Ask HN: How can I test my VPN client on more Android and Windows devices?

1 pointswillprice891y ago2 comments

I'm developing a cross-platform VPN client. Because the primary design goal is censorship circumvention, I'm using a custom protocol and cannot just put a wrapper around existing, well-tested, open-source solutions.

For iOS/macOS, once I've got the client working well on my own test devices, it generally works fine for other users. Unfortunately this isn't the case for Android or Windows. Even though the client works flawlessly on the devices I have available to test locally, many real-world users see poor performance or can't get it working at all.

While I always try to work with users to identify and fix the issues, most of them (understandably) prefer to just get a refund and switch to another VPN. So if I could do in-depth testing on a broader range of devices it would boost retention considerably.

Ideas I've considered so far for expanding my testing pool:

- Hire testers on a site like Fiverr

- Offer free accounts for testers

- Use a "real device cloud" testing service

- Hire a more experienced VPN developer with expertise on those platforms

- Open source the client code to get more developer eyes on it

2 comments

2 comments · 1 top-level

fensgrim1y ago· 1 in thread

I'd go for a "real device cloud", AWS/testgrid/browserstack/whatever.

However depending on your users' demographics, it might be a good idea to take a look at https://dontkillmyapp.com/ and get few used devices from huawei/xiaomi to run local tests as well, generally the farm experience wouldn't exactly match local experience (e.g. what happens when you leave the device without usb connection running overnight, what happens when you leave device unattended for a week). US/Europe? Don't bother. EMEA? Maybe get a couple devices running lineageos/grapheneos/calyx as well. China? There's https://www.alibabacloud.com/help/en/mobile-testing but it wouldn't match real user experience due to how dpi/GFW is set up.

Also check out https://developer.samsung.com/remote-test-lab and https://developer.huawei.com/consumer/en/digix-lab/ if you want to save some bucks on renting stuff.

willprice89OP1y ago

Wow, thanks for the great resources! Do you know of any real device clouds with Windows OS testing? Browserstack only seems to do web app testing in Windows browsers...