9 min Devops

Inside TestDevLab, the company that breaks tech for a living

Stress-testing the likes of Zoom, Microsoft, Orange and Twilio

Inside TestDevLab, the company that breaks tech for a living

When Skype wanted to stress-test itself, it ran into a problem. Every company that offered to put Skype’s software through the wringer was ill-equipped for the job. Ervins Grinfelds and Andrejs Frisfelds found the solution: build the IT outfit everyone else needed. The end result is TestDevLab, a Latvian-born QA testing provider that breaks technology on a daily basis.

Both Grinfelds and Frisfelds are co-founders and, to this day, co-CEOs of the company. Thirteen years since its founding, TestDevLab has grown far beyond Latvia. 150 employees reside in Riga, but the 500-person workforce is also spread across Málaga (Spain), Skopje (North Macedonia), Tartu (Estonia) and Vilnius (Lithuania). We’ve already mentioned Zoom, the posterchild among TestDevLab’s many clients, but agreements are also in place with Microsoft, Orange, Twilio, Pinterest and many more.

The company’s reach extends far beyond software, too, as shown by the treasure trove of smartphones running battery tests continuously. IoT devices such as hearing aids or airconditioning sensors are also put to the test. Although the clientele is diverse, large clients account for 60 percent of TestDevLab’s revenue.

Pushing software to the limit

Zoom proudly lets the world know about TestDevLab’s findings. These test results vary from latency benchmarks to simulating videocalls that boast 1,000 participants or more. In face-offs against major rivals Google Meet, Microsoft Teams and Webex, overall video quality measurements shows Zoom trumping others under various conditions. Notably, heavy packet loss allows Cisco Webex to soar above the rest, leaving Teams barely functional when facing over 120 milliseconds of latency. Upon recovering from a short network outage, Google Meet squeaks by as Webex in turn struggles.

It’s plainly obvious why Zoom would be happy about the end results – every benchmark puts them either solidly in the middle of the pack or a cut above the rest. TestDevLab’s findings aren’t always available like this – it’s up to the client to decide if it wants to make them public.

Tip: Zoom is more than a video conferencing tool and is eager to show that to the world

Even though stress-tests are a numbers game, some testbeds make for great showcases. For example, one latency test displays pre-recorded webcam footage on one phone, while another right next to it is displaying the other end of the simulated call. As QR codes flash by, computer vision is recording something the naked eye can spot with no trouble whatsoever. The tests aren’t just about measuring latency, throughput and resolution, as one might think. Instead, the automated scoring resembles the kind of rating humans would give a specific videocall.

1000014064

This metric, known as the VQTDL (No-Reference Video Quality Assessment Algorithm) and created by TestDevLab, predicts human feedback by quantifying subjective opinion. For instance: as visual creatures, we humans crave a consistent image over pitch-perfect audio. Whenever you’re randomly selected to rate the quality of a videocall right before exiting Zoom or Teams, just know that TestDevLab’s algorithms are just as picky as you are.

Diverse approach

Uniquely, TestDevLab has taken on no outside investment. With its origins steeped in Skype, it attracted said company as a client from its early goings. Ever since, it has doubled down on added value that co-CEO Ervins Grinfelds says nobody can match. Whereas competitors are playing catch-up, TestDevLab is building on top of the strengths of its proprietary algorithms and well-vetted testing environments. Grinfelds and Marketing Manager Jānis Tipainis tell us the company no longer needs to reach out to anyone.

Another key strength is the on-site security on offer, with well-guarded servers hidden from view. Large Synology racks will retain data for six months unless otherwise specified, with twelve months being the maximum cutoff for many clients. As one can imagine, the data quickly adds up due to the relentless test passes of various kinds. Constant penetration tests and other security checks aim to ensure perpetual data security.

These factors have made TestDevLab an attractive prospect for many. As a result, it covers many different testing scenarios. Our first proof of this is the so-called “tapster”, a collection of mechanical digits that will happily act out automated scripts to simulate real-world device usage. Along the way, apps can get found out in ways developers don’t have the time or resources to discover. A neat tidbit is the fact TestDevLab gets early access to new phone models. This makes it possible to give developers advance warning about their mobile app chugging through battery life or hogging memory on the latest hardware. One recurring issue to look out for are “loopholes”, where an app needlessly and repeatedly demands resources it has no need for, even after being closed by the user. Many phone batteries have faced an early retirement due to such problems – here’s hoping TestDevLab finds those loopholes before they have a chance to go in for the kill.

1000014065
The “tapster”, which can play pretend human through its robotic digits

Stress-tests aren’t just about churning through the same scripts either. While many of the testing processes are automated, some actions have to be done manually. This also helps to avoid a capricious problem where automated tests misbehave in unforeseen ways, which can completely void the validity of enormous test batches. And even though TestDevLab’s HQ is well taken care of with high-bandwidth, high-speed internet, real-world users often aren’t. A dedicated network degradation device can test app or device behavior when facing the dreaded conditions of airport Wi-Fi, rural areas with spotty connections or tunnels. The device in question lets testers pick and choose between the kinds of maladies. A rotary button then allows for analog control, allowing TestDevLab to find the exact breaking point for apps and appliances.

Once more, said breaking points may differ greatly. Some clients need to know how many resources will be needed in a heavy traffic scenario for Black Friday. If a retail site buckles under the pressure of the most lucrative shopping day of the year, millions in profits can go missing within minutes. A spin-off company called Loadero, launched from within TestDevLab in 2020, can load-test such scenarios on demand. To illustrate this point, there’s a fairground-style game featuring a hand grip that lets players try to overload a pretend web store as much as possible with simulated traffic, racking up the visitor count as one presses down harder.

Beyond the basics

As TestDevLab offers QA testing for IoT devices, the kinds of tests have to constantly adapt to the specific use case. How well do airconditioning sensors connect and when does the connection drop? How accurate are reported rental scooters locations? TestDevLab clearly tries to be ahead of the curve and won’t railroad itself into merely testing software or specific hardware. This approach ensures emerging technologies are met with the testing infrastructure they require.

Why not test in-house? Grinfelds and Tipainis tell us clients often do, but rightly leave it to TestDevLab to do most of the hard work. Even though the likes of Microsoft, Zoom and Orange have plenty of capital to throw around, even conventional software solutions are better off being vetted by an independent provider that can pool its expertise. After all, one app may malfunction in seemingly odd ways. A single vendor may not recognize this as part of an industry-wide trend, which TestDevLab can recognize far sooner owing to its wide clientele.

1000014374
TestDevLab’s network degradation device

TestDevLab will sit down with clients to discuss the performance figures and problematic scenarios found in testing. It’s up to the client to actually implement fixes. There’s a clear division of labor here that both sides have to uphold. If it’s your software, it’s up to you to make it as good as it can be. What TestDevLab does is pinpoint where you can improve.

Advocacy

The Latvian company will leave clients to develop their software. But there is advocacy in other areas that does directly impact said development. Accessibility is one area in which TestDevLab is positioning itself very clearly. Under the banner of TestDevLab A11y, various IT solutions are stress-tested in ways that once flew under the radar.

Tipainis points out around a billion people globally have some kind of impairment. To illustrate the issue in the clearest terms: as the famed “digital transformation” has transformed our interactions with banks, governments, shops and services, many individuals have simply been left behind. Some banks may only have an online presence or require in-app verification, but lack any functionality that allows, say, blind people to use their offerings. Plenty of browsers and websites fail to account for those trying to use them without the ability to move a mouse or (accurately) touch the screen. This has resulted in the visually impaired being unable to order Ryanair tickets or buy an apartment, to list just two of many examples.

The company’s A11y platform pinpoints such issues and forms the spearhead for a wider effort. In October, TestDevLab is organizing Quality Forge, a conference on digital accessibility. This represents TestDevLab’s contribution to change developers’ mindset. As much as, say, security, compliance and/or stability are relevant, one must account for all users. Even if compassion isn’t in one’s vocabulary, the simple fact is that it can be a profit-driver. This fact alone should hold plenty of sway.

Conclusion

As professional as TestDevLab’s approach is, one cannot help but notice the creative force required. This may seem strange as “testing” often gets associated with rote, routine tasks that lack any user agency once the methodology is in place. It isn’t just about simple execution, as evidenced by our visit in Riga. Being the go-to laboratory for many globally known brands, TestDevLab has found a niche that not many can fill with the same level of maturity.

Also read: Latvia as a breeding ground for IT success: Zabbix and DeskTime are living proof