Qapitol QA

3 strategies that top tech companies employ while testing their softwares

The famous 1983 HBR article titled “Quality is more than making a good product” highlighted the three paradoxes of Quality:

  1. Quality should always be customer-centric
  2. But when you ask your customers to define quality, they may have a hard time giving you a absolute definition, though they might come up with some comparative descriptions
  3. Not only this, customer’s perception of quality evolves over time, influenced by external factors like economic condition, purchasing power etc.

And these hold true more than 3 decades later today when technology has changed how businesses deliver value and quality to consumers by an unprecedented scale.

Today, quality is not a benchmark anymore. Its the bare minimum that your product needs to survive the competition and with software products, your offering often is about quality at a great pace than your peers.

So, while every tech organisation employs software testing in myriad ways to improve the quality of their offering at an accelerated pace, we take a look at how this feat is achieved continually by some of the world’s top tech companies.

How do the Top Tech companies in the world test their softwares?

The easy answer – They deploy several teams and go on a testing spree because they have deep pockets.

But dig deeper and you will find that these tech giants are as cost conscious as anyone else in the market and always on the lookout of practices that deliver better results and optimise resource utilisation. And as these companies go along in search of the ideal best-fit testing strategy which differs in intent and approach for each, here are the 3 aspects that remain common to each of these pursuits:

1. Incorporating Quality As Part Of Culture

Often the top tech companies view testing as vital to building quality softwares and hence aim towards building an organisational culture where everyone is aware of the importance of having a thorough testing schedule in place and consequences of a leaky testing framework.

  • Google runs Testing on the Toilet initiative to spread testing news and highlights throughout the company
  • Amazon’s Engineer accountability concept has a set of bug discovery hierarchies, ranging from level 5 (for minor bugs) to level 1 (when the CEO is alerted) to mark the urgency, COE (Correction of Exception) actions issued for developers who miss out on identifying high impact errors and which often involves a detailed discussion with their managers on reasons behind the miss, course correction and ways to avoid such mishaps in future.

With each of rigorous rituals, these companies ingrain testing as a organisation wide focus and not just limit it to a team. Emphasizing on superior quality as part of the culture also encourages teams to keep a track of the evolving nature of customer perception of quality of software applications by

  • Keeping a track of the evolving quality trends
  • Coming up with ways to innovate testing while keeping the intent intact
  • Resulting in testing and development teams working closely

2. Embracing Automation To Speed Up Testing

  • Facebook employs a wide range of test automation tools ranging from PHPUnit for back end unit testing to Jest (a Javascript test tool developed internally) and Watir for end-to-end testing.
  • As per a famous 2014 article by Dr. Werner Vogel, Amazon achieves 50M deployments a year across dev, test and production environments, which translates to about 1.6 deployments per sec. What enables the e-commerce giant to achieve this astronomical feat is the rigorous amount of support automation adopted at different stages of deployment.
  • Leading SaaS Unicorn in India was able to reduce its GTM by 45% by automating more than 95% of its regression tests using our tailorable test automation framework, resulting in cleaner releases and seamless user experience.

Test Automation ensures that repetitive tests are done faster with greater accuracy, thereby increasing the iteration speed, as is the case of Amazon. James Whittaker in his book “How Google Tests Software” talks about Google’s belief – “ ….human intelligence is too valuable to waste on something a computer can do…” Automation frees up your organisation’s manual efforts for more valuable and creative pursuits that can only be achieved by higher order intelligence.

3. Testing Approach Tailored To The T

Despite the tech titans having commonalities on how they test their softwares, what each of them constantly strive for is customisation of the whole approach to suit their business goals in a dynamic business scenario.

  • Spotify uses an intelligent mix of manual and automated testing approaches to test its softwares. The testers are part of different cross-functional teams who do most of the tests in the production environment to optimise resources spent on maintaining test environments.
  • Google believes in pyramid model of testing wherein it invests the maximum resources in unit and integration testing at the bottom and uses end-to-end testing less often. This enables it to go for small and frequent releases and minimise test failures.

    Source – Google testing Blog
  • Amazon’s testing philosophy resides on the fact that a software should be easy to deploy and easier to roll back in case of failure. Hence, they use Canary testing, wherein the updates are released to a small set of users to identify potential bugs or product missteps. This strategy enables Amazon to deploy huge number of deployments and easily roll back the ones which fail canary testing without any significant effect on the entire user base.
  • Netflix, the pioneer of microservices has a test automation in place in which tests do not have structural constraint, are low on cost but high on agility. Such a condition ensures that different teams are able to modify the test structure as per their requirement and at the same time, the tests are simple enough to be set up in manual and automation mode easily.

Every organisation is out there to provide greater value to its customers but the journey is different based on the industry, its testing philosophy, organizational vision about customer-centricity and what works for each on the ground level. We found during our work with unicorns and soonicorns that this holds true for them as well.

As unicorns, your challenges are different and hence the approach to delivering superior value to your customers at an accelerated pace should be backed by a quality engineering framework that’s tailormade to suit your situation and goals.

Based on the insights shared about how the top tech giants approach testing, it would be a great idea to write down about your testing philosophy and approach. This is where we have helped our clients in different segments create and implement bespoke testing strategies made to fit to their requirements. Get in touch with us for a free demo today.

Write to sales@qapitol.com for strengthening your quality engineering pursuit.

Share:

Share on facebook
Facebook
Share on twitter
Twitter
Share on linkedin
LinkedIn

Gartner, “Market Guide for Application Testing Services”, Jaideep Thyagarajan, Susanne Matson, Gunjan Gupta, Brett Sparks, Akshit Malik, 31 January 2021. GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.

Disclaimer: Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.