Software products are tested for functionality, usability, compatibility, security and performance using a series of verifications, validations and exploratory techniques. Automation tools play an important role in accelerating the testing process and improving product release velocity. Challenges such as preparing test environments and measuring code coverage are solved using tools such as Docker and JaCoCo. Automation infrastructure is also critical for development operations (DevOps) such as integration, delivery and deployment.
This article identifies some of the common testing challenges, product QA requirements and outlines few automation tools and techniques that help in solving the challenges, with the aim of delivering a defect-free product.
Need for quick testing feedback to reduce deployment time
Ability to get timely testing feedback is a frequent challenge faced by product teams. This impacts developers’ ability to resolve defects quickly and the opportunity to reduce product deployment time.
Adopting innovative testing techniques
The Testing team adopts the following methods to overcome process bottlenecks:
- Parallel execution of tests – multiple types of tests in parallel to development
- Mocking external systems – simulating test environments when code not ready
- Running tests in local machines instead of on staging environments – to save resources
Need for multiple testers to work in a common test environment
Very often, the possibility of defects leakage arises due to multiple changes worked upon by multiple testers on a single code branch. Dependency on other component services leads to blockers and delays in deployments.
Using Docker tech to isolate test environments
To reduce dependency on the environment, testers use Docker technology. Docker is a popular tool adopted by testers & developers to isolate test environments for speedy testing and shorter feedback loops.
Docker creates an image of the application and stores it in a local machine. Testers can access Services, Databases and the tech stack deployed in a local machine or any dedicated instance. Tests can be conducted without any disruption to other teams.
New code changes are accessed directly from Dev (via code branches). Functional and performance tests are conducted on the branches and after resolving defects, the code is then merged back to the main branch for integration with other modules. This technique helps in overcoming test environment restrictions which is a major obstacle to speed of deployment. Product teams can save valuable time.
Organizing manual and automated tests for all components
Test cases are written in a central a repository or in an Excel sheet and are mapped to the automated tests. For each regression and new changes, the status of the tests performed and data of the test results are updated automatically. The test cases are stitched to the automation scripts.
Complete code coverage by manual tests and automated scripts
Code coverage measures the lines of dev code executed during automated tests. The measure of quality of a product is derived from the end to end tests that are performed on the complete code. Analysing code coverage for measuring tests is a critical requirement.
JaCoCo for measuring code coverage
JaCoCo is an open source tool for measuring code coverage and visually reporting the results. Product component services and automation code are integrated with JaCoCo for analysing coverage for lines of code, branches of code, and complexity (number of test cases covering all code paths). JaCoCo provides analytical views of the coverage through configurable visual Dashboards.
Fixing integration bottlenecks and monitoring downtime
There are times when buggy code gets pushed into production due to process deficiencies and integration issues. Limited united tests also result in application breakages.
Automating process changes and maintaining uptime
Automated tests are performed on new changes being made to the code repository by running Webhooks (Bitbucket or other VCS). Unit tests and functional validations are performed and marked for failed and passed tests. Approvals are made mandatory from the respective owners of the code branch to make any changes. The processes ensure only well tested code is merged into the main repository.
A Service is created that is designed to periodically ping target systems to check the health of the systems and track down time. Any disruptions are notified to the stakeholders in real time via emails and messages. This speeds up bug-fixes, team communications and defect resolutions – helping to monitor systems efficiently and accurately.
Preventing disruptions during continuous deployment
One of the key challenges faced during continuous deployment is the unplanned time consumed for rolling back buggy code, time taken to push new code changes and care needed for preventing downtime.
Preparing for post-production rollback and releases
Processes must be kept simple for rollback actions such as a simple restart for configuration changes. Design services for Bundle/Jar based deployment to the instances. Communication and notifications provide clarity on build stability and buggy code. Deployments must be configured as rolling deployments and any changes must be pushed on the instances linearly and sequentially. QA and Devops teams needs to make sure the old and new changes can exist simultaneously to maintain backward compatibility. Ensure there is no downtime for the users by providing alternate services. Partial deployments must be planned because some changes are to be performed in production only.
Automation is a critical component across all phases of the testing life cycle. New tools are using artificial intelligence and machine learning techniques to make visual testing easier. Robotic Process Automation tools (RPA) are being used to automate business flows and supports testing activities in reducing time taken to analyse applications. Automation has added more power to the testers toolset and is revolutionising traditional testing through reducing testing time, costs and human labour.
To access our robust test automation solutions, write to [email protected].