Spot quiz for the DevOps and DevOps Security doyens here.
- What’s your deployment frequency?
- What’s the lead time for changes?
- What’s the time taken by your organization to restore service?
- What’s your change failure rate?
If you answered as below,
- Multiple times during a day
- Less than 1 day
- Less than an hour
- 0-15%
…then you can call yourself an Elite performer in DevOps. This is as per 2019 Accelerate State of DevOps Report by DORA (DevOps research & Assessment) and Google Cloud.
Elites have achieved high levels of automation. They have long surpassed the phase where QA is seen as a bottleneck between development and deployment. QA works closely with the dev team to push most of their tests into the continuous integration system. The idea is to get to no human intervention, so tests run on their own and generate their own test data. As ops teams develop more and more scripts to automate running, backing up and restore of production systems, rollbacks, QA works with them to bake in automated tests to enable things work as expected.
That’s one way of looking at it. The 2018 State of DevOps report by Puppet and Splunk came up with a great way of looking at DevOps maturity in an organization. The model has five stages of DevOps maturity, as well as Stage 0 that counts as building the foundation.
The 5 Stages of DevOps Evolution
Stage 0
This is about building the foundation, when the development and operations team are figuring out collaboration. They set up the tools and processes to make idea sharing and knowledge transfer easier, and also get some metrics in place. It is from here that the DevOps evolution of the organization starts.
Stage 1
At this stage, you can see that the development teams are using version control. They may look at removing systems that are redundant in their tech stacks, and may even refactor applications to work on fewer operating systems.
Stage 2
In this stage the dev and ops teams continue to “standardize the tech stack by further reducing the number of operating systems to a single OS or OS family and building on a standard set of technologies: databases, key value stores, message queues, identity stores and more.” This leads to reduction in overall system complexity and errors, and deployment can happen faster.
Stage 3
With the basic things in place, teams look at solving pain points such as late releases or critical defects that make it to production. The pace of development is faster than the ability to deploy. Hence reuse of deployment patterns starts, as does testing of infrastructure changes. These bring in a new level of trust and brings in efficiencies.
Stage 4
This stage is about the automation of systems configuration and provisioning. As the report says, “Automated system configuration makes it possible for ops teams to deliver systems to developers and QA that match the eventual production environment — and deliver them faster.”
Stage 5
By this stage the cumulative effects of automation and trust building are visible. “Resources are available via self-service, and incident response is automated. IT teams don’t automate just for the sake of automating; they do it to make the entire organization run with greater efficiency and precision,” says the report.
What about Security?
The fact remains that baking security into the software development process is challenging and that stalls DevOps evolution in a company. Well, it turns out that if you do DevOps well, you can do security well too. This is as per the 2019 State of DevOps report by Puppet, Circle CI and Splunk. The report notes that 22% of the firms at the highest level of security integration have reached an advanced stage of DevOps evolution.
Why is that? Because when one is looking at good software development and good security outcomes, the principles remain the same: culture, automation, measurement and sharing. The key to security is not a top-down approach; it’s a shared approach. Everyone is responsible for it. And when there is a culture of collaboration, sharing, reliability, issues can be identified early and resolved in the best possible way, says the report.
More Confident Teams; Better Organization Outcomes
When security is integrated across the software delivery lifecycle, teams tend to be far more confident about their security posture. With an approach that emphasizes cross team collaboration and trust, delivery teams are empowered to prevent, discover and fix security issues autonomously.
But does this translate into better outcomes? It does, according to the report. The report segmented companies into five levels of security integration into the software delivery lifecycle, starting from no integration at all to full integration.
61% percent of organizations that are at the highest level of security integration are able to deploy to production on demand at a significantly higher rate than firms at lower levels of integration. Fewer than half of the companies that have not integrated security into their delivery cycle at all are able to deploy on demand.
Use of Security Tools and Testing
While tools cannot solve all security challenges, they will enable you to automate routine tasks, so security teams can focus on finding security holes. Also, you can’t expect one single tool to capture all loopholes.
A comprehensive security program: What you require is a comprehensive security program that includes testing at all stages of the software delivery lifecycle. Here are some tests to incorporate into the programs
- Unit testing of code in compliance with secure coding standards. This ensures that issues are detected before changes are integrated into the software build. This can be done by the developers.
- Testing engineers look for application level vulnerabilities.
According to the report, “Application security testing tools that cover static, interactive and dynamic application security testing, software composition analysis tools, and practices such as threat modeling, unit testing and pen testing help security and development teams catch security risks and vulnerabilities before production, when they are cheaper to fix.”
A security expert in the scrum team: Embedding a security expert in the scrum team enables greater knowledge sharing and expands everyone’s awareness of security concerns. This may not be a scalable model for many companies, so it makes sense to incentivize developers for secure development. Similarly, ops is incentivized for secure deployment and infrastructure. And both teams are made responsible for mitigation.
Collaboration with the infra team: According to the report, “Security teams may not be aware of the operational concerns associated with specific security settings, and operations teams may not be aware of security holes caused by not enforcing specific settings.” So if they collaborate before deployment, the team can arrive at a more secure setup as well as make informed trade-offs.
In Search of Security
If your organization is committed to improving its security practices, the first step is to adopt DevOps practices. Companies with mature DevOps practices also have tightly integrated security across their software delivery cycle. Security is not a top-down approach, nor is it the responsibility of one team. Enable sharing and collaboration across all the teams, so security becomes ingrained into every process and practice.
Establish DevOps and QA best practices within your team to holistically advance security. Drop us a line and let’s explore how Qapitol teams can help.