This article is about the video: "Testing Strategy for DevOps: What to Test and When" made by Dave Farley.
Continuous Delivery is a commitment to always work on a project that is in a release state. That means following a sequence of steps that includes testing to ensure that is followed.
Dave suggests we use two phases of testing, the first one being the Commit Cycle, which is mostly TDD-style testing (Anything that Fails Fast). Followed by the Acceptance Cycle, which validates and confirms that our software is indeed ready for release (Production - Anything that defines Releasable).
After we are ready for release, we do two more things, we do the Release Cycle, which covers anything that Supports Release, and after that, the Product Cycle, which covers anything that informs Product Design.
Something very interesting that Dave talked about is Canary Testing. According to Peter Loshin "In software testing, a canary is a push of programming code changes to a small group of end users who are unaware that they are receiving new code. Because the canary is only distributed to a small number of users, its impact is relatively small and changes can be reversed quickly should the new code prove to be buggy.
For the rollout of incremental code changes, a canary release of new features or even a new version of the software enables the development team to quickly evaluate whether the new software release provides the desired outcome. Canary deployment tests, which can include automated testing, are included in the canary release after testing in a sandbox environment has been completed.
Canary deployment enables developers to migrate a small subset of users to new functionality in a new release. This minimizes the effect of potential issues related to new software and makes it easier for the development team to roll back a buggy release of new functionality when only a fraction of the overall user base is exposed to the new code.
The word canary describes the code push to a subset of users because canaries were once used in coal mining to alert miners when toxic gases reached dangerous levels. Like the canary in a coal mine, the group of users selected to receive new code in a canary test is unaware they are being used to provide an early warning."
Your testing strategy is central to your ability to deliver better software faster. Excellent teams have a laser focus on testing, and this focus extends way beyond what we do before we release our changes into production. Testing to support DevOps and Continuous Delivery is a core idea, and testing in Production is part of that game too.
Testing in production allows us to do a lot of things that we can't do in anywhere else, and so, it's an important part of any complete testing strategy. By consciously testing in production, we can guide our products to be better over time. We can treat deploying to production as a series of tiny experiments we can learn from, incrementing our systems step-by-step in the direction that the clients really love.