Although it wasn’t intended at the beginning, Agile (or, more specifically, Scrum) has led a rigid development process that seeks to remove responsibility and control from the developer, treating them merely as a worker in a production line.
To give this statement more colour, let’s take a look at a vanilla Agile workflow, a flavour of which is being played out in organisations across the world:
Features are carved up into a series of easily estimatable sub-tasks which are presented to a team of developers for estimation. The developers review each sub-task, usually described in vivid detail with a set of acceptance criteria, and come to a consensus on how long each will take to implement. Each sub-task is worked on by a developer, then handed over to a tester to check that it meets what was specified. If any issues are found, the tester will raise a bug ticket for the defect (usually a separate ticket for each issue found) and assign them and the original sub-task back to the developer. Once all bugs have been fixed and the tester is happy to sign off testing, the code is bundled into a release that will eventually go into Production.
In this workflow, rather than developers being responsible for the delivery from inception through to Production, control is handed to the person who divided the feature into sub-tasks. This is because to be able to effectively divide up a feature into components, someone has to consider how the sum of the sub-tasks creates the desired outcome. If this isn’t the developer, rather than them taking an active role in the build and being able to exercise their creativity, they take a back seat role.
Over the years, a number of developers have come to accept this state of affairs as the status quo, developing a learned helplessness that reinforces their dependence on the person pulling the strings. Preventing a developer from being creative removes the passion from their work, and, surprise surprise, results in lower quality work, work that contains defects, and work that may well not actually hold together when combined with the rest of the associated sub-tasks.
The lack of quality results in a low trust environment between developer and tester because the tester always finds obvious flaws when a sub-task is deemed to be complete and thrown over the fence for QA. Due to this lack of trust, the tester feels they need to test each and every thing the developer produces, or risk that a bug will ship to Production. Apart from being 100% correct in this belief, a large number of testers are needed and companies usually settle on a ratio of around 1 tester to 3 developers. In an age of skyrocketing tech salaries, this can quickly get expensive.
The No QA Movement
An approach some companies use to combat this problem is to remove QA as a function altogether. By removing the crutch of a QA department, the developer is forced to take more responsibility for the quality of their work, the threat of a Production outage being a powerful (and scary) motivator. This may be an acceptable approach for some companies but it ultimately comes down to your appetite for risk.
Great QAs add value by approaching problems differently, by being curious, and by not doing what they’re meant to. Due to reduced cognitive load, they’re able to keep a larger part of the overall system in their heads and are therefore in a better position to spot when a change in one area could lead to unintended consequences in another. In short, if the price of reversing a bug mistakenly shipped to production is high, you should have a tester assuring the quality of what you release.
But how can we get the benefit of having a quality assurance layer whilst fostering developer ownership and keeping costs down? At Moneybox, we’ve discovered an approach we like to call Consultative QA.
Consultative QA
At its heart, Consultative QA aims to push the responsibility of quality control onto the developer whilst keeping the overarching support and assurance of a seasoned QA professional. It is the antithesis of the traditional, fence throwing mindset that we’ve somehow ended up in with Agile.
To make this work, QA needs to be engaged right from the beginning. At Moneybox, this engagement point is when the developer has completed an Architectural Decision Record or ADR. ADRs warrant a post all of their own but, in short, the ADR outlines the technical solution to a Project Brief and is completed by the developer before they begin developing in earnest.
It’s important to note that the ADR is the lowest level the feature exists at, and should result in something tangible that can be shipped to your customers. By lowest level, I mean that the ADR maps one to one with a ticket on your ticketing system of choice and is not broken down into smaller tickets, sub-tasks, etc. The developer may choose to build and release the feature in parts but, if this is the case, it’s described in the ADR rather than through sub-tasks in a ticketing system.
The benefits of this approach are threefold:
- It ensures the developer is always looking at the bigger picture, driving ownership and overall responsibility.
- Thinking isn’t constrained through the use of sub-tasks and developers are free to come up with innovative solutions to the brief, ones that may not have been thought of.
- It prevents an explosion of tickets that need to be managed, tested, triaged and reported on.
ADR and QA Reviews
The real power of ADRs shows during the review phases, where they bring alignment and allow decisions to be challenged. I say ‘phases’ as there are actually 2 distinct phases:
First, the developer brings together all interested parties and walks them through the proposed design. This gives attendees the opportunity to ensure a) that the proposed design meets the requirements, and b) that the design fits into the overall architecture. Once feedback has been incorporated and the ADR iterated if necessary, we move onto the second phase: The QA Review.
The QA Review is the brainchild of Antony Konstantinou, one of Moneybox’s talented QA leads, and sits at the heart of Consultative QA. The review is organised separately from the ADR review to allow time for the tester to think holistically about the solution, what processes could be affected by the changes and what tests should be run. The output of this review session is a list of test cases that have been agreed between the tester and the developer, if a test can be automated or not and who should do the testing.
The last sentence above is very important. Consultative QA does not mean that the tester actually completes the testing themselves. Rather they are there to ensure that the testing has been completed. If a test can be automated, it should be automated by the developer. If the test is best handled by an experienced end-user of the system, that user should do the testing.
Conversations, not Tickets
If a bug is discovered, the tester discusses the issue with the developer, either in person or in an open channel. Bug tickets aren’t raised and assigned - all communication happens through the developer and tester talking to each other rather than communicating through a ticketing tool. You may decide that serious defects are raised to ensure that all the relevant information is captured but the emphasis is on serious. As a rule of thumb, Antony suggests that it will take longer than 24 hours to fix. The benefits of this are twofold: a) the burden of creating and managing tickets through a development flow is completely removed, and b) it encourages the developer and tester to work in harmony rather than the stop-start style of communication that takes place through a ticketing system.
However, the one benefit that the old ticketed approach does offer is that it proves that testing has taken place. This is especially important if, for instance, you work in a regulated industry like we do at Moneybox. In this scenario, rather than reverting to old habits, Antony has come up with another solution: ticket the manual scenarios and attach the proof to each ticket. These scenario tickets can then be grouped together into an epic and attached to the ADR, keeping the documentation and test cases in one place.
The Developer QA Ratio
Using this approach, we’ve managed to get a developer to QA ratio of around 6 to 1 with no loss of quality. In fact, I would argue that the overall quality has actually increased, as quality is baked in from the start rather than being dusted on at the end.
But what’s the optimum ratio? I think that depends on the environment you work in, however it’s important to keep the ratio high enough that there’s a healthy tension between development and overall QA capacity but not too high as to cause burn out. This keeps the onus on the developer to do as much of their own QA as possible, as there just isn’t anyone else to do it.
A Final Word
There are two final points I would like to make:
Firstly, it’s important to note that you need a mature development team to make this process work. Taking a team who are reliant on QA and suddenly removing their safety net will cause unease and likely result in bugs being shipped to Production. If you are to follow this model, I would fix the destination and then introduce change gradually but consistently over a period of time to allow the team to get comfortable with the process.
Secondly, we haven’t talked about Process, although, you’ve probably picked up that I have a mild distaste for Agile. The secret sauce that makes Consultative QA hang together is to use Shape Up instead of Agile.
I’ll be writing more on these topics in the future, stay tuned.