QA and Lean By André Vondran - Software Quality Assurance Analyst

Recently I’ve had the opportunity to participate in a project that is experimenting with a lean software development environment and so have made a few observations about where QA fits into a lean methodology.

Not being an expert in lean methodology, it’s possible (and likely) that the process my team is following isn’t strictly lean but some subjective interpretation of it.

The central idea of lean is that you never do anything until the moment when it is needed and since there is no concept of iterations, schedules tend to unfold organically. Priorities for tasked out work are set daily and decisions about what gets done is made the day it is worked on. Like agile, documentation takes a back seat to conversations about design and requirements.

This means that when decisions get made, we must rely on the frailty of human memory or a scribbled one-line note on a sticky as documentary evidence.

I’ve often said that the role of QA is to be the memory for the organization. We spend a lot of time conversing with various stakeholders in various departments.

We take these conversations and turn them into scenarios that form the basis of our criteria for testing. The result of all of this is that the QA engineer has a more holistic perspective of the end product. Our soul mates in the development process are the business analysts and architects of the system.

I was excited to get the opportunity to be a part of a lean project. I’ve been reading about it and I like the underlying concepts but mostly I wondered just how it would incorporate QA into the process.

So far my conclusion is that it doesn’t. When I say “it doesn’t”, I don’t mean this in a “the developers are so good they test all their own work and deliver perfect software” kind of way. I mean, in my opinion the biggest weakness of the lean process we’re following is that in encourages piecemeal delivery to such a degree that those working on the project completely lose sight of the big picture. The code is wonderfully modular, and the bits that get delivered to me for testing work very well in their own right, but in the end they don’t play well together. This team’s implementation of lean doesn’t encourage a collaborative environment but instead forces a handful of people into a role of systemic analysis.

And as the project progresses the team becomes increasingly dependent on the few people who have a systemic understanding of the system to clarify requirements and tell them how it should work.

Can the Lean concept work with QA? One of our team members made the observation that the process we were following tended to create a much higher burden on QA because developers had no impetus or opportunity for identifying the interdependencies. A big advantage of Lean is that you get to a mature stage of development extremely quickly and this can be very useful in situations where an organization has been paralyzed in the requirements analysis stage (which was the case with the client we’re working with). The client has been very impressed with short turnaround time. We’ve been able to deliver a working system in about half the time it took our predecessors just to hash through the requirements process.

I think for this particular client, it works very well because they know they will change their minds about what they want at any stage of the process. It was less important for them to have a top quality system. That said, we found an unusually high number of major issues post-release. Part of the legacy of our ‘lean’ philosophy was that we had no automated acceptance tests and since the original testers and business analysts are no longer on the project, we lost some of the original understanding (or memory) of how the system was supposed to behave. This problem was exacerbated by a lack of documentation.

Even the stories we developed from were simply keywords with no acceptance criteria and no elaboration as to what those keywords meant. As I tested the requirements, I spent alot of time talking to developers about what they had built, talking to the business about what they had asked for and trying to manage the large gaps between the two.

So back to the original question. Can a lean environment support a QA or testing role? I believe it can and I believe the problems with this lean project are rooted in the interpretation of the philosophy. I don’t think lean means: don’t do anything that takes time. And that was the tendency here. I

think the underlying philosophy is fantastic. Basically the idea is to be smart about what you spend your time on and when. In our case, the lean philosophy was more like bare bones. We accomplished a great deal but in the end I believe it was a skeleton of what we needed.

Spending a little more time in analysis and offering the developers more of an opportunity to understand the context of their work could have given us a little more flesh on those bones.

André Vondran SQA Analyst
www.aqasol.com.ar