We all know the traditional problem with software testing: it happens too late, and often in a rush, as users badger developers for delivery. If a software project runs over deadline, the chances are that the testing will suffer.
Agile development helps to solve that problem, but automating the testing is a critical part of that process. In agile development, the testing is meant to happen earlier rather than later, so that any problems with an application can be identified and fixed earlier on.
The testing is also supposed to happen frequently, leading to the mantra: “test early, test often”. That way, if your software development veers off course, you’ll notice quickly and recalibrate.
As the pressures on software developers increase, agility in development and testing becomes increasingly important. Damian McClellan, principal consultant at Readify, an Australian application development company, ships production-quality releases every two weeks.
"It takes a lot of time to do manual regression on that, so a good agile team will do a lot of automation," he says. Cloud-based and mobile applications will accentuate these pressures, making agile development and continuous testing increasingly important. One benefit of test automation is that it can positively affect the coding approach, argues Matt Frank, a consultant at Unboxed Consulting, an agile development firm.
"Teams that favour test automation normally also take a test-driven approach to writing code," he explains. "People thinking in a test-driven way often tend to focus on the intent of what needs to be built.” The ideal practice is to write requirements that can be directly converted into an acceptance test, he says.
Passive-aggressive bug reports
Writing automated tests also becomes a way of playing back requirements to a client and reducing any misunderstandings along the way. Test-driven development is an important tool when bridging the gap between development and testing teams. "People are used to having the test department as a separate division," explains McLellan.
This can lead to what he calls "passive-aggressive bug reports", in which the test team doesn't understand how the functional requirements laid out in the initial documentation changed during development. "So, testers say 'I expected it to do this, and it did that'. Then, that bug will come back to the development team, and they may not even know each other."
It is important that development teams work collaboratively with testers to build an atmosphere of trust and collaboration so that when the time comes to automate testing, the organisational structure supports what the software is trying to do.
Australia and New Zealand Banking Group (ANZ) used IBM's Rational Asset Manager to automate the testing process across the company. Rational Asset Manager keeps track of deployable artefacts across the company, and is linked to a baseline in the source control system, so that testing artefacts can be traced back to source code.
Automation plays a key role in how quickly you can operate in a changing environment," says Ben McCoy, service optimisation manager within ANZ's technology operation. "If we invest effort upfront to put the right levels of automation in place, it pays off when we need to make changes." One barrier to continuous testing and delivery is environmental set up.
Testing teams – especially those working on complex software – can take days to set up testing environments. Provisioning test environments automatically is an important part of any agile process. Consequently, server automation tools are an important part of the mix. Virtualising servers in the cloud can help to speed up the provisioning process.
Deployment automation can also help here. ANZ used Rational Build Forge to automate the deployment of software to non-production environments. This enabled it to understand exactly what was deployed, and match it against what was expected. Bridging the gap between testing and deployment is a crucial part of the devops movement. This seeks to break down barriers between development and operations personnel, so that they work more seamlessly together.
This can be challenging, and in many cases will require a broad organisational shift. Walls between developers, testers, and production teams have often been built up over long periods, and each of them have their own agendas. Developers have been told to create as much as possible. Testers are told not to let through anything that hasn’t been thoroughly vetted. Production teams are told not to introduce anything that might break their systems, which may make them suspicious of testers.
“You can see how these three organisations have nothing but different drivers,” explains Galles. “They need to work together on a common idea.”
Standard testing environments can help, says Frank Fabian, head of testing environments, delivery services, within ANZ's technology operation. "Because we have standard tools and processes, we are not reinventing the wheel every time," he explains. "We've reduced our training time as well, because all of our teams are using the same tools."
ANZ uses a standard toolset as the basis for a more collaborative approach, breaking down the boundaries that others identify as a perennial problem in development, testing, and operations. It can now manage its workflow more effectively, moving its resources around to respond to changing needs. Ultimately, technology will only ever be a tool, and it takes a cultural shift to truly realise the benefits of automated testing. But combining test automation with agile processes and a team committed to collaboration promises can yield good results. ®