Adobe edits the development cycle

Less bugs, more weekends off


Interview For years the Adobe Photoshop team has been trying to get away from the traditional death march to a more agile development style. For its CS3 release, it made the jump, with the help of VP Dave Story. The result? More weekends off, and a third fewer bugs to fix. Mary Branscombe quizzed co-architect Russell Williams on how they did it.

If it's such a good idea, why did it take so long – and how did you manage to change this time?

We had been trying to make the change for a couple of versions but hadn't really been able to make it stick. Nobody on the team had experience using an incremental development process, and we kept sliding back into our old ways because of our own habits and pressures from other groups at Adobe who wanted to see an early "feature complete" milestone. We'd always successfully delivered using the old method, so when things got difficult, we'd revert to things we knew would work.

The difference Dave Story [Adobe's vice president of Digital Imaging Product Development – Ed] made was he had successfully managed incremental development before at SGI and Intuit, and he had the commitment and experience to work through objections - from within or without.

And what actually changed about the way you're developing CS3?

The change we made was going from a traditional waterfall method to an incremental development model. Before, we would specify features up front, work on features until a "feature complete" date, and then (supposedly) revise the features based on alpha and beta testing and fix bugs. But we were scrambling so hard to get all the committed features in by the feature complete date - working nights and weekends up to the deadline - that the program was always very buggy at that point. We'd be desperately finding and fixing bugs, with little time to revise features based on tester feedback.

At the end of every cycle, we faced a huge "bugalanch" that required us to work many nights and weekends again. Of the three variables: features, schedule, and quality, the company sets the schedule and it's only slightly negotiable. Until feature complete, we could adjust the feature knob. But when we hit that milestone, quality sucked and we had only a fixed amount of time until the end. From there to the end, cutting features was not an option and all we could do was trade off our quality of life to get the quality of the product to the level we wanted by the ship date. We've never sacrificed product quality to get the product out the door, but we've sacrificed our home lives.

We couldn't cut features to meet the schedule because by the time we realize we're in trouble, the features have been integrated and now have interactions with other features, and trying to pull them out would introduce more bugs.

Probably the most effective thing we did was institute per-engineer bug limits: if any engineer's bug count passes 20, they have to stop working on features and fix bugs instead. The basic idea is that we keep the bug count low as we go so that we can send out usable versions to alpha testers earlier in the cycle and we don't have the bugalanch at the end.

The goal is to always have the product in a state where we could say "pencils down. You have x weeks to fix the remaining bugs and ship it". The milestones are primarily measurement points, with the acceptance metric being quality instead of something like "all features in" or "UI Frozen". We can keep adding or refining features until later in the cycle, and we can cut features if things are running behind (OK, when things are running behind).

Features are developed in separate, private copies of the source, and only merged into the main product when QE [Quality Engineering - Ed] has signed off on the quality level. Since each of those "sandboxes" has only one major feature under development at a time that differs from the main copy of the source, it's practical to send copies of the sandbox version out to testers to test that specific feature -- the rest of the program isn't too buggy to use. So the new feature gets reasonably tested and refined before being put into the main copy of the source. That keeps the main code base from becoming a mess of buggy and incomplete new features.

Making life easier for the developers matters – but has it been good for the code too?

The quality of the program was higher throughout the development cycle, and there have been fewer total bugs. Instead of the bug count climbing towards a (frighteningly high) peak at "feature complete", it stayed at a much lower plateau throughout development. And we were able to incorporate more feedback from outside testers because we didn't switch into "frantic bug fix mode" so early.

Did it change the way you put out betas?

An automatic process builds the program every night and runs a set of tests before posting the build on our internal servers for QE to test. We could take almost any of those daily builds and use them for demos.

The public beta was basically just "whatever build is ready on date X". There were only a couple of "we really gotta fix this before we send out the public beta" bugs. With past versions, we couldn't have done a public beta at all that far ahead of release - there would have been far too many bugs.

We weren't swamped with a pile of bugs from the hundreds of thousands of people who downloaded - it really was in the good shape we thought it was. With several hundred thousand downloads, there were fewer than 25 new bugs found.

Overall, did you end up with fewer bugs, more bugs, the same number of bugs fixed faster? Did you have to sacrifice features to work this way?

Some people feared this would mean fewer features. That hasn't been the case. We certainly had far fewer bugs overall and fewer during mid-cycle (about a third less in total last time I checked). Better quality, plenty of features, fewer nights and weekends: what's not to like? ®

Similar topics


Other stories you might like

  • Adobe lowers 2022 forecast, blames Ukraine war, strong dollar
    Extended 'summer season' also at fault, says software slinger as share price slides

    Creative software slinger Adobe booked in double-digit revenues rises in its latest quarter but lowered forecasts due to conflict in Ukraine and and currency challenges. As such, Wall Street frowned and the share price went down.

    The Photoshop maker reported turnover from sales of $4.39 billion for Q2 ended June 3, up 14 percent year-on-year. The vast bulk of this, some $4.07 billion, was subscription-based, something other software vendors must eye with some envy because investors love recurring revenues.

    The Digital Media division, which includes Creative Cloud and Document Cloud products, jumped 15 percent to $3.20 billion, higher than analysts had estimated. The Digital Experience wing was $1.1bn, up 17 per cent, again trumping analysts' projections of $1.08 billion.

    Continue reading
  • Microsoft fixes under-attack Windows zero-day Follina
    Plus: Intel, AMD react to Hertzbleed data-leaking holes in CPUs

    Patch Tuesday Microsoft claims to have finally fixed the Follina zero-day flaw in Windows as part of its June Patch Tuesday batch, which included security updates to address 55 vulnerabilities.

    Follina, eventually acknowledged by Redmond in a security advisory last month, is the most significant of the bunch as it has already been exploited in the wild.

    Criminals and snoops can abuse the remote code execution (RCE) bug, tracked as CVE-2022-30190, by crafting a file, such as a Word document, so that when opened it calls out to the Microsoft Windows Support Diagnostic Tool, which is then exploited to run malicious code, such spyware and ransomware. Disabling macros in, say, Word won't stop this from happening.

    Continue reading
  • Adobe apologizes for repeated outages of its Creative Cloud video collaboration service
    Frame.io admits it was 'slow to scale as demand rose

    Adobe-owned cloudy video workflow outfit Frame.io has apologized and promised to do better after a series of lengthy outages to its service, which became part of Adobe's flagship Creative Cloud in 2021.

    Frame.io bills itself as "The fastest, easiest, and most secure way to automatically get footage from cameras to collaborators – anywhere in the world" because its "Camera to Cloud" approach "eliminates the delay between production and post" by uploading audio and video "from the set to Frame.io between each take." In theory, that means all the creatives involved in filmed projects don't have to wait before getting to work.

    In theory. Customers say that's not the current Frame.io experience. Downdetector's listing for the site records plenty of complaints about outages and tweets like the one below are not hard to find.

    Continue reading

Biting the hand that feeds IT © 1998–2022