Uncle Sam mulls dumping monolithic software stacks for modular blocks

Anyone else heard of these microservices?

The US Department of Labor is dipping its toes into a radical approach to software architecture for the nation's unemployment insurance (UI) systems: breaking down monolithic software systems into a bunch of modular services. 

Okay, it might be just a few years late to the microservices game, but DoL deputy director of technology Larry Bafundo said that US states are still operating their UI software stack in monoliths that are "brittle and inflexible" when faced with stress situations - like a massive influx of claims due to a pandemic. 

Bafundo said the modernization approach championed by the DoL is one that embraces modular software "where components can be managed independently and more easily replaced with new solutions." Along with that comes an acceptance that there's no one perfect IT system for each state's UI stack, so the DoL instead suggests characteristics by which various products and approaches can be integrated.

Those include being cloud-based and scalable, built around open source and using agile software practices, with data analytics capabilities, cross-system integration, and the addition of process automation code that can "empower, rather than replace, staff," he said. So, all the buzzwords then. 

Effective modernization involves more than just technology

"Effective modernization involves more than just technology. It also requires a new mindset and approach to building and buying technology that centers on modernization as a continual process and the belief that IT systems are never complete," Bafundo said. 

In a roadmap of its UI modernization program that runs through September of this year, the DoL said one of the first moves is an AI prototyping partnership being launched with Stanford University's Regulation, Evaluation and Governance Lab, or RegLab.

RegLab (no relation) works with government agencies to modernize their systems. It's not immediately clear what the DoL will be designing with the help of RegLab - the DoL only mentions the partnership as a way to explore "effective and equitable" use cases for the technology so it can be used to assist state employees in streamlining their workflows. 

Um … did we miss a memo?

Nowhere in Bafundo's blog are microservices mentioned, but his tone and language points to the move, especially taking into account statements like "states should adopt modular systems that are easy to maintain incrementally, reuse existing software, and integrate new solutions through standard interfaces and an API-first approach" being highlighted. 

As microservices and APIs have proliferated, organizations have faced new challenges, like whether breaking monolithic software into a bunch of smaller parts that all have to talk to each other via APIs is the most efficient option. When it comes to a bunch of APIs, security is also a major concern. 

Sam Newman, who literally wrote the book(s) on microservices, said in 2020 that they shouldn't be the default choice for every new piece of software because, among other reasons, so few organizations manage to do it right. 

Even Amazon is rethinking its commitment to microservices, with a case study from the Prime Video team reporting that its distributed approach monitoring video streams for quality issues was making too many expensive calls to an S3 bucket and being bottlenecked by Step Function limits. 

To get around those issues, Amazon said it rebuilt the software it needed as a single monolithic process. 

"Microservices and serverless components are tools that do work at high scale, but whether to use them over monolith has to be made on a case-by-case basis. Moving our service to a monolith reduced our infrastructure cost by over 90%. It also increased our scaling capabilities," AWS said in its paper. 

Developers who responded to the report were dumbfounded, but the argument still stands: microservices might not be the best solution for every problem. Newman said as much, noting that monolithic software doesn't mean modularity is out of the picture. 

"Have you done some value chain analysis? Have you looked at where the bottlenecks are? Have you tried modularisation? Microservices should be a last resort," Newman previously told The Register. 

Whichever approach ultimately wins out, the Department of Labor has its work cut out for it, as a federal agency trying to herd cats - er, states - into adopting new technology. "Each state also has its own IT system for administering the program, which means the ease of use, efficiency, and effectiveness of different solutions can vary," Bafundo said in his blog post. 

A new unemployment claims intake platform is planned for rollout in New Jersey in August, but there's no word when other states will begin testing any additional systems they're planning to build.

We reached out to the DoL to ask some questions about its partnership plans with Stanford, and other aspects of its UI modernization plan. The Department confirmed receipt of our questions, but hasn't yet responded; we'll update this story when we hear back. ®