Streamlining Engineering Tasks: Cutting a 14-Hour Workflow to Just 2

Challenges with Traditional Agentic Software

Agentic software is commonly viewed as a straightforward approach: provide an LLM with tools, set a goal, and let the model tackle the hurdles. While this method shines in prototype stages, it poses significant challenges for workflows that demand repeatability, reliability, and maintainability. The perception of simplicity can often overshadow the complexities inherent in real-world applications.

One major hurdle is the unpredictability of large language models (LLMs). They excel in generating responses and solving problems in a controlled environment, but that skill doesn’t always translate well into environments with multiple variables and dependencies. In theory, this kind of software should alleviate the burden of heavy lifting from engineers, allowing models to flex their creative muscles. In practice, however, the inability to consistently manage outcomes can lead to frustration. For instance, if a model misinterprets instructions or generates erroneous outputs, it could derail an entire project. These unanticipated results can waste time and lead to costly errors if corrective actions are not taken swiftly.

The issue of maintainability adds another layer of complexity. As business needs evolve, the software must be updated and adapted. However, many teams fall into the trap of treating agentic systems as "set it and forget it." That's a risky approach. A lack of regular maintenance can lead to degradation of performance and even system failures. Unlike traditional software, which typically has clearly defined functionalities and predictable behaviors, agentic software can create a kind of black box that team members don’t always understand. This can breed reluctance to adopt the system fully, as users may be unsure of how to troubleshoot or enhance its capabilities when issues arise.

Real-World Application: Automating Maintenance

I faced this issue firsthand while automating a recurring software maintenance process. Initially, the task required approximately 14 hours of engineering work. The steps were intricate, encompassing the identification of outdated AI model servers, verification of upstream sources, grasping compatibility changes, rebuilding container images, and updating templates. After deploying to OpenShift, further validation was necessary along with promoting artifacts and creating pull requests. This multilayered process is indicative of the challenges faced by many engineers who attempt to integrate agentic software into their workflows.

This isn't just about one individual’s experience; it reflects a broader issue within software engineering. Many teams wrestle with intricate workflows that demand precision and timely execution. When automating these processes with agentic software, engineers often find themselves investing an exorbitant amount of time troubleshooting the software instead of focusing on their core tasks. Where one might expect time savings, the reliance on agentic systems often complicates matters further.

To make matters worse, the stakes are high. Downtime during maintenance can lead to significant financial losses, not to mention reputation damage. Teams subjected to tight deadlines can't afford unforeseen complications. Thus, although agentic software can hold promise, the cost of getting it right—or wrong—can be substantial.

Transforming the Process

By rethinking the approach without LLM reliance, I managed to condense this multifaceted workflow down to just 2 hours. The key was refining the procedures to enhance efficiency and ensure that every action had a clear, repeatable structure. Instead of deferring to an LLM to tackle the entire problem, I focused on streamlining the existing process through traditional scripting and automation tools.

It's easy to gloss over the value of tried-and-true methods in favor of the latest technology. But sometimes, the most effective solutions lie in a blend of old and new practices. By creating a step-by-step guide that engineers could follow, I minimized ambiguity. It standardized the process, allowing team members to jump in without deep-diving into the complexities typically required. What this did was free up mental bandwidth, giving engineers space to tackle higher-level challenges instead of getting bogged down in repetitive tasks.

(And this is the part most people overlook) — The real trick isn’t in using fancy tech, but in making systems that people can understand and follow. Traditional scripting is often underestimated in its ability to drive efficiency. A well-crafted script can outperform an experimental LLM model, rendering the software maintenance process not just swift, but reliable as well.

Implications for Future Development

The transition from reliance on agentic software to more traditional approaches has implications that businesses should consider moving forward. First, it raises the question of where to allocate resources and investment. While new technologies often promise improvement, the value of established methods cannot be understated. If teams opt to shift focus to new agentic systems without thorough vetting, they could end up spending more time fixing bugs than delivering solutions.

This also forces a re-evaluation of training programs. If you're working in this space, consider the skills your team will need. As businesses increasingly attempt to leverage AI, the demand for engineers skilled in traditional programming and systems management is likely to remain strong. Training initiatives should reflect that. Moving forward, companies may find that a hybrid model—integrating both traditional programming and agentic software—provides the best outcomes. This may allow teams to harness the strengths of LLMs while maintaining efficiency and clarity in their workflows.

Ultimately, the conversation should pivot from simply adopting the latest tech solutions to focusing on process effectiveness. The priorities must include efficiency, reliability, and continuous maintainability—all factors critical to ensuring that software systems serve their intended purpose without becoming a burden.