What’s next after Agile?
This is the first of four posts to introduce my views on how software development is changing in the agentic era. In this post, we focus primarily on process and we'll talk further about some of the topics mentioned here in future posts.
Consider a bold new era where engineers confidently execute technical plans knowing it’s the right one every time. That’s the era that follows agile. How do you achieve this? Prove then build. Learn how to ask questions. Understand non-determinism. Every engineering organization is seeking to leverage agents to achieve output orders of magnitude higher than before, and in a world where speed is a part of your moat, the real difference is and has always remained ensuring your engineers are working on the right things.
Prove then build. That is how your organization go from ideation to actual customers. Do it with high confidence that the products you ship will be achieve market fit. Let’s break it down.
Prove then build
What does it mean to prove then build? Let’s take a step back and look at a different industry: comic books. Every comic book starts with a simple script then storyboarding before the actual artwork even occurs. The storyboard is crucial: it determines the composition on every page from start to finish. If the storyboard is low quality, everything else that follows is as well. While agents can turn storyboards into artwork quickly, they cannot generate a proper storyboard. More deceptively, agents can generate many storyboards but if none are the right answer then it was all wasted effort.
Design the storyboard and then agents can help move that to the finish line by generating final artwork from it. That final artwork can be the starting point for artists or how the author and artists discuss what they'd like to see. For engineers, writing down the correct spikes for a project, and the right questions in them, is the equivalent.
How's that translate into product development? We're trying to prove what we're about to build is the right direction for the product. We start with understanding the product we're building and why it will achieve product market fit. From there, we figure out what bets we are making, both technical and non-technical. Then agents start to shine. Experimentation and prototyping is fast.
After trying out photo upload to interior colour scheme, someone on the team points out that they were chatting with the sales director and another useful feature was uploading floor plans and deciding on furniture placement. The team isn't sure if the Interior Design Agent is even capable of doing that, while the product manager isn't sure which feature should take higher priority.
Discussing, they find that there are two easy experiments to run that can use existing data. The first is to product a series of colour schemes to show to internal people in the company who are subject matter experts, and another is to produce a series of furniture recommendations. Both are just running off of existing data and then producing output that is put onto an S3 bucket. Then, the eng and product manager book meetings with the subject matter experts to walk through and see if the output is of sufficient quality.
There's no need to consider how the app might look, the upload flows, what vendors to choose for uploading, security concerns around user uploaded data etc. Those are items to consider when drafting up the production plan.
Understanding Agile
Before we talk about a new process, let’s reflect on the old one. Agile made a promise: in quick one or two week sprints you could iterate your way to a successful product market fit. At the end of each cycle was a course correction. But its main obstacles were sunk cost fallacy, system complexity and the high cost of experimentation. Engineering teams rarely scrapped work at the end of a cycle to take a new direction. Complex systems needed too many pieces stood up just to see a prototype. In many cases, one to two weeks was too short an iteration to see which direction to go in.
If we can carefully tease out exactly what we need to know, be able to answer those questions then we would have a process where we don’t need to be able to see the future.
It seems simple enough. First sprint, your team is to spend two weeks building out a prototype; that's Agile, fast and quick, quickly get something in the hands of users. But at the end of two weeks is a skeleton UX and a dummy API backend that pulls up the same five results. You want to show this to a customer but the product manager is concerned; they don't want to show something so unpolished to a paying customer. Instead, you shop it around internally but the feedback is non-existent; there's not enough of a product there to say anything.
The team keeps going. It takes another sprint before the backend is up and running. This time you can see some of the results and the UX looks alright. It's now been four weeks before showing it to the customer. They don't like the UX at all but the information it pulls up seems decent.
Since the company wants to ship this product in two months, there's no time to scrap the UX and start over. Instead, the team comes up with a few tweaks and then at the end of the next cycle looks for more feedback. The customer feels heard but now their concerns are around whether the results are relevant; on closer inspection it's too biased in a certain direction.
Fast forward a month later; there's a dozen nice-to-haves and when it goes to the first ten customers beyond the ones involved in the Agile process, none of them like the UX. It's clunky and weird, with flows that only made sense because the team was reacting to someone's feedback instead of re-thinking the core of it. Sound familiar? It's an oft repeated problem in Agile.
Learn how to ask questions
Start by asking the right questions: what are our bets, both technical and non-technical? What are you doing that is fundamentally different from before? What is just work? Understanding your bets sets what experimentation and prototyping needs to happen. Knowing how your product is fundamentally superior determines the why behind what you’re doing, allowing you to be nimble and pivot as needed as you learn more. Knowing what is just work sets the tone of where your focus and concern should be.
A bet is where you need to build confidence in whether a key feature must function in order for the project to be successful. For example, picture an LLM where the app is providing historical context to a prompt and a usual question is “would a structured perspective, or hierarchical view of that information result in superior output?”
Agents shine in answering these questions. But humans need to ask them. Agents can quickly stand up enough code to run experiments or prototypes. Then your team can discuss and change direction. This can happen on a daily basis, allowing high flexibility to change direction. What you’re doing is learning more about a concept; what you aren’t doing is build multiple versions of the same thing and hoping one sticks to the wall.
That is the paradigm shift that agents provide. Ask questions then get the answers in a day.
You might see the obvious bet here: use an LLM agent to immediately identify what ads can and cannot be displayed alongside each new ad that's uploaded. There might be an additional question: are there any performance concerns around updating these relationships for every new ad uploaded?
It's easy to set the team's focus at this point. If an LLM agent can accurately build the safe and unsafe relationships with other ads then you have a new product. If not, there's no product to be had; forget any other details.
Understand non-determinism
What are you doing that is fundamentally different from before? If you’re using generative AI as a part of your product you need to ask what the paradigm shift is. Tokens are much more expensive than DB calls. The key paradigm shift is that LLMs make decisions where there is a verifiably correct answer, and where it shines is that the sunk cost of building a deterministic system that can provide all the right answers is impossibly high.
Learn what LLMs do. You don’t give a skilled labourer a jackhammer and ask them what to jackhammer. Similarly, giving a person a spreadsheet doesn’t turn them into an accountant. Figure out what non-determinism gains you.
No more frontend devs. Dream achieved. Except now the website is ten times slower, there's constantly rendering errors with no consistency in reproducing the errors and resource consumption is way up. That didn't seem like a great use for an LLM did it? (And yes, someone actually did this 🙁)
In order to bolster the fraud detection in a payments system, an additional LLM agent is put into place that generates a confidence score based on guidelines, descriptions and other subject matter expert input to produce an answer. It's used on top of existing fraud detection and while resource intensive (ie. there's many payments per second that needs to be processed), it's worth it if it even bumps up the fraud detection rate by a percent.
This helps cover the gap that a pure rules-based system leaves for bad actors to exploit.
Conclusion
Once you’ve proven it works you can build it. Now comes the technical plans, product requirements, test cases and everything else. Tooling helps here too but it’s not about faster development. Ask what humans should do; and the answer is primarily about the architecture, the data models, the flows etc. Automate everything else so that your engineers can focus on the important bits. If you wouldn't give the top engineer in your company licence to skip code review, CI test runs, etc., you shouldn't do it for agents.
The core story hasn’t changed. Telling your eng to build the wrong things is still expensive. What we can do now is ensure velocity vs speed: prototype to understand the right path forward, not vibe code ten incorrect final products.