Structuring your org for the future

This is the second of four posts to introduce my views on how software development is changing in the agentic era. These are introductory thoughts to the structure of your org (whether you call it R&D or EPD). In the future, we'll have future series that deep dive into specific topics that help build healthy org dynamics.

As agents, agentic processes and non-deterministic systems become a part of daily life the difference between well-structured orgs has become greater. Agents act as an amplifier, fuel that accelerates what’s already going on in your organization and its institutions; good or bad. Therefore, to maximally benefit leaders must build the right rituals, talent pipeline and behavioural incentives. The best orgs leverage agents to build with ten times the confidence, not ten times the code.

Discuss don’t track

Agile and kanban encouraged quick stand-ups with the idea that people can showcase what they’ve finished yesterday and what they’ll work on today. It was to keep a fast pace. In practice the updates held little meaning because it didn't indicate whether people were moving in the right direction.

Teach your teams to discuss decisions that need to be made. A person building an eval system for an LLM might need to discuss what the definition of coverage should be; otherwise they may focus on building a set of evals that do not meet the needs of the system.

Use tools to offload the work of task management, that’s the point of systems like Jira or Linear. It’s better to have a team spend an hour in the morning discussing to use the rest of the day well than to keep stand-up short and have everyone execute their tasks blindly.

Decisions sometimes needs data. Ask what data you need not what you can collect. A team needs to know exactly what is going on in a system, whether the product is working well and whether there needs to be more instrumentation or data collection. It’s the difference between data informed to help make the right choices, versus letting existing metrics blindly drive decisions.

You have the data for the drop-off at each step. What you don't have is why they dropped off. Is it because of the amount of information they need to fill out, therefore the length of time in a particular step is the issue? Is it because they don't get a preview of how long the entire process might take? Is it because certain kinds of information is considered highly sensitive and therefore creates a lot of friction if they need to enter it before using completing the process?

The only information collected is the time per step. But that doesn't help answer the questions above. You need to run experiments to gather the data you need. Perhaps you could test a progress bar or organize information onto different steps to see if any of them cause greater drop-off. If there's enough traffic, you could use A/B tests.

Strategic vs Tactical

Understand strategy. At each level a good leader figures out the right things to do and their team figures out the right way to do it. A line manager ensures a team prioritizes the right projects while the team and its tech lead figures out the right way to do them. Above that, a senior manager or director ensures an engineering group is executing the right strategy, while the line managers ensures their roadmaps are done right. And so on.

How do you know what is the right thing to do? Ask why customers buy your solution, what sets you apart, what you are doing that is fundamentally superior to your competitors. Do this at each level. Each piece of your company should act like its own business unit. What’s their value proposition? Why should other teams use their output instead of going to a different vendor? If you're a dev tooling team, you're a business competing with external dev tool sass vendors.

Understand tactics. You need people capable of diving into edge cases, unhappy paths and other minutiae. That's the role of individual contributors assigned as project leads; they should be working on the nitty gritty details. But remember, don't confuse this high fidelity work as strategy. A set of features isn't a strategy. Figuring out a set of complex UX flows is not a strategy; if changing the exact features constitute a complete shift for the team, there was no vision. The paradigm is whether people are connecting what their working on to why they're working on it; if they can do that then they can connect many different ideas of "what" to the "why" and allow a team to be flexible.

Know the difference the between strategy and tactics. Most teams only have tactics nailed down. If you can change what people are working on but the why hasn’t changed, then you have a strategy. 

Here, the overall strategy is the thesis: if users have more control over the ads they see in their feed then they're happier and therefore stay longer on the platform and offset any decrease in short-term ad revenue with longer lifetime user activity. But what features does this imply? There are many possibilities here; those are the tactical decisions.

One possibility is to build a configuration settings page that allows users to decide what kind of topics are relevant to them, and offers suggestions based on their previous browsing behaviour and clickthroughs.

Another possibility is to provide buttons while browsing that allows them to boost or demote certain kinds of posts and send this information through to the overall ad auction system (perhaps as an additional bid that is trained specifically on this user data).

If while testing one you find it is not as promising as expected you can quickly pivot to another choice. There's no whiplash effect on the team; it's clearly obvious you're trying out different ways to give users more control over the ads they see. That ability to be nimble is because you have a strategy. Those specific projects and the gritty details in them are the tactical decisions.

Consider the whole software cycle

Software starts from an idea. It could be a business bet, a customer need or a feature request. In the past, the disciplines involved in converting ideas into software were far more limited. With agents, the number of people who can bring idea into the experiment phase are far greater. This has created the impression by many non-engineers that anyone can "write code" and produce products. While that's not the case there are significant gains using agents across eng and non-eng. We should look at how agents help at every step of the software cycle.

Product development is a long process, only one step of which is writing code that needs experts in every phase. We're still early in using agents so we don't know what the total possibilities are but we can already glimpse some of it.

A typical software development lifecycle looks something like this:

• Ideation, experiments and prototypes: Agents have already proven their value through vibe coding or quickly standing up systems to prove out features.
• Production planning: Technical and product documents are still mostly produced by hand. Agents can be used to do codebase research, or construct tools to help with information gathering to help with the planning process.
• Codebase LLM friendliness: Our codebases are artisanally built by engineers for other engineers. What we've not done is make them LLM friendly or fully understood what's needed to do so (eg. tagging, labelling, comments, agent-friendly libraries, etc).
• PR generation and review: There's a growing number of frameworks that are coming and going by the month that help with turning tickets from Linear or Jira into PRs. There's just as many new vendors pushing out review bots. We're seeing some early value here but everything still requires a human pair of eyes (lest you accidentally delete everything via agent vibing).
  • Note: If Linus Torvald needs a code review before shipping to prod then your coding agents sure do.
• Automated testing: Once more unto the breach! We're seeing another revival of self-healing test scripts and agents unlocking more interesting ways to test. One early point of view is to think about what actions you can take in your system and provide that as tooling to an agent whose prompt description is "you're a user that uses this product, this product is...".

Keep your org at the forefront of this exploration by staying nimble rather than rushing to define processes. In industry, if it's not settled yet then think of how it can be explored and let people try anything out. It's about focusing the energy of the people within your org on unsolved problems, rather than re-hashing established solutions.

There's a couple options in using agents to speed up the early design ideation process of product development.

• Engineers quickly build the proposed design for designers and product managers to play with in the dev/staging environment
• Designers vibe code apps that reflect what they're thinking and use this in design review instead of a mock-up

As well, there are ways to reduce friction in seeing the final product and giving a stamp of approval to the full production plan.

• Building up an agent-friendly UX components library and hooking it up to your coding agent environment
• Creating a comprehensive md file for rules and coding style that can be used in PR generation (perhaps as a step after initial code generation to immediately clean it up before human review)

It can be useful to identify a few champions in various areas to help lead exploration in specific areas in your org (eng and non-eng). Sometimes you want fewer chefs in the kitchen trying out recipes at a time to make it easier to understand if a process update was good.

Build a healthy talent pipeline

Senior engineers benefit more from agents because they’ve reached a point where coding is merely one step in product development. But you still need to hire and train up junior engineers or else you end up competing for talent from a diminishing pool of capable people. While the gap in value add between junior and senior members of a team has widened, the need for a healthy talent pipeline has remained crucial.

Remember that the fundamentals still matter. Giving someone a spreadsheet doesn’t turn them into an accountant. Buying a Cursor licence doesn’t turn a new grad into a principal engineer. Agents are like calculators where the people who are best able to use them are the ones who know what they need to calculate. Keep up training programs that teach people the software principles needed to help them become the next bar raiser in your team. We'll touch on how to train in the future.

Sound like a familiar situation? A common outcome is that the engineer is spinning in circles; either it feels like whack-a-mole or they're unable to figure out what's wrong. They leverage an agent but all it does is gaslight them.

Instead, at the end of the day, you have to pair them with a staff-level eng who walks them through what it means when an the SQL analyzer says this or that. The intermediate engineer still has to learn about indices or learn how a query actually traverses a database. Eventually, after a few weeks of code reviews, pairing sessions and data collection, the intermediate engineer finally improves the performance of the analytics page.

It may seem costly but six months later, they're now someone who can fix it independently or train up the next person.

Conclusion

Agents are amplifiers. Orgs that strive to be the best will enjoy that amplification. It’s important to be incredibly objective. Well aligned incentives means business plans become a reality rather than a suggestion. Structure your org to use agents well and you’ll be highly confident in the features your teams plan and ship. Strong strategy and talent pipeline means a healthy org today and tomorrow. Team processes and their daily rituals convert into the tactical decisions made on each project; good use of agents means high confidence output and poor use means mountains of pointless iterations that merely look like motion. Make sure you're an org where the amplification is magnifying the lead you have as a well-oiled machine.