Using AI Agents in Software Development

Introduction

Artificial intelligence is making a big impact in a lot of sectors, especially in IT and development.
We often hear the following quote

AI will replace developers

I don't believe we're at that point yet. AI does help developers in development but does not replace them entirely. However, hearing that quote over and over again makes you wonder how true that theory is.

Over the last couple of months, I experimented with AI agents by asking them to build the same medium-sized web application using different development strategies.

Development Strategy

To keep the comparison fair, I used the same application and the same overall requirements for both strategies.

1. Create the architecture documentation and instructions for the agent to follow, which tell the AI exactly how to build something.

2. Detailed prompt with the content/features required in the application, which tells the AI what to build and lets it decide how.

Strategy - Architecture Documentation

Providing documentation for the agent results in less token usage because the agent doesn't need to decide on everything.

On a small scale:

• Simple web application with a generic design.

• 1 Backend service.

The results were very good and almost perfect.

However, in a more complex application:

• Complex web application with professional design, around 5 different features, and an admin console.

• 3 Backend services.

The agent over-engineered the application and introduced multiple critical bugs.

To the point where the application was unusable, with components not showing and features not working, even after iterations of requesting testing and validation of the features.

What I observed, the more architectural freedom the agent had across multiple services, the more likely it was to introduce unnecessary complexity.

Strategy - Detailed Prompt

Providing the agent with a detailed prompt about the features needed in the application and the overall feel of it, while keeping the technical and architectural decisions in its hands, resulted in burning through the tokens faster. However, the application was not broken even when it came to a large-scale application.

To be fair, it did introduce bugs and required some iterations to fix everything. But overall results were surprisingly better.

• Cleaner code.

• Much better UI and theme consistency.

• Less hallucination.

• Fewer bugs.

Limitations

When it comes to the limitations of AI agents, I noticed multiple things while conducting my experiment.

Some of the solutions I found for these limitations are described in the next section.

• Once the context of the session becomes large, the agent starts hallucinating. It starts to assume something is working when it's not and implements or changes unrelated components.

• If the requirements are given all at once in one prompt and the agent is asked to implement all at once, it doesn't. It implements a skeleton of what you want, but when the requirements are large, it cannot be implemented at once.

• AI agents struggle with understanding requirements about UI/UX if they're not given clear instructions and samples of what to develop.

• AI agents generally won't prioritize security, testing, and automation unless those requirements are explicitly included.

• Asking the agent to run separate sub-agents with different purposes results in surprisingly better results than you would imagine.

• AI agents are almost addicted to fixing issues, even if you don't ask it to. It will touch unrelated files and change them unless you tell it not to do that.

Solutions

The solutions described here are not perfect and do not resolve the limitation entirely, but from my experience, they did provide enough help to consider them valid.

I hope they will provide enough value for you as well.

Solutions - Large Session Context

In a large session context, the simple solution for this issue would be to clear the context or start a new session. However, the agent would have to explore the project again and burn through your tokens.

To avoid that issue, prompt the agent to create a handoff document to help you move from one session to another easily.

Solutions - Handoff Document

I discovered this solution very late in my experiment, and it made a huge difference in how I moved from one session to another and between agents.
A handoff document should include the essential points of your project to make it easier for the agent to pick up the work.

Solutions - Phases

To implement big requirements, prompt the agent with your requirements even if they're a lot.
But ask it not to implement or change anything. The first response should provide the list of phases for implementation and wait for approval after each phase.

The agent must run all required testing and validation after the completion of each phase.
Using this method resulted in much better overall results in the end from the first iteration.

Solutions - UI Design

AI agents struggle with understanding UI requirements without clear instructions and samples of what to implement. If not asked specifically, you will end up with a generic-looking application.

To avoid this issue, I found two solutions:

• Provide the exact requirements of your design, such as fonts, colors, and behavior.

• Provide the agent with a prototype of your application.

Providing a prototype gets you much better results. There are multiple free tools that you can use to generate a prototype and use it to develop your final application.

Solutions - Security and Testing

Security and testing are not always considered by AI agents. You have to specifically ask for it.

For testing, make sure your prompt includes a section for testing. The agent must implement unit and integration testing for each new compoenent were applicable.

For security, you can include it in the first prompt. But don't consider it done.
Once you're satisfied with the application, ask the agent to "Act as" a security engineer and test your application.
You will be surprised by the results.

Solutions - Sub-agents

Deploying sub-agents with different purposes provides very good results. I still don't fully understand it.
But when you ask an agent to "Act as senior security engineer," the results become much better than not giving those instructions. It might be because working with a different set of instructions or rules makes it behave differently than having generic instructions.

I deployed multiple agents in parallel, all responsible for different parts of the application.

• Backend

• Frontend

• QA

• UI

• Security

Their responsibility was to only check the application and write reports without implementing anything.
The results were very good. They found multiple hidden issues and room for enhancements.
Once they were done, the original agent would go through the reports and fix the issues based on the findings.

Conclusion

Will AI replace developers?

Not in the near future.
AI agents are very capable development tools, but they're just that.
They are tools to accelerate development and identify issues. But they still require someone who understands architecture, quality, and security. They're excellent assistants, but not independent software engineers yet.

But don't take my words for it. Try it yourself and tell me about your experience.