Most software leaders we speak to have already said yes to AI. Developers use coding assistants, someone in QA is experimenting with generated test cases, and a project manager is pasting meeting notes into a chat window to draft the weekly status update. What few organisations have is a clear answer to the question that matters: is any of this making us deliver better software, faster, with fewer surprises?
This article is a practitioner's view of AI in software delivery. We cover where AI genuinely pays off across the software development lifecycle, where it disappoints, the guardrails you need before you scale it, and how to run a measured four-week pilot so your decision is based on evidence rather than enthusiasm.
The real question: faster at what?
AI tools make individual tasks quicker. Writing a function, drafting a test, summarising a long thread. But delivery speed is rarely limited by how fast one person types. It is limited by flow: how long work waits for a decision, a review, an environment, a tester or a release window.
If your developers write code twice as fast but every pull request still sits for three days waiting for review, your lead time barely moves. Worse, you now have a bigger queue of unreviewed code. That is why we start every AI conversation with the delivery system, not the tool.
Before you buy more licences, ask three questions:
- Where does work wait longest in our lifecycle today?
- Which of those waits is caused by effort (someone has to write, read or check something) rather than by a decision or dependency?
- Do we have the data to tell whether a change made things better?
AI helps with effort. It does not help much with waiting for a stakeholder to make up their mind.
Where AI genuinely pays off across the lifecycle
Used well, AI is a capable assistant at almost every stage of delivery. The pattern that works is consistent: AI produces a first draft, a human with context reviews and owns it.
Requirements and backlog refinement
AI is good at turning rough notes into structured user stories, suggesting acceptance criteria, spotting missing edge cases and flagging stories that are too large to finish in a sprint. A product owner who arrives at refinement with AI-drafted stories and a list of open questions makes that meeting far more productive. The product owner still decides what matters and why.
Coding assistance
AI coding assistants are most useful for boilerplate, unfamiliar syntax, small refactors, writing glue code and explaining legacy code that nobody remembers writing. In our experience, the gains are largest for well-understood, well-specified tasks and smallest for novel design work or deep domain logic.
Code review
AI can do a first pass on a pull request: summarising the change, flagging obvious bugs, inconsistent naming, missing error handling or tests. This lets human reviewers focus on design, intent and risk. It should shorten review time, not replace the reviewer.
Test generation and QA
AI for QA testing is one of the more practical uses we see. LLM-based test generation can draft unit tests from code, propose test cases from acceptance criteria, generate test data and help convert manual test scripts into automated ones. The QA engineer's role shifts towards deciding what is worth testing, reviewing generated tests for meaningful assertions and exploring the risky areas that no script covers.
Documentation and release notes
Few engineers enjoy writing documentation, so it is often missing or stale. AI can draft API docs, README updates, onboarding guides and release notes from merged pull requests and ticket descriptions. A human edits for accuracy and audience. This is low risk and usually an early, visible win.
Project management: status reporting and risk spotting
AI project management support works best on the administrative layer. It can draft status reports from ticket data, summarise meeting notes into decisions and actions, and scan the backlog or RAID log for warning signs such as tickets that have been in progress for too long, dependencies with no owner or scope that keeps growing. The project manager then spends more time on conversations and decisions and less on compiling slides.
AI across the delivery stages at a glance
The table below summarises how to use AI in software teams, stage by stage, and what to watch for at each one.
| Delivery stage | Where AI helps | Watch out for |
|---|---|---|
| Requirements and backlog | Drafting user stories, acceptance criteria, edge cases, splitting large items | Plausible-sounding stories that nobody has validated with a real user |
| Coding | Boilerplate, refactoring, explaining legacy code, unfamiliar languages | Code nobody fully understands; licence and IP exposure; security flaws |
| Code review | First-pass checks, change summaries, spotting missing tests | Reviewers rubber-stamping because "the AI already checked it" |
| Testing and QA | Generating unit tests, test cases from criteria, test data | Tests that pass but assert nothing useful; coverage numbers without real confidence |
| Documentation | API docs, READMEs, onboarding guides | Confident but outdated or invented details |
| Release | Release notes from merged changes, change summaries for stakeholders | Notes that describe intent rather than what actually shipped |
| Project management | Status drafts, meeting summaries, risk and dependency flags | Reports that look polished but hide the real issue; sensitive data in prompts |
Where AI disappoints
The failures we see are rarely about the technology. They are about what goes into it and what happens to what comes out.
Unclear requirements in, unclear code out
If a story says "improve the checkout", an AI assistant will happily generate something. It will be confident, syntactically correct and probably wrong. AI fills gaps with assumptions. When requirements are vague, you get faster delivery of the wrong thing, which then needs rework. Clear acceptance criteria matter more with AI, not less.
Amplifying a broken process
If your team already has long review queues, unstable test environments or a release process that depends on one person, AI will make more work arrive at those bottlenecks faster. Throughput at the start of the pipeline goes up; lead time does not. We cover the common causes of this in why software releases slip.
Unreviewed output
The most dangerous pattern is quiet trust. Generated code merged without a careful read, generated tests nobody checked, a status report sent to a client without anyone verifying the numbers. Each one is small. Together they erode quality and accountability, and the cost shows up later as escaped defects and awkward conversations.
Security and IP leakage
Developers paste code, logs, customer data and credentials into whatever tool is to hand. Without a policy, you may be sending proprietary source code or personal data to services your contracts and regulators never approved. Generated code can also introduce insecure patterns or snippets with unclear licensing.
Warning signs that AI is amplifying problems rather than solving them:
- More pull requests opened, but review time and work in progress are rising.
- Rework and bug-fix tickets are growing as a share of the backlog.
- Nobody can say which tools are in use or what data goes into them.
- Senior engineers are spending more time correcting generated code than before.
The guardrails a company needs
Guardrails are not about slowing teams down. They are what let you scale AI use without a nasty surprise six months later. Keep them short enough that people actually read them.
1. What code and data can go where
Classify your information into simple tiers and state which tools are approved for each. For example:
- Public or non-sensitive: any approved tool.
- Internal source code and documents: only tools with a business agreement that excludes your data from model training and meets your retention requirements.
- Customer data, personal data, credentials and secrets: never in prompts, unless a specific, approved setup exists for it.
Check your client contracts too. Some prohibit sending their code or data to third-party services at all.
2. Human review is mandatory
Every AI-generated artefact that ships, whether code, tests, documentation or a client report, is reviewed by a named person who understands it. Make this explicit in your definition of done. "The AI wrote it" is never an acceptable answer in a post-incident review.
3. Clear ownership
The person who merges the code owns the code. The person who sends the report owns the report. Assign an owner for the AI policy itself, usually someone in engineering leadership or the PMO, who maintains the approved tools list and reviews it regularly. A simple RACI for AI decisions helps; our guide on how to set up a PMO covers how governance like this fits into wider delivery oversight.
4. Security checks stay in the pipeline
Static analysis, dependency scanning and secret detection should run on all code, generated or not. If anything, tighten them when AI use increases.
How to run a measured 4-week AI pilot
A pilot should answer one question with data: did this change improve delivery for this team? Here is the structure we use.
Week 1: pick one bottleneck and baseline it
- Choose one team and one bottleneck. For example, slow code review, a thin regression suite or hours lost to status reporting.
- Pull baseline metrics from the last 6 to 8 weeks of delivery data. Useful measures include cycle time, pull request review time, escaped defects, rework rate and, where relevant, DORA's lead time for changes and change failure rate.
- Agree the guardrails for the pilot: approved tools, data rules, review expectations.
- Write down what "success" means before you start, for example a meaningful drop in review time with no rise in escaped defects.
Week 2: set up and start
- Configure the tools and run a short working session with the team on good prompting and good reviewing.
- Start using AI on the chosen bottleneck only. Resist adding other use cases mid-pilot.
- Capture quick qualitative notes: where it helped, where it got in the way.
Week 3: run and observe
- Keep going with normal work. Track the same metrics weekly.
- Watch the guardrails in practice. Is review still thorough? Is any sensitive data finding its way into prompts?
- Hold a mid-point retrospective and adjust how, not what, you are testing.
Week 4: compare and decide
- Compare pilot metrics with the baseline. Look at quality measures alongside speed, because faster with more defects is not a win.
- Gather team feedback and the cost of the tools.
- Make one of three decisions: scale to more teams, adjust and run again, or stop.
- If you scale, turn what you learned into a short rollout playbook: approved uses, guardrails, review checklist, metrics to keep watching.
For example, a team of eight that baselines review time, adds AI first-pass review for four weeks and sees review time fall while escaped defects stay flat has a clear case to expand. A team that sees review time fall but defects rise has learned something just as valuable, at low cost.
If you want a quick view of where your own process stands before you choose a bottleneck, start with our free Agile & AI Delivery Health Check. It takes a few minutes and highlights where flow is breaking down.
Fix flow first, then add AI
If your baseline shows work waiting in queues, unclear priorities or an overloaded release process, the best AI investment you can make is to fix those first. That usually means limiting work in progress, making the workflow visible on a Kanban board, tightening the definition of ready and done, and giving someone clear ownership of delivery governance.
A software delivery assessment is the quickest way to find out which of these applies to you. Where the problem runs deeper, across teams and ways of working, an Agile transformation programme or targeted Agile coaching and training will do more for lead time than any tool.
Next steps
At Arham Tech we are AI-enabled and human-led. Our AI in software delivery service, the AI Delivery Accelerator, is a fixed-scope, four-week engagement that finds where AI will genuinely speed up development, QA and project management in your organisation, proves it with a measured pilot, and leaves you with guardrails and a rollout playbook. Packages start from USD 7,500, with the exact fixed fee agreed before we begin.
If you would like to talk through where AI fits in your delivery lifecycle, book a free discovery call. There is no obligation, and you will leave with a clearer view of your next step.
Frequently asked questions
Will AI coding assistants replace developers or testers?
No. In our experience AI changes the shape of the work rather than removing the need for skilled people. Developers spend less time on boilerplate and more on design and review. Testers spend less time writing scripts and more on deciding what matters and exploring risky areas. Someone with context still has to understand, review and own every output that ships to customers.
How do we stop developers pasting sensitive code or data into AI tools?
Publish a short policy that classifies information into tiers and names the approved tools for each. Provide a sanctioned tool with a business agreement covering data retention and training, so people are not tempted to use personal accounts. Keep secret scanning in your pipeline, check client contracts for restrictions, and assign an owner who reviews the approved tools list regularly.
Which metrics should we track to prove AI is helping delivery?
Baseline before you start, then compare. Useful measures are cycle time, pull request review time, escaped defects, rework rate and DORA's lead time for changes and change failure rate. Always pair a speed metric with a quality metric. If work moves faster but defects or rework rise, the change has not improved delivery, it has moved the cost downstream.
How long does it take to see results from an AI pilot?
A focused pilot on one team and one bottleneck can give a useful answer in about four weeks: one week to baseline and agree guardrails, two weeks of running, and a final week to compare and decide. The key is narrow scope. Trying several use cases at once makes it hard to tell what caused any change you see.