AI Software Development: What Works in Production
A practical guide to AI software development: where AI fits each SDLC stage, how to measure real delivery gains, and the controls production teams need.
Across three randomized controlled trials covering 4,867 software developers, those with access to a generative AI coding assistant completed 26.08% more tasks than the control group, a result recorded in the 2024 Microsoft, Accenture and Fortune 100 field-experiment study. That number gets pasted into board decks as if it were a delivery forecast. It isn't one.
More code leaving the pipeline doesn't make a service more reliable. It moves the queue. Review backlogs lengthen, coverage falls behind the diff, and the integration nobody specced arrives in a 900-line pull request that a tired reviewer skims at 5pm. AI software development pays off only when generation capacity and validation capacity grow together. Speed without validation is defects arriving earlier, with better formatting.
The operating model that works is less about which model you license than about where you put the gates, what you measure, and who signs their name.
Where AI Fits Across the Software Delivery Lifecycle
AI software development means applying models and agents to the work that turns a requirement into a running service, from the first ticket to the 3am page. It behaves like a set of power tools rather than one machine. A router, a nail gun and a table saw each need a different guard, and a guard built for one will hurt you on another.
Error cost is what separates the stages. A bad summary of a user story costs a conversation in refinement. The same misreading shipped into a billing engine costs an incident review, a rollback and a customer credit. That gap should decide where you deploy AI first, not which tool has the slickest demo.
The diagram below shows where the tooling lands across the delivery lifecycle.

Pair every application with a check, and put the check live before the application is.
| Lifecycle stage | Typical AI application | Required check |
|---|---|---|
| Requirements and planning | Drafting user stories, summarizing tickets | Traced against source requirements |
| Design | Comparing architecture options, drafting decision records | Reviewed against team standards and architecture rules |
| Coding | Code completion, boilerplate, refactoring, API integration | Automated tests, static analysis, human review |
| Testing | Unit test generation, edge-case suggestions | Coverage and mutation testing |
| Review and security | Diff summaries, flagging risky patterns | Human sign-off, security scanning |
| Deployment and operations | Runbook drafts, incident summaries | Change approval, rollback plan |
The check is a staffing decision, not a checkbox
A stage without its check is the place where unreviewed output reaches users. Requirements drafts get traced back to what the business asked for. Designs get read against architecture rules the team already agreed on. Coding output runs the same pipeline as a human commit, with nobody granted a bypass because the diff was generated.
The check also names a person. Test generation needs a reviewer who knows which edge cases matter in this domain. Deployment runbooks need someone who has executed a rollback under pressure. When no name comes to mind, the stage isn't ready.
Deployment order: if a stage can't name its check and the owner of that check, it isn't ready for AI, however good the demo looked.
Start where mistakes are cheap and tests are strong: boilerplate, unit-test drafts, refactoring, documentation. Production logic and deployment runbooks come later, once repository-level evidence supports them. For a fuller stage-by-stage view of how the work itself changes, how AI changes each development stage is a useful companion read.
The tooling choices matter less than the fit. When we build agentic workflows, we use Copilot Studio, Power Platform and Azure AI process automation on the Microsoft stack the client already owns, so the governance controls in place are the ones their IT and security teams already run.
What Adoption and Productivity Data Shows
Adoption is settled. 84% of developers use or plan to use AI tools, according to the Stack Overflow 2025 Developer Survey of more than 49,000 respondents across 177 countries. The interesting spread sits between "using" and "using daily", and the 2025 sources measure it differently enough to be worth putting side by side.
| Source | Sample | Finding |
|---|---|---|
| Stack Overflow Developer Survey 2025 | More than 49,000 respondents, 177 countries | 84% use or plan to use AI tools |
| Stack Overflow Developer Survey 2025 | Professional developers | 51% use AI tools daily |
| Google DORA report 2025 | Nearly 5,000 technology professionals | 90% use AI, up 14 points from the previous year |
| JetBrains Developer Ecosystem 2025 | 24,534 developers, 194 countries | 85% use AI regularly, 62% use at least one coding assistant or agent |
| Randomized field study (Microsoft, Accenture, Fortune 100) | 4,867 developers | 26.08% more tasks completed, standard error 10.3% |
The percentages diverge because the questions diverge. Google's DORA survey counts any professional use across the delivery lifecycle, as the 2025 DORA report announcement sets out. JetBrains asked 24,534 developers across 194 countries about regular use and got 85%, with 62% relying on at least one assistant, agent or AI editor, per the JetBrains Developer Ecosystem 2025 survey. Stack Overflow's daily-use figure is the strictest of the three: 51% of professional developers touch these tools every working day.
Productivity evidence is thinner than adoption evidence, and noisier. The pooled field study puts the gain at a standard error of 10.3%, as the Microsoft Research publication reports. That error bar is the honest part of the result. It says gains vary widely by team, task and behaviour, and it kills any plan built on "we'll get 26% more output."
What the number actually licenses: a controlled pilot with explicit acceptance criteria, not a headcount reduction.
Near-universal adoption changes the question. It's no longer whether developers will try the tools. It's what happens to the work after the code gets written.
How to Measure Whether AI Improves Delivery
The 2025 DORA research, built on nearly 5,000 technology professionals globally, captures the gap in a single finding: more than 80% believed AI increased their productivity, AI adoption still showed a negative relationship with software-delivery stability, and 30% reported little or no trust in AI-generated code, according to Google Cloud's 2025 DORA report. Belief and delivery moved in opposite directions.
That's what an amplifier looks like. AI accelerates coding. It also accelerates defect creation, architectural drift and review workload. Which effect dominates is an operating question, and the answer shows up in your metrics within a quarter.
Six steps that produce a defensible answer
- Record a baseline first. Capture at least one full quarter of delivery data, by repository and workflow, before the first licence is issued. Without a pre-AI number, every later comparison is an argument about memory.
- Choose metrics that can move the wrong way. Change-failure rate, review rework, defect escape rate, commit-to-production cycle time and incident severity all punish you for shipping more bad code. Accepted-code volume never does.
- Segment the work. Low-risk changes such as unit-test drafts and documentation carry different failure costs than shared services or regulated logic. Averaging them hides the signal you need.
- Run the pilot against a comparison group. Pick teams on similar repositories doing similar work. One group with AI, one without, same gates, same review standard.
- Compare quality-adjusted delivery at the end of each period. If cycle time drops and change-failure rate holds, you have something. If velocity rises and rework rises with it, you've moved work, not removed it.
- Make expansion conditional on the numbers. Write the threshold down before the pilot starts, so the decision isn't a negotiation afterwards.
The rule I'd hold to: if review rework and change-failure rate aren't flat or falling, the extra throughput isn't throughput. It's inventory sitting in a queue.
The contrarian point deserves stating plainly. More AI-generated code can mean lower engineering performance when validation capacity doesn't grow with generation capacity. A working set of AI delivery metrics worth tracking helps you pick the numbers before the pilot starts rather than after the first incident.
Validation Gates for AI-Generated Changes
A passing benchmark proves a model can fix curated issues in public repositories. It says very little about whether it can change your codebase without breaking something your tests never covered.
SWE-bench is the honest version of that test: 2,294 tasks drawn from GitHub issues and pull requests across 12 Python repositories, where a system reads an issue, edits the codebase, and gets judged by whether the reported defect is fixed, as documented on the SWE-bench project site. The original retrieval-augmented baseline solved 1.96% of tasks. SWE-agent reached 12.47%. Multi-file engineering is hard, and agentic workflows improved the number fast.
Then the correction. Later audits of commonly resolved SWE-bench tasks found that a large share of issues marked "solved" relied on leaked solutions or on tests too weak to catch a wrong answer. When stricter, refreshed tests were applied, reported resolution rates fell sharply. A number that collapses when the test gets stricter was never measuring production readiness.
That gap is why gates exist at all. Once a model generates changes faster than people can read them, the bottleneck stops being writing code and becomes proving the code deserves to ship. The gate set below is what a change has to clear.

| Gate | What it checks | Owner | Evidence kept |
|---|---|---|---|
| Automated tests | Existing and new tests pass | Pipeline | Test report per change |
| Static analysis | Code quality and style rules | Pipeline | Analysis report |
| Dependency and licence checks | New or changed packages, licence terms | Platform team | Dependency manifest |
| Security scanning | Secrets, vulnerabilities, unsafe patterns | Security team | Scan results |
| Human review | Logic, architecture, fit with intent | Code owner | Approval record |
| Domain checks | Safety, traceability, compliance rules | Domain lead | Signed checklist |
| Hidden regression tests | Fresh, organization-specific issues | QA lead | Pass rate and regression rate |
The gate AI slips through
Standard gates catch broken code. They miss plausible code. A generated patch can pass every test, satisfy the linter and still hard-code an assumption that violates your architecture, import a library with the wrong licence, or log a field your privacy policy classifies as restricted. Each of those failures looks identical to a clean change at the pipeline stage, which is why the domain and hidden-regression gates exist and why review roles need explicit definition.
Who reviews what, and where an agent's summary stops being sufficient, is covered in human and agent review roles. The short version: an agent may summarize a diff, but a person owns the approval.
The control that makes all of this testable is evaluation on fresh, organization-specific issues with hidden regression tests, real repository context, dependency constraints and security checks. Track pass rate, patch-review time, regression rate, rollback frequency and human intervention, because those five numbers describe your repository instead of someone else's. When validation capacity grows alongside generation capacity, throughput holds; when it doesn't, every acceleration claim rests on unverified code. For the adversarial version of the same exercise, where the goal is breaking the gates rather than passing them, AI red teaming in production covers how to attack your own pipeline before someone else does.
Governance and Audit Controls for AI-Assisted Code
Most governance advice stops at "scan the code." Scanners are the easy part. The harder questions are ownership and evidence: who owns this code, how provenance is recorded, how model and dependency changes get monitored, and what an auditor can trace when a defect reaches production eighteen months later.
The workforce data explains why this is not theoretical. In the Stack Overflow 2025 Developer Survey, 45% of developers said debugging AI-generated code is time-consuming, 75.3% said they don't trust AI answers, 61.7% cited ethical or security concerns, and 61.3% wanted to fully understand the code before accepting it, per the Stack Overflow 2025 Developer Survey press summary. Distrust at that scale does not slow adoption on its own. It moves the risk downstream into review, which is exactly where governance has to act.

Map each control to the stage where it acts
Governance that lives in a policy document gets ignored. Governance that lives at the point of action gets followed.
- At intake, approve the tools and the data boundaries, and scope repository permissions so an agent can only reach what its task needs.
- During coding, block secrets from entering commits, and run software composition analysis on every new dependency the model suggests.
- At review, record the tool, model version and named reviewer for each AI-assisted change, so the audit trail answers "who approved this and with what."
- On model or prompt changes, require a documented review. Swapping a model is a change to your production system, not a vendor update.
- At incident time, run the response plan and confirm code and documentation ownership sits with the client, not the tool vendor.
The pattern across all five is that each control produces its evidence at the moment the decision happens. The record exists before anyone needs it, which is the only version of an audit trail that holds up under questioning.
Say a regulated team ships a defect traced to a generated helper function. If the commit records tool, model version and reviewer, the post-mortem reconstructs the decision in an hour. If it does not, the post-mortem becomes an archaeology project.
This is what makes AI software development an operating-control problem: generation capacity has already grown, so validation capacity has to grow with it. Ryshe's AI security and red team testing service covers the part scanners miss: prompt injection, data leakage, jailbreaks and compliance violations, followed by concrete fixes. Findings land in a Detailed Vulnerability Report with a severity rating of Critical, High, Medium or Low, evidence screenshots, reproduction steps and specific remediation guidance, so the output survives an audit conversation. The broader control set lives in this AI governance framework.
How to Roll Out AI in Phases
Rollouts fail when a team treats "AI" as one decision. It's a series of decisions, each tied to a risk tier with its own review requirement and its own condition for moving up.
| Risk tier | Example work | Review required | Condition to expand |
|---|---|---|---|
| Low | Boilerplate, documentation, unit-test drafts | Standard code review | Stable defect rate for one quarter |
| Medium | Refactoring, API integration | Code owner plus automated gates | Regression rate within agreed threshold |
| High | Core business logic, shared services | Senior review and hidden regression tests | Repository-level quality evidence |
| Regulated | Safety, compliance or traceability code | Domain sign-off and audit trail | Approved controls and documented ownership |
The sequence is deliberate. Low-risk work gets the team fluent with the tools while your defect data stays clean enough to read. Medium-risk work tests whether the gates hold under real refactoring pressure. High and regulated tiers only make sense after repository-level evidence exists, because that's the only evidence that speaks to your code.

A phase structure that has shipped
The aerospace data foundation project we delivered ran in four phases: a 3-week assessment, a 6-week platform build, a 5-week migration and integration, and a 2-week handoff and enablement. Each phase ended with something the client could inspect, and the handoff phase existed so their team could run the system without us.
Contrast that with what the same client had already bought. Before working with Ryshe, they had spent $180K on an AI vendor whose proof of concept never reached production. The pilot wasn't a technology failure. It had no named owner, no monitoring, and no written acceptance criteria, so nobody could declare it finished or fix what was missing.
Phase rule: every phase ends with a written acceptance decision. Approved, approved with conditions, or not approved. Silence is not approval.
The phase length matters less than the exit condition. A two-week phase with a signed decision beats a six-month phase that ends in a demo and a slide.
Confirm Your Pilot Gate Before Scaling AI
Before you widen the scope, run this test. All six items need to hold, not five.
- A baseline exists for at least one quarter, recorded by repository and workflow.
- Quality-adjusted metrics moved, with change-failure rate, review rework and cycle time improving against a comparison group.
- Validation gates run on every AI-assisted change, with no exceptions granted for generated diffs.
- Named owners exist for code, security and operations, and each one knows what they sign.
- An audit trail is active, recording tool, model version and reviewer per change.
- A written acceptance criterion for expansion was agreed before the pilot, not after the results.
If any item is missing, the pilot stays in its current risk tier. A no-go isn't a failed pilot. It's a decision to fix one specific thing before you scale.
How to read a no-go: it names the next piece of work. That's more useful than a soft yes that quietly expands scope.
A readiness review is the fastest way to find out which item will fail. Score data quality, governance, technology foundations and change capacity, then end with an explicit go or no-go recommendation rather than a summary slide everyone interprets differently.
The next two weeks
- Record the baseline for the repositories you've chosen, even if it's retroactive from your issue tracker and CI history.
- Select one low-risk workflow, such as unit-test drafts on a well-covered module, and put its gate owners in place.
- Write the acceptance criterion for expansion, with the numbers and the date attached.
If internal owners are missing, an outside team can fill those roles for a phase. Acceptance criteria and code ownership should still sit with you. Ryshe Forge includes the foundations this depends on: Azure and Microsoft architecture, identity, governance, monitoring and security, AI red teaming and production validation, then deployment with documentation and handoff. We don't introduce proprietary platforms that create new vendor dependencies, so the system you accept is one your team can run.
If you're planning an AI pilot this quarter, Ryshe builds it the way this article describes: a named senior architect, written acceptance criteria before implementation, red-team testing against prompt injection and data leakage, and an audit trail your compliance team can follow. We deliver through continuous monthly engagements and scoped projects for engineering-heavy organizations, on the Microsoft stack you already own. Visit Ryshe to scope a pilot against your own repositories rather than a demo environment.