Measuring AI coding productivity is not the same as counting generated lines or model requests. A change may be produced quickly and still take longer to understand, correct, test, and maintain. Measure outcomes that matter to the team, and collect enough context to explain when an agent helped or made the work harder.
Measure the whole path to an accepted change
Track the time from a well defined task to a reviewed, accepted result. Include time spent prompting, waiting, rerunning checks, correcting code, and reviewing a pull request. Record failed attempts and work that was rejected instead of counting only successful demos.
Useful measures can include task completion, tests passed, review effort, time to merge, defects found later, and developer experience. Keep each measure separate before you combine them. A score that hides how it was calculated is difficult for a team to act on.
| Measure | What it can show | Important limit |
|---|---|---|
| Accepted task rate | Whether work met the stated acceptance criteria | Task difficulty must be comparable |
| Time to reviewed change | How quickly a change became ready for a human decision | Fast output may need more correction |
| Review and rework | How much human effort followed generation | Review quality matters, not just minutes |
| Escaped defects | Whether quality held after the change shipped | Small samples cannot settle causation |
| Usage and spend | What the selected plan or provider meter recorded | Different products may use different units |
Compare similar work and keep the baseline visible
Choose tasks with clear acceptance checks and compare work of similar size, language, and risk. Record the repository, task type, model, coding tool, plan, team experience, and review rules. If a team changes its process at the same time it adopts an agent, do not attribute every difference to the model.
Protect developer privacy and explain why each measure is collected. Individual activity counts can encourage people to game the metric and do not reveal whether a task helped a user. Prefer team level outcomes and use interviews or surveys to understand the experience behind the numbers.
Run a small evaluation before making a team wide claim
Start with one workflow, compare it with a recent baseline, and repeat enough tasks to see variation. Publish the method, missing data, and limits alongside the result. Update the evaluation when models, plans, codebases, or review rules change.
LucentraCode’s benchmark worksheet can help record model, task, checks, usage, and review time. Use it to learn about your own team. A result from one group or repository should not be presented as proof that every developer will gain the same amount of time.