research-document MS-AICODING-2026-A1C4
Map conditional effects of AI coding assistance
Mission
Objective
Determine whether available empirical evidence supports a general productivity claim for AI coding assistance, identify boundary conditions, and define the next discriminating experiments.
Why this matters
Tool adoption is moving faster than valid measurement. A false universal claim can cause either wasteful rejection or unsafe over-adoption.
Scope
Included
Controlled and field evidence about developer speed, delivery outcomes, and merge readiness; evidence published through 2026-07-29.
Excluded
Security-specific effects, learning outcomes, labor-market effects, model comparisons, and a complete systematic review.
Existing context
Parent mandate:
prompts/Software-Engineering-Research-Mission-REP-v2.0.md.
The repository contained no accepted domain evidence before this mission.
Initial hypotheses
- AI assistance produces a positive average productivity effect.
- Effects vary materially with task, developer, repository familiarity, evaluation method, and workflow controls.
Required evidence
At least one controlled study, one realistic field study, one organizational study, contradictory findings, and explicit threats to validity.
Constraints
This is a bounded rapid evidence map, not an exhaustive systematic review. Vendor studies are treated as interested-party evidence.
Execution instructions
Preserve claim-level provenance, distinguish perception from observation, seek counterevidence, and propose tests that measure both speed and downstream cost.
Deliverables
Evidence records, hypothesis, theory, experiment proposal, journal, full REP, future questions, and generated registry updates.
Success criteria
The REP states what can and cannot be concluded, explains contradictions with testable moderators, and gives an executable next experiment.
Stop conditions
Stop after the evidence establishes that a universal effect is unjustified and the highest-value discriminating test is specified.
Verification
Completed on 2026-07-29. Outputs exist; ./ros registry build and
./ros validate are the required final checks.