evidence EV-BASELINE-2026-A4F8
Crossover software experiments trade participant control for learning and carryover risk
Evidence Record
Crossover designs are common in software-engineering experiments because each participant can serve as a control, reducing subject variability and sample requirements. They also introduce period, learning, and treatment carryover effects. Exposure to a structured handoff can teach a participant which questions to ask in a later control task, so counterbalancing alone cannot guarantee an uncontaminated comparison.
Limitations
The paper provides design guidance rather than data about this intervention. Whether carryover is material must be tested in a pilot.