evidence EV-BASELINE-2026-A4F8

Crossover software experiments trade participant control for learning and carryover risk

Evidence Record

Crossover designs are common in software-engineering experiments because each participant can serve as a control, reducing subject variability and sample requirements. They also introduce period, learning, and treatment carryover effects. Exposure to a structured handoff can teach a participant which questions to ask in a later control task, so counterbalancing alone cannot guarantee an uncontaminated comparison.

Limitations

The paper provides design guidance rather than data about this intervention. Whether carryover is material must be tested in a pilot.