Replace the basic CRUD experiment tracker with a scientifically rigorous A/B testing system:
- Add DiD analysis engine (services/experimentAnalysis.ts) that computes treatment vs control
group comparisons across 7 metrics (units, sessions, CVR, CTR, ROAS, revenue, ACOS)
- Implement Bayesian verdict system (Winner/Loser/Inconclusive) using posterior probability
with normal CDF approximation (Abramowitz & Stegun erf, no external deps)
- Build counterfactual time series for trend charts (actual vs estimated without change)
- Rewrite ExperimentsView as single component with 3 inline sub-views (list, detail, create)
replacing the previous modal-based ExperimentDetail and ExperimentForm
- Add control group support, change annotations (before→after diffs), and SEO experiment type
- New types: ExperimentChangeAnnotation, DiDMetricResult, DifferenceInDifferencesResult,
ExperimentVerdict
- Simplify App.tsx by removing experiment modal state (5 useState hooks eliminated)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Christian Vidal Wolf
