feat: rebuild Experiments tab with Difference-in-Differences analysis and Bayesian verdicts

Replace the basic CRUD experiment tracker with a scientifically rigorous A/B testing system:

- Add DiD analysis engine (services/experimentAnalysis.ts) that computes treatment vs control
  group comparisons across 7 metrics (units, sessions, CVR, CTR, ROAS, revenue, ACOS)
- Implement Bayesian verdict system (Winner/Loser/Inconclusive) using posterior probability
  with normal CDF approximation (Abramowitz & Stegun erf, no external deps)
- Build counterfactual time series for trend charts (actual vs estimated without change)
- Rewrite ExperimentsView as single component with 3 inline sub-views (list, detail, create)
  replacing the previous modal-based ExperimentDetail and ExperimentForm
- Add control group support, change annotations (before→after diffs), and SEO experiment type
- New types: ExperimentChangeAnnotation, DiDMetricResult, DifferenceInDifferencesResult,
  ExperimentVerdict
- Simplify App.tsx by removing experiment modal state (5 useState hooks eliminated)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Christian Vidal Wolf
2026-02-23 15:42:40 +01:00
co-authored by Claude Opus 4.6
parent 7ee33671cc
commit 24df5935cc
6 changed files with 1495 additions and 433 deletions
+2 -1
View File
@@ -98,11 +98,12 @@ export const ExperimentTypeBadge: React.FC<ExperimentTypeBadgeProps> = ({ type,
lg: 'px-3 py-1.5 text-sm',
};
const typeLabels = {
const typeLabels: Record<string, string> = {
pricing: 'Pricing',
advertising: 'Advertising',
content: 'Content',
promotion: 'Promotion',
seo: 'SEO',
};
return (