Commit Graph
9 Commits
Author SHA1 Message Date
Christian Vidal Wolf 30466a94ff fix(experiments): mathematically correctly aggregate ratios like ACOS and ROAS before averaging 2026-02-25 13:44:59 +01:00
Christian Vidal Wolf b39831c417 fix(experiments): change weekly boundaries to start on Sunday instead of ISO Monday 2026-02-25 13:35:10 +01:00
Christian Vidal Wolf 0fb312d537 fix(experiments): resolve bug that included adjacent week when experiment ended on midnight boundary 2026-02-25 13:32:39 +01:00
Christian Vidal Wolf 719390a5d5 refactor(experiments): change metric average calculation to strict weekly dividing rather than day fractions 2026-02-25 13:15:44 +01:00
Christian Vidal Wolf 6e94a32a47 fix(experiments): include full 24h of end_date in DiD overlap bounds 2026-02-25 13:10:00 +01:00
Christian Vidal Wolf 8ed073ff6e fix(experiments): use exact fractional proportions for overlapping weeks in DiD calculation 2026-02-25 12:59:03 +01:00
Christian Vidal Wolf 4de9fb5b89 fix(experiments): include overlapping weeks in date range logic and use exact ISO weeks 2026-02-25 08:39:13 +01:00
Christian Vidal Wolf 4935112a55 feat: enable custom baseline periods for Experiments to solve seasonality 2026-02-23 18:34:04 +01:00
Christian Vidal WolfandClaude Opus 4.6 24df5935cc feat: rebuild Experiments tab with Difference-in-Differences analysis and Bayesian verdicts
Replace the basic CRUD experiment tracker with a scientifically rigorous A/B testing system:

- Add DiD analysis engine (services/experimentAnalysis.ts) that computes treatment vs control
  group comparisons across 7 metrics (units, sessions, CVR, CTR, ROAS, revenue, ACOS)
- Implement Bayesian verdict system (Winner/Loser/Inconclusive) using posterior probability
  with normal CDF approximation (Abramowitz & Stegun erf, no external deps)
- Build counterfactual time series for trend charts (actual vs estimated without change)
- Rewrite ExperimentsView as single component with 3 inline sub-views (list, detail, create)
  replacing the previous modal-based ExperimentDetail and ExperimentForm
- Add control group support, change annotations (before→after diffs), and SEO experiment type
- New types: ExperimentChangeAnnotation, DiDMetricResult, DifferenceInDifferencesResult,
  ExperimentVerdict
- Simplify App.tsx by removing experiment modal state (5 useState hooks eliminated)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-23 15:42:40 +01:00