A/B Testing Resources
Everything you need to learn and master experimentation.
Articles, books, courses, tools, and a complete glossary for practitioners at every level.
A curated collection of articles, books, courses, tools, and terminology for A/B testing practitioners at every level. Whether you are running your first experiment or scaling a company-wide experimentation programme, these resources cover the statistical foundations, methodology, and organisational frameworks you need. All backed by our free statistical calculators for hands-on practice.
Statistical Calculators
Free tools to plan experiments, analyze results, and validate your data.
Learning Paths by Experience Level
Not sure where to start? These learning paths map the right resources to your current experience level and goals.
Beginner
New to A/B testing. You want to understand the basics before running your first experiment.
- 1Glossary - Learn the terminology
- 2Beginner courses - Structured learning
- 3Sample size calculator - Plan your first test
Intermediate
You have run experiments before and want to improve your statistical rigour and methodology.
- 1Articles - Methodology deep dives
- 2Books - Statistical foundations
- 3Tools comparison - Choose a platform
Advanced
You are building or scaling an experimentation program and need advanced techniques and organisational frameworks.
- 1Team models - Structure your team
- 2Advanced courses - Causal inference, Bayesian
- 3Bayesian calculator - Advanced analysis
What Each Resource Covers
A detailed overview of every resource section and how it fits into your experimentation practice.
Articles & Research
Our articles library includes in-depth guides written by our team on topics like non-inferiority testing, practical significance, and the region-beta paradox. We also curate the best external articles on experiment frameworks, one-tailed vs two-tailed testing, multiple comparisons, statistical power myths, non-binomial metrics, and interaction effects in overlapping experiments. Each external article includes a summary and key takeaways.
Books
The recommended books include titles from Georgi Georgiev on statistical methods, Ron Kohavi on trustworthy experiments at scale, Stefan Thomke on building experimentation culture, and Don Norman on user-centered design. Each book review includes a detailed summary, who it is useful for, and three key takeaways so you can decide which to read first.
Courses & Certifications
Our course directory covers 21 programmes across six categories: A/B testing fundamentals, statistics and Bayesian methods, CRO and conversion optimization, product analytics, data science foundations, and experimentation platform training from Optimizely and LaunchDarkly. Each course lists the provider, level, duration, price, and a detailed summary of what you will learn.
Glossary
The A/B testing glossary defines over 160 terms used in experimentation and statistics. Entries cover concepts like statistical significance, p-values, confidence intervals, minimum detectable effect, sample ratio mismatch, Bonferroni correction, and dozens more. The glossary is searchable and organised alphabetically with quick navigation.
A/B Testing Tools Comparison
The tools comparison evaluates 18+ experimentation platforms including Optimizely, VWO, LaunchDarkly, Statsig, GrowthBook, Amplitude Experiment, and more. Each tool is reviewed with pricing, supported platforms, statistical engine type, pros, cons, and best-fit use case. A quick comparison table lets you filter by category, free tier availability, and platform support.
Statistical Calculators
Nine free statistical calculators cover the full experiment lifecycle. Use the sample size calculator and duration calculator to plan tests, the significance calculator and Bayesian calculator to analyze results, and the SRM checker and normality test to validate your data.
Topics Covered Across All Resources
Whether you are looking for information on a specific statistical concept, trying to compare tools, or building a company-wide experimentation programme, this library covers it.
Statistical Foundations
Frequentist hypothesis testing, Bayesian inference, confidence intervals, credible intervals, p-values, statistical power, Type I and Type II errors, effect size estimation, minimum detectable effects (MDE), and sample size determination. The articles and books explore when to use one-tailed versus two-tailed tests, how to handle multiple comparisons with Bonferroni and Benjamini-Hochberg corrections, and how to apply t-tests to non-binomial metrics like revenue per user and average order value.
For hands-on practice, use our significance calculator for frequentist analysis, the Bayesian calculator for posterior distributions, the sample size calculator for power analysis, and the MDE calculator for reverse power analysis.
Experiment Design & Methodology
Hypothesis formulation, experiment briefs, the explore-exploit tradeoff applied to testing portfolios, non-inferiority testing, equivalence testing (TOST), and practical significance versus statistical significance. The resources explain how to structure experiments that produce actionable insights regardless of outcome, how to set non-inferiority margins, and when flat results are actually informative.
Advanced topics include interaction effects between overlapping experiments, the region-beta paradox (why small negative results damage programmes more than large failures), and how to balance incremental optimisation with transformational bets using portfolio frameworks like the 70-20-10 model.
Building Experimentation Programmes
Organisational models for experimentation teams (centralized, decentralized, Center of Excellence), scaling experimentation from startup to enterprise, getting leadership buy-in, defining an Overall Evaluation Criterion (OEC), and building a culture where most ideas are expected to fail. The books by Ron Kohavi and Stefan Thomke provide case studies from Microsoft, Google, Booking.com, and Amazon.
The courses cover practical programme management, from setting up prioritisation frameworks to proving ROI to stakeholders. Our team models guide compares the tradeoffs between each organisational structure with concrete recommendations for different company sizes and maturity levels.