Loading...
ABTestResult API
Run the same statistical calculations from our web tools via a simple REST API. Useful for automating analysis in your existing workflows.
API
Use Cases
Integrate statistical analysis into your existing workflows.
Automated Reporting
Pull test results into Google Sheets, Notion, or your internal dashboards via scheduled scripts. Schedule daily or weekly reports with statistical summaries.
Slack Alerts
Use Zapier or a simple script to post analysis results to Slack when your tests complete. Get notified when significance is reached.
Internal Tooling
Embed statistical calculations into your internal experimentation platform or analysis scripts. Build custom dashboards with our calculations.
Pre-launch Sample Size
Calculate required sample sizes programmatically when planning experiments. Automatically determine test duration based on traffic estimates.
Data Quality Monitoring
Automatically check for Sample Ratio Mismatch (SRM) in your experiments. Catch data quality issues before they invalidate your results.
CI/CD Integration
Integrate Bayesian analysis into your deployment pipeline. Make automated ship/no-ship decisions based on probability to be best and expected loss.
Sensitivity Analysis
Estimate Minimum Detectable Effect (MDE) for different traffic scenarios. Plan experiments that can detect business-meaningful effects.
Test Duration Planning
Calculate how long your test needs to run given your traffic, baseline conversion rate, and desired MDE. Account for multiple variants and metrics.
API Reference
Simple JSON in, JSON out. The same calculations as our web tools.
POST
/v1/analyzeAnalyze A/B test results with Z-tests (for conversion rates) or T-tests (for continuous metrics). Returns p-values, confidence intervals, and significance status.
// Request - Conversion Rate (Z-Test){ "metricType": "rate", "control": { "users": 10000, "conversions": 350 }, "variants": [ { "name": "Variant B", "users": 10000, "conversions": 420 } ], "confidenceLevel": 0.95, "sided": 2} // Response{ "significant": true, "pValue": 0.0023, "zScore": 3.05, "zCritical": 1.96, "relativeLift": 0.20, "absoluteLift": 0.007, "controlRate": 0.035, "variantRate": 0.042, "confidenceInterval": { "control": [0.0314, 0.0386], "variant": [0.0381, 0.0459], "difference": [0.0025, 0.0115] }, "srm": { "detected": false, "chiSquare": 0.82, "pValue": 0.365 }}POST
/v1/analyzeAnalyze continuous metrics (revenue, session duration) using T-tests. Requires standard deviation for each group.
// Request - Average Metric (T-Test){ "metricType": "average", "control": { "users": 5000, "mean": 45.50, "stdDev": 12.30 }, "variants": [ { "name": "Variant B", "users": 5000, "mean": 48.20, "stdDev": 13.10 } ], "confidenceLevel": 0.95, "sided": 2} // Response{ "significant": true, "pValue": 0.0001, "tScore": 3.89, "tCritical": 1.96, "relativeLift": 0.059, "absoluteLift": 2.70, "pooledStd": 12.71, "degreesOfFreedom": 9998, "confidenceInterval": { "control": [44.16, 46.84], "variant": [46.84, 49.56], "difference": [1.35, 4.05] }}POST
/v1/sample-sizeCalculate the sample size needed for your experiment. Supports conversion rates and continuous metrics with multiple comparison corrections.
// Request - Conversion Rate{ "metricType": "rate", "baseline": 3.5, "mde": 10, "confidenceLevel": 0.95, "power": 0.80, "sided": 2, "numVariants": 1, "successMetrics": 1, "guardrailMetrics": 0} // Response{ "samplesPerVariant": 14748, "totalSamples": 29496, "effectSize": 0.0165, "mdeAbsolute": 0.35, "parameters": { "alpha": 0.025, "beta": 0.20, "zAlpha": 1.96, "zPower": 0.84 }, "corrections": { "sidakApplied": false, "spotifyGuardrailApplied": false }}POST
/v1/bayesian/analyzeBayesian A/B testing with Beta-Binomial model for conversion rates or Normal-Normal model for continuous metrics. Returns probability to be best and expected loss.
// Request{ "metricType": "rate", "priorStrength": "uninformative", "credibility": 0.95, "control": { "visitors": 10000, "conversions": 350 }, "variants": [ { "name": "Variant B", "visitors": 10000, "conversions": 420 } ]} // Response{ "probabilityToBeBest": [0.023, 0.977], "expectedLoss": [0.0012, 0.0001], "posteriorMeans": [0.035, 0.042], "credibleIntervals": [ [0.032, 0.038], [0.039, 0.045] ], "relativeLift": 0.20, "monteCarloSamples": 100000, "verdict": { "type": "winner", "winner": "Variant B", "confidence": "97.7% probability to be best" }}POST
/v1/srm-checkCheck for Sample Ratio Mismatch (SRM) in your traffic allocation. Uses chi-square goodness of fit test with 0.01 significance threshold.
// Request{ "expectedRatio": [0.5, 0.5], "observedCounts": [10234, 9521]} // Response{ "srmDetected": true, "chiSquare": 25.734, "pValue": 0.00000038, "degreesOfFreedom": 1, "observedSplit": [51.8, 48.2], "expectedSplit": [50.0, 50.0], "recommendation": "Stop the experiment and investigate. Check assignment logic, bot traffic, and data pipeline."}POST
/v1/mde-estimateEstimate the Minimum Detectable Effect (MDE) given your sample size and statistical parameters. Includes Sidak correction for multiple comparisons.
// Request{ "metricType": "rate", "traffic": 50000, "baseline": 5.0, "confidenceLevel": 0.95, "power": 0.80, "numVariants": 1, "successMetrics": 1, "guardrailMetrics": 0} // Response{ "mde": 0.0041, "mdePercent": 8.2, "mdeInterpretation": "Can detect lifts of 8.2% or larger", "cohensD": 0.060, "parameters": { "zAlpha": 1.96, "zPower": 0.84, "nPerGroup": 25000 }, "sensitivityCurve": [ { "traffic": 25000, "mdePercent": 11.6 }, { "traffic": 50000, "mdePercent": 8.2 }, { "traffic": 100000, "mdePercent": 5.8 } ]}POST
/v1/duration-estimateCalculate how long your test needs to run given daily traffic and experimental parameters.
// Request{ "baseline": 3.5, "mde": 15, "dailyTraffic": 5000, "confidenceLevel": 0.95, "power": 0.80, "numVariants": 1} // Response{ "daysNeeded": 42, "weeks": 6, "samplesPerVariant": 10416, "totalSamples": 20832, "trafficPerVariant": 2500, "recommendation": "Run for at least 42 days (6 weeks) to capture 7 full business cycles. Avoid stopping early."}Supported Calculations
The same statistical methods available in our web tools.
Frequentist A/B testing (Z-test)
Continuous metrics (T-test)
Bayesian inference
Sample size calculation
MDE estimation
SRM detection
Duration estimation
Confidence intervals
Šidák correction
Guardrail metrics adjustment
Multiple variants support
Non-inferiority testing
Free Web Tools
Use these calculators directly in your browser while you wait for API access.