What it does
Reads the test's variant results, visitors, conversions, and conversion rate per variant, and states whether the difference looks statistically meaningful given the sample size, then recommends shipping the winner, killing the change, or running longer to get a clearer signal. Explains the recommendation in plain terms rather than just reporting a p-value.
Install
This skill isn't packaged in an installable registry. It's a plain SKILL.md file you can copy directly. Copy it from the SKILL.md tab and save it at the path for your assistant:
Use it in your AI assistant
Claude Code
.claude/skills/ab-test-result-summarizer/ (or ~/.claude/skills/ab-test-result-summarizer/ for all projects)
GitHub Copilot (VS Code)
.github/skills/ab-test-result-summarizer/
OpenAI Codex
.codex/skills/ab-test-result-summarizer/
Cursor
.cursor/skills/ab-test-result-summarizer/ (or ~/.cursor/skills/ab-test-result-summarizer/ globally)
Compatibility
Requires a connected A/B testing or experimentation tool that reports variant-level conversion data.
The complete SKILL.md for this skill:
---
name: ab-test-result-summarizer
description: Use when a user wants an A/B test's results summarized with a clear recommendation, or asks "should we ship this test" or for a "test readout".
---
# A/B Test Result Summarizer
## When to use
An A/B test has data in and the user needs a plain-language readout of whether the result is real, plus a clear ship, kill, or extend recommendation.
## Procedure
1. Read variant-level results, visitors and conversions per variant, and the test's running duration.
2. Check whether sample size and duration are sufficient to trust the result, has it run at least one full business cycle and reached a reasonable visitor count per variant.
3. State the observed conversion rate difference and whether it's likely a real effect or could plausibly be noise, in plain language.
4. Recommend one of ship the winning variant, kill the test and keep the control, or extend the test, with the reasoning stated.
5. Note any confound that could distort the result, a traffic source imbalance, a holiday in the test window, or a bug reported mid-test.
## Guardrails
- Don't declare a winner from an underpowered or too-short test, recommend extending instead of forcing a call.
- Flag if the test setup itself looks off, a badly imbalanced traffic split or one variant broken, before trusting the numbers at all.