Open source
$0 free for public repositories
Unlimited committers. Community support in the forum.
- Test selection on every pull request
- Flake quarantine
- The skip report
- 90 days of history
CI test selection and flake control
Run the tests your change can break. Skip the rest.
Bellwether records what every test loads, then runs only the ones a pull request can reach, and prints the reason for every test it leaves out. Flaky tests stop blocking merges. The whole suite still runs every night.
- 1,400 engineering teams
- 71% median CI time cut
- Free for open source
-
Every skip has a reason
Each test left out names the map entry that made it safe to leave out.
-
A full run every night
The whole suite still runs once a day, so a stale map is caught within hours.
-
Your code stays put
The recorder sends file paths and test IDs. Source never leaves your runner.
-
Leave in one commit
Delete two lines from the pipeline and it runs the whole suite again.
What Bellwether does on every pull request
One CLI step before your test command. Everything else happens in the map it keeps.
Free for open source
Test selection
On every pull request, Bellwether walks the diff against a recorded map and runs only the tests that can reach a changed file. Median suites drop to 13 percent of their size.
How it works
Free for open source
Flaky test quarantine
A test that fails and then passes on the same commit is rerun in isolation. If it disagrees with itself, it stops blocking merges and moves to a quarantine list with both runs attached.
How it works
Free for open source
The skip report
Every test Bellwether leaves out is listed with the reason it was safe to leave out. Ask about any test on any commit and get one of eight plain answers.
How it works
Team, from $24 a seat
Shard balancing
The selected tests are split across your parallel machines by each test’s recorded duration, so every shard finishes within seconds of the others. No shard lists to maintain.
How it works
Why we built it
Most of a test suite cannot fail on most commits
We spent years on build infrastructure watching engineers wait 40 minutes for tests that could not see their change. Bellwether has run in production pipelines since 2021, and it still only skips a test when it can print the reason.
- Teams using it
- 1,400
- Median CI time cut
- 71%
- Recording overhead
- 4%
From first recording to a shorter run
Setup is two lines in the pipeline file. The first recording is the only slow part, and nothing is skipped until it finishes.
Record
One full run with the recorder attached. It notes every file each test opens, imports or requires.
Select
On each pull request, the diff is walked against the map. Only tests that can reach a changed file are picked.
Run
Your own test command reads the selection file. Shards are balanced by each test’s recorded duration.
Audit
Every skipped test is listed with its reason, and a full run each night checks the map is still true.
What would selection save your pipeline?
Move the sliders to match your suite. The selection rate comes from real suites of the same size.
Tick a file, watch the suite shrink
A small repository with 18 tests. Pick what the pull request changed and Bellwether picks the tests. Try the base image.
3 of 18 tests selected
run time 2m 43s
Full suite: 13m 54s. Every skipped test keeps its reason in the skip report.
- billing/proration_spec.rb runs
- billing/invoice_totals_spec.rb runs
- billing/coupon_stacking_spec.rb skipped
- billing/tax_region_spec.rb skipped
- api/accounts/merge_spec.rb runs
- api/webhooks/retry_spec.rb skipped
- api/auth/session_spec.rb skipped
- search/indexer_spec.rb skipped
- search/ranking_spec.rb skipped
- search/facet_counts_spec.rb skipped
- jobs/scheduler_spec.rb skipped
- jobs/backoff_spec.rb skipped
- web/checkout_flow_spec.ts skipped
- web/checkout_errors_spec.ts skipped
- web/settings_form_spec.ts skipped
- core/money_spec.ts skipped
- core/result_spec.ts skipped
- core/clock_spec.ts skipped
Bellwether, run-everything or hand-cut shards
The three ways teams usually handle a slow suite, side by side for a 3,000-test suite that takes 38 minutes.
| What you get | Bellwether | Run everything | Manual sharding |
|---|---|---|---|
| Wait per push | 6–9 minutes | The full suite, 38 minutes | Suite ÷ shards, about 12 minutes on 4 |
| CI minutes billed | About 15% of the suite | 100% of the suite | 100% plus setup on every shard |
| Catches what the change can break | Yes, plus a full run nightly | Yes | Yes |
| Flaky tests | Quarantined with both runs attached | Block the merge until retried | Block the merge until retried |
| Upkeep | None, the map rebuilds itself | None | Rebalancing shard lists by hand |
| Cost | From $0 for open source, $24 a seat | Your CI bill | Your CI bill plus engineer hours |
$0 for open source, $24 a seat for everyone else
Every plan gets selection, flake quarantine and the skip report. Team adds private repositories and shard balancing.
Team
Most teams$24 per active committer / month
Billed yearly, or $29 month to month. 5,000 selected runs a month included, then $4 per 1,000.
- Everything in Open source
- Private repositories
- Shard balancing
- 400 days of history
- Email support, one working day
Enterprise
$1,900 per month, from
Annual contract. Self-hosted map store, SSO and audit log.
- Everything in Team
- Self-hosted map store
- SAML SSO and SCIM
- Audit log export
- 99.9% uptime SLA
Engineers on the time they got back
Wait per push before and after, measured on each team’s main pipeline over a month.
CI per push 38 min down to 7 min
“We stopped arguing about which tests to skip on Fridays. The skip report settles it, and the nightly run has caught two stale entries in six months, both before anyone merged.”
CI per push 52 min down to 11 min
“Our four hand-balanced shards drifted every sprint. Now one pipeline step picks the tests and spreads them by recorded duration. Nobody owns a shard list any more.”
CI per push 24 min down to 6 min
“Contributors used to wait half an hour to learn a typo fix passed. On the open-source plan it costs us nothing and the queue on release week is finally short.”
Shipped lately, and written down
Changelog
All releases- v2.14 New
Slow-shard summary in every run
- v2.13 New
First-party recorder for Go test
- v2.12 Improved
Random seeds kept with quarantined runs
- v2.11 Improved
More lockfiles treated as global by default
Questions teams ask in the first week
Setup, security and billing. The full list has more, and a person answers the contact form.
Setup
How long does setup take?
Two lines in the pipeline file and one full recording run. The recording takes as long as your suite plus about four percent. Until it finishes, Bellwether selects every test, so nothing changes for your team on day one.
Which CI systems and test runners does it support?
Anything that can run a shell step: hosted runners, self-hosted runners and build farms alike. Test runners with a first-party recorder are RSpec, Minitest, pytest, Jest, Vitest, Go test, JUnit 5 and Cargo test. Others work through the generic file-access recorder.
Security
Does Bellwether need our source code?
No. The recorder runs inside your pipeline and sends file paths, test identifiers and durations. File contents never leave your runner.
Where is the map stored?
On Open source and Team, in our EU region (Frankfurt), encrypted at rest with per-account keys. On Enterprise you can run the map store in your own cloud account and we never see it.
Billing
Who counts as an active committer?
Anyone who pushed a commit that Bellwether ran against in the billing month. Bots, release automation and people who only read the dashboard are not counted.
Is the open-source plan really free?
Yes, for public repositories under an OSI-approved licence, with no seat limit and no run limit. Private repositories need the Team plan.
Record one suite this afternoon.
Nothing is skipped until the first map is built, so you can install it today and decide next week.