Free on public repositories

The skip report

Every test Bellwether leaves out is listed with the reason it was safe to leave out. Ask about any test on any commit and get one of eight plain answers.

Start free Quickstart

Verdicts: Eight, four of them mean ‘run it’
Where it lives: Pull request check, CLI and dashboard
Export: JSON, per run or per month
History: 90 days free, 400 days on Team

Two monitors with code on a desk at night, a hand on the keyboard in the background

$ bellwether why api/webhooks/retry_spec.rb
skipped  api/webhooks/retry_spec.rb
  map recorded  2 hours ago, run 8841
  loads         12 files, none in this diff
  verdict       unreachable

A tool that skips tests has to be able to say why, or nobody should leave it switched on. Bellwether writes a skip report for every run and attaches it to the pull request as a check, so the answer is one click from the place people already look.

Eight verdicts, no ninth

Every test on every commit gets exactly one verdict. There is no partial credit: a test is never forty percent selected.

Verdict	Result	Meaning
reaches	runs	The map puts it on a changed file
stale	runs	Its map entry is older than the test file
unmapped	runs	It has never been recorded
pinned	runs	An always-run rule names it
unreachable	skipped	No path from the diff to anything it loads
quarantined	skipped	It disagreed with itself and is on the flake list
excluded	skipped	Your configuration excludes it
other shard	skipped	It runs on a different machine in this run

Four of the eight are ways of saying run it anyway, which is the direction the product is built to fail in.

Asking about one test

bellwether why <test> prints the verdict for any test on the current commit, with the map entry behind it. The same answer is in the dashboard next to every skipped test, and in the JSON export if you want to keep your own records.

The other three

Hands typing on a dark laptop keyboard in a dim room

Free for open source

Test selection

On every pull request, Bellwether walks the diff against a recorded map and runs only the tests that can reach a changed file. Median suites drop to 13 percent of their size.

How it works

Close-up of dark keyboard keys D, F and G with dust on the caps

Free for open source

Flaky test quarantine

A test that fails and then passes on the same commit is rerun in isolation. If it disagrees with itself, it stops blocking merges and moves to a quarantine list with both runs attached.

How it works

Rows of server blades with green status lights, shot from below

Team, from $24 a seat

Shard balancing

The selected tests are split across your parallel machines by each test’s recorded duration, so every shard finishes within seconds of the others. No shard lists to maintain.

How it works