Most of a test suite cannot fail on most commits. A change to the proration code in billing does not reach the search indexer, and yet a run-everything pipeline makes you wait for the indexer tests before you can merge. Bellwether removes that wait without guessing.
How the map is built
The first time you run it, Bellwether attaches a recorder to your test process and notes every file each test opens, imports or requires. That is not a model of your architecture. It is the list of files that test actually touched, kept per test, not merged into one coverage number.
Every full run that happens for any other reason, whether the nightly job or a release build, refreshes the map. There is no pipeline step whose only job is to keep Bellwether informed.
How a selection is made
On a pull request, bellwether select takes the diff against the base branch and
looks up every test whose map entry includes a changed file. Those tests are written
to .bellwether/selected, and your existing test command reads that file instead of
a glob. Everything else is written to the skip report with its reason.
Two rules decide the edge cases, and both fail towards running more:
- A test whose map entry is older than its own file always runs.
- A test that has never been mapped always runs.
When it selects everything
Change the base image, the lockfile or the CI definition and every test is downstream of the change. Bellwether selects the whole suite and says so. A tool that reported a saving on that commit would be describing the commit, not measuring it.
