What is the difference between blocking and non-blocking checks?
Blocking and non-blocking checks differ in what a failure does to the change. When a blocking check fails, the commit, push, or merge stops until a later run passes. When a non-blocking check fails, it reports the result, e.g. a red mark on the pull request, and the change moves on. The same test can run in either mode.
In a continuous integration and delivery (CI/CD) pipeline, a team marks some checks as required and leaves the rest as reports. The International Software Testing Qualifications Board (ISTQB) defines a quality gate as a milestone where the decision to move to the next phase rests on predefined quality criteria. A quality gate can use either kind of check.
Blocking checks keep changes that fail them out of the main branch, but each change waits for them. Non-blocking checks keep work moving, but nothing makes anyone read their failures.
What is a blocking check?
A blocking check is a check whose failure stops a change until the check passes or someone with permission overrides it. It can stop a change at three points:
- Before a commit or push. A Git hook that exits with a nonzero code stops the commit or the push. A pre-push hook runs on the developer's machine, so
git push --no-verifyskips it. - Before a merge. A code host can require a check to pass before a pull request merges. Under a GitHub branch protection rule, admins can bypass a required status check unless the team turns that off.
- In a merge queue. A merge queue runs the required checks again on each pull request combined with the main branch and the changes ahead of it.
A blocking check also needs a rule for when it cannot give a result. A check that fails closed stops the change, and a fail-open check lets it through. By default, GitHub Actions skips a required job when a job it needs fails, and the skipped check counts as passing, so it fails open.
What is a non-blocking check?
A non-blocking check is a check that runs and reports its result without stopping the change. Teams also call it an advisory check, and a failure it reports is often called a soft fail. A person decides what to do with the result.
A check is non-blocking when the code host does not require it, so a red result shows but the merge still works. A CI configuration can also ignore a failure. The GitHub Actions workflow syntax says that continue-on-error: true on a step allows the job to pass when that step fails:
on: pull_request
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- run: npm ci
- run: npm test
- run: npx playwright install --with-deps
- name: End-to-end tests (report only)
run: npx playwright test
continue-on-error: true
When that step fails, GitHub records its outcome as failure and its conclusion as success. The job passes, so the pull request shows a green check. In GitLab CI, allow_failure: true marks a failed job with a warning while the pipeline passes.
A check usually belongs in reporting mode when one of these is true:
- It is flaky. A flaky test fails at random, so a blocking version stops correct changes.
- It is untried. The team does not yet know how often it raises false alarms.
- Its result is a judgment. AI code review can give different comments on two runs of the same code, and some comments are wrong.
When should a check block, and when should it only report?
Here is an illustrative example. Acme Co. sells furniture online. Its build, linter, and unit tests are required checks on each pull request. The end-to-end test of checkout runs with continue-on-error: true, because a flaky wait made it fail at random for a month.
A developer asks a coding agent to "Let customers edit their delivery address during checkout." The checks then run:
- The agent opens a pull request with the change and 3 unit tests.
- The build, linter, and unit tests pass, so the required checks turn green.
- The end-to-end test fails. The address saved, but the cart emptied.
- The job still reports success, and nobody opens the log.
- A reviewer approves the pull request, and it merges.
- A customer reports an empty cart the next morning.
The reporting check found the bug, and nobody acted on it. A check should block when it is fast, gives the same result on the same code, and fails only on problems the team would not ship. Once Acme fixes the flaky wait, the checkout test meets that bar, and Acme requires it. Each remaining reporting check gets an owner who reads it.
This example is simplified. A real team would also send reporting results to the pull request's author.
What changes when a coding agent writes the code?
A coding agent acts on the results of the commands it runs. A check inside that loop, e.g. a pre-commit hook, stops the commit, and the agent reads the error. A required check on the pull request stops only the merge, and it often reports after the agent's session ends. A failed report there waits for a person to read it.
An agent told to make CI pass can also change which checks block. For a pull_request event, GitHub Actions runs the workflow file from the pull request's own merge commit. A change that adds continue-on-error: true to a failing step turns that job's required check green in the same pull request.
A team can require a code owner's review for changes to workflow, test, and linter configuration, so a person sees changes to how the checks run. It can also run slower checks inside the agent's loop, e.g. with an agent hook that runs the tests before the agent ends its turn. The team's definition of done can then name the checks that must pass before the agent says "done."
Can blocking and non-blocking checks be used together?
Blocking and non-blocking checks are usually used together. Fast, reliable checks block, while slower and untried checks report. A check often starts as a report, and the team makes it required once its false alarms are fixed. A flaky test can move the other way while someone fixes it, which is called test quarantine.
The mix has limits:
- Unread reports decay. A check that stays red teaches people to ignore red, so a real failure hides among the old ones.
- Each blocking check adds wait. A slow or flaky required check delays each merge, even for correct changes.
- Green covers only the chosen checks. A run with gaps does not establish that the whole app works.
Checks in CI also do not replace a person who tries the main workflows of an app before launch.
How do blocking and non-blocking checks compare?
Blocking and non-blocking checks differ on these attributes:
| Attribute | Blocking check | Non-blocking check |
|---|---|---|
| On failure | Stops the commit, push, or merge | Reports the failure, and the change continues |
| Main cost | Wait time on each change | Someone's attention to read results |
| Main risk | A flaky check stops correct changes | Real failures go unread |
| Good fit | Fast, reliable checks for problems the team will not ship | Flaky, untried, or judgment checks |
| GitHub setting | A required status check | Not required, or continue-on-error: true |
Who makes the release decision?
A required check can stop a merge, but it cannot say whether a change is ready for customers. People choose which checks are required, read the ones that report, and decide when to release. Those choices are human-in-the-loop decisions.
A required check covers only what someone wrote into it. RunStory runs your software in a separate environment, tries relevant workflows, and checks the results. It is in private alpha for CLIs and web apps, and your team keeps the final release decision.
FAQs
What is a soft fail in CI?
A soft fail in CI is a failed check that the pipeline reports without failing the build or stopping the merge. The failure stays visible as a warning or in the log. In GitHub Actions, a step set to continue on error behaves this way.
What happens to non-blocking results nobody reads?
Non-blocking results that nobody reads stop protecting the change. Old failures pile up, people learn to merge past a red mark, and a real failure hides among the old ones. A non-blocking check stays useful when a named owner reads its results.
Should AI checks be allowed to block merges?
AI checks usually work better as reports than as blocking checks. Their comments can change between runs on the same code, so a blocking AI check would stop correct changes at random. A person can choose which comments to act on, while fast, reliable tests do the blocking.
Can a check move from reporting to blocking?
A check can move from reporting to blocking once it stops raising false alarms. Adding a check as a report first shows how often it fails on correct code before it can stop anyone. The move can also run the other way, when a flaky test goes into quarantine until it is fixed.