# What is verification in a software factory?

Verification in a software factory is the stage where a separate agent runs the built software and reports pass or needs work, with evidence.

Last updated November 19, 2026, 8 min read

## Learning objectives

After reading this article you will be able to:

-   Define verification in a software factory
-   Explain how a check stage works
-   Measure how much of a phase is checking

## Related content

-   [What is a software factory?](https://specstory.com/learning/software-factories/software-factory)
-   [What is a software factory process?](https://specstory.com/learning/software-factories/software-factory-process)
-   [Why do software factories fail?](https://specstory.com/learning/software-factories/software-factory-failures)
-   [How to build a software factory](https://specstory.com/learning/software-factories/building-a-software-factory)

## What is verification in a software factory?

Verification in a software factory is the step where an agent that did not build a change runs the software and reports what happened. This section calls that step a check stage. The stage answers pass or needs work, and it attaches evidence to either answer.

A [software factory](https://specstory.com/learning/software-factories/software-factory) can build changes faster than a person can read them. When nobody reads most of the code, the checks carry the confidence that review used to carry. [One operator's published log](https://www.meditationsontech.com/p/what-its-like-to-operate-a-software) calls the factory's side of this manufactured trust, and says it is manufactured the way a part is, on a factory line.

On this page, the term has one meaning. Manufactured trust is confidence that comes from the checks a factory ran on a change, and not from a person reading the code.

A check stage is a form of [runtime verification](https://specstory.com/learning/verification/runtime-verification), because it judges the running software. It is also a form of [independent verification](https://specstory.com/learning/verification/independent-verification), because the checker did not write what it checks.

## How does the check stage work?

A check stage is one stop in the [factory process](https://specstory.com/learning/software-factories/software-factory-process). The steps and the rules below follow [one operator's post](https://www.meditationsontech.com/p/building-a-software-factory-aint) about a factory that the operator built earlier. The stage starts after an integrator agent has merged the builders' work and run every test. It handles each change in the same order:

1.  A checker agent starts the real software, e.g. the web app in a browser.
2.  The checker uses the change the way a user would, and tries one thing the builders did not report trying.
3.  The checker fills in a form with one of two answers and attaches evidence. The answers are pass and needs work.
4.  A fixer agent fixes each finding in one round.
5.  A recheck agent repeats the failed steps and confirms that nothing else broke.

That stage follows four rules:

-   **The checker runs the software.** A checker that only reads the diff repeats the builders' reasoning. Looking at [reading code](https://specstory.com/learning/verification/reading-vs-running-code) and at running it finds different defects.
-   **The form has no third answer.** The checker cannot answer "looks good." It answers pass or needs work, and it shows evidence either way.
-   **A crash counts as needs work.** A checker that stops without an answer has not passed the change.
-   **The checker is a different agent.** An agent that reviews its own work tends to agree with itself, which is the limit of [self-verification](https://specstory.com/learning/verification/ai-self-verification).

Evidence is a record of what the checker did and what the software did in return, e.g. a screenshot of the cart after each step. A sentence that says the change works is a claim. The article on [evidence](https://specstory.com/learning/verification/evidence-that-code-works) lists what counts. The guide on how to [build a software factory](https://specstory.com/learning/software-factories/building-a-software-factory) shows where the stage goes.

## What is an example of verification in a software factory?

Here is an illustrative example. Acme Co. sells furniture online and builds its store with a software factory. An operator at Acme asks the factory to "Let customers edit their delivery address during checkout." The factory handles the request in six steps:

1.  Two builder agents change the checkout code and report the work "done."
2.  An integrator agent merges their work, and every test passes.
3.  A checker agent starts the store, adds 2 items to the cart, and changes the delivery address during checkout.
4.  The checker opens the cart. The address saved, but the cart emptied.
5.  The checker answers needs work and attaches its steps and a screenshot of the empty cart.
6.  A fixer agent changes the code, and a recheck agent repeats the same steps. The cart keeps both items.

The builders' report was a [completion claim](https://specstory.com/learning/verification/coding-agent-done-claims), and the tests agreed with it. Only the stage that used the store found the failure. This example is simplified. A real check stage would also try the checkout on a phone screen and with an expired card.

## How much of a factory's time goes to checking?

[The same operator's log](https://www.meditationsontech.com/p/what-its-like-to-operate-a-software) reports that verifying and fixing the work takes, on average, as long as building it. The log also times one phase of work from start to finish. In one phase that one operator's log times, building took 222.5 minutes and checking took 218.8 minutes, out of 526.3 minutes from spec to commit.

Diagram: Where the time goes in one phase of a software factory

In this phase, checking and fixing took about as long as building. Both checkers found problems.

A separate check also finds problems after the builders report the work done. In 35 of 51 recent phases of one operator's factory, the build step reported the work done and the verify step still found a problem. That count comes from the same operator's post about the earlier factory.

One operator's logs are not a benchmark. They do show that checking and fixing can take about as long as building, and that a separate check keeps finding problems.

## What can a check stage miss?

A check stage has four limits:

-   **A stage checks only what it tries.** A checker that follows one path through the software misses the others. No findings is not the same as complete coverage. A factory with no check stage at all is the first case in the list of reasons that [software factories fail](https://specstory.com/learning/software-factories/software-factory-failures).
-   **The checker can share the builders' assumptions.** A checker needs a [test oracle](https://specstory.com/learning/test-quality/test-oracle), that is a source for the expected result. When it reads the same goal as the builders, it can accept the same wrong reading of that goal.
-   **Checking costs time and tokens.** In the log above, checking took about as long as building.
-   **A pass is evidence.** A pass does not prove that the software is correct. It shows that the steps the checker tried gave the expected results.

## How is factory verification different from code review?

Code review reads a diff. A reviewer judges the design and the wording of a change, and often never starts the software. A check stage starts the software and judges what it does. Review also spreads knowledge of the code among the people on a team, and a check stage does not.

The two also differ in who does the work. Review needs a person for each change, which becomes the [review bottleneck](https://specstory.com/learning/code-review/review-bottleneck) when agents open many changes a day. An agent runs the check stage, so it can run on every phase. A team can keep both, with the stage on every change and people on the decisions that need judgment.

A check stage is close to [end-to-end testing](https://specstory.com/learning/testing/end-to-end-testing), with one difference. An end-to-end test repeats a script that someone wrote earlier. A checker agent can choose its steps from the goal of the change.

## How does RunStory help with verification in a software factory?

A check stage that the factory's own agents run reads the same goal and rules that they do. RunStory independently runs the software against your change and returns evidence to the coding agent. After your agent makes the change, RunStory repeats the failing workflow to check that the problem is resolved. The [workflow guides](https://docs.specstory.com/runstory/workflows) show where that check fits in each way of working.

The operator cited on this page runs the factory that builds RunStory. Your team keeps the final release decision.

[Join the RunStory alpha →](https://specstory.com/runstory#alpha)

## FAQs

### What is manufactured trust?

Manufactured trust is confidence in a change that comes from the checks a software factory ran on it. No person read the code. The operator who named it says that the trust is manufactured the way a part is, on a factory line.

### Why should the checker be a different agent from the builder?

A checker that is a different agent starts without the builder's context, so it is less likely to repeat the builder's assumptions. An agent that reviews its own work tends to agree with itself. A separate checker has to reach its answer from what the running software does.

### What counts as evidence at a check stage?

Evidence at a check stage is a record of what the checker did and what the software did in return. A screenshot of an empty cart after an address change is evidence. A sentence that says the change works is a claim.

### What happens when a checker agent crashes?

A checker agent that crashes has not passed the change. A factory should count the crash as needs work and start a fresh checker. Reading silence as a pass lets unchecked changes through to the commit.

### Does a check stage replace the test suite?

A check stage does not replace the test suite. The integrator agent still runs every test before the checker starts. The tests repeat known cases at low cost, and the stage tries the change the way a user would.

---

Source: [Verification in a software factory | Checks | SpecStory](https://specstory.com/learning/software-factories/software-factory-verification)
