What are acceptance criteria?

What are acceptance criteria?

Acceptance criteria are the conditions that a software change must meet before the person who asked for it accepts it. Each criterion names one result that a person or a test can check. Teams usually write them under a user story or in a specification, and often shorten the term to AC.

The International Software Testing Qualifications Board (ISTQB) defines acceptance criteria as the criteria that a work product must satisfy for its stakeholders to accept it. A user story states who wants a change, what they want, and why. Its acceptance criteria state the conditions that the finished change must meet.

Each criterion can serve as a check's test oracle, the source of the expected result that the actual result is compared with. When a coding agent does the work, the criteria are the part of the request that can be checked against the running software. Without them, the task often ends with the agent's own report that it is "done."

How do acceptance criteria work?

For one change, acceptance criteria usually move through six steps:

1. A person asks for a change in a user story or in a prompt to a coding agent.
2. Before the build starts, the person who asked talks through examples with the team.
3. They write each agreed result as one criterion, and the list sets the scope.
4. A developer or a coding agent builds the change.
5. A person or an automated test checks each criterion on the running software.
6. The person who asked accepts the change when every criterion holds, or sends it back with the ones that failed.

The ISTQB Foundation Level syllabus describes a user story as a card, a conversation, and a confirmation, and the confirmation is its acceptance criteria. The criteria serve as the basis for the story's acceptance testing. A check that runs the finished software against one or more criteria is an acceptance test.

What is an example of acceptance criteria?

Here is an illustrative example. Acme Co. sells furniture online. A product owner at Acme asks the team to "Let customers edit their delivery address during checkout."

The owner's first draft of the criteria reads "The address form works." Before any code exists, a tester asks about the cart, and the team writes the story and these criteria:

As a customer, I want to change my delivery address at checkout,
so that my order ships to the right place.

Acceptance criteria
1. On the review step, the customer can replace the delivery address.
2. The review step and the order confirmation show the changed address.
3. After the address changes, the cart still holds its 2 items ($240.00).
4. An empty address shows "Enter a delivery address" and keeps the old one.

The work then takes five steps:

1. A developer gives the story and its criteria to a coding agent.
2. The agent edits the checkout code, writes unit tests for the address field, and reports that the work is "done."
3. The developer checks each criterion on a staging copy of the store at staging.shop.example.com.
4. Criteria 1, 2, and 4 hold. Criterion 3 fails, because the cart now has 0 items.
5. The developer sends the failed criterion to the agent, which edits the code. All 4 criteria then hold, and the owner accepts the change.

The broken change met the first draft, because the address form did work. Criterion 3 came from the tester's question, and the agent's unit tests did not check it although it was in the prompt. This example is simplified. A real story would need more criteria, e.g. one for an address the store cannot deliver to.

How are acceptance criteria written?

The ISTQB syllabus names two common formats. Rule-oriented criteria state rules in a list or a table, e.g. the Acme list above. Scenario-oriented criteria describe each case as a starting state, an action, and an expected outcome.

Scenarios usually take the Given-When-Then form of behavior-driven development (BDD). Cucumber's Gherkin reference calls each scenario "a concrete example that illustrates a business rule." As a scenario, criterion 3 reads:

Given a customer has 2 items in the cart
When the customer changes the delivery address
Then the cart still holds 2 items

In either format, a list of criteria that tests can check has these properties:

• One observable result. Each criterion names one thing a user can see or a test can check, e.g. the address on the review step.
• Concrete values. The criteria use real inputs and outputs, e.g. 2 items, so two people check them the same way.
• What must stay the same. At least one criterion names behavior that the change must not break, e.g. the contents of the cart.
• The failure case. At least one criterion says what happens after bad input, e.g. an empty address.
• No design. Each criterion describes the result, not the code that produces it.

Each rewrite below names a result that a test can compare:

A story that needs a long list of criteria is often too large and can be split.

How do acceptance criteria guide a coding agent?

Criteria in the prompt give a coding agent conditions to check before its final message says the task is "done," e.g. by running a test for each criterion. Where the criteria say nothing, the agent fills the gap with a default that nobody chose.

Criteria can also sit in a written spec, as in spec-driven development. In plan mode, the agent proposes a plan without editing files, so a person can check that the plan covers each criterion.

An agent can meet a vague criterion without doing what the person meant. "Errors are handled" holds for any code that catches an error. The agent's final report can list each criterion with its check and the result. A criterion with no check in that list is still open.

The weak point is who writes and checks the criteria. If the agent generates the criteria and the tests from the same prompt, a gap in the prompt appears in both. A safer setup has a person approve the criteria before the agent starts, and keeps the acceptance tests in files the agent does not edit. Checking the running software against criteria that a person approved is one part of how teams test AI-generated code.

What are the limits of acceptance criteria?

Acceptance criteria have these limits:

• They cover only what someone wrote. A need that nobody stated has no criterion and no check. Meeting every criterion is not the same as complete coverage.
• They can be wrong. Software can meet every criterion and still fail its users when the criteria record the wrong need.
• They check one change at a time. The criteria of each story can hold while a customer task that spans several stories fails. A user journey test follows one whole task.
• They go stale. Criteria kept in a spec stop describing the software when later changes skip the spec, which is spec drift.
• Some qualities resist a yes or no answer. A person has to judge them, e.g. whether the checkout page confuses customers.

How are acceptance criteria different from a definition of done?

Acceptance criteria belong to one change, while a definition of done applies to every change on a product. The two differ on these points:

For the Acme story, "The cart keeps its items when the address changes" is an acceptance criterion. "The test suite passes after the agent's last edit" belongs in the definition of done, because any change needs it. A change is finished only when it meets both, and a definition of done can require a passing check for each criterion.

FAQs

Who writes acceptance criteria?

Acceptance criteria are usually written by the person who asked for the change, often a product owner, with the people who will build and test it. They write the criteria before the build starts, because the agreed list sets the scope of the change.

How many acceptance criteria should a user story have?

A user story has no fixed number of acceptance criteria. It needs enough of them to cover its main result, its failure cases, and the behavior it must not break. When the list keeps growing, the story usually holds more than one change and can be split.

Can a coding agent write its own acceptance criteria?

A coding agent can draft its own acceptance criteria, but a person should review and approve them. When the agent generates the criteria and the tests from the same prompt, a gap in the prompt appears in both, and the tests cannot find it.

Are acceptance criteria the same as acceptance tests?

Acceptance criteria are not the same as acceptance tests. The criteria state the conditions that a change must meet. An acceptance test is a manual or automated check that runs the software to find out whether one or more of those conditions hold.