Skip to content
learning
Menu

Test environments and sandboxes

Where software runs while it is tested, from agent sandboxes and test isolation to staging environments, and how prompt injection reaches agents.

Start here

What is a sandbox for AI agents?

A sandbox for AI agents is an isolated place to run an agent's commands, with limits on the files and hosts they can reach, so mistakes do less harm.

10 min read

Concepts

  • What is test isolation (hermetic tests)?

    Test isolation means each test sets up its own state and data, so its result does not depend on other tests, the order they run in, or leftover files.

  • What is a staging environment?

    A staging environment is a copy of production used for final testing before release, running the same build with similar settings but no real users.

  • What is an ephemeral environment?

    An ephemeral environment is a short-lived, full copy of an app made for one change or pull request and then deleted, e.g. a preview environment.

  • What is approval fatigue in coding agents?

    Approval fatigue is the habit of approving an agent's permission prompts unread, because so many appear that the prompts stop protecting anything.

  • What is least privilege for AI agents?

    Least privilege for AI agents means giving an agent only the access its current task needs, e.g. a scoped token, and nothing that outlives the task.

Comparisons

Failures and bugs

Terms in this topic

Approval fatigue
Approval fatigue is a failure in which people approve a coding agent's permission prompts without reading them, because the prompts are so frequent that they stop protecting anything.
Container
A container is a packaged process that runs with its own filesystem and limits but shares the host's kernel, which makes it lighter and less isolated than a virtual machine.
Dev container
A dev container is a container defined in the repository that provides a ready-made development environment with its tools installed.
Ephemeral environment
An ephemeral environment is a short-lived copy of an application that is created for one change or test run and destroyed afterwards. A preview environment is one kind.
Hermetic test
A hermetic test is a test that depends only on what it declares and sets up itself, with no shared state, network, or leftover files. It gives the same result anywhere.
Indirect prompt injection
Indirect prompt injection is a prompt injection that arrives through content an agent reads during a task, e.g. a web page, instead of from the user.
Least privilege
Least privilege is a security principle that gives a person, program, or agent only the access its task needs, for no longer than it needs it.
Lethal trifecta
The lethal trifecta is a risk pattern in which an agent has access to private data, reads untrusted content, and can send data out, which lets prompt injection leak data.
Prompt injection
Prompt injection is an attack that hides instructions in content an AI model reads, e.g. a web page, so the model follows them instead of its user.
Sandbox
A sandbox is an isolated environment that limits which files and network hosts the code inside it can reach, so a bad command can damage only what the sandbox allows.
Staging environment
A staging environment is a production-like copy of an application that teams use for final testing before release, running the same build with similar settings but no real users.
Test isolation
Test isolation is a property of a test that runs with its own state and data, so its result does not depend on other tests, their order, or leftover files.
Test-order dependence
Test-order dependence is a flaw that makes a test pass or fail depending on which tests ran before it, usually because they share state.

13 terms from the Learning Center glossary.