Test environments and sandboxes

Terms in this topic

Approval fatigue

Approval fatigue is a failure in which people approve a coding agent's permission prompts without reading them, because the prompts are so frequent that they stop protecting anything.

Container

A container is a packaged process that runs with its own filesystem and limits but shares the host's kernel, which makes it lighter and less isolated than a virtual machine.

Credential proxy

A credential proxy is a service between an agent and an API that adds the real credential to each request, so the agent never holds the key itself.

Data retention

Data retention is how long a service keeps the data it receives, e.g. prompts and code sent to a model provider, before it deletes that data.

Database branching

Database branching is a way of giving each Git branch, pull request, or agent task its own quick copy of a database, so its writes and migrations cannot touch shared data.

Dev container

A dev container is a container defined in the repository that provides a ready-made development environment with its tools installed.

Egress control

Egress control is a network control that limits which outside hosts a sandbox or agent can reach, usually with an allowlist or a proxy.

Ephemeral environment

An ephemeral environment is a short-lived copy of an application that is created for one change or test run and destroyed afterwards. A preview environment is one kind.

Hermetic test

A hermetic test is a test that depends only on what it declares and sets up itself, with no shared state, network, or leftover files. It gives the same result anywhere.

Indirect prompt injection

Indirect prompt injection is a prompt injection that arrives through content an agent reads during a task, e.g. a web page, instead of from the user.

Least privilege

Least privilege is a security principle that gives a person, program, or agent only the access its task needs, for no longer than it needs it.

Lethal trifecta

The lethal trifecta is a risk pattern in which an agent has access to private data, reads untrusted content, and can send data out, which lets prompt injection leak data.

MicroVM

A microVM is a small virtual machine that starts quickly and gives each workload its own kernel, which isolates it more strongly than a container.

Prompt injection

Prompt injection is an attack that hides instructions in content an AI model reads, e.g. a web page, so the model follows them instead of its user.

Sandbox

A sandbox is an isolated environment that limits which files and network hosts the code inside it can reach, so a bad command can damage only what the sandbox allows.

Sandbox escape

A sandbox escape is a failure in which code inside an isolated environment breaks out of it and reaches the host or other systems the sandbox should keep it away from.

Staging environment

A staging environment is a production-like copy of an application that teams use for final testing before release, running the same build with similar settings but no real users.

Test data management

Test data management is the practice that creates, refreshes, and cleans up the data tests need, so each run starts from a known state without real personal data.

Test isolation

Test isolation is a property of a test that runs with its own state and data, so its result does not depend on other tests, their order, or leftover files.

Test-order dependence

Test-order dependence is a flaw that makes a test pass or fail depending on which tests ran before it, usually because they share state.

20 terms from the Learning Center glossary.