Skip to content
learning
Menu

Testing command-line tools and web apps

How command-line tools and web apps behave at runtime and how to test them, from exit codes and output streams to headless browsers.

Start here

How to test a command-line application

Testing a command-line application means running it the way users and scripts do and checking its output, errors, exit codes, and behavior in pipes.

11 min read

Concepts

  • What are exit codes?

    An exit code is the number a program returns when it finishes, where 0 usually means success and any other value signals an error that scripts can check.

  • What is a headless browser?

    A headless browser is a web browser that runs without a visible window, controlled by code, so tests and agents can load pages and act on them.

  • What is a browser test framework?

    A browser test framework is an open-source library that drives a real browser from test code, so a test can load pages, act on them, and check results.

  • What is a pseudoterminal (PTY)?

    A pseudoterminal (PTY) is a pair of virtual devices that acts as a terminal for a program, so another program can send it keystrokes and read its output.

  • What is an agent-native CLI?

    An agent-native CLI is a command-line tool designed for coding agents as well as people, with structured output, clear exit codes, and no hidden prompts.

  • What is browser automation?

    Browser automation is controlling a web browser with code or an agent to open pages, click, type, and read results, for testing or repetitive tasks.

  • What is Playwright MCP?

    Playwright MCP is a server that lets an AI agent drive a real browser through the Model Context Protocol, using page snapshots instead of screenshots.

  • What is the NO_COLOR convention?

    NO_COLOR is a convention in which a command-line tool turns off colored output when the NO_COLOR environment variable is not empty, keeping logs plain.

How-to guides

  • How to verify a CLI that a coding agent built

    Verifying a CLI that a coding agent built means installing it fresh, running the commands a user would, and checking exit codes, output, and files.

  • How to catch browser console errors in tests

    Catching console errors in tests means listening to the browser's console and page error events during a run and failing when unexpected errors appear.

  • How to handle Ctrl-C (SIGINT) in a CLI

    Handling Ctrl-C (SIGINT) in a CLI means catching the signal, stopping work, cleaning up files and child processes, and ending with exit code 130.

  • How to run a two-account access test

    A two-account access test signs in as two users and tries to read or change one user's data as the other, to check authorization in the running app.

  • How to test a CLI's --help and --version

    Testing a CLI's --help and --version flags means checking that each prints the expected text to stdout, exits with code 0, and needs no config or network.

  • How to test a terminal UI (TUI)

    Testing a terminal UI (TUI) means running it in a pseudoterminal, sending keystrokes, and comparing the screen it draws with a reviewed snapshot.

  • How to test interactive CLI prompts

    Testing interactive CLI prompts means feeding answers through stdin or a pseudoterminal, waiting for each prompt, and checking what the program does next.

  • How to test shell scripts

    Testing a shell script means running it with controlled input and environment, then checking its output, exit code, and the files it changes.

Comparisons

Common questions

Failures and bugs

  • What is a broken pipe error (SIGPIPE)?

    A broken pipe error happens when a program writes to a pipe or socket whose reader has closed, which sends it SIGPIPE and often ends it with exit code 141.

  • What is a hydration error?

    A hydration error happens when the HTML a server rendered does not match what the browser's JavaScript renders, so the framework warns or re-renders.

  • What is fail-open authentication?

    Fail-open authentication is an access check that lets a request through when the check itself errors or times out, instead of denying the request.

Terms in this topic

Agent-native CLI
An agent-native CLI is a command-line tool designed for coding agents as well as people, with structured output, clear exit codes, and no prompts without a terminal.
Browser automation
Browser automation is software control of a web browser, in which code or an agent opens pages, clicks, types, and reads what the page shows.
Browser test framework
A browser test framework is an open-source library that drives a real browser from test code, so a test can load pages, act on them, and check the results.
Chrome DevTools Protocol
The Chrome DevTools Protocol is an interface that lets a program drive and inspect a Chromium-based browser, and many automation tools and browser agents are built on it.
Command-line interface (CLI)
A command-line interface is a way of using a program that takes typed commands, flags, and input in a terminal or script, and returns output, errors, and an exit code.
Exit code
An exit code is the number that a program returns when it finishes, where 0 usually means success and any other value signals an error that scripts can check.
Fail-open
Fail-open is a behavior that lets an action through when a check errors or times out, while fail-closed code denies it. Fail-open authentication is a common security bug.
Headless browser
A headless browser is a web browser that runs without a visible window and is controlled by code, so tests and agents can load pages and act on them.
Hydration error
A hydration error is a web app failure that happens when the HTML a server rendered does not match what the browser's JavaScript renders on load.
NO_COLOR
NO_COLOR is a convention in which a command-line tool turns off colored output when the NO_COLOR environment variable is present and not empty.
Playwright MCP
Playwright MCP is an open-source server that lets an AI agent control a real browser through the Model Context Protocol, using structured page snapshots instead of screenshots.
Pseudoterminal (PTY)
A pseudoterminal (PTY) is a pair of virtual devices that acts as a terminal for a program, so another program, e.g. a test, can send it keystrokes and read its output.
SIGPIPE
SIGPIPE is the signal that a program receives when it writes to a pipe whose reader has closed. By default it ends the program, and shells report exit code 141.
Standard streams
The standard streams are the three channels that every process gets by default, which are standard input for data in, standard output for results, and standard error for diagnostics.
stderr
stderr is the standard error stream that a program writes errors and diagnostics to, so they stay separate from the results on standard output.
stdout
stdout is the standard output stream that a program writes its normal results to, so another program, a file, or a terminal can read them.
TTY
A TTY is a terminal device that a program can detect, and many programs change their output, color, or prompts when they write to a pipe instead of a TTY.
Two-account test
A two-account test is a behavior check that signs in as two different users and confirms that neither can see or change the other's data.

18 terms from the Learning Center glossary.