Why does CLI output change when piped?

Why does CLI output change when piped?

Output from a command-line interface (CLI) changes when piped because the program, or its language runtime, checks whether its input and output are connected to a terminal. A TTY is a terminal device that a program can detect. In a pipe or file, many programs turn off color, hold output in blocks, skip prompts, and print one item per line.

TTY is short for teletypewriter. In C, the isatty() function returns 1 when a file descriptor refers to a terminal and 0 when it does not, as its Linux manual page states. A shell script tests [ -t 1 ], where 1 is the file descriptor of stdout, and Python calls sys.stdout.isatty(). Node.js exposes process.stdout.isTTY, and Go programs often call term.IsTerminal from golang.org/x/term.

A person and a script can therefore get two versions of one command. A bug can sit in only one, so testing a command-line application means running it both ways.

What causes output to change in a pipe?

A TTY check covers one file descriptor at a time, so stdin, stdout, and stderr each count separately.

Why does color disappear in a pipe?

Terminal colors are ANSI escape codes, bytes that a terminal turns into formatting, e.g. \033[31m for red text. A pipe passes the bytes on unchanged, so the next program reads them as text. ls --color=auto and Git's default color.ui setting therefore add color only on a terminal. The Command Line Interface Guidelines advise turning color off when a stream is not a TTY or when NO_COLOR is set and not empty. Many tools accept a flag that forces color, e.g. ls --color=always, and Node.js reads the FORCE_COLOR environment variable.

What does buffering change in a pipe?

By default, the C library buffers stdout by line on a terminal and in blocks otherwise, and Python does the same. In a pipe, output waits until the block fills, the program flushes, or the program exits. A slow script piped into grep can then look stuck. In Python, print(line, flush=True) or python -u removes the delay, and GNU stdbuf -oL makes many C programs buffer stdout by line.

Why do prompts and progress bars stop?

A prompt needs a person to answer it, so a CLI should check stdin before it asks. The guidelines advise prompting only when stdin is a TTY, and otherwise failing with an error that names the flag to pass, e.g. --force. The command then exits with a nonzero exit code. Spinners and progress bars redraw one line in place, which only a terminal can show, so many tools draw them only on a TTY.

Why does the layout change?

ls prints names in columns on a terminal, and the POSIX standard requires one name per line by default when output goes anywhere else. A pipe also has no width, so programs fall back to the COLUMNS variable or a default, e.g. 80 columns in Python's shutil.get_terminal_size().

What does piped output look like?

Here is an illustrative example. Acme Co. sells furniture online, and its command-line tool, acme, lists orders. A developer asks a coding agent to "Show failed orders in red." The work takes five steps:

1. The agent wraps each failed row of acme orders list in the codes that start and reset red, with no TTY check.
2. The agent's test captures the output, checks that it contains A-1042, and passes, because the ID still sits between the codes.
3. In a terminal, the developer sees order A-1042 in red.
4. A nightly job runs acme orders list --status failed | cut -d' ' -f1 and passes each ID to acme refund, and every refund fails with error: unknown order.
5. The agent changes the code to add color only when sys.stdout.isatty() returns true and NO_COLOR is unset or empty.

Before step 5, cat -v shows what the job received, with the escape character printed as ^[:

$ acme orders list --status failed | cat -v
^[[31mA-1042  failed  $240.00^[[0m
$ acme orders list --status failed | cut -d' ' -f1 | cat -v
^[[31mA-1042

Each ID starts with an escape code. This example is simplified. A real CLI would need the same check on stderr, where its warnings go.

What changes when a coding agent writes the code?

A coding agent's shell tool usually captures command output in a pipe or a file. The agent's runs then take the pipe path, and the terminal path runs when a person types the command.

That split hides bugs on both paths. In step 2, the agent's test received the escape codes, but its check for the ID passed. After the fix in step 5, the agent's piped runs show no red, so they cannot confirm the requested feature. Code that prompts without checking stdin can also hang until the tool's timeout.

A practical adjustment is to name both modes in the task's acceptance checks, e.g. "no escape codes when piped, red rows in a terminal." To verify a CLI that a coding agent built, add one run under a terminal.

How can teams test both terminal and pipe modes?

Tests can run each command through pipes and again under a pseudoterminal (PTY). A PTY gives the program a terminal device while the test reads the other end. In Python, the standard pty module opens a PTY. Both tests drop NO_COLOR, so the developer's setting cannot change the result:

import os
import pty
import subprocess

CMD = ["acme", "orders", "list", "--status", "failed"]
ENV = {k: v for k, v in os.environ.items() if k != "NO_COLOR"}

def test_piped_output_has_no_escape_codes():
    result = subprocess.run(CMD, capture_output=True, text=True, env=ENV)
    assert "\x1b[" not in result.stdout

The terminal test closes its copy of the terminal end and reads until the command exits, because a full PTY buffer blocks the command:

def test_terminal_output_is_red():
    main_fd, tty_fd = pty.openpty()
    proc = subprocess.Popen(CMD, stdout=tty_fd, env=ENV)
    os.close(tty_fd)
    output = b""
    try:
        while chunk := os.read(main_fd, 1024):
            output += chunk
    except OSError:  # Linux raises EIO when the command exits
        pass
    finally:
        os.close(main_fd)
        proc.wait(timeout=10)
    assert b"\x1b[31m" in output

A fuller suite adds three kinds of check:

• Overrides. A terminal test with NO_COLOR set expects no escape codes, and a pipe test with --color=always expects them.
• Empty input. A pipe test with stdin from /dev/null expects a nonzero exit code and an error that names the flag to pass.
• Stored output. A golden file for each mode catches layout changes, and a snapshot test records only the mode it ran in.

PTY output ends lines with \r\n, so normalize line endings before comparing. Metamorphic testing needs no stored output. For a command whose layout does not depend on the mode, terminal output with its escape codes removed should equal the piped output.

Outside a test, the Linux script command runs a program under a PTY, e.g. script -qec "acme orders list" /dev/null. On macOS, the form is script -q /dev/null acme orders list. Programs that take over the whole screen need the method for testing a terminal UI.

Why does a CLI behave differently in CI?

A job in a continuous integration and delivery (CI/CD) pipeline usually runs each command without a terminal. The runner streams stdout and stderr to the job log through pipes or files. A prompt that expects a person fails at once or waits until the job times out.

GitHub Actions also sets the CI environment variable to true, and some tools read it. A command started with docker run gets a terminal only when the -t flag is passed.

To reproduce a CI run locally, set CI, give the command empty input, and pipe both output streams, e.g. CI=true acme orders list < /dev/null 2>&1 | cat. The TTY check is one of several reasons that tests pass locally and fail in CI.

How is a TTY different from a pipe?

A TTY connects a program to a person, and a pipe connects two programs. Both accept the same writes, but the C library and Python pick the buffering even for a program that never checks. A pipe also has a reader that can exit early, which causes a broken pipe. The two differ on these points:

FAQs

How do you force color output in a pipe?

Color output in a pipe can be forced with the tool's own flag, e.g. --color=always for ls. Some tools read an environment variable instead, e.g. Node.js reads FORCE_COLOR. The next program in the pipe then receives the escape codes as text.

Why is piped output delayed?

Piped output is delayed because C and Python programs hold stdout in a block buffer when it is not a terminal. Nothing reaches the pipe until the block fills, the program flushes, or the program exits. Flushing after each line removes the delay.

How do you get terminal behavior when output is captured?

Terminal behavior with captured output needs a pseudoterminal, which gives the program a terminal device while another program reads the other end. The script command runs a program under one, and a test can open one itself, e.g. with Python's pty module.

Should a CLI prompt for input when stdin is not a TTY?

A CLI should not prompt for input when stdin is not a TTY, because a script or a pipe supplies the input and no person can answer. The Command Line Interface Guidelines advise failing instead, with an error that names the flag to pass.