Introduction

Delta is a developer-first, cloud-augmented intelligent test runner built for high-velocity software engineering teams. By mapping dynamic test coverage at runtime (identifying exactly which lines of source code are executed by each test), Delta computes the exact intersection between code changes and the test suite.

When code changes, Delta queries the mapping database and returns only the affected tests, cutting test run times by up to 90%. This documentation guides you through installing the CLI, using it with pytest, and automating it via Git hooks.

Installation

The command-line interface pytest-deltatest is published on PyPI. It can be easily installed using pip or other python package managers in your virtual environment.

Prerequisites

• Python 3.8 or higher
• Git repository initialized
• Existing pytest test suite

Install CLI Package

Install the package inside your project's active virtual environment:

(.venv) $ pip install pytest-deltatest

If you use Poetry or Pipenv, install it as a development dependency:

# Poetry
$ poetry add pytest-deltatest --group dev

# Pipenv
$ pipenv install pytest-deltatest --dev

This automatically installs the CLI utility delta along with its core dependencies: pytest, pytest-cov, and coverage.

Verify Installation

Check if the CLI utility is correctly installed by verifying the version:

$ delta --version
pytest-deltatest version 0.4.6

Choose Your Setup Mode

Delta supports two modes of tracking code-to-test mapping. Choose the setup that best fits your workflow:

Delta Cloud Integration

Cloud integration enables sharing test mappings. If a team member writes a test or runs a test suite, your local workspace will immediately know which tests to run without you needing to run the full suite locally to generate a database.

1. Register Account

If you don't have a Delta account, register a new account from the CLI:

$ delta register
Email: [email protected]
Password: ********
   User registered successfully!

2. Authenticate

Login to save your secure access token to your local environment configuration (saved in ~/.delta/config.toml):

$ delta login
Email: [email protected]
Password: ********
   Authentication successful. API key saved.

3. Track Repository

Associate your local repository with Delta Cloud:

$ delta track --name my-application-repo
   Repository tracked successfully!
Repo ID: aa251edf-8d2b-4e1b-9fca-72bf7e8ea7bc

4. Seed Initial Cloud Mappings

Run your tests locally to build and populate coverage-context recording, then push it to Delta Cloud:

# Run tests and generate coverage context mapping
$ delta build-mapping

# Push mappings to Delta Cloud
$ delta push
   Coverage file found: .coverage
   Syncing mappings to Delta Cloud...
   Mapping pushed successfully!

5. CI/CD Integration & Environment Variables

In CI/CD environments (like GitHub Actions, GitLab CI, or Jenkins), you don't need to write a ~/.delta/config.toml file. Starting with pytest-deltatest >= 0.4.28, you can configure Delta directly using environment variables. This is the recommended way to run Delta in CI/CD pipelines.

Configure the following secrets/environment variables in your pipeline environment:

Example GitHub Actions workflow step:

- name: Run Delta (affected tests only)
  env:
    DELTA_API_KEY: ${{ secrets.DELTA_API_KEY }}
    DELTA_REPO_ID: ${{ secrets.DELTA_REPO_ID }}
  run: |
    delta run

Local Mode

If you prefer offline or isolated database tracking, you can use Local Mode.

Build Local Mapping Database

Delta provides a command to build the mapping database iteratively by executing your tests, extracting coverage contexts, and storing them in an SQLite database file:

$ delta build-mapping --local
Scanning for tests in: tests/
Building local mapping database...
[1/48] Running tests/test_auth.py::test_login... 
[2/48] Running tests/test_auth.py::test_logout... 
...
Database created at .delta/test_mapping.db with 48 test maps!

This command iteratively runs tests, creates .delta/test_mapping.db, and updates it. You should add the .delta/ folder to your .gitignore so database files aren't checked into your source control.

Bypassing Remote Mapping (Offline / Local Mode override)

You can force Delta to run strictly locally (offline) using the --local (or --no-remote) flag:

# Build database locally without connecting to the cloud
$ delta build-mapping --local

# Run affected tests using only the local database
$ delta run --local

The Git pre-commit hook also supports this flag to bypass remote queries:

# Run hook locally without cloud requests
$ delta pre-commit-hook --local

Similarly, for direct pytest invocations, you can use the --delta-local (or --delta-no-remote) option:

# Run smart test selection offline
$ pytest --delta --delta-local

Explaining Test Selection (Tracing Mapped Triggers)

If you want to inspect exactly why certain tests were selected by Delta, you can use the --explain option with delta run. This outputs a detailed breakdown of which tests were selected and their trigger reasons (such as covering specific lines in modified files):

# Explain selected tests without running them
$ delta run --explain --dry-run

Pytest Integration

The pytest-deltatest includes a pytest plugin that hooks directly into the pytest test collection lifecycle to filter out unaffected tests.

Running Affected Tests

To run only the tests affected by your unstaged and staged git changes compared to your base branch, add the --delta flag:

(.venv) $ pytest --delta

Example Terminal Output

Here is what you will see when executing pytest with Delta. Notice how the unaffected tests are automatically deselected, running in milliseconds rather than minutes:

$ pytest --delta
============================= test session starts ==============================
platform darwin -- Python 3.10.8, pytest-7.2.1, pluggy-1.0.0
plugins: cov-4.0.0, delta-0.4.6
Local mode: /Users/name/workspace/app/.delta/test_mapping.db
Analyzing git changes against base branch: master
Found 3 affected tests out of 145 total tests.

collected 145 items / 142 deselected / 3 selected

tests/test_auth.py ...                                                   [100%]

====================== 3 passed, 142 deselected in 1.42s =======================

Specifying a Base Branch

By default, Delta compares changes against the master branch. If your main development branch is named main or develop, pass it via the --delta-base option:

# Compare against main branch
$ pytest --delta --delta-base main

# Compare against develop branch
$ pytest --delta --delta-base develop

This allows Delta to calculate the diff between your current branch and the base branch, running only the tests that traverse those changed code lines.

Dry-Run Preview

If you want to quickly see which tests would be run without actually executing them, use the delta run CLI command with the --dry-run flag:

$ delta run --dry-run
Analyzing git changes against base branch: master
Selected Tests (Dry-Run):
 - tests/test_auth.py::test_login
 - tests/test_auth.py::test_logout
 - tests/test_settings.py::test_update_profile

Configure Defaults

To avoid typing --delta every time, you can configure it as a default option in your project's pytest.ini or pyproject.toml file:

[pytest]
addopts = --delta --delta-base main

[tool.pytest.ini_options]
addopts = "--delta --delta-base main"

Git Pre-commit Hook Integration

Automate your developer workflow by setting up a git pre-commit hook that automatically runs only the affected tests before allowing a commit to succeed.

Install Git Hook

To register the Delta git pre-commit hook in your local git repository, run the following command:

$ delta install-pre-commit

This command performs the following actions:

• Generates/updates a standard .git/hooks/pre-commit script that intercepts commits.
• Generates/updates a .git/hooks/post-commit script that updates the mapping database.
• Ensures your .gitignore is updated to ignore Delta's cache and local database folders.

Developer Workflow Example

Once installed, the hook runs automatically on every commit. Let's see an example workflow:

1. You modify src/auth.py to fix a bug, and stage it:

   $ git add src/auth.py

2. You attempt to commit your change:

   $ git commit -m "Fix login token validation"

3. The pre-commit hook triggers and executes only the affected tests:

   Repository: /Users/name/workspace/app
   Target branch: master
   Log file: /Users/name/workspace/app/.delta/pre-commit.log
   Mapping Service: Delta Cloud (repo aa251edf...)
   
   Running affected tests...
   tests/test_auth.py ..                                            [100%]
   All 2 affected tests passed!
   
   [main c1a2b3d] Fix login token validation
    1 file changed, 4 insertions(+), 1 deletion(-)

Bypassing Hooks

If you have an urgent change or are committing documentation and need to skip running tests, bypass the hook with the standard git flag:

$ git commit -m "urgent readme tweak" --no-verify

Tox Integration

If your project uses tox to manage virtual environments and run test suites, you can easily integrate DeltaTest into your tox.ini workflow.

1. Allow Environment Variables

By default, tox strips out all host environment variables to ensure test isolation. You must explicitly configure tox to pass your Delta API credentials and GitHub CI context variables through to the virtual environment using the passenv setting:

[testenv]
passenv =
    DELTA_*
    GITHUB_*

2. Add Delta Dependency

Ensure that pytest-deltatest is installed inside the virtual environments generated by tox. You can do this by adding it to the deps list in tox.ini:

[testenv]
deps =
    pytest
    pytest-cov
    pytest-deltatest

3. Update Test Commands

Replace the default pytest {posargs} runner with delta run -- {posargs} inside the environment commands section:

[testenv]
commands =
    delta run -- {posargs:tests}

Subprocess Coverage

By default, Python's coverage instrumentation only tracks code executed inside the parent pytest process. If your code under test spawns child processes — using subprocess.run, multiprocessing, concurrent.futures.ProcessPoolExecutor, or any OS-level fork — the lines executed inside those processes are invisible to coverage. This means DeltaTest cannot map those code paths to any test, causing those lines to appear unmapped and always re-run on every commit.

The Root Cause

Coverage instrumentation works by monkey-patching Python's import system at startup. A subprocess is a brand-new Python interpreter process — it starts clean, with no hooks installed. Unless the child process explicitly re-activates coverage before importing your code, none of its execution is recorded.

How --subprocess Solves This

The delta build-mapping --subprocess flag implements a two-part, zero-config mechanism to make child processes automatically record their own coverage and merge it back into the parent's .coverage file.

Part 1 — Install a sitecustomize.py bootstrap hook

Python automatically imports a file called sitecustomize.py from site-packages on every interpreter startup, including in child processes. When you run delta build-mapping --subprocess, Delta writes the following two-line file into your active virtual environment's site-packages directory:

# site-packages/sitecustomize.py (written by delta build-mapping --subprocess)
import coverage
coverage.process_startup()

coverage.process_startup() is a no-op unless the environment variable COVERAGE_PROCESS_START is set. Delta also writes this variable into .delta/.env pointing to your project's .coveragerc, so it is automatically available to all subprocesses spawned during a test run.

Part 2 — Enable parallel coverage collection

Each child process writes its coverage data to a separate file named .coverage.<hostname>.<pid>.<random> rather than writing directly to the shared parent .coverage file (which would cause race conditions). Delta automatically writes or updates your .coveragerc to enable this:

[run]
parallel = True
concurrency = multiprocessing

After all tests finish, Delta runs coverage combine to merge all the per-process .coverage.* files back into a single .coverage database. The mapping pipeline then reads from this combined file as normal — subprocess coverage is fully transparent to the rest of the toolchain.

End-to-End Flow

pytest (parent process)
  │  COVERAGE_PROCESS_START=.coveragerc is set in env
  │
  ├─ test_a runs → spawns subprocess.run(["python", "worker.py"])
  │                        │
  │               Python starts → sitecustomize.py auto-imported
  │                        │
  │               coverage.process_startup() activates
  │                        │
  │               worker.py executes → writes .coverage.host.12345.abc
  │
  ├─ test_b runs → spawns multiprocessing.Process(target=job)
  │                        │
  │               fork() → sitecustomize.py re-runs in child
  │                        │
  │               job() executes → writes .coverage.host.12346.def
  │
  └─ pytest session ends
           │
           ▼
  coverage combine
  .coverage.host.12345.abc  ┐
  .coverage.host.12346.def  ├─→  .coverage  (merged)
  .coverage  (parent)       ┘
           │
           ▼
  delta update-mapping → test_mapping.db updated with subprocess paths
  delta push            → cloud mapping includes subprocess coverage

Usage

# Build mapping with subprocess coverage tracking enabled
$ delta build-mapping --subprocess

# Works with all other flags too
$ delta build-mapping --subprocess --local
$ delta build-mapping --subprocess -- -n auto

Limitations

• Python subprocesses only. This mechanism works for any child process that runs the Python interpreter (CPython). Subprocesses running compiled binaries (Go, Rust, C, Node.js) require separate language-specific coverage instrumentation.
• Virtual environment scoped. The sitecustomize.py is written into the active virtual environment's site-packages. If you switch virtual environments, run delta build-mapping --subprocess once inside the new environment to reinstall the hook.
• Thread-safe, process-unsafe without parallel=True. Without parallel = True in .coveragerc, multiple processes writing to the same .coverage file simultaneously will corrupt the database. Delta sets this automatically when --subprocess is used.

Command Reference

Delta provides a full set of subcommands under the main delta command prefix.

VS Code Extension

The Python DeltaTest VS Code extension integrates Delta’s intelligent test selection directly into your editor with live, inline highlights and interactive controls.

Install from Marketplace

Key Features

• Line Highlights: Every line covered by at least one test is highlighted with an inline glow.
• Hover Tooltips: Hover over any highlighted line to see the exact tests executing it.
• Auto-Run Affected Tests: Runs the affected tests automatically on file save or line edits, integrating directly into VS Code’s native Test Results panel and logging to the Delta Auto-Run output channel.
• Update Mapping Button: Triggers delta build-mapping followed by delta push programmatically to sync your database.
• Run Affected Tests Button: Triggers delta run to execute only the tests affected by your current local changes.

Extension Settings

Configure the extension by opening your VS Code settings and searching for Delta Coverage:

Setting	Default	Description
deltaCoverage.enabled	true	Enable or disable inline coverage highlights.
deltaCoverage.apiKey	""	Your Delta Cloud API Key. If empty, the extension checks ~/.delta/config.toml.
deltaCoverage.pythonPath	""	Custom Python executable path. If you are using tox or a custom environment, point this to the relevant virtual environment (e.g. .tox/py310/bin/python).
deltaCoverage.branch	"main"	The base git branch to compare changes against.
deltaCoverage.testDir	"tests"	Relative path to the test directory.

$ delta build-mapping

$ pytest --delta --delta-base main

$ delta install-pre-commit --base-branch main

$ rm -f .git/hooks/pre-commit .git/hooks/post-commit

$ delta build-mapping --subprocess