Run Tests

The 10xgraph test command runs your project’s test suite and measures code coverage. It is a thin wrapper around pytest that reads your project configuration from 10xgraph.json (found in the current directory or a parent), so you can set coverage thresholds and test paths once and enforce them in CI without duplicating settings.

Prerequisites

Your environment needs pytest and, if you want coverage reports, pytest-cov. These are standard dependencies that belong in your project’s pyproject.toml or requirements.txt:

pip install pytest pytest-cov

Pytest discovers tests automatically from your project root, looking for files named test_*.py or *_test.py and functions named test_*. You can customize discovery with a pytest.ini or pyproject.toml configuration file (see pytest docs); the 10xgraph test command respects those settings.

Run your tests

From the folder that contains 10xgraph.json, run all tests:

10xgraph test

The command runs pytest from the project root using pytest’s own discovery rules: it reads testpaths from pytest.ini or pyproject.toml, or falls back to scanning the current directory. This behavior matches running pytest directly.

10xgraph test exits with pytest’s own exit code: 0 when the run succeeds, non-zero when tests fail or no tests are collected. If coverage is enabled, a coverage_threshold is set in 10xgraph.json, and coverage falls below it, the run also fails even if all tests pass.

Target a specific test path

Restrict the run to a directory or file by providing a path argument:

# Run all tests in a subdirectory
10xgraph test tests/unit

# Run a single test file
10xgraph test tests/unit/test_graph.py

When you provide a path, pytest only collects tests under that location. This is useful for running a fast subset of tests during local development before running the full suite in CI.

Measure code coverage

Add --coverage (short form -C) to enable code coverage reporting:

10xgraph test --coverage

This adds the following flags to pytest:

--cov=. --cov-report=term-missing --cov-report=html:htmlcov

The command prints a coverage summary in the terminal and writes a detailed HTML report to htmlcov/index.html. The HTML report shows which lines are covered, which are missing, and why: you can click into each file to see untested branches. This helps you understand coverage gaps without reading raw data.

To open the report in your default browser automatically:

10xgraph test --coverage --html

After the test run completes, the browser opens to htmlcov/index.html.

Filter tests by keyword expression

Run only tests matching a name pattern with the -k option:

10xgraph test -k "weather"

This forwards the expression directly to pytest. Only tests whose name or node ID matches the expression are collected and run. Expressions support and, or, and not: for example, -k "weather and not slow" runs tests with “weather” in the name, excluding those with “slow”.

Pass raw pytest arguments

Use -- to separate 10xgraph test options from raw pytest arguments that you want to pass through unchanged:

# Quiet output, short tracebacks
10xgraph test -- -q --tb=short

# Long tracebacks, no header
10xgraph test --coverage -- --tb=long --no-header

# Run only fast tests, skip slow and integration
10xgraph test -- -m "not slow and not integration"

Everything after -- is appended to the end of the pytest command. This lets you use any pytest feature without writing custom CLI options in 10xgraph test itself.

Configure your testing defaults

Add a test section to 10xgraph.json to set project-level defaults for path, coverage, and coverage threshold. CLI flags always take precedence over config values:

{
  "agent": "graph.react:app",
  "test": {
    "path": "tests",
    "coverage": true,
    "coverage_threshold": 80
  }
}

With the config above, a bare 10xgraph test is equivalent to running:

10xgraph test tests --coverage -- --cov-fail-under=80

Enforce coverage in CI

To fail the CI pipeline when code coverage drops below a target, set coverage_threshold in 10xgraph.json and run 10xgraph test --coverage in your CI workflow. The command exits with a non-zero code if coverage falls short, which stops the pipeline.

Here is a GitHub Actions example:

name: Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v4
        with:
          python-version: "3.12"
      - run: pip install -e ".[dev]"
      - name: Run tests with coverage
        run: 10xgraph test --coverage

The coverage_threshold from 10xgraph.json is enforced because --coverage is on, so you do not repeat it in the workflow. Other CI systems (GitLab CI, CircleCI, Jenkins, etc.) follow the same pattern: install dependencies, then run 10xgraph test --coverage. The exit code works the same way everywhere.

Control output verbosity

By default, 10xgraph test passes -v to pytest, printing one line per test. Use --quiet (-q) to pass -q instead:

10xgraph test --quiet

--verbose (-v) keeps the default -v and also turns on verbose CLI logging. For full tracebacks, pass --tb=long after --.

Common workflows

Run a fast smoke test during development:

10xgraph test tests/test_smoke.py -k "health"

This runs only tests in the smoke test file with “health” in the name. Useful for quick validation before committing.

Full coverage check with a visual report:

10xgraph test --coverage --html

This generates a detailed HTML coverage report and opens it in your browser. You can browse the report to find uncovered lines and decide which ones need tests.

Strict CI gate with a high threshold:

{
  "agent": "graph.react:app",
  "test": {
    "coverage": true,
    "coverage_threshold": 85
  }
}

10xgraph test

In CI, this ensures both that tests pass and that coverage never drops below 85%.

Run all except slow or integration tests locally:

10xgraph test -- -m "not slow and not integration"

This is useful when you want fast feedback during development. Mark slow tests with @pytest.mark.slow and integration tests with @pytest.mark.integration in your test code.

Troubleshoot common errors

“No module named pytest”

Pytest is not installed. Install it in your environment:

pip install pytest

If you use a virtual environment or uv, activate it first.

“No module named pytest_cov”

The pytest-cov plugin is not installed. Install it:

pip install pytest-cov

Coverage reporting requires this plugin. If you never use --coverage, you do not need it.

Coverage is below threshold, tests fail

The test command exits non-zero because coverage fell below coverage_threshold in 10xgraph.json. Either increase your test coverage, or lower the threshold if the current target is unrealistic. Check the coverage report (htmlcov/index.html after running with --coverage) to see which lines are uncovered.

Tests directory not found or no tests collected

Pytest did not find any tests. Check the path:

# Explicit path
10xgraph test tests

# Or list what pytest finds
pytest --collect-only

If the path is correct, ensure your test files are named test_*.py or *_test.py, and test functions are named test_*. You can customize this in pytest.ini or pyproject.toml.

Import errors in tests

If your tests import your agent code, ensure the project is installed in editable mode:

pip install -e .

This makes your source code importable from tests.

Next steps

Once your test suite is running, explore the unit testing tools in unit tests to write tests more effectively. For evaluation (checking whether an agent answers questions correctly), see evaluation instead.