The test pattern
Integration tests run real Orbit code on real inputs, and most of them follow one pattern:
flowchart LR
accTitle: The integration test pattern
accDescr: A YAML file declares the input and the expected behavior. A harness runs the real Orbit code path on the input and compares what Orbit produced with the expected behavior.
Spec[YAML file: input and expected behavior] --> Harness --> Code[Real Orbit code path] --> Compare[Compare with the expected behavior]- A YAML file declares the test: the input, such as seed rows, source files, or a query, and the behavior to expect.
- The harness runs the real code path, the way production compiles, indexes, or queries.
- The harness checks only what a caller can observe: result rows, the graph, messages, or the query plan. It never calls an internal function.
To add a test, add a file. The harness finds it with no registration and no Rust code. Similar tests share a folder, one for each behavior, so a reader finds every test of one behavior in one place.
Harnesses
| Harness | Checks | Infrastructure |
|---|---|---|
| Plan tests | The query plan and the generated SQL | None |
| Code-graph suites | The code graph, with graph queries | None |
| Command-line tests | The output and the local graph of the real binary | None |
| Query and indexer scenarios | The query response, and the graph that the indexer writes | ClickHouse |
| Container tests | The message broker, code indexing, and analytics events | ClickHouse, NATS, and an event collector |
The command-line and container tests are Rust tests, but they also check only observable behavior. The command-line tests run the real binary against temporary repositories. They cover concurrent readers and writers, repeated indexing, worktrees, nested repositories, unreadable files, and the bundled skills.
When they run
Every harness runs on each merge request that changes code and on each commit to main, and a failure blocks the merge. In CI, a run continues after a failure, retries a failed test, and reports each test. A check fails if a container test is in no CI lane or in two lanes, so no test drops out of CI.
Design rationale
We write tests as data, because a data file states what Orbit must do, not how the code does it. So the test stays valid through a refactor. We test the real code path, because a mock of an Orbit part repeats the assumptions of the code under test. We mock only the systems that Orbit does not own: GitLab authorization and repository storage.