The run at a glance
One run deploys a stack, proves that data reaches the graph, runs every suite, and removes the stack.
sequenceDiagram
accTitle: One end-to-end run
accDescr: The CI job deploys the stack and starts a test runner in the cluster. The smoke suite proves that data reaches the graph. The other suites then run in parallel, each with its own user. The job collects the results and removes the stack.
participant CI as CI job
participant K as Test cluster
participant R as Test runner
participant G as GitLab
participant O as Orbit
CI->>K: Deploy GitLab, Siphon, and Orbit
CI->>K: Start the test runner
R->>G: Smoke suite: admin user, feature flag, shared group
G-->>O: Changes reach the graph
R->>G: Query until the canary data appears
par Each other suite
R->>G: Seed data with a new user
R->>G: Query until the expected result appears
end
CI->>R: Read the results
CI->>K: Remove the stack1. Deploy the stack
On a merge request, the job builds a debug image of Orbit from the branch, in the background while the stack deploys. Setup removes old stacks, makes new secrets, and reads the list of replicated tables from the pinned GitLab commit. Then the deploy runs in waves, and each component waits only for what it needs:
flowchart LR
accTitle: The deploy waves
accDescr: A bootstrap step creates the namespaces and secrets. PostgreSQL and Redis come before GitLab. NATS and ClickHouse come before Orbit. The replication setup waits for GitLab, and Siphon waits for the replication setup, NATS, and ClickHouse.
Bootstrap --> DB[PostgreSQL and Redis] --> GitLab --> CDC[Replication setup] --> Siphon
Bootstrap --> Broker[NATS and ClickHouse] --> Orbit
Broker --> SiphonOrbit starts while GitLab is still booting, and a background step activates the license and creates the first administrator token.
2. Prove the pipeline with the smoke suite
The suites are Robot Framework files that run in a pod inside the cluster. The smoke suite runs alone first. It creates an administrator user and token, turns on the GQL query language, and creates a shared group with Orbit turned on. Then it creates a canary project, issue, and note, and queries Orbit until they appear in the graph.
When the smoke suite passes, the whole data path works.
A failure in a later suite then points to the feature under test, not to the stack.
3. Run the other suites in parallel
Every other suite starts at the same time, and each creates its own administrator user, because GitLab limits Orbit queries for each user. Indexing is eventually consistent, so a test polls Orbit until the expected result appears:
create the data in GitLab
deadline = now + budget × multiplier
repeat:
if a query through GitLab returns the expected node or edge:
pass
if now > deadline:
fail with the last result
wait a short timeA slower stack, such as the ClickHouse HA variant, gets a larger multiplier. A test reads like the steps that a user takes:
Group Membership Produces A MEMBER_OF Edge
${suffix}= Random Suffix
${user}= Create User e2e-member-${suffix}
Add Group Member ${SHARED_NAMESPACE_ID} ${user["id"]} 20
Start Indexing Budget 180
Wait For Edge Indexed Within Budget User ${user["id"]}
... MEMBER_OF Group ${SHARED_NAMESPACE_ID}4. Collect the results and remove the stack
Every run saves the test report and the Orbit logs. A failed run also saves the pods, events, and logs of every component. It also saves the ClickHouse tables, errors, and insert and merge logs, and the NATS streams and consumers. CI keeps these files for a few days. Start with the test report, then the Orbit logs, then the ClickHouse state.
The job always removes the stack. If the job stops first, the next run removes it.
Design rationale
We prove the data path once, in the smoke suite. Otherwise a broken stack fails every suite at once and hides the first error. We poll with a budget, because a fixed wait is too short on a slow stack and wastes time on a fast one.