An OwnStack product
Parallax runs one suite against Chromium, Firefox and WebKit at the same time and shows you where they disagree. The cell that is green on two engines and red on the third is not noise around the answer. It is the answer.
| Spec | chromium | firefox | webkit |
|---|---|---|---|
| signs a customer in | 412ms | 1180ms | 903ms |
| adds an item to the cart | 388ms | 1044ms | 861ms |
| picks a delivery date | 501ms | 1297ms | 2140ms |
| completes checkout | 733ms | 1610ms | 1288ms |
One run. WebKit is the only engine that fails the date picker, and it takes 2.1s to do it.
How it works
One process per browser, so a WebKit crash cannot take Chromium’s results with it and the matrix finishes in about the time the slowest engine takes.
A violated expectation and a broken runner are different things and are reported differently. A red build tells you which one you have.
Flake is retried, and a spec that only passes on the third attempt is marked flaky rather than quietly rounded up to green.
Writes JUnit XML your CI already knows how to render, with the browser in the classname so a failure says which engine broke.
RUN=$(curl -sf -X POST -H "Authorization: Bearer $PARALLAX_KEY" \ https://parallax-api.ownstack.org/api/suites/1/run | jq -r .id) curl -sf -H "Authorization: Bearer $PARALLAX_KEY" \ https://parallax-api.ownstack.org/api/runs/$RUN/junit -o junit.xml curl -sf -H "Authorization: Bearer $PARALLAX_KEY" \ https://parallax-api.ownstack.org/api/runs/$RUN | jq -e .green