I've seen test suites at every stage: 10 tests, 500 tests, 5,000 tests. The architecture that works at 50 tests will collapse at 500. Here's what I've learned about building test infrastructure that scales.
The testing pyramid (still relevant, with updates)
The classic pyramid (unit → integration → E2E) is still the right foundation. But most teams get the ratios wrong:
- Unit tests (60-70%): Fast, cheap, test logic in isolation. These should be the bulk of your suite.
- Integration tests (20-25%): API-level tests that verify service contracts, database queries, and component interactions without a browser.
- E2E tests (10-15%): Browser-based tests for critical user flows only. These are expensive to write and maintain. Be selective.
The mistake I see most often: teams with 2,000 E2E tests and 50 unit tests. The pyramid is inverted. CI takes 45 minutes. Flake rate is 20%+. The fix isn't more E2E tests. It's pushing coverage down the pyramid.
Architecture principles for scale
1. Test isolation is non-negotiable
Every test must be able to run independently, in any order, in parallel. No shared state between tests. No dependency on other tests running first. If you can't run a single test in isolation, your architecture is broken.
2. Test data management
At scale, test data becomes the hardest problem. Solutions that work:
- Factory pattern: Each test creates its own data via API calls in setup. No shared seed data.
- Database snapshots: Reset to a known state before each test suite (not each test, that's too slow).
- Transactional rollback: For integration tests, wrap each test in a transaction and roll back.
3. Parallel execution from day one
Design for parallelism immediately. If you add parallelism later, you'll spend weeks fixing shared state bugs. Playwright runs parallel by default. Use sharding in CI to distribute across multiple machines.
4. Tagging and selective execution
Not every test needs to run on every commit. Tag tests by priority:
@critical: Runs on every PR (login, checkout, core flows). Should be under 5 minutes.@regression: Runs on merge to main. Full suite, 10-15 minutes max.@nightly: Runs on schedule. Cross-browser, edge cases, performance. Can take longer.
5. Reporting and observability
At 10,000 tests, you need dashboards, not log files. Track:
- Test pass rate over time (target: 99%+)
- Flake rate per test (quarantine anything above 5%)
- CI execution time trend (set alerts if it exceeds your threshold)
- Coverage gaps (which features have no test coverage)
Common scaling failures
- Monolithic test suites: One repo, one config, one CI job. At scale, split by domain or service.
- No ownership: Tests without owners rot. Assign test ownership to the team that owns the feature.
- Testing implementation details: Tests coupled to internal implementation break on every refactor. Test behavior, not code structure.
Scaling your test suite and hitting walls? Let's talk. We've built test infrastructure for teams running thousands of tests in CI every day.