Designing Tests for CD

Test patterns, principles, and best practices for building fast, reliable test suites that enable Continuous Delivery
Tags:

  2 minute read  

There are common patterns to show how much of each kind of test is generally recommended. The most used are the Test Pyramid and the Test Trophy. Both are trying to communicate the same thing: design a test suite that is fast, gives you confidence, and is not more expensive to maintain than the value it brings.

Testing Principles

  • Balance cost and confidence
  • Move failure detection as close to the developer as possible
  • Increase the speed of feedback
    • CI to take less than 10 minutes.

Most of the tests are integration tests and emphasize maximizing deterministic test coverage in process with the development cycle, so developers can find errors sooner. E2E & functional tests should primarily focus on happy/critical path and tests that absolutely require a browser/app.

When executing continuous delivery, test code is a first class citizen that requires as much design and maintenance as production code. Flakey tests undermine confidence and should be terminated with extreme prejudice.

Testing Matrix

Feature Static Unit Integration Functional Visual Regression Contract E2E
Deterministic Yes Yes Yes Yes Yes No No
PR Verify, Trunk Verify Yes Yes Yes Yes Yes No No
Break Build Yes Yes Yes Yes Yes No No
Test Doubles Yes Yes Yes Yes Yes See Definition No
Network Access No No localhost only localhost only No Yes Yes
File System Access No No No No No No Yes
Database No No localhost only localhost only No Yes Yes

Testing Anti-patterns

“Ice cream cone testing” is the anti-pattern where the most expensive, fragile, non-deterministic tests are prioritized over faster and less expensive deterministic tests because it “feels” right.

Google Test Blog: Just Say No to More End-to-End Tests

Testing Best Practices

General testing best practices are documented here. Best practices specific to test types are documented within each test type page.

Test Pattern Resources

Testing Terms Glossary

Standardized definitions for testing terms to establish ubiquitous language and reduce ambiguity in testing discussions

Static Testing

Code analysis tools that evaluate non-running code for security vulnerabilities, complexity, and best practice violations

Testing Quickstart

Get your test suite to production-ready in one week

Unit Testing

Fast, deterministic tests that verify individual functions, methods, or components in isolation with test doubles for dependencies

Integration Testing

Deterministic tests that verify how units interact together or with external systems using test doubles for non-deterministic dependencies

Contract Testing

Non-deterministic tests that validate test doubles by verifying contract format against live external systems

E2E Testing

Understanding and implementing End-to-End (E2E) testing in software development

Functional Testing

Understanding and implementing Functional Testing in software development

Test Doubles

Understanding and implementing Test Doubles in software testing

Customer Experience Alarms

Active monitoring that sends requests to test critical customer workflows every minute to ensure system health and catch errors early

Testing Best Practices

Comprehensive guide to test-first approaches, naming conventions, test effectiveness, and practices for building maintainable test suites

Last modified October 20, 2025: docs: comprehensive documentation improvements (c4499d8)