by Adam BenderThis article was adapted from a GoogleTesting on the Toilet (TotT) episode. You can download a printer-friendlyversion of this TotT episode and post it in your office.An end-to-end test tests your entire system from one end to the other, treatingeverything in between as a black box. End-to-end tests can catch bugsthat manifest across

your entire system. In addition to unitand integration tests, they are a critical part of a balanced testing diet,providing confidence about the health of your system in a near production state.Unfortunately, end-to-end tests are slower, more flaky, and moreexpensive to maintain than unit or integration tests. Considercarefully whether an end-to-end test is warranted, and if so, how best to writeone.Let's consider how an end-to-end test might work for the following "login flow":

In order to be cost effective, an end-to-end test should focus onaspects of your system that cannot be reliably evaluated with smallertests, such as resource allocation, concurrency issues and APIcompatibility. More specifically:

• For each important use case, there should be one correspondingend-to-end test
  . This should include one test for each important classof error. The goal is the keep your total end-to-end count low.
• Be prepared to allocate at least one week a quarter per test to keepyour end-to-end tests stable in the face of issues like slow and flakydependencies or minor UI changes.
• Focus your efforts on verifying overall system behavior instead ofspecific implementation details; for example, when testing loginbehavior, verify that the process succeeds independent of the exact messages orvisual layouts, which may change frequently.
• Make your end-to-end test easy to debug by providing anoverview-level log file, documenting common test failure modes, and preservingall relevant system state information (e.g.: screenshots, database snapshots,etc.).

End-to-end tests also come with some important caveats:

• System components that are owned by other teams may change unexpectedly, andbreak your tests. This increases overall maintenance cost, but can highlightincompatible changes
• It may be more difficult to make an end-to-end test fullyhermetic; leftover test data may alter future tests and/or productionsystems. Where possible keep your test data ephemeral.
• An end-to-end test often necessitates multiple test doubles (fakes or stubs)for underlying dependencies; they can, however, have a high maintenance burdenas they drift from the real implementations over time.