Skip to content

Escaped Defects Analysis
What A structured review of why a defect reached production, who is accountable, and how to prevent recurrence.
Owner Project Manager facilitates; Engineering, QA, and Leadership participate.
Triggers After a production-reported issue has been fixed.

Summary

An escaped defect is a bug that reached production and was reported by users or clients, rather than being caught in QA. After such an issue is fixed, the team holds an Escaped Defects Analysis (EDA) meeting to determine why it escaped and who, if anyone, is accountable. The categories are deliberately specific so accountability is fair and evidence-based. Each person has a configurable allowance of escaped defects per review period, and the metric feeds into promotion and salary decisions. The goal is fairness and prevention, not blame for its own sake.

Full detail

When EDA runs

EDA runs after a production-reported issue has been fixed (not before; the immediate priority is always to resolve the client's problem first via Production Support and, if needed, a Hotfix). Once resolved, the EDA examines how it escaped.

Accountability categories

Every escaped defect is classified into exactly one of these:

Category Description Accountable
Dev fault The developer missed an impact area and did not inform QA, so QA had no way to know to test it. Developer
QA fault The developer did inform QA of the impact area, but QA missed testing it and the issue escaped. QA
No fault / pre-existing The issue existed in production before this release. It escaped, but no one is blamed due to lack of evidence tying it to a specific change. No one
Edge case / out of scope An edge case, an unrelated factor, or something outside the agreed scope caused it. No one

The classification rests on evidence: the impact analysis attached to the build, the test plan, and the defect's root cause. This is why the impact analysis and root-cause discipline in earlier phases matter; they are what make a fair EDA possible.

flowchart TD
    A[Production-reported issue fixed] --> B[EDA meeting]
    B --> C{Did dev identify and communicate the impact area?}
    C -->|no, dev missed/omitted it| D[Dev fault]
    C -->|yes, dev informed QA| E{Did QA test that area?}
    E -->|no, QA missed it| F[QA fault]
    E -->|yes, still escaped| G{Pre-existing in production?}
    G -->|yes, no evidence tying to this release| H[No fault / pre-existing]
    G -->|no| I{Edge case / out of scope / unrelated?}
    I -->|yes| J[Edge case - no fault]
    I -->|no| K[Re-examine evidence]
    D --> L[Record against individual's allowance]
    F --> L
    H --> M[Record, no accountability]
    J --> M
    L --> N[Capture prevention action]
    M --> N

Allowances, review period, and consequences

  • Each person is allocated a configurable number of escaped defects per review period. The threshold is set in each person's individual goals, not fixed in this playbook.
  • The threshold is lower for more senior positions: seniority carries a higher expectation of catching impact areas and edge cases.
  • The review cycle is every 6 months. Exceeding the allowance within a review cycle triggers the agreed penalty.
  • Promotion and salary reviews run annually and depend on this metric, among others. Consistently staying within allowance supports advancement; repeatedly exceeding it works against it.

Thresholds live in individual goals

The actual numbers (allowance per role, penalty) are intentionally not specified here. They are configured per person in their goals by Leadership, because they vary by seniority. This page defines the process and the 6-month review / annual promotion cadence; the numbers are set elsewhere.

Prevention is the real output

Beyond accountability, every EDA produces a prevention action: a missing test added, a logging gap closed (see Post-Production Monitoring), an impact-analysis habit reinforced, or a process tweak raised in the Monthly Retrospective. Recurring patterns across EDAs are a strong signal to change the process, not just the people.

Example

A production issue is reported and hotfixed: a discount rule misfired for one client tier. In EDA, the evidence shows the developer's impact analysis did not list the discount module as affected, even though the change touched shared pricing code. QA was never told to test it. This is classified Dev fault and recorded against that developer's allowance for the 6-month cycle. The prevention action is a checklist item: any change to shared pricing code must list the discount module as impacted. That item is added to the impact-analysis guidance.