Services / Assure
Test automation that people actually trust
We build test suites with the right shape: fast unit and integration layers doing the heavy lifting, a small set of end-to-end journeys, contract tests at service boundaries, and mutation testing instead of a coverage percentage. Flaky tests are quarantined within a day under a written process, because a suite people rerun until it goes green is not a gate.
- Coverage measured on changed lines, plus mutation score on high-risk modules
- Flake rate above one percent blocks the pipeline, no exceptions
- Load tests with pass criteria written against your SLOs
- Accessibility automated in CI, with manual screen reader passes because automation catches a third
At a glance
- Browser automation
- Playwright, Cypress, Selenium where mandated
- Unit and integration
- Jest, Vitest, pytest, JUnit 5, xUnit
- Contract testing
- Pact, OpenAPI and Protobuf compatibility checks
- Performance
- k6, Gatling, JMeter, Locust
- Accessibility
- axe-core, Pa11y, NVDA, VoiceOver, TalkBack
- Security in CI
- Semgrep, CodeQL, OWASP ZAP, Trivy, gitleaks
- Flake rate gate
- Above 1 percent blocks the pipeline
- Enquiries
- nitesh@redcubical.com
Strategy
The test pyramid, and the ice-cream cone you probably have
Test pyramid versus ice-cream cone
A healthy suite is mostly fast unit and integration tests with a thin layer of end-to-end journeys. The ice-cream cone is the inversion: hundreds of slow browser tests, few unit tests, and manual regression underneath. It produces a suite that takes an hour, fails randomly, and nobody trusts. The fix is not more end-to-end tests.
| Layer | Target proportion | Runtime per test | What it catches | Cost per test over its life |
|---|---|---|---|---|
| Unit | 60 to 70 percent | Under 10 ms | Logic errors, boundary and off-by-one conditions, calculation mistakes, invalid state transitions. Pinpoints the failing function | Lowest. Cheap to write, cheap to maintain, rarely needs changing unless behaviour changes |
| Integration and component | 20 to 30 percent | 50 ms to 2 s | Wiring defects, ORM and query mistakes, serialisation, transaction boundaries, migration correctness, framework configuration | Low to moderate. Needs a real database or a container, so setup and teardown discipline matters |
| Contract | 3 to 5 percent | Under 1 s | Breaking changes between services without running the whole estate. Catches provider changes that would break a known consumer | Low, and unusually high value per test in a multi-service system |
| End-to-end (browser or device) | 5 to 10 percent | 5 to 60 s | Integration of the whole stack on the journeys that matter: sign-up, checkout, payment, the primary workflow. Catches problems no lower layer can see | Highest. Slow, environment-dependent, and the main source of flake. Every one must justify itself |
| Manual and exploratory | Time-boxed sessions, not a count | Human hours | Usability problems, confusing copy, wrong-problem features, and the defect classes nobody wrote a test for because nobody imagined them | Not comparable. This is discovery work, and automating it would defeat the purpose |
These proportions are a target shape, not a rule to enforce with a linter. What matters is the effect: the blocking suite completes in under ten minutes, a failure tells you where the defect is rather than that something somewhere broke, and the same defect is not caught by four different layers at four different costs.
Symptoms of an inverted pyramid
- The suite takes over thirty minutes, so developers push and walk away rather than waiting for feedback.
- Rerunning a job is the accepted first response to a red build.
- A single failure produces a screenshot and a timeout message, and diagnosing it takes longer than fixing the defect.
- End-to-end tests exist because a unit test was hard to write, not because the journey needed integration coverage.
- Coverage looks respectable, yet defects keep escaping to production in the same modules.
How we invert it back
- Measure first. Runtime, pass rate and flake rate per test, plus which tests have never failed on a real defect. That last group is often a quarter of the suite.
- Map escaped defects to layers. For the last fifty production defects, ask which layer should have caught each. This tells you where coverage is genuinely missing rather than where it is merely thin.
- Delete rather than migrate. End-to-end tests that duplicate lower-layer coverage or have never caught anything are removed. This is the fastest win and the one teams resist most.
- Push assertions downward. Business rules verified in end-to-end tests are re-expressed as unit tests, and the browser test is reduced to proving the journey holds together.
- Add contract tests at boundaries, which removes most of the reason a full-estate integration environment existed.
- Keep ten to twenty end-to-end journeys covering the paths that make money and the paths that would embarrass you. Run them on every merge and nightly across browsers.
Measurement
Why coverage percentage is a bad target, and what to use instead
Better than coverage percentage
Line coverage measures whether a line executed, not whether anything was asserted about it. A suite can reach 90 percent while asserting almost nothing. Worse, once coverage becomes a target, teams write tests that satisfy the metric. Measure coverage on changed lines, mutation score on high-risk modules, and escaped defect rate.
| Metric | What it tells you | Target we work to | How it is measured | Why not the obvious alternative |
|---|---|---|---|---|
| Coverage on changed lines | Whether new and modified code is tested, which is where defects are introduced | Above 80 percent on the diff, enforced per pull request | Diff coverage report from the pipeline, posted on the pull request | Global coverage is dominated by legacy code and does not move when a risky change lands untested |
| Mutation score on high-risk modules | Whether tests would actually fail if the code were wrong. Mutants are deliberate small changes; a surviving mutant means no test noticed | Above 70 percent on pricing, entitlement, calculation and authorisation modules | Stryker, PIT or mutmut on a scoped module set, run nightly because it is expensive | Coverage cannot distinguish an assertion from a bare function call. Mutation testing can, and it is the only metric that directly measures assertion quality |
| Escaped defect rate | Defects reaching production per release or per thousand lines changed. The outcome metric everything else is a proxy for | A downward trend, with the absolute figure baselined per product | Production defect tickets classified by the layer that should have caught them | It is lagging, which is why it is used alongside leading indicators rather than instead of them |
| Defect escape by layer | Where your coverage is genuinely missing, as opposed to where it feels thin | No single layer accounting for more than 40 percent of escapes | Root cause classification during defect triage, which takes about five minutes per defect | Guessing where to add tests is how suites become slow without becoming safer |
| Flake rate | Whether the suite is trusted. The most important operational metric in testing | Below 1 percent of test runs. Above that blocks the pipeline | Pass and fail history per test across all runs on the main branch | A green build is meaningless if a red build is routinely dismissed as flake |
| Blocking suite duration | Whether developers get feedback while still holding context | Under 10 minutes to first actionable failure | Pipeline stage timings tracked as a time series | A slow suite is bypassed, and a bypassed gate is worse than no gate because it creates false confidence |
We report these monthly with the trend, not just the current value. One caution on mutation testing: it is computationally expensive and can take hours on a large codebase, so it runs nightly on a scoped set of modules rather than on every pull request. Applied to an entire codebase it becomes noise nobody reads.
Tooling
What we use at each layer, and why
Tool selection
Playwright for browser automation, Vitest or Jest for JavaScript and TypeScript units, pytest for Python, JUnit 5 for Java, Pact for contracts, k6 for load, and axe-core for accessibility. Selenium only where an existing estate or a client policy requires it. Tooling matters far less than suite shape and flake discipline.
| Tool | Layer | Strengths | Limitations | We choose it when |
|---|---|---|---|---|
| Playwright | End-to-end and API | Chromium, Firefox and WebKit from one API. Auto-waiting that removes most sleep-based flake. True parallelism across workers. Trace viewer with a timeline, DOM snapshots and network log. Multi-tab, multi-origin, file download and mobile emulation all supported | Younger ecosystem than Selenium. Debugging inside a browser dev console is less immediate than Cypress | Default for all new browser automation |
| Cypress | End-to-end and component | Outstanding developer experience with time-travel debugging in the browser. Component testing is genuinely good. Very fast to become productive | One browser tab and one origin per test without workarounds. Parallelism relies on the paid dashboard or custom sharding. WebKit support is still limited | An existing healthy Cypress suite, or component testing where the interactive runner is worth it |
| Selenium and WebDriver | End-to-end | Broadest browser and language support, mature grid infrastructure, frequently the only option approved in regulated enterprise estates | No auto-waiting, so explicit waits are needed everywhere and flake is far more likely. Slower, and much more code for the same test | Client policy mandates it, or a large existing suite makes migration uneconomic |
| Vitest and Jest | Unit and integration (JavaScript, TypeScript) | Fast, good mocking, snapshot testing, wide ecosystem. Vitest is notably quicker on Vite-based projects and shares config | Snapshot tests are easy to over-use and become change-detector tests nobody reads before approving | Vitest for new Vite projects, Jest where the project already uses it |
| pytest | Unit and integration (Python) | Fixtures compose cleanly, parametrised cases are concise, plugin ecosystem covers most needs including Django and async | Fixture scope and ordering surprises are the usual source of test interdependence | Any Python service or data pipeline |
| JUnit 5 and Testcontainers | Unit and integration (Java, Kotlin) | Mature, excellent IDE support. Testcontainers gives real PostgreSQL, Kafka or Redis per test class, which removes a whole category of mock-versus-reality defects | Container start-up adds seconds per suite, so reuse strategy matters | JVM services, and anywhere a real dependency beats a mock |
| Pact | Contract | Consumer-driven contracts verified in the provider pipeline. A provider cannot deploy a change that breaks a known consumer. Removes most of the need for a shared integration environment | Requires both sides to participate and a broker to run. Not useful for third-party APIs you do not control | Multiple services or clients built by different teams |
| k6 | Performance | Tests written in JavaScript, so developers maintain them. Low resource footprint per virtual user, clean thresholds that map to pass criteria, first-class CI integration | No browser-level rendering measurement. Very high virtual user counts need distributed execution | Default for API and service load testing |
| Gatling | Performance | Efficient asynchronous engine, strong reporting, expressive Scala or Java DSL, handles very high concurrency on modest hardware | Scala DSL is a barrier for some teams. Some features sit behind the commercial edition | High-concurrency scenarios on the JVM, or an existing Gatling investment |
| JMeter | Performance | Long-established, GUI-driven, huge plugin range, protocol support beyond HTTP including JDBC, JMS and LDAP | XML test plans are painful in version control and code review. Heavier per virtual user. GUI-first workflow resists automation | Non-HTTP protocols, or an enterprise estate with existing JMeter plans and skills |
| axe-core and Pa11y | Accessibility | Reliable, low false positive rate, integrates into unit, component and end-to-end layers, and can fail a build on new violations | Detects roughly a third of WCAG failures. Cannot assess meaning, reading order or whether a custom widget is usable | Every project, as a gate, always paired with manual screen reader passes |
We would rather inherit a well-shaped Selenium suite with a one percent flake rate than a badly-shaped Playwright suite at fifteen percent. Tool migration is often proposed as a fix for problems that are actually about test design, environment stability and data setup, and migrating carries all of those problems across intact.
Discipline
Flaky tests and test data, the two things that sink suites
Flaky test policy
A flaky test is worse than no test, because it teaches the team to ignore red builds. Our rule is absolute: a test that fails intermittently is quarantined out of the blocking suite within one working day, raised as a defect with an owner and a seven-day fix window, and deleted if unfixed. Aggregate flake rate above one percent blocks the pipeline.
| Root cause | How it presents | Fix | Prevention |
|---|---|---|---|
| Timing and race conditions | Passes locally, fails in CI. Fails more often on a loaded runner. Fixed-duration sleeps in the code | Replace sleeps with waits on an observable condition: element state, network response, or an application-emitted readiness signal | Auto-waiting frameworks, a lint rule banning fixed sleeps, and an application that exposes a settled state |
| Shared mutable state between tests | Passes alone, fails in a suite. Order-dependent. Fails when parallelism increases | Isolate per test: a fresh schema or transaction rollback, unique identifiers, no reliance on a record created by another test | Randomise execution order in CI so order dependence surfaces immediately rather than months later |
| Test data drift | Worked until a seed changed, a reference dataset was updated, or someone edited a record in the shared environment | Each test creates the data it needs and cleans up, or the environment is reset from a known snapshot per run | No test depends on data it did not create. Shared editable fixtures are treated as an anti-pattern |
| Unstable selectors | Fails after unrelated styling or markup refactors. Selectors chained through generated class names or nth-child positions | Stable test identifiers or accessible roles and names, which has the side benefit of surfacing accessibility gaps | A convention that test identifiers are production code, reviewed and not removed casually |
| Third-party and network dependency | Fails when a sandbox is down, rate-limited, or slow. Failure rate correlates with time of day | Mock at the boundary for functional tests. Verify the real integration in a separate, non-blocking suite | A firm rule that no blocking test depends on a system you do not control |
| Time and timezone dependence | Fails overnight, at month end, on the last day of February, or in a runner set to UTC | Inject a controllable clock. Never call the system clock directly in code under test | A clock abstraction from the first commit, and a scheduled CI run in a different timezone |
Quarantine is a holding pattern with a deadline, not a graveyard. Quarantined tests still run, still report, and are visible with an owner and an age on a dashboard. A quarantine that grows month over month means the process is not being enforced, and we escalate that in the monthly review rather than letting it accumulate quietly.
Test data management
- Synthetic generation is the default. Factories and builders produce valid domain objects with sensible defaults and explicit overrides, so a test states only what it cares about.
- Edge cases are seeded deliberately. Zero-value orders, maximum-length strings, non-Latin scripts, right-to-left text, leap days, daylight saving transitions, negative balances, duplicate names. Randomly generated data almost never produces the cases that break systems.
- Property-based testing for calculation-heavy logic. Hypothesis, fast-check or jqwik generate hundreds of inputs against an invariant and shrink any failure to a minimal case. This finds boundary defects hand-written examples miss.
- Production-shaped volumes in the performance environment. A table with a thousand rows and a table with fifty million produce different query plans. A load test against a small dataset proves nothing about production.
- Anonymised subsets where shape genuinely matters. Irreversible transformation with format-preserving masking, consistent pseudonyms so joins still work, and generalisation of anything indirectly identifying.
The compliance implications
- Copied production data stays in scope. Under GDPR and the India DPDP Act 2023, personal data in a test environment is still personal data, with the same lawful basis, retention, breach notification and data subject rights obligations.
- Test environments are weaker. Broader developer access, less monitoring, laxer network controls and frequently longer retention. That is precisely the wrong place for real personal data.
- Masking must be irreversible. A reversible transformation is pseudonymisation, which reduces risk but does not remove data from scope. Only genuinely irreversible anonymisation does.
- Beware indirect identifiers. Removing names is not enough. A postcode, date of birth and gender re-identify most individuals. Generalise or suppress quasi-identifiers, not just the obvious fields.
- PCI DSS. Cardholder data must not be in test environments. Use provider-issued test card numbers.
- HIPAA. Either apply the full de-identification standard, or your test environment is in scope for the Security Rule with all that implies.
Non-functional testing
Performance, accessibility and security testing in the pipeline
Non-functional gates
Performance tests need a realistic traffic mix, a ramp profile matching how load arrives, production-shaped data volumes, and pass criteria written against your SLOs. Accessibility runs as an axe-core gate plus scheduled manual screen reader passes. Security runs as SAST on every pull request, SCA continuously, and DAST against a deployed environment nightly.
Performance and load testing
- Load test. Expected peak concurrency sustained for thirty to sixty minutes. Pass criteria from the SLO, for example p95 under 500 ms and error rate under 0.1 percent.
- Stress test. Ramp beyond peak until something breaks, to find the actual ceiling and confirm the system degrades rather than collapses. Knowing your breaking point is a capacity planning input.
- Spike test. Instant jump to several times baseline, simulating a marketing send or a news mention. Tests autoscaling reaction time, connection pool headroom and queue behaviour.
- Soak test. Moderate load for four to twelve hours. This is the only way to find memory leaks, connection and file handle exhaustion, unbounded cache growth and log volume problems.
- Traffic shaping that reflects reality. Weighted journey mix from production analytics, realistic think time, a cache-hit ratio matching production, and an authenticated-to-anonymous split. One endpoint at maximum rate tells you about that endpoint and nothing else.
- Pass criteria against SLOs, not vibes. Written before the test runs, expressed as thresholds the tool enforces, so the result is a pass or a fail rather than a report someone interprets.
Accessibility testing
- Automated gate. axe-core in component and end-to-end tests. Any new violation of WCAG 2.2 AA rules fails the build. Existing violations are baselined with a remediation plan so the gate can be introduced without stopping delivery.
- Keyboard-only pass. Every interactive element reachable and operable by keyboard, visible focus indicators, no focus traps, logical tab order, modals returning focus on close.
- Screen reader passes. NVDA with Firefox and Chrome on Windows, VoiceOver with Safari on macOS and iOS, TalkBack on Android. Run on primary journeys each release and on any new component.
- Zoom and reflow. 200 percent browser zoom and 400 percent reflow without horizontal scrolling or content loss.
- Contrast and colour independence. Verified in light and dark themes, with status never conveyed by colour alone.
- Forms and errors. Every field programmatically labelled, errors associated with their field and announced, and instructions available before the field rather than only after failure.
- The honest limit. Automation catches roughly a third of WCAG failures. It cannot judge whether alternative text conveys meaning, whether reading order is sensible, or whether a custom component behaves as its role implies. A clean axe report is a floor, not a conformance claim.
Security testing in CI
Security testing belongs in the pipeline where it is cheap, not in a penetration test six weeks before launch where it is expensive. Each of these runs at the point where it gives the fastest useful signal.
- Secret scanning with gitleaks or TruffleHog on every push, plus history scanning on introduction and a documented rotation procedure, because a committed secret is compromised even after the commit is removed
- SAST with Semgrep or CodeQL on every pull request, scoped to changed files so it completes in minutes. Custom rules for your own framework misuse patterns, because generic rules produce generic noise
- SCA with Trivy, Grype or Dependabot on dependencies and container images. Critical findings with an available fix block the build. Findings without a fix are recorded with an expiry date so exceptions cannot silently become permanent
- DAST with OWASP ZAP against an ephemeral deployed environment nightly, authenticated so it reaches the application behind the login rather than only the marketing pages
- Infrastructure as code scanning with Checkov or tfsec on every Terraform plan, catching public buckets, permissive security groups and unencrypted volumes before apply
- Container image policy. Minimal or distroless base images, no root user, pinned digests, and a signed image with an SBOM attached
- Authorisation tests as functional tests. Automated checks that role A cannot read role B data, that object identifiers cannot be enumerated, and that every endpoint enforces authorisation server-side. Broken access control is consistently the most common serious finding, and it is entirely testable
- Findings triaged, not accumulated. Every scanner result gets accepted, fixed or suppressed with a reason and an owner. A dashboard with four hundred unread findings provides no security benefit and considerable audit risk
Honest answer
Release gates, and what test automation cannot do
The limits of automation
Automation proves that things which used to work still work. It cannot tell you a workflow is confusing, that a calculation matches a policy nobody wrote down, or that a feature solves the wrong problem. Those need a human with domain knowledge exploring the system deliberately. Automation is regression protection, not quality assurance.
| Gate | Stage | Threshold | On breach |
|---|---|---|---|
| Secret scan | Every push | Zero new secrets detected | Build fails and the rotation procedure is triggered immediately |
| Unit and integration suite | Every pull request | 100 percent pass, under 10 minutes | Merge blocked. A failure here is a defect, never a flake, because flaky tests are already quarantined |
| Changed-line coverage | Every pull request | Above 80 percent on the diff | Merge blocked pending either tests or a written, reviewed exception |
| Contract verification | Every pull request on a provider | All consumer contracts satisfied | Merge blocked. Deploying a provider change that breaks a known consumer is prevented rather than detected later |
| SAST and SCA | Every pull request | No new critical or high findings with an available fix | Merge blocked. Suppression requires a reason, an owner and an expiry |
| End-to-end journeys | On merge to main | All primary journeys pass on Chromium, plus Firefox and WebKit nightly | Deployment blocked and the release halted for investigation |
| Performance smoke | Nightly | p95 within 20 percent of the last release baseline | Investigated before the next release is cut, not after |
| Full performance suite | Before a major release | All SLO thresholds met under expected peak, with soak completed clean | Release deferred or scope reduced. This is a business decision, made with data |
| Flake rate | Continuous | Below 1 percent across the main-branch suite | Pipeline blocked for new feature merges until remediated. Tests are the product too |
| Exploratory session | Before a major release | Charters completed for new and changed areas, findings triaged | Release proceeds or is deferred on the findings, with the decision recorded |
Every one of these gates is agreed with you before it is switched on, and each has a documented override path requiring a named approver. A gate with no legitimate override route gets bypassed at the infrastructure level during the first genuine emergency, which is far worse than a recorded, deliberate exception.
What automation will not find
- Usability problems. A journey can pass every assertion and still confuse users. No assertion encodes "this makes no sense".
- Wrong requirements. Tests verify the system does what it was built to do. If the specification was wrong, the tests are wrong in exactly the same way and pass confidently.
- Missing features. There is no test for a validation rule nobody thought of. Absence is invisible to automation.
- Visual and layout defects. Overlapping elements, clipped text at unusual widths, broken dark mode. Visual regression tools help and are themselves a significant source of flake.
- Business rule correctness against undocumented policy. If the true rule lives in the head of one person in finance, only a conversation surfaces it.
- Data quality problems at scale. Duplicates, encoding corruption and slow drift in a production dataset are monitoring concerns, not test concerns.
- Novel security logic flaws. Scanners find known patterns. Chained business-logic abuse needs a human adversary, which is why penetration testing remains necessary.
The proper role of manual testing
- Charter-based exploratory sessions. Time-boxed to ninety minutes with a written charter, notes and findings. Structured investigation, not clicking around.
- New feature first pass. A human uses the feature as a user would before any automation is written, which is when the design problems are cheapest to fix.
- Domain review. Someone who understands the business checks that outputs are right, not merely consistent. Automation cannot tell a plausible wrong number from a correct one.
- Accessibility with real assistive technology. The two thirds of WCAG that automation cannot assess.
- Risk-based regression on the paths that would hurt most, where a human eye adds judgement automation lacks.
Answers
QA and test automation questions
What test coverage percentage should we target?
None. A global coverage percentage is a bad target because it is trivially satisfied by tests that execute code without asserting anything meaningful, and it treats a trivial getter as equal in value to a pricing calculation. Measure coverage on changed lines in a pull request, run mutation testing on your highest-risk modules, and track escaped defect rate. Those three change behaviour. A number on a badge does not.
Playwright or Cypress?
Playwright for most new work. It drives Chromium, Firefox and WebKit, runs genuinely parallel across workers, handles multiple tabs, origins and downloads without workarounds, and its trace viewer makes a failed CI run diagnosable. Cypress remains excellent for developer experience and component testing, and if you already have a large healthy Cypress suite, rewriting it is rarely the best use of money. We would not start a new browser suite in Selenium.
How do you deal with flaky tests?
A written process, not tolerance. Every test records a pass and fail history. A test that fails intermittently is quarantined out of the blocking suite within one working day, raised as a defect with an owner and a seven-day fix window, and deleted if not fixed. The hard rule is that aggregate flake rate above one percent blocks the pipeline, because a suite people rerun until it goes green has stopped being a gate.
Can we test with a copy of production data?
Usually not without work. Copying personal data into a lower environment extends your regulatory scope to that environment: under GDPR and the India DPDP Act it remains personal data and inherits the same obligations, and PCI DSS and HIPAA are stricter still. We prefer synthetic generation with deliberately seeded edge cases. Where production shape genuinely matters, we build an irreversibly anonymised subset with the transformation documented.
How much of accessibility can automated testing catch?
Roughly a third of WCAG failures, and that figure is consistent with published research from tool vendors themselves. Automation reliably finds missing alternative text, unlabelled form fields, contrast failures and invalid ARIA. It cannot judge whether alternative text is meaningful, whether reading order makes sense, whether a custom widget is usable by keyboard, or whether an error message tells a screen reader user what to do next. Automated gates plus scheduled manual NVDA and VoiceOver passes.
What does a load test need in order to be meaningful?
A realistic traffic mix rather than one endpoint hammered, a ramp profile that matches how load actually arrives, production-shaped data volumes so the query planner behaves the same way, and pass criteria expressed against your SLOs rather than "it did not fall over". Then four test types: load at expected peak, stress to find the breaking point, spike for sudden surges, and soak over several hours to expose leaks and connection exhaustion.
Does test automation replace manual QA?
No. Automation is regression protection: it confirms that things which used to work still work. It cannot tell you a workflow is confusing, that a message is misleading, or that a feature solves the wrong problem. Exploratory testing, charter-based sessions and domain review by someone who understands the business find a different class of defect. Teams that eliminate manual testing entirely usually rediscover the need after a costly release.
How long before an automation suite pays for itself?
Typically three to six months of active development. A test costs roughly two to three times its writing effort over its life once maintenance is counted, so a suite covering rarely-changing low-risk paths may never pay back. We target the journeys that generate revenue, the calculations that would be expensive to get wrong, and the paths with the worst historical defect record, and we leave stable low-risk areas to cheaper layers.
Start with a two-week QA assessment
Fixed fee. You get a current-state analysis of suite shape, runtime and flake rate, escaped defects mapped to the layer that should have caught them, a target pyramid with named test candidates, a gating proposal with thresholds, and a costed remediation plan. Yours to keep either way.