Verifying Product Outcomes

Resources

• The Testing Trophy — Kent C. Dodds distinguishes static, unit, integration, and end-to-end testing and explains their roles in confidence. Helps select evidence for an observable product outcome rather than equating a large test count with readiness. https://kentcdodds.com/blog/the-testing-trophy-and-testing-classifications

• The Practical Test Pyramid — Ham Vocke connects test layers, acceptance tests, and user-facing behavior with practical examples. Useful for mapping the definition of done and individual requirements to verification evidence, while retaining exploratory review for gaps that automated checks miss. https://martinfowler.com/articles/practical-test-pyramid.html

• What to look for in a code review — Google’s engineering practices cover design, functionality, complexity, tests, naming, style, and documentation. Provides a review rubric for architecture and maintainability that can guide agent reviewers without treating their approval as a substitute for accountable human judgment. https://google.github.io/eng-practices/review/reviewer/looking-for.html

• Playwright best practices — Official recommendations emphasize testing user-visible behavior, isolation, resilient locators, and web-first assertions. A practical basis for reproducible browser regression evidence against acceptance criteria, rather than tests that merely mirror implementation details. https://playwright.dev/docs/best-practices

• Accessibility testing with Playwright — Documents integrating axe scans into browser tests and explains the limits of automated accessibility detection. Useful for a layered quality gate that combines deterministic checks with manual keyboard and assistive-technology review. https://playwright.dev/docs/accessibility-testing

• WCAG quick reference — W3C’s filterable reference connects accessibility success criteria with techniques and failures. Supplies an external standard for reviewing changed UX and defining accessibility blockers beyond whether an automated scanner passed. https://www.w3.org/WAI/WCAG22/quickref/

• Application Security Verification Standard — OWASP’s ASVS supplies structured application-security requirements. Use it to select risk-appropriate security acceptance checks and demand evidence for authentication, authorization, data handling, and other affected controls rather than a generic claim that code is secure. https://owasp.org/www-project-application-security-verification-standard/

• Web Vitals — Google’s overview defines user-centered performance metrics and explains field and lab measurement. Helps evaluate the performance part of a system diff using explicit thresholds and representative conditions instead of a subjective impression that the interface feels fast. https://web.dev/articles/vitals

• fast-check — The project’s introduction explains property-based testing, generated inputs, and shrinking failures. Extends verification beyond hand-picked happy paths by testing invariants and producing smaller reproducible counterexamples for unexpected inputs. https://fast-check.dev/docs/introduction/

• Demystifying evals for AI agents — Anthropic distinguishes tasks, trials, graders, traces, and outcomes, including code-based, model-based, and human grading. Useful for evaluating delegated work and calibrating confidence: reviewer prose or a successful tool call is not proof that the requested end state was achieved. https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents

• Evolutionary Database Design — Pramod Sadalage and Martin Fowler describe versioned database changes, migration practices, and continuous integration of schema and data changes. Broadens the system diff beyond code and UX to compatibility, migration evidence, deployment ordering, and recovery consequences. https://martinfowler.com/articles/evodb.html

• Canarying releases — Google’s SRE Workbook explains controlled exposure and comparing candidate behavior with a baseline. Provides a concrete framework for release signals, confidence thresholds, blockers, and rollback decisions when pre-release tests cannot reproduce all production conditions. https://sre.google/workbook/canarying-releases/

• GitHub deployment environments — Official documentation covers deployment protection rules, required reviewers, and environment restrictions. A concrete implementation of the final quality gate: require completion evidence and human approval where risk demands it rather than allowing an agent’s finished message to authorize release. https://docs.github.com/en/actions/reference/workflows-and-actions/deployments-and-environments