Module 15 Quality engineering practice
Hours 223–238, about 6 hours. Reviewed at Gate 4.
Purpose
A suite that nobody trusts is worse than no suite. Module 15 is about keeping automated tests worth having: maintainable code, flaky tests found and fixed at their root cause, and simple flow metrics that show whether the suite is healthy. It also prepares the capstone estimate.
Outcomes
- Learning outcome 10: diagnose flaky tests, and measure suite health with flow metrics.
Depth by track
| Module | Band 3 | Band 4 quality assurance | Band 4 test engineering | Band 5 quality assurance | Band 6 quality assurance | Band 6 test engineering | Band 7 test engineering | Band 7 test management |
|---|---|---|---|---|---|---|---|---|
| Module 15 Quality engineering practice | Read and discuss | Read and discuss | With support | With support | Independent | Independent | Lead or coach | Lead or coach |
Session plan
| # | Session | Duration | Format | Who |
|---|---|---|---|---|
| 1 | Hours 223–230.5: maintainable test code: names, duplication, helpers, data builders | 1 hour | Core | Cohort |
| 2 | Hours 223–230.5: flaky tests are variation: root causes and fixes | 1 hour | Core | Cohort |
| 3 | Hours 223–230.5: flow metrics for testing | 30 minutes | Core | Cohort |
| 4a | Hours 223–230.5: flaky-test exercise, hands on | 1 hour | Breakout | Band 4 test engineering and above |
| 4b | Hours 223–230.5: describe a flaky test with the mentor | 1 hour | Breakout | Band 3, Band 4 quality assurance |
| 5 | Hours 230.5–238: AI assistants: drafting and explaining tests, and reviewing the output critically, including self-healing locators | 30 minutes | Core | Cohort |
| 6 | Hours 230.5–238: awareness: performance, load, and security testing, and who owns them | 30 minutes | Core | Cohort |
| 7 | Hours 230.5–238: estimating test effort for the capstone | 30 minutes | Core | Cohort |
| 8a | Hours 230.5–238: suite-health report | 1 hour | Breakout | Band 5 quality assurance, Band 6 quality assurance, Band 6 test engineering, Band 7 test management |
| 8b | Hours 230.5–238: reviewing test pull requests, led by Band 7 test engineering, then Band 7 test engineering's own suite-health report | 1 hour | Breakout | Band 7 test engineering, with others |
| 8c | Hours 230.5–238: flaky-test follow-up with the mentor | 1 hour | Breakout | Band 3, Band 4 quality assurance, Band 4 test engineering |
Each person's sessions add up to 6 hours: 3.5 hours in hours 223–230.5 and 2.5 hours in hours 230.5–238.
Activities
- Run the flaky-test exercise in
practice-repo/tests/flaky/(see its README), and complete the flaky-test investigation template. - Write the suite-health report for your suites.
- Review a test pull request from someone else for maintainability.
- Ask an AI assistant to draft a test for one of your scenarios. Review it line by line. Does every assertion check what the scenario says? Would a self-healing locator pass on the wrong element?
- Estimate the capstone: list each test, its layer, and its effort.
Evidence
Evidence 15:
| Part | Tracks |
|---|---|
| A flaky-test investigation with root cause and fix | Band 4 test engineering, Band 5 quality assurance, Band 6 quality assurance, Band 6 test engineering, Band 7 test engineering, Band 7 test management. Band 3 and Band 4 quality assurance describe one with the mentor. |
| A suite-health report using flow metrics (time from a defect report to a regression test, quarantined tests, run time, failure causes) | Band 5 quality assurance, Band 6 quality assurance, Band 6 test engineering, Band 7 test engineering, Band 7 test management |
| An effort estimate for the capstone | All tracks |
Capstone estimate
| Test or task | Layer | Hours | Assumptions |
|---|---|---|---|
| Total |
Compare the total with the capstone's hours: about 27.5 hours across hours 230.5–268. If it does not fit, cut scope with the product owner now, not in hours 253–260.5.
Assessment
Evidence 15 is assessed at Gate 4 (hour 268), with Evidence 14 and Evidence 16.
Resources
- What metrics help automatic testing?
- How does Six Sigma lead manual testing into automatic testing?
- How does artificial intelligence help automatic testing?
- The flaky-test exercise:
practice-repo/tests/flaky/README.md - Selenium documentation, waits (most flaky Selenium tests are missing an explicit wait): https://www.selenium.dev/documentation/webdriver/waits/
View this document on GitHub materials/modules/module-15-quality-engineering/index.md