Testing and Verification
Testing rules cover risk-based validation, doctests, feature matrices, deterministic tests, parser samples, command construction, dependency floors, and regressions.
Checks, examples, doctests, feature matrices, and regressions.
22 rules
Check Important Feature CombinationsTEST-CHECK-IMPORTANT-FEATURE-COMBINATIONSExercise default, disabled-default, all-feature, and important feature-pair builds. Feature flags change APIs and compile paths, so use a risk-based matrix instead of every combo.Check Maintainer Commands in CITEST-CHECK-MAINTAINER-COMMANDS-IN-CIPut documented maintainer commands, or intentionally stronger equivalents, in CI. This keeps local instructions honest while leaving slow or credentialed checks to special jobs.Check MSRV and PlatformsTEST-CHECK-MSRV-AND-PLATFORMSRun checks for declared Rust versions and supported platforms when they are promised. Compatibility claims are public contracts, so skip matrices only when no such claim exists.Choose Validation by RiskTEST-CHOOSE-VALIDATION-BY-RISKMatch the amount and kind of validation to the changed surface and failure cost. Cheap checks come first, but risky or uncertain behavior needs targeted evidence.Cover Async Routing Edge CasesTEST-COVER-ASYNC-ROUTING-EDGE-CASESTest late replies, unrelated input, timeouts, and unmatched async responses. These cases protect request correlation and cleanup without relying on wall-clock luck.Cover Local Logic with Unit TestsTEST-COVER-LOCAL-LOGIC-WITH-UNIT-TESTSUse unit tests for small local logic such as parsing helpers and policy branches. They give fast precise feedback, while boundary behavior still needs higher-level tests.Cover Navigation BoundariesTEST-COVER-NAVIGATION-BOUNDARIESExercise first, last, empty, oversized, and repeated-navigation states in tests. Boundary cases catch cursor and scrolling bugs that polished manual demos often miss.Cover Policy OutcomesTEST-COVER-POLICY-OUTCOMESTest allowed, denied, redacted, fallback, and unsupported policy outcomes. Policy value lives at decision boundaries, so assert caller-visible behavior instead of internals.Cover Public Boundaries with Integration TestsTEST-COVER-PUBLIC-BOUNDARIES-WITH-INTEGRATION-TESTSUse integration tests to prove public module, crate, or adapter boundaries. They catch composition failures unit tests miss, while local logic can stay unit-tested.Cover Public Examples with DoctestsTEST-COVER-PUBLIC-EXAMPLES-WITH-DOCTESTSCompile public documentation examples as doctests when they do not need fragile state. Executable examples catch stale guidance, reserving no-run or ignored examples for real limits.Fuzz Parsers, Formatters, and State MachinesTEST-FUZZ-PARSERS-FORMATTERS-AND-STATE-MACHINESUse fuzzing or property tests for input-heavy parsers, formatters, and state machines. Large input spaces hide failures, so keep long fuzzing outside PR gates unless it is stable.Keep Drift Claims AlignedTEST-KEEP-DRIFT-CLAIMS-ALIGNEDTie support claims, fixtures, examples, docs, and API paths to executable drift checks. Stable claims need evidence, but wording-only changes should not break broad tests.Keep Slow Checks out of PR CITEST-KEEP-SLOW-CHECKS-OUT-OF-PR-CIKeep long fuzzing, exhaustive matrices, and noisy benchmarks out of required PR CI. Fast gates preserve review flow, while heavier checks belong in release or scheduled validation.Match Evidence to SurfaceTEST-MATCH-EVIDENCE-TO-SURFACEValidate the actual changed surface, such as rendered docs, API behavior, or byte output. The narrowest relevant proof is more persuasive than unrelated broad test success.Prefer Deterministic TestsTEST-PREFER-DETERMINISTIC-TESTSPrefer tests controlled by fixed inputs, clocks, ordering, and local state. Deterministic failures are reproducible, while real integration checks should be isolated by cost.Prove Command Construction and DisplayTEST-PROVE-COMMAND-CONSTRUCTION-AND-DISPLAYTest both executable command shape and displayed command text when users rely on them. Quoting, redaction, ordering, and platform formatting can fail even when a local happy path works.Prove Contracts Not TriviaTEST-PROVE-CONTRACTS-NOT-TRIVIAWrite tests around observable contracts instead of private helper trivia. This preserves refactoring freedom unless the detail is itself the promised behavior, such as a stable user-facing output contract.Run Docs as a First-Class GateTEST-RUN-DOCS-AS-FIRST-CLASS-GATETreat documentation checks as real validation for examples, links, commands, and claims. Prose-only edits may need less proof, but API-facing docs should fail before stale guidance ships.Run Fast Format and Lint Gates EarlyTEST-RUN-FAST-FORMAT-AND-LINT-GATES-EARLYRun cheap format and lint gates early in the feedback loop. They remove mechanical failures quickly, but they do not replace validation of risky behavior.Use Realistic Parser SamplesTEST-USE-REALISTIC-PARSER-SAMPLESTest parsers with representative input, malformed cases, and safe degradation examples. Real samples catch compatibility failures, but fixtures should be minimized and scrubbed.Validate Declared Minimum Dependency VersionsTEST-VALIDATE-DECLARED-MINIMUM-DEPENDENCY-VERSIONSCheck that declared minimum dependency versions still build the supported behavior. Lockfile tests can hide newer API usage, so use targeted minimal-version checks when needed.Write Regression Tests for Bug FixesTEST-WRITE-REGRESSION-TESTS-FOR-BUG-FIXESAdd a test that fails before the bug fix and protects the repaired contract. Skip only when reproduction is impractical, and then record the closest trustworthy validation.