Verification

Evidence for a claim: checks, inspection, measurements, or comparison with a trusted source.

103 items in this guidance cluster.

Tagged guidance

Coding agentsCoding-agent guidance covers task setup, repo-owned context, tools, workspaces, verification, handoff proof, feedback loops, and maintainer review flow. It treats agents as workers in a system whose output should be reviewable, durable, and easy to integrate.Software change preferencesSoftware-change guidance covers broad preferences for change size, structure, validation, review artifacts, commit history, and handoff. It favors small coherent changes, explicit tradeoffs, and evidence matched to the changed surface.

Budget for feedback loopsReserve enough time and tokens for checks, failure inspection, and handoff proof. Planning for the feedback loop keeps validation from being squeezed out after the first edit.Encode nonfunctional requirementsPut invisible constraints such as security, accessibility, latency, and compatibility near the task. Encoding them upfront makes those requirements part of the implementation target.Make bad output hardTurn recurring bad agent output into scripts, templates, lint, or checks that fail fast. Mechanical enforcement lowers review cost more reliably than repeated prompt reminders.Prefer build-preserving editsMake multi-step edits in slices that keep compilation or tests close to green when the route allows it. Build-preserving work keeps failures close to the edit that caused them.Produce review packetsHand off agent work with purpose, changed files, evidence, skipped checks, risks, and follow-ups. A review packet lets maintainers inspect the output without replaying the session.Prove security impactSeparate security hypotheses from proof of reachability, exploitability, assets, and user impact. This keeps prioritization and mitigation tied to demonstrated risk.Report proof in handoffsReplace confidence language with the exact checks, inspection, screenshots, and skipped validation behind a handoff. Proof lets reviewers decide what to trust and what remains risky.Verify risky changes with canariesUse staged rollout, shadowing, dry runs, or partial publication for changes that can fail only under real conditions. Canaries reduce blast radius while evidence accumulates.

Define compaction invariantsState the budget and cut-point rules before deleting, summarizing, or moving context. Explicit invariants make compaction reviewable and reduce the risk that later work treats lossy or nondeterministic summaries as authoritative.Stage generated behaviorValidate generated, reloadable, or plugin-provided behavior in a staging path before promoting it over known-good behavior. Staging adds recovery cost only where runtime-loaded or generated artifacts can fail after deployment.Use conservative terminal defaultsChoose first-run terminal behavior that works with limited color, small viewports, ordinary keyboard input, and varied fonts or accessibility settings. Advanced styling can remain opt-in or capability-detected after the baseline is readable and usable.

Minimal but completeKeep the diff as small as the purpose allows while including the evidence that purpose needs. Reviewers should see one coherent unit they can understand, validate, and revert.Pin behavior with early testsAdd characterization tests before changing messy behavior when the existing contract is unclear. The baseline separates intentional behavior changes from accidental drift.Sync generated artifactsUpdate checked-in generated outputs when their source inputs change and they are part of review. Keeping them aligned prevents consumers from seeing stale artifacts.

Avoid unearned praiseReplace vague ranking words with observable behavior, evidence, or tradeoffs. Use evaluative claims only when the basis is explicit enough for readers to verify.Build docs like users read themReview rendered Rust documentation, links, cfg behavior, and examples in the modes users see. Match validation depth to the risk of feature, metadata, or public example changes.Compare libraries accuratelyCheck current upstream behavior before comparing nearby libraries or crates. Charitable, source-backed comparisons preserve trust while still helping users choose.Distinguish example rolesName whether an example is focused, canonical, survey, integration, or showcase work. That role controls how complete, copyable, and validated the example must be.Mark noncompiling examples honestlyLabel examples that are sketches, partial snippets, or intentionally not run. Honest labels keep readers from treating illustrative code as a supported copy-paste contract.Prove real use with examplesUse examples that show realistic ownership, errors, lifecycle, configuration, or integration shape. Keep them focused, but make them strong enough to prove the contract.Review correctness and risk firstLead documentation review with false contracts, drift, operability, and unsupported claims before style polish. This separates blocking risk from wording cleanup.Show side effects in live examplesShow setup, visible effects, and cleanup when examples touch live resources or persistent state. Gate costly or externally visible actions so examples stay honest and safe to adapt.State current behavior, not aspirationDescribe what the system does now and label limitations or future plans separately. Otherwise roadmap language becomes a false contract for readers and agents.Treat docs as contractsTreat supported behavior, commands, errors, and examples in docs as promises readers may rely on. Separate normative claims from background, examples, and future plans.Use source links as supportUse references to verify, frame, or contrast local guidance instead of supplying the wording. This keeps the repo voice original while still making claims checkable.Verify commands, paths, and linksCheck commands, file paths, and linked references because readers treat them as executable instructions. Note assumptions when credentials, services, or platforms prevent a full run.

Avoid single-run performance conclusionsDo not land performance conclusions from one short benchmark run. Repeat and contextualize timing evidence because warmup, scheduling, cache state, and background load can make a single result non-reproducible.Measure goal, change, and comparisonState the performance goal, baseline measurement, implementation change, and comparison result together. Those pieces let reviewers judge whether the patch improved the relevant workload enough to justify its tradeoffs.Optimize measured hotspotsOptimize code that measurement shows is on the important runtime path. This keeps review attention on changes whose user-visible impact justifies altering the code shape.Record benchmark provenanceRecord the commands, inputs, tool versions, build profile, and runner conditions behind benchmark numbers. Provenance makes future comparisons meaningful and helps separate real changes from environment drift.Run correctness before timingRun relevant correctness checks before interpreting performance timing. Fast code that changes behavior invalidates the benchmark claim and wastes review effort.Run timing benchmarks sequentiallySerialize timing-sensitive benchmarks when their numbers will be used as evidence. Concurrent runs compete for shared resources and can make the comparison describe the runner more than the code.

Classify prototype reuseClassify whether a rebuild is reusing behavior, evidence, replaceable internal shape, or load-bearing boundaries. That separation helps preserve proven compatibility while discarding prototype scaffolding.Explain PR problem model and proofUse PR descriptions to explain the problem, mental model, tradeoffs, validation, and documentation impact. Reviewers can then evaluate the diff against stated intent instead of reverse-engineering it.Label speculation as inferred or unknownMark review claims as observed, inferred, assumed, or unknown when the evidence level matters. Clear labels keep guesses from hardening into false project knowledge.

Add benchmarks for performance claimsUse benchmarks when Rust changes rely on speed, allocation, or hot-path claims. The evidence makes performance tradeoffs reviewable instead of relying on intuition.Keep CI high signalKeep required Rust checks strict, fast, deterministic, and actionable. Slow or flaky ritual trains maintainers to ignore failures while real drift accumulates.Keep compatible updates in lockfileLet lockfiles record newer compatible dependency versions when the manifest floor has not changed. This tests fresh releases without narrowing downstream compatibility.Release only after artifact validationValidate the actual release artifact before publishing instead of trusting the working tree. This catches missing files, stale generated content, and packaging mistakes while the release can still be fixed.Run feature gated validationExercise the feature combinations touched by a Rust change so gated code, docs, and integrations actually build. Choose representative combinations when exhaustive feature matrices would be too expensive.Use honest minimum dependenciesSet dependency requirements to the lowest compatible versions the crate actually supports. Raise minimums only for required APIs, fixes, features, security needs, or MSRV interactions.Validate package contents before releaseInspect and build the package users will receive, not just the repository checkout. This catches missing assets, accidental inclusions, README drift, and include/exclude mistakes before publication.Validate Rust docs as codeTreat Rust documentation examples, links, feature assumptions, and generated README content as code that must be checked. Use docs builds, doctests, feature-gated checks, and Markdown lint according to the changed surface.Validate semver breaks against external useCheck semver-breaking changes against real examples, dependents, or migration paths before treating an API cleanup as cheap. External evidence informs the cost even when security, soundness, or design repair still justify the break.Working Rust code not enoughTreat compilation as necessary but insufficient evidence for long-lived Rust code. Review API shape, docs, errors, tests, features, dependencies, and module organization because users and maintainers inherit those choices.

Prefer primary stable sourcesAnchor durable guidance in sources readers can inspect, compare, and challenge. Use links to clarify judgment, not to decorate rules or depend on private memory.

Check important feature combinationsExercise default, disabled-default, all-feature, and important feature-pair builds. Feature flags change APIs and compile paths, so use a risk-based matrix instead of every combo.Check maintainer commands in CIPut documented maintainer commands, or intentionally stronger equivalents, in CI. This keeps local instructions honest while leaving slow or credentialed checks to special jobs.Check MSRV and platformsRun checks for declared Rust versions and supported platforms when they are promised. Compatibility claims are public contracts, so skip matrices only when no such claim exists.Choose validation by riskMatch the amount and kind of validation to the changed surface and failure cost. Cheap checks come first, but risky or uncertain behavior needs targeted evidence.Cover async routing edge casesTest late replies, unrelated input, timeouts, and unmatched async responses. These cases protect request correlation and cleanup without relying on wall-clock luck.Cover local logic with unit testsUse unit tests for small local logic such as parsing helpers and policy branches. They give fast precise feedback, while boundary behavior still needs higher-level tests.Cover navigation boundariesExercise first, last, empty, oversized, and repeated-navigation states in tests. Boundary cases catch cursor and scrolling bugs that polished manual demos often miss.Cover policy outcomesTest allowed, denied, redacted, fallback, and unsupported policy outcomes. Policy value lives at decision boundaries, so assert caller-visible behavior instead of internals.Cover public boundaries with integration testsUse integration tests to prove public module, crate, or adapter boundaries. They catch composition failures unit tests miss, while local logic can stay unit-tested.Cover public examples with doctestsCompile public documentation examples as doctests when they do not need fragile state. Executable examples catch stale guidance, reserving no-run or ignored examples for real limits.Keep drift claims alignedTie support claims, fixtures, examples, docs, and API paths to executable drift checks. Stable claims need evidence, but wording-only changes should not break broad tests.Match evidence to surfaceValidate the actual changed surface, such as rendered docs, API behavior, or byte output. The narrowest relevant proof is more persuasive than unrelated broad test success.Prefer deterministic testsPrefer tests controlled by fixed inputs, clocks, ordering, and local state. Deterministic failures are reproducible, while real integration checks should be isolated by cost.Prove command construction and displayTest both executable command shape and displayed command text when users rely on them. Quoting, redaction, ordering, and platform formatting can fail even when a local happy path works.Prove contracts not triviaWrite tests around observable contracts instead of private helper trivia. This preserves refactoring freedom unless the detail is itself the promised behavior, such as a stable user-facing output contract.Validate declared minimum dependency versionsCheck that declared minimum dependency versions still build the supported behavior. Lockfile tests can hide newer API usage, so use targeted minimal-version checks when needed.Write regression tests for bug fixesAdd a test that fails before the bug fix and protects the repaired contract. Skip only when reproduction is impractical, and then record the closest trustworthy validation.

Confirm broad JJ operationsConfirm jj operations that abandon, rebase, squash, split, restore, publish, or go broad. Confirmation protects unrelated work when a command can reshape history or public state.Confirm GitHub remote topologyInspect origin, upstream, push remote, PR base, and PR head before publication. Fork and owned-repo layouts differ, so defaults can push or target the wrong repository.Create an operation log point before reshapingCreate a recent jj operation boundary before risky stack reshaping. Recovery is easier when there is a known point before rebases, splits, squashes, or repairs.Dry run surprising publicationUse dry-run for ambiguous or surprising remote publication, not every routine push. Extra verification pays off when remotes, bookmarks, force-like updates, or PR bases are unclear.Inspect state before mutatingInspect working-copy, stack, bookmark, conflict, and unowned state before mutation. Current state keeps edits scoped and avoids rewriting work the agent did not inspect.Name exact JJ mutation targetsSpecify revisions, filesets, bookmarks, and destinations for mutating jj commands. Explicit targets make intent reviewable and prevent defaults from acting on surprising state.Recover with operation logUse jj operation-log recovery before destructive cleanup habits. The operation log preserves recoverable graph states, but inspect the target before restoring.Stop repeated JJ retries and localize stateStop repeating a failing jj command after a transient retry and inspect relevant state. Diagnosis beats command spam when locks, sparse paths, bookmarks, or remotes disagree.Use evolog and operation logUse jj evolog for a changes evolution and jj op log for repository operations. Picking the right log distinguishes rewrite history from workspace, bookmark, and import events.Use ignore working copy carefullyUse --ignore-working-copy only for understood lock-safe inspection or metadata work. It may read stale file state, so do not use it to bypass normal synchronization before edits.

Budget tokens for feedback loopsLong-running agent work can exhaust context before validation, review, or handoff happens. Reserve space for feedback loops by summarizing durable state and avoiding unnecessary transcript growth.Characterize then fixBefore fixing incorrect behavior, add a passing test that records the observed result and labels it as known incorrect. Then fix the implementation, update that same expectation to the intended contract, and rewrite the rationale as durable regression context.Close the agent loopAgent work is incomplete when it stops at edits without checking the result or explaining residual risk. Finish the loop by validating, reporting evidence, and making the next human decision clear.Define good before judgment-heavy workWork that depends on taste or judgment drifts when success is not defined up front. Establish acceptance criteria, examples, and review dimensions before spending effort on subjective iteration.Encode nonfunctional requirementsPerformance, security, accessibility, privacy, and reliability expectations are easy to lose when they live only in prose. Encode important requirements in tests, budgets, schemas, automation, or review gates where the system can keep enforcing them.Label doc claims by evidenceDocumentation loses trust when speculation, observed behavior, policy, and verified facts look the same. Label claims by their evidence level so readers know what can be relied on and what still needs confirmation.Make bad output mechanically hardRepeated review failures usually belong in a tool, type, schema, template, lint, or checklist instead of another reminder. Encode stable rejection criteria early, but keep escape hatches when legitimate exceptions are common.Practice agent workoutsHigh-risk or repeated agent workflows are unreliable when first exercised during real work. Build safe practice tasks with explicit win conditions so the workflow, skill, or runbook can improve before pressure arrives.Produce review packetsReviewers lose time when a handoff contains only a diff and no acceptance evidence. Package the goal, validation, risks, and proof artifact in proportion to the review burden.Prove security impactSecurity reports create noise when they imply exploitability without showing reachability or consequence. Prove the attacker path and impact, or clearly label the finding as an unproven hypothesis.Remediate doc driftDocumentation drift misleads readers when behavior, workflow, or project shape has moved on. Fix the source of truth and nearby references together, choosing whether the code or the doc represents the intended contract.Report verification honestlyHandoffs become risky when verification is overstated, omitted, or blurred with assumptions. Report exactly what ran, what passed or failed, what was not checked, and what risk remains.Review proof not just codeCode review is weaker when it inspects the diff without checking whether the claimed behavior was proven. Ask for the smallest trustworthy evidence that the change works and that the likely failure mode was covered.Smallest trustworthy verificationVerification is weak when it is either skipped or broad without covering the likely failure. Run the cheapest check that could catch the risk introduced by the change, broadening only when shared contracts or user-visible behavior require it.Test observable behaviorTests that pin private structure can block useful refactors while missing broken user outcomes. Test observable outputs, errors, side effects, state, events, and boundaries, reserving private tests for dense rules that are otherwise hard to reach.Verify with canaries before cutoverAutomation is risky when production cutover is the first real proof. Use a canary path, rollback plan, and human approval for operations with remote access, credentials, production data, or long-lived service impact.Write PR narrativeA correct diff can still waste reviewer attention when it lacks purpose, risk, review order, and verification. Write the PR narrative as a guide to the decision the reviewer needs to make, scaled to the burden of the change.

Change shape controls review costChange boundaries determine how much intent, risk, validation, and revert cost a reviewer must reconstruct. Use the smallest coherent unit that preserves trust without splitting one decision into noise.Docs are contractsDocumentation near code is part of the supported behavior surface. Keep prose, examples, and implementation aligned so humans and agents can rely on the same contract.Measure before optimizingPerformance work needs a workload, baseline, goal, change, and comparison. Evidence keeps optimization from becoming readability, dependency, or API churn without a proven payoff.Tests should explain failuresA failing test should point at the broken contract, not merely report that a predicate was false. Choose assertion shapes that expose the relevant value, variant, order, or case name.

Rust API and release checksCargo release, semver, feature, packaging, and minimum-version checks protect downstream users from accidental compatibility changes. Use these checks when public API, crate metadata, or release artifacts are part of the changed surface.Rust docs validationDoctests, Rustdoc warnings, rendered docs, and docs.rs-style builds prove documentation surfaces the way users will see them. Use this mechanism when examples, public API docs, feature docs, or release-facing documentation changes.Rust lints and formattingRust and Markdown formatting plus targeted project lints encode stable mechanical preferences. The mechanism removes repeat style debate from review while leaving design judgment to humans and higher-level guidance.Rust tooling profileThe broad Rust tooling profile gathers lints, docs, release, CI, performance, and agent workflow checks in one place. Use it as a menu of mechanical supports, not as a substitute for project-specific risk judgment.Testing and benchmarkingTest, feature-matrix, fuzzing, property, and benchmark tools should be chosen by risk and feedback cost. This mechanism helps match validation depth to the behavior, integration surface, or performance claim under review.

Core agent instructionsBroad baseline instructions for coding agents working in repositories that adopt these development preferences. The snippet keeps AGENTS.md short while routing agents to canonical guides, principles, mechanisms, and the reviewed rule pack.Rust agent instructionsCompact Rust maintainability instructions for agents working on APIs, Rustdoc, tests, dependencies, and reviewable implementation changes. The snippet routes agents toward reader locality, explicit boundaries, documented contracts, validation evidence, and pragmatic performance work.