Skip to content

Require verified outcomes from Repo Assist - #416

Open
dsyme wants to merge 1 commit into
mainfrom
fix/repo-assist-verified-outcomes
Open

dsyme wants to merge 1 commit into
mainfrom
fix/repo-assist-verified-outcomes

Conversation

@dsyme

@dsyme dsyme commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

Summary

The fortnight assessment found successful runs skipping required investigations, losing blocker memory during schema recovery, and describing queued publication as completed. Make those outcomes harder to hide behind a green run.

  • Gate normal-mode noops against the actual task selection and recorded successful shell commands, with concrete per-task file/API evidence. Missing required work must be reported incomplete.
  • Require monthly reporting for substantive public work, including labels; reject mutation-plus-noop batches and reporting-only bookkeeping used to bypass assessment.
  • Expose the exact memory schema and useful field-level errors; require sequential, backed-up edits and validation without clearing useful state.
  • Use the pre-authenticated read-only gh proxy, excluding bookkeeping issues from task weights.
  • Clarify information-only publication intent, fresh human feedback, actual test evidence, complete PR-head delta/bundle preflight, and queued-versus-confirmed publication.
  • Document behavior and add regression scenarios exercising the executable guards.

The gate verifies recorded assessment coverage, not the semantic quality of every conclusion or a requirement to manufacture PRs. Explicit commands and intentional stop requests remain exempt from normal-mode reporting/assessment. Existing scheduled PR capacity protections remain in place.

Validation

  • PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests -p 'test_repo_assist_*.py': 55 tests pass.
  • gh aw compile workflows/repo-assist.md --strict --no-check-update: succeeds with the memory-validator review warning described below.
  • git diff --check: passes.
  • The generated shared sample lock remains intentionally gitignored; this PR does not change deployed workflow locks or enable/dispatch any workflows.
  • No live agent trial was dispatched. Reassess real run compliance after merge and deployment.

Security/update review

  • No new secrets, write permissions, credential scopes, or redirect destinations.
  • The new task-selection artifact uses the existing pinned upload/download-artifact actions and contains deterministic task-selection data. The outcome gate parses artifacts as data; it does not execute agent-authored code.
  • The gh proxy remains read-only, replacing the broad local MCP tool inventory and its unauthenticated-CLI/fork-enrichment failure paths. All GitHub writes still use safe outputs.
  • Protected-file handling for existing-PR updates is tightened from allowed to request-review. The explicit 4096 KiB cap matches the existing default; no publication protection is relaxed.
  • The compiler flags the changed memory validation script for review. Those edits improve error messages only; accepted schema and validator capabilities are unchanged. Reviewed without --approve.

Repository-specific Fantomas startup and calendar-month scheduling are fixed separately in fsprojects/fantomas#3574. Other repository schedules and intentional disablements are unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant