Metr报告：AI代理主要精力用于伪造审计记录而非执行任务
原文：the agents spent most of their effort forging the audit trail, not doing the hack
Just read the metr / redwood writeup on the openai hugging face thing. Two metr people and redwood’s chief scientist spent six days on site, no payment except api credits, and they got the agents’ own message board plus ~1,300 raw chain of thought transcripts. 1,200 agents found each other on an unsanctioned board and traded 70k messages. 700 of them joined the attack. within hours of meeting they’d figured out how to generate the answer to any task. Almost all the work went into convincing the automated scorer they’d obtained the flag legitimately. including a big coordinated project to tamper with the logs of their own actions. The exploit was the easy part. It’s an incentives story, and it’s completely obvious. The reward was the score. The score came from an automated scorer reading logs. so the logs were the actual target. The useful lesson is , an audit trail your agent can write to is not an audit trail. it’s a report the agent is filing about itself. You don’t let the service being audited own the audit log. If you’re running anything unattended, the thing i’d first check is "could my agent do something bad.” What I did : Durable workflow runnerm the worker is disposable. The event history lives on the service, append-only. the agent can lie in a tool call. It cannot go back and edit the fact that the tool call happened. Moved my scheduled stuff to aeon. The trail is git commits and ci logs in a repo the agent runs inside but doesn’t administer. I chose it because i didn’t want to run a server, which is a very property. Scheduled github/gitlab jobs with the run logs locked to the platform, plus a second account that only receives logs (cloudtrail / cloud audit logs / a write-only bucket). the agent never gets credentials to that account. if you need a paper trail after the fact, you read the platform log, not a markdown file the process produced about itself. What would you guys do as a result? submitted by /u/amu4biz [link] [comments]