Open to Debate: should we trust AI agents with our information?

Debating whether AI agents can be trusted with personal data

Masters of Scale's first debate asks whether AI agents should be trusted with personal information. Whittaker, of Signal, says agents are other people's software running with your permissions, so prompt injection and platform control make them a hazard. Chilson, of the Abundance Institute, says agents are the first tools that let people use their own data, and that engineering and liability will fix them. Prompt injection is the crux, unsolved to one side and a 1990s virus problem to the other.

Takeaways

• Prompt injection is unsolved and already exploited. Anthropic's Claude browser agent failed to resist prompt injection 11% of the time. Whittaker points to a zero-click Copilot data theft, a Perplexity browser hijacked by a Reddit comment, a Google Calendar attack that turned on a boiler, and UK agency doubt it is solvable.
• Agents demand sweeping permissions and obey text as commands. Booking a party needs contacts, a credit card, browser control, the calendar and Signal messages. Whittaker says agents want root-level permissions, act without explicit consent and treat natural language as commands, which is what makes hijacking possible.
• Meta's Muse leaked an address and lost Chase. Whittaker says Muse gave out a user's home address in a Facebook Marketplace sale and logged contact lists to build dossiers. Chase blocked Muse because it cannot tell a real user from a prompt-injected agent at the keyboard.
• Whittaker says agents shift power to big platforms. She argues agents pull agency back to AI companies and operating system vendors that grab data from independent developers and shut down APIs. Agentic integrations become the internet's front end, hollowing out application-layer privacy and reversing the open internet's promise.
• Chilson says agents flip the internet's data bargain. For twenty years companies pooled data users could not touch and sold ads against it. Agents let individuals use their own data instead. Chilson opted out of five US data brokers with a six-word Muse prompt and says agents make Tor convenient.
• Whittaker says duty of care sits with companies. She says trillion-dollar companies carry the responsibility, since users click through terms of service and install something closer to malware. Her assistant Emma is accountable under a labour contract, and agents are not.

How Chilson would make agents safe

• Tiered permissions, so an agent's access matches the task, with legal liability and investor scrutiny pushing companies to fix harms.
• Defence in depth, the layered approach that tamed 1990s computer viruses, using front-end filters and back-end permissions.