A UCSC student team working with Scale AI has found a soft spot in agentic AI: ambiguity. At New Tech on November 5, they'll show how attackers exploit it and how to defend against it.
Tell a human assistant to "clean up the shared drive" and they'll probably ask what you mean. Tell an AI agent the same thing and it has to decide on its own. Archive or delete? Everything, or just last year's files? Whose files count?
That gap between what you said and what you meant is where a team of UC Santa Cruz students is hunting for trouble.
Working on a project for Scale AI, the San Francisco company known for the data and evaluations used to train and test frontier models, the team is studying how secure AI agents remain when their instructions are fuzzy. Their working conclusion: ambiguity isn't just a usability problem. It's an attack surface.
The New Front Door
Agents are no longer chatbots. They read email, browse the web, write code, move files, and call other tools on our behalf. When a task is unclear, a well-built agent does the sensible thing and looks for clarification. It might ask the user, check a document, or consult another model.
That is exactly the moment an attacker wants. If someone controls the answer, by planting instructions in a web page, a file, or a tool response, the agent may resolve its uncertainty in the attacker's favor. No password cracked. No firewall breached. The agent simply follows what it believes is helpful guidance.
It's an awkward kind of threat. Traditional defenses look for malicious code. Here, the malicious input can look like an ordinary, polite answer to an ordinary, polite question.
Break It, Then Fix It
The team's approach is classic security research: demonstrate the vulnerability, then defend against it. They are building scenarios that show how frontier agentic systems can be steered off course through ambiguous task definitions, then testing safeguards that stop those exploits before they turn into real breaches.
The stakes rise as companies hand agents more autonomy and more access. An agent that can be talked into the wrong action is only as safe as the least trustworthy source it consults when it's unsure.
Why It Matters Here
Santa Cruz has a long habit of producing people who ask uncomfortable questions about technology. This team is asking them at the frontier, for a company that helps shape how AI gets tested. That's a pipeline worth noticing: UCSC talent, real industry problems, and work that could make the tools we'll all use a little harder to fool.
See Them Live
The team presents at Santa Cruz Works New Tech #198 on Thursday, November 5, at 6 pm at River Row, alongside ElevationXR, Farmblox, g-SPACE, and Printed Living. Bring your hardest questions. Get tickets here.
Related Articles
New Tech #198: We're Lighting the 20th-Anniversary Fireworks Early — The full November 5 lineup, from orbit to orchard to agent security.
From the Chancellor: Harnessing the Power of AI—While Avoiding its Perils — UCSC's chancellor on AI's promise in higher education and the risks that come with it.
Kai Lenny Just Rode a 200-Foot Wave. Mathematically Speaking. — What happens when AI output arrives faster than humans can check it.
The AI Money Machine: Circle Jerking? — Following the dollars that fuel the AI boom, and asking how long they last.
The Sphere Is Spectacular. Now Make It Fit in a Parking Lot. — Fellow November 5 presenter ElevationXR brings Sphere-style immersion to modular domes.