An agent that cannot find an answer does not fall silent. It answers from whatever is adjacent, in the same confident voice it uses when it is right, and the first person to find out otherwise is the customer reading it. Chalky reads your agents, topics, actions, instructions and prompt templates against 30 published checks — what each one is grounded on, what it can do without being asked twice, and whether anything would tell you when it goes wrong.
None of the four below is a bug. Each is the platform doing exactly what it was configured to do, which is why nothing in the org raises a hand.
An action pointed at a corpus that is not enabled, or at an object holding no records, returns nothing. Nothing is not an error. The model continues, answers from whatever else is in context, and is no less fluent for having had no material — so the answer that came from your knowledge base and the answer that came from the model’s own priors are indistinguishable to the person reading them.
It is not a rare shape. On one real org, six actions were grounded on Knowledge — in an org where Knowledge was not switched on.
ForcedLeak, published by Noma Labs in September 2025 and rated CVSS 9.4, submitted instructions through Web-to-Lead. They sat in the record until an ordinary employee interaction brought them into a prompt, and then executed and exfiltrated CRM data. Salesforce patched it by re-securing an expired domain and enforcing an allowlist on the links an agent can emit.
That patch is the point. The platform did not prevent it, and Salesforce’s own documentation describes prompt defence as heuristics that reduce the odds rather than remove them. So the question a scan has to answer is not whether the Trust Layer is on. It is which fields a stranger can write reach a grounded prompt, and what that agent is allowed to do once they are there.
Routing is the planner deciding which topic a request belongs to. With no example utterances it makes that decision on the topic name and scope alone, so requests land in the wrong topic or in none — and the topic somebody built, tested and shipped simply never runs. Nothing reports a topic that is never chosen.
When an action errors, times out or returns empty, the conversation does not halt. The model carries on without the data and produces something plausible. That makes a failed action and a successful one look the same from the outside, which is the whole reason the check reads the instructions rather than the logs: an agent that has been told what to say when a lookup returns nothing behaves differently from one that has not.
On one real Agentforce org, twenty-two deterministic checks ran and ten of them failed. Five custom topics carried no example utterances at all. Six actions were grounded on Knowledge, in an org where Knowledge is not enabled. Not one of twenty-one topic instructions said what the agent should do when an action fails.
Three prompt templates had no active version — every version unpublished — and twenty-five of twenty-five template versions had no output schema, so a generation that comes back short comes back well-formed. Field-completion templates, which look things up and reformat them, were configured on a full reasoning model. Every scorer was one Salesforce ships; nothing tested a topic built in that org.
And three actions could act on the outside world with no confirmation: create a Slack channel, send a message to one, and modify automation metadata. None of it was throwing an error. The agents were answering.
Twelve checks on grounding: six on whether the sources resolve at all, six on whether the records behind them are fit to answer from.
AG-005 · High
An action pointed at a corpus that is not enabled, or an object holding no records. The agent answers from adjacent material and sounds no less certain.
AG-016 · High
Actions such as queryRecords and getRecordFieldsAndValues decide what to read when they run, so there is no field list to review. The honest answer to “what is this agent grounded on?” is everything the running user can see — and that is the finding, not any particular field.
AG-003 · High
A merge field naming a field the org does not have. The agent substitutes silently, so the prompt reaches the model with a hole in it and the model fills the hole.
AG-002 · Medium
Nothing for the planner to match a request against, so the topic is routed to by name alone or not at all.
AG-006 to AG-011 · Low to Medium
Null rates above threshold on a grounded field, degenerate cardinality, free-text dispersion, duplicates in the retrieval set, records too old to be current, and Knowledge articles that are archived or draft while still retrievable. Configuration can be perfect and the data behind it still unable to answer.
Nine checks on the paths through an agent: what happens when something fails, what happens when something runs twice, and what a stranger can reach.
AG-025 · High
Web-form fields, inbound email bodies, case descriptions and chat transcripts grounded into a prompt for an agent that can take actions or emit links. ForcedLeak is the worked example; the check is which fields reach the prompt and what the agent can do once they have.
AG-030 · High
An action that writes, sends, refunds, cancels or deletes, reachable by the planner with no confirmation and no human in the loop. To a planner an irreversible action is just another tool.
AG-031 · High
Planners retry and users repeat themselves. Where the action creates, sends or moves money, the second run is a second real-world effect.
AG-019 · High
A topic whose instructions say nothing about an action erroring, timing out or returning empty. Unhandled failure does not stop the conversation, it changes the answer.
AG-022 · High
An outcome where the do-nothing case and the success case are indistinguishable downstream: no signal, no counter, no field that would look different. It is the failure that reports itself as success, and it is the one nobody raises a ticket about.
AG-018 · Beta
An agent, topic, action or template changed since the version that is live, so the org’s behaviour and its configuration disagree. Read the active version; never infer it from a save that succeeded.
AG-017 · High
A generation invoked once per record where a batched path exists. Metered per call, this is the difference between a line item and a conversation with finance.
AG-024 · Medium
A large model configured for a task that is lookup or formatting. Retrieval earns no reasoning budget; judgement does.
AG-015 · Medium
A topic promising to answer about something its grounding does not cover. Requests route in correctly and are answered from nothing relevant, which is the worst of both.
AG-027 to AG-029 · Low to Medium
Answers buried below the background, so retrieval returns the background. No content saying a thing is unsupported, so the agent constructs a plausible answer from what is adjacent. Articles that never use the words customers use, so they lose retrieval to something worse.
AG-032 · Medium
A topic with no test utterances, or utterances not re-run since it last changed. The agent form of the standard we hold ourselves to: the tests changed in the same edit as the behaviour, or they prove nothing.
The agent, the topic, the action and the value we read, so your admin can open it in Agent Builder and see the same thing we did.
What goes wrong if it is left, written in business terms, then the remediation and the published source the rule came from.
5 of the 30 are read by a model rather than measured, because no pattern settles them. They are labelled as such, and two runs of the same estate can differ on them.
Rules that could not be measured say so and say why. A check we could not run is never reported as a check that passed, and a component your org would not hand over stops the scan rather than shrinking the report.
Priced on the number of agents in your org — the one number you already know, which is why the price can sit here rather than arriving after a scan you have already sat through. Never per seat, never per conversation. The rate falls as the estate grows, and it falls furthest on the subscription. Charities pay half, applied automatically. The scan itself is free: connect a sandbox and the preview comes by email, naming every check that failed.
£50 an agent. 30checks every month, and what changed since the last one: what is new, what regressed, what got fixed.
Starts at £250 rather than £750, and an agent estate does not stand still. Cancel from the link in your monthly email and the billing stops at the end of the month you have paid for — that month still runs. Ending the connection in your own Setup stops the scans and leaves the billing where it is.
£150 an agent. 30 checks against every agent, topic, action, instruction and prompt template in the org. Every finding names the component, the evidence read from your own org, and what to change.
A year of the subscription is £3,000 — more than the report, because it is twelve reads of an estate that keeps changing rather than one.
Charities and nonprofits pay half, applied automatically when the scan finds Nonprofit Cloud or the Nonprofit Success Pack.
10 hours a pass across 5 agents at £95 an hour, twelve times a year. Reviewing what is there — not fixing any of it.
The subscription is £3,000 for the same twelve passes, so it saves £8,400 a year.
The report is £750, so a single pass by hand costs £200 more than buying it.
Defaults are £95 an hour and 2 hours an agent, which is a deliberately low estimate for reading one agent’s topics, actions, instructions and templates against 30 rules. The real org this was measured on averages twenty-four of those components per agent.
Create a minimal integration user in your sandbox, log in once with Salesforce’s own CLI, and paste the connection URL into the connect page. About ten minutes.
Chalky counts the agents in your org, and the topics, actions, instructions and prompt templates each one carries. The agent count is what you are charged on; the rest is what gets checked.
30 checks across four channels: the estate itself, your Knowledge corpus, and the 5 that need judgement. Minutes, not hours.
Every finding with the component named and the evidence beside it. The connection is yours to end from your own Setup whenever you like.
One connection to a sandbox, on demand, and the same connection that reads your flows reads your agents. Every finding names the component, the evidence from your own org, and what to change.
