An agent that cannot find an answer does not fall silent. It answers from whatever is adjacent, in the same confident voice it uses when it is right, and the first person to find out otherwise is the customer reading it. Chalky reads your agents, topics, actions, instructions and prompt templates against thirty published checks — what each one is grounded on, what it can do without being asked twice, and whether anything would tell you when it goes wrong.
None of the four below is a bug. Each is the platform doing exactly what it was configured to do, which is why nothing in the org raises a hand.
An action pointed at a corpus that is not enabled, or at an object holding no records, returns nothing. Nothing is not an error. The model continues, answers from whatever else is in context, and is no less fluent for having had no material — so the answer that came from your knowledge base and the answer that came from the model’s own priors are indistinguishable to the person reading them.
It is not a rare shape. On one real org, six actions were grounded on Knowledge — in an org where Knowledge was not switched on.
ForcedLeak, published by Noma Labs in September 2025 and rated CVSS 9.4, submitted instructions through Web-to-Lead. They sat in the record until an ordinary employee interaction brought them into a prompt, and then executed and exfiltrated CRM data. Salesforce patched it by re-securing an expired domain and enforcing an allowlist on the links an agent can emit.
That patch is the point. The platform did not prevent it, and Salesforce’s own documentation describes prompt defence as heuristics that reduce the odds rather than remove them. So the question a scan has to answer is not whether the Trust Layer is on. It is which fields a stranger can write reach a grounded prompt, and what that agent is allowed to do once they are there.
Routing is the planner deciding which topic a request belongs to. With no example utterances it makes that decision on the topic name and scope alone, so requests land in the wrong topic or in none — and the topic somebody built, tested and shipped simply never runs. Nothing reports a topic that is never chosen.
When an action errors, times out or returns empty, the conversation does not halt. The model carries on without the data and produces something plausible. That makes a failed action and a successful one look the same from the outside, which is the whole reason the check reads the instructions rather than the logs: an agent that has been told what to say when a lookup returns nothing behaves differently from one that has not.
On one real Agentforce org, twenty-two deterministic checks ran and ten of them failed. Five custom topics carried no example utterances at all. Six actions were grounded on Knowledge, in an org where Knowledge is not enabled. Not one of twenty-one topic instructions said what the agent should do when an action fails.
Three prompt templates had no active version — every version unpublished — and twenty-five of twenty-five template versions had no output schema, so a generation that comes back short comes back well-formed. Field-completion templates, which look things up and reformat them, were configured on a full reasoning model. Every scorer was one Salesforce ships; nothing tested a topic built in that org.
And three actions could act on the outside world with no confirmation: create a Slack channel, send a message to one, and modify automation metadata. None of it was throwing an error. The agents were answering.
Twelve checks on grounding: six on whether the sources resolve at all, six on whether the records behind them are fit to answer from.
AG-005 · High
An action pointed at a corpus that is not enabled, or an object holding no records. The agent answers from adjacent material and sounds no less certain.
AG-016 · High
Actions such as queryRecords and getRecordFieldsAndValues decide what to read when they run, so there is no field list to review. The honest answer to “what is this agent grounded on?” is everything the running user can see — and that is the finding, not any particular field.
AG-003 · High
A merge field naming a field the org does not have. The agent substitutes silently, so the prompt reaches the model with a hole in it and the model fills the hole.
AG-002 · Medium
Nothing for the planner to match a request against, so the topic is routed to by name alone or not at all.
AG-006 to AG-011 · Low to Medium
Null rates above threshold on a grounded field, degenerate cardinality, free-text dispersion, duplicates in the retrieval set, records too old to be current, and Knowledge articles that are archived or draft while still retrievable. Configuration can be perfect and the data behind it still unable to answer.
Nine checks on the paths through an agent: what happens when something fails, what happens when something runs twice, and what a stranger can reach.
AG-025 · High
Web-form fields, inbound email bodies, case descriptions and chat transcripts grounded into a prompt for an agent that can take actions or emit links. ForcedLeak is the worked example; the check is which fields reach the prompt and what the agent can do once they have.
AG-030 · High
An action that writes, sends, refunds, cancels or deletes, reachable by the planner with no confirmation and no human in the loop. To a planner an irreversible action is just another tool.
AG-031 · High
Planners retry and users repeat themselves. Where the action creates, sends or moves money, the second run is a second real-world effect.
AG-019 · High
A topic whose instructions say nothing about an action erroring, timing out or returning empty. Unhandled failure does not stop the conversation, it changes the answer.
AG-022 · High
An outcome where the do-nothing case and the success case are indistinguishable downstream: no signal, no counter, no field that would look different. It is the failure that reports itself as success, and it is the one nobody raises a ticket about.
AG-018 · Beta
An agent, topic, action or template changed since the version that is live, so the org’s behaviour and its configuration disagree. Read the active version; never infer it from a save that succeeded.
AG-017 · High
A generation invoked once per record where a batched path exists. Metered per call, this is the difference between a line item and a conversation with finance.
AG-024 · Medium
A large model configured for a task that is lookup or formatting. Retrieval earns no reasoning budget; judgement does.
AG-015 · Medium
A topic promising to answer about something its grounding does not cover. Requests route in correctly and are answered from nothing relevant, which is the worst of both.
AG-027 to AG-029 · Low to Medium
Answers buried below the background, so retrieval returns the background. No content saying a thing is unsupported, so the agent constructs a plausible answer from what is adjacent. Articles that never use the words customers use, so they lose retrieval to something worse.
AG-032 · Medium
A topic with no test utterances, or utterances not re-run since it last changed. The agent form of the standard we hold ourselves to: the tests changed in the same edit as the behaviour, or they prove nothing.
The agent, the topic, the action and the value we read, so your admin can open it in Agent Builder and see the same thing we did.
What goes wrong if it is left, written in business terms, then the remediation and the published source the rule came from.
Four of the thirty are read by a model rather than measured, because no pattern settles them. They are labelled as such, and two runs of the same estate can differ on them.
Rules that could not be measured say so and say why. A check we could not run is never reported as a check that passed, and a component your org would not hand over stops the scan rather than shrinking the report.
Priced from the size of the estate the report has to read — never per seat, never per agent conversation. Charities pay half, applied automatically at the quote. The scan itself is free: connect a sandbox and the preview comes by email, naming every check that failed. The report is the part you pay for, and only once you have seen what is in it.
The price climbs with the estate up to 338 elements and is flat after that at £1,950. Four of the thirty checks are read by a model rather than measured, which is most of what the later elements cost to serve.
The scan is free and the report is on the self-serve checkout: connect a sandbox, the preview naming every failed check comes by email, and you unlock the report once you have seen what is in it.
Indicative. The price you pay is counted from your own org, not from the numbers you typed here, and it is shown before anything is charged. All prices exclude VAT. Orgs running the Nonprofit Success Pack or Nonprofit Cloud pay half. 1% of every purchase goes to removing CO₂ from the atmosphere, through Stripe Climate.
Create a minimal integration user in your sandbox, log in once with Salesforce’s own CLI, and paste the connection URL into the connect page. About ten minutes.
Chalky counts the agents, topics, actions, instructions and prompt templates in the org. Nothing is charged before you have seen the number.
Thirty checks across four channels: the estate itself, your Knowledge corpus, and the four that need judgement. Minutes, not hours.
Every finding with the component named and the evidence beside it. The connection is yours to end from your own Setup whenever you like.
One read-only connection to a sandbox, and the same connection that reads your flows reads your agents. Every finding names the component, the evidence from your own org, and what to change.
