Ray's Knowledge Base

Agents do not reliably load a skill, so must-follow rules need always-on context or tools

PitfallVerified 27 Sep 2026Holds anywhere
Pitfall. The symptom, what causes it, and the fix that was run and seen to work.

Symptom#

A tool's skill is installed in the agent harness, yet the agent ignores it: it answers a question the tool covers by searching the web, or it calls the tool without the flags and habits the skill asks for (for example it guesses file paths and cuts output with | head).

Cause#

A skill loads only when the model decides its description matches the task. Some models, or some turns, never load it, so the rules in it are not in the context. Text the harness always adds, such as session-start hook output, is the only guidance such an agent sees.

Fix#

  • Put the few rules that must always hold into text every agent always sees, such as the session-start context lines: one line per rule, with the exact command.
  • For actions the agent should take, offer real tools (a pi extension's registerTool, an MCP server, for example inside a Claude Code plugin). Tools are in the tool list at every step, so they work whether or not the skill loads.
  • Keep the skill for the detailed how-to, and make its description name the situations it is for.

Evidence#

In rkb (project rkb), a test with natural questions against a small knowledge base: pi with a Codex model searched the web although an rkb lesson answered the question, and Claude ran rkb search without --toon, guessed a path (cat ~/.rkb/...) and cut output with | head. After the session-start line carried the search-first rule and rkb offered tools, a new Claude Code session listed the rkb tools and used them.