Context#

On each user message, rkb hook prompt may add a lesson as hidden context. The first measurement, with Laya on the GPU, was 15 questions against 11 real lessons (2026-09-27): the right lesson first 7 of 9 times with relevance 0.49 to 0.93; wrong lessons in second place often at 0.82 to 0.86; two questions no lesson answers reached 0.77. It set the margin at 0.05. The harder eval set (tests/fixtures/eval, 124 queries) and the new Laya question (173a08ddf5) then gave a sweep over minimum relevance and margin.

Decision#

Add only the top lesson, and only when its relevance is at least hooks.recall_min_relevance (0.85) and at least hooks.recall_min_margin (0.10) above the next lesson. With BM25 only, add nothing.

Why#

A wrong hidden lesson misleads the agent, while a missed lesson is still one rkb_search away, so recall favors precision. On the 124 eval queries with the new question, margin 0.10 added 12 right lessons and 2 wrong ones; margin 0.05 added 20 right and 5 wrong. On the real lessons the new question keeps the top score of questions no lesson answers at 0.52 or lower.

Rejected options#

• Up to 2 lessons at 0.8: would have added wrong second-place lessons on most matching questions.
• Margin 0.05: about one wrong add per four right ones on the eval set.
• Minimum 0.80 with margin 0.10: 19 right and 7 wrong on the eval set.
• A threshold alone at 0.9: kept only 1 of the right lessons.
• Recall with BM25 scores: a BM25 score means different things for different queries.