Q0 "What is intent recognition?"
"Intent recognition is figuring out what the user actually wants, so the system knows which tool to use.
It is not answering the question. It only decides who should answer it.
In our system the tools are things a human already approved — an approved Q&A item, or an approved SQL query. So intent recognition is really: match this question to one of our approved capabilities."
"Would you like me to go into how we do it?"
→ 说完就停。别一口气把细节全倒出来。
Q1 "So it decides the answer?"
"No — it decides who answers, not what the answer is.
It picks the tool. The tool produces the answer."
Answer shape: correct the assumption in one line, then give the two sides below.
A question comes in. We map it to one item on a list.
The list is fixed, and someone approved it. That is the whole point — it is how we keep the system testable.
It does not write the answer.
It is not free-form. The model cannot invent a new capability.
"A good intent recogniser must also be able to say 'none of these'.
If it can only pick the closest option, then when a user asks about the weather, it will still pick something — and it will sound confident."
Q2 "How does yours work?"
"Four steps: we search, we re-rank, we cut, and then a small planner picks up to three capabilities.
The important part is that the planner can only choose from what the search returned."
Answer shape: say those four words, then expand only if they want the detail.
A question comes in │ ├─ 1 Search find candidate capabilities — Q&A items and SQL templates │ ├─ 2 Re-rank a stronger model reads the question and the candidate together │ ├─ 3 Cut each type has its own cut-off — zero candidates is fine │ └─ 4 Plan a small planner picks 1 to 3 of them, then code checks the plan
"Every capability is indexed by the questions a person would ask to trigger it. So we search questions against questions.
Then the planner only chooses from that short list. It cannot invent a tool — it can only pick from what the search returned."
Q3 "Can you go deeper?"
"Sure — which part? The matching, the shortlist, what happens when nothing fits, or the planner?"
Answer shape: let them choose. Then give the first line only, and stop. Go deeper if they ask again.
Q3.1 — "Why not just train a classifier?"
Short: "Because then adding a new capability means retraining. With search, adding a capability is just adding data."
More: "One example question is enough to start, and the business team can write it — they don't need us. There's no class imbalance problem, and no cold-start problem."
Fair trade-off, if they push: "A classifier can win once the list is stable and you have a lot of labelled data. That's a later optimisation. And the two aren't exclusive."
Q3.2 — "How do you build the shortlist?"
Short: "Each type is handled separately. Q&A gets up to three, SQL gets up to three, and we put them side by side."
Why separately: "Their scores are not on the same scale. If I rank them against each other, the winner is decided by text shape, not by meaning."
One more detail: "One capability has many phrasings. We collapse to the capability first and keep its best phrasing — otherwise one popular item takes all three slots, and we lose variety."
Q3.3 — "What if nothing matches?"
Short: "Then the list is empty, and that is a normal outcome. We say we don't have it."
Why it matters: "A model will use whatever you show it. So showing it three bad candidates is worse than showing it none."
The real story, if you have time: "In testing, a warranty question matched a stock query at 0.5183. The lowest genuine match was 0.4981. So no threshold could separate them — if I raise it, I kill real answers first. That's when I decided 'none of these' has to be a real outcome, not a leftover."
Q3.4 — "What if two capabilities look the same?"
Short: "Then the top two scores are very close, and we treat that as low confidence — we don't just take the winner."
What we do: "Either ask one clarifying question, or refuse. Picking by a tiny gap is basically luck, and luck is not something I want in the system."
Prevention, if they push: "We also check this before anything goes live. Every phrasing has to find its own capability when we search with it. If it finds someone else's, that phrasing is bad data — we fix it or delete it."
Q3.5 — "Why cap the plan at three steps?"
Short: "Because I want the plan to be testable."
Two reasons: "Errors multiply — at 95% per step, three steps is about 86%, six steps is about 73%. And a three-step trace is something a human actually reads; a ten-step one, nobody reads."
And the nice part: "If a question really needs more, we don't improvise. We refuse, and offer to save it as a workflow for someone to approve. So the system gets better by refusing."
Q3.6 — "The planner is an LLM. What stops it making things up?"
Short: "It can only choose from the shortlist. A tool that wasn't retrieved can't even be written down."
Then the checks, all plain code: "Three steps or fewer. Every step must come from the shortlist. And required values must come from the user's question — the model cannot invent a date or an account number."
The line I'd use: "I don't ask the model to judge things that code can decide. Counting steps is code. Only the last check is a model: does this plan actually answer this question?"
If they mention prompt injection: "Text inside a document can't name a tool we didn't retrieve, so injection can't change the plan. That's structural, not a filter."
Q3.7 — "How do you know it works?"
Short: "Because the plan is a small object, I can compare it to an expected plan, exactly. No model judging it."
What we measure: "Plan accuracy. Whether the right capability made the shortlist. And the number I care about most — how often we answer confidently and wrongly."
The honest part: "I also measure how often we refuse when we shouldn't. A system tuned only for safety quietly stops being useful — and nobody complains about an answer they never received."
Q4 "Anything you would like to add?"
1. "Intent recognition decides who answers, not what the answer is."
2. "We match the question against an approved list — so the model can't invent a tool."
3. "It must be able to say 'none of these'."
4. "The plan is three steps or fewer, so I can test it."
"Let me give you the short answer first."
"That's a good question — let me think for a second."
"To be honest, that part isn't built yet. What we do today is…"