How to Create Clear and Effective Labels and Training Phrases for Triage LLM

Labels are used in the Triage Large Language Model (LLM) to distinguish between different models. Labels should be short, clear, and distinct from others. Each label must also include a concise description that clearly defines what the label is. This description can reiterate the tag or model name, refine the definition that the LLM should already understand, or explain why the label is important.

Example of a good label:

• Order Issues – This is a good label because the LLM can easily understand that it is about problems related to customer’s orders.

Example of a bad label:

• Issues – This is a bad label because it is too vague, which makes it difficult for the LLM to determine the specific type of issue.

Good and Bad Label Descriptions

A label description should clearly define what the label represents to help the LLM categorize your end user’s question more accurately.

Good description example:
"This label is used for customer inquiries related to problems with their orders, such as missing, damaged, or incorrect items."

✅ Why it's good:

• Clearly defines what "Order Issues" includes.
• Provides specific examples (missing, damaged, incorrect items).

Bad description example:
"This label is for order-related problems."

❌ Why it's bad:

• Too vague; doesn’t specify what kind of order issues.
• Lacks examples, making it harder for the LLM to differentiate from other labels.

Phrases vs. Sentences in Label Descriptions

While writing label descriptions, you may be tempted to use single phrases instead of full sentences. However, using only phrases can reduce clarity and effectiveness. 

Example of a phrase-based label description: 

"Order-related problems: missing items, damaged goods, incorrect orders." 

Example of a sentence-based label description: "

“This label is used for customer inquiries related to problems with their orders, such as missing, damaged, or incorrect items." 

Why full sentences are better: 

• More clarity – Full sentences provide a structured definition, making it easier for the LLM to differentiate labels. 
• Better prediction – Having a full sentence can actually pre
• Improved consistency – Sentence-based descriptions ensure uniformity across multiple labels, reducing ambiguity. 

Using just phrases can make it harder for the model to interpret the intended scope, leading to misclassification or overlap with other labels. Therefore, full, well-structured sentences are recommended for label descriptions.

Training Phrases

Similar to the Solve Widget, training phrases should match how customers typically describe their issues. For example, for the Order Issues label, common training phrases could be:

• "The order I received is broken."
• "I did not receive my order."
• "My order is missing."
• "My package never arrived."
• "I got the wrong item."
• "The item I received is damaged."

These phrases help the LLM recognize and categorize customer concerns more accurately. For best practice, we recommend using at least five training phrases per label.