What is Deflection Insights?
At a high level, Deflection Insights enables customer support teams to assess the quality of Deflections based on the quality of interactions between a user and AI chatbot. Deflection Insights is only available for Discover customers.
Why are Deflection Insights useful?
Deflections are an industry standard for evaluating the straightforward and mechanical outcomes of human escalations. However, Deflections alone cannot accurately represent the quality of interactions between users and AI chatbots. Deflection Insights offers an AI-driven interpretation of conversation transcripts for chat only.
Let’s Dive Deeper
🧑🏫 Think of Deflection Insights as a third-party auditor impartially reviewing conversation transcripts and assigning grades. 👩🎓 This is what Deflection Insights does. Deflection Insights assesses conversation quality without concern for how workflows are created. |
How are Deflection Insights measured?
Deflection Insights measures the quality of chatbot conversations – one-on-one interactions between a user and the chatbot.
To comprehensively assess conversation quality, Deflection Insights examines the contributions from both the user and the chatbot. The chatbot's performance is judged based on how relevant its responses are to the user's queries.
|
Important Note
Deflection Insights does not evaluate the accuracy or currency of the information provided by the chatbot. Assessing such aspects requires specific knowledge about the company that is beyond Forethought's ability to determine. To gauge the accuracy of information, alternative methods such as Customer Satisfaction Scores (CSAT), article click rates, or identifying gaps in the Knowledge Base should be considered. |
Does Deflection Insights apply to all conversations?
No. Deflection Insights rating will be available for deflected chatbot conversations only as understanding the quality of Deflected conversations is top-of-mind for most Forethought customers.
Deflection Insights vs Deflections
In the industry, "Deflections" refers to a method used to decide if a customer conversation requires the intervention of human agents. This could mean creating a support ticket for an agent to address later or transferring the conversation directly to a live agent. While "Deflections" offer a clear perspective in terms of Return on Investment (ROI), they don't provide insight into the quality of these interactions.
On the other hand, "Deflection Insights" specifically assesses the quality of the conversation by examining the responses exchanged between the end user and the chatbot. For a high-quality interaction, the chatbot must deliver information that directly answers the user's question, and the user must remain engaged, allowing the chatbot the chance to provide accurate and relevant responses.
Deflection Insight’s Relevance Definition
This criterion assesses whether the chatbot's responses align directly with the customer's query. It measures the extent to which the chatbot provides useful and applicable information or solutions that directly address and contribute to resolving the customer's stated problem or question. This evaluation excludes the analysis of user behavior, which is assessed separately, ensuring that the focus remains strictly on the suitability and applicability of the chatbot’s responses.
Chatbot Relevance Cheat Sheet |
|
| Relevance Results | Definition |
| Relevant | This indicates the chatbot's response aligns with the customer's query. A relevant answer would address the customer's problem directly and provide useful, applicable information or solutions. It should be closely related to the customer's query and not stray off-topic or provide extraneous information. |
| Somewhat relevant |
|
| Irrelevant | This indicates the chatbot's response does not align with the customer's query. An irrelevant answer fails to address the customer's problem directly and does not provide useful or applicable information or solutions. It may stray off-topic, introduce unrelated subjects, or include extraneous information that does not contribute to solving the customer's issue or answering their question. Additionally, if the chatbot's response consists largely of clarifying questions without progressing towards a solution or relevant information, it is also considered irrelevant. |
Deflection Insight’s Engagement Definition
This criterion determines whether the user remained actively involved throughout the conversation, meaningfully progressing through all necessary steps to arrive at an answer. A "Yes" indicates that the user consistently interacted with the chatbot, responding to prompts and following the dialogue flow designed to resolve their query. A "No" suggests that the user disengaged at some point, either by not responding, prematurely ending the conversation, or bypassing crucial steps needed for a resolution.
User Engagement Cheat Sheet |
||
| Did the user provide answers to Form or button? | Was the chatbot able to retrieve information based on user input? | Engaged Status |
| Yes | Yes | Engaged |
| No | Yes | Drop off |
| Yes | No | Drop off |
| No | No | Drop off |
How to improve your Deflection Insights Quality
| Chatbot Relevance | User Engaged | Recommendations |
| Relevant | Yes | No actions needed as the response is relevant and the user is engaged. |
| Relevant | No |
|
| Irrelevant | Yes |
|
| Irrelevant | No |
|
| Somewhat relevant | Yes |
|
| Somewhat relevant | No |
|