Articles in this section

Capturing every intent for deeper insights in the landing page metrics

The landing page metrics have been revamped to provide a more comprehensive understanding of user interactions with your chatbot. Previously, these metrics only considered the last intent used, resulting in valuable data loss as the number of intents used was undercounted if there were multiple intents. Therefore, to address this issue, we have changed the way we calculate the chat column logic. This article will explain the metrics on the landing page and demonstrate how the new chat column logic will be calculated with examples.

 

What are the metrics on the landing page?

Screenshot 2024-03-27 at 10.15.07 AM.png

  Metric Name

Definition
Total chats Counts the number of times a user opens a chat widget. It will be deemed a single chat session if the user revisits the widget within 3 hours of the last interaction. If the user restarts the conversation or refreshes the browser tab within 3 hours, it will only count as 1 chat session.
Total deflected chats

This metric measures the number of chat sessions in which one of the following events did not occur:

  1. User was not successfully transferred to a human agent 
  2. User did not succeed in creating a ticket. 

If any of the events above occurred during a chat session, that chat session will be marked as not deflected. All other chat sessions will be marked as deflected.

Chat deflection rate It is calculated by dividing the total deflected chats by the total number of chats.
Total estimated cost savings This metric calculates the cost savings you can attain from deflecting chats. It multiplies an estimated cost per ticket ($15) with the total number of deflected chats. If your ticket cost differs, divide by $15 and multiply by deflected chats for accurate results.
Chats The count represents how many times an Intent was utilized within a Chat Session. In cases where multiple Intents are employed within a single Chat Session, each used Intent will increase by 1. If an Intent is used multiple times in a Chat Session, it will still only be counted once.
Deflected The number of deflected Chats that used this Intent. If the same Intent was used multiple times during a deflected chat, it would be recorded as one.
Deflection rate

Deflected divided by Chat

 

How does each different conversation affect the chat metrics?

 

1. Chat Conversation with Single Intent but without Knowledge Retrieval

When a user interacts with a chatbot and an intent is identified, this is counted as a "+1" to the specific intent within the chat metrics. For instance, if the chatbot identifies an intent related to 'Store hours,' then this will contribute a "+1" to the chat metrics specifically for the 'Store hours' intent.

Visual Materials Landing Page Metrics Examples -1.png

 

2. Conversation with Multiple Intents but without Knowledge Retrieval

When different intents are identified, each unique intent is counted separately with a "+1”. For example, in this chat conversation, there are two different intents: ‘Online order inquiry’ and ‘Payment method inquiry’. Both of these intents are counted as +1 each.

Visual Materials Landing Page Metrics Examples -2.png

 

3. Chat Conversation with Multiple Same Intents but without Knowledge Retrieval

If the chatbot identifies the same intent more than once within a single conversation, it's counted as just "+1" for that intent in the entire conversation. To put it simply, we only count the value of each unique intent. In the given example, the intent was detected twice in the chat, but we only counted it as "+1" because it was the same intent.

Visual Materials Landing Page Metrics Examples -3.png

 

4. Chat Conversation with No Intent and Knowledge Retrieval

When a user's question doesn’t match a specific intent, the chatbot activates Knowledge Retrieval. This enables the chatbot to scan the relevant information from the knowledge base and use it to respond to the user's query. If the chatbot can deliver an answer, it will count as a "+1" to the Knowledge Retrieval metric.

Single Intent w KR-1.png

 

5. Chat Conversation with Single Intent and then Handed Off

If a user asks a question and the chatbot isn’t able to resolve the query, a standard handoff will be triggered, and a “+1” will be added to the Standard Handoff metric.

Visual Materials Landing Page Metrics Examples -5.png

 

6. Chat Conversation where the user proactively asks to get connected with Customer Support

If the user proactively wants to speak to an agent, an intent called Customer Support Intent will be triggered. This intent is activated when a user explicitly expresses their desire to connect with a human agent for assistance during a chat session. Whenever this intent is triggered, it will be counted as a "+1" to the chat metric for this specific intent.

Visual Materials Landing Page Metrics Examples -6.png

Was this article helpful?
0 out of 0 found this helpful

More resources

  • Need support?

    Submit a request and our support team will assist you

  • Business hours

    Monday to Friday 8am - 5pm PST excluding US holidays

  • Contact us

    support@forethought.ai