Triage LLM Simulations

The Triage LLM Simulation empowers you to test your Triage LLM models using real historical ticket data. By running simulations, you can gain valuable insights into how your model would perform before making it live, allowing you to make informed decisions about fixing or improving your model.

Why Triage LLM Simulation Matters

1. Real-World Testing: You can run tests on your models using actual past data, which helps you understand how they will work in real situations.
2. Detailed Reports: The simulation generates a report that provides insights into your model's predictions, helping you identify potential issues before going live.
3. Scalable Testing: Efficiently test multiple versions of your model, enabling you to make improvements at scale.
4. Continuous Improvement: Regular testing and analysis help you keep refining your models over time.

Get started

1. Click Triage > Simulations > Create new simulation.
   

   

2. Provide a test name for your simulation.
   
3. Select the model and version you want to test.
4. Choose a test set criteria:
   • Date range: The simulation will use the oldest 1,000 tickets within the selected time range.
   • Ticket IDs: Only the first 1,000 ticket IDs will be used for the simulation.
5. Click Submit to start the simulation.
6. Monitor the status of your simulation on the Simulations page.
   

   

7. Click on your simulation to view it in more detail:
   • Labels tab: See the labels, descriptions, and metrics (total tickets, predicted tickets, correct predictions, and accuracy) for your chosen model and version.
     

     

   • Tickets tab: View all the tickets that Forethought fetched from your helpdesk.