Overview

In TrustTest there are specialized components designed to assess if an AI model response is compliant with a specific set of criteria. They provide a systematic way to measure various aspects of model outputs against predefined criteria, ensuring reliable and consistent evaluation across different use cases.

---

Key Areas

Heuristic evaluators — rule-based, no extra LLM call:

• Language — FastText (expected_languages)
• Equals, Regex, BLEU
• Signature evaluators — Virus, Spam, Phishing, XSS
• Bias comparison

LLM judges:

• Correctness, Completeness, Tone
• URL correctness, True/False
• Answer relevance, RAG poisoning
• Custom — CustomEvaluatorExpected / CustomEvaluatorObjective

---

Why It Matters

• Quality Assurance Evaluators provide objective metrics to ensure AI responses meet quality standards and requirements.

• Consistent Assessment By standardizing evaluation criteria, evaluators enable reproducible and comparable results across different models and use cases.

• Flexible Evaluation The modular design allows for custom evaluators to be created for specific needs while maintaining a consistent interface.

• Comprehensive Analysis Different types of evaluators can be combined to provide a holistic assessment of model performance across multiple dimensions.

• Trust and Reliability Systematic evaluation helps build confidence in AI systems by providing clear metrics and explanations for assessment results.