> ## Documentation Index > Fetch the complete documentation index at: https://neuraltrust-92b43583-develop.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # LLMs & Embeddings ## LLM Clients TrustTest provides a flexible abstraction layer for working with different LLM providers through its `LLMClient` interface. This architecture allows for seamless integration with various LLM services while maintaining a consistent interface for generating questions, evaluations, and other LLM-powered features. ### Architecture The core of this system is the `LLMClient` abstract base class, which defines two main methods: * `complete(instructions, system_prompt)`: For single-prompt completions * `complete_chat(messages)`: For multi-turn conversations Each implementation handles provider-specific details while exposing a unified interface. ### Supported Providers `get_llm_client(provider=..., model=..., **kwargs)` supports: `openai`, `azure`, `google`, `anthropic`, `ollama`, `vllm`, `groq`, `deepseek`, `http`. | Provider | Extra | Notes | | ----------- | ----------------------------------------- | ---------------------------------------------------------------------------------------------------- | | `openai` | `trusttest[openai]` | Default models in `set_config` | | `azure` | `trusttest[azure]` or `trusttest[openai]` | Azure OpenAI judge / generator | | `google` | `trusttest[google]` | | | `anthropic` | `trusttest[anthropic]` | | | `ollama` | `trusttest[ollama]` | Local | | `vllm` | `trusttest[vllm]` | OpenAI-compatible | | `groq` | `trusttest[openai]` | Uses `OpenAiClient` against Groq | | `deepseek` | `trusttest[deepseek]` | | | `http` | none | Judge/generator against any HTTP endpoint — **not** the same as `HttpTarget` (the system under test) | ```shell theme={null} uv add "trusttest[openai]" uv add "trusttest[anthropic]" uv add "trusttest[google]" uv add "trusttest[ollama]" uv add "trusttest[vllm]" uv add "trusttest[azure]" uv add "trusttest[deepseek]" ``` #### HTTP judge / generator `HTTPClient` is an LLM client. `HttpTarget` is the model you are testing. Do not mix them. ```python theme={null} from trusttest.llm_clients import get_llm_client from trusttest.evaluators import TrueFalseEvaluator judge = get_llm_client( provider="http", model="ignored", url="https://your-judge.example.com/v1/complete", payload_config={ "messages": [ {"role": "system", "content": "{{ system_prompt }}"}, {"role": "user", "content": "{{ instructions }}"}, ] }, concatenate_field="choices.0.message.content", headers={"Authorization": "Bearer ..."}, ) evaluator = TrueFalseEvaluator(llm_client=judge) ``` `concatenate_field` supports dot paths. Placeholders default to `{{ instructions }}` and `{{ system_prompt }}`. You can also pass `llm_client=` on individual probes and evaluators instead of changing global config. ### Usage Example ```python theme={null} import asyncio import os from trusttest.llm_clients import get_llm_client # Set up environment variables for the provider os.environ["DEEPSEEK_BASE_URL"] = "https://api.deepseek.com" os.environ["DEEPSEEK_API_KEY"] = "" # Initialize the client llm = get_llm_client(model="deepseek-chat", provider="deepseek") # Make a completion request response = asyncio.run(llm.complete("Return a json saying hello")) print(response) ``` The abstraction allows for easy switching between providers while maintaining consistent behavior across the application. ## Embeddings Clients TrustTest provides a flexible abstraction layer for working with different embedding providers through its `EmbeddingsModel` interface. This architecture allows for seamless integration with various embedding services while maintaining a consistent interface for generating vector representations of text. ### Architecture The core of this system is the `EmbeddingsModel` abstract base class, which defines the main method: * `embed(texts)`: Converts a sequence of texts into numerical vector representations Each implementation handles provider-specific details while exposing a unified interface. ### Supported Providers `get_embeddings_model(provider=..., model=...)` supports `openai`, `azure`, `google`, and `ollama`. ```python theme={null} from trusttest.embeddings import get_embeddings_model embeddings = get_embeddings_model( provider="azure", model="text-embedding-3-small", ) ``` Use `trusttest[azure]` (or `trusttest[rag-azure]`) for Azure embeddings. Vector knowledge bases need embeddings plus a `topic_summarizer` LLM via `set_config`. ### Usage Example ```python theme={null} import os from trusttest.embeddings import get_embeddings_model os.environ["OPENAI_API_KEY"] = "" embeddings = get_embeddings_model( provider="openai", model="text-embedding-3-small", ) texts = ["Hello world", "TrustTest is great"] vectors = embeddings.embed(texts) print(vectors.shape) ``` The abstraction allows for easy switching between providers while maintaining consistent behavior across the application. ## Global Configuration TrustTest provides a global configuration system to manage LLM and embeddings settings across your application. The configuration can be set using the `set_config()` function, which accepts a dictionary with settings for different components: ```python theme={null} import trusttest from trusttest.config import get_config, load_config, TrustTestConfigError trusttest.set_config({ "evaluator": { "provider": "openai", "model": "gpt-4o-mini", "temperature": 0.2, "retry_config": {"attempts": 3, "initial_delay": 1.0, "max_delay": 10.0, "exp_base": 2.0}, "extra_args": {}, }, "question_generator": { "provider": "openai", "model": "gpt-4o-mini", "temperature": 0.5, }, "translation": { "provider": "openai", "model": "gpt-4o-mini", "temperature": 0.2, }, "embeddings": { "provider": "openai", "model": "text-embedding-3-small", }, "topic_summarizer": { "provider": "openai", "model": "gpt-4o-mini", "temperature": 0.2, }, }) ``` **Tasks:** `evaluator`, `question_generator`, `translation` (used by `StaticDatasetProbe.translate_into_language`), `topic_summarizer`, `embeddings`. **LLM fields:** `provider`, `model`, `temperature`, optional `retry_config` (`attempts`, `initial_delay`, `max_delay`, `exp_base`), optional `extra_args`. Config `provider` literals are `openai`, `azure`, `deepseek`, `vllm`, `google`, `anthropic`, `ollama`, `http`. Use `get_llm_client(provider="groq", ...)` for Groq. **Embeddings fields:** `provider` (`openai`, `azure`, `google`, `ollama`) and `model`. **Defaults** when you do not set config: `gpt-4o-mini` (all LLM tasks) and `text-embedding-3-small`. ### Config files On import, TrustTest looks in the current working directory for `.trusttest_config.json`, then `trusttest_config.json`. The JSON object uses the same keys as `set_config`. If neither file exists and you never called `set_config`, `get_config()` raises `TrustTestConfigError`. ```python theme={null} from trusttest.config import get_config, load_config, TrustTestConfigError load_config() # re-read CWD files try: cfg = get_config() except TrustTestConfigError: trusttest.set_config({...}) ``` ## Implementing Custom Clients Both LLM and Embeddings clients can be easily extended by implementing custom providers. The base classes provide a clear interface that you need to implement. ### Custom LLM Client To create a custom LLM client, inherit from `LLMClient` and implement the required methods: ```python theme={null} from trusttest.llm_clients.base import LLMClient, ChatMessage, BaseLLMResponse class CustomLLMClient(LLMClient): async def complete( self, instructions: str, system_prompt: Optional[str] = None, response_schema: Type[BaseModel] = BaseLLMResponse, ) -> Dict[str, Any]: # implement your custom logic here raise NotImplementedError async def complete_chat( self, messages: Sequence[ChatMessage], response_schema: Type[BaseModel] = BaseLLMResponse, ) -> Dict[str, Any]: # implement your custom logic here raise NotImplementedError ``` The LLMClient expects to define the response schema, this is a pydantic model that will be used to parse the response from the LLM. Once implemented, you are ready to use them: ```python theme={null} custom_llm = CustomLLMClient() evaluator = CorrectnessEvaluator(llm_client=custom_llm) ``` ### Custom Embeddings Client To create a custom embeddings client, inherit from `EmbeddingsModel` and implement the required method: ```python theme={null} from trusttest.embeddings import EmbeddingsModel import numpy as np class CustomEmbeddingsModel(EmbeddingsModel): def embed(self, texts: list[str]) -> np.ndarray: # Implement your custom embedding logic pass ```