j( ddlZddlZddlmZmZddlmZmZmZm Z m Z ddl Z ddl m Z ejeZGddej"Zededeeeefd ed efd Zededeeeefd ed efd Zd e ed efdZy)N)contextmanagerasynccontextmanager)AnyListDictOptionalType) exceptionsc eZdZdZdefdZej ddedee e jde defd Z ejd eeee fde de fd Z ddedee e jde defd Zd eeee fde de fd Zy) OpikBaseModelz This class serves as an interface to LLMs. If you want to implement a custom LLM provider in evaluation metrics, you should inherit from this class. model_namec||_y)z Initializes the base model with a given model name. Args: model_name: The name of the LLM to be used. N)r )selfr s z/Users/manta/Documents/Projects/TheRoad-I1/backend/.venv/lib/python3.12/site-packages/opik/evaluation/models/base_model.py__init__zOpikBaseModel.__init__s %Ninputresponse_formatkwargsreturnc y)aV Simplified interface to generate a string output from the model. Args: input: The input string based on which the model will generate the output. kwargs: Additional arguments that may be used by the model for string generation. Returns: str: The generated string output. Nrrrrs rgenerate_stringzOpikBaseModel.generate_string" rmessagesc y)ak Do not use this method directly. It is intended to be used within `get_provider_response()` method. Generate a provider-specific response. Can be used to interface with the underlying model provider (e.g., OpenAI, Anthropic) and get raw output. Args: messages: A list of messages to be sent to the model, should be a list of dictionaries with the keys kwargs: arguments required by the provider to generate a response. Returns: Any: The response from the model provider, which can be of any type depending on the use case and LLM. Nrrrrs rgenerate_provider_responsez(OpikBaseModel.generate_provider_response1rrc Ktdw)ae Simplified interface to generate a string output from the model. Async version. Args: input: The input string based on which the model will generate the output. kwargs: Additional arguments that may be used by the model for string generation. Returns: str: The generated string output. 2Async generation not implemented for this providerNotImplementedErrorrs ragenerate_stringzOpikBaseModel.agenerate_stringDs ""VWW c Ktdw)a Do not use this method directly. It is intended to be used within `aget_provider_response()` method. Generate a provider-specific response. Can be used to interface with the underlying model provider (e.g., OpenAI, Anthropic) and get raw output. Async version. Args: messages: A list of messages to be sent to the model, should be a list of dictionaries with the keys "content" and "role". kwargs: arguments required by the provider to generate a response. Returns: Any: The response from the model provider, which can be of any type depending on the use case and LLM. r!r"rs ragenerate_provider_responsez)OpikBaseModel.agenerate_provider_responseVs$""VWWr%)N)__name__ __module__ __qualname____doc__strrabcabstractmethodrr pydantic BaseModelrrrrrr$r'rrrr r s%3% ?C  "$x'9'9":;     $  T#s(^, 8;   *?CXX"$x'9'9":;X X X$XT#s(^,X8;X Xrr model_providerrrrc+K |j|fi|y#t$r9}tjd|t j t |d}~wwxYww)a  Provides a context manager for getting and managing the response from a model provider. Ensures that errors during the interaction with the model provider are handled appropriately and logged. Args: model_provider: Instance of a class derived from `OpikBaseModel` responsible for interfacing with the model. messages: List of dictionaries containing the messages or inputs to be passed to the model. **kwargs: Additional keyword arguments to customize the generation of the model responses. Yields: Any: The response generated by the model provider. Raises: exceptions.BaseLLMError: If the response generation from the model provider fails due to an exception. z'Failed to call LLM provider, reason: %sN)r ExceptionLOGGERerrorr BaseLLMErrorr,)r1rres rget_provider_responser8ksV0.7n77KFKK . >B%%c!f--.s$AA A4AAAc K |jdd|i|d{}|y7 #t$r9}tjd|t j t |d}~wwxYww)a Asynchronous context manager for getting a response from a model provider. This function asynchronously interacts with the specified `model_provider` to generate a response based on the given list of `messages` and additional optional keyword arguments. If an error occurs during this process, it is logged, and a custom exception is raised. Args: model_provider: The model provider from which to request the response. messages: A list of dictionaries containing the messages for the model provider to process. **kwargs: Additional keyword arguments passed to the model provider's response generation method. Yields: Any: The response generated asynchronously by the model provider. Raises: exceptions.BaseLLMError: If there is an error during the asynchronous interaction with the model provider. rNz6Failed to call LLM provider asynchronously, reason: %sr)r'r3r4r5r r6r,)r1rrresponser7s raget_provider_responser;ss6.CCC  !'    . MqQ%%c!f--.s0A-(& (A-( A*4A%%A**A-outputc4|tjd|S)a Checks the output of a model and verifies that it is not None. This function ensures that the output returned from a language model (LLM) has a valid, non-null value. If the output is found to be None, an error is raised with a detailed message. This can help in debugging issues related to incorrect environment configuration or missing API keys. Args: output: The output string generated by the language model to be validated. Returns: The output of the language model that was validated. Raises: exceptions.BaseLLMError: Raised if the output is evaluated to None. The error message contains suggestions to verify environment configurations and check model API key availability. zReceived None as the output from the LLM. Please verify your environment configuration and ensure that the API keys for the models in use (e.g., OPENAI_API_KEY) are set correctly.)r r6)r<s rcheck_model_output_stringr>s)$~%% k  Mr)r-logging contextlibrrtypingrrrrr r/opikr getLoggerr(r4ABCr r,r8r;r>rrrrEs :22   8 $[XCGG[X|.!.-1$sCx.-A.MP...<!.!!.-1$sCx.-A!.MP!.!.!.Hhsmr