jxddlmZmZmZmZddlmZddlZddlZddl m Z m Z ddl m Z er ejeZej"GddZej"Gd d Zej"Gd d Zej"Gd dZej"GddZy))ListOptionalDict TYPE_CHECKING) defaultdictN)score_statistics test_result) score_resultc^eZdZUdZeej ed<ee e jfed<y)DatasetItemResultsz4Results for a single dataset item across all trials. test_resultsscoresN) __name__ __module__ __qualname____doc__rr TestResult__annotations__rstrr ScoreStatisticsz/Users/manta/Documents/Projects/TheRoad-I1/backend/.venv/lib/python3.12/site-packages/opik/evaluation/evaluation_result.pyr r s->{--.. &666 77rr cdeZdZUdZeed<eed<eeed<eeed<eed<eee fed<y) 'EvaluationResultGroupByDatasetItemsViewz4View of evaluation results grouped by dataset items. experiment_id dataset_idexperiment_nameexperiment_url trial_count dataset_itemsN) rrrrrrrintrr rrrrrs;>Oc]"SM!//00rrceZdZUdZeed<eed<eeed<eeed<eed<ee jed<e ee jfed<y ) $EvaluationResultAggregatedScoresViewaw Represents an aggregated view of evaluation results and scores for an experiment. This class is designed to encapsulate information about an experiment, its related dataset, trial counts, test results, and aggregated score statistics. It serves as a comprehensive representation useful in evaluation pipelines or result summaries. Attributes: experiment_id: Unique identifier for the experiment. dataset_id: Unique identifier for the associated dataset. experiment_name: Human-readable name of the experiment. experiment_url: URL link to the experiment for easy access. trial_count: Number of trials conducted in the experiment. test_results: Collection of test results from the experiment. aggregated_scores: Aggregated statistical scores for evaluation results keyed by score name. rrrr r!raggregated_scoresN)rrrrrrrr#rr rrr rrrrr%r%$sV(Oc]"SM!{--..C!1!A!AABBrr%ceZdZUeed<eed<eeed<eejed<eeed<e ed<e je Z eejed<d efd Zd efd Zd eeeffd Zy )EvaluationResultrrrrr r!)default_factoryexperiment_scoresreturnc tj|j}t|j|j |j |j|j|j|S)a Aggregates evaluation scores from test results and returns the aggregated scores view. The method calculates aggregated scores from test results and encapsulates the results in an EvaluationResultAggregatedScoresView object, which contains information about the experiment and computed aggregated scores. The aggregated scores dictionary has keys for each found score name and values containing the statistics for that score. Returns: EvaluationResultAggregatedScoresView object containing details about the experiment and the aggregated scores calculated from test results. )rrrr r!rr&) r calculate_aggregated_statisticsrr%rrrr r!)selfr&s raggregate_evaluation_scoresz,EvaluationResult.aggregate_evaluation_scoresOsd-LL    4,, 00..((**/  rc|j}t|j|j|j|j |j |S)z Create a view of evaluation results grouped by dataset items. Returns: EvaluationResultGroupByDatasetItemsView containing organized results with aggregated score statistics )rrrr r!r")_build_results_per_dataset_itemrrrrr r!)r.r"s rgroup_by_dataset_item_viewz+EvaluationResult.group_by_dataset_item_viewksM<<> 6,, 00..(('   rc|jstjdiStt}|jD],}|j j }||j|.i}|jD]=\}}|jdtj|}t||||<?|S)z Build dataset item results with aggregated score statistics. Returns: Dict mapping dataset item IDs to their results and aggregated score statistics z)No test results available for aggregationc|jS)N)trial_id)xs rzBEvaluationResult._build_results_per_dataset_item..sAJJr)key)rr) rLOGGERdebugrlist test_casedataset_item_idappenditemssortr r-r )r.results_by_dataset_item test_result_r=dataset_items_resultsdataset_item_resultsr&s rr1z0EvaluationResult._build_results_per_dataset_item}s  LLD EI"-d"3 --L*44DDO #O 4 ; ;L I.!#5L5R5R5T 1O1 % %*> % ? 0 P P$! 6H1:K6 !/ 2 6U%$rN)rrrrrrrr rr# dataclassesfieldr;r*r ScoreResultr%r/rr2rr r1rrrr(r(CsOc]"{--..SM!8I 8I8I9tL445 -Q 8 ,S $%c;M6M1N%rr(c`eZdZUdZeej ed<dee e jffdZ y)EvaluationResultOnDictItemsa Evaluation result for dict items evaluation without experiment tracking. This class provides a similar interface to EvaluationResult but is designed for lightweight evaluations that don't require experiment or dataset management. It can aggregate scores across test results just like the regular evaluation. Attributes: test_results: Collection of test results from the evaluation. rr+c@tj|jS)z Aggregates evaluation scores from test results. Returns: Dictionary mapping score names to their aggregated statistics. )r r-r)r.s rr/z7EvaluationResultOnDictItems.aggregate_evaluation_scoress ??@Q@QRRrN) rrrrrr rrrrr rr/rrrrIrIs: {--.. S c#333 4 SrrI)typingrrrr collectionsrloggingrEr r metricsr getLoggerrr9 dataclassr rr%r(rIrrrrRs66#+!   8 $ 888 111 CCC< T%T%T%n SSSr