j] ddlmZmZmZmZddlmZddlmZm Z  ddl m Z GddejZded ed ed eed eef d Zy#e $rdZ Y3wxYw))AnyListOptionalUnion)MetricComputationError) base_metric score_result) rouge_scorerNceZdZdZ ddedededededeedeeffd Zd ed e ee efd ed e jfdZ xZS)ROUGEa9 A metric that computes the ROUGE, or Recall-Oriented Understudy for Gisting Evaluation score between an output and reference string mainly used for evaluating text summarization. ROUGE is case insensitive, meaning that upper case letters are treated the same way as lower case letters. This metrics is a wrapper around the Google Research reimplementation of ROUGE, which is based on the `rouge-score` library. References: - https://github.com/google-research/google-research/tree/master/rouge - https://huggingface.co/spaces/evaluate-metric/rouge Args: name: The name of the metric. Defaults to "rouge_metric". track: Whether to track the metric. Defaults to True. rouge_type: Type of ROUGE score to compute. Defaults to "rouge1". Must be one of the following: - "rouge1": unigram (1-gram) based scoring - "rouge2": bigram (2-gram) based scoring - "rougeL": Longest common subsequence based scoring - "rougeLSum": splits text using '\n'" use_stemmer: Whether to use stemming when computing ROUGE. Defaults to False. split_summaries: Whether to split summaries into sentences. Defaults to False. tokenizer: A tokenizer to use when splitting summaries into sentences. Defaults to None. project_name: Optional project name to track the metric in for the cases when there are no parent span/trace to inherit project name from. Example: >>> from opik.evaluation.metrics import ROUGE >>> rouge_metric = ROUGE() >>> result = rouge_metric.score( ... output="The quick brown fox jumps over the lazy dog.", ... reference="The quick brown fox jumps over the lazy dog." ... ) >>> print(result.value) 1.0 nametrack rouge_type use_stemmersplit_summaries tokenizer project_namect ||||hd}||vrtd|d|d||_t |||||_y)N)r rr>rouge1rouge2rougeL rougeLsumzInvalid rouge_type 'z'. Must be one of .rrrr)super__init__r _rouge_type_build_rouge_backend_rouge) selfr rrrrrrvalid_rouge_types __class__s /Users/manta/Documents/Projects/TheRoad-I1/backend/.venv/lib/python3.12/site-packages/opik/evaluation/metrics/heuristics/rouge.pyrzROUGE.__init__-sm d%lKG . .(&zl2DEVDWWXY &*!#+   output referenceignored_kwargsreturnc R|js tdt|tr|js td|g}nht|trXt |dk(r tdt d|Ds td|D]}|jrtd|j}|j td|jj||}||j}tj||j|d|d  S) a Compute the ROUGE score based on the given rouge_type between the output and reference strings. Args: output: The output string to score. reference: The reference string or list of reference strings. **ignored_kwargs: Additional keyword arguments that are ignored. Returns: score_result.ScoreResult with: - `value`: The ROUGE score (float). - `name`: The metric name. - `reason`: A short explanation (e.g. "rouge1 score: 0.91"). Raises: MetricComputationError: - If the candidate or any reference is empty. - If the reference is not a string or a list of strings. zCandidate is empty.zReference is empty.rc3<K|]}t|tyw)N) isinstancestr).0items r# zROUGE.score..lsCz$,sz0Reference must be a string or a list of strings.zEncountered empty reference.z!ROUGE backend is not initialized.z score: z.4f)valuer reason)striprr+r,listlenallrr score_multifmeasurer ScoreResultr )r r%r&r'ref_strrouge_score_typeresultsrouge_f1_values r#scorez ROUGE.scoreGs2||~()>? ? i %??$,-BCC" I  4 (9~",-BCCCCC,F%}}01OPP% ++ ;; ()LM M++)))V< !12;;'' &'xs/CD  r$) rouge_metricTrFFNN)__name__ __module__ __qualname____doc__r,boolrrrrrr r8r= __classcell__)r"s@r#r r sF#"! %#'&*         C= sm 48 8 d3i(8  8  ! ! 8 r$r rrrrr(cXt tdtj|g|||S)Nzh`rouge-score` libraries are required for ROUGE score calculation. Install via `pip install rouge-score`.)rrr)r ImportError RougeScorerrs r#rrs? 5   # #  '  r$)typingrrrropik.exceptionsropik.evaluation.metricsrr rouge_scorer rF BaseMetricr r,rCrr$r#rNs--2=( t K " "t n  }  c] wLsA AA