jBUdZddlmZddlmZmZmZmZmZee ge fZ hdZ de d<e Dchc]}|jc}Z ddhZde d <dd Zdd Zdd Zdd ZGddZeZej-deej-deeZgdZycc}w)z Capability registry for evaluation models. The registry is designed to grow beyond vision support (e.g. audio in the future). ) annotations)CallableDictIterableOptionalSet>yi-vlgpt-4ogpt-4.1qwen-vlclaude-3qwen2-vl claude-3-5 gpt-5-mini gpt-4-turbo gpt-4o-mini gpt-4-vision gpt-4.1-mini gpt-4.1-nano phi-3-visiongemini-1.5-prophi-3.5-visiongpt-4.1-previewgemini-1.5-flashgemini-2.0-flashchatgpt-4o-latestgemini-pro-visionllama-3.2-11b-visionllama-3.2-90b-visionllavacogvlmpixtralzSet[str]VISION_MODEL_PREFIXESz-visionz-vlVISION_MODEL_SUFFIXESc<d|vr|S|jdd\}}|S)N/)split) model_name_suffixs /Users/manta/Documents/Projects/TheRoad-I1/backend/.venv/lib/python3.12/site-packages/opik/evaluation/models/model_capabilities.py_strip_provider_prefixr-4s+ *  a(IAv McN ddl}|j|S#t$rYywxYw)Nr)modelF)litellmsupports_vision Exception)r)r1s r,_litellm_supports_visionr4;s0&&Z&88 s  $$ct|}||h}|D]S}t|ry|jtfdtDrytfdt DsSyy)NTc3@K|]}j|ywN) startswith).0prefix normalizeds r, z-vision_capability_detector..KsQ;Pz$$V,;Pc3@K|]}j|ywr7)endswith)r9r+r;s r,r<z-vision_capability_detector..MsO9Nvz""6*9Nr=F)r-r4loweranyr#r$r)stripped candidates candidater;s @r,vision_capability_detectorrFDsa%j1Hh'J #I .__& Q;PQ Q O9NO O  r.ct|}||h}|D]#}|j}d|vryd|vsd|vs#yt|S)a Heuristically determine whether a model accepts video inputs. Providers rarely expose structured metadata for video support, so we fall back to naming conventions (e.g. models whose names contain ``video`` or ``qwen`` + ``vl``). When those heuristics fail we delegate to the vision detector since current SDK integrations treat video as an extension of multimodal/vision APIs. videoTqwenvl)r-r@rFrBs r,video_capability_detectorrKRsX&j1Hh'J __& j  Z DJ$6   &j 11r.ceZdZdZd dZ ddZddZddZddZddZ ddZ dd Z e dd Z e dd Zy )ModelCapabilitiesRegistryz: Central registry for model capability detection. ci|_yr7_capability_detectors)selfs r,__init__z"ModelCapabilitiesRegistry.__init__ns DF"r.c"||j|<y)zE Register a detector callable for a capability name. NrO)rQ capabilitydetectors r,register_capability_detectorz6ModelCapabilitiesRegistry.register_capability_detectorqs 2:"":.r.ct|sy|jj|}|y ||S#t$rYywxYw)z] Return True when the supplied model name supports the requested capability. F)rPgetr3)rQrTr)rUs r,supportsz"ModelCapabilitiesRegistry.supportsysK--11*=   J' '  s + 77c&|jd|S)zC Convenience wrapper for vision-capable detection. visionrYrQr)s r,r2z)ModelCapabilitiesRegistry.supports_visions}}Xz22r.c&|jd|S)zB Convenience wrapper for video-capable detection. rHr\r]s r,supports_videoz(ModelCapabilitiesRegistry.supports_videos}}Wj11r.chtj|j|jyr7)r#addr-r@r]s r,add_vision_modelz*ModelCapabilitiesRegistry.add_vision_models$!!$"="=j"I"O"O"QRr.c4|D]}|j|yr7)rb)rQ model_namesr)s r,add_vision_modelsz+ModelCapabilitiesRegistry.add_vision_modelss%J  ! !* -&r.ct|Sr7)rFr]s r,_supports_visionz*ModelCapabilitiesRegistry._supports_visions )*55r.ct|Sr7)r-r)s r,r-z0ModelCapabilitiesRegistry._strip_provider_prefixs %j11r.ct|Sr7)r4ris r,r4z2ModelCapabilitiesRegistry._litellm_supports_visions ' 33r.N)returnNone)rTstrrUCapabilityDetectorrkrl)rTrmr) Optional[str]rkbool)r)rorkrp)r)rmrkrl)rdz Iterable[str]rkrlr)rmrkrpr)rmrkrm)__name__ __module__ __qualname____doc__rRrVrYr2r_rbrerg staticmethodr-r4r.r,rMrMisqG::);: : 3 2 S.62244r.rMr[rH)rMMODEL_CAPABILITIES_REGISTRYModelCapabilitiesrFrKNrrrq)rv __future__rtypingrrrrrrmrprnr#__annotations__r@r$r-r4rFrKrMryrVrz__all__)r:s0r,rs #::seTk*!#x!D7LL6KF6KL#,e"4x4 2.=4=4@8988 (88 & 0 KMsB