i<dZddlZddlZddlZddlmZmZddlZddlm Z ddl m Z m Z ddl mZddlmZGd d eZy) z Qdrant Semantic Cache implementation Has 4 methods: - set_cache - get_cache - async_set_cache - async_get_cache N)Anycast) print_verbose)QDRANT_SCALAR_QUANTILEQDRANT_VECTOR_SIZE)EmbeddingResponse) BaseCachecTeZdZ d dZdefdZdZdZdZdZ d Z d Z y) QdrantSemanticCacheNc ddl} ddlm} m} m} ddlm} | td||_td|j| td||_ ||_ ||nt|_ i}|r)t|tr|j!dr| |}|r)t|tr|j!dr| |}|xs$| j#dxs| j#d }|xs| j#d }d d i}|r||d <| t%d||_||_td|j&||_| |_| | j.|_| td|j,j3|j&d|jd|j*}|j4dk7rt%d|j6|j9ddrn|j,j3|j&d|j|j*}|j9|_td|j:y||dk(rdddii}n*|dk(r ddt<ddi}n|d k(rd d!dd"i}n td#|j,j?|j&d|j|jd$d%|d&|j*'}|j9drn|j,j3|j&d|j|j*}|j9|_td(|j:ytd))*Nr)_get_httpx_clientget_async_httpx_clienthttpxSpecialProvider)get_secret_strz-collection_name must be provided, passed Nonez0qdrant semantic-cache initializing COLLECTION - z2similarity_threshold must be provided, passed Nonez os.environ/ QDRANT_URLQDRANT_API_BASEQDRANT_API_KEYz Content-Typezapplication/jsonzapi-keyzQdrant url must be providedz'qdrant semantic-cache qdrant_api_base: ) llm_providerzNQuantization config is not provided. Default binary quantization will be used. /collections/z/exists)urlheadersz1Error from qdrant checking if /collections exist resultexistsz.Collection already exists. Collection details:binary always_ramFscalarint8)typequantilerproductx16) compressionrzBQuantization config must be one of 'scalar', 'binary' or 'product'Cosine)sizedistance)vectorsquantization_config)rjsonrz+New collection created. Collection details:z#Error while creating new collection) os&litellm.llms.custom_httpx.http_handlerrrrlitellm.secret_managers.mainr Exceptioncollection_namersimilarity_thresholdembedding_modelr vector_size isinstancestr startswithgetenv ValueErrorqdrant_api_baseqdrant_api_keyr sync_clientCaching async_clientget status_codetextr*collection_inforput)selfr8r9r/r0r)r1 host_typer2r+rrrrrcollection_existscollection_detailsquantization_paramsnew_collection_statuss v/Users/manta/Documents/Projects/TheRoad-I1/.venv/lib/python3.12/site-packages/litellm/caching/qdrant_semantic_cache.py__init__zQdrantSemanticCache.__init__s   @  "KL L.>t?S?S>T U  'PQ Q$8!.&2K8J  /3/O4N4N5#1"A .#.>3L3L4"0!?  Vryy6 V"))DU:V (F2995E+F!#56 !/GI   ":; ;.,?@T@T?UVW ,.2-55   & ` !,,00''( d6J6J5K7SLL1   ( (C /CDUDZDZC[\   ! ! #H -h 7!%!1!1!5!5++,M$:N:N9OP "6" $6#:#:# %9% !%))+H5%)%5%5%9%9//0 d>R>R=ST LL&:&"(:'>'>'@$B4CWCWBXY EFFcached_responsec||S tj|}|S#t$rtj|}Y|SwxYwN)r*loadsr.ast literal_eval)rBrKs rH_get_cache_logicz$QdrantSemanticCache._get_cache_logicsV  "" " @"jjO  @!..?O @s ??c td|ddlm}|d}d}|D] }||dz } ttt j |j|ddd }|d dd } t|}t|tsJd t|j| ||d dgi} |jj|jd|jd|j| y)Nz)qdrant semantic-cache set_cache, kwargs: ruuidmessagescontentTzno-storezno-cachemodelinputcachedata embeddingpointsr?responseidvectorpayloadr/pointsrrr*)r litellm._uuidrTrrlitellmr^r1r4r3uuid4r:rAr8r/r) rBkeyvaluekwargsrTrUpromptmessageembedding_responser^r]s rH set_cachezQdrantSemanticCache.set_cachesA&JK&*%G gi( (F "    **#'T:  'v.q1+> E %%%% djjl+' &$)     ''( d6J6J5K7SLL  rJc td||d}d}|D] }||dz } tttj|j |ddd}|dd d }|d d dd diddd}|j j|jd|jd|j|} | jd} | yt| trt| d k(ry| d d} | d dd} td|jd| d|d| | |jk\r1| d dd} td| d|d| |j!| Sy)!Nz.sync qdrant semantic-cache get_cache, kwargs: rUrVrWTrXrYr]rr^ quantizationF@ignorerescore oversamplingr rdparamslimit with_payloadr/points/searchrgrscorerer?&semantic cache: similarity threshold: , similarity: , prompt: , closest_cached_prompt: ragot a cache hit, similarity: , Current prompt: , cached_prompt: rK)rrrrir^r1r:postr8r/rr*r3listlenr0rQ)rBrkrmrUrnrorpr^r]search_responseresults similarity cached_prompt cached_values rH get_cachezQdrantSemanticCache.get_cachesFvhOP*%G gi( (F "    **#'T:  'v.q1+>  ##$'!   **//''( d6J6J5K>ZLL0  "&&(2 ? gt $7|q QZ(  9-f5  4T5N5N4O~^h]iistzs{|UVcUd e  22 2"1:i0P&P!::j"599."ML'1'<'<**#'T:$004 & :r : > >z4 P (= ( " (/'9'9**#'T:(" 'v.q1+> E %%%% djjl+' &$)    ##''( d6J6J5K7SLL$   a 6 ""2 sB9F FBFF-F5F6BF F FFFc Ktd|ddlm}m}|d}d}|D] }||dz } ||Dcgc]}|d c}ng} ||j| vrq|j dij d d} |j |j|d d d | d |j dij d dd d{} n-tj |j|d d d d{} | ddd} | ddd ddidd d} |jj|jd|jd|j| d{}|jd}|d|jdid<yt|t r$t#|dk(rd|jdid<y|dd}|ddd }td!|j$d"|d#|d$|||jdid<||j$k\r1|ddd%}td&|d'|d(||j'|)Sycc}w77X7w)*Nz/async qdrant semantic-cache get_cache, kwargs: rrrUrVrWrrrTrXrrrrYr]r^rsFrtrur ryrr}rgrgzsemantic-similarityr~rer?rrrrrarrrr)rrrrr1r=rrir<rr8r/rr* setdefaultr3rrr0rQ)rBrkrmrrrUrnrorrrrpr^r]rrrrrs rHasync_get_cachez#QdrantSemanticCache.async_get_cache]sGxPQI*%G gi( (F )'5 5nQ|_n 5   !d&:&:>P&P!::j"599."ML'1'<'<**#'T:$004 & :r : > >z4 P (= ( " (/'9'9**#'T:(" 'v.q1+>  ##$'!   !% 1 1 6 6''( d6J6J5K>ZLL!7!   "&&(2 ?GJF  j" -.C D gt $7|q KN!!*b12GHQZ(  9-f5  4T5N5N4O~^h]iistzs{|UVcUd e DN*b)*?@ 22 2"1:i0@X t$%rJr )__doc__rOrr*typingrrrilitellm._loggingrlitellm.constantsrrlitellm.types.utilsr base_cacher r rrJrHrs5  *H1!f%)f%rJ