j<dZddlZddlZddlZddlmZmZddlZddlm Z ddl m Z m Z ddl mZddlmZGd d eZy) z Qdrant Semantic Cache implementation Has 4 methods: - set_cache - get_cache - async_set_cache - async_get_cache N)Anycast) print_verbose)QDRANT_SCALAR_QUANTILEQDRANT_VECTOR_SIZE)EmbeddingResponse) BaseCachecReZdZ d dZdefdZdZdZdZdZ d Z d Z y) QdrantSemanticCacheNcddl}ddlm} m} m} ddlm} | td||_td|j| td||_ ||_ i} |r)t|tr|jdr| |}|r)t|tr|jdr| |}|xs$|jdxs|jd }|xs|jd }d d i} |r|| d <| t!d||_||_td|j"| |_| |_| | j*|_| td|j(j/|j"d|jd|j&}|j0dk7rt!d|j2|j5ddrn|j(j/|j"d|j|j&}|j5|_td|j6y||dk(rdddii}n*|dk(r ddt8ddi}n|d k(rd d!dd"i}n td#|j(j;|j"d|jt<d$d%|d&|j&'}|j5drn|j(j/|j"d|j|j&}|j5|_td(|j6ytd))*Nr)_get_httpx_clientget_async_httpx_clienthttpxSpecialProvider)get_secret_strz-collection_name must be provided, passed Nonez0qdrant semantic-cache initializing COLLECTION - z2similarity_threshold must be provided, passed Nonez os.environ/ QDRANT_URLQDRANT_API_BASEQDRANT_API_KEYz Content-Typezapplication/jsonzapi-keyzQdrant url must be providedz'qdrant semantic-cache qdrant_api_base: ) llm_providerzNQuantization config is not provided. Default binary quantization will be used. /collections/z/exists)urlheadersz1Error from qdrant checking if /collections exist resultexistsz.Collection already exists. Collection details:binary always_ramFscalarint8)typequantilerproductx16) compressionrzBQuantization config must be one of 'scalar', 'binary' or 'product'Cosine)sizedistance)vectorsquantization_config)rjsonrz+New collection created. Collection details:z#Error while creating new collection)os&litellm.llms.custom_httpx.http_handlerrrrlitellm.secret_managers.mainr Exceptioncollection_namersimilarity_thresholdembedding_model isinstancestr startswithgetenv ValueErrorqdrant_api_baseqdrant_api_keyr sync_clientCaching async_clientget status_codetextr*collection_inforputr)selfr7r8r/r0r)r1 host_typer+rrrrrcollection_existscollection_detailsquantization_paramsnew_collection_statuss ~/Users/manta/Documents/Projects/TheRoad-I1/backend/.venv/lib/python3.12/site-packages/litellm/caching/qdrant_semantic_cache.py__init__zQdrantSemanticCache.__init__s   @  "KL L.>t?S?S>T U  'PQ Q$8!. /3/O4N4N5#1"A .#.>3L3L4"0!?  Vryy6 V"))DU:V (F2995E+F!#56 !/GI   ":; ;.,?@T@T?UVW ,.2-55   & ` !,,00''( d6J6J5K7SLL1   ( (C /CDUDZDZC[\   ! ! #H -h 7!%!1!1!5!5++,M$:N:N9OP "6" $6#:#:# %9% !%))+H5%)%5%5%9%9//0 d>R>R=ST LL&:&"(:'>'>'@$B4CWCWBXY EFFcached_responsec||S tj|}|S#t$rtj|}Y|SwxYwN)r*loadsr.ast literal_eval)rArJs rG_get_cache_logicz$QdrantSemanticCache._get_cache_logicsV  "" " @"jjO  @!..?O @s ??c td|ddlm}|d}d}|D] }||dz } ttt j |j|ddd }|d dd } t|}t|tsJd t|j| ||d dgi} |jj|jd|jd|j| y)Nz)qdrant semantic-cache set_cache, kwargs: ruuidmessagescontentTzno-storezno-cachemodelinputcachedata embeddingpointsr>responseidvectorpayloadr/pointsrrr*)r litellm._uuidrSrrlitellmr]r1r3r2uuid4r9r@r7r/r) rAkeyvaluekwargsrSrTpromptmessageembedding_responser]r\s rG set_cachezQdrantSemanticCache.set_cachesA&JK&*%G gi( (F "    **#'T:  'v.q1+> E %%%% djjl+' &$)     ''( d6J6J5K7SLL  rIc td||d}d}|D] }||dz } tttj|j |ddd}|dd d }|d d dd diddd}|j j|jd|jd|j|} | jd} | yt| trt| d k(ry| d d} | d dd} td|jd| d|d| | |jk\r1| d dd} td| d|d| |j!| Sy)!Nz.sync qdrant semantic-cache get_cache, kwargs: rTrUrVTrWrXr\rr] quantizationF@ignorerescore oversamplingr rcparamslimit with_payloadr/points/searchrfrscorerdr>&semantic cache: similarity threshold: , similarity: , prompt: , closest_cached_prompt: r`got a cache hit, similarity: , Current prompt: , cached_prompt: rJ)rrrrhr]r1r9postr7r/rr*r2listlenr0rP)rArjrlrTrmrnror]r\search_responseresults similarity cached_prompt cached_values rG get_cachezQdrantSemanticCache.get_cachesFvhOP*%G gi( (F "    **#'T:  'v.q1+>  ##$'!   **//''( d6J6J5K>ZLL0  "&&(2 ? gt $7|q QZ(  9-f5  4T5N5N4O~^h]iistzs{|UVcUd e  22 2"1:i0P&P!::j"599."ML'1'<'<**#'T:$004 & :r : > >z4 P (= ( " (/'9'9**#'T:(" 'v.q1+> E %%%% djjl+' &$)    ##''( d6J6J5K7SLL$   a 6 ""2 sB9F FBFF-F5F6BF F FFFc Ktd|ddlm}m}|d}d}|D] }||dz } ||Dcgc]}|d c}ng} ||j| vrq|j dij d d} |j |j|d d d | d |j dij d dd d{} n-tj |j|d d d d{} | ddd} | ddd ddidd d} |jj|jd|jd|j| d{}|jd}|d|jdid<yt|t r$t#|dk(rd|jdid<y|dd}|ddd }td!|j$d"|d#|d$|||jdid<||j$k\r1|ddd%}td&|d'|d(||j'|)Sycc}w77X7w)*Nz/async qdrant semantic-cache get_cache, kwargs: rrrTrUrVrrrTrWrrrrXr\r]rrFrsrtr rxrr|rfrgzsemantic-similarityr}rdr>r~rrrr`rrrr)rrrrr1r<rrhr;rr7r/rr* setdefaultr2rrr0rP)rArjrlrrrTrmrnrrrror]r\rrrrrs rGasync_get_cachez#QdrantSemanticCache.async_get_cacheYsGxPQI*%G gi( (F )'5 5nQ|_n 5   !d&:&:>P&P!::j"599."ML'1'<'<**#'T:$004 & :r : > >z4 P (= ( " (/'9'9**#'T:(" 'v.q1+>  ##$'!   !% 1 1 6 6''( d6J6J5K>ZLL!7!   "&&(2 ?GJF  j" -.C D gt $7|q KN!!*b12GHQZ(  9-f5  4T5N5N4O~^h]iistzs{|UVcUd e DN*b)*?@ 22 2"1:i0@X t$%rIr )__doc__rNrr*typingrrrhlitellm._loggingrlitellm.constantsrrlitellm.types.utilsr base_cacher r rrIrGrs5  *H1!b%)b%rI