iz ddlZddlZddlmZmZmZmZmZmZm Z m Z ddl Z ddl m Z mZmZmZddlZddlmZddlmZddlmZddlmZddlmZdd lmZdd lmZdd l m!Z!eZ"Gd d eZ#de e jHe%fde%fdZ& d)de'de'de'de'dege'ff dZ( d)de'de'de'de'dege'ff dZ) d*de'dee'dee'dee'de'f dZ*de%fdZ+dedee'fd Z,Gd!d"eZ-Gd#d$eZ. d*d%ee'd&ee'd'ee'de.fd(Z/y)+N)AnyCallableDictLiteral NamedTupleOptionalUnioncast)AsyncAzureOpenAI AsyncOpenAI AzureOpenAIOpenAI)verbose_logger) DualCache)BaseLLMException) BaseOpenAILLM)get_azure_ad_token_provider)get_secret_str)GenericLiteLLMParams)_add_path_to_api_basec eZdZ ddeej deej deeeje fdee ffd Z xZ S)AzureOpenAIErrorrequestresponseheadersbodyc0t|||||||y)N) status_codemessagerrrr)super__init__)selfrrrrrr __class__s p/Users/manta/Documents/Projects/TheRoad-I1/.venv/lib/python3.12/site-packages/litellm/llms/azure/common_utils.pyr!zAzureOpenAIError.__init__s) #  )NNNN) __name__ __module__ __qualname__rhttpxRequestResponser Headersdictr! __classcell__)r#s@r$rrsl ,0-18<# %--(  5>>*  % t 345  tn  r%rrreturnci}d|vr|d|d<d|vr|d|d<d|vr|d|d<d|vr|d|d<|jDcic]\}}djd||}}}i||Scc}}w)Nzx-ratelimit-limit-requestszx-ratelimit-remaining-requestszx-ratelimit-limit-tokenszx-ratelimit-remaining-tokensz{}-{} llm_provider)itemsformat)ropenai_headerskvllm_response_headerss r$process_azure_headersr8+sN#w.7> (8 34(72;B ,< 78"W,5<=W5X12%09@ *: 56:A9HA~q)1, 6" 5n 55 sA, tenant_id client_id client_secretscopecddlm}m}tjd|j dr t |}n|}|j dr t |}n|}|j dr t |}n|}tjd|||||| td||||} tjd| || |} tjd| | S) z Get Azure AD token provider from `client_id`, `client_secret`, and `tenant_id` Args: tenant_id: str client_id: str client_secret: str scope: str Returns: callable that returns a bearer token. r)ClientSecretCredentialget_bearer_token_providerz$Getting Azure AD Token from Entra IDz os.environ/z,tenant_id %s, client_id %s, client_secret %sz8tenant_id, client_id, and client_secret must be provided credential %stoken_provider %s)azure.identityr>r?rdebug startswithr ValueError) r9r:r;r<r>r? _tenant_id _client_id_client_secret credentialtoken_providers r$ get_azure_ad_token_from_entra_idrKBs$Q?@M*#I.  M*#I.   .' 6&6  Z/>3ISTT' JOJ*5.z5AN,n= r%azure_usernameazure_passwordcddlm}m}tjd|||||||}tjd||||}tjd||S)a Get Azure AD token provider from `client_id`, `azure_username`, and `azure_password` Args: client_id: str azure_username: str azure_password: str scope: str Returns: callable that returns a bearer token. r)UsernamePasswordCredentialr?z2client_id %s, azure_username %s, azure_password %s)r:usernamepasswordr@rA)rBrOr?rrC)r:rLrMr<rOr?rIrJs r$)get_azure_ad_token_from_username_passwordrRzsl$U<  ,J *5.z5AN,n= r%azure_ad_tokenazure_client_idazure_tenant_idc |d}tjdd}|xstjd}|xstjd}|| tdd t|}| td d t j ||||d }t j|}||Stj}|j|d |d|d|d|d} | jdk7r!t| j| j | j } | jdd}| jdd} | tdd | tdd t j||| |S)z Get Azure AD token from OIDC token Args: azure_ad_token: str azure_client_id: Optional[str] azure_tenant_id: Optional[str] scope: str Returns: `azure_ad_token_access_token` - str N,https://cognitiveservices.azure.com/.defaultAZURE_AUTHORITY_HOSTz!https://login.microsoftonline.comAZURE_CLIENT_IDAZURE_TENANT_IDiz/AZURE_CLIENT_ID and AZURE_TENANT_ID must be set)rriz6OIDC token could not be retrieved from secret manager.)rTrUazure_authority_host oidc_token/z/oauth2/v2.0/tokenclient_credentialsz6urn:ietf:params:oauth:client-assertion-type:jwt-bearer)r: grant_typer<client_assertion_typeclient_assertion)data access_token expires_inz(Azure AD Token access_token not returnedz&Azure AD Token expires_in not returned)keyvaluettl)osgetenvrrjsondumpsazure_ad_cache get_cachelitellmmodule_level_clientpostrtextget set_cache) rSrTrUr<r[r\azure_ad_token_cache_keyazure_ad_token_access_tokenclient req_tokenazure_ad_token_jsonazure_ad_token_expires_ins r$get_azure_ad_token_from_oidcr{s$ }>99 C&E3D)EO%E3D)EO/"9E   /JL  $zz..$8$   #1":":;S"T".**  ( (F  /!22DE(.%] *   I#!--NN  $..*"5"9"9.$"O 3 7 7 d K"*%O  !(%M   $) % '&r%azure_client_paramscb|jdd}|d|vr||d<|jd|S)Nazure_endpointz/openai/deploymentsbase_url)rspop)r|r~s r$!select_azure_base_url_or_endpointrsC(,,-=tDN! N 2.<  +  # #$4 5 r%litellm_paramsc(|jd}|jdxs td}|jdxstjd}|jdxstjd}|jdxstjd }|jd xstjd }|jd xstjd }|jdxstjdd}|d}|*|r(|r&|r$t j dt ||||}|*|r(|r&|r$t j dt||||}|r:|r8|r6|jdr%t j dt||||}nP|Ntjdur} t jdt!| t/d"t!| | d} ~ wwxYw)#a Get Azure AD token from various authentication methods. This function tries different methods to obtain an Azure AD token: 1. From an existing token provider 2. From Entra ID using tenant_id, client_id, and client_secret 3. From username and password 4. From OIDC token 5. From a service principal with secret workflow 6. From DefaultAzureCredential Args: litellm_params: Dictionary containing authentication parameters - azure_ad_token_provider: Optional callable that returns a token - azure_ad_token: Optional existing token - tenant_id: Optional Azure tenant ID - client_id: Optional Azure client ID - client_secret: Optional Azure client secret - azure_username: Optional Azure username - azure_password: Optional Azure password Returns: Azure AD token as string if successful, None otherwise azure_ad_token_providerrSAZURE_AD_TOKENr9rZr:rYr;AZURE_CLIENT_SECRETrLAZURE_USERNAMErMAZURE_PASSWORD azure_scope AZURE_SCOPErWN:Using Azure AD Token Provider from Entra ID for Azure Authr9r:r;r<0Using Azure Username and Password for Azure AuthrLrMr:r<oidc/%Using Azure OIDC Token for Azure AuthrSrTrUr<TzvUsing Azure AD token provider based on Service Principal with Secret workflow or DefaultAzureCredential for Azure Authr*Azure AD Token Provider could not be used.z'Error calling Azure AD token provider: zm. Follow docs - https://docs.litellm.ai/docs/providers/azure/#azure-ad-token-refresh---defaultazurecredential)r<z3Azure AD token provider returned non-string value: z%Azure AD token must be a string, got zFailed to get Azure AD token: )rsrrirjrrCrKrRrDr{roenable_azure_ad_token_refreshrrE Exceptionerrorstr BaseAzureLLM*_try_get_default_azure_credential_providercallable isinstancetype TypeError RuntimeError) rrrSr9r:r;rLrMr<etokens r$get_azure_ad_tokenr s3:-001JK#''(89^>N"";/O299=N3OI"";/O299=N3OI"&&7299<M$''(89XRYYGW=XN#''(89XRYYGW=XN   } - E2E }>&9} H #C' #  '   OP"K)) #      % %g .DE5)%%   '  1 1T 9 E  &Ae&T # # *~/EGGH $8,C#D Q+-EeS)$$I$u+W "GU } UVV!& >K O  !M N   9#a&Bop G  8   Q  #J3q6(!S T!?AxHIq P Qs7# I-AK-J> J>$J99J>L9L  LcXeZdZededeegeffdZ d!deedeedeedeee e e e fd ee d ed eedeee e e e ffd Zd e deedeed eedeedede fdZ d"ded edededeeej(fd e deedeedeegefdedeee e fdee e ffdZede d eede fdZe d"deed eeeeeeffdeedefdeeeedfdef dZedeedefdZd eeefdededeefd Zy)#rr<r/cddlm}tjd t ||j }tjd|S#t $r+}tjdt|Yd}~yd}~wwxYw)z Try to get DefaultAzureCredential provider Args: scope: Azure scope for the token Returns: Token provider callable if DefaultAzureCredential is enabled and available, None otherwise r)AzureCredentialTypez7Attempting to use DefaultAzureCredential for Azure Auth)razure_credentialzJSuccessfully obtained Azure AD token provider using DefaultAzureCredentialzDefaultAzureCredential failed: N)9litellm.types.secret_managers.get_azure_ad_token_providerrrrCrDefaultAzureCredentialrr)r<rrrs r$rz7BaseAzureLLM._try_get_default_azure_credential_providers|  VW &A!!4!K!K' #  \ + *   #B3q6(!K L s-A A?!A::A?Napi_keyapi_base api_versionrwr _is_asyncmodelcd}t} || d<||j| d} | r"t| ttt t fr| S|j|xsi|||||} |j|rr| jd|dd} d| vr| d| d<d | vr| d | d <d | vr| d | d <tjd | d |d ur t di| }nxt di| }nl|d ur t di| }n\tdi| }nP|}|Lt|ttfr6t|jtr|jjd||j|| d|S)Nis_asyncazure)client_initialization_params client_type)rrr model_namerrrz /openai/v1/)rrtimeout max_retries http_clientz"Using Azure v1 API with base_url: rT api-version) openai_clientrr)localsget_cached_openai_clientrr r rr initialize_azure_sdk_client_is_azure_v1_api_versionrsrrC _custom_queryr- setdefaultset_cached_openai_client) r"rrrrwrrrrr cached_clientr| v1_paramss r$get_azure_openai_clientz$BaseAzureLLM.get_azure_openai_clients  .4X$3<$Z0 > 99-I#:M!K1A6;#W)("&"B"B-3! '" #C# ,,[9 366yA#+*K 8  33+>y+IIi( $77/B=/QIm, $77/B=/QIm,$$8:9N8OP$$/$<)$ .. ,.> %%nm]S =BE$((7  $$Y/'/  L 'G##+ ' # $ +  !S T&O--# ' #  %.*C*CG*L  !H I9- ) ) N'/55=  n  S*E %+'  ))#W%F%FK  Jx$=|h.H3J<|CPXM?Zdemdn o &&,'>   t 151L1L1N  .151K1K1M  .  "1<  .  -4  * " .=T  9 :@ 3 #"S S$$%QR Ss9 II%$I%rrrSr acompletionc |jdtjd} |jdtjd} |jdtjdd}| |jds|dz }||z }||tj ||d }|||d <n'|%|j d rt|| | | }||d <| | |d<| dur tdi|} | Stdi|} | S)Nr9rZr:rYrrrWr])rrrrrrrrrSrTr) rsrirjendswithroclient_sessionrDr{r r )r"rrrrrrrrSrrrwr9r:r<r|s r$,_init_azure_client_for_cloudflare_ai_gatewayz9BaseAzureLLM._init_azure_client_for_cloudflare_ai_gateways9#&&{BII>O4PQ "&&{BII>O4PQ ""  IIm%S T  >$$S)C 5' "H +'j&55*" 3 "18#I.+!,,W5%A'5(1(1# &N9G#$45&2AX#$=>d")@,?@ %;':; r%rc|xs t}d|vr|S|jxs<tjxs*tjxst dxs t d}|r||d<|St |}|rd||d<|S)Nzapi-keyAZURE_OPENAI_API_KEY AZURE_API_KEYzBearer Authorization)rrro azure_keyrr)rrrrSs r$ _base_validate_azure_environmentz-BaseAzureLLM._base_validate_azure_environments(A+?+A  N  " " / /   /45 /o.  !(GI N,N; )00@'AGO $r%route)z/openai/responsesz/openai/vector_storesdefault_api_version)latestpreviewc~|xstjxs td}|td|dt j |}|xsi}t tt|jdxs|}t|j}d|vr|r||d<||vrt||}n|}tj|rNd|vrJt j |}t|j|j j#dd }t j |j| } t| S) a} Get the base Azure URL for the given route and API version. Args: api_base: The base URL of the Azure API. litellm_params: The litellm parameters. route: The route to the API. default_api_version: The default API version to use if no api_version is provided. If 'latest', it will use `openai/v1/...` route. AZURE_API_BASEz^api_base is required for Azure AI Studio. Please set the api_base parameter. Passed `api_base=`rr)r ending_pathz /openai/v1z/openai)path)params)rorrrEr)URLr rrrsr-rrrr copy_withrreplace) rrrr original_urlr query_paramsnew_url parsed_url final_urls r$_get_base_azure_urlz BaseAzureLLM._get_base_azure_urlsC"Sw//S>BR3S  pqypzz{| yy* (-2 # 2 2= A B #"  L//0   ,*5L '  +X5QGG  0 0 =7*"YYw/ (('__44Y M)IIg&00 0E 9~r%c|y|dvS)NF>v1rrr)rs r$rz%BaseAzureLLM._is_azure_v1_api_versions  999r% param_key env_var_keycV|j|}||Stj|S)a!Resolve the environment variable for a given parameter key. The logic here is different from `params.get(key, os.getenv(env_var))` because litellm_params may contain the key with a None value, in which case we want to fallback to the environment variable. )rsrirj)r"rrr param_values r$rzBaseAzureLLM._resolve_env_var"s/%((3  " yy%%r%)NNNFN)N)r&r'r( staticmethodrrrrr r r rr r-boolrrintfloatr)Timeoutrrrrrrrrrrr%r$rrs (2s7# $F&* )-#S#S3-Sc] S  +/D E S!SS}S % %5v{JK LSj####3- # SM # c] ## #ZBF444 4  4 uemm+, 44#4! 4"*(2s7*;!<44{,<<=>4 {,, -4l'/0D'E : SW 83-8 ';T#s(^'K!LM8WIJCOP8&eC9L1M,M&NO 8 88t:hsm::: &"38n &9< &KN & # &r%rc>eZdZUeeed<eeed<eeed<y)AzureCredentialsrrrN)r&r'r(rr__annotations__rr%r$rr1s sm c]#r%rrrrc"|xstjxs td}|xstjxs td}|xs<tjxs*tj xstdxs td}t |||S)zEResolve Azure credentials from params, litellm globals, and env vars.rrrr)rrr)rorrrrrr)rrrresolved_api_baseresolved_api_versionresolved_api_keys r$get_azure_credentialsr7s !XG$4$4XGW8XQw**Qn=P.Q  + ?? +    + 0 1 + / *  " ( r%)rW)NNN)0rkritypingrrrrrrr r r)openair r r rrolitellm._loggingrlitellm.caching.cachingr)litellm.llms.base_llm.chat.transformationr litellm.llms.openai.common_utilsr3litellm.secret_managers.get_azure_ad_token_providerrlitellm.secret_managers.mainrlitellm.types.routerr litellm.utilsrrmrr,r-r8rrKrRr{rrrrrrr%r$rs RRR EE+-F:85/ ' (65)<#=6$66@ 5555  5 b#g 5x@ &&&&  & b#g &V&*%) ['['c]['c][' C= ['  ['| 4 F(F c]FRX&=X&v z#!!%sm c]# r%