iq fddlmZmZddlmZddlmZddlmZm Z ddl ddl m Z GddZ y ) )AnyDict)uuid)+_convert_vertex_datetime_to_openai_datetime)BatchJobStatusCreateBatchRequest)*) LiteLLMBatchceZdZdZededefdZedede fdZ ede e e fde e e ffdZedede fdZedede fd Zedede fd Zededefd Zed e de fd Zede de fdZy)VertexAIBatchTransformationz Transforms OpenAI Batch requests to Vertex AI Batch requests API Ref: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/batch-prediction-gemini requestreturnc(dtj}|jd}| tdt t |gd}|j |}tdt|j|}t|||| S) zN Transforms OpenAI Batch requests to Vertex AI Batch requests zlitellm-vertex-batch- input_file_idz+input_file_id is required, but not provided)urisjsonl) gcsSourceinstancesFormat)outputUriPrefix)predictionsFormatgcsDestination) inputConfig outputConfigmodel displayName) ruuid4get ValueError InputConfig GcsSource_get_model_from_gcs_file OutputConfigGcsDestination_get_gcs_uri_prefix_from_fileVertexAIBatchPredictionJob)clsr request_display_namer input_configr output_configs ~/Users/manta/Documents/Projects/TheRoad-I1/.venv/lib/python3.12/site-packages/litellm/llms/vertex_ai/batches/transformation.py9transform_openai_batch_request_to_vertex_ai_batch_requestzUVertexAIBatchTransformation.transform_openai_batch_request_to_vertex_ai_batch_requests"7tzz|nE O4  JK K$/m_5w% 11-@&2%) # A A- P' *$&,   responsec t|j|dt|jddd|j |d|j |d|j | S)N24hrs createTime)vertex_datetimebatch) idcompletion_window created_atendpointrobjectstatus error_file_idoutput_file_id)r +_get_batch_id_from_vertex_ai_batch_responserr0_get_input_file_id_from_vertex_ai_batch_response3_get_batch_job_status_from_vertex_ai_batch_response1_get_output_file_id_from_vertex_ai_batch_response)r&r-s r*;transform_vertex_ai_batch_response_to_openai_batch_responsezWVertexAIBatchTransformation.transform_vertex_ai_batch_response_to_openai_batch_response0su>>xH%B ( \2 >NNJJ8TPP  r,c6|jdgxsg}|Dcgc]}|j|}}t|dkDr|djnd}t|dkDr|djnd}|jd}d|||t ||dScc}w)z` Transforms Vertex AI batch list response into OpenAI-compatible list response. batchPredictionJobsrN nextPageTokenlist)r8datafirst_idlast_idhas_morenext_page_token)rr@lenr4bool)r&r- batch_jobsjobrFrGrHrJs r*?transform_vertex_ai_batch_list_response_to_openai_list_responsez[VertexAIBatchTransformation.transform_vertex_ai_batch_list_response_to_openai_list_responseFs\\"7<B " !  K KC P!  "%TQ47::D!$TQ$r(++D",,7 _-.    sBc`|jdd}|sy|jd}|r|dS|S)z Gets the batch id from the Vertex AI Batch response safely vertex response: `projects/510528649030/locations/us-central1/batchPredictionJobs/3814889423749775360` returns: `3814889423749775360` namer1/rC)rsplit)r&r-_namepartss r*r<zGVertexAIBatchTransformation._get_batch_id_from_vertex_ai_batch_responseas; VR( C !uRy,u,r,cd}|jd}||S|jd}||S|jdd}t|dk(r|S|dS)zJ Gets the input file id from the Vertex AI Batch response r1rrrr)rrK)r&r-rr( gcs_sourcers r*r=zLVertexAIBatchTransformation._get_input_file_id_from_vertex_ai_batch_responsessi ||M2   !%%k2   ~~fb) t9> Awr,cZ|jdtjdd}|r|jddz}|r|dk7r|S|jd}||S|jd}||S|jdd}|jdr|S|jddzS) zK Gets the output file id from the Vertex AI Batch response outputInfogcsOutputDirectoryr1rRz/predictions.jsonlrrr)r OutputInforstripendswith)r&r-r;r)gcs_destinationoutput_uri_prefixs r*r?zMVertexAIBatchTransformation._get_output_file_id_from_vertex_ai_batch_responses'll<FJJ "  +2237:NNN n0DD! ! ^4  ! !'++,<=  "! !+//0A2F  % %&: ;$ $ '',/CCCr,c Nddddddddddddd }|jd d }||S) z Gets the batch job status from the Vertex AI Batch response ref: https://cloud.google.com/vertex-ai/docs/reference/rest/v1/JobState failed validating in_progress completed cancelling cancelledexpired) JOB_STATE_UNSPECIFIEDJOB_STATE_QUEUEDJOB_STATE_PENDINGJOB_STATE_RUNNINGJOB_STATE_SUCCEEDEDJOB_STATE_FAILEDJOB_STATE_CANCELLINGJOB_STATE_CANCELLEDJOB_STATE_PAUSEDJOB_STATE_EXPIREDJOB_STATE_UPDATINGJOB_STATE_PARTIALLY_SUCCEEDEDstaterh)r)r&r- state_mapping vertex_states r*r>zOVertexAIBatchTransformation._get_batch_job_status_from_vertex_ai_batch_responsesM&. ,!-!.#. ($0#. -!*"/-8 4  ||G-DE \**r,rc0|jdd}|dS)aK Gets the gcs uri prefix from the input file id Example: input_file_id: "gs://litellm-testing-bucket/vtx_batch.jsonl" returns: "gs://litellm-testing-bucket" input_file_id: "gs://litellm-testing-bucket/batches/vtx_batch.jsonl" returns: "gs://litellm-testing-bucket/batches" rRr)rsplit)r&r path_partss r*r$z9VertexAIBatchTransformation._get_gcs_uri_prefix_from_files #))#q1 !}r, gcs_file_uricddlm}||}|jdd}|jd}ddj|dd}|S)a Extracts the model from the gcs file uri When files are uploaded using LiteLLM (/v1/files), the model is stored in the gcs file uri Why? - Because Vertex Requires the `model` param in create batch jobs request, but OpenAI does not require this gcs_file_uri format: gs://litellm-testing-bucket/litellm-vertex-files/publishers/google/models/gemini-1.5-flash-001/e9412502-2c91-42a6-8e61-f5c294cc0fc8 returns: "publishers/google/models/gemini-1.5-flash-001" r)unquotez publishers/rxrRN) urllib.parser}rSjoin)r&r{r} decoded_uri model_pathrUrs r*r!z4VertexAIBatchTransformation._get_model_from_gcs_filesW )l+  &&}5a8   %chhuRay123 r,N)__name__ __module__ __qualname____doc__ classmethodrr%r+VertexBatchPredictionResponser r@rstrrrOr<r=r?rr>r$r!r,r*r r s]  #  $  8 4   * CH~ c3h  4-4- --"4 *D4D DD8+4+ ++4 # #  CCr,r N)typingrr litellm._uuidr#litellm.llms.vertex_ai.common_utilsrlitellm.types.llms.openairrlitellm.types.llms.vertex_ailitellm.types.utilsr r rr,r*rs(I*,ZZr,