~h!26ddlZddlZddlZddlZddlZddlZddlmZddlm Z ddl m Z m Z ddl mZddlmZddlmZmZdd lmZdd lmZdd lmZmZmZmZmZerdd lmZej@e!Z"d dgZ#dddddddddddddgdddddddZ$dZd Z%e Gd!d"Z&e Gd#d$Z'Gd%d&e(eZ)e Gd'd(Z*d)e(e+zd*e(e+zfd+Z,e Gd,d-Z-e Gd.d/Z.e)j^fd0e0e.d1e)d*e*fd2Z1e)jde)j^e)jfe)jhiZ5d3ed*e+fd4Z6d5e(d6e0e(d*e(fd7Z7iddfd8e0e*d9e+e)e)fe+e(e(fzd:e8d;e8d*e0e+e(eff d<Z9d=e(d>e(d?e(d*e'fd@Z:dAe(d*e8fdBZ;GdCdDZ<GdEdFe<Z=GdGdHe<Z>GdIdJe<Z?GdKdLe<Z@GdMdNe@ZAGdOdPeAZBGdQdRe@ZCGdSdTe@ZDeDZEGdUdVeDZFeFZGGdWdXe@ZHeHZIgdYZJy)[N) Generator)deepcopy)asdict dataclass)Enum)Thread) TYPE_CHECKINGAny) TokenUsage)Tool) RateLimiter_is_package_availableencode_image_base64make_image_urlparse_json_blob)StoppingCriteriaListcerebrasz fireworks-ai json_schemaFz4A free form text description of the thought process.Thoughtstring) descriptiontitletypez3Valid Python code snippet implementing the thought.Code)thoughtcoderrThoughtAndCodeAnswerobject)additionalProperties propertiesrequiredrrT)schemanamestrict)rrc"fd|S)Nct|dr9t|jDcic]\}}|k7s ||c}}S|Scc}}w)N__dataclass_fields__)hasattrritems)objkvconvert ignore_keys M/opt/mcp/mcp-sentiment/venv/lib/python3.12/site-packages/smolagents/models.pyr.z1get_dict_from_nested_dataclasses..convertCsI 3. /.4Sk.?.?.AUdaQ*_AwqzMU U Vs A  A )r+r/r.s `@r0 get_dict_from_nested_dataclassesr2Bs 3<c6eZdZUeed<eed<dZedzed<y)ChatMessageToolCallFunction argumentsr$Nr)__name__ __module__ __qualname__r __annotations__strrr1r3r0r5r5KsN I"Kt"r3r5c8eZdZUeed<eed<eed<defdZy)ChatMessageToolCallfunctionidrreturncd|jdt|jjdt|jjS)NzCall: z : Calling z with arguments: )r?r;r>r$r6selfs r0__str__zChatMessageToolCall.__str__XsDy 3t}}/A/A+B*CCTUXY]YfYfYpYpUqTrssr3N)r7r8r9r5r:r;rDr1r3r0r=r=Rs )) G Ittr3r=c0eZdZdZdZdZdZdZedZ y) MessageRoleuser assistantsystemz tool-callz tool-responsec@|Dcgc]}|jc}Scc}wN)value)clsrs r0roleszMessageRole.rolescs!$%A%%%sN) r7r8r9USER ASSISTANTSYSTEM TOOL_CALL TOOL_RESPONSE classmethodrOr1r3r0rFrF\s- DI FI#M&&r3rFc eZdZUeed<dZeeeee fzdzed<dZ ee dzed<dZ e dzed<dZ edzed<dZed dede dzdedzd dfd Zd Zd efd Zy) ChatMessageroleNcontent tool_callsraw token_usagecBtjt|dS)Nr[)r/)jsondumpsr2rBs r0model_dump_jsonzChatMessage.model_dump_jsonpszz:4ERSSr3datar@c |jdr5|dDcgc]"}ttd i|d|d|d$}}||d<||d|jd|jd||Scc}w) NrZr>r?rr>r?rrXrY)rXrYrZr[r\r1)getr=r5)rMrar[r\tcrZs r0 from_dictzChatMessage.from_dictss 88L ! |, $8J2j>JrRVx^`ag^hJ ",D fHHY'xx -#   s'A5ct|SrK)r2rBs r0dictzChatMessage.dicts /55r3c .t|jxsd}|jrj|dj|jDcgc]B}t j |j j|j jdDc}z }|Scc}w)N )toolr6) r;rYrZjoinr^r_r>r$r6)rCrenderedrls r0render_as_markdownzChatMessage.render_as_markdowns{t||$* ??  !%JJ (:(:I`I`ab H  sAB NN)r7r8r9rFr:rYr;listrhr rZr=r[r\r r`rUrfror1r3r0rWrWhs 15GS4S#X' '$ .537J()D07Ct%)Kd")T T d  UYHY er  "6 C r3rWr6r@ctt|tr|S tj|S#t$r|cYSwxYwrK) isinstancerhr^loads Exception)r6s r0parse_json_if_neededrvs;)T" ::i( (   s ) 77cbeZdZUdZdZedzed<dZedzed<dZ edzed<dZ e dzed<y)ChatMessageToolCallStreamDeltaz>Represents a streaming delta for tool calls during generation.Nindexr?rr>) r7r8r9__doc__ryintr:r?r;rr>r5r1r3r0rxrxs?HE3:Bd D#*37H)D07r3rxcPeZdZUdZedzed<dZeedzed<dZ e dzed<y)ChatMessageStreamDeltaNrYrZr\) r7r8r9rYr;r:rZrqrxr\r r1r3r0r}r}s4GS4Z>BJ34t;B%)Kd")r3r} stream_deltasrXc i}d}d}d}|D]}|jr2||jjz }||jjz }|jr||jz }|jsj|jD]d}|j H|j |vr:t |j|jtdd||j <||j }|jr|j|_|jr|j|_|js|jjrGt|jjdkDr%|jj|j_ |jjs$|jxj|jjz c_ Ytd|t|||j!D cgc]]} | jrOt#t| jj| jj| jxsdd_c} t%|| Scc} w) zI Agglomerate a list of stream deltas into a single stream delta. rjrr$r6r?rr>z/Tool call index is not provided in tool delta: r>rc input_tokens output_tokens)rXrYrZr\)r\rrrYrZryrxr?rr5r>r$lenr6 ValueErrorrWvaluesr=r ) r~rXaccumulated_tool_callsaccumulated_contenttotal_input_tokenstotal_output_tokens stream_deltatool_call_delta tool_calltool_call_stream_deltas r0agglomerate_stream_deltasrsT IK%j  # # ,":":"G"G G  <#;#;#I#I I     <#7#7 7   " "#/#:#: j"((4&,,4JJHf.11!0!5!5%@bTV%WI./D/DE !77L7L MI&))'6'9'9 &++)8)=)= &//*3388SAYAYA^A^=_bc=c6E6N6N6S6SI..3*33==%..88Or)rr!r")r$r parameters)rr>)rinputsr*appendr$r)rlr!r"keyrLs r0get_tool_json_schemars$++&JH &&(! U =E !$E&Me#j(9 OOC ! II++ ($   r3rYstop_sequencescX|D]$}|t| d|k(s|dt| }&|SrK)r)rYrstop_seqs r0remove_stop_sequencesr sA"0 CM># $ 0.X/G0 Nr3 message_listrole_conversionsconvert_images_to_image_urlsflatten_messages_as_textc vg}t|}|D]%}|j}|tjvr$t d|dtjd||vr |||_t |j tr|j D]}t |tsJdt|z|ddk(s/|r Jd||r8|jdd tt|jdid ut|d|d<t|d kDr|j|d d k(rt |j tsJdt|j z|r%|d dxxd|j d dzz cc<|j D]O}|ddk(r.|d dd ddk(r|d dd dxxd|dzz cc<9|d dj|Q|r|j d d} n |j } |j|j| d(|S)aw Creates a list of messages to give as input to the LLM. These messages are dictionaries and chat template compatible with transformers LLM chat template. Subsequent messages with the same role will be concatenated to a single message. Args: message_list (`list[dict[str, str]]`): List of chat messages. role_conversions (`dict[MessageRole, MessageRole]`, *optional* ): Mapping to convert roles. convert_images_to_image_urls (`bool`, default `False`): Whether to convert images to image URLs. flatten_messages_as_text (`bool`, default `False`): Whether to flatten messages as text. zIncorrect role z, only z are supported for now.z%Error: this element should be a dict:rimagez0Cannot use images with flatten_messages_as_text= image_urlurl)rrrrXzError: wrong content:rYrktext)rXrY)rrXrFrOrrsrYrqrhr;updaterrpoprr) rrrroutput_message_listmessagerXelementelrYs r0get_clean_message_listrsx 13L)L,|| {((* *tfGKvF$QSTZQ[J[[F+B/ :AA"E F(!//!,V4!//  & &#LL& O,Z r3r tool_name_keytool_arguments_keyc bt|\}} ||}|j |d}t |tr t|}tttjdt||S#t$r/}td|dt|j d|d}~wwxYw)NzKey tool_name_key=z1 not found in the generated tool call. Got keys: z insteadr>rr) rrurrqkeysrdrsr;rvr=uuiduuid4r5)rrrtool_call_dictionary_ tool_nameetool_argumentss r0get_tool_call_from_textrRs-d3!(7 *--.@$GN.#&-n=  tzz|  ,)~V  !=""STXYmYrYrYtTuSvv~   sA66 B.?*B))B.model_idc\|jdd}d}tj|| S)a[ Check if the model supports the `stop` parameter. Not supported with reasoning models openai/o3 and openai/o4-mini (and their versioned variants). Args: model_id (`str`): Model identifier (e.g. "openai/o3", "o4-mini-2025-04-16") Returns: bool: True if the model supports the stop parameter, False otherwise /rz^(o3[-\d]*|o4-mini[-\d]*)$)splitrematch)r model_namepatterns r0supports_stop_parameterrds0$R(J+Gxx, ,,r3ceZdZ ddededededzfdZededzfdZededzfd Z dd e e d e edzd e eefdzd e e dzde eefdzdedee zdzde eeffdZ dd e e d e edzd e eefdzd e e dzde f dZdZde de fdZde fdZede eefddfdZy)ModelNrrrrc f||_||_||_||_d|_d|_||_yrK)rrrkwargs_last_input_token_count_last_output_token_countr)rCrrrrrs r0__init__zModel.__init__ws;)A%*"4 37$48%$, r3r@cNtjdt|jS)NzAttribute last_input_token_count is deprecated and will be removed in version 1.20. Please use TokenUsage.input_tokens instead.)warningswarn FutureWarningrrBs r0last_input_token_countzModel.last_input_token_counts$  :  +++r3cNtjdt|jS)NzAttribute last_output_token_count is deprecated and will be removed in version 1.20. Please use TokenUsage.output_tokens instead.)rrrrrBs r0last_output_token_countzModel.last_output_token_counts$  ;  ,,,r3messagesrresponse_formattools_to_call_fromcustom_role_conversionsr tool_choicec t|jd|j} t||xst|| } i|jd| i} |t |j xsdr|| d<||| d<|r3d|D cgc] } t| c} i} ||| d<| j| | j|| Scc} w) a$ Prepare parameters required for model invocation, handling parameter priorities. Parameter priority from high to low: 1. Explicitly passed kwargs 2. Specific parameters (stop_sequences, response_format, etc.) 3. Default values in self.kwargs r)rrrrrjstoprtoolsr) rrrtool_role_conversionsrrrrr)rCrrrrrrrrrmessages_as_dictscompletion_kwargsrl tools_configs r0_prepare_completion_kwargsz Model._prepare_completion_kwargss($*::.H$JgJg#h 2 4M8M)E%=   kk )   %&t}}':;,:!&)  &3B / 0 AST.t4TL&.9 ]+  $ $\ 2   (  Us4B5c td)aProcess the input messages and return the model's response. Parameters: messages (`list[dict[str, str | list[dict]]] | list[ChatMessage]`): A list of message dictionaries to be processed. Each dictionary should have the structure `{"role": "user/system", "content": "message content"}`. stop_sequences (`List[str]`, *optional*): A list of strings that will stop the generation if encountered in the model's output. response_format (`dict[str, str]`, *optional*): The response format to use in the model's response. tools_to_call_from (`List[Tool]`, *optional*): A list of tools that the model can use to generate responses. **kwargs: Additional keyword arguments to be passed to the underlying model. Returns: `ChatMessage`: A chat message object containing the model's response. z0This method must be implemented in child classesNotImplementedError)rCrrrrrs r0generatezModel.generates2""TUUr3c&|j|i|SrK)r)rCargsrs r0__call__zModel.__call__st}}d-f--r3rctj|_|jsD|jJdt |j|j |jg|_t|jdkDsJd|jD]0}t|jj|j_ 2|S)zXSometimes APIs do not return the tool call as a specific object, so we need to parse it.z-Message contains no content and no tool callsrz*No tool call was found in the model output) rFrQrXrZrYrrrrrvr>r6)rCrrs r0parse_tool_callszModel.parse_tool_callss",, !!??. _0_ _.'9K9KTMdMde"G 7%%&*X,XX* ++ ^I+? @R@R@\@\+]I   ( ^r3ci|jd|ji}dD]}t||st||||< ddg}|D]}t||st d|d |S)zG Converts the model into a JSON-compatible dictionary. r) custom_role_conversion temperature max_tokensprovidertimeoutapi_base torch_dtype device_map organizationprojectazure_endpointtokenapi_keyz,For security reasons, we do not export the `z5` attribute of your model. Please export it manually.)rrr)getattrprint)rCmodel_dictionary attributedangerous_attributesattribute_names r0to_dictz Model.to_dicts kk      GItY'.5dI.F + G !(32 Nt^,B>BRSHI   r3rc b|di|jDcic]\}}|| c}}Scc}}w)Nr1)r*)rMrr,r-s r0rfzModel.from_dicts0A'7'='='?@tq!ad@AA@s +)Fr$r6N)NNNNFr"NNN)r7r8r9boolr;rpropertyr{rrrqrWrhr r rrrrrrUrfr1r3r0rrvs*/#"-# -"&-- - * - ,d ,,-t--,015049=-2)35!{#5!S D(5!c3h$. 5! !J- 5! "&c3h$!6 5!'+5!4Z$&5! c3h5!t,01504 V{#VS D(Vc3h$. V !J- V V6.     @Bc3hBGBBr3rc eZdZdZ d deeefdzffd ZdZ d de e de edzdeeefdzd e e dzd e f d Z xZ S) VLLMModelaModel to use [vLLM](https://docs.vllm.ai/) for fast LLM inference and serving. Parameters: model_id (`str`): The Hugging Face model ID to be used for inference. This can be a path or model identifier from the Hugging Face model hub. model_kwargs (`dict[str, Any]`, *optional*): Additional keyword arguments to pass to the vLLM model (like revision, max_model_len, etc.). N model_kwargsc tds tdddlm}ddlm}|xsi|_t| di|||_ |dd|i|j |_ |jJ|||_ d|_ y) NvllmzNPlease install 'vllm' extra to use VLLMModel: `pip install 'smolagents[vllm]'`r)LLM) get_tokenizermodelFr1) rModuleNotFoundErrorrr!vllm.transformers_utils.tokenizerrrsuperrrr tokenizer_is_vlm)rCrrrrr __class__s r0rzVLLMModel.__init__+s %V,%&vw wC(.B "6"  =x=4+<+<= zz%%%&x0 r3cddl}ddl}ddlm}m}||j |j j j`|j||jjy)Nr)destroy_distributed_environmentdestroy_model_parallel) gctorchvllm.distributed.parallel_staterrr llm_enginemodel_executor driver_workercollectcuda empty_cache)rCrrrrs r0cleanupzVLLMModel.cleanup?sT  :: ! %%44B ')  r3rrrrr@c |ddlm}|jd||j ||d|}|r d|ddind}|j d}|j dg} |j d d} |j d d|j j || d d } ||jdd|jdd|jdd| } |jj| | |} | djdj}t| dj|_t| djdj|_t#t$j&|||dt)t| djt| djdjS)Nr)SamplingParams)rrrr guided_jsonrr#rrrrTF)radd_generation_prompttokenizenr rgri)r rrr)sampling_paramsguided_options_requestoutrrrXrYr[r\r1)rrrr rr apply_chat_templaterdrroutputsrrprompt_token_idsr token_idsrrWrFrQr )rCrrrrrrrr"prepared_stop_sequencesrpromptr!r$ output_texts r0rzVLLMModel.generatePs (;D;; *.,,&6)1    _n-1OPX1Y!Zsw$((4"3"7"7"C!%%gt4mT233 "& 4 )jja  =#6zz,5(  jj!! +#9"  !fnnQ',, '*3q6+B+B'C$(+CFNN1,=,G,G(H%&&#:KL" Q!8!89!#a&.."3"="=>   r3rKr)r7r8r9rzrhr;r rrrqrWr r __classcell__r s@r0rr s/338nt+(!(,01504 8 {#8 S D(8 c3h$. 8 !J- 8  8 r3rc eZdZdZ ddededeeefdzdeeefdzffd Z dde e d e edzd eeefdzd e e dzd e f d Z xZ S)MLXModelaA class to interact with models loaded using MLX on Apple silicon. > [!TIP] > You must have `mlx-lm` installed on your machine. Please run `pip install smolagents[mlx-lm]` if it's not the case. Parameters: model_id (str): The Hugging Face model ID to be used for inference. This can be a path or model identifier from the Hugging Face model hub. tool_name_key (str): The key, which can usually be found in the model's chat template, for retrieving a tool name. tool_arguments_key (str): The key, which can usually be found in the model's chat template, for retrieving tool arguments. trust_remote_code (bool, default `False`): Some models on the Hub require running remote code: for this model, you would have to set this flag to True. load_kwargs (dict[str, Any], *optional*): Additional keyword arguments to pass to the `mlx.lm.load` method when loading the model and tokenizer. apply_chat_template_kwargs (dict, *optional*): Additional keyword arguments to pass to the `apply_chat_template` method of the tokenizer. kwargs (dict, *optional*): Any additional keyword arguments that you want to use in model.generate(), for instance `max_tokens`. Example: ```python >>> engine = MLXModel( ... model_id="mlx-community/Qwen2.5-Coder-32B-Instruct-4bit", ... max_tokens=10000, ... ) >>> messages = [ ... { ... "role": "user", ... "content": "Explain quantum mechanics in simple terms." ... } ... ] >>> response = engine(messages, stop_sequences=["END"]) >>> print(response) "Quantum mechanics is the branch of physics that studies..." ``` Nrtrust_remote_code load_kwargsapply_chat_template_kwargsc tds tdddl}|xsi|_|jj dij d||xsi|_|j j ddt |d |dd||j|jfi|j\|_ |_ |j|_ d |_ y) Nmlx_lmzSPlease install 'mlx-lm' extra to use 'MLXModel': `pip install 'smolagents[mlx-lm]'`rtokenizer_configr1rT)rrFr1)rrr5r2 setdefaultr3r rloadrrr stream_generateis_vlm)rCrr1r2r3rr5r s r0rzMLXModel.__init__s%X.%e  &," ##$6;FFGZ\mn*D*J' ''223JDQ T(TTVT%0V[[%S$BRBR%S" DN%55 r3rrrrr@c  | td|jd|||d|}|jd}|jdg}|jdd}|jdd|jj|fd|i|j } d} d |j |j|jfd | i|D]2} | d z } | jz t fd |Ds- d  nt| |_ | |_ ttj |d t!t| | S)Nz(MLX does not support structured outputs.rrrrrrrrrjr+r c3JK|]}j|xdk7yw)rN)rfind).0r stop_indexrs r0 z$MLXModel.generate..s$Ld$**T"22Jr9Ls #r#rr%r1)rrrr r&r3r9rrrrrrrWrFrQr )rCrrrrrrstopsr prompt_idsrresponser@rs @@r0rzMLXModel.generatesj  &GH H;D;; )1    %((4!%%fb1!%%gt4mT27T^^77qqQUQpQpq  ,,,TZZpPZp^op H Q M HMM !DLeLLKZ(  (+:$(5%&&3DE" _+   r3)FNNr)r7r8r9rzr;rrhr rrqrWr rr-r.s@r0r0r0s%T#(-1<@  #s(^d*  %)cNT$9 8,01504 * {#* S D(* c3h$. * !J- *  * r3r0c VeZdZdZ ddedzdedzdedzdeffd Zdeed d fd Z dd ee deedzd ee dzd e ee ffdZ dd ee deedzde eefdzd ee dzd e f dZ dd ee deedzde eefdzd ee dzd eef dZxZS)TransformersModela=A class that uses Hugging Face's Transformers library for language model interaction. This model allows you to load and use Hugging Face's models locally using the Transformers library. It supports features like stop sequences and grammar customization. > [!TIP] > You must have `transformers` and `torch` installed on your machine. Please run `pip install smolagents[transformers]` if it's not the case. Parameters: model_id (`str`): The Hugging Face model ID to be used for inference. This can be a path or model identifier from the Hugging Face model hub. For example, `"Qwen/Qwen2.5-Coder-32B-Instruct"`. device_map (`str`, *optional*): The device_map to initialize your model with. torch_dtype (`str`, *optional*): The torch_dtype to initialize your model with. trust_remote_code (bool, default `False`): Some models on the Hub require running remote code: for this model, you would have to set this flag to True. kwargs (dict, *optional*): Any additional keyword arguments that you want to use in model.generate(), for instance `max_new_tokens` or `device`. **kwargs: Additional keyword arguments to pass to `model.generate()`, for instance `max_new_tokens` or `device`. Raises: ValueError: If the model name is not provided. Example: ```python >>> engine = TransformersModel( ... model_id="Qwen/Qwen2.5-Coder-32B-Instruct", ... device="cuda", ... max_new_tokens=5000, ... ) >>> messages = [{"role": "user", "content": "Explain quantum mechanics in simple terms."}] >>> response = engine(messages, stop_sequences=["END"]) >>> print(response) "Quantum mechanics is the branch of physics that studies..." ``` Nrrrr1c  ddl}ddlm}m}m} m} m} |stjdtd}d} |jdxs|jd} | s| |d<tjd | ||jjrd nd }tj!d |d |_ |j%|||||_| j%|||_d|_| |j(j*dd|_t5|ld|j" |d|y#t$r tdwxYw#t.$rf}dt1|vrM|j%|||||_| j%|||_| |j*dd|_n|Yd}~d}~wt2$r}t/d|d||d}~wwxYw)Nr)AutoModelForCausalLMAutoModelForImageTextToText AutoProcessor AutoTokenizerTextIteratorStreamerzhPlease install 'transformers' extra to use 'TransformersModel': `pip install 'smolagents[transformers]'`zThe 'model_id' parameter will be required in version 2.0.0. Please update your code to pass this parameter to avoid future errors. For now, it defaults to 'HuggingFaceTB/SmolLM2-1.7B-Instruct'.z#HuggingFaceTB/SmolLM2-1.7B-Instructimax_new_tokensrzN`max_new_tokens` not provided, using this default value for `max_new_tokens`: rcpuzUsing device: F)rrr1)r1T) skip_promptskip_special_tokensz Unrecognized configuration classz0Failed to load tokenizer and model for model_id=z: )rrr1)r transformersrHrIrJrKrLrrrrrdloggerwarningr is_availableinfor from_pretrainedr processorr streamerrr;rur r)rCrrrr1rrrHrIrJrKrLdefault_max_tokensrMrr s r0rzTransformersModel.__init__!s      MMQ   =H!$45QL9Q'9F# $ NN`as`tu   #(::#:#:#<%J nZL12  `4DD%'"3 EDJ +::8Wh:iDNDL01I1IW[quvDM `dll2BX`Y_`g# %z  J 1SV;1AA) +&7 B "/!>!>x[l!>!m 4T^^QUko p  `PxkQSTUSVWX^_ _ `s1D38AE 3E GAF55 GGGrr@rcHddlm}m}Gdd|}||||gS)Nr)StoppingCriteriarc*eZdZdeefdZdZdZy)?TransformersModel.make_stopping_criteria..StopOnStrings stop_stringsc.||_||_d|_yNrj)r^r stream)rCr^r s r0rzHTransformersModel.make_stopping_criteria..StopOnStrings.__init__ks$0!!*  r3cd|_yr`)rarBs r0resetzETransformersModel.make_stopping_criteria..StopOnStrings.resetps   r3c|jj|ddd}|xj|z c_t|jDcgc]}|jj |c}ryycc}w)NrrTrPF)r decoderarr^endswith)rC input_idsscoresr generated stop_strings r0rzHTransformersModel.make_stopping_criteria..StopOnStrings.__call__ssg NN11)A,r2BX\1]  y( TM^M^_k ,,[9_``s "A7N)r7r8r9rqr;rrcrr1r3r0 StopOnStringsr]js !T#Y !  ! r3rl)rQr[r)rCrr r[rrls r0make_stopping_criteriaz(TransformersModel.make_stopping_criteriags)G ,  $]>9%M$NOOr3rrc |jd||d|}|jd}|jdd}|jdd}|jdxsO|jdxs<|jjdxs|jjdxsd}t |dr |j n |j j||d d d d }|j|jj}t |d r|d }t |dr|j j n |j } |r|j|| nd} ||d<td|d | d|S)N)rrrrrrMrirWptT)rreturn_tensorsrr return_dictrh)r )r use_cachestopping_criteriar1) rrrdrr)rWr r&tordevicermrh) rCrrrrrrrM prompt_tensormodel_tokenizerrss r0_prepare_completion_argsz*TransformersModel._prepare_completion_args|s>nn "& o &(():):; =+ .)+6M6=dK6P$..22VZVdVdVdD ' '/ ' Rjn /=*+  /    r3rc R| td|jd |||d|}|djd}|jjd i|}|d|df} t |dr|j j| d} n|jj| d} | t| |} ||_ t| |_ ttj| | |j!D cic]\} } | dk7s | | c} } d t#|t|   Scc} } w) NITransformers does not support structured outputs, use VLLMModel for this.r<rr rrWTrer#rr%r1)rrxshaperrr)rWrfr rrrrrWrFrQr*r ) rCrrrrrgeneration_kwargscount_prompt_tokensr$generated_tokensr,rrLs r0rzTransformersModel.generatesZ  &hi i9D99 )1    09??B!djj!!  q"5"667 4 %..//0@VZ/[K..//0@VZ/[K  %/ ^LK':$(+,<(=%&&"CTCZCZC\%pZS%`cgo`oc5j%p#0!"23  &qs 0 D#>D#c +K| td|jd ||||d|}|djd}||_t |j j d|ji|}|jd} d} |jD],} | dz } | r|nd} d } t| dt| d  d}.|j| |_ yw) Nrz)rrrrrr rX)targetrTrFr)rYrZr\r1) rrxr{rrrrrXstartr}r rmr) rCrrrrrr|r}threadis_first_tokencount_generated_tokensnew_textrs r0generate_streamz!TransformersModel.generate_streams   &hi i9D99 )+1    09??B':$tzz22J ;kYj;kl !" $H "a ' "2@.aL"N( &LPQR  #$  $  )?%sCC)NNNFrpr)r7r8r9rzr;rrrqrmrWr rhr rxrrr}rr-r.s@r0rFrFs%R $!%"&"' Da*Da$JDa4Z Da  DaLPT#YPNdP0,004 . {#. S D(. !J- . c3h . f,01504 * {#* S D(* c3h$. * !J- *  * ^,01504 +?{#+?S D(+?c3h$. +? !J- +? ) *+?r3rFc deZdZdZ d dedeeefdzdedzdedzffd ZdZ d Z xZ S) ApiModelao Base class for API-based language models. This class serves as a foundation for implementing models that interact with external APIs. It handles the common functionality for managing model IDs, custom role mappings, and API client connections. Parameters: model_id (`str`): The identifier for the model to be used with the API. custom_role_conversions (`dict[str, str`], **optional**): Mapping to convert between internal role names and API-specific role names. Defaults to None. client (`Any`, **optional**): Pre-configured API client instance. If not provided, a default client will be created. Defaults to None. requests_per_minute (`float`, **optional**): Rate limit in requests per minute. **kwargs: Additional keyword arguments to pass to the parent class. Nrrclientrequests_per_minutec t|dd|i||xsi|_|xs|j|_t ||_y)Nrr1)r rr create_clientrr rate_limiter)rCrrrrrr s r0rzApiModel.__init__sJ 5(5f5'>'D"$4 2 2 4 '(;<r3ctd)z/Create the API client for the specific service.z8Subclasses must implement this method to create a clientrrBs r0rzApiModel.create_client's!"\]]r3c8|jjy)z,Apply rate limiting before making API calls.N)rthrottlerBs r0_apply_rate_limitzApiModel._apply_rate_limit+s ""$r3r) r7r8r9rzr;rhr floatrrrr-r.s@r0rrs^,:>!,0 = ="&c3h$!6 =d = #T\ =^%r3rc eZdZdZ ddedzdedzdedzdeeefdzdedzf fd Zd Z dd e e d e edzd eeefdzd e e dzde f dZ dd e e d e edzd eeefdzd e e dzde ef dZxZS) LiteLLMModelaModel to use [LiteLLM Python SDK](https://docs.litellm.ai/docs/#litellm-python-sdk) to access hundreds of LLMs. Parameters: model_id (`str`): The model identifier to use on the server (e.g. "gpt-3.5-turbo"). api_base (`str`, *optional*): The base URL of the provider API to call the model. api_key (`str`, *optional*): The API key to use for authentication. custom_role_conversions (`dict[str, str]`, *optional*): Custom role conversion mapping to convert message roles in others. Useful for specific models that do not support specific message roles like "system". flatten_messages_as_text (`bool`, *optional*): Whether to flatten messages as text. Defaults to `True` for models that start with "ollama", "groq", "cerebras". **kwargs: Additional keyword arguments to pass to the OpenAI API. Nrrrrrc |stjdtd}||_||_||n|j d}t |d|||d|y)NzThe 'model_id' parameter will be required in version 2.0.0. Please update your code to pass this parameter to avoid future errors. For now, it defaults to 'anthropic/claude-3-5-sonnet-20240620'.z$anthropic/claude-3-5-sonnet-20240620)ollamagroqrrrrr1)rrrrr startswithr r)rCrrrrrrr s r0rzLiteLLMModel.__init__Cs| MMR   >H   (3 %$$%CD !  $;%=   r3cJ ddl}|S#t$r}td|d}~wwxYw)zCreate the LiteLLM client.rNzWPlease install 'litellm' extra to use LiteLLMModel: `pip install 'smolagents[litellm]'`)litellmr)rCrrs r0rzLiteLLMModel.create_clientbs6    # %i  s " "rrrrr@c 6|jd|||||j|j|jd|jd |}|j |j jdi|}|jj|_ |jj|_ tj|jdj j#hd|t%|jj|jjS) NT) rrrrrrrrrrrXrYrZincluderr[r\r1)rrrrrrr completionusage prompt_tokensrcompletion_tokensrrWrfchoicesr model_dumpr rCrrrrrrrDs r0rzLiteLLMModel.generatems,=>'/~~'C'C$(0(H(H%$$   Q  ' ' 2 2;\ 2 ]"%^^99&nn>>%  r3c +K|jd|||||j|j|j|jdd |}|j |j jdi|dddidD]_}t|ddrx|jj|_ |jj|_ tdt|jj|jj|j s|j d }|j"rt|j"j$|j"j&rY|j"j&D cgc]9} t)| j*| j,| j.| j0 ;c} nd Et|d drTt3d |ycc} ww)NT) rrrrrrrrr include_usagerastream_optionsrrjrrYr\rryr?rr>rYrZ finish_reason#No content or tool calls in event: r1)rrrrrrrrrrrrrrr}r rdeltarYrZrxryr?rr>r rCrrrrrreventchoicers r0rzLiteLLMModel.generate_streams%>&+kk&C&C!}}q)<<0 & 4 4"<<22*0)@)@$!& ;&+kk#(88%*ZZ). $"  #6?DA(+Nug)VWW= X $sD G#A#G#3>G 1G#G#NNNNNr)r7r8r9rzr;rhrrrrqrWr rrr}rr-r.s@r0rr0sE( $#"9=04  * * t  "&c3h$!6  #'+  > ,01504  {#  S D(  c3h$.  !J-     J,01504 3X{#3XS D(3Xc3h$. 3X !J- 3X ) *3Xr3rc |eZdZdZ d dedeeeefdeeefdzdeeefdzdedzf fd Z d Z xZ S) LiteLLMRouterModelu" Router‑based client for interacting with the [LiteLLM Python SDK Router](https://docs.litellm.ai/docs/routing). This class provides a high-level interface for distributing requests among multiple language models using the LiteLLM SDK's routing capabilities. It is responsible for initializing and configuring the router client, applying custom role conversions, and managing message formatting to ensure seamless integration with various LLMs. Parameters: model_id (`str`): Identifier for the model group to use from the model list (e.g., "model-group-1"). model_list (`list[dict[str, Any]]`): Model configurations to be used for routing. Each configuration should include the model group name and any necessary parameters. For more details, refer to the [LiteLLM Routing](https://docs.litellm.ai/docs/routing#quick-start) documentation. client_kwargs (`dict[str, Any]`, *optional*): Additional configuration parameters for the Router client. For more details, see the [LiteLLM Routing Configurations](https://docs.litellm.ai/docs/routing). custom_role_conversions (`dict[str, str]`, *optional*): Custom role conversion mapping to convert message roles in others. Useful for specific models that do not support specific message roles like "system". flatten_messages_as_text (`bool`, *optional*): Whether to flatten messages as text. Defaults to `True` for models that start with "ollama", "groq", "cerebras". **kwargs: Additional keyword arguments to pass to the LiteLLM Router completion method. Example: ```python >>> import os >>> from smolagents import CodeAgent, WebSearchTool, LiteLLMRouterModel >>> os.environ["OPENAI_API_KEY"] = "" >>> os.environ["AWS_ACCESS_KEY_ID"] = "" >>> os.environ["AWS_SECRET_ACCESS_KEY"] = "" >>> os.environ["AWS_REGION"] = "" >>> llm_loadbalancer_model_list = [ ... { ... "model_name": "model-group-1", ... "litellm_params": { ... "model": "gpt-4o-mini", ... "api_key": os.getenv("OPENAI_API_KEY"), ... }, ... }, ... { ... "model_name": "model-group-1", ... "litellm_params": { ... "model": "bedrock/anthropic.claude-3-sonnet-20240229-v1:0", ... "aws_access_key_id": os.getenv("AWS_ACCESS_KEY_ID"), ... "aws_secret_access_key": os.getenv("AWS_SECRET_ACCESS_KEY"), ... "aws_region_name": os.getenv("AWS_REGION"), ... }, ... }, >>> ] >>> model = LiteLLMRouterModel( ... model_id="model-group-1", ... model_list=llm_loadbalancer_model_list, ... client_kwargs={ ... "routing_strategy":"simple-shuffle" ... } >>> ) >>> agent = CodeAgent(tools=[WebSearchTool()], model=model) >>> agent.run("How many seconds would it take for a leopard at full speed to run through Pont des Arts?") ``` Nr model_list client_kwargsrrc Jd|i|xsi|_t|d|||d|y)Nrrr1rr r)rCrrrrrrr s r0rzLiteLLMRouterModel.__init__sH * "   $;%=   r3cn ddlm}|di|jS#t$r}td|d}~wwxYw)Nr)Routerz]Please install 'litellm' extra to use LiteLLMRouterModel: `pip install 'smolagents[litellm]'`r1)litellm.routerrrr)rCrrs r0rz LiteLLMRouterModel.create_clientsE  - +**++ # %o  s 4 /4r) r7r8r9rzr;rqrhr rrrr-r.s@r0rrsy<D049=04   c3h( CH~,  "&c3h$!6  #'+  (,r3rcJeZdZdZ ddededzdedzdedeeefdzdeeefdzd edzd edzd edzffd Zd Z dde e de edzdeeefdzde e dzde f dZ dde e de edzdeeefdzde e dzdeef dZxZS)InferenceClientModelaA class to interact with Hugging Face's Inference Providers for language model interaction. This model allows you to communicate with Hugging Face's models using Inference Providers. It can be used in both serverless mode, with a dedicated endpoint, or even with a local URL, supporting features like stop sequences and grammar customization. Providers include Cerebras, Cohere, Fal, Fireworks, HF-Inference, Hyperbolic, Nebius, Novita, Replicate, SambaNova, Together, and more. Parameters: model_id (`str`, *optional*, default `"Qwen/Qwen2.5-Coder-32B-Instruct"`): The Hugging Face model ID to be used for inference. This can be a model identifier from the Hugging Face model hub or a URL to a deployed Inference Endpoint. Currently, it defaults to `"Qwen/Qwen2.5-Coder-32B-Instruct"`, but this may change in the future. provider (`str`, *optional*): Name of the provider to use for inference. A list of supported providers can be found in the [Inference Providers documentation](https://huggingface.co/docs/inference-providers/index#partners). Defaults to "auto" i.e. the first of the providers available for the model, sorted by the user's order [here](https://hf.co/settings/inference-providers). If `base_url` is passed, then `provider` is not used. token (`str`, *optional*): Token used by the Hugging Face API for authentication. This token need to be authorized 'Make calls to the serverless Inference Providers'. If the model is gated (like Llama-3 models), the token also needs 'Read access to contents of all public gated repos you can access'. If not provided, the class will try to use environment variable 'HF_TOKEN', else use the token stored in the Hugging Face CLI configuration. timeout (`int`, *optional*, defaults to 120): Timeout for the API request, in seconds. client_kwargs (`dict[str, Any]`, *optional*): Additional keyword arguments to pass to the Hugging Face InferenceClient. custom_role_conversions (`dict[str, str]`, *optional*): Custom role conversion mapping to convert message roles in others. Useful for specific models that do not support specific message roles like "system". api_key (`str`, *optional*): Token to use for authentication. This is a duplicated argument from `token` to make [`InferenceClientModel`] follow the same pattern as `openai.OpenAI` client. Cannot be used if `token` is set. Defaults to None. bill_to (`str`, *optional*): The billing account to use for the requests. By default the requests are billed on the user's account. Requests can only be billed to an organization the user is a member of, and which has subscribed to Enterprise Hub. base_url (`str`, `optional`): Base URL to run inference. This is a duplicated argument from `model` to make [`InferenceClientModel`] follow the same pattern as `openai.OpenAI` client. Cannot be used if `model` is set. Defaults to None. **kwargs: Additional keyword arguments to pass to the Hugging Face InferenceClient. Raises: ValueError: If the model name is not provided. Example: ```python >>> engine = InferenceClientModel( ... model_id="Qwen/Qwen2.5-Coder-32B-Instruct", ... provider="nebius", ... token="your_hf_token_here", ... max_tokens=5000, ... ) >>> messages = [{"role": "user", "content": "Explain quantum mechanics in simple terms."}] >>> response = engine(messages, stop_sequences=["END"]) >>> print(response) "Quantum mechanics is the branch of physics that studies..." ``` Nrrrrrrrbill_tobase_urlc  | | td||n|}|tjd}i|xsi|||||| d|_t |d||d| y)NzReceived both `token` and `api_key` arguments. Please provide only one of them. `api_key` is an alias for `token` to make the API compatible with OpenAI's client. It has the exact same behavior as `token`.HF_TOKEN)rrrrrr)rrr1)rosgetenvrr r) rCrrrrrrrrrrr s r0rzInferenceClientModel.__init__\s  !4>  * =IIj)E "     f(D[f_efr3c2ddlm}|di|jS)zCreate the Hugging Face client.r)InferenceClientr1)huggingface_hubrr)rCrs r0rz"InferenceClientModel.create_client}s34!3!344r3rrrrr@c P|6|jdtvr!tddjtz|jd |||d|j d|}|j |jjd i|}|jj|_ |jj|_ tjt!|j"dj$|t'|jj|jjS) NrzKInferenceClientModel only supports structured outputs with these providers:z, T)rrrrrrrrr1)rSTRUCTURED_GENERATION_PROVIDERSrrmrrrrchat_completionrrrrrrWrfrrrr rs r0rzInferenceClientModel.generates  &4+=+=j+IQp+p]));<= >%  r3c +K|jd|||||j|jdd|}|j|jj j jdi|dddidD]_}t|ddrx|jj|_ |jj|_ tdt|jj|jj|j s|j d }|j"rt|j"j$|j"j&rY|j"j&D cgc]9} t)| j*| j,| j.| j0 ;c} nd Et|d drTt3d |ycc} ww)NTrrrrrrrrrrrjrrrrrrrr1)rrrrrchat completionscreaterrrrrrr}r rrrYrZrxryr?rr>rrs r0rz$InferenceClientModel.generate_streams%>&+kk&C&C!}}q)<<0 & 4 4"<<22*0)@)@$!& ;&+kk#(88%*ZZ). $"  #6?DA(+Nug)VWWA X$$sD G!A#G!1>G /G! G!) zQwen/Qwen2.5-Coder-32B-InstructNNxNNNNNr)r7r8r9rzr;r{rhr rrrqrWr rrr}rr-r.s@r0rr"s7v:# /39=""#gg*gTz g  g CH~, g"&c3h$!6gtgtg*gB5,01504 " {#" S D(" c3h$. " !J- "  " N,01504 3X{#3XS D(3Xc3h$. 3X !J- 3X ) *3Xr3rc<eZdZdZ ddededzdedzdedzdedzdeeefdzd eeefdzd effd Zd Z dd e e de edzdeeefdzde e dzde ef dZ dd e e de edzdeeefdzde e dzde f dZxZS)OpenAIServerModelaThis model connects to an OpenAI-compatible API server. Parameters: model_id (`str`): The model identifier to use on the server (e.g. "gpt-3.5-turbo"). api_base (`str`, *optional*): The base URL of the OpenAI-compatible API server. api_key (`str`, *optional*): The API key to use for authentication. organization (`str`, *optional*): The organization to use for the API request. project (`str`, *optional*): The project to use for the API request. client_kwargs (`dict[str, Any]`, *optional*): Additional keyword arguments to pass to the OpenAI client (like organization, project, max_retries etc.). custom_role_conversions (`dict[str, str]`, *optional*): Custom role conversion mapping to convert message roles in others. Useful for specific models that do not support specific message roles like "system". flatten_messages_as_text (`bool`, default `False`): Whether to flatten messages as text. **kwargs: Additional keyword arguments to pass to the OpenAI API. Nrrrrrrrrc  Ti|xsi||||d|_t |d|||d| y)N)rrrrrr1r) rCrrrrrrrrrr s r0rzOpenAIServerModel.__init__sR "  (    $;%=   r3c~ ddl}|jdi|jS#t$r}td|d}~wwxYw)NrzZPlease install 'openai' extra to use OpenAIServerModel: `pip install 'smolagents[openai]'`r1)openairOpenAIrrCrrs r0rzOpenAIServerModel.create_clientsK   v}}2t1122 # %l  " < 7<rrrrr@c +K|jd |||||j|jdd|}|j|jj j jd i|dddidD]^}|jrx|jj|_ |jj|_ tdt|jj|jj|js|jd}|j rt|j j"|j j$rY|j j$D cgc]9} t'| j(| j*| j,| j. ;c} nd Dt1|d drSt3d |ycc} ww)NTrrrrjrrrrrrrr1)rrrrrrrrrrrrrr}r rrrYrZrxryr?rr>rrrs r0rz!OpenAIServerModel.generate_streams%>&+kk&C&C!}}q)<<0 & 4 4"<<22*0)@)@$!& ;&+kk#(88%*ZZ). $"  #6?DA(+Nug)VWWA X$$sD G  A#G 0>G .G  G c 6|jd |||||j|jdd|}|j|jj j jd i|}t|jdd|_ t|jdd|_ tj|jdjj!hd|t#|jj$|jj& S) NTrrrrrrrrr1)rrrrrrrrrrrrrWrfrrrr rrrs r0rzOpenAIServerModel.generateOs>%  r3)NNNNNNFr)r7r8r9rzr;rhr rrrrqrWr rr}rrr-r.s@r0rrsv6 $"#'"/39=).  * t  Dj  t  CH~, "&c3h$!6 #' 43,01504 3X{#3XS D(3Xc3h$. 3X !J- 3X ) *3Xp,01504  {# S D( c3h$.  !J-    r3rceZdZdZ d dededzdedzdedzdeeefdzdeeefdzf fd Zd ZxZ S) AzureOpenAIServerModelaThis model connects to an Azure OpenAI deployment. Parameters: model_id (`str`): The model deployment name to use when connecting (e.g. "gpt-4o-mini"). azure_endpoint (`str`, *optional*): The Azure endpoint, including the resource, e.g. `https://example-resource.azure.openai.com/`. If not provided, it will be inferred from the `AZURE_OPENAI_ENDPOINT` environment variable. api_key (`str`, *optional*): The API key to use for authentication. If not provided, it will be inferred from the `AZURE_OPENAI_API_KEY` environment variable. api_version (`str`, *optional*): The API version to use. If not provided, it will be inferred from the `OPENAI_API_VERSION` environment variable. client_kwargs (`dict[str, Any]`, *optional*): Additional keyword arguments to pass to the AzureOpenAI client (like organization, project, max_retries etc.). custom_role_conversions (`dict[str, str]`, *optional*): Custom role conversion mapping to convert message roles in others. Useful for specific models that do not support specific message roles like "system". **kwargs: Additional keyword arguments to pass to the Azure OpenAI API. Nrrr api_versionrrc b|xsi}|j||dt|d||||d|y)N)rr)rrrrr1)rr r) rCrrrrrrrr s r0rzAzureOpenAIServerModel.__init__sR&+ *"0   '$;    r3c~ ddl}|jdi|jS#t$r}td|d}~wwxYw)Nrz_Please install 'openai' extra to use AzureOpenAIServerModel: `pip install 'smolagents[openai]'`r1)rr AzureOpenAIrrs r0rz$AzureOpenAIServerModel.create_clientsM   "v!!7D$6$677 # %q  rr) r7r8r9rzr;rhr rrr-r.s@r0rrts.&*""&/39=  d  t  4Z  CH~,  "&c3h$!6 28r3rc>eZdZdZ ddedeeefdzdeeefdzffd Z ddee deedzd eeefdzd ee dzdeeefdzd e d eeeefzdzd effd Z dZ ddee deedzd eeefdzd ee dzd e f dZxZS)AmazonBedrockServerModela< A model class for interacting with Amazon Bedrock Server models through the Bedrock API. This class provides an interface to interact with various Bedrock language models, allowing for customized model inference, guardrail configuration, message handling, and other parameters allowed by boto3 API. Parameters: model_id (`str`): The model identifier to use on Bedrock (e.g. "us.amazon.nova-pro-v1:0"). client (`boto3.client`, *optional*): A custom boto3 client for AWS interactions. If not provided, a default client will be created. client_kwargs (dict[str, Any], *optional*): Keyword arguments used to configure the boto3 client if it needs to be created internally. Examples include `region_name`, `config`, or `endpoint_url`. custom_role_conversions (`dict[str, str]`, *optional*): Custom role conversion mapping to convert message roles in others. Useful for specific models that do not support specific message roles like "system". Defaults to converting all roles to "user" role to enable using all the Bedrock models. flatten_messages_as_text (`bool`, default `False`): Whether to flatten messages as text. **kwargs Additional keyword arguments passed directly to the underlying API calls. Examples: Creating a model instance with default settings: ```python >>> bedrock_model = AmazonBedrockServerModel( ... model_id='us.amazon.nova-pro-v1:0' ... ) ``` Creating a model instance with a custom boto3 client: ```python >>> import boto3 >>> client = boto3.client('bedrock-runtime', region_name='us-west-2') >>> bedrock_model = AmazonBedrockServerModel( ... model_id='us.amazon.nova-pro-v1:0', ... client=client ... ) ``` Creating a model instance with client_kwargs for internal client creation: ```python >>> bedrock_model = AmazonBedrockServerModel( ... model_id='us.amazon.nova-pro-v1:0', ... client_kwargs={'region_name': 'us-west-2', 'endpoint_url': 'https://custom-endpoint.com'} ... ) ``` Creating a model instance with inference and guardrail configurations: ```python >>> additional_api_config = { ... "inferenceConfig": { ... "maxTokens": 3000 ... }, ... "guardrailConfig": { ... "guardrailIdentifier": "identify1", ... "guardrailVersion": 'v1' ... }, ... } >>> bedrock_model = AmazonBedrockServerModel( ... model_id='anthropic.claude-3-haiku-20240307-v1:0', ... **additional_api_config ... ) ``` Nrrrc @|xsi|_|xsztjtjtjtjtj tjtj tji}t| d||d|d|y)NF)rrrrr1) rrFrRrPrQrSrTr r)rCrrrrrr s r0rz!AmazonBedrockServerModel.__init__s+0b #:#    0 0  ! !;#3#3  ! !;#3#3  % %{'7'7 >   $;%*    r3rrrrrrr@c t |d|d|||d|} | jdd| jdgD]!} | jdgD] } d| vs| d= #d|ji| S) a Overrides the base method to handle Bedrock-specific configurations. This implementation adapts the completion keyword arguments to align with Bedrock's requirements, ensuring compatibility with its unique setup and constraints. N)rrrrr toolConfigrrYrmodelIdr1)r rrrdr) rCrrrrrrrrrrrYr s r0rz3AmazonBedrockServerModel._prepare_completion_kwargss$"G> 1$;)E     lD1),,Z< (G";;y"5 (W$ ( ( t}}   r3c~ ddl}|jdi|jS#t$r}td|d}~wwxYw)NrzcPlease install 'bedrock' extra to use AmazonBedrockServerModel: `pip install 'smolagents[bedrock]'`)zbedrock-runtime)boto3rrr)rCrrs r0rz&AmazonBedrockServerModel.create_client<sK   u||D1C1CDD # %u  rc | td|jd|||jdd|}|j|jj di|}|ddddd|ddd<|d d |_|d d |_tj|dd|t|d d |d d   S)Nz/Amazon Bedrock does not support response_formatT)rrrroutputrrYrrr inputTokens outputTokensrrr1) rrrrrconverserrrWrfr rs r0rz!AmazonBedrockServerModel.generateFs  &NO O"A$"A"A# 1$($@$@)- #  #   '4;;''<*;<4,015049=-237) {#) S D() c3h$. ) !J- ) "&c3h$!6 ) '+) 4S>)D0)  ) VE,01504 ! {#! S D(! c3h$. ! !J- !  ! r3r)rFrrrr0rFrrrrr OpenAIModelrrAzureOpenAIModelrAmazonBedrockModelrWrK)Kr^loggingrrrrcollections.abcrcopyr dataclassesrrenumr threadingrtypingr r monitoringr rr utilsrrrrrrQr getLoggerr7rRrCODEAGENT_RESPONSE_FORMATr2r5r=r;rFrWrhrvrxr}rQrqrrSrTrPrrrrrrrrrr0rFrrrrrrrrrr__all__r1r3r0rsW  %)%"kk1   8 $#-~"> %*$Z&$ $Y#$ #F++! $')6 ## #  tt t &#t & (( (VC$J3: 88 8 ** *FQEZEZ;./;7B;;~;00{// t,3S cIK).%* ?{#?; 34tCH~E?#'?# ?  $sCx. ?D#csWj$-c-d-$gBgBTh h Vk uk \J?J?Z'%u'%TRX8RXjZ,Z,zxX8xXvQ Q h 68.68r*w xw t. r3