language_models
langroid/language_models/init.py
LLMConfig
¶
Bases: BaseSettings
Common configuration for all language models.
LLMMessage
¶
Bases: BaseModel
Class representing an entry in the msg-history sent to the LLM API. It could be one of these: - a user message - an LLM ("Assistant") response - a fn-call or tool-call-list from an OpenAI-compatible LLM API response - a result or results from executing a fn or tool-call(s)
api_dict(has_system_role=True)
¶
Convert to dictionary for API request, keeping ONLY the fields that are expected in an API call! E.g., DROP the tool_id, since it is only for use in the Assistant API, not the completion API.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
has_system_role |
bool
|
whether the message has a system role (if not, set to "user" role) |
True
|
Returns: dict: dictionary representation of LLM message
Source code in langroid/language_models/base.py
LLMFunctionCall
¶
Bases: BaseModel
Structure of LLM response indicating it "wants" to call a function.
Modeled after OpenAI spec for function_call
field in ChatCompletion API.
from_dict(message)
staticmethod
¶
Initialize from dictionary. Args: d: dictionary containing fields to initialize
Source code in langroid/language_models/base.py
LLMFunctionSpec
¶
Bases: BaseModel
Description of a function available for the LLM to use.
To be used when calling the LLM chat()
method with the functions
parameter.
Modeled after OpenAI spec for functions
fields in ChatCompletion API.
Role
¶
Bases: str
, Enum
Possible roles for a message in a chat.
LLMTokenUsage
¶
Bases: BaseModel
Usage of tokens by an LLM.
LLMResponse
¶
Bases: BaseModel
Class representing response from LLM.
to_LLMMessage()
¶
Convert LLM response to an LLMMessage, to be included in the message-list sent to the API. This is currently NOT used in any significant way in the library, and is only provided as a utility to construct a message list for the API when directly working with an LLM object.
In a ChatAgent
, an LLM response is first converted to a ChatDocument,
which is in turn converted to an LLMMessage via ChatDocument.to_LLMMessage()
See ChatAgent._prep_llm_messages()
and ChatAgent.llm_response_messages
Source code in langroid/language_models/base.py
get_recipient_and_message()
¶
If message
or function_call
of an LLM response contains an explicit
recipient name, return this recipient name and message
stripped
of the recipient name if specified.
Two cases:
(a) message
contains addressing string "TO: message
is empty and function_call/tool_call with explicit recipient
Returns:
Type | Description |
---|---|
str
|
name of recipient, which may be empty string if no recipient |
str
|
content of message |
Source code in langroid/language_models/base.py
OpenAIChatModel
¶
Bases: str
, Enum
Enum for OpenAI Chat models
AnthropicModel
¶
Bases: str
, Enum
Enum for Anthropic models
GeminiModel
¶
Bases: str
, Enum
Enum for Gemini models
OpenAICompletionModel
¶
Bases: str
, Enum
Enum for OpenAI Completion models
OpenAIGPTConfig(**kwargs)
¶
Bases: LLMConfig
Class for any LLM with an OpenAI-like API: besides the OpenAI models this includes: (a) locally-served models behind an OpenAI-compatible API (b) non-local models, using a proxy adaptor lib like litellm that provides an OpenAI-compatible API. We could rename this class to OpenAILikeConfig.
Source code in langroid/language_models/openai_gpt.py
create(prefix)
classmethod
¶
Create a config class whose params can be set via a desired prefix from the .env file or env vars. E.g., using
you can have a group of params prefixed by "OLLAMA_", to be used with models served viaollama
.
This way, you can maintain several setting-groups in your .env file,
one per model type.
Source code in langroid/language_models/openai_gpt.py
OpenAIGPT(config=OpenAIGPTConfig())
¶
Bases: LanguageModel
Class for OpenAI LLMs
Source code in langroid/language_models/openai_gpt.py
436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 |
|
unsupported_params()
¶
List of params that are not supported by the current model
Source code in langroid/language_models/openai_gpt.py
rename_params()
¶
Map of param name -> new name for specific models. Currently main troublemaker is o1* series.
Source code in langroid/language_models/openai_gpt.py
chat_context_length()
¶
Context-length for chat-completion models/endpoints Get it from the dict, otherwise fail-over to general method
Source code in langroid/language_models/openai_gpt.py
completion_context_length()
¶
Context-length for completion models/endpoints Get it from the dict, otherwise fail-over to general method
Source code in langroid/language_models/openai_gpt.py
chat_cost()
¶
(Prompt, Generation) cost per 1000 tokens, for chat-completion models/endpoints. Get it from the dict, otherwise fail-over to general method
Source code in langroid/language_models/openai_gpt.py
set_stream(stream)
¶
Enable or disable streaming output from API. Args: stream: enable streaming output from API Returns: previous value of stream
Source code in langroid/language_models/openai_gpt.py
get_stream()
¶
Get streaming status. Note we disable streaming in quiet mode.
Source code in langroid/language_models/openai_gpt.py
tool_deltas_to_tools(tools)
staticmethod
¶
Convert accumulated tool-call deltas to OpenAIToolCall objects. Adapted from this excellent code: https://community.openai.com/t/help-for-function-calls-with-streaming/627170/2
Parameters:
Name | Type | Description | Default |
---|---|---|---|
tools |
List[Dict[str, Any]]
|
list of tool deltas received from streaming API |
required |
Returns:
Name | Type | Description |
---|---|---|
str |
str
|
plain text corresponding to tool calls that failed to parse |
List[OpenAIToolCall]
|
List[OpenAIToolCall]: list of OpenAIToolCall objects |
|
List[Dict[str, Any]]
|
List[Dict[str, Any]]: list of tool dicts (to reconstruct OpenAI API response, so it can be cached) |
Source code in langroid/language_models/openai_gpt.py
1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 |
|
MockLM(config=MockLMConfig())
¶
Bases: LanguageModel
Source code in langroid/language_models/mock_lm.py
chat(messages, max_tokens=200, tools=None, tool_choice='auto', functions=None, function_call='auto', response_format=None)
¶
Mock chat function for testing
Source code in langroid/language_models/mock_lm.py
achat(messages, max_tokens=200, tools=None, tool_choice='auto', functions=None, function_call='auto', response_format=None)
async
¶
Mock chat function for testing
Source code in langroid/language_models/mock_lm.py
generate(prompt, max_tokens=200)
¶
agenerate(prompt, max_tokens=200)
async
¶
MockLMConfig
¶
Bases: LLMConfig
Mock Language Model Configuration.
Attributes:
Name | Type | Description |
---|---|---|
response_dict |
Dict[str, str]
|
A "response rule-book", in the form of a dictionary; if last msg in dialog is x,then respond with response_dict[x] |
AzureConfig(**kwargs)
¶
Bases: OpenAIGPTConfig
Configuration for Azure OpenAI GPT.
Attributes:
Name | Type | Description |
---|---|---|
type |
str
|
should be |
api_version |
str
|
can be set in the |
deployment_name |
str
|
can be set in the |
model_name |
str
|
can be set in the |
model_version |
str
|
can be set in the |
Source code in langroid/language_models/openai_gpt.py
AzureGPT(config)
¶
Bases: OpenAIGPT
Class to access OpenAI LLMs via Azure. These env variables can be obtained from the
file .azure_env
. Azure OpenAI doesn't support completion
Attributes:
config (AzureConfig): AzureConfig object
api_key (str): Azure API key
api_base (str): Azure API base url
api_version (str): Azure API version
model_name (str): the name of gpt model in your deployment
model_version (str): the version of gpt model in your deployment
Source code in langroid/language_models/azure_openai.py
set_chat_model()
¶
Sets the chat model configuration based on the model name specified in the
.env
. This function checks the model_name
in the configuration and sets
the appropriate chat model in the config.chat_model
. It supports handling for
'35-turbo' and 'gpt-4' models. For 'gpt-4', it further delegates the handling
to handle_gpt4_model
method. If the model name does not match any predefined
models, it defaults to OpenAIChatModel.GPT4
.
Source code in langroid/language_models/azure_openai.py
handle_gpt4_model()
¶
Handles the setting of the GPT-4 model in the configuration.
This function checks the model_version
in the configuration.
If the version is not set, it raises a ValueError indicating
that the model version needs to be specified in the .env
file. It sets OpenAIChatMode.GPT4o
if the version is
one of those listed below, and
OpenAIChatModel.GPT4_TURBO
if
the version is '1106-Preview', otherwise, it defaults to
setting OpenAIChatModel.GPT4
.