shcherbak-ai/contextgem v1.3.0 on GitHub

[1.3.0] - 2026-10-25

Added

Added ModelRefusalError, a subclass of ModelExtractionError, raised when the last model tried, or its provider, declines to answer, after its retries and the fallback models: the refusal of a model (e.g. OpenAI's refusal field, or a refused answer of a decision model), an answer stopped with the content_filter finish reason (e.g. the refusal of a Claude model, the safety filters of Gemini and Azure OpenAI, or the guardrails of Amazon Bedrock), or a request or an answer that the provider blocks. Its message names the model and what declined to answer, never the answer or the text of the document. See Refusals.
Added max_retries_invalid_data to DocumentDecisionModel: how many times a call whose response cannot be used, or that the model or its provider declines to answer, is made again, as for an LLM (see "Changed").
Added supports_vision, supports_reasoning, supports_tools and supports_parallel_tool_calls to DocumentLLM, which declare a capability of the model where the detection of litellm is wrong for it, e.g. for a local model: None by default, i.e. detected with litellm.supports_vision(), litellm.supports_reasoning(), litellm.supports_function_calling() and litellm.supports_parallel_function_calling(), as before. They replace the private attributes _supports_vision, _supports_reasoning, _supports_tools and _supports_parallel_tool_calls, which the documentation and the messages told you to set (see "Changed"). The messages on a capability that an LLM lacks name the function of litellm that it was detected with, or the parameter if it is set to False. See Explicit capability declaration.
Added UnknownPricesWarning, a subclass of ContextGemWarning that can be imported from contextgem: the category of the warning on a model with auto_pricing=True that genai-prices has no prices for, whose costs stay at zero. A warning filter on it, warnings.filterwarnings("error", category=UnknownPricesWarning), makes such a model raise where it is created, e.g. for a budget of your own. See Stopping a model whose prices are unknown.
A ContextGemWarning is now issued when reasoning_effort is set for an LLM that does not send it: for a model that does not support reasoning, as detected or set with supports_reasoning, and for a model whose parameters litellm does not list it among. The parameter was left out of the requests without a word.
Before you build on ContextGem, a new page of the documentation: what ContextGem already does that is easy to build again, or to get wrong, when you build on it, or when a coding agent does, each with a link to the page that describes it. The summary of llms.txt points to it first. The page on optimizing for cost now says what each request to an LLM sends.
The documentation now says that a document holds copies of the aspects and concepts given to it, which extraction parameters put aspects and concepts in the same request, where the log goes (standard output, not the root logger) and that the loggers of litellm log the prompts at the DEBUG level, that the call history of a model keeps the prompts until reset_usage_and_cost(), how to stub an empty answer, a refusal and a blocked request in a test, with a stand-in that builds its responses with litellm.mock_completion and sends no request whatever it is given, and which download the counting of the input tokens can make.
Testing without API calls, a new page of the documentation: litellm.acompletion and litellm.adecisions, which ContextGem looks up at each request, are the supported points for replacing the requests to the models in your tests, with examples of a stand-in for an LLM and for a decision model.
The docstring of each public class now names all its public attributes and properties, the inherited ones included, e.g. custom_data, where your own data goes, such as an ID of your own for a paragraph, and unique_id. The docstrings of Paragraph and Sentence say that the references of aspects and extracted items are the document's own paragraphs and sentences. A docstring that shows a usage example now gives the address of the API page that shows it, as the installed package has no usage examples.
The docstring of the package, the description on PyPI and the project URLs now point to llms.txt, the index of the documentation for coding agents and LLMs, with a Markdown copy of each page. The docstring of the package says that the documentation describes the latest release, and the docstrings the installed version.

Changed

The prompts of LLMs list the paragraphs of a document, and their sentences, more compactly: a paragraph on one line, <paragraph id="P0">text</paragraph>, with its additional context in an <additional_context> element before the text, where a paragraph took several lines with its ID given twice. With sentence-level references (reference_depth="sentences"), each sentence of a paragraph is on a line of its own, <sentence id="P0-S0">text</sentence>, within the <paragraph> element. On the evaluation cases, this cut the input tokens of a run by 10% to 42%, most on documents of many short paragraphs such as table rows, with no measured loss of quality, and halved the retries for invalid data. The prompts of aspects, and of concepts with references, change, as does one line of the instructions of every extraction prompt.
A refusal of a model or of its provider is now told apart from invalid data: its answer is not used, also if it is valid JSON, each refusal is logged as one, and ModelRefusalError is raised when the last model tried still declines to answer after its retries, where an LLM's refusal was reported as invalid JSON. As before, a refused answer of an LLM is requested again up to max_retries_invalid_data times and then sent to the fallback models, as the verdict of a model or of a content filter can differ on another request.
A request that the provider blocks for its content, or whose answer it blocks, which litellm raises as ContentPolicyViolationError for both (e.g. Vertex AI and Anthropic for a blocked answer), is now a refusal: it is requested again up to max_retries_invalid_data times, as another refusal is, where it was not sent again to the same model, then sent to the fallback models, and raises ModelRefusalError instead of ModelAPIError if none of them answers, with the error of litellm as its original_error. To handle such a request, catch ModelRefusalError (or ModelExtractionError, its base) where you caught ModelAPIError. Set max_retries_invalid_data=0 to send it to each model once.
A decision model now asks again for a response that cannot be used, e.g. an answer without valid probabilities, and after a refusal of the model or of its provider, up to its new max_retries_invalid_data times (3 by default, as for an LLM), before the fallback models are tried, where it was asked once. Only the requests whose answers cannot be used are sent again, with those that were not sent after them: e.g. for an aspect, the requests on the paragraphs that were answered are not. A refused answer on some of the paragraphs of an aspect is accepted as before, and not asked for again of the model itself (see the next entry). A model that gives the same answers to the same request gains nothing from the retries, which then only add requests: set max_retries_invalid_data=0 for it, which also keeps the requests of earlier versions. See Errors.
The paragraphs, and the sentences, that a decision model refuses to answer on for an aspect while it answers on the others, which are left out of the aspect, are now sent to its fallback models, if it has any, where none of them was asked. In their order, each extracts the aspect from a document of what is still undecided, with its own retries, and its decisions on it are added to the aspect, in the order of the document: an item of a fallback LLM has no probabilities. What a fallback decision model refuses to answer on in turn goes to the next fallback model, and a fallback model that fails decides nothing. What none of them decides is left out, as before, and the warning on it names only that. See Errors.
A refused answer of a decision model to a question of a concept, or to the question of an aspect on every paragraph, now raises ModelRefusalError instead of ModelExtractionError when the retries and the fallback models are used up. litellm also gives a refused answer for a question that a decision model leaves unanswered, or answers with an answer of another kind.
A failed request whose error has no HTTP status code is no longer retried as a transient error, unless it is a TimeoutError or a ConnectionError of Python: the next fallback model is tried, and ModelAPIError is raised if none is left, with the error as its original_error. litellm gives a status code to each error of a request, also to one that it cannot classify (APIConnectionError), i.e. such an error comes from elsewhere, e.g. from a stand-in for litellm.acompletion or litellm.adecisions in a test, and would fail again in the same way: retrying it only delayed the error.
The messages on an extraction that fails for an answer of an LLM that cannot be used now say that the LLM returned invalid data, not invalid JSON: such an answer can be valid JSON that does not match the answer format, or a value that a concept does not accept. The line logged at the ERROR level for such a call is now LLM returned invalid data for aspects (or concepts), where it was LLM did not return valid JSON for aspects: change a log filter or an alert that matches the old text.
In a chat, a request whose answer is empty, a refused answer without a text, and a request or an answer that the provider blocks, are now sent again up to max_retries_invalid_data times (3 by default), as the same data is asked for again in extraction, where the next fallback LLM was tried at once. After a round of tool calls, only that request is sent again, with the results of the tool calls: the tools are not called again. A refused answer with a text is returned with a warning logged, and an answer cut off at the max output tokens is not sent again. A refused answer without a text, or a blocked request, raises ModelRefusalError instead of ModelAPIError once the retries and the fallback LLMs are used up. To handle such an answer or request, catch ModelRefusalError (or ModelExtractionError, its base) where you caught ModelAPIError. Set max_retries_invalid_data=0 to send each request of a chat once.
The private attributes _supports_vision, _supports_reasoning, _supports_tools and _supports_parallel_tool_calls of DocumentLLM can no longer be set: setting one raises AttributeError. Pass the parameter of the same name without the underscore instead, e.g. DocumentLLM(..., supports_vision=True), or assign it (llm.supports_vision = True). Reading one still gives the capability that applies. An LLM serialized with an earlier version, and a model group that holds one, can no longer be loaded as is: the serialized data holds the four private attributes, which raise AttributeError (property '_supports_vision' of 'DocumentLLM' object has no setter). Remove the _supports_vision, _supports_reasoning, _supports_tools and _supports_parallel_tool_calls keys from the data of each LLM, its fallback LLMs included, before loading it, and set with its parameter a capability that you had set, e.g. "supports_vision": true: the others are detected.
The line logged at the ERROR level for an answer of an LLM that is not JSON is now LLM output is not JSON: it cannot be parsed as a JSON array or object, where it was Error when validating parsed JSON: parsed_json is None: change a log filter or an alert that matches the old text. The lines on a model created without api_key or api_base (API key was not provided, API base was not provided) are no longer logged: a key is often read from an environment variable, and most providers need no API base. The line logged by get_cost() for a model without prices now says that its costs are not calculated, and how to have them calculated.
The warning that the context window or the max output of a model cannot be checked, e.g. for a model that litellm does not know (Could not validate max input tokens), is now logged once for the LLM and its model, where it was logged for each request, and says to register such a model with litellm.register_model(). See Explicit capability declaration.

Removed

A multi-class LabelConcept no longer logs a message at the INFO level when it is created, which suggested a catch-all label such as "other" for every such concept, also one whose labels already cover all cases. The docstring of LabelConcept gives the advice.
A model created with auto_pricing=True, or assigned it, no longer issues a ContextGemWarning that the prices of genai-prices are not 100% accurate. It was issued for every such model, i.e. such a model raised where it was created when the warnings of ContextGem were made errors, e.g. in tests, as the logging page suggests. The documentation of auto_pricing says that the prices are approximate. The warning on a model that genai-prices has no prices for stays, as an UnknownPricesWarning (see "Added").
Additional context of a paragraph or a sentence with line breaks no longer logs a warning at the WARNING level when it is created: its line breaks are sent to the models as spaces, as the text of a paragraph is, which the docstrings of Paragraph and Sentence now say.

Fixed

Classes of the same name that inherit from JsonObjectClassStruct, e.g. an Address in each of two modules of an application, are no longer mixed up in the structure of a JsonObjectConcept. The base class registered its subclasses by their names, and a class of a structure was replaced with the subclass of the same name defined last, also a class that did not inherit from it, so that the structure got the fields of another class, without an error. A class is now converted by itself, and a name in the string annotations of a subclass, e.g. in a module with from __future__ import annotations, is looked up as Python looks up such names: among the classes defined in the body of the class, e.g. a nested class, among the subclasses defined in the same function or in a function that encloses it, and in its module.
The documentation said that all the classes of a nested class structure of a JsonObjectConcept must inherit from JsonObjectClassStruct. They need no base class: a dataclass, a Pydantic model or another class with type annotations is converted with its nested classes, at any depth. Only classes defined inside a function, in a module with from __future__ import annotations, whose annotations are strings that name classes local to the function, are to inherit from it. The page on JsonObjectConcept, the docstrings and the examples now say so, and the ValueError for such classes without the base class says what to do. The errors on the structure of a JsonObjectConcept now name the concept, where they named its class (JsonObjectConcept).
A request that the client of the OpenAI SDK rejects before sending it, e.g. for a missing API key of an OpenAI or Azure OpenAI model, is no longer retried as a transient error: litellm raises it as a server error (InternalServerError or APIError, 500), as it no longer recognizes the message of the SDK (litellm 1.105.2 with openai 2.54.0), and each such request was sent again 3 times, about 6 seconds in all, with each retry logged as a transient error, before the next fallback model was tried or ModelAPIError was raised. It now fails at once, e.g. in tests run without API keys.
The input tokens of the requests of a model are no longer counted again in the process once tiktoken cannot download the token encodings of the model, e.g. without network access, or on Windows behind a proxy that refuses the download: each request tried the download again, which took about 12 seconds a request behind such a proxy, before the check of the context window was skipped. The check is now skipped for the next requests of that model, with one warning, which links to the way to prepare the encodings without network access.
The log no longer has the escape codes of its colors where it is not written to a terminal, e.g. to a file or to the log of a CI job, which showed them as text. Set FORCE_COLOR to keep the colors there, e.g. in a Jupyter notebook, and NO_COLOR to have no colors at all.
create_image() now accepts a PIL image without a format, e.g. one created in memory with PIL.Image.new(), and saves it as PNG: it raised ValueError, while its docstring said that it takes a PIL image.
The log level OFF no longer removes the handlers that your application added to the contextgem logger: only the handler of ContextGem is removed, and your handlers get the messages again once the level is set to another one with reload_logger_settings().
The capabilities of an LLM are now detected again when its model is changed, unless they are set with their parameters. The LLM kept those of the model it was created with, e.g. was not sent images after its model was changed to one with vision.
The warnings on the parameters of an LLM created without a system message, e.g. on tools set for a model without tool calls, are now issued once: they were issued twice when the LLM was created, which Python's default warning filter shows once, but a filter that shows each warning, or a test that counts them, did not. The LLM was validated twice: the input that a validation error of such an LLM reports for a check of several params, e.g. of its fallback LLMs, is now all the params that it is created with, with the API key redacted, as for an LLM with a system message of its own, where it was the LLM.
In a chat with tools, the list of messages that a request is sent with is no longer changed after the request is sent. The messages of the next requests of the turn were added to it, so that a stand-in for litellm.acompletion that keeps the requests, e.g. in a test, saw each request with the messages of the requests after it.
The extraction prompt of a multi-class LabelConcept no longer contradicts itself: it said that the concept may have no extracted items, and that its items must never be empty, as a label is always selected. It now asks for one or more items, or for exactly one with singular_occurrence=True. Only the prompts with multi-class label concepts change.
The warnings issued during an extraction or a chat are now attributed to the line of your code that calls the extraction method or chat(), as the warnings issued when a model is created are: that genai-prices has no prices for the model, issued when the usage of each call is recorded, that the references of a multimodal concept are added only for its text, that tools are ignored in the extraction of an LLM or a model group, that an ollama_chat/ model is sent images, and that a chat uses the default system message. They were attributed to the code of ContextGem or of asyncio, which a warning filter by the module of your code does not match. When a warning filter makes the warning on unknown prices an error, the extraction method now stops after the call is recorded in the usage: its tokens were recorded, but not the call. Stopping a model whose prices are unknown, new in the documentation, gives such a filter, e.g. for a budget, which makes the model raise where it is created.
A model identifier with an empty provider or model name, e.g. typesafe/ or /gpt-6-luna, is now rejected with ValueError, as an identifier without / is: such a model was created.

Security

The errors logged when the output of a model cannot be used no longer give its values, which can hold text of the document: they say where the output is invalid and how, e.g. `0.extracted_items.1.title`: Extra inputs are not permitted for a field of an item that an LLM wrote beside its value. The text of a validation error of pydantic gives the input of each error, of which such a log line held up to about 50 characters, e.g. of an extracted string, and the messages on a date that cannot be parsed and on a label that is not a label of the concept gave the whole value. These errors are logged at the ERROR level, which the default level (INFO) shows. The ValueError raised for a label or a rating that a concept does not accept, e.g. in extracted items assigned to it, no longer gives the value either, and names the concept. The messages on an answer of a decision model that cannot be used no longer give what the model returned either, e.g. an option that was not offered, the option it picked, or a probability that is not within [0, 1]: they name the question and what is wrong with the answer, with the options offered.