shcherbak-ai/contextgem v1.2.0

on GitHub

[1.2.0] - 2026-10-22

Added

• Added Document.from_docling(), which creates a document from a file converted with Docling, from the DoclingDocument of the conversion. See Converting files with Docling.
  • Formats: all the input formats of Docling 2.141, each covered by test files, e.g. PDF, also scanned, DOCX, PPTX, XLSX, HTML, email, images, audio and video. The supported formats give the extensions, and what some formats need, e.g. LibreOffice for DOC, PPT, XLS and RTF.
  • Paragraphs: each text of the body of the file is a paragraph, in reading order, and each row of a table is a paragraph with each value after the header of its column. The additional context of a paragraph says what it is, e.g. List item or Table row, with the headings of its section, and its sheet, slide or speaker where the file has them. Page headers and footers, comments, speaker notes, hidden slides and sheets, and the text inside pictures are left out. On the 219 converted documents of Docling's own tests, no text and no table cell of their bodies was lost.
  • Sentences: a paragraph that is not running text, e.g. a heading, a table row, a form field or code, is one sentence, its whole text, so that a reference sentence of a table row is the whole row. A SaT model, which is made for running text, split 26 of the 324 table rows of the test files into parts, often within the text of a cell. The other paragraphs are split into sentences by the SaT model when sentences are needed.
  • Pages and positions: the page_numbers of each paragraph are the pages that its text is on (see below), and its custom data holds the reference of its Docling item and the page and the bounding box of each part of its text, e.g. to highlight it in the file, or the times of a cue or a segment of a transcript.
  • Images: the pictures that the Docling document has image data for are attached as images, and so are the frames of a video and the images of the pages of an image file, e.g. a scan. The other pictures are counted, with the reason, in a message at the INFO level.
  • Installation: the new docling extra installs Docling 2.141 or later (uv add "contextgem[docling]"), except on an Intel Mac: docling-parse, the PDF parser of Docling, has no packages for it from version 7, which Docling needs. A Docling document converted elsewhere, e.g. loaded from JSON, needs only docling-core.
• Added page_numbers to paragraphs and sentences: the numbers of the pages of the document that the text is on, which Document.from_docling() sets for the formats with pages, e.g. PDF, and which can be set on the paragraphs that you create. Sentences segmented from a paragraph inherit them. They are not sent to the models. Added reference_page_numbers to aspects and extracted items: the pages of their reference paragraphs and sentences, e.g. to cite the pages of an extracted item without looking them up in its references.

Changed

• Upgraded pinned dependency version: litellm==1.105.2. Versions remain pinned to maintain stability and avoid occasional breaking changes and API inconsistencies observed in previous unpinned releases. openai remains pinned to 2.54.0, the latest version supported by litellm.
• The prompts of LLMs with sentence-level references (reference_depth="sentences") now give the additional context that a sentence shares with its paragraph once, with the paragraph, as the requests to decision models already did. A sentence segmented from a paragraph inherits its additional context, which the prompts gave again under each sentence: for documents created from four files converted with Docling, whose paragraphs all have additional context, the prompts were 7% to 19% longer, and more for a longer document. The additional context of a sentence that differs from that of its paragraph is still given with the sentence.
• A decision model that extracts an aspect with sentence-level references (reference_depth="sentences") no longer asks about the sentence of a paragraph that is one sentence as a whole, e.g. a table row of a document created with Document.from_docling(), or a short paragraph: the sentence takes the answer on the paragraph. Such a request repeated the text of the paragraph, and its answer could differ from the one on the paragraph, which left the paragraph out of the aspect. This saves one request for each such paragraph that belongs to the aspect. A sentence with additional context of its own is still asked about.
• Splitting the paragraphs of a document into sentences now runs the SaT model only on the paragraphs that have no sentences. Previously it ran on all of them, and the sentences of the paragraphs that had sentences, e.g. given with them, were not used.

Removed

• Removed the names that the __init__.py files under contextgem.internal imported from their modules. A name that was imported from one of these packages, e.g. from contextgem.internal import ReasoningEffort, is now imported from its module: from contextgem.internal.typings.types import ReasoningEffort, the path the API reference shows. The public API (contextgem) is unchanged.
• Removed the trace() method that import contextgem added to the logging.Logger class, i.e. to every logger of the process. ContextGem never logged with it. The TRACE level of CONTEXTGEM_LOGGER_LEVEL works as before.

Fixed

• An extraction with sentence-level references (reference_depth="sentences") now splits into sentences the paragraphs that have none in a document whose other paragraphs have sentences, e.g. a document created with some paragraphs given with their sentences. Previously, such a document was taken as split, and the paragraphs without sentences gave no items and no references to an aspect or a concept with sentence-level references. pre_segment_sentences=True split them already.
• Each sentence segmented from a paragraph now gets a copy of the custom data of the paragraph of its own. Previously, the values nested in the custom data, e.g. a dict, were shared by the paragraph and all its sentences, so that a change to them on one sentence changed them on the paragraph and on the other sentences, until the document was serialized and loaded again.
• The items extracted for an aspect now get a copy of the custom data of their reference paragraph or sentence of their own, as the sentences of a paragraph do (above). Previously, the values nested in the custom data were shared by the item and its reference, so that a change to them on one changed them on the other.
• extract_all() with raise_exception_on_extraction_error=False now extracts the concepts of the other aspects and of the document when the extraction of an aspect fails, and leaves out the concepts of that aspect, with a warning that names them. Previously, it raised ValueError (the aspect "is not yet processed"), and the concepts of the document were not extracted. extract_concepts_from_aspect() called on an aspect that is not extracted still raises ValueError.