shcherbak-ai/contextgem v1.1.0

on GitHub

[1.1.0] - 2026-10-17

Added

• Added a documentation page on agentic workflows, with an example for each of the three points where ContextGem runs in them: extracting the metadata of documents at ingestion, extracting on demand in a tool that an agent calls (also served over MCP), and deciding the next step of a workflow with a decision model, which can also choose the tools that an agent gets for its turn. The README links to it.
• Added release_sat_model(), which releases the SaT model that is kept loaded (see "Changed"), to free its memory, e.g. in a long-running process that segments no more documents. A document that needs the model then loads it again. See Releasing a SaT model.

Changed

• The async extraction methods of LLMs, decision models and model groups (extract_all_async(), extract_aspects_from_document_async(), extract_concepts_from_aspect_async() and extract_concepts_from_document_async()) now split the paragraphs of a document into sentences with the SaT model in a worker thread, instead of in the event loop, which the segmentation blocked: the other tasks of the loop, e.g. the requests of other extractions or of a web server, go on while the model runs. For a document of 353 KB, another task of the loop was stopped for about 5 s during the extraction, and is now stopped for about 0.3 s at most, on the machine it was measured on. The segmentation takes as long as before, as the model uses all the cores of the CPU for a document. The sentences are assigned in the event loop, to the paragraphs that have none by then: when two tasks segment the same document at the same time, each paragraph keeps the sentences of the task that finishes first.
• Splitting the paragraphs of a document into sentences with a SaT model is now several times faster for a document with many paragraphs, and takes less memory. The model pads the texts that it processes in a batch to the longest of them, and runs on the padding as well: the paragraphs were passed to it in their order, so that most of them, which are short, were padded to the longest paragraph of each 1,000. They are now passed sorted by length, in batches of 16, and the sentences are the same. For a document of 1,818 paragraphs (353 KB), the segmentation took about 6 s instead of about 57 s, with a peak memory of about 2.2 GB instead of about 3.8 GB, and for a document of 62 paragraphs (7.5 KB) about 0.8 s instead of about 1.1 s, including the loading of the model, on the machine it was measured on. The segmentation of a text into paragraphs (paragraph_segmentation_mode="sat") is unchanged.
• A SaT model is now loaded once in a process, for the first document that needs it, and kept loaded for the documents that follow, instead of being loaded for each document and released after it. The model used last is kept: a document with another sat_model_id loads its model, which is then kept in place of the one before. For 10 short documents (7.5 KB each) segmented into sentences one after another, each document after the first took about 0.16 s instead of about 0.8 s, on the machine it was measured on. A loaded model holds a large amount of memory: about a gigabyte for the default model, and more once it has segmented a long document, or several documents at the same time, as it keeps the memory that it used for them. The peak memory is the same, as a model is loaded while a document is segmented either way, but the process holds it also between documents: about 1.8 GB of committed memory after those 10 documents, and about 2.2 GB after a document of 353 KB, against about 1.2 GB once the model is released with release_sat_model() (see "Added").
• Documents that are segmented with the same SaT model at the same time, e.g. in threads, now share one model, which is loaded once, instead of each loading a model of its own. The paths of a folder with a model refer to one model in any form, e.g. relative or absolute, and a thread that loads a model, which can take minutes when it is downloaded, keeps no thread that needs another model waiting. For 8 short documents (7.5 KB each) segmented into sentences in 8 threads at once, the peak memory of the process was about 4.8 GB instead of about 10.7 GB, and the segmentation took about 6 s instead of about 29 s, on the machine it was measured on.
• Creating a long document, adding aspects or concepts to it, and loading it from serialized data are now faster. A document checks that its paragraphs are found in its text in their order when it is created and on each assignment to it, e.g. in each add_aspects() and add_concepts() call. The check looked for each paragraph from the start of the text, i.e. took time proportional to the number of paragraphs times the length of the text, and now reads the text once. Also, add_aspects() and add_concepts() no longer check the new list on a copy of the whole document, pipeline or aspect first: a rejected list leaves the instance unchanged, as before. For a document of 1,818 paragraphs (353 KB of text), creating it takes about 30 ms instead of about 180 ms, each add_aspects() or add_concepts() call about 2 ms instead of about 160 ms, and Document.from_dict() about 90 ms instead of about 145 ms, on the machine it was measured on. The errors for paragraphs that are not found in the text, or not in their order, are unchanged.
• import contextgem no longer imports wtpsplit-lite, with ONNX Runtime and NumPy, which is now imported when a SaT model is first loaded, i.e. for the segmentation of paragraphs in the "sat" mode or of sentences. A process that uses no SaT model imports ContextGem about 0.13 s faster.