github icereed/paperless-gpt v0.30.0
v0.30.0 — AI Workflows
on GitHub

4 hours ago

paperless-gpt now lets different documents be processed differently. An invoice can get a title prompt built around its invoice number, a contract its own custom fields, private mail only a title, each with its own prompts and steps, set up and tested in the UI.

Around that: searchable PDFs that become a new version of the same document instead of a new document, no more OCR for pages that already have clean text, and a long list of fixes, many of them for documents that ended up with the wrong tags, values or a fail tag without anyone noticing.

The theme is control: over what happens to which document, and over what you can see before it happens.

Upgrading: nothing to do for most setups. Please read Before you upgrade at the bottom if you mount your volumes differently from the examples, used AI workflows from @MrMoep's fork, or customised the custom field prompt.

🧭 AI Workflows: different prompts for different documents

Until now there was one automatic path: every document tagged paperless-gpt-auto got the same prompts and the same steps. That stops working once an archive holds very different things.

An AI workflow is another processing path with its own trigger tag. Tag a document with it (by hand, or with a paperless-ngx workflow or mail rule) and paperless-gpt processes it with that workflow's prompts and steps, then swaps the trigger for the workflow's completion tag.

  • Own prompts, only where they differ. Each prompt is either the shared global one or the workflow's own version; the editor makes clear which one you are editing. Only the steps a workflow actually runs show a prompt.
  • Steps per workflow. Title, tags, correspondent, document type, created date and custom fields are each Default, On or Off, and the editor shows what Default currently means. Optional OCR first, with its own page limit and OCR prompt.
  • Test on a document before you save. The editor runs the workflow, unsaved changes included, on a real document and shows the result next to the current values. Nothing is written to paperless-ngx.
  • Workflows are files. Each one is a folder in prompts/workflows/, next to the global prompts, so the prompts volume you already have keeps them. You can also write them by hand or keep them in git; changes are picked up without a restart.
  • No loops. A completion tag can't be a trigger, workflows can't be chained, and paperless-gpt's own tags can't be triggers. The editor flags this while you type; files written by hand follow the same rules.
  • Plays well with the default path. A document carrying both a workflow trigger and paperless-gpt-auto is processed once, by the workflow.

→ AI Workflows guide

Thanks @MrMoep for building this and bringing a working implementation in the feature request. (#1108, #1110, #1115, #1116, closes #1096)

📄 Searchable PDFs as a new version (paperless-ngx 3.0)

With PDF_UPLOAD_MODE=version, the searchable PDF is added as a new version of the same document. It keeps its ID, tags, custom fields, notes and storage path, and the original stays available as the previous version. Nothing is deleted. On paperless-ngx 3.0 this is the safe replacement for PDF_REPLACE.

new stays the default; older paperless-ngx versions get a clear error instead of a silent failure, and PDF_REPLACE is ignored in version mode.

Thanks @kzaoaai! (#1125)

⚡ No OCR for pages that already have text

Born-digital PDFs (invoices, statements, letters generated as PDF) already contain exact text. OCR on them costs time and tokens and can only make the text worse. With OCR_SKIP_DIGITAL_PAGES=true, paperless-gpt checks each page of the original file and keeps the text of pages that have a clean, visible text layer. Scans, pages with another tool's OCR layer, and pages with broken text, including umlauts mangled into f¸r or für, still go to OCR. In a real archive we checked, about half of all pages needed no OCR at all.

Off by default; image mode only, and off whenever a searchable PDF or hOCR is produced. Inspired by doc-router. (#1148)

🛡️ Fixes for wrong or lost data

  • Tag suggestions no longer remove paperless-gpt's own tags. Applying suggestions replaced a document's tags, which silently dropped PDF_OCR_COMPLETE_TAG, AUTO_TAG_COMPLETE, FAIL_TAG and workflow completion tags. They are now kept. Thanks @nirvdrum for the report. (#1167, closes #1156)
  • Correspondents are matched regardless of case. When the model answered MONTANA Energieversorgung for an existing Montana Energieversorgung, paperless-gpt tried to create a duplicate, paperless-ngx refused, and the whole update failed with the fail tag. Document types had the same problem. And if a correspondent still can't be used, the other suggestions are applied and the document is flagged for review instead of losing everything. Thanks @MichiFr for the report. (#1147, from #1146)
  • No more "Unknown" correspondent. When nothing fits, the prompt answers Unknown; paperless-gpt created a correspondent with exactly that name. It now leaves the correspondent unchanged. Thanks @MaxRink! (#1169, from #1162)
  • No more invented amounts in custom fields. The default custom field prompt used 1.664,58 € as its formatting example, and models sometimes returned it when a document had no amount. The prompt now only contains placeholders. Thanks @MichiFr for the report. (#1168, from #1163)
  • PDF_REPLACE only deletes after a confirmed import. On paperless-ngx 3.0 the import status wasn't read correctly, so the original could be deleted while its replacement failed to import. Beyond that, the original is now only deleted once paperless-ngx reports the import as done; if that can't be confirmed, the original is kept and the run says so. Thanks @kzaoaai! (#1112, #1136)
  • An invalid date no longer fails the whole update. One impossible date in a custom field (say 2026-09-31) made paperless-ngx reject everything; it is now skipped and the rest applied. Thanks @tdabasinskas! (#1150, closes #1149)
  • Custom field answers in other shapes are understood. Replies as a single object or a map of field names, or indented with non-breaking spaces, used to be dropped silently. Thanks @shiaho777! (#1132 from #1129)
  • Year-first dates are normalised. 2023.01.01 or 2023-01-01T15:04:05Z were dropped on apply. Thanks @shiaho777! (#1130)
  • Partial updates are reported, not hidden. In manual review a partially applied document used to show "nothing was applied"; it now names the fields that were left out and flags the document like the background processing does.

🧩 Settings, prompts and setup

  • Warnings when your data isn't persisted. In a container, settings, prompts, workflows and the undo history are lost on every update unless config/, prompts/ and db/ are volumes. paperless-gpt now warns at startup and on the Settings and AI Workflows pages, with the volume line to add. It also spots the /root/prompts mount from setups older than October 2024, which has silently had no effect since. Thanks @MichiFr for the report that led here. (#1120, from #1118)
  • Improved default prompts reach existing installations. A prompt in prompts/ that you never edited is updated to the current default on startup. Prompts you changed are never touched. (#1168)
  • Review decisions survive a reload. Thanks @shiaho777! (#1138)
  • Configurable Docling output format (DOCLING_OUTPUT_FORMAT). Thanks @mw88-personal! (#1152)
  • LOG_LEVEL now applies to OCR logging too. Thanks @mw88-personal! (#1153)

🔌 Providers

  • OPENAI_HEADERS adds HTTP headers to OpenAI-compatible requests, e.g. for API gateways. The headers stay on those requests only, are also used for LLM OCR, and are not sent along when a redirect points to another host. Thanks @zhzy0077! (#1122 from #1077)
  • Retry-After is honoured. When a provider answers 429 with a cooldown, paperless-gpt waits for it instead of retrying straight into the limit. Thanks @MaxRink! (#1170 from #1161)
  • Google AI is rate-limited like every other provider, so LLM_REQUESTS_PER_MINUTE and LLM_MAX_RETRIES apply. Thanks @shiaho777! (#1128)
  • An empty model reply counts as an empty suggestion instead of failing and retrying. Thanks @titulartarantula! (#1157, #1158)

🔧 Under the hood

  • Startup calls to paperless-ngx are bounded, so the web UI comes up even when paperless-ngx is slow; tags are fetched 1000 per page. Thanks @MaxRink! (#1169)
  • Dependency updates close a batch of known vulnerabilities in indirect dependencies. Thanks @Svector-anu! (#1151)
  • AI workflows are covered by an end-to-end test suite that runs on every pull request. (#1166)
  • go test no longer overwrites a developer's local prompts, workflows or settings. (#1134)

⚠️ Before you upgrade

  • Volumes. If the startup log or the Settings page warns that a directory is not on a volume, add the line it shows. The example compose file now mounts all three: ./prompts:/app/prompts, ./config:/app/config, ./db:/app/db.
  • AI workflows from @MrMoep's fork are moved from config/settings.json to prompts/workflows/ on the first start (one log line per workflow). Back up both first. One deliberate difference: a workflow's completion tag now replaces AUTO_TAG_COMPLETE instead of being added next to it. Workflows that break the loop rules are skipped with an error in the log.
  • Custom field prompt. If you customised prompts/custom_field_prompt.tmpl, remove or neutralise the 1.664,58 € example there. Untouched prompts are updated automatically.

🙏 Credits

Check out latest releases or
releases around icereed/paperless-gpt v0.30.0

Don't miss a new paperless-gpt release

NewReleases is sending notifications on new releases.

Get notifications