lfnovo/open-notebook v1.13.0

v1.13.0 — Content Core 2.0: Our Biggest Content-Processing Upgrade

on GitHub

We recommend all users upgrade. v1.13.0 is our biggest content-processing upgrade yet: it rebuilds extraction on top of Content Core 2.0 (a major architecture release), adds opt-in Docling (OCR + layout-aware parsing) and Crawl4AI (JavaScript rendering) engines, brings EPUB and Reddit support, moves PDF extraction to MIT-licensed pdfplumber, fixes audio extraction, and keeps the default image lean by making the heavy runtimes opt-in. Plus a batch of podcast correctness fixes. Validated with the release confidence process: full backend/frontend suites, the fresh-install + upgrade image gate, and manual verification of the opt-in runtimes, real podcast/TTS and PDF extraction on the pushed image.

✨ A major content-processing upgrade (Content Core 2.0)

Extraction now runs on content-core 2.x — a ground-up refactor of the extraction stack. What it means for you:

• Docling document engine (opt-in) — real OCR for scanned PDFs and images plus layout-aware parsing. Toggle OCR in Settings → Content Processing (on by default; turn it off to speed up text-native docs). (#1104)
• Crawl4AI URL engine (opt-in) — renders JavaScript-heavy pages locally with no API key, or offloads to a Crawl4AI server (CRAWL4AI_API_URL). Selectable alongside Firecrawl, Jina and Simple. And your persisted engine choice now actually takes effect — the source-processing graph reads the saved Content Settings and passes them to content-core (it previously always ran auto and silently ignored the selection). (#432)
• New formats — EPUB books are now supported, and Reddit links extract the post and its comments automatically.
• Better PDF & audio — PDF extraction moves to MIT-licensed pdfplumber (replacing AGPL PyMuPDF), and audio moves to direct ffmpeg calls — faster, and it fixes extraction from MP3 files carrying chapter metadata. (#1103)
• Faster, clearer failures — unsupported uploads are rejected instantly at ingestion with a 415 naming the detected type, instead of a background job retrying for ~an hour. (#975)

🪶 Lean by default, powerful on demand

• The heavy runtimes are opt-in. Docling and local Crawl4AI are no longer bundled into every image — they install automatically on first container startup when enabled. Set OPEN_NOTEBOOK_ENABLE_DOCLING=true and/or OPEN_NOTEBOOK_ENABLE_CRAWL4AI=true; downloads are cached on the /app/data volume (only the first boot is slow), and a failed install degrades gracefully (the app still starts, the engine is reported unavailable) with loud logs. A new GET /api/capabilities probe reports what's actually installed, and Settings → Content Processing disables the Docling/Crawl4AI engines and the OCR toggle (with an env-var hint) until their runtime is available — so the UI never advertises an engine that isn't there. A remote Crawl4AI server needs no local install. Recorded as ADR-007. (#1122)
• Source extraction now logs the effective engine (url_engine / document_engine / docling_ocr) at INFO right before it runs, so you can confirm which engine actually processed a source instead of guessing. (#1125)

🐛 Notable fixes

• Podcast correctness: renaming a speaker profile no longer breaks the episode profiles using it (references are stored by record ID, not name); episode audio paths are stored relative to the podcasts folder so previously generated episodes survive a DATA_FOLDER relocation; episode cards resolve and show the real outline/transcript/speaker model instead of "— / —"; and podcast generation now honors an explicitly supplied speaker_profile all the way into the generation command. (#630, #1030, #1114, #1058)
• Clicking a chat/Ask citation that points at a deleted source, insight or note now shows a friendly "this content no longer exists" state in all three dialogs. (#455)
• Source insights get created/updated timestamps and the API returns null (not the literal string "None") for legacy insights. (#1045)
• Optional model defaults (transformation, tools, large context, TTS, STT) can now be cleared via PUT /api/models/defaults (chat and embedding still can't be null). (#1091)
• Creating a transformation from the empty state works again — after deleting every transformation, the "New Transformation" button did nothing. (#999)
• The source detail view now fetches through the shared React Query hook (consistent caching and 404 handling), and the settings UI reads the provider list from GET /api/providers so new providers need zero frontend edits. (#1106, #1082)
• uv sync alone now provides the full dev toolchain (mypy included). (#1101)

⚠️ Behavior changes for self-hosters

1. Docling and local Crawl4AI are now opt-in. If you relied on Docling parsing/OCR or local Crawl4AI rendering, set OPEN_NOTEBOOK_ENABLE_DOCLING=true and/or OPEN_NOTEBOOK_ENABLE_CRAWL4AI=true — the first container boot then installs them (cached afterwards). The default image no longer ships Chromium or the multi-GB ML stack. A remote Crawl4AI server (CRAWL4AI_API_URL) needs no local install.
2. Persisted engine selections now take effect. If you had a document/URL engine selected in Settings → Content Processing, it was previously ignored (everything ran auto); it now actually applies. Re-check your selection after upgrading — and remember to click Save for it to persist.
3. Frontend bind override changed. Docker images force the Next.js frontend to bind 0.0.0.0; the HOSTNAME variable is no longer honored as a bind override — use the new FRONTEND_BIND_HOST variable instead. (#994)
4. Podcast profile migrations (19–22) apply automatically on startup. Profiles whose legacy model mapping never converged (e.g. a provider credential was never configured) may need you to re-pick the models/speaker in the profile form once. Podcast episodes stored under non-standard absolute paths are treated as legacy-invalid — regenerate to restore playback.
5. docker-compose.yml SurrealDB command now uses the YAML list (exec) form, so SURREAL_USER / SURREAL_PASSWORD values containing spaces are passed correctly. (#1093)

🙏 Thanks

• @pelazas — fixed the transformation editor being unreachable from the empty state (#999).
• @lfnovo — the opt-in runtime system (#1122), the Crawl4AI URL engine and persisted-engine wiring (#432), the OCR toggle (#1104), the content-core 2.x upgrade (#1103), the podcast correctness batch (#630, #1030, #1114, #1058), the 415 ingestion guard (#975), and the rest of the release.

And a collective thank-you to everyone who filed issues and reported bugs that shaped this release — the deleted-citation UX, the engine-selection gap, the podcast speaker/rename cases, and more.

Full details in the CHANGELOG.