This release introduces enhanced table of contents (TOC) extraction capabilities using LLM-powered analysis, enabling more accurate chapter structure detection and hierarchy recognition.
What's Changed
Features
-
LLM-Powered TOC Level Extraction: Implemented LLM-based analysis to automatically extract and recognize hierarchical levels in table of contents, improving chapter structure accuracy in #336
- Resolves #268
-
Enhanced TOC Page Processing: Modified the TOC detection algorithm to pass all identified TOC pages to the LLM for comprehensive analysis, rather than processing them individually in #338
- Improves the accuracy of chapter hierarchy detection
- Provides better context for LLM analysis by including all TOC pages
Refactoring
- LLM Analyzer Refactoring: Refactored
llm_analyser.pyto improve code maintainability and extensibility in #339
Background
Previously, pdf-craft used statistical analysis to detect TOC pages and extract chapter structure. While effective for basic cases, this approach had limitations in accurately determining chapter hierarchies and handling complex TOC layouts. This release introduces LLM-powered analysis to better understand TOC structure and extract hierarchical information.
How It Works
The new TOC extraction process:
- Identify TOC Pages: Uses statistical analysis to detect which pages contain table of contents
- Collect All TOC Pages: Gathers all identified TOC pages for comprehensive analysis
- LLM Analysis: Passes all TOC pages to an LLM to extract chapter titles and their hierarchical levels
- Structure Generation: Uses the extracted hierarchy information to build accurate EPUB navigation structure
This approach combines the efficiency of statistical detection with the semantic understanding capabilities of LLMs, resulting in more accurate chapter organization in the final output.
Usage
The TOC extraction improvements are automatically applied when using the appropriate toc_mode:
from pdf_craft import transform_epub, BookMeta, TocExtractionMode
# Use AUTO_DETECT for statistical analysis (default for EPUB)
transform_epub(
pdf_path="input.pdf",
epub_path="output.epub",
toc_mode=TocExtractionMode.AUTO_DETECT,
book_meta=BookMeta(
title="Book Title",
authors=["Author"],
),
)
# Use LLM_ENHANCED for LLM-powered extraction (requires toc_llm configuration)
from pdf_craft import LLM
toc_llm = LLM(
key="your-api-key",
url="https://api.openai.com/v1",
model="gpt-4",
token_encoding="cl100k_base",
)
transform_epub(
pdf_path="input.pdf",
epub_path="output.epub",
toc_mode=TocExtractionMode.LLM_ENHANCED,
toc_llm=toc_llm,
book_meta=BookMeta(
title="Book Title",
authors=["Author"],
),
)Notes
- Important: When using
TocExtractionMode.LLM_ENHANCED, thetoc_llmparameter must be configured. The conversion will fail iftoc_llmis not provided. - This feature is most beneficial for books with complex chapter hierarchies
- The statistical TOC page detection remains as the first step, with LLM analysis enhancing the extraction quality
Full Changelog: v1.0.8...v1.0.9