⭐️ Highlights

In this release, pandera adds support for ParsedFields and ParsedColumns so you can define columns/fields that are derived from other columns/fields. This unlocks a pandera.decisions API that delegates to decision models (like Jev, or a variety of open weights alternatives) to derive columns from a source column/field:

import enum
import pandas as pd
import pandera.pandas as pa
import pandera.decisions as decisions

class Department(str, enum.Enum):
    billing = "billing"
    """Payment, invoices or subscription issues."""
    technical = "technical"
    """Bugs, outages or integration problems."""
    sales = "sales"
    """Pricing, plans or account expansion."""

class Frustration(enum.IntEnum):
    calm = 0
    """Calm, simply stating facts."""
    annoyed = 1
    """Frustrated but civil."""
    angry = 2
    """Very angry, strong language."""

class Triage(pa.DataFrameModel):
    ticket_body: str
    department: Department = pa.ParsedField(
        description="Which team should handle this ticket",
        parser=decisions.Choice(),
    )
    frustration: Frustration = pa.ParsedField(
        description="How frustrated the customer appears",
        parser=decisions.Score(),
    )
    is_urgent: bool = pa.ParsedField(
        description="The message conveys time-sensitivity",
        parser=decisions.Noul(),
    )

    class Config:
        parser_source = "ticket_body"

What's Changed

• fix(pandas): filter invalid rows using complete check results by @chivector in #2611
• fix(pandas): add missing columns without an explicit dtype by @chivector in #2609
• Accept non-string column labels in schema-level unique checks by @feiiiiii5 in #2607
• docs: add financial transaction reconciliation tutorial by @Numroid in #2605
• feat(synthesis): pluggable data-synthesis backend primitives by @cosmicbboy in #2578
• fix(io): preserve datetime comparison bounds in pandas schemas by @chivector in #2613
• fix: detect pyspark on PYTHONPATH without package metadata for numpy2 patch by @cosmicbboy in #2620
• docs(polars): clarify LazyFrame validation typing by @Blues-berry in #2623
• docs(contributing): install uv without pip into the system Python by @thisisshivamtiwari in #2621
• fix: prevent RCE in tensordict load (GHSA-vpwq-239f-3cf7) by @cosmicbboy in #2627
• fix(model): don't require optional dtype params in Annotated metadata by @finnhoang05 in #2625
• docs: generalize the derived-columns spec beyond one decision model by @cosmicbboy in #2527
• fix(ibis): empty tables vacuously satisfy checks by @inchang-ing in #2629
• fix(io): restore nested tuple column labels by @bosprimigenious in #2632
• docs: fix :param names that do not match the signatures by @mmporong in #2619
• docs: update top banner for DataFusion and fix table dark-mode rendering by @cosmicbboy in #2585
• feat: map Enum and Literal to categorical data types by @cosmicbboy in #2515
• fix(polars): make Enum dtypes comparable and hashable by @DG47 in #2634
• test(pandas): pin signedness in the integer dtype check by @Trenyx in #2630
• fix(io): preserve single-level MultiIndex schemas by @bosprimigenious in #2636
• docs: advertise ParsedField/ParsedColumn and the decisions API by @cosmicbboy in #2641
• perf(polars): read the frame schema once in collect_column_info by @evnchn in #2647

New Contributors

• @chivector made their first contribution in #2611
• @Blues-berry made their first contribution in #2623
• @thisisshivamtiwari made their first contribution in #2621
• @finnhoang05 made their first contribution in #2625
• @inchang-ing made their first contribution in #2629
• @bosprimigenious made their first contribution in #2632
• @mmporong made their first contribution in #2619
• @Trenyx made their first contribution in #2630
• @evnchn made their first contribution in #2647

Full Changelog: v0.35.0...v0.36.0