TL;DR
- Breaking Changes: Check the migrations for saved models, trimming limits and the retired CBPS reference loader; missing values no longer count as zero, and refits keep predictions until explicitly refreshed.
- New Features: Optional cross-fitting uses held-out predictions for population outcome estimates (AIPW), while saved IPW/CBPS models keep the preparation details needed for reuse.
- Bug Fixes: Weighting, missing-data statistics, model reuse and diagnostics now handle more edge cases correctly, with clearer errors when inputs cannot be used safely.
- Documentation: Formula errors and weight-trimming instructions explain what went wrong and which settings to use.
- Code Quality & Refactoring: Reusing weighting models avoids duplicate predictions, and temporary bootstrap fits use less memory without changing results.
- Tests: More comparisons against recorded R results check numerical behavior; test data and source notices stay together, and Bento flags preserve the requested test selection.
Breaking Changes
-
The CBPS reference dataset is no longer included in installed balance.
balance.datasets.load_cbps_dataandload_data("sim_data_cbps")have been removed.load_data()andload_data("sim_data_01")still work as before.
Migration: use the repository-onlyload_cbps_reference()for CBPS reference examples; see Tests for the data and loader location. -
Numerical change: Missing values no longer count as zeros in weighted statistics.
Means, sample variances, variances of weighted means, standard deviations and confidence intervals now leave out a missing value and its weight, separately for each column. Missing weights are also left out; actual zero values still count. Previously, a missing value counted as zero but its weight stayed in the mean's denominator.
Migration: usefillna(0)first only if you intend to replace missing values with zero. This change does not affect inputs without missing values or weights. -
Saved models now remember how categories became model inputs.
When categories are converted into model columns, reusing a model needs its original conversion record (preprocessing_state). Unknown categories, newly missing category values, formulas whose transformations cannot be reproduced, and outdated category-column labels now raise errors instead of silently changing model columns or setting them to zero.
Migration: refit older saves missing this record, or using old category-column labels containing NumPy type names. Include intended categories when fitting; for newly missing values, use a learner that supports accepting missing categories directly (native missing-category handling) if you cannot include them at fit time. -
Reusing IPW/CBPS on new data needs repeatable transformations.
transformations="default",quantizeandfct_lumpdepend on the data. They cannot be applied to different covariate data unless their original parameters are retained; reusing the model on the same data still works.
Migration: fit withtransformations=Noneor repeatable (deterministic) transformations that keep their original parameters. Fit a new model on the scoring sample/target if you want new bins. -
Rake recognizes
1and1.0as the same category.
Integer-valued floats, including pandas categorical levels, now get labels without.0in fitting andpredict_weights(). This avoids false sample/target mismatches withtransformations=None.
Migration: refit older saved rake models with labels such as"1.0"before reusing them, even on their original rows. -
trim_weightscatches overlapping limits and impossible totals.
A singleweight_trimming_percentiletrims both tails. Values of at least0.5, two nonzero tails summing to at least1, or calculated lower/upper bounds that cross now raiseValueErrorrather than clipping arbitrarily;0.95previously collapsed CBPS weights to two values.
Migration: use0.05for 5% at each tail, or(0.0, 0.05)for only the top 5%. A one-sided 100% limit is still allowed when the other tail is disabled. Zero weights stay zero when preserving an already-zero total; preserving a positive original total raises an error if trimming removes all weight. -
Refitting or transferring an outcome model keeps existing predictions.
Sample and target prediction columns, including manually added ones, are kept rather than refreshed automatically. Warnings name columns that may be out of date; these warnings are not errors. Predicting on the target still copies a shared target before writing.
Migration: rerunframe.predict_outcomes(on="target"), oron="sample", when you need predictions from the new model. -
Bootstrap intervals need the model's original training data.
Stored outcome models now keep one extra copy of their raw training inputs; deep-copying a frame copies those inputs too.
Migration: refit older models without these inputs to get bootstrap confidence intervals, or useci_method="analytic"for uncertainty that treats the predictions as fixed.
New Features
-
BalanceFrame.aipw(n_folds=..., random_seed=...)can cross-fit the outcome model.
This optional mode fits on groups of training rows and predicts rows left out of each fit;cross_fitted_aipw_point_estimateprovides the standalone function. The default full-sample estimate is unchanged.- Each fold keeps the selected variables, original fitting weights (not weights changed since fitting), and common covariate layout; it refits any scalers and learners. Stored frame, model and prediction state is untouched.
- The residual correction still uses existing adjusted sample weights. Cross-fitting the weighting model and AIPW confidence intervals are not included.
- Legacy or transferred frame models without matching records of the original fit must be refit. Weighted fits with incomplete training rows need the original fit weights passed explicitly to the standalone function.
-
IPW and CBPS save how training data were prepared for the model.
Their model dictionaries keeppreprocessing_state, which is reused when predicting weights for held-out rows. IPW saves it withstore_fit_metadata=Trueorstore_fit_matrices=True; CBPS saves it withstore_fit_metadata=True. It isNoneif the code that builds model inputs reports no preparation record.
Bug Fixes
-
Numerical change: IPW now saves the model fitted to all training rows.
With built-in logistic regression,max_de=Noneandnum_lambdas > 1,store_fit_metadata=Truepreviously saved the last cross-validation fold's model, trained on 90% of rows. Weights,model["fit"], coefficients and stored probabilities now use all rows at the chosen regularization level (lambda). This fixes differing weights fromfit()andadjust()under theBalanceFrame.fit(method="ipw")default; other paths are unchanged. -
Numerical change: Poststratify keeps useful numeric bins when values are missing.
Withna_action="add_indicator"andtransformations="default", numbers used to becomeobjectvalues and be lumped byfct_lump, often into one cell with no adjustment, instead of binned byquantize.
Fitting andpredict_weights()now transform before filling missing values, as rake does. Missing numeric values and rare missing non-numeric levels keep their own"__NaN__"cell instead of being lumped with rare levels.na_action="drop"is unchanged. -
Numerical change: Poststratify no longer invents cells for unused categories.
Pandas'groupby(observed=False)created combinations absent from the data: sample-only cells silently got zero weights withstrict_matching=True, and target-only cells got infinitecell_weight_ratiovalues. Only observed combinations are now used, as withobjectcolumns; strict matching raises when a sample cell is missing from the target.objectinputs are unchanged. -
Numerical change: Rake keeps each weight on the right row.
When training has dropped rows for missing values, filtered training weights are restored to scoring-row positions before prediction repeats that row drop, without filtering weights twice. Fitting and reuse both use retained row positions, so repeated index labels neither weight dropped rows nor duplicate retained weights. Non-finite caller-filtered training weights are rejected before restoration.
Returned weights keep the full sample index and its name. This also fixes named-indexKeyErrorfailures, a covariate namedindexreplacing the weights' index, and unwanted renaming to"index". -
Rake warns when it stops short of the requested match.
converged=1now means the final deviation is at mostconvergence_rate. Arate_tolerancestop above that limit returnsconverged=0and warns with the remaining deviation, including when available sample combinations cannot reproduce the target. The stopping rule and fitted weights have not changed. -
Formula errors are clearer without changing valid expressions.
Only standalone dots expand into variables; decimal numbers, attribute access and quoted labels are left alone. Malformed formula tokens raise actionableValueErrormessages, as does a standalone dot when no variables are available. -
Dropping rows for a formula keeps data, weights and outcomes together.
Warnings identify affected factors/columns, original sample/target row positions and labels, and short examples of inputs and calculated values. All formulas use the same retained rows; fitting and prediction use the corresponding weights and outcomes. Other row-drop helpers also align covariates and weights by position when index labels repeat. -
Category calculations keep the same model columns when saved and reused.
Formulas first use the original boolean/numeric values, then store category results in a form that survives pickling. NumPy category labels and category-comparison columns stay stable. Dropping missing rows keeps unused levels labelled the same way as observed levels; see Breaking Changes for older saves and unsupported reuse. -
Filling missing values respects column types and leaves your input alone.
Nullable numeric columns use anobjectfallback only when a string missing-value marker needs it; numeric fills keep native types. Category columns keep all levels and add markers only where needed during weighting fits and prediction. A new missing-value-indicator frame does not replace missing values in the caller's original data. -
Outcome predictions keep category meanings across batches and transfers.
The saved conversion record (preprocessing_state) survives pickling. Bootstrap fits keep the original category set and value-type rules, even for rare levels absent from a resample, without silently discarding failed predictions. Unsupported formula reuse and unknown levels in category-indicator columns raise actionable errors. -
Different outcome learners now agree on a usable input format.
The choice no longer depends on outcome-column order. Auto mode uses one-hot encoding (numeric indicator columns for categories) if any learner needs it; forcing native category inputs raises an actionable error if a learner cannot accept them. -
Numerical change: Outcome-model bootstrap intervals use the data that were actually fitted.
Resampled fits keep the original selected variables, complete training rows and actual fit weights, even after adjustment or transfer. The point estimate uses the saved model without refitting it; see Breaking Changes for older models and the extra training-data copy. -
Samplekeeps weight choices and column roles consistent.
Selecting, renaming, reading or updating active weights stays aligned with the sample's underlying responder data. Row/column filtering keeps accessors in sync; conversions preserve the frame's state, declared column roles (such as weights and outcomes), weight history and fitted outcome models. Pickled objects are rebuilt from internal frames; the guard against directSampleconstruction remains. -
In-place fitting keeps your before-fitting baseline.
fit(inplace=True)now preserves a separate unadjusted baseline as it updates the frame in place. -
Changing an outcome target does not discard the existing weighting work.
fit_outcome_model(target=...)keeps sample weights, weighting models, adjustment history and the unadjusted baseline. If target data differ, it warns that retained weights may still describe the previous population, while subsequent outcome predictions use the new target. -
Outcome summaries still work when AIPW is unavailable.
The outcome-model summary remains visible and explains which AIPW requirement is not met. -
Numerical change: Covariate views use the weights you supply.
Weight names no longer decide whether explicit weights are used. Default unnamedRangeIndexweights stay positional even if data labels are reordered. Complete, unique label matches align before length checks and ignore unused weight labels; incomplete unique matches raise rather than silently assigning weights by position. -
Numerical change: Zero or missing weights no longer move weighted quantiles.
Those rows are excluded from interpolation by position, so they cannot alter quantile comparisons (QQ plots). With zero or missing total weight, quantiles returnNaNwith a warning, not an error, including a scalar request. Inputs containing only contributing rows are unaffected by this exclusion. -
Numerical change: Distribution and mean-gap diagnostics handle zero weights and constants correctly.
Continuous Kullback–Leibler divergence (KLD) leaves out zero-weight rows before fitting its density estimate (KDE) and choosing integration limits. Absolute standardized mean difference (ASMD) stays infinite for unequal constants with zero reference standard deviation; equal constants now have zero deviation. -
Numerical change: Cramér–von Mises distance (CVMD) compares both distributions on the same basis.
Numeric and categorical comparisons use an equal mixture of sample and target probabilities, normalizing each set of weights separately, even when they have no values in common. Non-finite, negative and zero-total weights are rejected before probability normalization. -
Numerical change: Weight diagnostics count the intended observations.
Weighted-median breakdown includes the observation that first reaches 50% of weight. Kish effective sample size leaves out missing-weight rows; design effect (Deff) and effective sample proportion (ESSP) still use valid weights and require at least one positive weight. Frequency tables keep default unit weights aligned with filtered row indices. -
Command-line diagnostics keep their batch identifiers.
Every output row retains batch keys; conflicting diagnostic names get abatch_prefix. Wrong numbers of batch keys and missing keys raise errors rather than cutting off identifiers or silently losing observations before grouping. -
The diff-diff adapter accepts the upstream survey settings.
Replicate-weight column lists, stratum-assignment lists and combined-weight boolean settings are now accepted. -
CSV output respects pandas' file options.
Appending to a file, creating one only if it does not exist, text encoding and filename-based compression all keep their pandas behavior. -
Distribution plots honor your weight and histogram choices.
Single numeric seaborn plots now receive the resolved histogram default rather than the originalNone. Plotly respectsweighted=Falseand uniformweight=Nonewithout adding empty weight columns.
Documentation
-
dot_expansion()explains how to fix different formula errors.
Its Raises section distinguishes invalid variable arguments, malformed tokens and expanding a standalone dot without variables. -
IPW/CBPS trimming guidance now states the required setting.
Percentile trimming requiresweight_trimming_mean_ratio=None, rather than the default20. Conflicting options fail before fitting and explain that fix; IPW ignores both trimming options whenmax_deis set.
Code Quality & Refactoring
-
Reusing a transferred weighting model avoids a second prediction pass. Cached sample probabilities are reused after formula-induced row drops; weights and output positions are unchanged.
-
Temporary bootstrap fits use less memory. They skip unused raw-training-input copies, including fallback point-estimate fits, without changing estimates, intervals or the original model's fit record.
Tests
-
R-inspired test data stay together, with each source identified.
tests/r_inspired/contains the test inputs and reference material, including required files from nested folders. It is a mixed-license folder: sources are declared and attributed component by component, not covered by one blanket package-license claim. This organization and packaging do not change production numerical behavior. -
The CBPS reference checks use the same data outside the installed package.
The unchanged CSV/R capture and localload_cbps_reference()loader live intests/r_inspired/fixtures/cbps/; repository-only reference tests use that loader and the tutorial reads this bank directly. Original CBPS correlation guards and fixture checks are retained. -
Bento test flags no longer widen your test selection.
Flags alone run this package's test directory. When you name Python tests explicitly, leading flags do not expand that selection. -
Survey outcome tests compare with complete R predictions.
Their own fixture bank keeps the original survey API inputs, full outcome-prediction expectations and calibrated generalized-regression (GREG) expectations. -
MatchIt diagnostic tests use the original LaLonde data and weights.
Their own bank keeps matching weights and independent balance-diagnostic expectations; these comparisons do not imply equivalent causal estimates. -
Paired weight-impact tests keep the complete R expectations.
A self-contained bank uses the original LaLonde/MatchIt inputs and full paired R comparison results. -
glmnet tests keep separate inputs and expectations for each comparison.
Self-contained banks retain original inputs with complete fixed-lambda/selected-fit IPW expectations, recorded cross-validation (CV) folds and independent deviance expectations, and independent weighted-scaler expectations. -
matrixStats/survey tests cover weighted statistics on recorded inputs.
Self-contained fixtures cover weighted means, repeated selections, reliability variance and quantiles. -
recipes and forcats/hardhat tests check the resulting model inputs against R.
Separate interaction and category fixtures keep complete R training/scoring matrices and executed factor-matrix comparisons. -
Survey Kalton/badcal tests compare raking and poststratification with R.
Self-contained fixtures keep original inputs and R numerical comparisons. These banks do not claim that every R uncertainty calculation or numerical convention matches balance.
Contributors
Full Changelog: 0.23.0...0.24.0