evidentlyai/evidently v0.2.0

Stable Metrics, Tests and Presets

on GitHub

Breaking Changes:

NOTE: Dashboards, Profiles, Tabs and Profile Sections are now DEPRECATED and will be completely REMOVED in the nearest releases.

Deleted NumTargetDriftPreset (use TargetDriftPreset instead)
Deleted CatTargetDriftPreset (use TargetDriftPreset instead)

Renamed Parameters:

• classification_threshold -> probas_threshold
  this afects:
  ClassificationQualityMetric , TestAccuracyScore, TestPrecisionScore, TestRecallScore, TestF1Score, TestTPR, TestTNR, TestFPR, TestFNR, TestPrecisionByClass, TestRecallByClass, TestF1ByClass, ClassificationPreset, BinaryClassificationTestPreset

• threshold-> stattest_threshold
  this afects:
  ColumnDriftMetric, TestColumnValueDrift, BinaryClassificationTestPreset, BinaryClassificationTopKTestPreset, MulticlassClassificationTestPreset

• all_features_stattest  -> stattest & all_features_threshold -> stattest_threshold
  this afects:
  DataDriftTable, DatasetDriftMetric, TestNumberOfDriftedColumns, TestShareOfDriftedColumns, DataDriftPreset, TargetDriftPreset, DataDriftTestPreset, NoTargetPerformanceTestPreset

• cat_features_stattest -> cat_stattest  & cat_features_threshold -> cat_stattest_threshold 
  this afects:
  DataDriftTable, DatasetDriftMetric, TestNumberOfDriftedColumns, TestShareOfDriftedColumns, DataDriftPreset, TargetDriftPreset, DataDriftTestPreset, NoTargetPerformanceTestPreset

• num_features_stattest -> num_stattest & num_features_stattest -> num_stattest_threshold 
  this afects:
  DataDriftTable, DatasetDriftMetric, TestNumberOfDriftedColumns, TestShareOfDriftedColumns, DataDriftPreset, TargetDriftPreset, DataDriftTestPreset, NoTargetPerformanceTestPreset

• per_feature_stattest -> per_column_stattest & per_feature_stattest -> per_column_stattest_threshold
  this afects:
  DataDriftTable, DatasetDriftMetric, TestNumberOfDriftedColumns, TestShareOfDriftedColumns, DataDriftPreset, TargetDriftPreset, DataDriftTestPreset, NoTargetPerformanceTestPreset

Renamed Tests:

• TestColumnValueDrift  -> TestColumnDrift
• TestColumnValueRegExp -> TestColumnRegExp 
• TestValueQuantile -> TestColumnQuantile

Updates:

Added top_error parameter to RegressionErrorBiasTable metric #422
Added ClassificationDummyMetric metric #445
Added RegressionDummyMetric metric #445
Added ConflictPredictionMetric metric #455
Added ConflictTargetMetric metric #455

Added API reference DRAFT https://docs.evidentlyai.com/reference/api-reference

Added new Statistical Tests:

• Apps-Singleton test #363
• Total Variation Distance (TVD) #391

Fixes:

#431
#438
#451
#458