New Features
-
Image type support in IDataView
PR#3263 added support for in-memory image as a type in IDataView. Previously it was not possible to use an image directly in IDataView, and the user had to specify the file path as a string and load the image using a transform. The feature resolved the following issues: 3162, 3723, 3369, 3274, 445, 3460, 2121, 2495, 3784.Image type support in IDataView was a much requested feature by the users.
Sample to convert gray scale image in-Memory | Sample for custom mapping with in-memory using custom type
-
Super-Resolution based Anomaly Detector (preview, please provide feedback)
PR#3693 adds a new anomaly detection algorithm to the Microsoft.ML.TimeSeries nuget. This algorithm is based on Super-Resolution using Deep Convolutional Networks and also got accepted in KDD'2019 conference as an oral presentation. One of the advantages of this algorithm is that it does not require any prior training and based on benchmarks using grid parameter search to find upper bounds it out performs the Independent and identically distributed(IID) and Singular Spectrum Analysis(SSA) based anomaly detection algorithms in accuracy. This contribution comes from the Azure Anomaly Detector team.Algo Precision Recall F1 #TruePositive #Positives #Anomalies Fine tuned parameters SSA (requires training) 0.582 0.585 0.583 2290 3936 3915 Confidence=99, PValueHistoryLength=32, Season=11, and use half the data of each series to do the training. IID 0.668 0.491 0.566 1924 2579 3915 Confidence=99, PValueHistoryLength=56 SR 0.601 0.670 0.634 2625 4370 3915 WindowSize=64, BackAddWindowSize=5, LookaheadWindowSize=5, AveragingWindowSize=3, JudgementWindowSize=64, Threshold=0.45 Sample for anomaly detection by SRCNN | Sample for anomaly detection by SRCNN using batch prediction
-
Time Series Forecasting (preview, please provide feedback)
PR#1900 introduces a framework for time series forecasting models and exposes an API for Singular Spectrum Analysis(SSA) based forecasting model in the Microsoft.ML.TimeSeries nuget. This framework allows to forecast w/o confidence intervals, update model with new observations and save/load the model to/from persistent storage. This closes following issues 929 and 3151 and was a much requested feature by the github community since September 2018. With this change Microsoft.ML.TimeSeries nuget is feature complete for RTM.Sample for forecasting | Sample for forecasting using confidence intervals
Bug Fixes
Serious
-
Math Kernel Library fails to load with latest libomp: Fixed by PR#3721 this bug made it impossible for anyone to check code into master branch because it was causing build failures.
-
Transform Wrapper fails at deserialization: Fixed by
PR#3700 this bug affected first party(1P) customer. A model trained using NimbusML(Python bindings for ML.NET) and then loaded for scoring/inferencing using ML.NET will hit this bug. -
Index out of bounds exception in KeyToVector transformer: Fixed by PR#3763 this bug closes following github issues: 3757,1751,2678. It affected first party customer and also github users.
Other
- Download images only when not present on disk and print warning messages when converting unsupported pixel format by PR#3625
- ML.NET source code does not build in VS2019 by PR#3742
- Fix SoftMax precision by utilizing double in the internal calculations by PR#3676
- Fix to the official build due to API Compat tool change by PR#3667
- Check for number of input columns in concat transform by PR#3809
Breaking Changes
None
Enhancements
- API Compat tool by PR#3623 ensures future changes to ML.NET will not break the stable API released in 1.0.0.
- Upgrade the TensorFlow version from 1.12.0 to 1.13.1 by PR#3758
- API for saving time series model to stream by PR#3805
Documentation and Samples
- L1-norm and L2-norm regularization documentation by PR#3586
- Sample for data save and load from text and binary files by PR#3745
- Sample for LoadFromEnumerable with a SchemaDefinition by PR#3696
- Sample for LogLossPerClass metric for multiclass trainers by PR#3724
- Sample for WithOnFitDelegate by PR#3738
- Sample for loading data using text loader using various techniques by PR#3793
Remarks
- Microsoft.ML.TensorFlow, Microsoft.ML.TimeSeries, Microsoft.ML.OnnxConverter, Microsoft.ML.OnnxTransformer nugets are expected to be upgraded to release in ML.NET 1.2 release. Please give them a try and provide feedback.