Bugfixes
-
#8014: Setting
model_confidence=cosine
inDIETClassifier
,ResponseSelector
andTEDPolicy
is deprecated and will no longer be available. This was introduced in Rasa Open Source version2.3.0
but post-release experiments suggest that using cosine similarity as model's confidences can change the ranking of predicted labels which is wrong.model_confidence=inner
is deprecated and is replaced bymodel_confidence=linear_norm
as the former produced an unbounded range of confidences which broke the logic of assistants in various other places.We encourage you to try
model_confidence=linear_norm
which will produce a linearly normalized version of dot product similarities with each value in the range[0,1]
. This can be done with the following config:- name: DIETClassifier model_confidence: linear_norm constrain_similarities: True
This should ease up tuning fallback thresholds as confidences for wrong predictions are better distributed across the range
[0, 1]
.If you trained a model with
model_confidence=cosine
ormodel_confidence=inner
setting using previous versions of Rasa Open Source, please re-train by either removing themodel_confidence
option from the configuration or setting it tolinear_norm
.model_confidence=cosine
is removed from the configuration generated by auto-configuration.