Metrics

The metrics module provides a suite of functions to evaluate the performance of the machine learning algorithm. There metrics module is divided into two packages: scikit-learn for Outlier Detection and TorchMetrics for Detection and Impact Assessment. This means over 100+ evaluation metrics are implemented to evaluate the performance of the model.

Implementation

The metrics module is implemented in the evaluation module, in which the metrics are defined. To use the metrics from the package, substitute Metric_1 in the code snippets below for the name of the class you want to use (e.g roc_auc_score for AUROC from scikit-learn or AUROC for TorchMetrics)

For OutlierDetection, specify also if the input to the metrics is probabilistic(y_scores) or the predictions(y_pred) with the parameter probabilities (for more information, see scikit-learn documentation for each specific metric). Then, define as many metrics as you want as follows:

evaluation:
    metrics:
        Metric_1: {probabilities: 'True/False', parameter_1: 'value_parameter_1'}
        ...

For Detection and Impact Assessment, define as many metrics as you want as follows:

evaluation:
    metrics:
        Metric_1: {parameter_1: 'value_parameter_1'}
        ...