evaluators.xai package

Submodules

evaluators.xai.XAI module

class evaluators.xai.XAI.XAI(config: dict, model: Module, dataloader: DataLoader)

Bases: object

Class for handling eXplainable AI (XAI) within the toolbox. It explains the predictions of the deep learning models by using any attribution method from captum.attr such as Saliency, InputXGradient, or IntegratedGradients. It should work for all kinds of output data (categorical / regression) and all kinds of input / output dimensionality. However, only plots for some input / output dimensionality combinations are currently available. In any case, a general plot showing average overall attributions of input features vs output features should always be generated. It also supports aggregation of the outputs (e.g. by mean) over specific dimensions, and aggregation of the outputs based on a mask (e.g.: consider only predictions that have low error, consider only predictions given a certain ground truth class, or based on a custom compartmentalization).

Parameters:

  • config –

    Global configuration dictionary for the case, containing all hyperparameters. Here we use: config[‘debug’]: if >= 1, it runs the XAI only for config[‘debug’] samples and prints some extra info config[‘task’]: ‘Classification’ or ‘ImpactAssessment’ (regression) config[‘data’][‘num_classes’]: Number of output classes or features that have been predicted config[‘data’][‘features’]: All feature names config[‘data’][‘features_selected’]: Indices of the features that were selected from config[‘data’][‘features’] config[‘save_path’]: XAI plots will be saved in config[‘save_path’] / ‘xai’ config[‘evaluation’][‘xai’][‘params’][‘out_agg_dim’]: None or tuple, if tuple, output dims wrt which we aggregate the output config[‘evaluation’][‘xai’][‘params’][‘mask’]: Used for masked aggregation, it selects the aggregation mode.

    It must be one of [“none”, “events[-full]”, “correctness[-full]”, “labels[-full]”, “custom[-full]”]

    config[‘evaluation’][‘xai’][‘params’][‘type’]: Captum attribution method (str) config[‘data’][‘data_dim’] & config[‘arch’][‘input_model_dim’]: Used by adapt_variables

  • model – Pytorch model

  • dataloader – The dataloader used by the Pytorch model

xai(events: Tensor | None = None)

Performs XAI over a dataloader, saving the plots of the attributions independently for each sample

Parameters:

events – tensor with the same shape as model outputs y indicating to what event that output corresponds (or 0 if it corresponds to no event). It cannot be None if agg_mode == “events[-full]”

evaluators.xai.XAI.agg_y(y: ~torch.Tensor, mask: ~torch.Tensor | None = None, out_agg_dim: tuple | None = (2, 3), num_classes: int = -1, cross_agg: bool = False, agg_dim_fn=<built-in method mean of type object>, agg_mask_fn=<built-in method mean of type object>) → Tensor

This wrapper aggregates an output tensor over dimensions out_agg_dim using agg. function agg_dim_fn. For output maps / images this makes it easier to attribute with respect to a single output scalar, as opposed to individual pixel output attribution.

Instead, if a mask is provided (not None), it uses this mask to aggregate over the classes, either by simple direct masking if cross_agg=False where the mask just selects some wanted pixels from each output class, or by generating all possible combinations of the output classes with the masking classes if cross_agg=True. In this last case, the new number of classes is the product of original_classes x aggregation_classes. Samples selected by the mask are then aggregated using agg_mask_fn

It also expands the output channel dimension if it has only a size of 1 and num_classes>1. E.g.: it expands from binary classification to 2-class multiclass output

Note: Output classes must be located in dimension 1 of y, and for masked aggregation, dimension 1 must have a size of 1

Parameters:

  • y – Model predictions or labels to be aggregated

  • mask – If provided (not None), mask to aggregate over the classes

  • out_agg_dim – None or tuple, if tuple, output dims wrt which we aggregate the output

  • num_classes – Number of output classes or features that have been predicted

  • cross_agg – If True, generate all possible combinations of the output classes with the masking classes

  • agg_dim_fn – pytorch function used to aggregate y if not using masked aggregation. It must accept dim param

  • agg_mask_fn – pytorch function used to aggregate y if using masked aggregation. It must accept axis param

Returns:

aggregated y

evaluators.xai.XAI.attribute(config: dict, model: Module, dataloader: DataLoader, num_classes: int, out_agg_dim: tuple | None, agg_mode: str, events: Tensor | None = None, debug: bool = False) → Tuple[Dict[str, ndarray], Dict[str, ndarray], Dict[str, ndarray], Dict[str, ndarray], List[int], List[int]]

Attribute over an arbitrary amount of input and output dimensions. It also supports aggregation of the outputs (e.g. by mean) over specific dimensions, and aggregation of the outputs based on a mask (e.g.: consider only predictions that have low error, consider only predictions given a certain ground truth class, or based on a custom compartmentalization).

Parameters:

  • config – Global configuration dictionary for the case, containing all hyperparameters. Here we use: config[‘evaluation’][‘xai’][‘params’][‘out_agg_dim’]: None or tuple, if tuple, output dims wrt which we aggregate the output config[‘evaluation’][‘xai’][‘params’][‘type’]: Captum attribution method (str) config[‘data’][‘data_dim’] & config[‘arch’][‘input_model_dim’]: Used by adapt_variables

  • model – Pytorch model

  • dataloader – The dataloader used by the Pytorch model

  • num_classes – number of output classes / features for the problem

  • out_agg_dim – tuple of dimensions wrt which we aggregate the output, or None, to perform no dimension-wise aggregation

  • agg_mode – Used for masked aggregation, it selects the aggregation mode. It must be one of [“none”, “events[-full]”, “correctness[-full]”, “labels[-full]”, “custom[-full]”]

  • events – tensor with the same shape as model outputs y indicating to what event that output corresponds (or 0 if it corresponds to no event). It cannot be None if agg_mode == “events[-full]”

  • debug – if >= 1, it runs the XAI only for debug samples and prints some extra info

Returns:

tuple containing: Dictionaries with a key for every attributed event with numpy arrays:

attributions: array with shape ([out x, out y, out t], out classes, [in x, in y, in t], in features) inputs: array with shape ([in x, in y, in t], in features) labels: array with shape ([out x, out y, out t], [out classes]) predictions: array with shape ([out x, out y, out t], out classes)

List with the shape of x and y after processing:

x_shape: list containing [[in x, in y, in t], in features] y_shape: list containing [[out x, out y, out t], out classes]

evaluators.xai.XAI.event_at_positon(arr: ndarray, t: int, position: str = 'end') → ndarray

Takes an arr of shape (*), and creates a new one of shape (t, *)

This is used for processing arrays before passing them to plot_attributions_1d, in the case where there is a sinle output (instead of one output for every time step)

Parameters:

  • arr – array to transform, shape (*)

  • t – dimensionality of the first dimension of arr after processing

  • position –

    where to place the original array with respect to the final array position == ‘end’: the original arr is at the last position of the 0th dimension,

    and the rest of the elements are zero.

    position == ‘beginning’: the original arr is at the first position of the 0th dimension,

    and the rest of the elements are zero

    position == ‘all’: the original arr is repeated t times over t index 0

Returns:

transformed array with shape (t, *)

evaluators.xai.XAI.get_agg_mask(agg_mode: str, y: Tensor, y_labels: Tensor, sample: Dict[str, Tensor], events: Tensor | None = None, error_threshold: float = 0.5) → Tensor

Compute the aggregation mask according to the agg_mode

Parameters:

  • agg_mode – It selects the aggregation mode. It must be one of [“none”, “events[-full]”, “correctness[-full]”, “labels[-full]”, “custom[-full]”]

  • y – Model predictions

  • y_labels – Labels

  • sample – The dictionary returned by the dataloader containing the batch

  • events – tensor with the same shape as model outputs y indicating to what event that output corresponds (or 0 if it corresponds to no event). It cannot be None if agg_mode == “events[-full]”

  • error_threshold – Threshold to be used for agg_mode == ‘correctness[-full]’

Returns:

tensor of the final mask to be used for masked aggregation

evaluators.xai.XAI.get_multiindex_df(data: ndarray, rows: List[List[str]], columns: List[List[str]], row_names: List[str] | None = None, column_names: List[str] | None = None, default_name: str = 'values') → DataFrame

Builds a multiindex + multicolumn pandas dataframe. For instance, from a data matrix of shape (a, b, c, d), if len(rows) = 2, and len(columns) = 2, the output dataframe will have two column levels, two index levels, and a x b x c x d total rows. It is assumed that rows are taken from the first dimensions of data, and then columns are taken from the remining dimensions

Parameters:

  • data – array with a shape that is consistent with the rows + columns provided

  • rows – a list of lists that will be used to build a MultiIndex, optionally empty list. For instance, the list at position 0 will contain a label for each of the features of data in the 0th dimension.

  • columns – a list of lists that will be used to build a MultiIndex, optionally empty list For instance, the list at position 0 will contain a label for each of the features of data in the 0th dimension.

  • row_names – a list of row names for the final DataFrame, len(row_names) = len(rows)

  • column_names – a list of column names for the final DataFrame, len(column_names) = len(column_names)

  • default_name – the default name to use for row and columns if they are not provided

Returns:

MultiIndex pd.DataFrame containing the data

evaluators.xai.XAI.plot_attributions_1d(attributions: ndarray, inputs: ndarray, labels: ndarray, predictions: ndarray, masks: ndarray, feature_names: List[str], class_names: List[str], plot_classes_predictions: List[str] | None = None, plot_classes_attributions: List[str] | None = None, figsize: Tuple[float] = (10, 10), color_list=['tab:blue', 'tab:orange', 'tab:green', 'tab:red', 'tab:purple', 'tab:brown', 'tab:pink', 'tab:gray', 'tab:olive', 'tab:cyan'], title: str | None = None, outlier_perc: float = 2.0, margin_perc: float = 25.0, alpha: float = 0.8, attr_factor: float = 0.5, attr_baseline: str = 'middle', kind: str = 'stacked', names_position: str = 'left', label_fontsize: float = 11, is_classification: bool = True, **kwargs) → Tuple[Figure, axis]

Plot 1D attributions (e.g. of a inputs) given an output timestep. If you want to see absolute attributions, pass np.abs(attributions) instead.

Parameters:

  • attributions – array with shape (out classes, in t, in features)

  • inputs – array with shape (in t, in features)

  • labels – array with shape (out t, [out classes]) out classes is optional

  • predictions – array with shape (out t, out classes)

  • masks – array with shape (out t, out classes)

  • feature_names – List of in features names

  • class_names – List of out classes names

  • plot_classes_predictions – Classes to plot for predictions. If None, first class is ignored if there are more than 2 classes

  • plot_classes_attributions – Classes to plot for attributions. If None, first class is ignored if there are more than 2 classes

  • figsize – Matplotlib figsize

  • timesteps – Global x values

  • timestep – output timestep with respect to which which attributions are plotted. If None, use the first timestep where the ground truth output class != 0

  • color_list – list of Matplotlib-compatible colors

  • title – title of the plot

  • outlier_perc – Get rid of N% of attribution outliers

  • margin_perc – Add some % of margin to feature plots

  • alpha – transparency of the bars

  • attr_factor – multiply attribution values by this number (e.g., if 0.5, the attribution can only take maximum 50% of the height of the plot. Recommended: 0.5 for signed attributions, and 1 for absolute attributions (this is done by default)

  • attr_baseline – Where to place the attribution bars with respect to the plot. attr_baseline=’middle’: start attribution bars from the middle of the plot attr_baseline=’feature’: start attribution bars from the feature value for that timestep

  • kind – How to plot attributions: kind=’stacked’: plot attributions for each class stacked on top of each other kind=’sidebyside’: plot attributions for each class side by side

  • names_position – Where to position the names of the features names_position=’left’: position them to the left of the plot names_position=’top’: position them to the top of the plot

  • label_fontsize – font size of the labels

  • is_classification – whether it is a classification or a regression problem

  • *kwargs –

    kwargs to be passed to plt.subplots

Returns:

Matplotlib figure and ax

evaluators.xai.XAI.plot_attributions_nd(data: ndarray, x_shape: List[int], y_shape: List[int], feature_names: List[str], class_names: List[str], figsize: Tuple[float] = (17, 9)) → Tuple[Figure, axis]

Plots average attributions over all dimensions except for output classes and input features It should work for any kind of model and input / output dimensionality

Parameters:

  • data – attributions array with shape ([out x, out y, out t], out classes, [in x, in y, in t], in features)

  • x_shape – List [[in x, in y, in t], in features]

  • y_shape – List [[out x, out y, out t], out classes]

  • feature_names – List with the names of the input features

  • class_names – List with the names of the output classes / features

  • figsize – figsize to pass to plt.subplots

Returns:

Matplotlib figure and ax

Module contents