Counterfactual Explanation Bake-Off: A Review and Experimental Evaluation for Time Series Classification

Li, Peiyu; Bahri, Omar; Boubrahimi, Soukaina Filali; Hamdi, Shah Muhammad

doi:10.1007/s10994-026-07056-4

Counterfactual Explanation Bake-Off: A Review and Experimental Evaluation for Time Series Classification

Open access
Published: 06 May 2026

Volume 115, article number 114 (2026)
Cite this article

You have full access to this open access article

Download PDF

Machine Learning Aims and scope Submit manuscript

Counterfactual Explanation Bake-Off: A Review and Experimental Evaluation for Time Series Classification

Download PDF

Peiyu Li¹,
Omar Bahri¹,
Soukaina Filali Boubrahimi¹ &
…
Shah Muhammad Hamdi¹

1603 Accesses
Explore all metrics

Abstract

Machine learning models often excel in prediction tasks but frequently lack interpretability, limiting their use in critical domains where understanding and trust are essential. Counterfactual explanations bridge this gap by identifying the minimal feature modifications necessary to alter a model’s prediction to a desired outcome. This paper offers a comprehensive review and evaluation of counterfactual explanation methods specifically for univariate time series classification. By categorizing methods based on their underlying principles, we highlight their strengths, limitations, and the inherent compromises between properties such as validity, proximity, sparsity, and plausibility. Our experimental evaluation assesses these key properties across diverse datasets. The results make it evident that no current method achieves all these properties simultaneously. By synthesizing advancements and pinpointing areas for improvement, this work aims to guide future research and foster the development of interpretable AI solutions for time series applications.

Interactive Counterfactual Generation for Univariate Time Series

Denoising Optimization-Based Counterfactual Explanations for Time Series Classification

Streaming Counterfactual Subset Selection with Bias Awareness

1 Introduction

Machine learning has advanced rapidly to address real-world classification challenges and automate decision-making in various domains. Time series classification (TSC), in particular, has gained significant attention due to its applications in fields such as electrocardiogram (ECG) signal analysis (Pyakillya et al., 2017), motion data recognition (Yao et al., 2017), and industrial monitoring (Susto et al., 2018). Despite these advancements, many machine learning models operate as "black boxes," offering high accuracy but lacking interpretability (Pasquale et al., 2015; Rudin, 2019). This opacity makes it difficult to trace how specific features contribute to the final decision, particularly for models with millions of learned parameters.

In sensitive domains such as healthcare, finance, and applications affecting human life, this lack of transparency poses significant concerns. Decision-makers must not only trust the predictions but also ensure that the reasoning aligns with human understanding and ethical standards. When model decisions cannot be explained, stakeholders struggle to understand their rationale, undermining trust and limiting the adoption of machine learning in critical applications.

To address these challenges, the field of explainable AI (XAI) (Arrieta et al., 2020) has emerged with the goal of making machine learning models more interpretable. XAI techniques provide transparent explanations for model decisions while maintaining high levels of performance. Among various explanation techniques, counterfactual explanations (Wachter et al., 2017) have gained significant attention due to their intuitive nature. A counterfactual explanation answers the question, “What changes to the input would lead to a different outcome?" By providing alternative scenarios that result in different model predictions, counterfactuals offer a post-hoc explanation that helps users understand the conditions under which a model’s decision would change. This is especially useful in cases where actionable insights are needed, such as understanding what a loan applicant could have done differently to receive approval or how a patient’s medical history could have been modified to change a diagnosis.

Within the realm of time series classification, counterfactual explanation methods offer a promising avenue for enhancing interpretability (Delaney et al., 2021; Ates et al., 2021). These methods identify the smallest changes to input features that would result in a different classification outcome, providing users with valuable insights into model behavior. Time series data, however, introduces unique challenges due to its sequential nature and temporal dependencies. The complexity of maintaining sequential patterns and plausibility in the generated counterfactuals makes time series data more difficult to handle compared to tabular data. Understanding how different time steps influence the model’s decision-making process is essential for effective interpretation, particularly in high-stakes environments where timely, accurate decisions are critical. For example, consider a traffic management system that classifies pedestrian activity levels based on sensor data. A counterfactual explanation model could identify minimal changes in foot traffic patterns—such as reducing pedestrian flow during specific peak intervals through rerouting strategies or event rescheduling—that would alter the classification outcome from “overcrowded” to “moderate.” Such counterfactuals provide actionable insights by highlighting precise temporal segments where targeted interventions may mitigate congestion, thereby informing decision-making processes for urban planners and traffic management authorities.

While counterfactual explanation techniques have been extensively developed and evaluated for tabular data, their application to time series classification has been relatively underexplored. Despite the significant potential of counterfactuals to bridge the interpretability gap in time series models, there remains a notable lack of comprehensive reviews and evaluations tailored specifically to this domain. This work seeks to address this gap by providing a comprehensive review and experimental evaluation of counterfactual explanation methods specifically tailored for time series classification.

In this work, we provide an empirical examination of state-of-the-art counterfactual explanation methods for time series classification. Through extensive experiments on diverse datasets, we demonstrate the strengths and limitations of existing methods, offering valuable insights for researchers and practitioners. Our findings highlight opportunities for further advancements and lay the groundwork for future research in this critical area of machine learning.

2 Related Works

In recent years, there has been a burgeoning interest in explainable artificial intelligence (XAI) and interpretable machine learning (Goebel et al., 2018). Various taxonomies categorizing the problems have been published (Guidotti et al., 2018; Bodria et al., 2023; Carvalho et al., 2019; Gilpin et al., 2018; Li et al., 2020; Murdoch et al., 2019; Samek et al., 2019). A fundamental classification distinguishes between intrinsic interpretability, also known as explanation by design, and post hoc interpretability, which deals with black-box models (Adadi & Berrada, 2018; Guidotti et al., 2018). Further, Martens et al. (2007) delineates black-box explanations into model explanations, outcome explanations, and black-box inspections. Model explanations, aimed at elucidating a model’s overall logic, are typically addressed by global explainers (Craven & Shavlik, 1995). In contrast, outcome explanations focus on the rationale behind specific model outputs, often explored by local explainers (Ribeiro et al., 2016). Black-box inspections involve directly analyzing the internal structure or parameters of a model to understand its behavior, such as examining feature attributions or activations in neural networks (Zeiler & Fergus, 2014). An important distinction is also made between model-specific and model-agnostic methods, depending on whether the explanation technique applies solely to a particular model or across various models (Adadi & Berrada, 2018; Carvalho et al., 2019; Guidotti et al., 2018). Gilpin et al. (2018) elaborates on the nuances between explanation, interpretability, and explainability. Moreover, Arrieta et al. (2020) provides a taxonomy specifically tailored to deep learning model explainers, while Carvalho et al. (2019) discusses both quantitative and qualitative evaluations of explanation methodologies.

The literature on XAI and interpretable ML not only spans across the domains of machine learning and artificial intelligence but also extends into social studies (Byrne, 2019), recommendation systems (Zhang et al., 2020), and domain-specific applications (Tjoa & Guan, 2020). The term “interpretability" often refers to a model’s passive characteristic that renders it comprehensible to humans, whereas “explainability" denotes an active effort to make the model’s decision-making process clear.

Counterfactual explanations, within the scope of these discussions, are classified as post-hoc local explanations. They serve to uncover the rationale behind a decision made by a pre-trained black-box machine learning system on a particular instance, thereby acting as an active method of explanation. These explanations are not inherent to the model but are generated to elucidate the specific conditions under which the model’s output would change, thereby enhancing the transparency of machine learning systems.

Wachter et al. (2017) introduced w-CF, the first counterfactual explanation model within the context of explainable AI (XAI). This model generates counterfactual explanations by optimizing a loss function that combines prediction loss and L1 distance loss, where the prediction loss steers the counterfactual toward the desired target class, and the L1 distance loss ensures proximity to the original instance. Building on this foundational work, several methods have been developed to enhance the performance of w-CF from various perspectives (Dhurandhar et al., 2018; Mothilal et al., 2020; Van Looveren & Klaise, 2021). For example, the Contrastive Explanation Method (CEM) (Dhurandhar et al., 2018) introduces an L2 reconstruction error loss through an autoencoder to ensure that counterfactuals remain close to the data manifold. DiCE (Mothilal et al., 2020) incorporates diversity constraints to generate diverse counterfactual instances, while Counterfactual Explanations Guided by Prototypes (CEGP) (Van Looveren & Klaise, 2021) accelerates the counterfactual search process by introducing a prototype loss, thereby eliminating the computational bottleneck associated with numerical gradient evaluations in black-box models.

The aforementioned methods have been predominantly applied to tabular and image data; however, counterfactual explanation techniques for time series data remain comparatively underdeveloped. Native Guide (NG) (Delaney et al., 2021), the pioneer model specifically tailored for univariate time series, adopts an instance-based approach. This model identifies and modifies discriminative regions of the time series by employing the nearest unlike neighbor (NUN) from the target class. NG offers two distinct models for generating counterfactual explanations: NG-CAM and NG-DBA. The NG-CAM model generates counterfactuals using class activation mapping to identify the most distinguishable segments, replacing them with those from an in-sample NUN. Although NG-CAM ensures proximity, sparsity, plausibility, and diversity in its counterfactuals, it relies on class activation weights to direct perturbations, which can be limiting when the underlying classifier (e.g., k-nearest neighbor model) does not provide a feature-weight vector w. This can be a limitation when the underlying machine learning classifier, like k-nearest neighbor, does not provide a feature-weight vector w. In such cases, NG-DBA offers an alternative by using Dynamic Time Warping Barycenter Averaging (DBA) between the query instance $\textbf{x}$ and the native counterfactual $\mathbf {x_{NUN}'}$ to generate the counterfactual explanation. However, this approach often results in low sparsity because it alters the entire time series rather than focusing on specific, discriminative subsequences.

Further developments include SG-CF by Li et al. (2022a), which builds upon w-CF. SG-CF uses shapelet-based loss functions to create more contiguous counterfactual explanations, fixing the perturbation window to the presence of a shapelet and minimizing the distance between the counterfactual and the shapelets. Another method, MG-CF (Li et al., 2022b), also for univariate time series, uses shapelet replacement to guide perturbations. While SG-CF and MG-CF improve the contiguity and interpretability of counterfactual explanations, their reliance on pre-mined shapelets introduces significant computational challenges. TimeX (Boubrahimi et al., 2022), another method based on w-CF, introduces a Dynamic Barycenter Average Loss that minimizes the distance between the counterfactual and a centroid approximation of the target class. This centroid is designed to be maximally representative of the class, with the aim of improving both the interpretability and the contiguity of the counterfactual explanations generated. Although TimeX achieves high contiguity and proximity by perturbing only a single contiguous segment, it does not always guarantee the generation of valid counterfactuals. Yan and Wang (2023) introduces a causal generative model CounTS for producing actionable and feasible counterfactual explanations in time-series prediction, with a focus on preserving confounders and ensuring identifiability. LASTS (Spinnato et al., 2023) is a model-agnostic explainer that integrates saliency maps, prototype-based counterexemplars, and subsequence-based rules to provide complementary, human-interpretable explanations for time series classifiers. It leverages subsequence discovery and surrogate models to generate interpretable decision rules alongside example-based reasoning. Hao et al. (2023) introduces a visual analytics framework, TimeTuner, that integrates counterfactual explanations with interactive visualization to diagnose time-series representations for forecasting models. It supports exploring the impact of different transformation methods (e.g., smoothing, sampling) on prediction performance using coordinated multiple views and explanation metrics such as SHAP and correlation.

Wang et al. (2021) introduced LatentCF++, a counterfactual generation method that perturbs latent representations of time series using an autoencoder-based framework. Glacier (Wang et al., 2024) extends LatentCF++ by introducing flexible temporal constraints—example-specific, global, or uniform—and supports optimization in both input and latent spaces. The method is model-agnostic and leverages constraint-based perturbations to balance validity, proximity, and compactness.

Li et al. (2023) introduced CELS, a method that generates intuitive counterfactuals by leveraging learned saliency maps. CELS provides insightful explanations for the decisions made by time series classifiers and generates counterfactuals. While CELS could generate high proximate and sparse counterfactuals, the validity could not be ensured. More recently, Li et al. (2025) proposed Info-CELS, which eliminated the saliency map normalization to ensure higher validity.

To the best of our knowledge, this work is the first to provide an in-depth empirical analysis and categorization of counterfactual explanation methods specifically designed for univariate time series classification. While Theissler et al. (2022) provides a broad literature review of explainability techniques for time series classification—focusing mainly on attribution and saliency-based methods and introducing a taxonomy based on explanation granularity—their work does not address counterfactual explanations nor include any empirical comparisons. More recently, Kan et al. (2024) benchmarked six counterfactual methods for time series data; however, their evaluation is limited to methods supported by the DiCE framework and PyTorch-based classifiers, and focuses primarily on heuristic and optimization-based methods, restricting both coverage and generalizability.

In contrast, our work significantly extends the existing literature by:

We evaluate nine diverse counterfactual explanation methods, encompassing a total of 16 models, and covering a wide range of methodological paradigms, including optimization-based, shapelet-based, instance-based, and saliency-guided approaches.
Benchmarking these methods across 20 UCR datasets using a unified and reproducible experimental setup;
Assessing explanation quality across four key properties—validity, proximity, sparsity, and plausibility;
Performing trade-off analyses to identify how methods balance competing explanation objectives, offering nuanced insights into each method’s strengths and limitations.
Conducting a case study on two well-known UCR datasets—GunPoint and Coffee, to demonstrate how dataset characteristics, such as intra-class variability and pattern stability, influence the quality and interpretability of counterfactual explanations.

By synthesizing methodological innovations and providing actionable empirical insights, our work offers a more targeted, comprehensive, and in-depth evaluation of counterfactual explanations for time series models. It not only complements prior reviews but also serves as a practical reference for researchers and practitioners seeking interpretable AI solutions in sequential domains.

3 Counterfactual Explanations for Time Series Classification

3.1 Problem Definition

In this work, we address the problem of generating counterfactual explanations specifically for time series data. Assume a set of N time series $\mathcal {D}=\left\{ \textbf{x}_1, \textbf{x}_2, \ldots , \textbf{x}_N\right\} $, where each $\textbf{x}_i \in \mathbb {R}^{D \times T}$, with D being the number of dimensions (e.g., $D=1$ for univariate time series) and T the length of the time series. Given a black-box time series classification model $f: \mathbb {R}^{D \times T} \rightarrow \mathcal {Y}$, where $\mathcal {Y}$ is a set of C class labels, the goal is to find a counterfactual explanation for an instance $\textbf{x} \in \mathcal {D}$.

Let $\hat{\textbf{y}} = f(\textbf{x})$ be the predicted probability distribution over the C classes. The top predicted class $z = \operatorname {argmax} \hat{\textbf{y}}$ is the class of interest for which we seek a counterfactual explanation. The goal is to generate a counterfactual $\mathbf {x'} = \textbf{x} + \Delta $ such that the perturbed instance $\mathbf {x'}$ is classified as a different class $z' \ne z$. In binary classification, $z'$ is the class different from z, while in multi-class settings, $z'$ is often defined as the class with the second-highest predicted probability.

3.2 Properties of Counterfactuals

The most critical properties commonly emphasized in counterfactual explanations across the literature are validity, proximity, sparsity, plausibility, and efficiency. The key challenge lies in balancing these criteria when introducing the perturbation $\Delta $. We detail each of these five properties separately below to facilitate a deeper understanding.

Validity. As we described in Sect. 3, a counterfactual explanation is a perturbation $\Delta $ of the original instance $\textbf{x}$ to generate a different output $z'$ by the black-box model f. Therefore, validity is the most important property that ensures the generated counterfactual instance $\mathbf {x'}$ satisfies $f(\mathbf {x'}) = z'$. Common evaluation criteria used in the literature are the “flip label rate" (Mothilal et al., 2020; Li et al., 2022b), and “target probability" (Li et al., 2022a).
Proximity. Proximity aims to keep the counterfactual instance close to the original instance. Common measures for proximity include the L1 distance (Manhattan distance), L2 distance (Euclidean distance), and $L_{\infty }$ distance. Some studies utilize a combination of these measures (Delaney et al., 2021; Bahri et al., 2022a), while others focus solely on the L1 distance (Li et al., 2022a, 2023) to evaluate proximity.
Sparsity. Sparsity seeks to minimize the number of data point perturbations required to generate counterfactuals. It is typically measured either by calculating the percentage of unchanged data points after perturbation (Boubrahimi et al., 2022; Li et al., 2022a) (where a higher value is preferred) or by counting the number of altered data points (where a lower value is better) (Bahri et al., 2022a, 2022b).
Plausibility. Plausibility requires the counterfactuals to be realistic. In other words, they must lie within the data manifold of the training data. In essence, plausible counterfactuals should resemble the natural structure of the data, ensuring that they do not appear as anomalies or outliers. Different outlier detection methods, such as the local outlier factor (Breunig et al., 2000), isolation forests (Liu et al., 2008), and one-class-support vector machine (Schölkopf et al., 2001) have been used as crucial metrics in the literature (Delaney et al., 2021; Boubrahimi et al., 2022).

4 Categorization of Counterfactual Explainers

Counterfactual explanations, as discussed in the literature (Guidotti, 2024), can be broadly classified into two main strategies based on their approach to deriving explanations:

1.
Optimization (OPT): This category includes counterfactual explainers that use optimization strategies. They are characterized by defining a loss function that encapsulates the desired properties of counterfactual explanations. These explainers then utilize existing optimization algorithms to minimize this loss function. This approach ensures that the explanations adhere to specific quantitative criteria, optimizing them for the defined loss.
2.
Heuristic Search Strategy (HSS): Counterfactual explainers in this category rely on heuristic search strategies to find suitable explanations. These strategies do not follow a rigid algorithmic path but use heuristic and local search techniques to progressively minimize a cost function. Each iteration aims to refine the counterfactuals incrementally, guided by heuristic choices that are often tailored to exploit the problem’s specific structure or peculiarities.

While this high-level distinction aligns with the classification proposed by Guidotti (2024) for general counterfactual methods, their taxonomy is designed primarily for tabular data and does not account for the structural properties of time series or the increasing methodological diversity of counterfactual explanations in that domain. Similarly, Theissler et al. (2022) provides a taxonomy of explainable AI methods for time series, but their work focuses mostly on saliency-based attribution techniques and does not offer a structured breakdown of counterfactual explanation approaches.

To address this gap, we propose a refined taxonomy tailored specifically to counterfactual explanations for time series classification. Our categorization goes beyond optimization versus heuristic distinctions by organizing methods based on the underlying mechanism used to generate or guide perturbations. In particular, we extend existing taxonomies by explicitly introducing method families such as instance-based, shapelet-based, and saliency-guided counterfactuals, which have not been formally distinguished in prior work. This more structured categorization provides a clearer view of how recent time series counterfactual methods are designed and implemented, particularly as many incorporate components from multiple methodological paradigms.

To reflect this diversity, we refine the taxonomy into five subcategories based on the guiding principles of the methods:

Instance-based Methods: These methods generate counterfactuals by leveraging existing instances from the target class as references. A common approach involves identifying a “nearest unlike neighbor"—a real example with a different predicted label—and using it to guide the generation of counterfactuals.
Heuristic Methods: Heuristic-based approaches generate counterfactuals through predefined rules, greedy strategies, or locally guided modifications rather than solving a formal optimization problem. These methods often use attribution signals or data-driven heuristics to alter input features to induce a label change iteratively.
Shapelet-based Methods: These approaches focus on identifying and modifying discriminative subsequences (shapelets) within the time series. By perturbing or replacing key shapelets that contribute most to the classification decision, these methods aim to induce label changes efficiently. Shapelet-based methods balance interpretability and sparsity, but their success often hinges on the quality of shapelet extraction.
Optimization-based Methods: These methods treat counterfactual generation as an optimization problem, where an objective function is designed to capture specific goals such as inducing a label change (validity), preserving similarity to the original instance (proximity), minimizing the number of changed features (sparsity), or ensuring realism (plausibility). Depending on the approach, one or more of these properties are incorporated into the loss function, which is then minimized using gradient-based optimization techniques.
Saliency Map-based Methods: These methods exploit saliency or attribution maps to identify which regions of the input time series contribute most to the model’s decision. Counterfactuals are then generated by perturbing only the most influential regions, making the changes both effective and interpretable.

It is important to note that the taxonomy presented in Fig. 1 is not mutually exclusive: some counterfactual methods belong to multiple categories because they integrate different generation mechanisms. For example, Info-CELS and CELS generate counterfactuals by optimizing a multi-objective function while using saliency maps to restrict perturbations to relevant time steps. This places them in both the optimization-based and saliency-guided categories. MG constructs counterfactuals by inserting class-relevant shapelets, guided by a rule-based heuristic rather than a loss-driven search. SG combines gradient-based optimization with shapelet-informed perturbation, incorporating both learned objectives and shape-matching constraints. Rather than forcing a strict separation, our taxonomy reflects the fact that many recent methods blend optimization techniques with guidance from interpretable signals such as saliency maps or shapelets. This overlap more accurately represents how counterfactual explanations for time series are formulated in practice. In the following subsections, we discuss each category shown in Fig. 1 and describe the methods grouped under them. For example, Info-CELS and CELS are covered under saliency map-based methods, SG and MG under shapelet-based methods, Wachter, TimeX, and Glacier under optimization-based methods, and NG-DBA and NG-CAM under instance-based methods.

4.1 Instance-Based Counterfactual Explainer

The instance-based model would generate counterfactuals from the in-sample instance that guide the final counterfactual generation. There are two models that use native guides that incorporate the nearest unlike neighbor to guide the perturbation.

Native Guide (NG) (Delaney et al., 2021) is the first counterfactual explanation method tailored specifically for time series classifiers. NG employs an instance-based approach that adapts existing counterfactual instances by focusing on and modifying the discriminative areas of the time series that influence the classification decision. As shown in Fig. 2, the core idea behind NG is to retrieve existing counterfactual instances from the case base for any query and use them to guide the counterfactual generation process. By integrating feature attribution information or deep features into the counterfactual generation, NG produces sparse counterfactuals that highlight the most relevant and meaningful areas of the time series. In Delaney et al. (2021), the nearest unlike neighbor (NUN) is used as the ‘native counterfactual’, ensuring the plausibility of the explanation, as it inherently falls within the distribution of valid instances.

1.
NG-CAM. When applying NG to deep learning classifiers, where the model functions as a black box, feature-weight vectors w can be extracted using the Class Activation Mapping (CAM). These feature weights help NG modify only specific contiguous subsequences of the time series, rather than altering the entire sequence. The process is formalized as: Given the native counterfactual $\mathbf {x_{NUN}^{'}}$ and the feature-weight vector w, the most influential contiguous subsequence (as determined by the magnitude of the weights in w) is identified. The corresponding region in $\textbf{x}$ is then replaced with the values from the native counterfactual.
$$\begin{aligned} \textbf{x}&= <x_1, x_2, x_3, x_4, x_5, ..., x_d> \ s.t. \ f(\textbf{x}) = z \end{aligned}$$
(1)
$$\begin{aligned} \mathbf {x'}&= <x_1, {x^{\prime }_2}, {x^{\prime }_3}, {x^{\prime }_4}, x_5, x_6, ..., x_d> \ s.t. \ f(\mathbf {x'}) = z' \end{aligned}$$
(2)
This process begins with a small subsequence and iteratively increases the subsequence length until $f(\mathbf {x'}) = z'$. In the worst-case scenario, the subsequence length could span the entire series, reverting to the original native counterfactual $\mathbf {x_{NUN}^{'}}$. This adaptive method improves both the proximity and plausibility of the generated counterfactuals. Moreover, diversity can be introduced by using other in-sample instances, such as the next nearest unlike neighbor, to generate alternative counterfactuals for the query, enriching the explanatory power. However, this method is limited to cases where the feature-weight vector is available, such as in deep learning models like CNNs or RNNs. For traditional machine learning classifiers, like k-nearest neighbors or decision trees, NG cannot use CAM to guide counterfactuals generation with high proximity and plausibility. Additionally, the reliance on feature weights can introduce bias, as the counterfactuals depend on how accurately these weights capture the true discriminative regions.
2.
NG-DBA. In cases where a traditional machine learning classifier (such as k-nearest neighbor) is the black box for which we seek a counterfactual explanation, a feature-weight vector w is not accessible. To address this, Dynamic Time Warping Barycenter Averaging (DBA) is applied between the query instance $\textbf{x}$ and the native counterfactual $\mathbf {x_{NUN}^{'}}$ to generate the counterfactual explanation. However, this approach tends to result in low sparsity, as it requires altering the entire time series to obtain the final counterfactual explanation, rather than targeting specific, discriminative subsequences.

4.2 Shapelet-Based Counterfactual Explainer

Shapelets are time series subsequences that are highly representative of a class (Ye & Keogh, 2009), offering interpretable insights valuable to domain experts.

1.
MG (Li et al., 2022b): The first step is to extract shapelets from the training dataset using the Shapelet Transform (ST) algorithm proposed by Lines et al. (2012). Once the shapelets are extracted, counterfactual explanations are generated by first predicting the query instance $\textbf{x}$ using a pre-trained model f. The target class for the counterfactual is set as the opposite label of the prediction. To generate the explanation, the most prominent shapelet from the target class is identified, and its corresponding segment is used to replace the matching segment in the query instance, while the rest of the series remains unchanged. This process creates the counterfactual explanation for the input time series. Notably, MG-CF does not rely on class activation maps, making it compatible with both traditional machine learning and deep learning classifiers.
2.
SG (Li et al., 2022a): SG extends MG by introducing an optimization process to refine the counterfactual generation. A shapelet-guided loss term, $L_{shapelet}$, is added to guide perturbations in $\textbf{x}$ to produce a counterfactual $\mathbf {x'}$ that aligns with the distribution of the target class. Before generating the counterfactual, the most prominent shapelets for each class are mined and integrated into the objective function to direct perturbations toward an interpretable explanation. The loss term $L_{shapelet}$ ensures that the counterfactual instance $\mathbf {x'}$ is close to the prominent shapelet $\mathbf {x_{shapelet}}$ from the target class $z'$. The shapelet’s prominence is measured by its discriminative power, and the Shapelet Transform algorithm is again used to mine these shapelets. The shapelet-guided loss is defined as:
$$\begin{aligned} L_{shapelet} = d(\mathbf {x'}, \mathbf {x_{shapelet}}) \end{aligned}$$
(3)
The total loss combines this with the prediction and L1 loss terms:
$$\begin{aligned} L = L_{pred} + L_{L1} + L_{shapelet} \end{aligned}$$
(4)

A significant drawback of shapelet-guided counterfactuals is the computational burden of mining shapelets, which involves evaluating all possible subsequence lengths to identify the most discriminative ones. This process is time-consuming, so to expedite the search, a fixed subsequence length is often set as a parameter. However, this can lead to suboptimal shapelet extraction, where the most discriminative shapelets are not identified, resulting in counterfactuals that may lack validity. Balancing efficiency and accuracy remains a key challenge in shapelet-based counterfactual methods.

4.3 Optimization Strategies

1.
Wachter. Wachter et al. (2017) are among the pioneers in proposing counterfactual explanations. The objective function introduced by Wachter aims to minimize the perturbation of data points while encouraging a change in the label of the perturbed instance. This function is expressed as follows:
$$\begin{aligned} \arg \min _{\mathbf {x'}} \max _{\lambda } \lambda \underbrace{\left( f(\mathbf {x'})-z'\right) ^{2}}_{\mathcal {L}_{\text {pred}}} + \underbrace{d\left( \textbf{x}, \mathbf {x'}\right) }_{\mathcal {L}_{\text {L1}}}, \end{aligned}$$
(5)
Where $z'$ represents the target class that differs from the original class z associated with the query time series x, the first term, $\mathcal {L}_{\text {pred}}$, refers to the prediction loss, which is measured by the squared difference between the model’s predicted probability for the perturbed instance, $f(\mathbf {x'})$, and the desired class $z'$. The second term, $\mathcal {L}_{\text {L1}}$, captures the perturbation by measuring the distance between the original instance $\textbf{x}$ and the counterfactual instance $\mathbf {x'}$. In theory, any distance metric d, such as the Euclidean distance, could be used. Wachter et al. recommend using the L1 norm, or Manhattan distance, weighted by the inverse median absolute deviation. The parameter $\lambda $ controls the trade-off between the two terms. A low $\lambda $ prioritizes keeping $\mathbf {x'}$ similar to $\textbf{x}$, while a high $\lambda $ focuses on achieving predictions closer to the desired outcome $z'$. In Wachter et al. (2017), it is recommended to maximize $\lambda $ while minimizing the loss. As a result, $\lambda $ becomes a parameter of the search process, with a tolerance $\epsilon $ applied to constrain the classification of $\mathbf {x'}$, ensuring that $\Vert f(\mathbf {x'}) - z'\Vert \le \epsilon $. Therefore, the loss measures both the deviation of the counterfactual outcome $f(\mathbf {x'})$ from the desired outcome $z'$ and the distance between the counterfactual $\mathbf {x'}$ and the original instance $\textbf{x}$. The loss function can be minimized using any appropriate optimization algorithm, such as the Nelder-Mead (Simplex) method (Powell, 1973). If the black-box model is differentiable and gradients are accessible, gradient-based methods like Adam (Kingma & Ba, 2014) can be employed. In summary, the Wachter method operates as follows: given an instance x to be explained, a black-box model f, and the desired outcome $z'$, Wachter randomly initializes $\mathbf {x'}$ and $\lambda $. It then optimizes the loss by updating the values of $\mathbf {x'}$. If the constraint $|f(\mathbf {x'}) - z'| < \epsilon $ is not satisfied, $\lambda $ is increased and another optimization step is performed until the constraint $|f(\mathbf {x'}) - z'| < \epsilon $ is met. A limitation of Wachter is that determining the initial value of $\lambda $ and the value of $\epsilon $ a priori is unclear. Additionally, Wachter is an exogenous explainer specifically designed to explain differentiable classifiers acting on tabular data, and it returns a single counterfactual. Wachter can be adapted into the time series dataset, and usually serves as a baseline in counterfactual explanation in time series classification tasks (Delaney et al., 2021; Li et al., 2022a; Höllig et al., 2022).
2.
TimeX. (Boubrahimi et al., 2022) This work builds upon Wachter’s work to enhance interpretability and contiguity by introducing a new loss term that incorporates the knowledge of the counterfactual’s class of interest. To improve interpretability, a new loss term is integrated into the existing objective function, focusing on the proximity to the average time series instance within the class of interest. The Dynamic Barycenter Averaging (DBA) method is employed to approximate the centroid of the desired class, maximizing its representativeness. The DBA algorithm utilizes dynamic time warping (DTW) to establish optimal alignment between sequences, employing a warping path across a warping matrix D that minimizes the final warping cost (Keogh & Ratanamahatana, 2005). The canonical DTW formulation is given in Eq. 6, where M and N indicate the lengths of the time series x and y, respectively. The D matrix is initialized with $D_{0,0} = 0$ and $D_{i,j} = \infty $. The cost function g typically represents the squared differences between $x_i$ and $y_j$.
$$\begin{aligned} D_{i, j}=g\left( x_{i}, y_{j}\right) +\min \left\{ D_{i, j-1}, D_{i-1, j}, D_{i-1, j-1}\right\} \quad \text {s.t }: i \in (1, M), j \in (1, N) \end{aligned}$$
(6)
The DBA algorithm consists of two phases. The first phase calculates the DTW distance between each time series sample and a temporary average sequence $\mathcal {C}$, aligning the coordinates of the average sequence with those in the time series set $\mathcal {D}$. The second phase updates each coordinate of the average sequence to reflect the barycenter of associated coordinates determined in the first phase. The DBA loss is defined in Eq. 7, while the new proposed loss is presented in Eq. 8.
$$\begin{aligned} \mathcal {L}_{dba}&= \theta \Vert \tilde{TS} - dba_{\tilde{y}}\Vert \end{aligned}$$
(7)
$$\begin{aligned} \mathcal {L}&= \mathcal {L}_{pred} + \mathcal {L}_{L1} + \mathcal {L}_{dba} \end{aligned}$$
(8)
To tackle the contiguity violation identified in Wachter’s original loss formulation, it is proposed to update only the gradients of the most significant contiguous segment within the time series TS. The importance of contiguous segments in the original time series is assessed using saliency maps of the TS feature vector. If saliency maps are unavailable, they are generated through continuous perturbation and evaluation, as described in the subsequent subsection.
3.
Glacier (Wang et al., 2024). Glacier provides a unified framework for counterfactual explanation in time–series classification by performing gradient-based optimization either (i) in the latent representation space of an auxiliary autoencoder or (ii) directly in the original feature space, under one of four temporal-segment constraints—local, global, unconstrained, or uniform. LatentCF++ (Wang et al., 2021) corresponds to Glacier’s unconstrained latent-space variant; we therefore treat LatentCF++ as a special case of Glacier in this paper. For the latent?space counterfactual search process, Glacier first maps the input time series $\textbf{x}$ to its latent representation $z=\texttt {Encode}(\textbf{x})$ using a pre-trained encoder. Decoding this vector yields an initial reconstruction $\textbf{x}^{*}=\texttt {Decode}(z)$, whose class score is obtained from the classifier as $y^{*}=f(\textbf{x}^{*})$. Then $\textbf{x}^{*}$ is optimized by an objective function that balances validity against sparsity. The prediction-margin loss
$$\begin{aligned} \mathcal {L}_p\!\left( y^{*},\tau \right) =\bigl \Vert y^{*}-\tau \bigr \Vert \end{aligned}$$
(9)
encourages the prediction score $y^{*}$ to cross the decision boundary $\tau $, whereas the constraint loss
$$\begin{aligned} \mathcal {L}_c\!\left( \textbf{x}',\textbf{x},\varvec{c}\right) =\left| \varvec{c}^{\top }\!\left( \textbf{x}'-\textbf{x}\right) \right| \end{aligned}$$
(10)
penalizes perturbation outside the time segments specified by a constraint vector $\varvec{c}$. Combining the two terms gives the total loss
$$\begin{aligned} \mathcal {L}\!\left( \textbf{x}',y^{*},\textbf{x},w,\tau ,\varvec{c}\right) = w\,\mathcal {L}_{p}\!\left( y^{*},\tau \right) +(1-w)\,\mathcal {L}_{c}\!\left( \textbf{x}',\textbf{x},\varvec{c}\right) , \end{aligned}$$
(11)
where the weight $w\!\in \![0,1]$ controls the trade-off between the two objectives. During optimization, Adam updates the latent vector z with respect to $\nabla _{z}\mathcal {L}$; the updated z is decoded to a new $\textbf{x}^{*}$, and $y^{*}$ as well as $\mathcal {L}$ are recomputed. This loop continues until one of three stopping conditions is met: (i) the loss falls below the tolerance tol, (ii) the prediction crosses the boundary ($y^{*}\ge \tau $), or (iii) a maximum number of iterations is reached. The last reconstruction produced at termination, denoted $\textbf{x}'$, is returned as the counterfactual explanation. For the feature?space counterfactual search variant, no auto-encoder is used and the optimization is performed directly on the input itself. The process starts by setting the candidate counterfactual to the original sample, $\textbf{x}^{*}=\textbf{x}$, and obtaining its class score $y^{*}=f(\textbf{x}^{*})$. The same prediction-margin loss $\mathcal {L}_{p}$ (Eq. 9), constraint loss $\mathcal {L}_{c}$ (Eq. 10), and total loss $\mathcal {L}$ (Eq. 11) are then evaluated, with the constraint vector $\varvec{c}$ and weight w playing the same roles as before. At each optimization step, Adam updates the input sample itself,
$$ \textbf{x}^{*}\leftarrow \textbf{x}^{*}-\alpha \,\nabla _{\textbf{x}^{*}}\mathcal {L}, $$
where $\alpha $ is the learning rate. After every update, the classifier recomputes $y^{*}$ and the losses are updated. The procedure terminates under the same stopping criteria as in the latent-space case—namely, when the loss drops below the tolerance tol, the prediction crosses the boundary ($y^{*}\ge \tau $), or the maximum number of iterations is reached. The final value of $\textbf{x}^{*}$ at termination, denoted $\textbf{x}'$, is taken as the resulting counterfactual explanation. Glacier offers four variants of the constraint vector $\varvec{c}$:
- Example-specific constraints are derived for each test case with LIMESegment (Sivill & Flach, 2022). They highlight the most influential segments of the current series, so edits are encouraged mainly inside those regions, often leading to compact, instance-tailored counterfactuals.
- Global constraints come from interval-level permutation importance (Breiman, 2001) computed on the training data. All examples are steered toward the same class-discriminative intervals, yielding counterfactuals that are consistent across the dataset.
- Unconstrained sets $\varvec{c}=[0, 0,..., 0]$, where all time steps are favoured to change.
- Uniform sets $\varvec{c}=[1, 1,..., 1]$, penalizing the change of all the time series equally,
Combining the two search spaces—autoencoder (AE) and no-autoencoder (NoAE)—with the four constraint types gives eight Glacier variants: Glacier-AE-local, Glacier-AE-global, Glacier-AE-uniform, Glacier-AE-unconstrained, Glacier-NoAE-local, Glacier-NoAE-global, Glacier-NoAE-uniform, and Glacier-NoAE-unconstrained.

4.4 Saliency Map Guided Counterfactual Explainer

1.
CELS (Li et al., 2023). CELS learns a saliency map for the interests of an instance and generates a counterfactual explanation guided by the learned saliency map. CELS generates counterfactual explanations through an optimization-driven process that selectively modifies the most influential parts of a time series instance to flip its predicted label. It starts by initializing a one-dimensional saliency map, where each value corresponds to the importance of a specific time step. These values are randomly sampled between 0 and 1, allowing the method to softly indicate which regions of the input should be considered for modification. To guide the perturbation, CELS then identifies a nearest unlike neighbor ($\textbf{nun}$)—an instance from the training set that is close to the input but belongs to a different class. The target label of this neighbor becomes the desired outcome for the counterfactual. Using this guidance, the method enters an iterative optimization loop with a perturbation function (Eq. 12) involved. At each step, the saliency map determines which time steps to perturb, producing a candidate counterfactual instance. An objective function (Eq. 16) evaluates this candidate by balancing three goals: achieving a successful label flip (Eq. 13), minimizing the number of changes (sparsity) (Eq. 14), and preserving temporal coherence (smoothness) (Eq. 15). The optimization updates the saliency map using gradient descent to better satisfy these goals over time.
$$\begin{aligned} \textbf{x}'= & \textbf{x} \odot \left( 1-\varvec{\theta }\right) + \textbf{nun} \odot \varvec{\theta } \end{aligned}$$
(12)
$$\begin{aligned} L_{\operatorname {Max}}= & \left( 1-P\left( \hat{\textbf{y}}_{z'} \mid \textbf{x}'\right) \right) \end{aligned}$$
(13)
$$\begin{aligned} L_{\textrm{Budget}}= & \frac{1}{T} \sum _{t=1}^T\varvec{\theta }_{ t} \end{aligned}$$
(14)
$$\begin{aligned} L_{\textrm{TReg}}= & \frac{1}{T} \sum _{t=1}^{T-1}\left( \varvec{\theta }_{t}-\varvec{\theta }_{t+1}\right) ^2 \end{aligned}$$
(15)
$$\begin{aligned} L(P(\hat{\textbf{y}} \mid \textbf{x}); \varvec{\theta })= & \lambda *L_{\textrm{Max}}+L_{\textrm{Budget}}+L_{\textrm{TReg}} \end{aligned}$$
(16)
Once the optimization converges, the continuous saliency map is thresholded to isolate the most impactful time steps. A final counterfactual explanation is then constructed using these selected time steps, effectively highlighting the key regions that caused the classification to change. CELS ultimately returns both the saliency map and the counterfactual explanation, offering an interpretable view of how minimal, targeted changes can influence the model’s prediction.
2.
Info-CELS (Li et al., 2025). Info-CELS builds upon the previous CELS model to address the limitations in validity for counterfactual explanations in time series classification. The original CELS model introduced the use of saliency maps to generate counterfactual explanations by identifying important time steps for perturbation. While CELS showed strong performance in sparsity and proximity, it faced challenges in ensuring the validity of the generated counterfactuals due to its use of saliency map thresholding, which introduced noise and reduced the reliability of the explanations. Info-CELS eliminates the saliency map thresholding step in CELS, leading to more informative and valid counterfactuals. By directly using the learned saliency map for perturbations without normalization, Info-CELS achieves smoother transitions in the generated counterfactuals, reducing abrupt changes and enhancing the clarity of the explanations. This approach improved the validity significantly while maintaining descent sparsity and proximity compared to CELS.

5 Experiments

In this section, we benchmark a set of counterfactual explanation methods, previously detailed in Sect. 4.

5.1 Datasets

Our benchmark consists of twenty real-world datasets from the UCR Time Series Classification Archive (Dau et al., 2019). Although the UCR archive contains 128 datasets, existing literature on counterfactual explanation methods typically evaluates performance on smaller subsets of these datasets rather than the entire archive. This is primarily because generating and evaluating counterfactual explanations involves more complex computational and methodological considerations compared to standard time series classification tasks. Therefore, our dataset selection was determined by taking the union of all datasets previously used in the original papers introducing the counterfactual-explanation methods evaluated in this study, except for Glacier. Specifically, all other methods had been previously evaluated on both binary-class and multi-class datasets, whereas Glacier’s original paper included only binary-class evaluations. To ensure that our dataset selection encompasses both binary and multi-class tasks and avoids bias toward any particular method, we did not include Glacier’s datasets when forming the union. However, for completeness and fair comparison in our experiments, we adapted Glacier’s optimization method to handle multi-class datasets.

As detailed in Table 1, the final dataset collection includes 9 binary-class datasets and 11 multi-class datasets, with the number of classes ranging from 3 to 15. These datasets collectively span all nine domain types present in the UCR archive—including image, spectro, sensor, simulated, traffic, device, motion, ECG, and HAR (Human Activity Recognition)—and cover series lengths ranging from 24 to 900 samples. Following common practice, we strictly adhere to the archive’s provided train/test splits: classifiers are trained solely on the training sets, and the corresponding test sets are reserved exclusively for generating and evaluating counterfactual explanations. Notably, our evaluation considers counterfactuals generated for every instance in each test set, without randomly sampling a subset of instances, thereby eliminating potential sampling bias in the experimental results.

Table 1 Datasets details

Full size table

5.2 Black-Box Classification Models

For each dataset, we applied a single black-box classifier per counterfactual explanation method, with the choice determined by the design and optimization requirements of the explanation model. For methods that rely on gradient-based optimization—where backpropagation through the classifier is required to optimize counterfactuals—such as Wachter, SG, TimeX, CELS, Info-CELS, and Glacier, we used the Fully Convolutional Network (FCN). FCN provides a fully differentiable prediction function, which is essential for these methods to perform gradient-based updates. It was the first model used in the literature on time series counterfactual explanations (Delaney et al., 2021), and has since been widely adopted in many subsequent works. FCN also demonstrates strong, competitive performance across many UCR datasets (Ismail Fawaz et al., 2019). The architecture follows Wang et al. (2017) and the implementation of Ismail Fawaz et al. (2019), consisting of three convolutional layers with batch normalization and ReLU activation, followed by global average pooling and a softmax output. For counterfactual methods that do not require access to gradients—specifically MG and NG-DBA—we used the ROCKET framework (Dempster et al., 2020) combined with a Random Forest (RF) classifier. We used the ROCKET+RF (RR) classifier for these methods to maintain alignment with their intended setup while enabling counterfactual generation in multiclass settings, where the target class is determined as the second most probable class in the model’s prediction. Classification accuracies for both setups are reported in Table 1.

5.3 Evaluation Metrics

To rigorously evaluate our proposed method, we employ a comprehensive set of metrics to assess various aspects of performance and compare them with established baselines:

5.3.1 Validity Metric

Flip Label Rate: It quantifies the proportion of instances where the label flips from the original class to the target class after perturbation. We computed the flip rate following the formula of Eq. (17).
$$\begin{aligned} flip\_rate = \frac{num\_flipped}{num\_testsample}, \end{aligned}$$
(17)
where we denote the num_flipped as the number of generated counterfactual explanation samples that result in a different prediction label compared to the original input, and the num_test_samples is the total number of samples in the test dataset used to generate these counterfactual explanations. According to the validity property as we mentioned in Sect. 3.2, a counterfactual instance should result in a prediction that is different from the prediction of the original instance. Therefore, the flip rate directly aligns with this validity property by quantifying how often the generated counterfactual instances meet this criterion. A high flip rate 1 indicates that all the generated counterfactual instances are valid.

5.3.2 Proximity Metric

We evaluate proximity using standard distance-based metrics—L1, L2, and $L_{\infty }$ Distance to quantify the magnitude of change between the original input and the generated counterfactual. These metrics are widely adopted in counterfactual explanation literature due to their interpretability and their direct correspondence with the principle of minimal change.

While time-series-specific measures like Dynamic Time Warping (DTW) can account for temporal misalignments, they are less appropriate in this context. DTW may return low distances despite widespread temporal perturbations due to its warping flexibility, which contradicts the goal of evaluating how minimally the input was changed. Moreover, counterfactuals are intended to adopt target-class characteristics and do not aim to preserve the original shape. For these reasons, we opt for L1, L2, and $L_{\infty }$ as consistent and conceptually aligned proximity metrics in our benchmark.

L1 Distance: Also known as the Manhattan distance, this metric captures the sum of absolute differences between the original and counterfactual instances across all time steps. It provides insight into the magnitude of the alterations made to the instance by measuring the total change in each time step.
$$\begin{aligned} L1\ distance = \sum _{t=1}^T |\textbf{x}_{t} - \textbf{x}_{t}'|, \end{aligned}$$
(18)
L2 Distance: The L2 distance, also known as the Euclidean distance, measures the overall change between the original and counterfactual instances. Unlike the L1 distance, which sums the absolute differences, L2 distance sums the squared differences and takes the square root, emphasizing larger deviations more strongly due to the squaring operation.
$$\begin{aligned} L2\ distance = \sqrt{\sum _{t=1}^T (\textbf{x}_{t} - \textbf{x}_{t}')^2}, \end{aligned}$$
(19)
$L_{\infty }$ Distance: Also known as the Chebyshev distance, $L_{\infty }$ distance measures the maximum absolute difference between the original and counterfactual instances across all time steps. Unlike L1 and L2, which aggregate differences, $L_{\infty }$ focuses on the largest deviation, providing insight into the most significant change made to any individual time step.
$$\begin{aligned} L_{\infty }\ distance = \max _{t=1}^T |\textbf{x}_{t} - \textbf{x}_{t}'|, \end{aligned}$$
(20)

where all of the T in the above Eqs. (18), (19), and (20) represent the total number of time steps (length) of the time series data.

5.3.3 Sparsity Metric

Unchanged Time Steps: The sparsity level is evaluated by calculating the percentage of time steps that remain unchanged after perturbation. This metric helps in understanding the extent of modification applied to achieve the counterfactual, with a higher percentage indicating greater sparsity. The highest sparsity is an indicator that the number of time series data point perturbations made in $\textbf{x}$ to achieve $\textbf{x}'$ is minimal. Therefore, a higher sparsity level is desirable. The equations designed by Boubrahimi et al. (2022) are shown in Eqs. (21) and (22), where T represents the total number of time steps (length) of the time series data.
$$\begin{aligned} Sparsity= & 1- \frac{\sum _{t=1}^{T}g({\textbf{x}'_t}, \textbf{x}_t)}{T} \end{aligned}$$
(21)
$$\begin{aligned} g(x, y)= & \left\{ \begin{array}{ll} 1, & \text{ if } x \ne y \\ 0, & \text{ otherwise } \end{array}\right. \end{aligned}$$
(22)

5.3.4 Plausibility Metric

Isolation Forest (IF) (Liu et al., 2008). IF is an anomaly detection algorithm that identifies anomalies by constructing random binary trees—called Isolation Trees (iTrees)—that recursively partition the data. The core assumption is that anomalies are typically few and different, making them easier to isolate and more likely to appear closer to the root of the tree. As a result, they exhibit shorter average path lengths compared to normal points. To quantify this, an anomaly score s(x, n) for a data point x is computed based on the average path length E(h(x)) from the root node to a terminating node across multiple iTrees. This score is normalized using the expected average path length c(n) of a binary search tree of size n:
$$\begin{aligned} s(x, n) = 2^{- \frac{E(h(x))}{c(n)}}, \end{aligned}$$
(23)
where E(h(x)) is the average path length of x across all trees in the ensemble, and c(n) is the average path length of an unsuccessful search in a binary search tree with n instances. The anomaly score s(x, n) ranges between 0 and 1, where higher values indicate a greater likelihood of being anomalous.
Local Outlier Factor (LOF) (Breunig et al., 2000). The LOF is a density-based method that identifies anomalies by comparing the local density of a data point to that of its neighbors. Anomalous instances typically reside in regions of significantly lower density relative to their surroundings. To compute the LOF score for a point x, the first step is to define the reachability distance from x to a neighbor y:
$$\begin{aligned} \text {reach}\_\text {dist}_k(x, y) = \max \left\{ \text {k-distance}(y),\ d(x, y) \right\} , \end{aligned}$$
(24)
which ensures that very close neighbors do not overly influence the local density estimate. Next, the local reachability density (LRD) of point x, which is the inverse of the average reachability distance to its k-nearest neighbors, is computed as:
$$\begin{aligned} \text {lrd}_k(x) = \left( \frac{1}{|\mathcal {N}_k(x)|} \sum _{y \in \mathcal {N}_k(x)} \text {reach}\_\text {dist}_k(x, y) \right) ^{-1}, \end{aligned}$$
(25)
where $\mathcal {N}_k(x)$ denotes the set of k-nearest neighbors of x. A smaller average reachability distance implies a denser local neighborhood. Finally, the Local Outlier Factor score is obtained by comparing the local density of x to those of its neighbors:
$$\begin{aligned} \text {LOF}_k(x) = \frac{1}{|\mathcal {N}_k(x)|} \sum _{y \in \mathcal {N}_k(x)} \frac{\text {lrd}_k(y)}{\text {lrd}_k(x)}. \end{aligned}$$
(26)
A LOF score close to 1 indicates that x has a similar density to its neighbors (i.e., it is likely normal), while a score significantly greater than 1 suggests that x is located in a sparser region and is likely to be an anomaly.
One-Class Support Vector Machine (OC-SVM) (Schölkopf et al., 2001). The OC-SVM is an unsupervised anomaly detection algorithm that learns a decision boundary to separate normal data points from potential outliers. Unlike traditional binary SVMs, OC-SVM is trained using only examples from a single (normal) class. Its objective is to find a function that captures the regions in input space where most of the data lie. OC-SVM maps the input data into a high-dimensional feature space via a kernel function and seeks a hyperplane that maximally separates the origin from the mapped data points, with the goal of enclosing the normal data within a bounded region. The optimization problem can be formulated as:
$$\begin{aligned} \min _{w, \rho , \xi } \frac{1}{2} \Vert w\Vert ^2 + \frac{1}{\nu n} \sum _{i=1}^n \xi _i - \rho \end{aligned}$$
(27)
subject to:
$$\begin{aligned} (w \cdot \phi (x_i)) \ge \rho - \xi _i, \quad \xi _i \ge 0, \quad \forall i = 1, \dots , n, \end{aligned}$$
(28)
where $\phi (\cdot )$ is the feature map induced by the kernel, $\nu \in (0, 1]$ is a parameter controlling the trade-off between the fraction of outliers and the margin, $\xi _i$ are slack variables, and $\rho $ is the offset. At prediction time, a test point x is considered an anomaly if the decision function $f(x) = (w \cdot \phi (x)) - \rho < 0$. This value f(x) serves as an anomaly score. If $f(x) \ge 0$, the point is considered normal. If $f(x) < 0$, the point is classified as an anomaly. The smaller the value of f(x), the more likely the point is to be an outlier.

The outputs of these anomaly detection methods are then used to quantify the plausibility of counterfactuals based on how often they are flagged as out-of-distribution. For each method, we use the trained anomaly detector to evaluate every generated counterfactual. If a counterfactual is flagged as an anomaly by a specific anomaly detection algorithm, it is counted as out-of-distribution. We then calculate the plausibility score by dividing the number of such detected anomalies by the total number of counterfactual examples. This results in a ratio ranging from 0 to 1, representing the proportion of counterfactuals that are flagged as out-of-distribution. In other words, it measures what percentage of the generated counterfactuals fall outside the distribution of the training data—lower values indicate higher plausibility.

5.4 Experimental Results

We began by evaluating the validity, proximity, sparsity, and plausibility of nine counterfactual explanation methods, along with eight variants of Glacier—namely Glacier-AE-local, Glacier-AE-global, Glacier-AE-uniform, Glacier-AE-unconstrained, Glacier-NoAE-local, Glacier-NoAE-global, Glacier-NoAE-uniform, and Glacier-NoAE-unconstrained (see Sect. 4.3). These variants are abbreviated in the following figures as Glacier-AE-Loc, Glacier-AE-Glob, Glacier-AE-Unif, Glacier-AE-Unc, Glacier-NoAE-Loc, Glacier-NoAE-Glob, Glacier-NoAE-Unif, and Glacier-NoAE-Unc, resulting in a total of sixteen models evaluated across twenty univariate time series datasets.

5.4.1 Validity Evaluation

The results in Fig. 3 assess the validity of 16 counterfactual explanation models using the Flip Label Rate metric. A Flip Label Rate of 1.00 indicates that all generated counterfactuals successfully flipped the predicted label to the desired target class, satisfying the core requirement for validity. In the scatter plot, each dot represents the Flip Label Rate for a specific dataset, while the blue star denotes the average performance across all 20 datasets for each method, providing a high-level summary of overall validity.

NG-DBA clearly stands out, achieving perfect validity (flip rate = 1.00) on every dataset. This result stems from its robust design: it selects the second most probable class as the target, retrieves a nearest neighbor from that class, and applies DTW Barycenter Averaging (DBA) to interpolate between the original input and the guide. If the interpolated counterfactual does not flip the prediction, the interpolation weight is gradually increased until the final output becomes the guide itself, ensuring a valid flip. In contrast, other heuristic-based methods such as NG-CAM and MG exhibit greater variability. Their performance depends on the quality of the extracted shapelets or class activation maps, which may lack consistency across datasets, thus limiting their ability to produce valid counterfactuals reliably.

Saliency map–based methods show a clear performance distinction in terms of validity. InfoCELS ranks second overall in average Flip Label Rate, achieving near-perfect validity on most datasets, with only a few falling between 0.8 and 0.9. In contrast, its predecessor CELS shows consistently lower validity across all datasets. This gap highlights the benefit of removing saliency map normalization in InfoCELS, which allows for smoother and more effective perturbations aligned with informative regions.

Other gradient-based optimization methods, including Wachter, TimeX, and SG, also demonstrate strong validity, with Flip Label Rates above 0.8 on most datasets. While not as consistent as NG-DBA or as robust as InfoCELS, their blue stars are positioned in the upper portion of the plot, and most of their scatter points lie near 1.0. This suggests these methods are generally effective at generating valid counterfactuals, though they exhibit slightly more variability across datasets.

The final set of methods includes the eight variants of Glacier, all of which are optimization-based. These variants generate counterfactuals either in the latent space of an autoencoder (AE-based) or directly in the input space (NoAE-based), guided by different perturbation constraints. Among them, Glacier-AE-Glob, Glacier-AE-Unc, Glacier-NoAE-Glob, and Glacier-NoAE-Unc achieve relatively higher validity, with average Flip Label Rates around 0.9, ranking among the stronger-performing methods overall. This suggests that global and unconstrained constraints are more effective for achieving label flips. In contrast, Glacier-AE-Unif and Glacier-NoAE-Unif consistently rank the lowest, likely due to the overly rigid nature of uniform perturbations. The overall difference between AE-based and NoAE-based variants is modest, with AE-based methods performing slightly better, indicating a mild benefit from leveraging learned latent representations, though this advantage appears limited in practice.

We summarize the relative performance of methods using a Critical Difference (CD) diagram based on average ranks across all 20 datasets (Fig. 4). The diagram confirms that NG-DBA outperforms all other methods, standing alone with the best average rank (1.1). This result supports our earlier claim about its robust label-flipping guarantee through progressive interpolation. InfoCELS and Glacier-AE-Unc form the next best-performing group, with comparable performance and strong average ranks (4.8 and 4.5, respectively). This highlights the importance of unconstrained perturbation (in Glacier) and removing saliency normalization (in InfoCELS) for enhancing validity. Other methods, such as TimeX and Glacier-NoAE-Unc, also show relatively strong validity, while NG-CAM and Glacier-Glob variants exhibit more moderate performance. In contrast, the lowest-ranking methods include MG, Glacier-Unif, and CELS, which consistently fail to produce valid counterfactuals across datasets. Their poor performance can be attributed to overly restrictive perturbation constraints or less reliable guidance mechanisms.

5.4.2 Proximity Evaluation

The results in Figs. 5, 6, and 7 evaluate the proximity of 16 counterfactual explanation models using three standard distance-based metrics: L1, L2, and $L_\infty $. These metrics quantify how much the generated counterfactuals deviate from the original inputs, with lower values indicating smaller perturbations and thus better proximity. In each figure, individual dots represent proximity scores for specific datasets, while the blue star shows the average distance across all 20 datasets for each method, positioned further to the left for more proximity-efficient methods.

Among all methods, Glacier-NoAE-Unif consistently achieves the best proximity performance, ranking first across all three metrics. Its blue star appears at the far left in Figs. 5, 6 and 7, confirming that this variant produces counterfactuals with minimal changes to the original input. Glacier-AE-Unif also performs exceptionally well, ranking second in both L1 and L2 distances. These results suggest that applying uniform constraints—which penalize all time step modifications equally—effectively reduces the magnitude of perturbations, leading to highly proximate counterfactuals. However, as discussed in Sect. 5.4.1, this advantage in proximity comes at the cost of significantly lower validity.

In contrast, NG-DBA shows the worst proximity in terms of L1 distance, as it modifies the entire time series using DTW Barycenter Averaging. MG ranks last under both L2 and $L_\infty $ distances due to its heuristic shapelet-replacement strategy, which often introduces large localized changes. These two instance- and shapelet-based methods are consistently positioned toward the far right in all proximity plots, indicating poor performance across all distance metrics.

Across the eight Glacier variants, proximity performance is strongly influenced by the type of perturbation constraint. Uniform-constrained variants (Glacier-AE-Unif and Glacier-NoAE-Unif) clearly outperform others, followed by local and global variants. In contrast, unconstrained variants such as Glacier-AE-Unc and Glacier-NoAE-Unc rank mid-to-low in proximity, reflecting the trade-off between validity and compactness.

The critical difference (CD) diagram for proximity (Fig. 8), based on the average ranks across the three distance metrics—$L_1$, $L_2$, and $L_\infty $—shows clear differences among the methods. The diagram confirms that Glacier-NoAE-Unif achieves the best overall proximity, standing alone with the lowest average rank (2.5). Glacier-AE-Unif follows closely as the second-best performer (3.3), forming a top-performing group that significantly outperforms the rest. In contrast, NG-DBA and MG exhibit the worst proximity, both with an average rank of 14.0. These results are fully consistent with the trends observed in Figs. 5, 6 and 7, where Glacier-NoAE-Unif and Glacier-AE-Unif appear at the far left with the smallest deviations, while NG-DBA and MG appear at the far right, reflecting large perturbations from the original input.

In summary, methods like Glacier-NoAE-Unif, Glacier-AE-Unif produce the most proximity-efficient counterfactuals across all three metrics. In contrast, NG-DBA and MG generate larger, less interpretable changes, highlighting a recurring trade-off where methods that prioritize label flipping or single segment replacement tend to sacrifice proximity.

5.4.3 Sparsity Evaluation

The results in Fig. 9 evaluate the sparsity of the generated counterfactual explanations, measured as the proportion of unchanged time steps after perturbation. Higher sparsity indicates that fewer data points were modified to achieve the counterfactual, which enhances interpretability by focusing on minimal, localized changes. In the scatter plot, each dot represents the sparsity score for a specific dataset, while the blue star denotes the average sparsity across all 20 datasets for each method.

Among all methods, CELS achieves the highest sparsity, meaning it alters the fewest time steps on average. However, as noted in Sect. 5.4.1, this comes at the cost of reduced validity, since the saliency-thresholding design sometimes fails to ensure a successful label flip. TimeX, NG-CAM, Glacier-NoAE-Unif, and InfoCELS also rank among the top five methods, highlighting that optimization-based approaches with sparsity constrain are effective in preserving most of the original time series.

In contrast, NG-DBA and Wachter exhibit the lowest sparsity scores, ranking at the bottom in Fig. 9. This result aligns with their design: both methods tend to modify the entire time series—NG-DBA by interpolating with a full-sequence barycenter and Wachter by applying global optimization without constraints—leading to highly non-sparse perturbations. Interestingly, unconstrained Glacier variants (Glacier-NoAE-Unc and Glacier-AE-Unc) also show very low sparsity (ranks 14 and 13), as their design allows full flexibility in selecting and modifying time steps without explicit constraints. This absence of targeted penalties for distributed changes often results in broad perturbations across the series, reducing sparsity. In contrast, the uniform-constrained variant Glacier-NoAE-Unif achieves relatively high sparsity, reflecting the effect of penalizing modifications across all time steps equally.

Another notable pattern is the difference between AE-based and NoAE-based Glacier variants. NoAE variants generally produce higher sparsity compared to their AE counterparts. This is likely because latent-space optimization (AE-based) involves decoding perturbations back into the input space, which often spreads changes more broadly, reducing sparsity.

The Critical Difference (CD) diagram for sparsity (Fig. 10) provides an aggregated view of the methods’ rankings based on their ability to minimize the number of modified time steps across all 20 datasets. The diagram confirms that CELS (rank 2.1) stands out as the most sparsity-efficient method, followed by TimeX (3.4), NG-CAM (3.8), InfoCELS (5.2), and Glacier-NoAE-Unif (5.3). On the other end, Wachter and NG-DBA tie for the lowest sparsity (rank 15), while unconstrained Glacier variants and global Glacier variants also cluster toward the bottom, confirming that methods without sparsity-oriented constraints or with single segment replacements inherently struggle to achieve sparse counterfactuals.

5.4.4 Plausibility Evaluation

The results in Figs. 11, 12, and 13 evaluate the plausibility of 16 counterfactual explanation models using three anomaly-detection-based metrics: Isolation Forest (IF), Local Outlier Factor (LOF), and One-Class SVM (OC-SVM). These metrics assess how realistic the generated counterfactuals are, with lower values indicating higher plausibility (i.e., a smaller proportion of counterfactuals flagged as out-of-distribution). Each figure presents dataset-specific scores as individual dots and the average across all datasets as a blue star, positioned farther left for more plausible methods.

Across all three metrics, the blue stars indicate clear differences in average plausibility among methods. For LOF (Fig. 12), most methods achieve relatively high plausibility, with average out-of-distribution (OOD) rates below 0.3 for all methods. In contrast, for IF (Fig. 11), the averages range more widely, with top-performing methods near 0.2–0.3 and the lowest-performing methods exceeding 0.5. For OC-SVM (Fig. 13), the variation is even larger, with top-performing methods near 0.1–0.3, while the weakest methods exceed 0.6. These differences reflect the sensitivity of each metric: LOF, based on local density, is more forgiving to small, localized changes, while IF and OC-SVM penalize global or high-dimensional deviations more strongly.

Among all methods, Wachter demonstrates the strongest plausibility overall, ranking first for LOF and OC-SVM and among the top three for IF. Its optimization framework enforces smooth perturbations without introducing unrealistic patterns, enabling counterfactuals to remain within the data manifold. For IF, MG, NG-CAM, and Wachter achieve average OOD scores below 0.3, while NG-DBA performs the worst, exceeding 0.5 due to its DTW barycenter averaging approach, which heavily alters global structure. For LOF, Wachter, TimeX, InfoCELS, CELS, NG-CAM, and SG maintain extremely low scores below 0.1, indicating strong local consistency. For OC-SVM, Wachter, TimeX, InfoCELS, and SG rank in the top four (below 0.3), whereas several AE-based Glacier variants appear at the bottom with scores above 0.5, reflecting reconstruction artifacts and constraint-driven distortions introduced during latent-space optimization.

Figure 14 presents a Critical Difference (CD) diagram based on the average ranks across three outlier detection metrics: Isolation Forest (IF), Local Outlier Factor (LOF), and One-Class SVM (OC-SVM). The diagram identifies Wachter as the best-performing method, followed by NG-CAM, TimeX, and InfoCELS, forming the top-performing group in plausibility. At the other extreme, NG-DBA ranks last, with most Glacier variants also clustered toward the bottom. These findings underscore a key insight: methods that apply smooth, proximity-aware perturbations or saliency-guided adjustments tend to generate more realistic counterfactuals, while approaches that rely on global averaging or latent-space reconstructions without strong plausibility constraints often produce unrealistic and out-of-distribution counterfactuals.

General Discussion for All CD Diagrams: Although the CD diagrams above provide a useful summary of overall rankings for each metric, it is important to note that the observed differences are not statistically significant under the Nemenyi test, as indicated by the horizontal connections in Figs. 4, 8, 10, and 14. CD tests measure statistical significance, not practical significance. Therefore, ties in the CD diagrams do not imply that all methods perform equally well; rather, they indicate that rank differences are relatively small and vary by dataset. This outcome is expected in the context of counterfactual explanations, as different methods are designed to optimize different objectives (e.g., validity, sparsity, proximity, plausibility), which naturally leads to trade-offs rather than universal superiority on a single metric. As shown in the scatter plots, performance differences are real but often inconsistent across datasets, which reduces statistical power under this test. Additionally, different datasets exhibit varying performance patterns for each metric, reflecting the diverse complexities of our benchmark. This variation is natural since the datasets come from different domains, each with distinct characteristics and levels of difficulty—a point we further explore in our case study (Sect. 5.6).

5.5 Trade Off Analysis

In the previous comparison of evaluation based on different metrics, we observed a trade-off between validity and other metrics such as proximity, sparsity, and plausibility across different methods. Next, we provide a detailed trade-off analysis between validity and these metrics, as we aim to disclose the advantages and disadvantages of each method in generating counterfactual explanations. By analyzing these trade-offs, we can better understand the inherent compromises that methods make when prioritizing certain properties over others.

5.5.1 Trade Off Between Validity and Proximity

Figure 15 highlights the trade-off between validity and proximity for each method across datasets, using L1 as the proximity measure. Ideally, a desirable counterfactual method should lie in the top-left region of each subplot—achieving high validity (Flip Rate close to 1.0) while maintaining low L1 proximity (i.e., minimal changes to the input). The original (unnormalized) L1 proximity values vary greatly across datasets, ranging from less than 1 to over 1000, as shown in Fig. 5. Plotting these values together with validity (which ranges from 0 to 1) on the x-axis and y-axis in their original scales would result in a highly skewed and unreadable figure. To address this and enable fair visual comparison across datasets, we applied min-max normalization to the L1 values before plotting them in Fig. 15, mapping all values to the [0, 1] range, where 0 corresponds to the smallest L1 value and 1 to the largest.

Figure 15 reveals three distinct behavioral patterns. The first group in Fig. 15, consisting of InfoCELS, Wachter, Glacier-AE-Unc, Glacier-AE-Glob, and Glacier-NoAE-Glob, clusters near the top-left corner, representing methods that strike a good balance between validity and proximity. This is achieved through targeted optimization strategies: InfoCELS removes saliency normalization and leverages saliency maps for focused time steps perturbation, while Wachter minimizes prediction loss with $L_1$ regularization to control perturbations. Similarly, Glacier variants use global or unconstrained constraints combined with optimization flexibility, allowing effective label flipping without excessive modifications.

The second group occupies the upper region of the plot, where validity remains consistently high but proximity varies considerably. This group includes TimeX, Glacier-NoAE-Unc, and NG-DBA. These methods strongly prioritize successful label flips, often relaxing proximity constraints. Consequently, they achieve near-perfect validity, but their proximity spans from moderate to large.

The third group aligns along the left-most section of the plot, characterized by very high proximity but substantial variability in validity. This group includes methods such as CELS, NG-CAM, MG. These approaches are designed to restrict perturbations to specific regions or structures, effectively minimizing changes. However, this tight control often reduces their ability to generate counterfactuals that successfully flip the predicted class.

Overall, these patterns underscore a fundamental trade-off between validity and proximity. While a few methods, such as those in the first group, approach the ideal balance, others demonstrate that prioritizing one property inevitably compromises the other. These findings highlight the need for adaptive or multi-objective optimization frameworks to reduce this trade-off and achieve counterfactual explanations that are both valid and minimally perturbed.

5.5.2 Trade Off Between Validity and Sparsity

Figure 16 illustrates the trade-off between validity and sparsity across all methods. Ideally, a well-performing method should cluster toward the top-right corner of each subplot, achieving high validity (Flip Rate near 1.0) while modifying only a small portion of the input (high sparsity). This balance is essential for interpretable counterfactuals that are both effective and concise.

The results reveal three distinct clusters. The first group, located near the upper-right corner, includes InfoCELS and TimeX, which achieve relatively high sparsity while maintaining strong validity. These methods balance interpretability and effectiveness by applying targeted perturbations—InfoCELS through saliency-guided optimization and TimeX via a single continuous segment constraint—thereby minimizing unnecessary changes without compromising predictive success.

The second group is characterized by methods that exhibit very high validity but comparatively low sparsity. This group includes NG-DBA, Wachter, Glacier-NoAE-Unc, and Glacier-AE-Unc. Their designs prioritize successful label flips without constraining the number of time steps that can be modified. For example, NG-DBA interpolates across the entire time series, while unconstrained Glacier variants allow optimization flexibility, both leading to dense perturbations.

The third group is characterized by methods that exhibit very high sparsity but substantial variability in validity. Methods such as CELS, Glacier-NoAE-Unif, MG, and NG-CAM consistently enforce strong constraints on the number of perturbed time steps. While this design achieves high sparsity, it limits the model’s flexibility to alter class-discriminative regions effectively, making it harder to flip predictions. As a result, these methods often sacrifice validity when sparsity constraints dominate the optimization objective.

Overall, the trade-off between validity and sparsity highlights a recurring challenge: methods designed to minimize the number of perturbed time steps often do so at the expense of validity, while approaches focused on guaranteeing label flips tend to introduce dense modifications that compromise interpretability. Only a few methods, such as InfoCELS and TimeX, achieve a balanced trade-off, maintaining strong validity without substantially increasing the perturbation scope. This observation underscores the importance of adaptive strategies that can dynamically balance these two objectives, rather than rigidly prioritizing one at the cost of the other.

5.5.3 Trade Off Between Validity and Plausibility

Figure 17 presents the trade-off between validity and plausibility for each method, using LOF (Local Outlier Factor) as the plausibility measure. A desirable method should ideally lie in the top-left of each subplot—achieving high validity (Flip Rate near 1.0) while maintaining low OOD values (indicating plausible counterfactuals that reside within the data distribution).

The results reveal three main clusters. The first group, located near the upper-left corner, includes Wachter, InfoCELS, TimeX, and SG. These methods successfully combine strong validity with low OOD scores, reflecting their design choices: Wachter uses $L_1$-regularization to constrain perturbations, while InfoCELS, TimeX, and SG leverage saliency-driven, segment-based, or shapelet-based optimization to ensure perturbations remain meaningful and realistic.

The second group occupies the upper region of the plot, spanning from left to right, and is characterized by methods with very high validity but comparatively varied plausibility. This cluster includes NG-DBA and Glacier-AE-Glob. These approaches prioritize achieving successful label flips while placing less emphasis on maintaining the realism of counterfactuals. For instance, NG-DBA employs DTW barycenter averaging to interpolate between instances, which often introduces unnatural patterns, whereas AE-based Glacier variants may produce reconstruction artifacts and distortions under rigid global constraints.

The third group consists of methods with high plausibility and varied validity, such as Glacier-AE-Unif, Glacier-NoAE-Unif, CELS, NG-CAM. These approaches maintain strong plausibility because they enforce strict constraints or saliency-driven perturbations that keep counterfactuals close to the data manifold. However, these restrictions reduce their flexibility to modify class-discriminative regions effectively, resulting in lower and less consistent validity across datasets.

Overall, these results demonstrate that methods emphasizing smooth optimization and guided perturbations—such as Wachter and InfoCELS—achieve the most favorable balance between validity and plausibility. In contrast, techniques relying on full-pattern transformations or latent-space reconstructions tend to sacrifice plausibility for high validity.

5.6 Case Study

To better demonstrate the interpretability and practical usefulness of different counterfactual explanation models, we present a case study using two well-known datasets from the UCR archive: GunPoint and Coffee. These datasets are chosen because their class differences are visually interpretable, yet they represent different levels of complexity. GunPoint is more challenging, as samples from the same class can differ in magnitude and local shape, making it harder to generate natural counterfactuals. In contrast, Coffee exhibits highly consistent class patterns, making counterfactual generation relatively straightforward. This contrast allows us to examine how dataset variability influences the behavior and quality of different counterfactual methods.

5.6.1 GunPoint Dataset

The GunPoint dataset records the motion of a hand drawing a gun versus pointing with a finger. Each time series shows the x-axis position of the hand, so certain parts of the sequence strongly indicate the action. For example, as shown in Fig. 18, samples from the Gun class in Fig. 18a exhibit noticeable oscillations at the beginning and end of the series, representing the gun being taken out and put back into the holster. In contrast, the same regions for the Point class in Fig. 18b remain smooth and stable, reflecting the gradual movement of finger pointing.

To show the difference in counterfactuals generated among all the models on the GunPoint dataset, we select a sample labeled Point that has valid counterfactuals from all methods. We compare the original instance, labeled as Point, with the corresponding counterfactuals that flip the prediction to the opposite class (Gun). From Fig. 19, we can see that different models generate counterfactuals with varying levels of change, reflecting their design strategies.

Wachter does not include a sparsity constraint, so its perturbations spread across the whole time series data. However, due to the $L_1$ distance constraint in its objective, the overall proximity remains relatively reasonable. NG-DBA modifies the entire series because it interpolates between the original instance and the nearest unlike neighbor, also resulting in very low sparsity. In contrast, NG-CAM changes only one important segment identified by Class Activation Mapping, making the modification localized and easy to interpret. Similarly, MG, TimeX, and SG are designed to modify a single continuous segment, but they behave differently. MG replaces the segment with a shapelet from another class, which in this example creates an abrupt jump and a noticeable change in magnitude. This issue is more pronounced in the GunPoint dataset because samples within the same class often have different magnitudes (refer to Fig. 18). As a result, replacing a segment without adjusting for this difference can distort the overall pattern and make the counterfactual look less natural. TimeX and SG use optimization with distance constraints, which helps keep the overall shape close to the original. However, this sometimes introduces unnatural patterns—TimeX shows oscillations in the modified region, and SG adds spiky fluctuations. These artifacts suggest that even though distance constraints maintain global similarity, they do not always produce smooth or realistic changes. Both methods rely on shapelets and DBA to guide localized perturbations. But when a single shapelet or DBA reference does not capture the full variation within a class—as seen in Fig. 18, where samples from the same class differ in magnitude—it can cause mismatched adjustments. This mismatch often leads to unrealistic counterfactuals, despite the optimization objective enforcing proximity.

CELS and InfoCELS apply perturbations at the time-step level, resulting in sparse modifications across the series. Both methods introduce small-magnitude changes, but some abrupt spikes can still be observed, especially at the start and end regions, which reduce smoothness. Removing the saliency normalization step in InfoCELS slightly reduces these sharp artifacts, leading to somewhat smoother transitions compared to CELS. While the difference between them is modest, both approaches tend to produce counterfactuals with fewer oscillations or spiky fluctuations than segment-based methods like TimeX and SG.

The Glacier family uses gradient-based optimization to minimize a combined loss function that balances prediction change and a constraint term. The constraint vector $\varvec{c}$ is not a hard restriction; it softly penalizes changes outside preferred regions. This penalty does not enforce sparsity directly—it only influences the overall modifications without strictly limiting their spread. As a result, even with different constraint strategies—local, global, uniform, and unconstrained—perturbations can still occur across the entire time series. Consequently, Glacier often achieves good proximity because of the $L_c$ constraint loss term, but does not guarantee high sparsity. AE-based and NoAE-based variants show clear differences in the naturalness of their modifications. AE-based models (the third row in Fig. 19) optimize in latent space and then decode to the input space, which can introduce reconstruction artifacts. In particular, local, global, and uniform AE-based variants often produce unnatural patterns, such as wavy distortions, due to both reconstruction error and overly restrictive region weighting. In contrast, the unconstrained AE-based variant tends to generate smoother and more natural changes. NoAE-based models (bottom row in Fig. 19), which directly optimize the input space, generally produce more natural perturbations than AE-based Glacier models.

5.6.2 Coffee Dataset

The Coffee dataset distinguishes between Robusta and Arabica coffee beans using Fourier transform infrared spectroscopy signals. Unlike GunPoint, Coffee exhibits highly consistent class patterns, making it easier to generate realistic counterfactuals. As shown in Fig. 20, both classes share a similar shape but differ in key details: Robusta has a higher second peak, while Arabica generally has the last peak lower than the first two.

We select an instance labeled as Arabica and compare its counterfactuals across all models, flipping the prediction to Robusta. From Fig. 21, we observe that all methods produce smooth and interpretable counterfactuals for Coffee in general. Segment-level methods (MG, TimeX, SG) perform well because the reference patterns (shapelet or DBA) derived from each class are sufficiently representative, thanks to the stability of patterns within each class from the dataset. This ensures that shapelet replacement in MG, DBA-guided optimization in TimeX, and shapelet-guided optimization in SG do not introduce large discrepancies between the original and modified segments. As a result, these methods avoid the abrupt magnitude mismatches commonly observed in GunPoint, where intra-class variability is higher. Time-step-level methods (CELS and InfoCELS) also maintain high sparsity and generally produce smooth transitions on the Coffee dataset. The stability of class patterns reduces the potential negative effects of saliency map normalization seen in more variable datasets like GunPoint. As a result, CELS and InfoCELS perform similarly and reliably, indicating that for datasets with stable patterns like Coffee, saliency map normalization does not introduce significant noise, making the advantage of InfoCELS less critical. Glacier variants also generate natural counterfactuals on the Coffee dataset. Both AE-based and NoAE-based Glacier models produce smooth and realistic changes, with only slight differences between them. While AE-based variants may introduce minor reconstruction artifacts under strict local or uniform constraints, these effects are negligible in Coffee compared to more complex datasets like GunPoint. Overall, all Glacier variants deliver natural and interpretable counterfactuals for Coffee.

These case studies highlight the strong influence of dataset characteristics on counterfactual generation. On GunPoint, where intra-class variability is high, segment-based methods often introduce abrupt or oscillatory changes, and even time-step-level approaches show minor artifacts, while some Glacier variants produce unnatural distortions due to reconstruction or constraint effects. In contrast, the Coffee dataset’s consistent class patterns allow all methods to generate smooth and interpretable counterfactuals with minimal artifacts. Overall, dataset stability plays a critical role in ensuring the naturalness and reliability of counterfactual explanations.

6 Future Research Directions

Our findings reveal clear trade-offs among validity, sparsity, proximity, and plausibility, showing that no single method excels across all metrics. These trade-offs highlight the need for multi-objective optimization and comprehensive evaluation frameworks to balance competing goals. Insights from the case study further emphasize the impact of dataset characteristics, calling for data-aware adaptation. Other important directions include improving plausibility, enabling domain-specific customization, and incorporating user feedback. Based on these insights, several promising future research directions are outlined below:

Comprehensive Evaluation Metrics: Future research should focus on developing a comprehensive evaluation framework that integrates validity, proximity, sparsity, and plausibility. This framework should not prioritize any single metric but rather assess how well methods balance these multiple dimensions. It would help avoid situations where counterfactuals are either valid but unrealistic or interpretable but ineffective in altering predictions.

Methods to Improve Plausibility: Given that plausibility ensures the realism of counterfactuals, future research could explore ways to enhance plausibility without compromising validity. This could involve integrating domain-specific constraints or generative models that ensure counterfactuals lie within the data manifold while still achieving classification changes.

Multi-Objective Optimization: One promising direction is to explore multi-objective optimization approaches that balance validity, plausibility, sparsity, and proximity dynamically. Such methods could adaptively balance competing objectives based on the application context, prioritizing validity in high-stakes domains where outcome changes are critical, while preserving plausibility and sparsity to support interpretability in user-facing or decision-support scenarios.

Domain-Specific Counterfactual Generation: In some domains, plausibility may be more critical (e.g., in healthcare, where counterfactuals need to be medically feasible), while in others, sparsity or proximity may be more valuable. Future work should focus on creating customizable counterfactual methods tailored to specific domains, balancing metrics based on the requirements of the task.

Dataset-Aware Adaptation: The case study demonstrates how dataset characteristics strongly influence method behavior. Future research should explore dataset-aware strategies that adjust perturbation mechanisms (e.g., shapelet-based, saliency-guided, or latent-space optimization) based on intra-class variability, temporal continuity, and domain constraints. For example, shapelet-based methods may excel in stable datasets like Coffee but require adaptive adjustments for highly variable datasets such as GunPoint.

Integrating User Feedback: Another promising research direction is developing interactive counterfactual generation methods where user feedback guides the balance between validity, sparsity, and plausibility. This user-centric approach can help fine-tune counterfactual explanations to align with the interpretability needs of different stakeholders.

7 Conclusion

This work provides the first large-scale, framework-agnostic evaluation of 16 counterfactual explanation models for univariate time series classification across 20 UCR datasets. Our findings confirm that no single method dominates across all four key dimensions—validity, proximity, sparsity, and plausibility. Instead, performance strongly depends on the underlying design strategy, which introduces distinct trade-offs across categories:

Instance-based (or Heuristic-based) methods (NG-DBA, NG-CAM) rely on fixed perturbation strategies. NG-DBA guarantees perfect validity by interpolating toward the nearest unlike neighbor but sacrifices sparsity due to large-scale changes. NG-CAM uses class activation maps to identify and modify the most discriminative segment, achieving strong validity and sparsity. However, its effectiveness heavily depends on the quality of the class activation map—poor or noisy importance estimates can lead to suboptimal or unrealistic modifications.

Shapelet-based methods (MG, SG) leverage discriminative shapelets but differ in how they apply them. MG directly replaces a segment with a shapelet from another class, which can work well when the selected shapelet is representative, achieving good validity and sparsity. However, when shapelets are poorly matched, this approach often introduces abrupt magnitude shifts and disrupts global patterns, reducing plausibility. In contrast, SG uses shapelets as guidance for optimization, which generally produces smoother transitions than direct replacement but still struggles with plausibility on datasets with high intra-class variability. Both methods strongly depend on the quality and representativeness of the chosen shapelets.

Optimization-based methods (Wachter, Glacier family, TimeX) generate counterfactuals by minimizing composite objectives that incorporate different loss terms depending on the design. Wachter achieves decent validity and proximity by optimizing prediction and distance losses, but suffers from low sparsity because perturbation is spread across the whole time series. TimeX constrains perturbations to a single contiguous segment and incorporates a DBA-based alignment term to maintain class consistency. This design balances validity and sparsity but can introduce oscillatory distortions when DBA references fail to represent intra-class variation. The Glacier family employs gradient-based optimization in either input or latent space under various region-weighting strategies (local, global, uniform, unconstrained). While Glacier variants often achieve strong proximity, they do not enforce sparsity and tend to perturb the whole time series data. AE-based models further risk reconstruction artifacts due to decoding, particularly under strict region constraints, whereas NoAE-based variants generally produce smoother and more natural perturbations.

Saliency-based methods (CELS, InfoCELS) perturb individual time steps guided by saliency-map importance scores, achieving high sparsity and strong interpretability. Removing saliency normalization in InfoCELS generally improves validity and reduces some abrupt artifacts, making edits smoother on many datasets. However, as shown in the GunPoint case study, both methods can still introduce unrealistic local fluctuations, especially in datasets with high intra-class variability. While InfoCELS tends to perform better overall, its improvements are not always sufficient to guarantee natural-looking counterfactuals.

These results highlight that the choice of counterfactual method should be guided by the priorities of the application. In high-stakes domains, validity is essential because a counterfactual that fails to change the model’s decision provides no actionable insight. When the goal is to keep explanations close to the original input for realism or regulatory purposes, proximity becomes critical. In settings where interpretability depends on minimal, localized changes—such as user-facing explanations—sparsity should be emphasized. Finally, for domains where realism and consistency with the data distribution are vital, such as healthcare or safety-critical systems, plausibility must be prioritized to ensure counterfactuals are feasible and trustworthy. Understanding these trade-offs enables practitioners to balance competing objectives and select or adapt methods that best fit their operational and interpretability requirements.

Our case study further demonstrates that dataset characteristics significantly influence counterfactual behavior. On variable datasets like GunPoint, where intra-class differences in magnitude and local patterns are high, several methods produce unrealistic or oscillatory perturbations, even when validity is maintained. In contrast, the same methods generate smooth and interpretable counterfactuals on stable datasets like Coffee, where class patterns are consistent. These observations underscore the importance of developing data-aware adaptation strategies that can adjust perturbation mechanisms based on properties such as intra-class variability, temporal continuity, and structural complexity.

While this work focuses on univariate datasets, extending counterfactual frameworks to multivariate time series (MTS) remains an open and critical challenge, as most univariate counterfactual models cannot be directly adapted to handle the complexity of inter-variable dependencies and higher dimensionality. Although some initial research has explored MTS counterfactuals, significant gaps remain in ensuring plausibility and interpretability across multiple correlated channels. In future work, we aim to conduct a comprehensive survey of MTS counterfactual approaches, examining their limitations and design principles, and outlining promising directions for advancing research in this complex setting.

In addition, our benchmark focuses on evaluating counterfactual explanation methods using a fixed black-box model to ensure fairness and isolate explanation quality. However, we recognize that the performance of a counterfactual method may vary when applied to different classifiers. Exploring how explanation quality changes across various black-box models is an important direction for future work, as it would provide a more comprehensive understanding of method robustness and generalizability.

Data Availability

A public repository for this work is available at: TS-Counterfactual-Explanation-Bake-off. The repository includes the implementation, configuration details, and supplementary materials relevant to the experiments and results presented in this manuscript.

References

Adadi, A., & Berrada, M. (2018). Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access, 6, 52138–52160.
Article Google Scholar
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2020). Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58, 82–115.
Article Google Scholar
Ates, E., Aksar, B., Leung, V. J., & Coskun, A. K. (2021). Counterfactual explanations for multivariate time series. In: 2021 International Conference on Applied Artificial Intelligence (ICAPAI), pp. 1–8. IEEE.
Bahri, O., Boubrahimi, S. F., & Hamdi, S. M. (2022). Shapelet-based counterfactual explanations for multivariate time series. arXiv preprint arXiv:2208.10462.
Bahri, O., Li, P., Boubrahimi, S. F., & Hamdi, S. M. (2022). Temporal rule-based counterfactual explanations for multivariate time series. In: 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 1244–1249. IEEE.
Bodria, F., Giannotti, F., Guidotti, R., Naretto, F., Pedreschi, D., & Rinzivillo, S. (2023). Benchmarking and survey of explanation methods for black box models. Data Mining and Knowledge Discovery, 37(5), 1719–1778.
Article MathSciNet Google Scholar
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.
Article Google Scholar
Breunig, M. M., Kriegel, H.-P., Ng, R. T. & Sander, J. (2000). LOF: identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 93–104
Byrne, R. M. (2019). Counterfactuals in explainable artificial intelligence (XAI): Evidence from human reasoning. In: IJCAI, California. pp. 6276–6282.
Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019). Machine learning interpretability: A survey on methods and metrics. Electronics, 8(8), 832.
Article Google Scholar
Craven, M., & Shavlik, J. (1995). Extracting tree-structured representations of trained networks. Advances in Neural Information Processing Systems,8.
Dau, H. A., Bagnall, A., Kamgar, K., Yeh, C.-C.M., Zhu, Y., Gharghabi, S., Ratanamahatana, C. A., & Keogh, E. (2019). The UCR time series archive. IEEE/CAA Journal of Automatica Sinica, 6(6), 1293–1305.
Article Google Scholar
Delaney, E., Greene, D., & Keane, M. T. (2021). Instance-based counterfactual explanations for time series classification. International Conference on Case-Based Reasoning (pp. 32–47). New York: Springer.
Google Scholar
Dempster, A., Petitjean, F., & Webb, G. I. (2020). Rocket: Exceptionally fast and accurate time series classification using random convolutional kernels. Data Mining and Knowledge Discovery, 34(5), 1454–1495.
Article MathSciNet Google Scholar
Dhurandhar, A., Chen, P.-Y., Luss, R., Tu, C.-C., Ting, P., Shanmugam, K., & Das, P. (2018). Explanations based on the missing: Towards contrastive explanations with pertinent negatives. Advances in Neural Information Processing Systems,31.
Filali Boubrahimi, S., & Hamdi, S. M. (2022). On the mining of time series data counterfactual explanations using barycenters. In: Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pp. 3943–3947.
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018). Explaining explanations: An overview of interpretability of machine learning. In: 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), pp. 80–89. IEEE.
Goebel, R., Chander, A., Holzinger, K., Lecue, F., Akata, Z., Stumpf, S., Kieseberg, P., & Holzinger, A. (2018). Explainable ai: the new 42? In: International Cross-domain Conference for Machine Learning and Knowledge Extraction, pp. 295–303. Springer.
Guidotti, R. (2024). Counterfactual explanations and how to find them: Literature review and benchmarking. Data Mining and Knowledge Discovery, 38(5), 2770–2824.
Article MathSciNet Google Scholar
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2018). A survey of methods for explaining black box models. ACM Computing Surveys (CSUR), 51(5), 1–42.
Article Google Scholar
Hao, J., Shi, Q., Ye, Y., & Zeng, W. (2023). Timetuner: Diagnosing time representations for time-series forecasting with counterfactual explanations. IEEE Transactions on Visualization and Computer Graphics, 30(1), 1183–1193.
Article Google Scholar
Höllig, J., Kulbach, C., & Thoma, S. (2022). TSEVO: Evolutionary counterfactual explanations for time series classification. In: 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 29–36. IEEE.
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., & Muller, P.-A. (2019). Deep learning for time series classification: A review. Data Mining and Knowledge Discovery, 33(4), 917–963.
Article MathSciNet Google Scholar
Kan, Z., Rezaei, S., et al. (2024). Benchmarking counterfactual interpretability in deep learning models for time series classification. arXiv preprint arXiv:2408.12666.
Keogh, E., & Ratanamahatana, C. A. (2005). Exact indexing of dynamic time warping. Knowledge and Information Systems, 7, 358–386.
Article Google Scholar
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980
Li, P., Bahri, O., Boubrahimi, S. F., & Hamdi, S. M. (2022). SG-CF: Shapelet-guided counterfactual explanation for time series classification. In: 2022 IEEE International Conference on Big Data (Big Data), pp. 1564–1569. IEEE.
Li, P., Bahri, O., Filali, S., & Hamdi, S. M. (2023). CELS: Counterfactual explanations for time series data via learned saliency maps. In: 2023 IEEE International Conference on Big Data (BigData), pp. 718–727. IEEE.
Li, P., Bahri, O., Hosseinzadeh, P., Boubrahimi, S. F., & Hamdi, S. M. (2025). Info-cels: Informative saliency map-guided counterfactual explanation for time series classification. Electronics, 14(7), 1311.
Article Google Scholar
Li, P., Boubrahimi, S. F., & Hamdi, S. M. (2022). Motif-guided time series counterfactual explanations. In: International Conference on Pattern Recognition, pp. 203–215. Springer.
Li, X.-H., Cao, C. C., Shi, Y., Bai, W., Gao, H., Qiu, L., Wang, C., Gao, Y., Zhang, S., Xue, X., et al. (2020). A survey of data-driven and knowledge-aware explainable AI. IEEE Transactions on Knowledge and Data Engineering, 34(1), 29–49.
Google Scholar
Lines, J., Davis, L. M., Hills, J., & Bagnall, A. (2012). A shapelet transform for time series classification. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 289–297
Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008). Isolation forest. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 413–422. IEEE.
Martens, D., Baesens, B., Van Gestel, T., & Vanthienen, J. (2007). Comprehensible credit scoring models using rule extraction from support vector machines. European Journal of Operational Research, 183(3), 1466–1476.
Article Google Scholar
Mothilal, R. K., Sharma, A., & Tan, C. (2020). Explaining machine learning classifiers through diverse counterfactual explanations. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp. 607–617.
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019). Definitions, methods, and applications in interpretable machine learning. Proceedings of the National Academy of Sciences, 116(44), 22071–22080.
Article MathSciNet Google Scholar
Pasquale, F. (2015). The black box society: the secret algorithms that control money and information. Harvard University Press.
Book Google Scholar
Powell, M. J. (1973). On search directions for minimization algorithms. Mathematical Programming, 4, 193–201.
Article MathSciNet Google Scholar
Pyakillya, B., Kazachenko, N., & Mikhailovsky, N. (2017). Deep learning for ECG classification. Journal of Physics: Conference Series, 913, 012004.
Google Scholar
Ribeiro, M. T., Singh, S., Guestrin, C. (2016). “ Why should i trust you?” explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144.
Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence, 1(5), 206–215.
Article Google Scholar
Samek, W., Montavon, G., Vedaldi, A., Hansen, L. K., & Müller, K.-R. (2019). Explainable AI: Interpreting, Explaining and Visualizing Deep Learning (Vol. 11700). Germany: Springer.
Book Google Scholar
Schölkopf, B., Platt, J. C., Shawe-Taylor, J., Smola, A. J., & Williamson, R. C. (2001). Estimating the support of a high-dimensional distribution. Neural Computation, 13(7), 1443–1471.
Article Google Scholar
Sivill, T., & Flach, P. (2022). Limesegment: Meaningful, realistic time series explanations. In: International Conference on Artificial Intelligence and Statistics, pp. 3418–3433. PMLR.
Spinnato, F., Guidotti, R., Monreale, A., Nanni, M., Pedreschi, D., & Giannotti, F. (2023). Understanding any time series classifier with a subsequence-based explainer. ACM Transactions on Knowledge Discovery from Data, 18(2), 1–34.
Article Google Scholar
Susto, G. A., Cenedese, A., & Terzi, M. (2018). Time-series classification methods: Review and applications to power systems data. Big Data Application in Power Systems, 179–220.
Theissler, A., Spinnato, F., Schlegel, U., & Guidotti, R. (2022). Explainable ai for time series classification: A review, taxonomy and research directions. IEEE Access, 10, 100700–100724.
Article Google Scholar
Tjoa, E., & Guan, C. (2020). A survey on explainable artificial intelligence (XAI): Toward medical XAI. IEEE Transactions on Neural Networks and Learning Systems, 32(11), 4793–4813.
Article Google Scholar
Van Looveren, A., & Klaise, J. (2021). Interpretable counterfactual explanations guided by prototypes. In: Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part II 21, pp. 650–665. Springer.
Wachter, S., Mittelstadt, B., & Russell, C. (2017). Counterfactual explanations without opening the black box: Automated decisions and the GDPR. The Harvard Journal of Law & Technology, 31, 841.
Google Scholar
Wang, Z., Samsten, I., Mochaourab, R., & Papapetrou, P. (2021). Learning time series counterfactuals via latent space representations. In: International Conference on Discovery Science, pp. 369–384. Springer.
Wang, Z., Samsten, I., Miliou, I., Mochaourab, R., & Papapetrou, P. (2024). Glacier: Guided locally constrained counterfactual explanations for time series classification. Machine Learning, 113(7), 4639–4669.
Article MathSciNet Google Scholar
Wang, Z., Yan, W., & Oates, T. (2017). Time series classification from scratch with deep neural networks: A strong baseline. In: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 1578–1585. IEEE.
Yan, J., & Wang, H. (2023). Self-interpretable time series prediction with counterfactual explanations. In: International Conference on Machine Learning, pp. 39110–39125. PMLR.
Yao, S., Hu, S., Zhao, Y., Zhang, A., & Abdelzaher, T. (2017). Deepsense: A unified deep learning framework for time-series mobile sensing data processing. In: Proceedings of the 26th International Conference on World Wide Web, pp. 351–360.
Ye, L., & Keogh, E. (2009). Time series shapelets: A new primitive for data mining. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 947–956
Zeiler, M. D., & Fergus, R. (2014). Visualizing and understanding convolutional networks. In: European Conference on Computer Vision, pp. 818–833. Springer.
Zhang, Y., Chen, X., et al. (2020). Explainable recommendation: A survey and new perspectives. Foundations and Trends® in Information Retrieval, 14(1), 1–101.
Article MathSciNet Google Scholar

Download references

Acknowledgements

We would also like to thank Eamonn Keogh and his team at UCR for their efforts in collecting, cleaning, and curating the datasets used in this study and making them freely available.

Funding

This project has been supported in part by funding from the Division of Atmospheric and Geospace Sciences within the Directorate for Geosciences, under NSF awards #2301397, #2204363, and #2240022, and by funding from the Office of Advanced Cyberinfrastructure within the Directorate for Computer and Information Science and Engineering, under NSF award #2305781.

Author information

Authors and Affiliations

School of Computing, Utah State University, Logan, UT, USA
Peiyu Li, Omar Bahri, Soukaina Filali Boubrahimi & Shah Muhammad Hamdi

Authors

Peiyu Li
View author publications
Search author on:PubMed Google Scholar
Omar Bahri
View author publications
Search author on:PubMed Google Scholar
Soukaina Filali Boubrahimi
View author publications
Search author on:PubMed Google Scholar
Shah Muhammad Hamdi
View author publications
Search author on:PubMed Google Scholar

Contributions

P.L. conducted the literature review, implemented the experiments, and wrote the initial draft of the manuscript. S.F.B., S.M.H., and O.B. provided supervision, contributed to the conceptual framing of the study, and offered guidance throughout the research process. All authors reviewed and approved the final version of the manuscript.

Corresponding author

Correspondence to Peiyu Li.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Editor: Gustavo Batista.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Cite this article

Li, P., Bahri, O., Boubrahimi, S.F. et al. Counterfactual Explanation Bake-Off: A Review and Experimental Evaluation for Time Series Classification. Mach Learn 115, 114 (2026). https://doi.org/10.1007/s10994-026-07056-4

Download citation

Received: 14 April 2025
Revised: 31 March 2026
Accepted: 17 April 2026
Published: 06 May 2026
Version of record: 06 May 2026
DOI: https://doi.org/10.1007/s10994-026-07056-4

Keywords

Profiles

Peiyu Li View author profile
Shah Muhammad Hamdi View author profile

Counterfactual Explanation Bake-Off: A Review and Experimental Evaluation for Time Series Classification

Abstract

Similar content being viewed by others

Interactive Counterfactual Generation for Univariate Time Series

Denoising Optimization-Based Counterfactual Explanations for Time Series Classification

Streaming Counterfactual Subset Selection with Bias Awareness

Explore related subjects

1 Introduction

2 Related Works

3 Counterfactual Explanations for Time Series Classification

3.1 Problem Definition

3.2 Properties of Counterfactuals

4 Categorization of Counterfactual Explainers

4.1 Instance-Based Counterfactual Explainer

4.2 Shapelet-Based Counterfactual Explainer

4.3 Optimization Strategies

4.4 Saliency Map Guided Counterfactual Explainer

5 Experiments

5.1 Datasets

5.2 Black-Box Classification Models

5.3 Evaluation Metrics

5.3.1 Validity Metric

5.3.2 Proximity Metric

5.3.3 Sparsity Metric

5.3.4 Plausibility Metric

5.4 Experimental Results

5.4.1 Validity Evaluation

5.4.2 Proximity Evaluation

5.4.3 Sparsity Evaluation

5.4.4 Plausibility Evaluation

5.5 Trade Off Analysis

5.5.1 Trade Off Between Validity and Proximity

5.5.2 Trade Off Between Validity and Sparsity

5.5.3 Trade Off Between Validity and Plausibility

5.6 Case Study

5.6.1 GunPoint Dataset

5.6.2 Coffee Dataset

6 Future Research Directions

7 Conclusion

Data Availability

References

Acknowledgements

Funding

Author information

Authors and Affiliations

Contributions

Corresponding author

Ethics declarations

Conflict of interest

Additional information

Publisher's Note

Rights and permissions

About this article

Cite this article

Share this article

Keywords

Profiles