Interpreting User Opinions: A Multidimensional Approach Leveraging Explainable AI and Generative Models

Abstract

In today’s digital landscape, user-generated opinions—such as online reviews, user comments, and social media posts—offer valuable insights into people’s experiences, sentiments, and concerns, influencing decisions across businesses, organizations, and public policy. Advanced machine learning techniques, particularly Large Language Models (LLMs) like BERT and GPT, facilitate the automated analysis of this vast, unstructured data to extract actionable information. However, beyond high classification accuracy, there is a growing demand for explainability to ensure transparency and trust in automated systems. Understanding why an opinion is classified in a particular way is critical for informed decision-making. This paper proposes a multidimensional, explainable framework that combines LLM-based classification across latent dimensions (e.g., sentiment, topic, emotion), interpretable AI for identifying influential words, and generative AI for producing human-readable explanations. Unlike standard explanations generated solely by models such as GPT, our method integrates Explainable AI (XAI) techniques to pinpoint influential words for each classification dimension and organizes them into structured, dimension-aware outputs—significantly enhancing interpretability and alignment with model predictions. Experimental results—based on text-level metrics, latent space representations, and qualitative assessments from both automated tools and human experts—demonstrate the effectiveness of our approach in improving transparency, interpretability, and usability in opinion analysis.

1 Introduction

In today’s digital landscape, user-generated content—including social media posts, online discussions, comments, and feedback—provides valuable insights into public opinion, preferences, and concerns (Kirchner et al., 2021; Mudambi & Schuff, 2010). As the volume and influence of online opinions increase, such evidence increasingly informs strategic decisions across sectors such as technology, healthcare, finance, and public policy.

Recent advances in Transformer-based Large Language Models (LLMs), such as BERT and GPT (Myers et al., 2024), have substantially improved automated opinion analysis (e.g., sentiment, topic, and trend detection). However, general-purpose pre-training may miss domain-specific nuances, making task and domain adaptation (e.g., fine-tuning) important for accurate predictions, particularly in multidimensional settings that jointly model sentiment, emotion, and topic.

As classification systems become more powerful, explainability is essential to support trust and real-world use (Guidotti et al., 2021). Explainable AI (XAI) methods such as LIME and Integrated Gradients help identify the words that drive predictions (Ali et al., 2023; Ribeiro et al., 2016; Sundararajan et al., 2017), while organizing outputs across multiple dimensions further improves interpretability (Cantini et al., 2024, 2025). Generative models (e.g., GPT) can transform these structured signals into fluent, human-readable explanations, producing clearer and more evidence-grounded justifications of the predicted labels when coupled with XAI (Belcastro et al., 2024).

This paper introduces a multidimensional and explainable methodology for classifying user-generated opinions by combining fine-tuned LLMs, XAI techniques, and generative models. Our approach consists of three main steps: (i) mapping user opinions into high-dimensional latent spaces and classifying them across multiple dimensions; (ii) interpreting each dimension’s classification using XAI to pinpoint influential words; and (iii) generating coherent, interpretable explanations that explicitly justify the classifications. The framework is designed for practical deployment across diverse analytical scenarios, enabling on-demand explanation generation for individual opinions and reuse of topic structures in large-scale or continuously updated datasets.

To evaluate the framework, we conduct experiments on diverse datasets from multiple domains, including product reviews, hotel service evaluations, social media discussions, and political discourse. Opinions are analyzed along various dimensions such as sentiment, topic, emotion, and domain-specific factors (e.g., candidate support or psychological distress). Using textual and semantic similarity metrics, LLM-as-a-Judge analysis, and expert reviews, we show that our combined approach (GPT + XAI) consistently yields clearer, more informative, and contextually aligned explanations than GPT alone.

In contrast to prior studies that treat a single dimension in isolation—whether sentiment, topic, emotion, or domain-specific factors (e.g., political stance, mental-health status) (Guidotti, 2024; Krugmann & Hartmann, 2024; Qu et al., 2025; Wang et al., 2025)—our approach integrates multiple analytical layers within an explainable architecture, using LLMs as post-hoc synthesizers of specialized model outputs (e.g., BERT-based classifiers, BERTopic) and XAI signals (e.g., LIME). Compared to our previous study (Cosentino et al., 2025), we introduce the following major contributions:

• Broader Opinion Scope: We extend the analysis beyond product reviews to include a wider range of user-generated content—such as social media posts, political opinions, and mental health narratives—capturing dynamic patterns across multiple domains.

• End-to-end integration: Our contribution lies in the construction of a unified pipeline that jointly produces multidimensional outputs (sentiment, emotions, topics, domain-specific labels) accompanied by explicit token-level rationales.

• Multidimensional Classification: Our model simultaneously classifies opinions across multiple dimensions—sentiment, emotion, topic, and domain-specific signals (e.g., political alignment, mental state)—enabling richer and more context-aware interpretation.

• Enhanced Explainability: We integrate XAI techniques (e.g., LIME) with generative models (e.g., GPT) to highlight key lexical features and produce fluent, human-readable explanations aligned with model outputs.

• BERTopicXAI for Topic Transparency: We propose BERTopicXAI, an interpretable extension of BERTopic that provides token-level visualizations, enhancing the explainability of unsupervised topic modeling.

• Structured and Aligned Outputs: The generated explanations are not only fluent but also explicitly structured by classification dimension—linking key expressions to sentiment, topic, emotion, and domain-specific attributes—thus improving transparency and interpretive depth.

• Comprehensive Evaluation: We present a robust evaluation pipeline, that combines textual/semantic metrics, ablation from base, to intermediate and advanced configurations, and both automated (LLM-as-judge) and human expert evaluations.

The remainder of the paper is structured as follows. Section 2 offers a concise analysis of related works. Section 3 describes the proposed methodology. Section 4 presents the dataset, analyzes input examples with their generated explanations, and provides a comparative evaluation. Finally, Sects. 5 and 6 discuss limitations and conclude the paper.

2 Related Work

In the era of artificial intelligence, Transformer-based LLMs have revolutionized data analysis by enabling efficient and context-aware insight extraction. LLMs are typically categorized into decoder-based models (e.g., GPT) and encoder-based models (e.g., BERT), which excel in causal language generation and semantic representation, respectively. Leveraging advanced Natural Language Processing (NLP) techniques, these models can interpret user queries and generate detailed, human-readable reports across domains such as education, healthcare, finance, and information technology (Ayanouz, 2020; Belcastro et al., 2023; Gilardi et al., 2023; Hossain et al., 2022; Meng et al., 2023; Wang et al., 2022).

Despite their success, interpreting deep learning models remains challenging, particularly in high-stakes contexts where understanding model reasoning is crucial. Explainable AI (XAI) methods address this issue by improving transparency and trust. Techniques such as LIME and Integrated Gradients quantify feature importance, while self-explanatory architectures generate explanations alongside predictions, albeit with limited flexibility (Kumar & Raman, 2022).

Recent research has increasingly leveraged LLMs to enhance XAI. For instance, LLMs have been used to translate SHAP outputs into natural language (Hsu et al., 2024) and to generate sentiment explanations with improved user trust (Krugmann & Hartmann, 2024). Comparative studies also examine how encoder-based models differ in human alignment, robustness, and contrastivity (Mersha et al., 2025). Narrative-driven LLM explanations reduce cognitive load and improve comprehension (Martens et al., 2025), while counterfactual approaches explore fairness and trustworthiness (Crisan et al., 2024; Guidotti, 2024; Wang et al., 2025). Other works employ LLMs to refine perturbation strategies within XAI methods (Angiulli et al., 2024). Together, these studies highlight the growing role of LLMs as mediators between complex model internals and human-understandable reasoning.

While XAI research in supervised learning has matured, explainability for unsupervised methods—particularly topic modeling—remains underexplored (Morichetta et al., 2019). Unsupervised approaches uncover latent structures rather than direct predictions, making the generation of faithful explanations more difficult. Incorporating XAI into topic modeling could enhance transparency and interpretability in applications such as sentiment analysis or opinion mining (Laato et al., 2022).

Despite these advances, most prior studies evaluate models either for predictive performance or for explanation quality, but rarely integrate both dimensions. Our framework addresses this gap by combining topic modeling, sentiment and emotion classification, and XAI to enable multidimensional and interpretable opinion analysis. The proposed BERTopicXAI extends BERTopic with interpretable word clustering, uncovering latent discussion themes. In contrast to works where LLMs act as classifiers or direct translators of XAI outputs (Krugmann & Hartmann, 2024; Martens et al., 2025), our approach employs the LLM as a post-hoc synthesizer, integrating outputs from multiple specialized models into a single coherent narrative. This design bridges high-performing LLMs and explainable analytics, providing both quantitative robustness and human-readable interpretability across domains.

3 Proposed Methodology

This section presents our methodology for explainable multidimensional classification of user opinions across diverse domains, including product feedback, service evaluations, social media discussions, and public sentiment analysis in forums. Our goal is to classify and interpret user opinions accurately while providing clear explanations that illustrate the reasoning behind classification decisions.

The methodology consists of three main phases: (i) representing user opinions across multiple latent space dimensions and classifying them; (ii) interpreting classification results by applying explainable AI techniques; and (iii) generating human-readable explanations with generative AI. The complete execution flow of this methodology is illustrated in Fig. 1.

Fig. 1

Execution flow of the proposed three-step methodology

The first phase involves encoding user opinions into high-dimensional latent spaces, enabling comprehensive classification across multiple dimensions. Given the diversity of user-generated content, an opinion may express various attributes—such as sentiment (positive, neutral, or negative), discussed topics, emotional tone, or domain-specific aspects (e.g., political stance or mental health indicators). To effectively represent these attributes, pre-trained LLMs, such as BERT, RoBERTa, and ALBERT, are fine-tuned for multidimensional opinion classification. These models generate embeddings capturing contextual semantics, allowing separate classifiers to analyze different dimensions such as sentiment, topic, and emotion. The effectiveness of classification is validated through benchmarking against labeled datasets, ensuring reliable predictions across contexts.

The second phase enhances explainability by identifying influential elements within opinions that drive classification decisions. Given the complexity of LLM-based classifiers, as an XAI technique LIME is used to compute token-level importance scores and feature attributions. To improve topic-level transparency, we introduce BERTopicXAI, which enables interpretable token-to-topic mappings in unsupervised settings. Moreover, multidimensional analysis—across sentiment, emotion, and topical relevance—offers a layered understanding of each opinion, enriching transparency, interpretability, and practical usability.

The final phase synthesizes clear, human-readable explanations of classification decisions. While numerical and visual XAI outputs provide technical insights, generative AI (e.g., GPT) translates these structured outputs into coherent natural language narratives. By integrating sentiment classification, topical analysis, and explainability insights, GPT generates context-aware explanations that articulate both the reasoning behind the classifications and the key contributing factors. This approach enhances accessibility and trust, providing stakeholders with meaningful justifications alongside classification results.

In the following subsections, we provide detailed descriptions of each methodological step.

3.1 Multi-Dimensional Opinion Classification Using Fine-Tuned BERT Models

In this section, we present our multi-dimensional opinion classification strategy, which leverages fine-tuned BERT models to enhance the extraction of contextual information from user-generated content. The process begins by encoding diverse inputs—such as reviews, mood posts, and support messages—into high-dimensional latent spaces. These embeddings, derived from one or more BERT-based models, capture the semantic richness of each opinion. Dedicated classifiers then operate on these representations to assign labels across multiple dimensions, ensuring a comprehensive understanding of the expressed content.

In our case studies, we consider three core dimensions (sentiment, emotion, topic) common to all datasets, along with additional context-specific dimensions designed to capture domain-dependent aspects of opinion expression:

• Sentiment: Distinguishing between positive and negative opinions.

• Emotion: Identifying emotional states, including commonly used expressions found on social media.

• Topic: Dynamically derived themes that reflect the primary subject matter.

• Context-specific dimensions: Additional dimensions defined for specific datasets to address domain-related nuances. For example, a dataset focused on mental health may include a depression indicator, while one centered on political discourse could introduce a candidate support dimension to capture partisan alignment.

Once all dimensions are defined for a given use case, a dedicated classifier is employed for each dimension to ensure consistent and reliable predictions. To identify the most effective models, we conducted a thorough evaluation aligned with previous research (Cantini et al., 2024; Zhang & Shafiq, 2024). We focused on BERT-based architectures to ensure comparability with prior studies, achieve efficient inference, and ensure stable inference—since, for a fixed set of model weights, predicted outputs (and thus evaluation metrics) are deterministic. We also experimented with prompt-based Generative AI models to assess their potential for classification through natural language instructions; however, their outputs proved less consistent due to the inherent stochasticity of generative inference and exhibited higher latency during output generation.

In particular, for the sentiment dimension, we compared several BERT variants—including BERT, DistilBERT, ALBERT, and RoBERTa—using an annotated review dataset.¹ Through fine-tuning, we aimed to enhance each model’s capacity to capture subtle sentiment cues, enabling more precise interpretation and classification of nuanced expressions in user reviews. Evaluation on a held-out test set provided a consistent basis for comparing accuracy and assessing the models’ robustness in detecting sentiment across diverse user-generated content.

For emotion classification, we employed a classifier fine-tuned on the SemEval-2018 English dataset (Mohammad et al., 2018),² which includes 11 emotion labels: Anger, Anticipation, Disgust, Fear, Joy, Love, Optimism, Pessimism, Sadness, Surprise, and Trust. These represent an extension of Plutchik’s classification (Plutchik, 1980), adapted to capture emotions commonly expressed on social platforms. The classifier supports multi-label prediction, recognizing that a single expression may convey multiple emotions simultaneously. This approach has been widely adopted in recent studies (Ameer et al., 2023; Jabreel & Moreno, 2019) to capture emotional nuance in user opinions. The use of the SemEval-2018 fine-tuned model was motivated by its broad coverage of emotional categories and its established role in prior research, ensuring both granularity and comparability while maintaining computational efficiency.

For topic classification, we adopted the BERTopic framework (Grootendorst, 2022), which leverages transformer-based embeddings (e.g., BERT, RoBERTa) to capture semantic relationships among documents. Unlike sentiment or emotion classification, which directly predict labels for individual texts, BERTopic first identifies latent themes by clustering semantically similar documents and then assigns topic labels to new inputs based on their proximity to these clusters. We selected this method for its strong performance in maintaining topic coherence and diversity. Previous studies have shown that BERTopic outperforms statistical models by producing more clearly delineated and independent topics (Egger & Yu, 2022; Gan et al., 2024). Our choice was further motivated by its modular design and compatibility with our explainability extension (i.e., BERTopicXAI, introduced in Sect. 3.3), enabling the framework to deliver both robust thematic structure and transparent topic assignments—a key requirement of our system.

Table 1 presents representative examples illustrating the outcomes of our multi-dimensional classification, where positive opinions generally exhibit affirming language, emotional positivity, and clear thematic focus, whereas negative opinions often display critical language, pessimistic sentiment, and negative emotional cues. Details about the training and evaluation process, including model architectures, configuration parameters, model selection, and their application to the datasets under analysis, are reported in Sect. 4.2. Further details about the context-specific dimension(s), including their definition and implementation, are also provided in the same section.

Table 1 Sample opinions showcasing sentiment analysis, identified topics, and expressed emotions

3.2 Interpreting Classification Outcomes Using Explainable AI

In this section, we focus on explaining the classification outcomes produced by our models. Explainability is essential for validating predictions and ensuring transparency in automated opinion analysis. To achieve this, we employ a range of explainability techniques.

One approach leverages the attention mechanisms inherent in LLMs, which highlight the tokens or phrases that most strongly influence classification decisions. In addition, post-hoc methods such as Integrated Gradients (IG) and LIME (Local Interpretable Model-Agnostic Explanations) provide instance-level insights into model behavior. LIME is particularly effective for local explainability and has demonstrated strong performance in previous studies (Ribeiro et al., 2016).

In the examples and analyses presented in this paper, LIME is employed to generate word-level explanations across multiple dimensions, including sentiment, emotion, political stance, and mental health indicators. For instance, Table 2 illustrates a sentiment classification case where positively contributing words are highlighted in shades of green, while negative ones appear in red. Terms like “nice,” “helpful,” and “professional” characterize positive sentiment, whereas “lost,” “needs,” and “usage” are associated with negative sentiment. These visualizations provide a clear and intuitive view of the textual features influencing each classification decision.

LIME calculates attribution scores relative to the predicted class, and the highlighted intensity in our visualizations (see Table 2) directly corresponds to the magnitude of these scores (Ribeiro et al., 2016; Sundararajan et al., 2017). Prior work has empirically evaluated these explanation techniques. For instance, user-centered studies have shown alignment with human reasoning for both methods (Lottridge & Woolf, 2023), while other work has introduced performance metrics to quantitatively assess explanation accuracy against ground-truth feature importance, demonstrating that LIME can meaningfully capture model reasoning (Nguyen, 2022). Taken together, these findings indicate that LIME has been validated through both user studies and quantitative analyses, supporting our qualitative use of this method in this study. Additionally, to validate the explanations generated by LIME, all visual highlights presented in this paper were subjected to a systematic human inspection by the authors. Each highlighted token was manually assessed for both its semantic relevance within the text and its coherence with the model’s predicted class. This verification step ensured that the presented attributions were meaningful and faithful to the model’s behavior in our use cases.

For the topic dimension, we adopt a different strategy using BERTopicXAI, an interpretable extension of the BERTopic algorithm proposed in this work. Since topic modeling requires aggregating multiple opinions—such as reviews, political posts, or mood-related updates—to identify dominant themes, BERTopicXAI highlights the main tokens defining each topic. It enhances the explainability of BERTopic by providing insights into the internal dynamics of topic assignment, helping explain why a particular opinion is associated with a specific theme (see next subsection for further details). Together, these explainability techniques ensure that classification outcomes are interpretable across all dimensions, thereby improving both the transparency and robustness of our analysis.

Table 2 LIME-based sentiment interpretation: positive and negative words shown in green and red, with shade intensity indicating contribution

3.3 BERTopicXAI

BERTopicXAI,³ introduced in this paper, enhances the interpretability of topic modeling by generating visual explanations of documents and their associated topics. This is especially valuable when analyzing social media posts, where multiple overlapping concepts often appear (Egger & Yu, 2022; Jaradat & Matskin, 2019). By associating each document with multiple relevant topics, BERTopicXAI enriches explainability, improving the organization and comparison of textual content. It differs from instance-based methods like LIME by offering topic-level contextual insights.

BERTopicXAI follows a three-step process: (i) identifying top topics, (ii) extracting c-TF-IDF scores, and (iii) generating visual representations. In the first step, BERTopic computes topic probabilities for a given document and ranks them in descending order, excluding the outlier topic due to its lack of semantic structure. Topic selection is controlled through three strategies: a percentile cutoff [e.g., top 90% (Gayoso-Diz et al., 2013)], a fixed probability threshold (e.g., 0.20), or manual selection of a predefined number of top topics. These strategies ensure that only the most relevant topics are retained, enhancing interpretability and reducing classification noise.

In the second step, BERTopicXAI retrieves the c-TF-IDF scores for the input document. This metric adapts traditional TF-IDF to topic modeling by treating each cluster as a single document and adjusting the inverse document frequency accordingly. It quantifies the importance of each term within a topic, calculated as Grootendorst (2022):

$$\begin{aligned} \text {c-TF-IDF}_{t,c} = tf_{t,c} \cdot \log \left( 1 + \frac{A}{tf_t}\right) \end{aligned}$$

(1)

where $tf_{t,c}$ is the frequency of term t in class c which is the collection of documents concatenated for each topic. This value is multiplied with inverse class frequency calculated by A, the average number of words per topic, divided by $tf_{t}$, the frequency of term t across all topic clusters. This formula ensures positive values and highlights words that are both frequent within a topic and rare across others.

Finally, in the third step, visual explanations are generated. Each top topic is assigned a unique color, and words in the input text are highlighted accordingly. The highlight opacity reflects the normalized c-TF-IDF score, making highly representative terms more prominent. The result is a color-coded visualization that intuitively shows how words contribute to topic assignments.

Fig. 2

Overview of the BERTopicXAI framework and its three main steps for generating interpretable topic-based explanations

Figure 2 illustrates an example of this process. Such visualizations help readers grasp how documents relate to different topics and facilitate comparison across texts. For instance, a hotel review may simultaneously address parking and location, making it relevant to multiple topics. Additionally, BERTopicXAI offers an option to display topic probability distributions and word scores for top topics providing a clearer view of each topic’s influence within the text. The source code and experimental setup are publicly available at https://github.com/SCAlabUnical/BERTopicXAI.

It is important to emphasize that BERTopicXAI is not designed as a new topic modeling algorithm, but as an interpretability-oriented extension of the standard BERTopic approach. Its novelty lies in enhancing transparency and controllability of topic assignments rather than altering the underlying clustering process. Specifically, compared to the base method, BERTopicXAI introduces: (i) configurable probability thresholds and percentile cut-offs to filter weakly associated words or topics, (ii) token-level visual encoding where opacity reflects normalized c-TF-IDF scores, and (iii) tabular export of topic–word probability distributions to support expert validation and reproducibility. These extensions make BERTopicXAI a practical module for interpretable opinion analysis and ensure its seamless integration with the multi-dimensional classification and GPT-based explanation components of our framework.

3.4 Generating Human-Readable Explanations Using Generative AI

In this section, we outline our methodology for generating clear and comprehensive explanations of opinion classification outcomes across multiple dimensions using GPT. Specifically, we employ GPT-4o via the OpenAI API, configured with a temperature of 0 to encourage deterministic behavior, improving output consistency and reducing creative variation (Davis et al., 2024). Our approach is designed not only to explain sentiment analysis results but also to interpret additional aspects of opinions—such as emotional tone, political support, and other context-specific dimensions.

The GPT-adv prompt provides a structured input format for GPT-4o to generate human-readable explanations aligned with the model’s classification decisions.

This prompt currently operates across three dimensions—Sentiment, Topics, and Emotions—but can be extended to additional dimensions as needed. It requires GPT-4o to produce a concise explanation (e.g., 100 words) that justifies the classifications for a given opinion $x, using only the provided word-importance pairs ($si, $ti, and $ei). The explanation must rely exclusively on this input, without introducing unsupported information or assumptions.

We compare GPT-adv with a baseline configuration, referred to as GPT-base:

The basic prompt reflects the standard approach used in chat interfaces, where GPT classifies the opinion based solely on its content and the prompt request, without access to explicit classification outputs.

Although the results presented in this paper are based on GPT-4o, our framework is model-agnostic, allowing seamless integration of future LLMs as the technology evolves. GPT-4o was selected to ensure reproducibility and computational feasibility, making the results both verifiable and efficient to obtain.

4 Experimental Results

Our study utilizes labeled opinion datasets drawn from diverse sources—including product reviews, discussion threads, and mood-related posts—to provide a comprehensive perspective on how users express opinions across different domains and contexts.

4.1 Datasets

The Booking dataset is essential for analyzing structured, service-related feedback, where users explicitly express satisfaction or dissatisfaction with specific aspects of their hotel experience. Sourced from Booking.com,⁴ this dataset contains over 515,000 customer reviews and ratings for 1,493 hotels across Europe, enriched with geographic metadata. These reviews provide insights into various facets of user satisfaction, such as service quality, comfort, cleanliness, and overall experience.

The Amazon dataset offers a different perspective, focusing on consumer trust, usability, and product performance. We analyze a curated set of Amazon product reviews,⁵ comprising approximately 10,000 entries from 100 electronics products. Each review includes structured fields such as star rating, title, review content, user verification status, reviewer location, and date. This structure supports in-depth analysis of sentiment trends and recurring themes in consumer feedback.

The Depression dataset addresses the emotionally nuanced and often implicit nature of opinions related to mental health. We include the Detecting Signs of Depression from Social Media Text dataset (Poświata & Perełkiewicz, 2022),⁶ which captures a wide spectrum of user expressions—from general mood updates to indicators of depression. It includes 12,136 posts, organized chronologically by user ID, text, and date. For consistency across datasets, the original labels (“moderate,” “severe,” and “not depressed”) were converted into a binary classification: depressed vs. not depressed. This binarization addresses potential data sparsity in the “moderate” and “severe” categories, yielding more balanced, statistically robust classes for model training, albeit at the cost of granularity.

The Political dataset captures opinion dynamics in politically sensitive contexts. Introduced in Belcastro et al. (2022) and available at,⁷ it focuses on public discourse related to the 2020 U.S. presidential election. Tweets are labeled as pro-Biden or pro-Trump, allowing for the analysis of partisan opinion expression. The dataset includes tweets posted between September 1 and October 31, 2020, by users in 11 key swing states. For our study, we extracted a balanced subset of 77,188 tweets from Florida.

By integrating these four datasets—Booking, Amazon, Depression, and Political—our system is evaluated on multidimensional opinion analysis across varied domains. This setup enables the generation of fine-grained explanations that encompass sentiment, topic, emotion, and context-specific features, supporting a robust and transparent assessment of the proposed methodology.

4.2 Application of Multidimensional Classifiers: Sentiment, Emotion, Context-Specific and Topic

This section explores the use of BERT-based architectures for three key tasks in opinion data analysis: sentiment classification, topic modeling, and emotion recognition. BERT and its variants have been widely adopted for natural language understanding (Devlin et al., 2018), and we selected them as the backbone of our framework due to their proven effectiveness and reproducibility in comparable studies (Cantini et al., 2024; Zhang & Shafiq, 2024). To assess performance stability and ensure reproducibility, we repeat each classifier experiment over $R=5$ independent runs with different seeds while keeping all hyperparameters fixed. Table 3 reports Accuracy, Precision, Recall, and F1 as mean and standard deviation across runs. The corresponding seed values are provided in “LLM Configuration for Reproducibility” section in Appendix 7 to enable full reproducibility. The following sections describe the dataset configurations, experimental setup, and comparative results for each classification dimension in our multidimensional framework.

Experimental Setup and Evaluation Protocol. All experiments were conducted using a Google Colab environment equipped with an NVIDIA T4 GPU. All transformer-based classifiers were implemented using the PyTorch and Transformers libraries. Models were fine-tuned with a maximum sequence length of 128 tokens, a batch size of 4, the AdamW optimizer (learning rate 2e-5), and trained for 10 epochs. Unless otherwise specified, the data were split into 80% for training, 10% for validation, and 10% for testing. For all classification tasks, we adopted a unified evaluation pipeline: each model was trained independently on the designated training split and monitored on the validation split for early stopping and model selection, with final results computed on the test set. Standard metrics were applied depending on the task type—Accuracy, Precision, Recall, and F1-score for binary and multi-class tasks, and micro-averaged variants for multi-label prediction. This configuration ensures fair comparability and consistent evaluation across sentiment, emotion, and context-specific classifiers.

Table 3 Classification performance on sentiment, emotion, and context-specific tasks, reported as mean and standard deviation over $R=5$ runs with different seeds

Sentiment Classification. For sentiment polarity detection, we fine-tuned and tested multiple transformer models on the Booking.com⁸ dataset, sampled to 50,000 reviews and annotated with binary labels (positive vs. negative). The annotations were considered as reference ground truth, consistent with standard evaluation practices in sentiment analysis. After fine-tuning, inference was performed across all four opinion datasets described previously (Booking, Amazon, Political, and Depression), allowing us to assess generalization performance across domains. Table 3 (first block) reports the averaged results for the sentiment task. BERT achieves the highest performance across all metrics (Accuracy = 0.947, F1 = 0.944), confirming its strong suitability for binary opinion polarity detection. Despite slight variations, all models show stable behavior, with DistilBERT and ALBERT reaching competitive scores, while BERTweet performs moderately lower, as expected, due to its primary specialization in informal microtext rather than product or booking reviews.

Emotion Classification. In contrast to sentiment polarity detection, emotion recognition was modeled as a multi-label classification task, where each instance could express multiple affective states simultaneously. We employed the SemEval-2018 dataset (Mohammad et al., 2018), which provides eleven emotion categories such as joy, sadness, anger, and optimism, with thousands of annotated instances used for training and evaluation. The annotations originate from publicly validated benchmark resources and are widely adopted in prior studies (Ameer et al., 2023; Andreou et al., 2023; Chochlakis et al., 2023; Haryono et al., 2023; Ririanti & Purwinarko, 2021), ensuring comparability with previous work. Results in Table 3 (second block) highlight a natural performance decrease compared to sentiment classification, as the task involves multi-label prediction and greater conceptual overlap among affective categories. Nevertheless, BERT achieves the highest overall F1-score (0.689), outperforming RoBERTa and clearly surpassing the other variants. BERTweet, while optimized for social media, performs slightly worse due to its sensitivity to noisy emotional expressions and the inherent sparsity of emotion labels. Overall, these findings confirm that transformer-based models—particularly BERT—remain robust even when dealing with subtle affective nuances.

Context-Specific Classification: Mental Health (Depression) and Candidate Support (Political). To assess the adaptability of transformer-based classifiers in specialized domains, we extended our analysis to the Depression and Political datasets previously described in Sect. 4.1. Unlike the general-purpose sentiment and emotion tasks, these datasets feature context-specific target classes corresponding to distinct contextual phenomena: mental health status (Depressed vs. Not Depressed) in social media posts and candidate support (Trump vs. Biden) in political opinions. Each model was applied to its corresponding dataset, and the results are reported in Table 3 (last two blocks). In the Depression classification task, RoBERTa achieves the highest F1-score (0.898), confirming its robustness in emotionally loaded or vulnerable language. A similar pattern emerges in the Political setting, where RoBERTa again leads (F1 = 0.903), closely followed by BERT and BERTweet, while DistilBERT and ALBERT perform satisfactorily but are slightly affected by class imbalance, leading to small drops in Recall. It is worth noting that BERT remains competitive, whereas BERTweet performs particularly well on texts containing colloquial mental health expressions, reflecting the advantage of stylistic alignment with its pre-training domain.

Class Distribution and Task Difficulty. Table 4 summarizes the class distributions across the four datasets, covering the main analytical dimensions addressed in this study: sentiment, emotion, and the context-specific categories defined for each context—candidate support in the political domain and mental health status in the depression dataset. These characteristics provide essential insights into the balance and variability of each dataset, which directly affect model performance. The remaining dimension, topic, characterized by its unsupervised nature and interpretive role, will be discussed separately in the next section. For instance, the Booking dataset shows a relatively balanced sentiment distribution (55.18% Positive vs. 44.82% Negative), whereas both Depression and Political exhibit strong skewness toward Negative sentiment and specific emotion categories (Anger and Sadness respectively). Such imbalances justify the adoption of micro-averaged metrics and explain why context-specific models tend to show lower Recall in minority classes.

Table 4 Class distribution across the four datasets (Booking, Amazon, Depression, Political) along the dimensions of sentiment (positive/negative), emotion (main four classes), and context-specific—mental health (depression) and candidate support (political)

Topic Modeling and Human Validation. To complement classifier-based predictions with an unsupervised exploration of semantic structure, we applied BERTopic v0.17.0 (Grootendorst, 2022) to each dataset. Documents were first embedded using the 768-dimensional paraphrase-MPNet-base-v2 model which has shown strong performance in clustering tasks (Lis & Kowalski, 2023; Nasiri Lahmer et al., 2024; Zhou et al., 2025), then projected into a lower-dimensional space through UMAP and clustered using HDBSCAN.

To optimize topic extraction, we systematically tuned key hyperparameters. We focused on UMAP’s n_neighbors for balancing local vs. global structure and HDBSCAN’s min_cluster_size for cluster granularity. Using the Booking dataset, we explored n_neighbors $\in \{10,15,30\}$ and min_cluster_size $\in \{50,75,100,125,150\}$. Each configuration was assessed quantitatively using semantic coherence, topic relevance, and qualitative researcher judgment (Grootendorst, 2022). Following an evaluation of all candidate configurations, the optimal setting of n_neighbors = 15 and min_cluster_size = 150, was selected for the corpus, as this configuration yielded the most semantically coherent and interpretable topics, a selection methodology consistent with prior studies (Cantini et al., 2024; Cosentino et al., 2025). The same tuning protocol was applied to the other three datasets, yielding dataset-specific optima. To support reproducibility, we fixed the random seed and excluded the outlier topic from downstream analyses to enhance clarity and overall topic quality.

Each topic was automatically labeled using ChatGPT, which generated concise, human-readable names by summarizing the top representative keywords from each cluster. To assess the semantic coherence of the extracted topics, we conducted a human validation study with three independent annotators experienced in NLP and social media analysis. The study included two complementary tasks (Chang et al., 2009; Lau et al., 2014):

• Word intrusion: annotators identified an intruder term among the topic keywords.

• Topic labeling: annotators reviewed the top keywords and three representative documents for a topic and proposed an appropriate label.

We conducted 20 word-intrusion trials and 20 topic-labeling trials, each evaluated by all three annotators. Topics were randomly sampled from the four datasets, and for each topic we provided the top keywords and three representative documents. For word intrusion, the task achieved an average accuracy of 81.7%. Inter-annotator agreement, measured using Fleiss’ $\kappa$, was $\kappa =0.70$, indicating substantial agreement according to the Landis–Koch scale (Landis & Koch, 1977; Moons & Vandervieren, 2025). This result supports the interpretability of the extracted topics despite the inherent subjectivity of semantic judgments. For topic labeling, we assessed consistency in two ways. First, a qualitative check showed that for 18 out of the 20 sampled topics (90%), annotators produced labels with a clearly shared meaning. Second, to obtain a formal agreement measure, we mapped the free-text labels to semantic categories based on their core meaning (e.g., “Charging Problem,” “Battery issues,” and “Poor charging quality” in “Charging Complaints”) and computed Fleiss’ $\kappa$ on the resulting categorical annotations. This analysis yielded $\kappa =0.86$, indicating almost perfect agreement (Landis & Koch, 1977). Overall, these findings suggest that the topics are meaningful and consistently interpretable by human readers.

Figure 3 provides an overview of the topic structure across the Booking, Amazon, Depression, and Political datasets by projecting BERTopic clusters into a two-dimensional UMAP space. Each cluster is shown with a distinct color and an associated label, while star markers indicate the sample opinions used in the qualitative case studies (see Sect. 4.4).

Fig. 3

Two-dimensional UMAP projection of BERTopic clusters computed on the Booking, Amazon, Depression, and Political datasets. Each color represents a distinct topic, with cluster names automatically derived from representative keywords. Star-shaped markers indicate sample opinions used as qualitative examples

For the selection of top topics within our BERTopicXAI analysis, we applied a 90th-percentile cutoff to the per-document topic probabilities. This filtering systematically excludes minor clusters and concentrates interpretation on the most dominant topics for each document.

4.3 Explainable Method: LIME Application

To generate local explanations, LIME perturbs the input text by randomly masking words and then fits a simple interpretable model to approximate the predictions of the original classifier (Ribeiro et al., 2016). The quality of these explanations depends on several key parameters. The number of perturbations controls how many altered samples are generated, with larger values generally leading to more stable surrogate models. The kernel width determines how perturbed samples are weighted relative to their similarity with the original text, thereby shaping the locality of the explanation (Duong et al., 2023). The feature selection strategy determines which input tokens are included in the created explanation (Knab et al., 2025).

In our experiments, we adopted the default kernel width of 25 for the LimeTextExplainer. The kernel width parameter controls how perturbed text instances are weighted based on their similarity to the original input, thereby shaping the locality of the surrogate model. A value of 25 was proposed in the original LIME framework as a robust setting that balances locality and stability in text classification tasks (Ribeiro et al., 2016). Specifically, this value ensures that perturbations moderately distant from the input still contribute to the explanation, while maintaining sufficient emphasis on local behavior. Consistent with prior studies (Amparore et al., 2021), we retain this default to provide faithful and comparable explanations without introducing additional hyperparameter bias. We employed the adaptive auto strategy for feature selection, allowing the library to automatically choose the most suitable method for our desired explanation size.

To assess the stability of our LIME explanations, we conducted a systematic analysis of the top influential tokens while varying the num_samples parameter. Following Visani et al. (2022), we leveraged two metrics: Variables Stability Index (VSI), measuring whether the same features are consistently selected across repeated runs, and Coefficients Stability Index (CSI), measuring the consistency of word attribution scores across repeated explanations. Additionally, we measured feature overlap using Jaccard similarity and report the Stability Ratio, i.e., the fraction of top-k features that appear in all runs. We generated multiple independent explanations for the test documents across different num_samples values and observed top-10 influential words. Table 5 summarizes the stability of LIME explanations for different values of num_samples. Using num_samples = 300 yields the highest overall stability (VSI = 89.33%, CSI = 70.90%) (Visani et al., 2022). At this setting, the mean Jaccard similarity is 0.807, indicating substantial overlap in the selected features across runs, and the Stability Ratio shows that 76% of top features appear in all runs. Increasing num_samples from 100 to 300 leads to clear improvements, while moving from 300 to 500 provides only marginal changes. For this reason, we set num_samples = 300 in all experiments. Detailed per-document results are reported in “Per-Document Stability Analysis” section in Appendix 7.

Table 5 LIME stability metrics across different num_samples values

4.4 Comparative Case Studies: Input Examples and Generated Explanations

This section presents a comparative evaluation of our proposed methodology against a baseline approach. The objective is to assess how providing GPT with additional structured inputs—namely, classifications and interpretability data across multiple dimensions—affects the quality and clarity of the generated explanations. Specifically, we compare the outputs of our enhanced model (GPT-adv), which leverages fine-grained interpretability data, with a baseline model (GPT-base) that relies solely on the raw opinion text. For details on prompt structure and configuration, refer to Sect. 3.4.

We begin with an instance from the Booking dataset, where a user shares a positive review of a hotel stay. This opinion is annotated across the considered dimensions: it carries a positive sentiment, addresses themes such as “Celebratory Sweet Treats” and “Hospitality Excellence,” and conveys emotions like “Joy” and “Love.” Token-level interpretations are assigned scores and visualized using color gradients—e.g., green for positive sentiment—to support interpretability. Figure 3a shows the UMAP projection of topics from the Booking dataset, with the star indicating the reviewselected for detailed analysis.

The explanations generated by both approaches are shown below (in these examples, N—the target explanation length—was set to approximately 100 words):

While GPT-base effectively summarizes the overall satisfaction conveyed in the review, it does not explicitly link specific lexical elements to the underlying classification dimensions. In contrast, GPT-adv highlights key tokens—such as “lovely,” “nice,” and “comfortable”—and maps them to sentiment and emotion labels, while also identifying relevant topics like “Hospitality Excellence.” This structured alignment enhances interpretability and provides a more transparent rationale for the model’s decisions.

The next case, drawn from the Amazon dataset, features a critical review of a toothbrush. It is labeled with negative sentiment, associated with topics such as “Toothbrush Usage” and “Usage Issues,” and reflects the emotion “Anger.” Figure 3b shows the topic distribution of Amazon reviews in a UMAP-compressed 2D space, with a star indicating the review analyzed in this example.

Below are the corresponding generated explanations:

Here, GPT-base captures the overall dissatisfaction by summarizing negative phrases such as “too long” and “slides down near the top,” but it remains surface-level and lacks structured interpretation. In contrast, GPT-adv not only identifies these phrases but explicitly associates them with specific dimensions—highlighting negative sentiment, usability-related topics such as “Toothbrush Usage” and “Usage Issues”, including aspects like “grip” and “brushing”, and the emotion of anger. This layered explanation provides a clearer understanding of how the opinion was classified, offering both greater interpretability and analytical depth.

We now examine a case from the Depression dataset, where the user expresses intense psychological distress. The opinion is classified as having negative sentiment, is associated with a depressive condition, and is linked to topics such as “Emotional Struggles” and “Sleep Patterns.” The emotions “Sadness” and “Anger” are prominently identified, capturing the severity and emotional weight of the user’s expression. Figure 3c visualizes the topic distribution within the dataset, with the star indicating the location of the selected post analyzed in this example.

Below are the generated explanations:

In this highly sensitive example, GPT-adv goes beyond capturing emotional tone—it provides a structured and clinically meaningful interpretation by mapping key phrases to distinct classification dimensions: sentiment, mental state, topics, and emotions. Unlike GPT-base, which summarizes emotional distress, GPT-adv explicitly associates terms like “dying,” “problems,” and “sleeping pills” with the diagnosed depressive condition and emotional states such as sadness and anger. The inclusion of the Mental State dimension is particularly valuable, as it enables a more precise and empathetic understanding of the user’s psychological state—an essential requirement in mental health analysis. This multidimensional mapping not only enhances interpretability but also improves the potential for responsible and context-aware applications.

Finally, turning to political discourse, we consider an opinion from the Political dataset. This post expresses a positive sentiment, supports Trump, and focuses on themes such as “Health and Caregiving” and “Voting.” The dominant emotion identified is “Optimism,” reflecting a hopeful and affirmative tone. Figure 3dillustrates the topic distribution within the dataset, with a star indicating the specific post analyzed in this example.

The base and advanced explanations are compared as follows:

In this example, GPT-adv delivers a more structured and informative explanation by explicitly linking key linguistic elements to multiple dimensions, including sentiment, topic, emotion, and political stance. While GPT-base recognizes the overall positive tone, GPT-adv clarifies the direction of political alignment through references to Trump, “cast my vote,” and campaign-related language. The inclusion of the Candidate Support dimension is particularly valuable in distinguishing not just positivity, but partisan intent. Moreover, the integration of emotionally charged terms (e.g., “proud,” “trail,” “brutality”) and topical markers (e.g., “abortion,” “RNCConvention”) enriches the interpretability, yielding a more nuanced and multidimensional understanding of political discourse.

In summary, the presented examples demonstrate how incorporating structured inputs—comprising multidimensional classifications and token-level interpretability signals—enables GPT-adv to produce structured, informative, and context-aware explanations. While GPT-base offers general summaries of user opinions, GPT-adv maps key expressions to explicit dimensions such as sentiment, topic, emotion, and domain-specific labels (e.g., mental state or political support). This structured alignment between input and output enhances interpretability, offering deeper insight into model decisions and supporting more transparent, nuanced, and actionable opinion analysis across diverse domains.

4.5 Comprehensive Evaluation: Comparative Analysis and Ablation Study

In this section, we compare the responses generated by the different prompt configurations across all selected opinions. To further illustrate the system’s interpretability, we curated a set of 100 opinion samples per dataset, ensuring a balanced and varied distribution across classification dimensions. For the ablation study, we introduce an intermediate prompt that serves as a midpoint between the base and advanced configurations. This version provides GPT with the classification outputs for each dimension (e.g., sentiment, topic, emotion), but excludes word-level importance scores. While we do not include examples generated by this intermediate prompt in the case studies due to space constraints, its outputs are included in the comparative evaluation.

We assess the quality of the generated explanations using three complementary approaches:

• 1. Textual and Semantic Metric Analysis: This evaluation involves computing a set of established metrics from textual analysis (Hansen, 2023), including semantic similarity scores such as Dice, TF-IDF, ROUGE-L, and S-BERT. While these metrics are traditionally used in summarization tasks, we adapt them here to assess the informativeness and relevance of the generated explanations relative to the original opinions.

• 2. GPT-Based Evaluation: In this setting, GPT itself is used as an evaluator. It analyzes each explanation across key dimensions, including the accuracy of detail coverage—ensuring that the main elements of the original opinion are preserved—and semantic depth, assessing how well the explanation captures implicit meanings and emotional tone.

• 3. Domain Expert Evaluation: Finally, we incorporate assessments from domain experts who independently evaluate the explanations and collectively identify the most informative and accurate one based on their professional judgment.

4.5.1 Textual and Semantic Metric Analysis

The analysis of textual and semantic metrics is essential for evaluating the quality of the generated explanations. Table 6 presents a summary of the scores obtained across four opinion datasets, assessing both the linguistic quality and the semantic alignment of each explanation with its corresponding original opinion.

To assess textual quality, we computed the following four metrics:

• Readability (Coleman-Liau Index): Estimates the education grade level required to comprehend the generated explanation, with higher values indicating greater syntactic and lexical complexity. Coleman-Liau Index is used as indicator of linguistic complexity as higher values corresponds to a more complex text.

• Quality: Measures redundancy by analyzing repeated textual patterns (duplicate n-grams), where lower scores indicate more concise and less repetitive outputs.

• Coherence: Assesses the logical flow and sentence connectivity using cosine similarity between adjacent sentences, with embeddings computed via Latent Semantic Analysis (LSA).

• Complexity: Evaluates lexical and structural diversity through text entropy, where higher scores reflect more varied and information-rich language.

To evaluate semantic similarity between the original opinion and the generated explanation, we considered the following metrics:

• Dice Similarity: Measures the lexical overlap of content words (excluding stopwords) between the opinion and the explanation, reflecting surface-level similarity.

• TF-IDF Similarity: Computes cosine similarity between TF-IDF vectors of the opinion and explanation, capturing similarity in word usage and term relevance.

• ROUGE-L: Evaluates the longest common subsequence between the opinion and explanation, widely used to assess content preservation in summarization tasks.

• S-BERT Similarity: Uses cosine similarity between Sentence-BERT embeddings to assess semantic closeness, capturing deeper contextual alignment between texts.

Table 6 Evaluation scores based on textual and semantic similarity metrics between original opinions and generated explanations across multiple opinion datasets

The trends in Table 6 quantitatively confirm the interpretability gains highlighted in the examples discussed in Sect. 4.4. While GPT-base produces general summaries, it lacks explicit connections between lexical elements and classification dimensions. In contrast, GPT-adv consistently yields more structured and semantically grounded explanations, reflected in higher scores for readability, coherence, and complexity. These results align with the examples, where GPT-adv effectively maps key tokens to sentiment, emotion, and topic labels, offering clearer, more transparent rationales. Improvements in ROUGE-L and S-BERT further demonstrate GPT-adv’s ability to preserve both textual surface and underlying meaning. Despite added detail, redundancy remains low, confirming its conciseness. The intermediate model, GPT-interm, performs between the two, underscoring the added value of token-level interpretability in generating informative and aligned explanations.

We acknowledge that the textual and semantic metrics (e.g., ROUGE-L, Dice, S-BERT, Coleman–Liau) adopted in our evaluation act as indirect proxies for explanatory quality rather than direct measures of interpretability. Their role is to capture surface-level similarity and linguistic structure, which, while informative, may not fully reflect user-perceived utility. To mitigate this limitation, we triangulated three complementary perspectives: (i) automatic metrics, (ii) LLM-as-a-judge scoring, and (iii) human expert evaluation. This multi-perspective design ensures that explanation quality is not assessed through a single lens but validated across diverse and complementary criteria.

4.5.2 GPT Evaluation

The evaluation of the generated explanations was conducted using the LLM-as-a-Judge paradigm, a methodology increasingly adopted in recent literature (Dong et al., 2024; Zheng et al., 2023). This approach leverages LLMs to assess outputs according to predefined criteria, enabling a structured and consistent evaluation process. In our setup, the judgment model receives the original opinion, the classifications across multiple dimensions, and the corresponding explanation. Based on this input, the model evaluates the explanation’s consistency, informativeness, and structure relative to the original opinion.

The assessment is carried out across four dimensions, each rated on a scale from 1 (worst) to 10 (best):

• 1. Detail Coverage: Measures how thoroughly the explanation captures the key aspects of the opinion. For example, if a review criticizes both product durability and customer service, an explanation addressing only one of these points would receive a lower score.

• 2. Clarity & Structure: Assesses the organization and readability of the explanation. High-scoring explanations clearly separate components such as sentiment justification, topic discussion, and emotional tone, facilitating interpretability.

• 3. Mapping to Dimensions: Evaluates how effectively the explanation links specific phrases from the opinion to the predicted classification dimensions. A strong explanation explicitly references key words (e.g., using quotation marks) and correctly associates them with sentiment labels, emotions, or topics.

• 4. Overall Analytical Insight: Examines the depth and reasoning of the explanation. High-quality explanations go beyond restating the opinion by offering interpretative insights—such as contextualizing the sentiment or identifying implicit emotional cues.

The use of GPT-4o as an explanation generator is accompanied by three evaluation channels (textual/semantic metrics, LLM-as-a-Judge, and a human expert panel). The model’s judgment, therefore, is not the only measure, but it remains problematic because it can trigger a self-referential loop. GPT-4o was employed in two independent contexts, each serving a distinct role: (i) as a generator, constrained by structured prompts grounded in XAI-derived features (LIME attributions and BERTopic outputs), and (ii) as a judge, where it received only the opinion, predicted labels, and explanation—without access to the generation prompts or attribution scores. This separation minimizes the risk of self-referential evaluation. To deliberately limit the generator’s freedom and ensure its outputs were constrained, we designed a structured prompt that required it to stay within a specific word limit and to refrain from including prohibited information or its own deductions. To verify that the generator adhered to the prompt constraints, we conducted a systematic compliance check. This analysis confirmed that the explanations respected the word limit (mean: 101 words, SD: 5.4) and contained no prohibited information. To quantitatively verify that the generated explanations were grounded in XAI-derived features, we computed a token-overlap grounding metric on a sample of 400 instances (100 per dataset). For each instance, we constructed a source set of salient tokens ($T_{\text {XAI}}$) as the union of the top-10 tokens produced by each XAI module (LIME for sentiment/emotion and c-TF-IDF for topics), and a target set ($T_{\text {out}}$) as the tokens explicitly enclosed in quotation marks in the GPT-4o explanation, as required by our prompt. Both sets were normalized by lowercasing, lemmatizing, and removing punctuation and English stopwords. Grounding was then defined as the percentage of normalized tokens in $T_{\text {out}}$ that exactly match tokens in $T_{\text {XAI}}$. Averaged over all instances, the grounding score was 84%, indicating that the explicitly referenced tokens closely track high-attribution XAI terms. As a chance-level reference, we computed the expected overlap when randomly sampling tokens from each document (matching the size of $T_{\text {XAI}}$ and applying identical normalization), obtaining 15%. This demonstrates that the LLM is specifically prioritizing XAI-identified tokens rather than arbitrary contextually relevant terms from the input text. Full reproducibility details (exact model versions, temperature settings, and system prompts) are provided in “LLM Configuration for Reproducibility” in Appendix.

Table 7 presents the aggregated evaluation scores for the different prompting strategies. The results indicate that the advanced configuration consistently outperforms the baseline across all evaluation criteria. In particular, GPT-adv shows marked improvements in detail coverage and in establishing structured mappings between textual elements and classification dimensions. By explicitly referencing relevant words and systematically aligning them with the corresponding dimensions, GPT-adv enhances both the granularity and transparency of the explanation. This structured evaluation approach supports a more interpretable and comprehensive understanding of user opinions, confirming the effectiveness of GPT-adv in generating meaningful explanatory insights.

Table 7 Evaluation scores comparing GPT-base, GPT-interm, and GPT-adv outputs across multiple datasets and criteria

4.5.3 Domain Expert Evaluation

As part of the human evaluation, we involved 40 participants with backgrounds in natural language processing, data analysis, and related fields. The group consisted mainly of PhD students and researchers in artificial intelligence and medical informatics, ensuring advanced expertise in NLP methodologies and domain-specific knowledge. Each participant took part in an online study where they reviewed sets of explanations—comprising base, intermediate, and advanced outputs—for selected opinions drawn from four datasets: Booking, Amazon, Political, and Depression. For each dataset, five representative opinions were selected, and the three corresponding explanations were displayed in random order to minimize presentation bias. A summary of the human evaluation protocol is provided in Table 10 (“Details About Human Evaluation Protocol” section in Appendix 7).

Participants were asked to evaluate and select the best explanation for each opinion based on the following four criteria:

• (i) Which explanation captures all essential aspects of the opinion?

• (ii) Which explanation is clearer, more readable, and well-organized?

• (iii) Which explanation most effectively links specific words from the opinion to predefined classification dimensions (i.e., sentiment, topics, and emotions)?

• (iv) Which explanation provides the deepest analytical insight into the rationale behind the dimension-based evaluation?

To ensure transparency, all participants received written guidelines describing how to evaluate explanations according to the four criteria. We quantified inter-annotator agreement among the 40 participants using Fleiss’ $\kappa$, computed from their forced-choice selections for each instance. The overall agreement was $\kappa =0.75$ (ranging from 0.71 to 0.78 across datasets), indicating substantial agreement according to the Landis–Koch scale. We further assessed the consistency between human preferences and the LLM-as-a-judge scores by computing correlation coefficients. We observed strong alignment (Pearson’s $r=0.82$ and Spearman’s $\rho =0.79$), suggesting that the LLM-based evaluation is broadly consistent with human judgments and can serve as an auxiliary assessment signal.

Fig. 4

Participant preference scores for GPT-base, GPT-interm, and GPT-adv explanations across multiple datasets, expressed as average selection percentages

Figure 4 reports the distribution of preferences expressed by the participants across the four evaluation criteria. The results show that GPT-adv was consistently preferred over the other two approaches. This preference is likely due to its enriched content, which integrates multidimensional classifications together with representative tokens supporting those dimensions. As a result, GPT-adv produces explanations that are more informative and better organized, while also being more interpretable and analytically rich. Overall, the human evaluation indicates that GPT-adv provides deeper explanatory value than both GPT-base and GPT-interm.

This human evaluation should be regarded as a pilot-scale study aimed at validating the methodological soundness of our approach rather than supporting broad statistical generalizations. Accordingly, the number of evaluated instances is smaller than in the automated experiments, as human assessments prioritize qualitative depth, interpretive consistency, and expert judgment over scale.

5 Discussion and Limitations

The results of this work highlight the potential of integrating multidimensional classification, explainability techniques, and generative models into a unified pipeline, while also exposing key challenges in natural language interpretability. The superior performance of our GPT-adv model, which combines multidimensional classifications with XAI outputs to generate more informative and analytically rich explanations, aligns with the findings of Martens et al. (2025), who showed that LLM-generated narratives enhance user understanding over raw XAI outputs. Our framework advances this direction by demonstrating the LLM’s ability to synthesize multiple analytical dimensions—sentiment, topics, and emotions—into a coherent, human-readable explanation. Unlike studies such as Krugmann and Hartmann (2024), where the LLM acts as both classifier and explainer, our approach employs it as a post-hoc synthesizer that integrates outputs from specialized models. By contextualizing these dimensions and their representative tokens, our framework meets the need for deeper, more reasoned explanations (Qu et al., 2025), ultimately demonstrating the LLM’s value as a synthesizer of diverse analytical outputs into a single, interpretable narrative.

Our primary contribution is architectural rather than algorithmic: we introduce a unified, end-to-end framework that delivers multidimensional, evidence-grounded explanations of user opinions by tightly coupling specialized predictors (e.g., sentiment/emotion classifiers and BERTopic) with instance- and topic-level interpretability signals and a controlled generative layer. While the individual components (BERTopic, LIME, and LLMs) are established, prior work often produces explanations that are either faithful but fragmented (token attributions without a coherent narrative) or fluent but weakly grounded (free-form LLM rationales not explicitly constrained by model evidence). In contrast, our pipeline enforces dimension-aware explanation generation conditioned on explicit outputs and token-level rationales, improving traceability between predicted labels and the linguistic cues that support them. Within this architecture, BERTopicXAI further strengthens topic transparency by visualizing the tokens driving topic assignment and exposing multi-topic affiliations, yielding topic explanations that go beyond keyword lists or cluster labels. Overall, the novelty lies in providing a modular and extensible explainability pipeline that consistently connects what is predicted to why it is predicted across domains.

LIME, used to highlight the most influential tokens, proved effective in complementing classification outputs, although they remain sensitive to parameter choices. Consistent with prior work, this study employed a qualitative approach to interpreting influential tokens, which was further validated through structured human evaluation. However, we acknowledge that a more systematic sensitivity analysis is required to quantitatively assess the faithfulness of these attributions. We frame the rigorous validation of the explainability results as an important avenue for future research. Our reliance on LLMs for both explanation generation and, in one evaluation channel, for judgment introduces known challenges, including the risk of stylistic bias and the potential for a self-referential loop. While our human expert panel provides an essential external validation that mitigates this risk, we acknowledge that interpretability is not an objective property but an evolving, context-dependent construct. Therefore, we position our findings as a demonstration of a powerful framework whose outputs, while beneficial, warrant careful and context-aware evaluation.

Expert validation confirmed the added value of the advanced configuration, although the number of participants was limited to a group of PhD students in AI and data science. Future studies should involve larger and more diverse participant groups, include Likert scales or a “None of the above” option for more nuanced evaluations, and collect qualitative feedback to capture subjective impressions of explanation quality. Broader evaluations, including domain specialists, will be essential to consolidate these findings—particularly in sensitive contexts such as mental health. In this regard, the inclusion of the Depression dataset was intended to demonstrate the framework’s cross-domain flexibility rather than serve diagnostic purposes; any clinically reliable extension will necessarily require interdisciplinary collaboration. Furthermore, our framework’s sentiment and emotion classifiers primarily interpret the literal meaning of text, which limits their handling of sarcasm, irony, and other figurative expressions. This is particularly evident in Political dataset where sarcasm and irony are frequently used to express an implicit stance. For example, a sarcastic tweet using positive words like “brilliant” to convey criticism would likely be misclassified as positive. This initial error then leads to a misleading explanation, as the XAI would highlight “brilliant” as the reason for the incorrect prediction. This creates explanation noise and fundamentally misrepresenting the author’s true intent in this specific domain. Promising enhancements include integrating sarcasm detection modules or developing hybrid LLM-based components to pre-process and flag figurative language before classification. Visual inspection of latent topics also remains a valuable yet manual step; while our visualizations effectively clarified cluster structure, scalability and readability on larger datasets remain open challenges.

The study’s contribution lies not only in quantitative performance but also in qualitative gains such as structured outputs, explicit word-to-dimension mappings, and enhanced readability. This added interpretability is particularly important in sensitive domains like political analysis and mental health, where transparency and user trust are essential. The identified limitations should thus be viewed as natural steps in an evolving line of research that paves the way for broader validation, refined evaluation metrics, and interdisciplinary collaboration.

6 Conclusions

Within today’s digital landscape, user-generated opinions are central to understanding public sentiment, shaping business strategies, and informing policy decisions. This work presents a novel methodology for the multidimensional and explainable classification of user opinions by integrating advanced language models with interpretable AI techniques. Our approach leverages BERT-based classifiers fine-tuned on domain-specific datasets and GPT-based generators to produce coherent and structured textual explanations. We also introduce BERTopicXAI, a new extension of BERTopic that provides token-level interpretability for unsupervised topic modeling.

Our framework is model-agnostic, allowing for the easy integration of future LLMs as the technology evolves. The specific model used in this work was selected to prioritize reproducibility and computational feasibility, ensuring that the results are both verifiable and efficient to obtain.

By combining classification outputs with token-level attribution methods (e.g., LIME), we enable a fine-grained mapping of opinions across multiple dimensions, including sentiment, emotion, topic, and domain-specific labels such as political stance or mental health status. This results in not only accurate classifications but also in context-aware and transparent natural language justifications that explicitly link lexical cues to their corresponding dimensions.

Comprehensive evaluations conducted on four diverse datasets—Booking, Amazon, Depression, and Political—demonstrate that our proposed system significantly outperforms baseline methods across readability, coherence, semantic similarity, and informativeness. The generated explanations are more aligned with underlying classifications and are easier to interpret by both humans and automated judges.

By moving beyond one-dimensional opinion summaries to structured, interpretable, and generative explanations, this work establishes a generalized and extensible framework for transparent opinion analysis. It sets the foundation for practical applications in business intelligence, political analysis, and mental health monitoring—where both classification performance and explanation quality are essential.

Appendix

1.1 Per-Document Stability Analysis

Table 8 presents detailed feature stability for five documents at varying number of sample levels. For stability analysis, documents are randomly selected from Booking.com dataset. The stability analysis validates our use of num_samples = 300 throughout Sect. 4.3 as observed in Tables 5 and 8. Per-Document stability analysis confirms that Jaccard similarity and Stability ratio after number of samples increased from 300 to 500 relatively stayed the same.

Table 8 Per-document stability metrics across num_samples values

1.2 Random Seeds for Classifier Runs

Across the four datasets (Booking, Amazon, Depression, and Political), we ensure reproducibility by fixing all sources of randomness in the experimental pipeline (e.g., model initialization and data shuffling). To assess performance stability and ensure reproducibility, we repeat each classifier experiment over $R=5$ independent runs with different random seeds, $\mathcal {S}=\{42, 54, 66, 78, 90\}$, while keeping all other hyperparameters unchanged across runs.

1.3 LLM Configuration for Reproducibility

To ensure full reproducibility of our generative and evaluative experiments, this appendix documents the exact configurations used for the GPT-4o model in its two distinct roles. The prompt for the generator is detailed in Sect. 3.4; the prompt and parameters for the judge are provided here (Table 9).

Table 9 LLM configuration parameters

1.3.1 System Prompt for the LLM-as-a-Judge

The following system prompt was used to instruct the model in its role as an impartial judge. This prompt defines the evaluation criteria and ensures a structured, parsable JSON output for each assessment.

1.4 Details About Human Evaluation Protocol

Table 10 summarizes the overall human-evaluation protocol, including the number of participants, the number of evaluated instances, the evaluation procedure, and the scale.

Table 10 Summary of the human evaluation protocol