Self-adapting Machine Learning-based Systems via a Probabilistic Model Checking Framework

Probabilistic model checking is a formal technique based on methods for reasoning about and analyzing systems that exhibit probabilistic and uncertain behavior. This approach has been extensively explored in the self-adaptive systems literature [6, 8, 49] to synthesize optimal adaptation strategies. To generate these strategies, it is necessary to instantiate a formal model of the system under adaptation and to specify an adaptation goal in the form of a property (written as a temporal logic formula) that the model checker can verify for optimality. Additionally, these techniques are a natural fit for planning the need for adaptation in self-adaptive systems since they support proactive adaptation schemes such as look-ahead [49]. This consists of having the model checker, via the formal model of the system, simulate the different possible future states to synthesize the adaptation strategy (sequence of adaptation tactics to execute) that maximizes system utility.

This work leverages the PRISM model checker [40], which is a probabilistic model checker commonly used in the literature [41, 49]. We define the formal models as Markov Decision Processes (MDPs) [55], which allow to model systems’ dynamics through a set of states, whose transitions are either probabilistic or partially controlled by an actor and which model the evolution of the state of the system in discrete timesteps. More formally, an MDP is defined by a 4-tuple \((\mathcal {S}, \mathcal {A}, P_a, R_a\) ), where \(\mathcal {S}\) is the set of states, \(\mathcal {A}\) is the set of actions, \(P_a\) is the probability matrix that gives the probability of transitioning to state \(s^{\prime }\) from state s by executing action a at time t, and \(R_a\) is the reward associated to this transition. At each timestep, a set of adaptation tactics is available for adaptation and the model checker has to select one to execute. Not adapting is considered a possible tactic available for adaptation that has no impact on the system and which we refer to as NOP. The choice between adaptation tactics corresponds to a nondeterministic choice and is guided by the optimization of a user-defined property. This property encodes the goal of the adaptation process and generally corresponds to optimizing system utility. The definition of system utility is domain dependent: it can be associated, for example, with the cost of operating a system, the user experience, or the energy consumption.

To specify these properties, PRISM relies on probabilistic computation-tree logic (PCTL) [31], which is an extension of Computation-Tree Logic (CTL) [21]. CTL is a formal language for specifying well-formed formulas (WFFs) according to a pre-defined grammar. These formulas are then used by model checkers to verify system properties, typically expressed in terms of liveness and/or safety. Examples of temporal CTL operators are \({\bf F}\) and \({\bf X}\) , which specify that a given WFF eventually has to hold or has to hold at the next state, respectively. PCTL extends CTL allowing the definition of formulas that account for probabilities associated with state transitions. By exploiting the fact that it is possible to associate rewards with specific states or state transitions in the formal model, and by specifying reward-based properties as a function of state-specific constraints of the system, PRISM generates optimal strategies that comply with these constraints. To specify such properties in PRISM using PCTL, one would leverage PRISM’s R operator, which informs PRISM that we want to reason about Rewards. Such properties are expressed as \({\bf R}\) \({\rm\small QUERY}\) REWARD_PROPERTY. \({\rm\small QUERY}\) can be set to any of \([=?\) , \(min=?\) , \(max=?]\) and allows the user to specify whether PRISM should find the exact, minimum, or maximum value of the rewards for the reward property REWARD_PROPERTY. Finally, REWARD_PROPERTY is defined in terms of CTL operators, such as the operators \({\bf F}\) and \({\bf X}\) described above. For example, REWARD_PROPERTY can be defined as

This would lead PRISM to compute the reward accumulated along a path until the system eventually reaches a state satisfying \({\rm\small PROP}\) (i.e., a state where variable z is 2).

While it is fairly trivial to compute values for features such as (1) the amount of new samples with which the ML model has not been trained and (2) the time elapsed since the model was last retrained, computing the model’s real-time predictive performance may not be as straightforward. This is particularly the case in contexts for which ground-truth labels may take a non-negligible time to become available. Without these labels, we only have access to a set of unlabeled samples, model predictions, and delayed (outdated) labeled samples, thus rendering it challenging to estimate a model’s predictive quality in real time.

To address this challenge, the ML literature has proposed numerous approaches to estimate the quality of models’ predictions, [16, 16, 27, 35]. Existing methods can be coarsely classified according to the type of ML models that they target (e.g., deep neural networks vs. generic ML predictors) and to their ability to estimate aggregate quality on an entire (unlabeled) test set [30] vs. identify misclassified test inputs [27].

These approaches typically leverage the probability distribution that a model outputs¹ to gain insights into the level of uncertainty associated with each prediction. This uncertainty thus constitutes a meaningful proxy to estimate model performance/error. For instance, the approach by Jiang et al. [35] targets deep neural networks and estimates the error on unlabeled test sets by measuring the disagreement rate of instances of the same network trained with a different run of Stochastic Gradient Descent (SGD). Another approach is to develop self-trained ensembles of deep networks that predict which inputs will be misclassified by the classifier based on (known) errors in the training dataset [16]. Other works, like ATC [27], take a model-agnostic approach and can estimate the correctness of individual model predictions—and then use the point-wise predicted labels to compute aggregate estimates of a model’s predictive quality (e.g., class error rate) as well as of ground-truth class probabilities.

By leveraging such techniques, our framework can be applicable to a wide range of domains, without requiring assumptions on ground-truth label availability. Leveraging such techniques increases the framework’s applicability and generalizability. However, the framework’s accuracy is ultimately dependent on the accuracy of both (1) the predictors for estimating the benefits of executing each adaptation tactic and (2) the approaches employed for estimating the current quality of the ML model.

The proposed framework targets systems composed of ML and non-ML components and is designed to automate the analysis of the tradeoffs associated with adapting (e.g., retraining) an ML component at a given moment with the goal of maximizing system utility. Our design aims to ensure the following key properties: (1) generic—designed to be applicable to different types of offline supervised ML models (e.g., neural networks, random forests); (2) tractable—designed to be usable by a probabilistic model checker like PRISM, which requires identifying an adequate level of abstraction to model ML components in order to enable systematic analysis via model checking; (3) expressive—designed to capture the general and key dynamics of ML models; and (4) extensible—designed to be easily extended to incorporate additional adaptation tactics (e.g., transfer learning, unlearning), as discussed by [13], and customized to capture application specific dynamics.

Our framework makes the following assumptions:

Similarly to previous work [6, 8, 49], the proposed framework leverages a self-adaptation manager that adopts a MAPE-K [37] architecture, as illustrated in Figure 1. The following paragraphs briefly describe each module of the architecture.

Environment. Generates events that constitute the inputs to the ML-based system, and hence to its ML component(s). These events may cause ML mispredictions and a decrease in system utility.

ML-based system. Implements the domain-specific tasks, for which it relies on at least one ML component and may rely on several other components, both ML and non-ML. ML-based systems include, for instance, financial fraud detection [3, 54] and machine translation systems [23, 59]. These systems normally rely on both ML and non-ML components to fulfill their objectives, namely detecting fraudulent transactions and translating sentences, respectively. A system component that incorporates an ML model is considered an ML component. Such components can be adapted via the execution of the tactic selected by the self-adaptation manager.

Self-adaptation manager. Provides the required functionalities for ML adaptation. The novelty of the proposed framework with respect to existing self-adaptation managers lies in the operation of the Analyze and Plan components: Analyze—contains the cost/benefit predictors (referred to as Adaptation Impact Predictors (AIPs), introduced in Section 4.4), which leverage historical data of previous adaptations and of their impact on the ML component’s quality (e.g., accuracy) to estimate its future performance in case an adaptation is or is not executed, and Plan—comprises the adaptation planner, which relies on a formal model of the system being adapted and on a probabilistic model checker to synthesize the adaptation strategy that optimizes system utility.

The self-adaptation manager triggers the execution of any of the tactics available to adapt the system. Although the diagram in Figure 1 considers two example tactics (retraining ML components or NOP (not performing any adaptation)), as discussed in our prior work [13], the ML literature has proposed a number of approaches that can be leveraged as adaptation tactics, such as:

The self-adaptation manager should thus abide by the following key requirements:

The following sections describe the Analyze and Plan components, explaining how these requirements are met.

This section details how we formally model the ML components to capture their error in a compact but meaningful way (realizing R2) and its impact on system utility (realizing R3).

ML component definition. Depending on the domain of operation of a system, the most appropriate type of ML component varies. For instance, while in cyber-physical systems it is common to see reinforcement learning ML components [38], in the context of fraud detection [44, 54] or medical diagnosis systems [25], offline trained ML components (e.g., decision trees or neural networks) are more common. We focus our analysis on offline trained ML components and specifically on ML classifiers. (Note that it is possible to transform a regressor into a classifier by discretizing the target domain, although this implies introducing an intrinsic prediction error due to the chosen discretization granularity.)

ML component state. Since the goal of the proposed framework is to model ML components, and in particular the impact of their mispredictions on system utility, we require a way to evaluate their classification performance. Classification models are typically evaluated based on a popular construct known as confusion matrix [61], which provides a statistical characterization of the model’s quality by describing the distribution of its misclassification errors. For a classification problem with N classes, the confusion matrix normalized by rows \(\mathcal {C}\) (the rows represent the actual sample class and sum to 1) contains, in each cell \((i,j)\) , the ratio of samples of class i (ground truth) that have been classified as being of class j (prediction). For the simpler case of binary classification problems, the confusion matrix is reduced to a \(2\times 2\) matrix where each cell specifies the following: True-Positive Rate (TPR)—percentage of examples of the positive class that the model classified as such; True-Negative Rate (TNR)—percentage of examples of the negative class that the model classified as such; False-Positive Rate (FPR)—percentage of examples of the negative class that the model classified as positive; False-Negative Rate (FNR)—percentage of examples of the positive class that the model classified as negative. This representation allows for extracting further error metrics such as the model’s accuracy and F1-score.

The row-normalized confusion matrix has the following relevant properties: (1) generic—can be computed for different ML models (e.g., random forest or neural network); (2) tractable—is compact and abstract enough to be encoded into a formal model; (3) expressive—captures the predictive performance of the ML model and allows for computing several error metrics; and (4) extensible—can be used to model the impacts of executing different adaptation tactics [13] (e.g., retrain, NOP) by updating its cells. These properties make it a natural fit to model ML components and hence realize R2. Depending on the predictive models used by the model checker to estimate the evolution of the confusion matrix (or the cost of executing an adaptation tactic), the state of the ML component can be extended with additional variables, e.g., that describe the expected data shifts on the input or output.

ML component interface. Since the framework aims to adapt offline-trained ML components, we define the base interface as being composed of the methods query and retrain. The ML component interface can be extended to incorporate additional adaptation tactics (e.g., transfer learning [51] or unlearning [9, 10]), if those are indeed available for the managed system. As the name suggests, retrain models the execution of a retrain procedure of the ML component by triggering an update of its row-normalized confusion matrix. The techniques employed to predict how the confusion matrix of an ML component evolves as a result of a retrain procedure are described in Section 4.4. The query method models the process of asking the ML component for predictions for a set of inputs. Specifically, this method should abstract over the concrete input/output values of the samples and of the predictions, requiring only the total number of inputs for the ML component and the expected distribution of (real) output classes \({\bf O}\) (given by the probability \(p_i\) for an input to be of class i, for all classes \(i\in [1,N]\) ). The method returns a (non-normalized) confusion matrix \(\mathcal {C}^*\) that reports in position \((i,j)\) the (absolute) number of inputs of class i that are classified as of class j by the model. \(\mathcal {C}^*\) can be simply computed by multiplying each row i of the normalized confusion matrix \(\mathcal {C}\) by \(p_i\) . The interface can be extended to account for more adaptation tactics that allow to tailor the framework to specific adaptation scenarios.

Dealing with uncertainty. As shown by recent work [7, 33, 49], capturing uncertainty and including it when reasoning about adaptation contributes to improved decision-making. To capture uncertainty, we leverage the probabilistic framework proposed by Moreno et al. [48], which allows to account for different sources of uncertainty in the system (e.g., uncertainty on the effects of an adaptation tactic or on the input class distribution) and which generates memoryless strategies (strategies that depend only on the current state of the system). This framework accounts for uncertainty by modeling the source of uncertainty as a probabilistic tree that is approximated via the Extended Pearson-Tukey (EP-T) [36] three-point approximation. The current state of the source of uncertainty is represented by the root node of the probability tree and the child nodes are its possible realizations.

A key requirement of our framework is the ability to predict the costs and benefits of executing adaptation tactics on the ML components (requirement R1). For this purpose, the proposed framework associates with each adaptation tactic a dedicated component, which we call the AIP. The AIP is in charge of predicting (1) the adaptation tactic’s cost that is charged to the system utility and (2) the impact of the adaptation on the future quality of the ML component. We also include an adaptation tactic corresponding to performing no changes to the ML component (NOP). While the AIP for tactic NOP always predicts zero costs (this tactic inherently has no cost), its model quality predictor captures the evolution of the model’s performance if no action is taken, e.g., the possible degradation of accuracy of the ML component due to data shifts. Overall, this approach allows the model checker to quantify the impact of different adaptation tactics on system utility and reason about their cost/benefit tradeoffs.

We focus on the problem of how to predict the performance evolution of the ML component and describe, in the next section, how we tackle the problem of implementing AIPs for the retrain and NOP tactics for generic ML components. Indeed, for tactics such as retrain the problem of estimating their costs has been investigated in the system’s community. The literature has shown that data-driven approaches [11] based on observing previous retraining procedures, possibly mixed with white-box methods [66], can generate accurate predictive models of the retrain cost.

Predicting future quality of ML components. Given the reliance on a row-normalized confusion matrix \(\mathcal {C}\) to characterize the performance of ML components, predicting their performance evolution requires estimating how \(\mathcal {C}\) will evolve in the future, e.g., due to shifts affecting the quality of the current model or as a consequence of retraining the model to incorporate newly available data.

The proposed method abstracts over the specific adaptation \(a()\) by modeling it as a generic function \(\mathcal {M}^{\prime } \leftarrow a(\mathcal {M}, \mathcal {I}, \mathcal {N})\) that produces a new ML model \(\mathcal {M}^{\prime }\) and takes as input (1) model \(\mathcal {M}\) prior to the execution of the adaptation; (2) data \(\mathcal {I}\) , used to generate model \(\mathcal {M}\) ; and (3) new data, \(\mathcal {N}\) , that became available since the last adaptation, e.g., by deploying the model in production and gathering new samples and corresponding ground-truth labels. We assume that both \(\mathcal {I}\) and \(\mathcal {N}\) contain ground-truth labels. Additionally, we assume that \(\mathcal {M}\) and \(\mathcal {M}^{\prime }\) are generic supervised ML models that are queried and returned predictions for the input samples. These two assumptions allow to determine the confusion matrices of models \(\mathcal {M}\) and \(\mathcal {M}^{\prime }\) at any future time interval, since their predictions can be compared with the ground-truth labels.

We seek to build blackbox regressors (e.g., random forests or neural networks) that, given model \(\mathcal {M}\) obtained at time 0 with dataset \(\mathcal {I}\) , and given new data \(\mathcal {N}\) available at time \(t\gt 0\) , predict the confusion matrices of both models ( \(\mathcal {M}\) and \(\mathcal {M}^{\prime }\) ) at time \(t+k\) , where \(k\gt 0\) is the prediction lookahead window.

Adaptation impact dataset. In order to train such a blackbox regressor, we build an Adaptation Impact Dataset (AID) by systematically simulating the execution of the adaptation tactic using production data in different points in time. This allows for gathering observations characterizing the execution of the adaption tactic in different environmental contexts, such as (1) different sets of data used to adapt the model, (2) variation in the time passed since the last execution of the tactic, and (3) different ML performance before and after adaptation.

The first step of the procedure consists of monitoring model \(\mathcal {M}_0\) of an ML component in production over T time intervals. During this period, given the absence of AIPs, we assume that no adaptation is executed. Next, we deploy \(\mathcal {M}_0\) on a testing platform (so as not to affect the production environment) and systematically apply adaptation \(a()\) at each time interval \(i\gt 0\) , i.e., \(a(\mathcal {M}_0, \mathcal {I}_0, \mathcal {N}_i)\) . This yields a new model \(\mathcal {M}_i\) , which we evaluate at every future time interval \(i\lt j\le T\) , obtaining the corresponding confusion matrices, noted as \(\mathcal {C}_{i}(j)\) . Overall, this procedures yields T models, resulting from the adaptation of \(\mathcal {M}_0\) at different time intervals, and produces \(T\cdot (T-1)\) measurements of the confusion matrices at times \(j\gt i\) . This testing platform is required to support the data pre-processing pipeline, model building, and inference stages of the ML components targeted by the adaptation. Such testing platform is then leveraged by the framework to create and evaluate different versions of these components, eschewing the need to reproduce the full production system, comprising the whole set of ML and non-ML components. We expect such testing platforms to be normally available due to the common DevOps/MLOps practice [60] of testing ML models’ quality prior to their actual deployment in production.

For each of the aforementioned \(T\cdot (T-1)\) measurements, we generate an AID entry, \(e_{i,j,k}\) , which describes the quality at time \(j+k\) of model \(\mathcal {M}_j\) obtained by executing \(a(\mathcal {M}_i, \mathcal {I}_i, \mathcal {N}_j)\) at time j on model \(\mathcal {M}_i\) , where \(\mathcal {I}_i\) denotes the data used at time i to generate model \(\mathcal {M}_i\) , and \(\mathcal {N}_j\) the new data gathered from time i until time j. Each entry \(e_{i,j,k}\) has as target variables the \(N^2-N\) independent entries of the confusion matrix at time \(j+k\) of model \(\mathcal {M}_j\) and stores the following features:

Overall, the AID can be seen as composed of pairs of features, where each pair describes a specific “characteristic” of the data or model at two different points in time, e.g., amount of data available at time i and j or distribution of predicted classes at time \(j+k\) by models \(\mathcal {M}_i\) and \(\mathcal {M}_j\) . The last step of the process consists of extending the AID by encoding the variation of each feature as follows: (1) for scalar features (e.g., amount of data) we encode their variation using the ratio and difference and (2) for features described via probability distributions (e.g., prediction’s uncertainty) we quantify their variation using the JSD [46] (inspired by previous work [54]), which yields a scalar measurement of the similarity between two probability distributions. This generic methodology can also be applied to the case of the NOP tactic. In this case, the dataset describes how the accuracy of a model originally obtained at time i will evolve at time \(j+k\) , based on the information available at time j.

Building the AIPs. We exploit the AID dataset to train a set of independent AIPs, which can be simple linear models or blackbox predictors such as random forests or neural networks. Each AIP is trained to predict the value of a different cell of the confusion matrix. Given an n-ary classification problem, we have \(n^2-n\) independent values for the corresponding confusion matrix, since each row must sum to 1. For the case of binary classification, where \(n=2\) , it is sufficient to predict the values of the two elements on the diagonal, which, being in different rows, are not subject to any mutual constraint. For the general case of \(n\gt 2\) , it is necessary to ensure that the predictions of the AIPs targeting different cells of the same row sum to 1. This can be achieved by using a softmax function [4] to normalize the predictions generated by the AIPs into a probability distribution.

Integrating the AIPs in the formal model. As for the integration of the AIPs in the formal model, which is checked via a tool such as PRISM, a key practical issue is related to the fact that these tools do not typically allow for interacting with external processes (which could be used to encapsulate the implementation of the AIPs) during model analysis. This would be beneficial for cases when the model checker is used to reason on a look-ahead horizon of \(l\gt 1\) time intervals. In such a case, up to \(a^l\) possible adaptation strategies are generated, where a is the number of adaptation tactics available, thus requiring up to \(l\cdot a^l\) predictions.

This problem can be circumvented by integrating directly the AIPs as part of the formal model to be checked. This approach is reasonable if the AIPs are implemented via simple methods, such as linear models, but is cumbersome and unpractical for the case of more complex models, such as neural networks. An alternative approach, which is the one currently implemented in our framework, is to precompute all the predictions that will be required during the model checking phase and provide them as input constants to the model checker tool. This approach is viable only when the lookahead window and the set of available adaptations are small but allows us to use arbitrary external predictors.

In general, label delay can be thought of as a form of temporal misalignment between the data and the labels. More formally, for a delay d and current timestamp t, ground-truth labels are available for transactions completed up to time instant \(t-d\) . This means that in order to estimate the current quality of the model, one either resorts to (1) delayed labels, hence using stale data as a proxy for the current model’s quality, or (2) methods (see Section 3.2) that aim to predict the current model’s predictive quality in the absence of labeled data.

To test the proposed framework, which requires knowledge of both the current performance of the ML model and the distribution of the target (features BF2 and BF4, c.f. Section 4.4) in order to estimate the impact of an adaptation, we explore three different approaches to label estimation. The first, less realistic approach assumes that (1) labels are immediately available. As this is not typically the case in reality, for our second and third approaches we relax this assumption by (2) leveraging only delayed labels and (3) resorting to methods for predicting model performance under unseen data distributions. Among the approaches described in Section 3.2, we use our own variant of the ATC method proposed in [27], that we name CB-ATC. We build on ATC due to its reduced computational complexity and to its ability to estimate the individual ground-truth labels of point-wise model predictions.

The formal model of the system requires modules for each of the different moving parts that have an impact on the system. Thus, we model (1) the environment under which the system is operating; (2) the actual system, to analyze how mispredictions affect system utility, to simulate the execution of the tactics, and to understand their impact on system utility; and (3) the adaptation tactics, which in our case consist of either retraining the model or sticking with the current one.

Since we assume that the two tactics (NOP and retrain) cannot be executed simultaneously, we further consider an adaptation manager module that prevents this from happening and nondeterministically selects which tactic to execute. As shown in Listing 1, whenever there is a new event generated by the environment (line 5)—for the fraud detection system an event consists of a batch of transactions—the adaptation manager enters the selectTactic state and can select to execute one tactic among the available ones.² For example, while tactic nop (do nothing) can always be executed (line 8), tactic retrain can only be executed when there is newData with which to train the ML component (line 9). Finally, since our approach assumes that time is divided into fixed-sized intervals, we further model a clock whose purpose is to keep track of the passing of each time interval. The clock module is implemented as in [48].

Synthesizing optimal adaptation policies. As can be seen in Listing 1, each tick of the clock triggers the accrual of a reward.³ For this specific use case, the rewards (lines 18–22) consist of the total costs incurred by the system during that time period due to the tactics executed and possible SLA violations. We account for the latency of executing the adaptation tactic by considering that there is a percentage of transactions that is classified resorting to the previous, non-adapted model, while the remaining transactions are classified resorting to the adapted model. Specifically, the tactic is assumed to take a given percentage of the time interval to execute. This percentage is given by tacticLatency. During this period, the system is receiving and classifying transactions. The percentage of transactions classified during this period is encoded in percentTxs. Thus, if the non-adapted model violates any threshold, the cost the system incurs is proportional to tacticLatency \(\times\) percentTxs \(\times\) sum of costs of SLA violations. For the remainder of the time instant, i.e., 1 - tacticLatency, the remaining transactions 1 - percentTxs are classified with the adapted model, hence leading the system to possibly incur a different cost due to variations in the system’s TPR and FPR.

To generate optimal adaptation policies, PRISM requires the specification of a property. Since for this use case system utility is defined as the total costs incurred by the system (Equation (5)) and since the goal is to minimize these costs, the property that leads to the optimal adaptation policy corresponds to minimizing system utility, which is defined in PCTL (reward-based property specification logic, c.f. Section 3.1) as \({\bf R}^{\text{systemUtility}}_{\text{min}=?} [{\bf F} \: {`{\rm\small{END}}^{\prime }}]\) , which means “minimum system utility when time ‘end’ is reached,” and where \({\bf R}\) and \({\bf F}\) are the operators described in Section 3.1, \(\text{min=?}\) is the query, and \(\text{systemUtility}\) specifies the reward structure to use as target. ”end” defines the simulation horizon, i.e., how many future time intervals we want the formal model to simulate.

Extending the tactic’s repertoire. To reason about self-adaptation considering a broader set of adaptation tactics, the formal model needs to be changed only through the addition of the corresponding tactics’ modules such that the adaptation manager can consider them as available when making its nondeterministic choice. This can be achieved by adding these tactics to Listing 1, in addition to nop and retrain.

As discussed in Section 4.4, the framework instantiates an AIP for each adaptation tactic, trained using the features presented in Table 1. In this case, since there are two adaptation tactics (retrain and nop), the framework instantiates one AIP for each, which is composed of two predictors: one for predicting the increase/decrease in the TPR and a second one to predict the TNR. Thanks to the properties of the confusion matrix, by predicting the future TPR and TNR, we can fully characterize the ML component’s confusion matrix in the following time interval. These predictions are then provided as inputs to the formal model and leveraged by the probabilistic model checker to synthesize an optimal adaptation strategy.

Figure 4 compares the proposed framework (represented by line AIP) against the baselines. To evaluate whether the use of the framework allows to improve system utility over baselines that do not explicitly estimate the benefits of retrain, we define the SLA thresholds as \(\text{RECALL} \ge 70\%\) [3] and \(\text{FPR} \le 1\%\) [68] and fix the retrain latency to 0 and the retrain cost to 8. These SLA threshold values were set based on values typically employed by related works in this domain [3, 68]. As Figure 4(a) shows, by leveraging the proposed framework, it is possible to have the fraud detection system minimize its total costs and be closer to the optimal possible cost. This answers RQ2 and shows that the framework does improve system utility over simpler, model-free baselines due to its ability to estimate the benefits of executing the retrain tactic. As for the number of SLA violations, as shown in Figure 4(b), the AIP violates slightly more SLAs than all other baselines except No-retrain (which, as expected, is by far the approach that violates the most SLAs). However, as seen previously, this does not translate into higher incurred costs, which is the quality attribute under optimization.

In order to evaluate how different execution contexts impact the need for retrain and answer RQ3, we ran experiments for different SLA thresholds, retrain costs, and retrain latencies. Specifically, we tested the values shown in Table 2 for each dimension, fixing the remaining two dimensions to the values of the base scenario (recall threshold = 70, retrain cost = 8, retrain latency = 0). Figure 5 displays these results.

Regarding the recall threshold (Figure 5(a)), the results show that, as expected, the cost incurred by the approaches increases as the recall threshold increases. This is justified by the fact that an increase in the recall threshold yields a more difficult problem—the system tolerates less incorrect classifications of fraud transactions. This is translated into an increase of the SLA violations, thus increasing the cost. The optimum and AIP approaches also suffer a cost increase since retraining does not prevent them from violating the thresholds.

Focusing now on the retrain cost (Figure 5(b)), we see that if the cost is very low, the decision of whether to retrain is fairly trivial and so all approaches that retrain the ML component are close to the optimum. However, as the retrain cost increases, we start to notice how being careful in selecting when to retrain, accounting for the costs and benefits of the tactic, does pay off, as AIP is closer to the optimum than the other approaches. As expected, the No-retrain approach is not affected by this dimension.

Finally, regarding the retrain latency dimension, the tested values correspond to percentages of the time interval that are occupied with the process of retrain. That is, retrain latency = 0: retrain is assumed instantaneous; retrain latency = 1: during the first 10% hours of the time interval the model is being retrained and as such transactions are classified using the existing (non-retrained) model. The same rationale applies to retrain latency = 5. The results (Figure 5(c)) show that this dimension has relatively little impact on the cost of any approach, although as expected the total cost of the optimum solution increases slightly as the retrain latency grows. In fact, even if this baseline can always determine correctly whether it is worth retraining the model at any time t, if the retrain latency grows, a fraction of the transactions in input for the tth interval will be classified using an old model, thus suffering from an increase in misclassifications and SLA violations.

This section answers RQ1 by evaluating the performance of the AIPs resorting to the mean absolute error (MAE) and to the PCC (corr-coef), and considering different sets of features employed by each predictor. Specifically, we consider three different feature sets: S—minimal set with only the basic features (c.f. Section 4.4); M—medium set, which includes the basic features and output characteristics; L—encompasses the features of the previous sets and the input characteristics. Table 3 displays these results. Interestingly, we see that an increase in the size and complexity of the feature set does not yield better AIPs. Additionally, the results also show that the models responsible for predicting the future TPR and TNR when the model is retrained achieve a higher accuracy (lower MAE and higher correlation) than their NOP counterparts (which predict the future TPR and TNR when the model is not retrained). Overall, on the one hand, the accuracy of the AIPs proposed in this work is, as shown in Figure 5, good enough to allow implementing effective adaptation strategies. On the other hand, the absolute accuracy metrics reported in Table 3 confirm that predicting the future performance of ML models is far from trivial and that the proposed predictive methodology still has significant margins of improvement (e.g., by identifying different features or blackbox predictors or possibly combining white-box methods [24]).

Since the purpose of the framework is to enable runtime adaptation of ML components in order to improve system utility, we evaluate the time complexity of the process of generating the adaptation strategy. This process corresponds to querying the AIPs and having PRISM verify the property of interest for the formal model of the system. Table 4 shows the average and standard deviation of the time overhead due to the whole process and also of the formal model verification alone. These values correspond to the execution context defined in Section 6.1 and were obtained by running the experiments on a machine with an AMD EPYC 7282 [email protected], with 16 cores and 128 GB RAM. As can be seen, the process of generating the adaptation strategy takes around 3.5 seconds, which is perfectly affordable considering that retraining ML components has a much higher time overhead. This answers RQ4 and shows that it is feasible to employ the proposed framework in an online scenario.

We now evaluate the impact of label delay when estimating model performance and how the accrued estimation error affects the performance. We structure this study in two parts.

We start by analyzing the effectiveness of ATC and CB-ATC (see Section 4.5) in predicting the current confusion matrix of the fraud detection classifier, as well as the fraud rate. Recall that this information constitutes some of the input features to the AIP predictors, which are then used to predict the expected variation of the confusion matrix in the following time window, depending on whether the model is retrained or not.

Next, we evaluate to what extent using ATC and CB-ATC to predict the input features for the AIPs impacts the effectiveness of the self-adaptative framework to optimize system utility.

Estimation of confusion matrix and fraud rate. Figures 6 and 7 evaluate the performance of both ATC and CB-ATC when used to estimate the current (i.e., before adaptation is enacted) TPR, TNR, and fraud rate of the fraud detection model. Specifically, we report the MAE (Figure 6) and the PCC (Figure 7) for each approach and for each value of label delay tested. Regarding the MAE, we observe that CB-ATC substantially reduces the estimation error for all metrics (TPR: Figure 6(a); TNR: Figure 6(b); fraud rate: Figure 6(c)) when compared against the delayed labels baseline. ATC, however, is actually outperformed by the delayed labels baseline. This is due to the fact that ATC was developed to estimate the accuracy of the classifier and not the performance of individual classes, which is the current case. Regardless, and particularly for the TNR and the fraud rate (Figures 6(b) and 6(c)), all approaches have a low estimation error.

In terms of the PCC (Figure 7), both ATC and CB-ATC present much higher correlations than the delayed approach, for all metrics evaluated. In this setting, high correlation means that the estimations obtained by both ATC and CB-ATC provide meaningful insights into the actual real values of the metrics. Since among the three approaches CB-ATC is the one that overall presents the best tradeoff between the error (low) and the correlation (high), we expect it to yield better performance when leveraged by the framework.

These experimental results corroborate our hypothesis on ATC’s limitations (Section 4.5.1) and that CB-ATC can effectively solve these issues, leading to lower estimation error. Impact on system utility. Next we evaluate the impact that using these three approaches has on system utility (i.e., total cost) when dealing with different values of delay. Specifically, we consider delay intervals ranging from 2 to 34, which correspond to, approximately, 1 and 14 days, respectively (recall that a delay interval corresponds to 10 hours). Figure 8 reports the total cost incurred by each baseline for the same environmental settings of Figure 4. These results confirm the superiority of CB-ATC, which globally appears to be the most competitive solution across the considered delay values up to delay 12 (corresponding to 5 days). More precisely, the average of the total cost in the range of delay values [2, 12] for the approaches CB-ATC (AIP_cbatc), ATC (AIP_atc), and delayed metrics (AIP_del) achieve an average of 1,359, 1,407, and 1,453, respectively. This was expected given the conclusions drawn by the analysis of Figures 6 and 7.

Further, these results show that, for relatively small delay values (i.e., delay 2, corresponding to approximately 1 day), the use of the CB-ATC method allows to achieve a system utility that is close to (i.e., 8% worse than) the one obtained in a setting where labels are immediately available (i.e., the AIP baseline). As the delay grows beyond 12 (corresponding to 5 days), the performances of CB-ATC, as well as ATC, tend to degrade relatively to the baseline, which has only access to delayed information, leading them to achieve a performance that is on par with a random approach. Conversely, in the [2, 12] interval of delays, CB-ATC achieves an average gain of approximately 10% with respect to a random approach.

We suspect that the root cause of the problem is that the quality of the AIP models degrades significantly as the delay grows, independently of whether the AIPs’ input features are being predicted (via ATC or CB-ATC) or if delayed values are being used. Regardless, and as previously observed, CB-ATC can predict these features more accurately than an oracle that simply outputs delayed information. We argue that this limitation might be imputed to the modeling approach that we currently use to construct the AIPs, which we intend to address in future work by investigating alternative ML modeling techniques and different feature engineering methods.

In this work we have created AIPs only for the nop and retrain tactics. However, even in the absence of AIPs capable of estimating the effects of additional tactics, the planning component (i.e., the probabilistic model checker) of our framework can still be effectively used to support “what-if” analysis aimed at identifying in which scenarios the use of additional adaptation tactics would optimize system utility.

In order to demonstrate these capabilities, we consider the availability of a third tactic, which we refer to as component replacement, that consists of replacing the ML model used for fraud detection with a rule-based model defined by human experts. We assume that the rule-based model offers a fixed TPR of \(75\%\) and that the current performance of the ML model is \(TPR=75\%\) and \(TNR=98\%\) . The SLA thresholds and costs are the ones previously defined. We then use the planning component of our framework to conduct a what-if analysis that aims to identify the optimal adaptation tactic when varying (1) the TNR for the rule-based model, (2) the TNR for the ML model after it is retrained, and (3) the costs for the retrain and component replacement tactics.

Figure 9 shows the results of this analysis, indicating with different colors the optimal tactic for each setting of rule-based model TNR, retrained ML model TNR, and tactic costs. The figure reports on the X axis the TNR of the ML-based component after retrain and on the Y axis the TNR of the rule-based model. However, Figures 9(a) and 9(b) consider different execution costs for each tactic. This study demonstrates that the model checker is capable of selecting between multiple adaptation tactics by reasoning on the impact of each alternative tactic on system utility and identifying the one that yields the maximum gains.

The findings regarding the predictability of the impacts of retrain are dataset and domain dependent and so they cannot be generalized to other domains or datasets (external validity). This also applies to the time complexity of the approach, which depends on the complexity of the formal model. Thus, further research is required to understand how the proposed framework and architecture fare in different domains (e.g., self-adaptive intrusion detection, spam detector, or machine translation systems) and for systems that rely on other types of ML models (e.g., neural networks, support vector machines, linear models). Analogous considerations apply to the evaluation of the CB-ATC method, which has only been conducted in the context of the case study considered in this work. The proposed method should be evaluated on a broader set of datasets in order to verify whether the benefits observed in our study generalize to different domains.

We have evaluated the use case for specific execution contexts (regarding system SLAs, tactic cost, and latency) that impact the difficulty of the problem (internal validity). Specifically, for the considered use case and evaluation, retrain latency was assumed to be lower than one time interval. This assumption holds for the considered use case since the ML components employed by the fraud detection system are relatively simple and hence it is reasonable to assume that their training takes less than 10 hours. However, for more complex ML components, such as large language models (LLMs) [47], this assumption might not hold. A simple approach to circumvent this issue would consist in setting the time interval used by the model checking tool to be at least as large as the retrain latency, while preserving the frequency with which the framework analyzes the need for adaptation (i.e., the frequency with which we query the framework). On the one hand, this would avoid the need to estimate the ML model’s quality over multiple time steps in the future, which would be necessary if the time interval was set to be smaller than the retrain latency. On the other hand, it would still preserve the framework’s reactivity to possible changes in the environment, as this is dictated by the frequency with which we query the framework.

Further, although in this article we have constructed and evaluated AIPs to predict the impact of executing only two tactics (nop and retrain), our framework has been designed to support other tactics, such as the ones identified in our prior work [14]. Additionally, this work constitutes a first step toward enabling long-term planning of ML adaptations, which the probabilistic model checker intrinsically supports. In fact, the current challenge hindering this extension lies in the computational complexity of creating the AID when accounting for longer horizons (i.e., larger than one time interval). This same challenge also impacts the extension to a setting with multiple adaptation tactics, due to the need to account for possible dependencies among tactics (i.e., tactics whose outcome depends on whether a different tactic was executed, e.g., retrain and fine-tune). Nonetheless, the methodology presented in this work established the required building blocks for coping with more than two tactics (also discussed in Section 6.6) and long-term planning extensions.

Finally, the several building blocks that constitute the framework are what makes it general. Specifically, to instantiate the framework for additional applications, it is necessary to (1) create an AID by following the proposed methodology (Section 4.4); (2) instantiate AIPs for the available adaptation tactics and metrics of interest,² leveraging the AID that was created; and (3) create a formal model of the system, capturing the error of the ML component as described in Section 4.3. Further, the proposed strategy to formally model ML components does not constrain the applicability of the framework to specific types of ML models. In fact, classification models can be evaluated through confusion matrices independently of the underlying ML algorithms they employ (e.g., neural networks, random forests).