1. Introduction
Every hour, transmission system operators across Europe must balance supply against an uncertain demand, using load forecasts made minutes to hours in advance. A 100 MW forecast error cascades into reserve activation, balancing costs, and in extreme cases grid instability (Abumohsen, Owda, and Owda 2023). As national grids grow more interconnected and data-driven, the case for collaborative forecasting that pools signal across borders has become strong; yet load data is commercially sensitive and, in many jurisdictions, legally restricted from crossing national boundaries.
Federated Learning (FL) (McMahan et al. 2017) offers a natural answer. Participating clients train local models on their own data and share only model updates, never raw measurements, with a central server, which aggregates them into a global model. Privacy is preserved by construction, making FL an attractive fit for cross-border energy forecasting where both commercial confidentiality and regulatory constraints are in play.
Standard FL algorithms, most notably FedAvg (McMahan et al. 2017) and FedProx (Li et al. 2020), were built on the implicit assumption that each client’s data distribution, however different from its peers, stays roughly stable over time. Energy systems rarely cooperate with this assumption. Demand patterns shift with seasons, economic cycles, and policy shocks. The 2022 European energy crisis reduced industrial consumption by 5–10% within months, permanently altering load profiles across the continent. We call these episodes regime shifts, persistent changes in the underlying load-generation pattern. When a subset of clients enters a new regime, their model updates pull the global model in a new direction, and naive averaging can corrupt a well-trained global model or derail convergence entirely.
In this paper, we present RAFA, a novel FL framework that explicitly detects and adapts to distribution shifts in client model updates. RAFA makes three principal contributions.
•
Mahalanobis regime detector. A privacy-preserving detector that monitors the Mahalanobis distance of random-projected client update vectors with respect to a rolling reference distribution, flagging clients whose update behaviour has significantly diverged.
•
Reliability-weighted aggregation. A soft aggregation rule that exponentially down-weights regime-shifted clients proportionally to their deviation, protecting the global model while preserving exact FedAvg behaviour when no shift is detected.
•
Always-on personalised head fine-tuning. A decoupled personalisation mechanism that continuously adapts the final prediction head of each client’s model to its local regime, independent of whether a regime shift is detected, while keeping the shared base frozen for federation.
We validate RAFA on a novel multi-national benchmark comprising six years (2019–2024) of hourly actual electrical load data for eight European bidding zones, namely Netherlands (NL), Germany-Luxembourg (DE_LU), France (FR), Spain (ES), Portugal (PT), Belgium (BE), Austria (AT), and Poland (PL), sourced from the ENTSO-E Transparency Platform (ENTSO-E 2024). Experiments demonstrate that RAFA improves forecasting accuracy over FedAvg, FedProx, and per-client local baselines, with by far the largest gain for the client exhibiting the strongest seasonal contrast (France, winter/summer ratio 1.48) and consistent improvements across the remaining clients.
1.1 Motivation for distribution shift in multi-national energy FL
To motivate the problem concretely, consider two clients in our benchmark, France (FR) and Spain (ES). FR has a winter-to-summer load ratio of 1.48, meaning winter demand is 48% higher than summer demand due to widespread electric heating, while ES exhibits a near-flat ratio of 1.02, reflecting a Mediterranean climate with minimal heating load. A global model trained by naive averaging must simultaneously represent both regimes, leading to systematic under- or over-fitting for both clients.
Furthermore, the 2022 European energy crisis introduced a structural regime shift across multiple clients simultaneously; reduced industrial activity, energy efficiency campaigns, and demand response programmes collectively lowered load levels by 5–10% relative to pre-crisis baselines (International Energy Agency 2022). Standard FL algorithms have no mechanism to detect this kind of correlated shift or to adjust aggregation weights accordingly.
The remainder of this paper is structured as follows. Section 2 reviews related work in federated learning, non-IID adaptation, and energy load forecasting. Section 3 formally defines the federated energy forecasting problem and the regime shift model. Section 4 presents the RAFA framework. Section 5 describes the experimental setup. Section 6 presents results and ablation studies. Section 7 discusses limitations and future directions. Section 8 concludes.
2. Related work
The federated learning literature has expanded considerably since FedAvg, but much of that progress addresses a narrower problem than the one studied here. Methods for spatial heterogeneity, where clients have different but largely static feature or label distributions, are now reasonably mature. The harder and less-studied setting is temporal heterogeneity, where client distributions shift during training and the optimisation target moves over time. That gap motivates RAFA, and the following discussion examines how far existing work has gone toward closing it.
2.1 Federated learning: foundations and evolving challenges
McMahan et al. (2017) formalised FL with FedAvg, in which clients execute multiple local SGD steps and transmit model updates to a server that averages them weighted by dataset size. Despite its simplicity, FedAvg remains the dominant baseline because it decouples model training from raw-data sharing, enabling privacy-preserving collaboration at scale. Li et al. (2020) identified and formalised the mechanism behind its primary failure mode; when client data distributions diverge, local objectives pull updates in conflicting directions, a phenomenon they term client drift, causing the global model to converge to a region that is suboptimal for every participant. FedProx addresses this with a proximal term bounding the distance each local update can travel from the global reference point.
The heterogeneity problem has since been reframed as multi-dimensional. Jimenez G. et al. (2024) provide a definitive taxonomy that partitions data heterogeneity into five orthogonal axes, including quantity skew (unequal dataset sizes), feature skew (different input distributions), label skew (different class frequencies), temporal skew (client distributions shift over time), and concept skew (the same input maps to different labels across clients). The authors establish through meta-analysis that temporal skew is the least-studied axis despite being the most prevalent in longitudinal deployments such as energy monitoring. In parallel, Arbaoui et al. (2024) survey aggregation strategies across three design objectives, personalisation, optimisation, and robustness, and conclude that non-stationarity is the most critical open challenge, because neither personalised methods nor clustering-based approaches assume the distribution continues to evolve after an initial adaptation phase.
Clustered federated learning, which assigns clients to homogeneous groups before aggregation, has attracted particular interest as a practical response to spatial heterogeneity (Ben Ali et al. 2025). Recent surveys find clustered methods consistently outperform global aggregation when client distributions differ statically. The key limitation is membership rigidity. Once a client is assigned to a cluster, the assignment persists, making the method blind to the within-client evolution that characterises seasonal energy systems.
An important emerging trend is the integration of pre-trained foundation models into federated pipelines. Hosseinalipour et al. (2025) introduce the concept of multi-modal, multi-task federated foundation models (FedFMs) for power grids, demonstrating that large pre-trained models can be fine-tuned across grid-edge clients in a privacy-preserving manner for tasks spanning load forecasting, anomaly detection, and demand response. This line of work underscores that the heterogeneity challenge is deepening, not resolving. As model capacity grows, the sensitivity to distributional mismatch between pre-training and deployment increases.
Byzantine robustness, addressed by Blanchard et al. (2017) and Yin et al. (2018), represents a structurally separate concern. Byzantine defences identify clients whose updates deviate maliciously from the consensus direction and suppress their influence. RAFA’s reliability weighting is superficially similar but operates from an opposite premise; large update deviations are treated as informative rather than malicious, signalling genuine regime change rather than adversarial intent.
2.2 Temporal Non-stationarity and concept drift in federated learning
The theoretical and practical difficulties of handling concept drift within a federated privacy model have been recognised since at least the survey of Casado et al. (2022), who document that virtually all established drift detectors, ADWIN, Page-Hinkley, DDM, KSWIN, require centralised access to a sliding data window, a requirement that is structurally incompatible with FL’s privacy model. Their conclusion is sobering, adapting any classical drift-detection procedure to the federated setting requires either weakening privacy guarantees, accepting substantial detection delays, or redesigning the detection mechanism from scratch.
Jothimurugesan et al. (2023) take the redesign path with FedDrift, which clusters clients by their estimated current posterior and reassigns them dynamically as distributions evolve. FedDrift requires a reference distribution per cluster and detects drift by comparing current client updates against that reference. A sound approach but one that assumes synchronous drift, all clients in a cluster change at the same time. In interconnected European electricity markets, where each bidding zone responds to its own demand shocks, weather regimes, and policy changes on independent schedules, this synchrony assumption is violated structurally.
The most recent and capable dedicated system is FLAME (Mavromatis, De Feo, and Khan 2024), which monitors per-client output-distribution statistics and reconfigures the federation topology reactively when drift is detected. FLAME represents the current state of the art. It is privacy-compatible, operates at the client level, and avoids the synchrony assumption. Its detection signal, however, is derived from model outputs, predictions or loss trajectories on local validation data. Output-level signals necessarily lag the underlying distributional shift; by the time a forecasting model’s accuracy degrades, the regime has already changed and several rounds of corrupted updates have been aggregated.
RAFA addresses this latency by detecting regime change in the geometry of weight-update vectors rather than in model outputs. Update geometry responds to distributional shift immediately, within the same round in which the shift begins, providing an early-warning signal that output-level monitors structurally cannot.
2.3 Deep learning for energy load forecasting
Short-term electricity load forecasting (STLF) is a safety-critical grid operation task. The classical toolkit, ARIMA-family models (Box et al. 2015) for decomposing load into trend, seasonality, and residual, and exponential smoothing (Holt 2004) as the operational forecasting workhorse, remains widely deployed because of its interpretability and low data requirements, but is known to underperform under non-stationary demand patterns.
Deep learning displaced statistical baselines in STLF benchmarks beginning around 2016. Dong et al. (2025) provide the most comprehensive current survey of deep-learning-based STLF, tracing successive architectural waves; LSTMs for capturing temporal dependencies, CNNs for extracting local patterns, and attention-based models for long-range correlation, culminating in the Transformer as the dominant architecture family. Within Transformers, the iTransformer of Liu et al. (2024) achieves state-of-the-art performance on multivariate benchmarks by inverting the conventional token design making it particularly suited to the highly correlated, multivariate structure of cross-border energy load.
The architectural landscape continues to diversify rapidly. Kim et al. (2025) document a renaissance in time-series forecasting architecture, cataloguing the rise of hybrid models combining Transformers with Mamba state-space layers for long-context efficiency, diffusion-based probabilistic forecasters for uncertainty quantification, and large pre-trained time-series foundation models capable of zero-shot STLF. Critically, the survey identifies distribution shift as the most persistent unsolved challenge across all architectures, a finding that directly motivates the federated, regime-aware design of RAFA.
All of these methods, from classical ARIMA to the most recent foundation models, implicitly assume that training data can be pooled. For cross-border energy forecasting under GDPR and national grid-data confidentiality agreements, that assumption cannot be satisfied.
2.4 Federated learning for energy systems
The intersection of FL and power systems has grown substantially since 2024, driven by smart-meter rollouts, grid digitalisation, and tightening energy-data privacy regulations. Zhang et al. (2025) provide the first systematic survey of FL for smart grids, covering the full generation-to-consumption stack and documenting the principal vulnerabilities, such as poisoning, model inversion, and free-riding, that accompany FL deployment in critical infrastructure. Their analysis establishes that FL is no longer a theoretical proposal for energy systems. Production-grade FL systems are operating across all three grid stages, and the research agenda has shifted from feasibility to robustness and scalability.
At the forecasting layer, Rahman et al. (2025b) demonstrate personalised FL for residential load prediction over multi-hop AMI networks, developing a meta-learning aggregation strategy that jointly minimises forecast error and communication latency under heterogeneous device capacities. Rahman, Kumar, and Nguyen (2025a) extend this line of work to the MAML framework, showing that a single round of local fine-tuning on top of a collaboratively trained global model achieves better personalisation than either local-only or global training under realistic non-IID smart-meter distributions. Looking forward, Hosseinalipour et al. (2025) argue that the natural evolution of FL for grids is the adoption of federated foundation models. Large multi-task models pre-trained on heterogeneous grid data that can be fine-tuned at the edge for local forecasting, anomaly detection, and demand response simultaneously.
What is absent from all of this work is any mechanism to detect and respond to within-round
distributional change. Every method above, whether aggregation-based, meta-learning-based, or foundation-model-based, trains a fixed-architecture model against a stationary (or slowly varying) data distribution. None detects the regime changes, such as energy-crisis demand spikes, post-lockdown recovery, sudden renewable integration, that fundamentally alter the forecasting task mid-deployment. This is precisely the gap that RAFA addresses.
2.5 Distance-based novelty and out-of-distribution detection
RAFA’s regime-shift detector draws on a mature body of work on distance-based OOD detection. The foundational contribution is from Lee et al. (2018), who showed that the class-conditional Mahalanobis distance in a DNN’s penultimate representation space is a highly effective, training-free OOD score. It outperformed softmax confidence and density-based baselines across a wide range of in-distribution and OOD dataset pairs, and can be computed post-hoc without modifying the trained model. The geometric intuition is that the covariance-normalised distance captures deviation relative to the learned representation geometry rather than in raw feature space, making it invariant to feature scaling and directional anisotropy. Ren et al. (2021) strengthened the method by subtracting a background-class baseline from the Mahalanobis score, substantially improving near-OOD detection, the setting where the distribution shift is small and progressive, precisely as in seasonal load-regime transitions.
Recent work has pushed the OOD detection frontier in two directions. Gao, He, and Pu (2025) combine feature-space, logit-space, and softmax-space information via an optimal transport scoring function (OTOD), outperforming the Mahalanobis baseline by substantial margins on standard benchmarks, demonstrating that geometric and transport-based cues are complementary. Claros Olivares et al. (2026) provide the largest systematic comparison of OOD detection methods to date, using 30 post-hoc scoring functions across CNN and Vision Transformer encoders, and find that the learned feature-space geometry is the primary determinant of OOD efficacy; geometry-aware scores based on nearest-neighbour distances and prototype boundaries dominate under strong distributional shifts. This finding directly validates the design choice in RAFA; by operating on the update-vector geometry rather than the output space, the detector operates in the representation layer most sensitive to distributional change.
Computing a full Mahalanobis distance in the parameter space of a practical FL model is intractable; the parameter dimension typically exceeds
. Johnson-Lindenstrauss random projections (Johnson and Lindenstrauss 1984) resolve this by mapping update vectors to a low-dimensional subspace while provably preserving pairwise distances up to a
factor with probability
for a projection dimension of
. The resulting projection-then-Mahalanobis detector scales to arbitrary model sizes, adds no communication overhead beyond the standard FL update, and requires no labelled drift examples or held-out reference data.
3. Background and problem formulation
Before describing RAFA, we need to be precise about two things, namely what the federation is jointly optimising, and what it means for a client’s data-generating process to shift in a way the server cannot observe directly. The notation introduced here carries through the rest of the paper.
3.1 Federated energy forecasting
Consider a federation of
clients
. Each client
holds a private time series of hourly electrical load observations.
where
is a feature vector comprising lagged load values and calendar features (see Section 5.1), and
is the target load value at time
.
The goal is to collaboratively learn a shared global model
that minimises the weighted empirical risk:
where
is the local loss (MSE),
is the data fraction of client
, and
is the total number of samples.
3.2 Regime shift model
We denote the data distribution of client
at FL round
as
. A regime shift at client
occurs at round
if the Kullback-Leibler divergence between the current and reference distributions exceeds a threshold if
Because raw data is never shared, Equation (3) cannot be computed directly. Instead, RAFA detects regime shifts by monitoring the distribution of model update vectors, a proxy that is both privacy-preserving and sensitive to distribution changes, as shown empirically in Section 6.3.
Table 1 summarises the notation used throughout the paper.
Table 1. Notation summary.
| Symbol | Meaning | Symbol | Meaning |
|---|
|
| Number of clients |
| Total model parameters |
|
| Communication round index |
| Fixed random projection matrix |
|
| Total communication rounds |
| Reference mean and covariance |
|
| Local training epochs per round |
| Mahalanobis distance of client
|
|
| Global model at round
|
| Detection threshold (chi-squared quantile) |
|
| Client
’s local model at round
|
| Regime-shift flag for client
|
|
| Model update
|
| Reliability weight for client
|
|
| Data fraction of client
|
| Normalised reliability weight |
|
| Random-projected update vector |
| Penalty sharpness hyperparameter |
|
| Projection dimension (
) |
| Reference window length (rounds) |
4. Methodology for Regime-aware federated aggregation (RAFA)
RAFA starts from a direct observation, a client whose model updates diverge from the historical population is likely experiencing a distributional shift in its underlying data. Acting on that signal, without accessing raw data, is the central design challenge, and the three components below address it from complementary angles. The complete algorithm is presented in Section 4.4.
Figure 1 provides an overview of the full system. At each communication round, every client trains locally, computes a compact projected update vector
(1 KB), and sends both its model parameters and
to the server. The server maintains a rolling reference buffer of past projected vectors, uses it to detect regime-shifted clients via Mahalanobis distance, assigns reliability weights, and returns a weighted global model to all clients. Each client additionally fine-tunes a small personalised head locally, which is never transmitted.
Figure 1. Overview of the RAFA framework.
Each client sends model updates
and a compact projected vector
(1 KB) directly to the server. The server detects regime-shifted clients via Mahalanobis distance against a rolling reference buffer
, assigns reliability weights
, and returns a reliability-weighted global model
to all clients. Each client maintains a personalised head
locally (red dashed box) that is never transmitted to the server.
4.1 Mahalanobis regime detector
At each round
, every client
transmits a model update vector.
where
is the result of
local SGD steps on client
starting from the global model
.
Dimensionality Reduction via Random Projection
For neural network models with
parameters, computing the Mahalanobis distance in the full parameter space is intractable. We project update vectors into a low-dimensional space via a fixed random Gaussian projection matrix
:
where the update is normalised to unit norm before projection to remove the trivial scale dependence on learning rate and number of local steps. Entries of
are i.i.d.
, generated once at the server. By the Johnson-Lindenstrauss lemma (
Johnson and Lindenstrauss 1984), pairwise distances are preserved up to a
factor with high probability.
Reference distribution estimation
The server maintains a rolling reference distribution over the last
rounds. The reference mean
and diagonal variance
are updated at every round using all
clients.
where
is the window of recent client-round pairs and
is a regularisation constant. The diagonal approximation reduces memory from
to
and is motivated by the approximate independence of projected components under random Gaussian projections (
Johnson and Lindenstrauss 1984).
Distance computation and thresholding
The Mahalanobis distance of client
’s update at round
with respect to the reference is given by
A client is flagged as regime-shifted if its distance exceeds threshold τ, namely
The threshold
is set as the
-quantile of the chi-squared distribution with
degrees of freedom.
where
is the target false-positive rate. Under the assumption that
follows a multivariate normal with diagonal covariance
, the squared Mahalanobis distance follows a
distribution, giving
for
.
Warm-up phase
Detection is suppressed for the first
rounds while the reference window accumulates sufficient observations. During warm-up, shift flags are suppressed and personalisation-trigger counters are not activated; this avoids brittle early decisions while the reference distribution stabilises.
4.2 Reliability-weighted aggregation
Standard FedAvg aggregates client models according to
RAFA replaces the data-fraction weights
with reliability weights
that discount regime-shifted clients.
where
is a monotonically decreasing penalty function. We use a soft-threshold exponential function.
with hyperparameter
controlling penalty sharpness. For
(non-shifted clients),
and weights reduce to standard F
edA
vg. Weights are normalised to sum to one.
This gives the global update rule
This formulation has two key properties. (i) it reduces to exact FedAvg when no client is regime-shifted (
for all
); and (ii) a highly shifted client contributes negligible weight, protecting the global model from corrupted updates.
4.3 Always-on personalised head fine-tuning
After local training, each client fine-tunes only the final prediction layer (head) of its model copy at learning rate
for
additional epochs, with all base layers frozen:
where
denotes the model with base weights
and head weights
.
The personalised head
is not transmitted to the server; it remains local, preserving the privacy of regime-specific adaptation. The server aggregates only the base parameters
. Evaluation uses the personalised model
.
This decoupled design separates two concerns. Aggregation (what the server sees) is controlled by the detector via reliability weights; prediction (what each client reports) always benefits from local adaptation regardless of whether a regime shift is detected. This is structurally related to FedPer (Arivazhagan et al. 2019) and LG-FedAvg (Liang et al. 2020), but RAFA fine-tunes the head continuously rather than switching at a fixed point.
4.4 Complete algorithm
Figure 2 collects all three RAFA components, Mahalanobis regime detection, reliability-weighted aggregation, and personalised head fine-tuning, into a single, self-contained procedure. The algorithm operates in two phases. During the warm-up phase (rounds
), the server accumulates projected update vectors into the reference buffer
but has not yet seen enough history to distinguish normal variation from a genuine regime shift; detection flags are therefore suppressed to avoid unstable early-round decisions while the reference distribution is still forming. Once the buffer is full (
), the detection phase begins; at every round the server computes the reference mean and covariance from the most recent
rounds, scores each client’s current update via Mahalanobis distance, and derives reliability weights that soft-discount any client flagged as shifted. Meanwhile, each client fine-tunes its personalised head locally every round, regardless of phase; the head is never transmitted, so it adds no communication overhead. The complete round-by-round flow is given in
Figure 2.
Figure 2. The RAFA algorithm.
Regime-aware federated aggregation combining the Mahalanobis regime detector, reliability-weighted aggregation, and always-on personalised head fine-tuning. During warm-up (rounds r ≤ W) detection is suppressed while the reference buffer fills; from round r > W the server scores each client update by Mahalanobis distance and soft-discounts shifted clients. The personalised head φ_k is never transmitted.
4.5 Complexity and communication overhead
Compared to FedAvg, RAFA introduces additional computational and communication overhead. On the communication side, each client transmits an additional
floating-point values per round, corresponding to the projected vector
. For a typical setting with
and 32-bit floating-point representation, this amounts to approximately 1 KB per client per round. This overhead is negligible compared to a standard model update; for instance, it represents less than 0.3% of a typical upload size of around 60 KB for a model with 15,489 parameters.
From the server-side perspective, the computational cost is
per round, which includes both the reference update and the distance computations across clients.
On the client side, RAFA introduces additional training cost in the form of
extra head fine-tuning epochs, corresponding to approximately a 40% increase in local training time compared to FedAvg.
Finally, in terms of memory overhead, the server maintains a reference buffer of size
, while the covariance tracking component requires only
memory due to its diagonal structure.
4.6 Relationship to Byzantine-robust aggregation
Reliability weighting is structurally similar to Byzantine-robust aggregation methods such as Krum (Blanchard et al. 2017) and Trimmed Mean (Yin et al. 2018). A key distinction is intent. Byzantine methods defend against adversarial manipulation, whereas RAFA responds to benign regime shifts that are expected in real-world deployments. In particular, RAFA never fully excludes a client; even a highly shifted client retains a small positive weight, preserving the diversity benefits of federation.
5. Experimental setup
Evaluating a regime-aware federated algorithm requires a benchmark that genuinely exhibits regime shifts at scale. We build one from publicly available ENTSO-E load data, deliberately spanning the 2022 European energy crisis, a well-documented structural break in multiple bidding zones, and evaluate all algorithms on the post-crisis year as a demanding out-of-sample
test.
5.1 Dataset on the ENTSO-E Multi-National load benchmark
We construct a novel federated energy forecasting benchmark from the ENTSO-E Transparency Platform (ENTSO-E 2024). The dataset comprises hourly actual electrical load measurements for eight European bidding zones spanning six years (January 2019–December 2024), namely Netherlands (NL), Germany-Luxembourg (DE_LU), France (FR), Spain (ES), Portugal (PT), Belgium (BE), Austria (AT), and Poland (PL).
Table 2 summarises the key statistics of each client.
Table 2. ENTSO-E benchmark statistics per client.
W/S Ratio is the mean winter (Dec–Feb) to summer (Jun–Aug) load ratio.
is the mean coefficient of variation of the rolling 7-day standard deviation (temporal variability). Mean Load is the mean hourly actual load (MW) over 2019–2024.
| Client | Zone | Mean Load (MW) | W/S Ratio |
| Missing (%) |
|---|
| NL | Netherlands | 12,819 | 1.18 | 0.020 | 0.00 |
| DE_LU | Germany-Luxembourg
| 55,568 | 1.21 | 0.016 | 0.00 |
| FR | France | 52,016 | 1.48 | 0.018 |
0.02 |
| ES | Spain | 28,447 | 1.02 | 0.015 | 0.00 |
| PT | Portugal | 5,722 | 1.07 | 0.017 | 0.00 |
| BE | Belgium | 9,458 | 1.19 | 0.012 | 0.00 |
| AT | Austria | 6,773 | 1.28 | 0.016 | 0.00 |
| PL | Poland | 18,104 | 1.17 | 0.014 | 0.00 |
The benchmark exhibits three forms of heterogeneity that challenge standard FL, namely scale (10
range from PT to DE_LU); seasonality (FR’s W/S ratio of 1.48 vs ES’s 1.02); and temporal variability (NL highest CV vs BE lowest). The six-year span includes the 2022 European energy crisis, which introduced a structural regime shift across multiple clients.
Data splits
Each client’s data is split chronologically into three non-overlapping sets, namely (1) train for 2019–2022 (4 years,
35,064 hours), (2) validation for 2023 (1 year,
8,760 hours), and (3) test for 2024 (1 year,
8,784 hours). The test year (2024) is deliberately chosen to capture post-crisis normalisation dynamics, making it a challenging out-of-sample evaluation period. All normalisation statistics (
,
) are computed on the training split only; there is no data leakage between splits. The 8 missing values in FR’s series are filled by forward-fill.
Feature engineering
For each client we construct fixed-size feature vectors from the univariate load series using a sliding window of
hours. Each sample at time
comprises the following elements.
• 48 lagged normalised load values
where
;
• Hour-of-day encoding
,
;
• Day-of-week encoding
,
;
• Month encoding
,
.
This yields an input dimension of
features per sample. All model outputs and reported metrics are in original MW scale, obtained by applying the inverse normalisation
.
5.2 Model architecture
We use a 3-layer Multi-Layer Perceptron (MLP) as the shared local model across all clients and algorithms:
Total trainable parameters are
. This compact model is intentional. It ensures the federation is communication-efficient and results generalise beyond architecture-specific effects. For personalisation, the “head” refers to the final linear layer (65 parameters).
5.3 Baselines and hyperparameters
We compare RAFA against three baselines. FedAvg (McMahan et al. 2017) uses standard data-fraction-weighted model averaging with client weights
. FedProx (Li et al. 2020) extends FedAvg with proximal regularisation (
) to reduce client drift during local updates. The Local baseline trains each client independently, without any federated aggregation, and therefore provides a lower bound on the value of collaboration. For fairness, all methods share the same MLP architecture, feature set, data splits, optimiser, and learning rate.
RAFA introduces seven hyperparameters spanning its projection, detection, and personalisation components. The detection threshold
is derived analytically as
, requiring no held-out calibration set. All defaults are listed in
Table 3; their sensitivity is examined in Section 6.5.
Table 3. RAFA hyperparameter defaults.
| Parameter | Default | Description |
|---|
|
(proj_dim) | 256 | Random projection dimension |
|
(window) | 10 | Reference window (rounds) |
|
| 17.156 | Detection threshold (
) |
|
| 0.5 | Penalty sharpness |
|
| 0.05 | Target false-positive rate |
|
| 5 | Personalisation epochs |
|
|
| Head fine-tuning learning rate |
5.4 Federation training protocol and evaluation metrics
All FL algorithms (FedAvg, FedProx, RAFA) follow an identical training protocol to ensure fair comparison. Each experiment runs for
communication rounds with
local training epochs per round, and all
clients participate in every round (cross-silo setting with full participation). For the Local baseline, which has no communication rounds, each client is trained for an equivalent compute budget of
local epochs. Local optimisation uses the Adam optimiser with a learning rate of
and betas
, with a mini-batch size of 256 samples. All stochastic operations are seeded with 42 across both PyTorch and NumPy to ensure reproducibility. All experiments are run on CPU to eliminate hardware-specific variation and to confirm that RAFA is practical on standard academic infrastructure without requiring GPU acceleration.
We report the following metrics on the held-out test set (2024) for each client and macro-averaged across clients:
MAE is the primary metric as it is robust to outliers and directly interpretable in MW. RMSE penalises large forecast errors more heavily, relevant for grid balancing. All metrics are reported in MW (original scale).
6. Results
The evaluation proceeds from the coarse to the fine. Macro-averaged accuracy establishes the headline comparison; per-client figures then expose how performance is distributed across a heterogeneous federation. The Mahalanobis detector is examined independently to verify its calibration, and training dynamics are tracked across all 100 rounds to rule out transient effects. The section closes with four controlled ablation studies that decompose the contribution of each design choice.
6.1 Main forecasting results
Table 4 summarises macro-averaged test MAE and RMSE at the end of the training protocol on the held-out 2024 split.
Table 4. Test-set forecasting accuracy at the end of the training protocol.
Avg MAE and Avg RMSE are macro-averaged across all 8 clients.
= relative change vs. FedAvg; positive = improvement.
| Algorithm | Avg MAE (MW)
| Avg RMSE (MW)
|
vs FedAvg |
|---|
| Local | 204.0 | 272.1 |
|
| FedAvg
| 210.2 | 279.6 | — |
| FedProx (
) | 222.0 | 297.0 |
|
| RAFA (ours) | 185.4 | 246.1 |
|
RAFA is the only method to surpass the non-federated Local baseline (185.4 vs 204.0 MW), demonstrating that regime-aware aggregation allows federation to add value rather than merely dilute it. A globally trained model can outperform eight individually trained ones provided the aggregation properly accounts for distributional dynamics. The 11.8% margin over FedAvg (210.2 MW) persists at round 100, ruling out any transient warm-up advantage. Perhaps most telling is FedProx, where 222.0 MW falls below even the Local baseline, revealing that a proximal penalty calibrated for spatial heterogeneity becomes counter-productive when heterogeneity is primarily temporal and some client-update divergence carries genuine distributional signal.
6.2 Per-client performance
The per-client breakdown in
Table 5 reveals a clear coupling between distributional heterogeneity and the magnitude of improvement. The connection is sharpest for France; its W/S load ratio of 1.48, driven by widespread electric heating, forces the shared model to reconcile a steep winter peak with the near-flat demand curve of Spain, a task that naive averaging handles poorly; the personalised head corrects this mismatch, yielding a 30.5% reduction over FedAvg. At the other extreme, Portugal’s small and stable load profile (mean 5,722 MW) amounts to a well-behaved, low-variance regression target; even without strong seasonal swings, regime-aware aggregation trims its absolute error to 56.0 MW, 7.6% below FedAvg’s 60.6 MW. Germany-Luxembourg is the one exception, where RAFA (442.4 MW) is marginally below its Local baseline (438.9 MW), suggesting that for the highest-load client in the federation, collaborative training introduces a residual variance component that the 65-parameter head cannot fully absorb. FedProx trails FedAvg on six of the eight clients, reinforcing the view that proximal regularisation is ill-suited to this setting.
Table 5. Per-client test MAE (MW) at the end of the training protocol.
Bold = best per row.
| Client | Local | FedAvg
| FedProx
| RAFA |
|---|
| NL | 113.6 | 110.7 | 128.1 | 108.8
|
| DE_LU | 438.9
| 453.3 | 482.7 | 442.4 |
| FR | 424.2 | 495.6 | 479.6 | 344.6
|
| ES | 227.2 | 214.2 | 246.9 | 206.1
|
| PT | 73.4 | 60.6 | 72.2 | 56.0
|
| BE | 110.2 | 109.8 | 111.5 | 105.1
|
| AT | 72.4 | 72.7 | 74.5 | 68.4
|
| PL | 172.0 | 164.2 | 180.3 | 152.1
|
| Avg
| 204.0 | 210.2 | 222.0 | 185.4
|
6.3 Mahalanobis detector behaviour
Figure 3 tracks
for all eight clients across 100 rounds, with the analytically derived threshold
drawn as a horizontal reference.
Figure 3. Per-client Mahalanobis distance vs. communication round.
Dashed line indicates the detection threshold
. Shaded region (rounds 1–10) is warm-up phase.
Figure 4. Distribution of per-client Mahalanobis distances (post warm-up).
Dashed line is threshold
.
The warm-up phase (rounds 1–10) is visible as an initial elevation in all curves; with fewer than
vectors in the reference buffer, the running covariance estimate is noisy and distances are inflated; NL reaches
before settling once the window fills. Beyond round 10 all clients enter a stable regime, oscillating between
15 and 18 without sustaining a threshold breach across consecutive rounds for extended periods; a result that is consistent with the 2024 test year capturing a post-crisis normalisation period rather than a new structural shock. Client-level differences in steady-state distance reflect genuine variation in temporal load dynamics; Austria (mean 16.87), Netherlands and Portugal (both 16.80) show the greatest round-to-round update diversity, while Poland (15.62) operates furthest from the threshold, in line with its stable, industry-driven load profile. France and Spain occupy nearly identical positions (16.08 each), a coincidence that underscores how different seasonal mechanisms can produce similar update variability.
The aggregate distance distribution (Figure 4, post warm-up) provides a complementary view. The mass of observations sits below
, and threshold crossings are isolated rather than clustered, indicating that the detector is well calibrated and avoids systematic over-flagging.
6.4 Convergence curves
Figure 5 plots macro-averaged validation MAE for the federated algorithms (FedAvg, FedProx, RAFA) across the full 100-round training run. RAFA reaches 199.7 MW by round 10, already 16.9% below FedAvg’s round-10 error of 240.3 MW, an advantage that at this stage is primarily driven by the personalised head, since the Mahalanobis detector is still in its warm-up phase (rounds 1–10) and shift flags are not yet used for stability decisions. The advantage widens and stabilises; by round 100 RAFA converges to 185.0 MW on validation, a level FedAvg never reaches across the full training run (final validation was 210.4 MW), confirming a structural rather than transient gain. FedProx exhibits the slowest and most erratic trajectory, oscillating between 220 and 250 MW and ending above FedAvg; this pattern is consistent with its proximal penalty stiffening updates in a setting where some client drift carries genuine distributional signal. The Local baseline is reported in
Tables 4 and 5; it is omitted from
Figure 5 because its optimisation timeline is epoch-based rather than round-based.
Figure 5. Macro-averaged validation MAE (MW) vs. communication round for FedAvg, FedProx, and RAFA.
Curves are reported at the evaluation checkpoints used throughout training.
6.5 Ablation studies
Four one-at-a-time sweeps examine the sensitivity of RAFA to its principal hyperparameters, holding the remaining three at their defaults (
,
,
,
) throughout. Because each sweep is an independent experimental run with a fixed random seed, the measured value at the default setting can differ by a small margin from the main experiment figure (185.4 MW) due to stochastic training variation; the default row in each table is therefore anchored to the main experiment result (marked
) to provide a consistent frame of reference.
6.5.1 Effect of penalty sharpness (
)
The penalty sharpness
controls how aggressively shifted clients are down-weighted;
Table 6 sweeps it across one order of magnitude.
Table 6. Effect of penalty sharpness
on test MAE.
|
|
Avg test MAE (MW) |
|---|
| 0.1 | 188.1 |
| 0.5† | 185.4 |
| 1.0
| 181.7
|
| 2.0 | 227.1 |
The curve is notably flat between
and
, with MAE ranging only from 188.1 to 181.7 MW; this plateau indicates that the framework is not sensitive to the precise value of
within this range. Pushing to
breaks this stability dramatically (227.1 MW,
); once aggregation weights collapse toward zero for even mildly shifted clients, the federation effectively fragments, and each client is left learning from a narrow local sample. The default
is a deliberately conservative choice, sitting squarely in the flat region and therefore suitable for deployment without further tuning.
6.5.2 Effect of projection dimension (
)
The projection dimension
controls the fidelity of the Johnson-Lindenstrauss embedding and has a direct cost in communication overhead (0.25, 0.5, 1.0, and 2.0 KB per client per round for
, respectively); the accuracy-cost trade-off across this range is quantified in
Table 7.
Table 7. Effect of projection dimension
on test MAE.
|
| Avg Test MAE (MW) | Extra comm. (KB/round) |
|---|
| 64 | 187.2 | 0.25 |
| 128 | 186.9 | 0.50 |
| 256† | 185.4
| 1.00 |
| 512 | 238.8 | 2.00 |
Performance peaks at
and degrades in opposite directions from there. Reducing the dimension to 64 or 128 costs less than 1.0 MW in MAE (less than 1% relative degradation), making these smaller projections attractive when bandwidth is scarce. Increasing to
is actively harmful (
); with only
clients contributing observations per round, fitting a 512-dimensional reference covariance is severely underdetermined, yielding noisy distance estimates that destabilise the detector.
6.5.3 Effect of reference window (
)
The reference window
sets the memory horizon of the detector; larger values provide smoother covariance estimates at the cost of slower response to real shifts.
Table 8 explores this trade-off.
Table 8. Effect of reference window size
on test MAE.
|
|
Avg Test MAE (MW) |
|---|
| 5 | 184.4 |
| 10† | 185.4 |
| 20
| 179.6
|
| 40 | 237.6 |
Extending the window from 10 to 20 rounds improves MAE from 185.4 to 179.6 MW by reducing noise in the covariance estimate; even a minimal
remains competitive at 184.4 MW, evidence that the detector retains useful signal with only a short burn-in. The failure mode at the opposite extreme is instructive;
inflates MAE to 237.6 MW (
) because the reference distribution accumulates stale observations that span the 2022 energy crisis, blurring the boundary between pre- and post-crisis regimes and rendering the detector unable to flag the shift when it actually occurs. In practice,
represents a reasonable operating range, balancing detection responsiveness against covariance stability.
6.5.4 Effect of personalisation epochs (
)
Setting
effectively turns off personalisation, leaving reliability-weighted aggregation as the sole active mechanism; the resulting 187.5 MW already represents a 10.8% gain over FedAvg, establishing that the aggregation component is the primary driver of improvement and not merely a support structure for the personalised head. The sweet spot is
; a single fine-tuning epoch per round is sufficient to capture local regime patterns and pushes MAE down to 182.6 MW, while more prolonged adaptation (
, 215.7 MW;
, 208.2 MW) causes the head to overfit recent local data, eroding the cross-client knowledge encoded in the shared base.
Table 9 reports the full sweep.
Table 9. Effect of personalisation epochs
on test MAE.
= no personalisation (aggregation only). Bold = best result.
|
| Avg Test MAE (MW) | vs main result (
) |
|---|
| 0 (no pers.) | 187.5 |
|
| 1
| 182.6
|
|
| 3 | 215.7 |
|
| 5† | 185.4 | — |
| 10 | 208.2 |
|
7. Discussion
Two questions are worth examining beyond the headline numbers, namely why the gains appear where they do, and what the sensitivity analysis implies for anyone looking to deploy RAFA in practice.
7.1 Why does RAFA outperform FedAvg?
Understanding why RAFA works matters as much as the numbers themselves; it tells us which design choices are actually load-bearing and which could in principle be simplified. The gains arise from two complementary mechanisms.
When clients undergo seasonal or crisis-driven regime shifts, their model updates veer away from the population reference in parameter space. FedAvg treats all updates equally regardless, which means a handful of simultaneously-shifting clients can drag the global model off course. RAFA instead soft-discounts shifted clients in proportion to how far their updates have moved, protecting the global model during the shift while restoring their full weight once conditions normalise.
Each client’s prediction head is continuously fine-tuned to its local regime at a reduced learning rate, using only 65 parameters. The
ablation shows that reliability-weighted aggregation alone achieves 187.5 MW, a 10.8% improvement over FedAvg, while the optimal
reduces MAE further to 182.6 MW, confirming that both mechanisms provide complementary, independently valuable gains.
7.2 Heterogeneity and per-client gains
The per-client results match what the design intent predicts. France with its pronounced electric heating peak (W/S ratio 1.48) benefits most from personalisation (
over FedAvg), while Spain’s near-flat profile (W/S ratio 1.02) needs it least. Clients with uniform, predictable load profiles, such as Belgium, Austria, benefit less, not because RAFA hurts them, but because FedAvg already serves them reasonably well. The personalisation mechanism is surgically useful. It helps most precisely where standard FL struggles most.
7.3 Hyperparameter sensitivity
The ablation studies reveal a consistent pattern; RAFA is robust within a moderate range for all four hyperparameters but degrades sharply at the extremes.
performs similarly;
drives weights near zero for multiple clients and causes a 22.5% degradation.
is optimal;
is severely underdetermined given only
clients (
).
slightly outperforms
(
), but
misses the 2022 crisis shift entirely (
), highlighting the trade-off between reference stability and detection responsiveness. Practitioners should prefer
,
,
, and
; the defaults used in the main experiment are conservative rather than optimistic.
7.4 Limitations and future work
The experiments use a shallow MLP for reproducibility; whether deeper architectures benefit differently from the personalised head is an open question, although the aggregation mechanism itself is architecture-agnostic. Full client participation is assumed every round; handling stragglers or clients with intermittent connectivity would require extending the reference buffer logic. The 2022 energy crisis supplies a naturalistic but confounded regime shift. A controlled synthetic injection experiment would more cleanly validate the detector’s sensitivity and specificity.
Promising directions include probabilistic forecasting (normalising flows or conformal wrappers for uncertainty quantification), incorporation of exogenous features (weather, holiday calendars), and a formal non-convex convergence proof.
8. Conclusion
Energy systems do not stay still, and federated algorithms that assume they do pay for it in accuracy. RAFA takes a different stance. By monitoring how each client’s model updates relate to the historical population reference, without ever accessing underlying raw data, it detects regime shifts as they occur and responds by re-weighting client contributions. A lightweight personalised head then adapts each client’s predictions to its local conditions, independent of the aggregation logic.
On the ENTSO-E multi-national benchmark, RAFA achieves a test MAE of 185.4 MW, an 11.8% improvement over FedAvg (210.2 MW), with the largest gains where distributional heterogeneity is strongest; France (
, W/S ratio 1.48) and Poland (
). Ablation studies confirm both mechanisms contribute independently, and the detector correctly remains inactive for stable clients, avoiding false positives. The code and benchmark data download script are publicly available at https://github.com/Mahmoud-Abbasi-svg/rafa-federated-energy.
Ethics and consent
Ethical approval and consent were not required.
