Briefly, FLEXGRID focuses on all the energy sector stakeholders in modern smart grids and proposes an innovative smart grid management architecture through which they can interact efficiently in order to optimally plan and operate their grids (transmission and distribution networks) and their DERs. Figure 1 depicts the high-level functional architecture of FLEXGRID. The red boxes depict stakeholders combined with their existing functionalities while the purple boxes depict innovative functionalities (digital services) that FLEXGRID develops. Arrows in Figure 1 try to highlight interaction and data exchange which occurs sometimes between existing and FLEXGRID’s functionalities and sometimes among FLEXGRID’s functionalities.
From the grid management perspective, FLEXGRID develops innovative energy market architectures able to ensure the optimal use of DERs through advanced market clearing algorithms (dispatch and pricing). In order to achieve this, the proposed architecture guarantees an efficient interaction between TSOs (which operate TN and interact with MO to conduct market clearing) and DSOs (which operate DN and interact with FMO to conduct market clearing). More specifically, the Market Operator (MO, which is responsible for the efficient and network-aware market clearing at TN-level) interacts with Flexibility Market Operator(s) (FMOs, which are responsible for the efficient and network-aware market clearing at DN-level which includes operating reserves for frequency control and availability reserves for congestion management).
To develop innovative energy market architectures, FLEXGRID considers the “market” domain, in which the MO and FMO collaborate to improve operation and coordination of the wholesale (TN-level) and local (DN-level) energy markets, as shown in the Figure 2. Meanwhile, in the “network” domain, the TSO-DSO collaboration achieves improved network operation outcomes. The goal of FLEXGRID is to provide attractive trade-offs between optimal market and network operations, i.e., between economic efficiency and reliability, in a future with highly distributed RES penetration scenarios.
From the perspective of a DER investor, FLEXGRID accurately models modern energy sector stakeholders, often referred to as Energy Service Providers (ESPs), which are competitive stakeholders owning (or operating on behalf of end users / customers) various types of DER assets that interact with the DN and/or TN. ESPs are able to participate in various energy markets to ensure their financial sustainability by providing competitive energy services (such as trade of capacity, energy and flexibility services). Thus, FLEXGRID provides to ESPs the necessary intelligence for optimal and robust planning of their investments in DERs and to schedule their operation in an efficient way according to the various market needs. ESPs own DERs which are compatible (in timing and bidding format) with today’s existing markets, so these DERs can be introduced easily in these markets.
From the perspective of the interaction between modern smart grids and end users, FLEXGRID focuses on the effective DN level aggregation of flexibility assets from end users and their efficient exploitation. FLEXGRID develops management tools for a specific type of ESP (denoted as FSP/Aggregator). FLEXGRID provides intelligent retail pricing and auction algorithms to independent FSPs to efficiently interact with end users through a B2C flexibility market. In addition, it provides algorithms that facilitate: i) the optimal planning of the quantity and the location of DERs (flexibility assets) according to the needs of various markets, and ii) the efficient scheduling of those DERs (optimal participation in and across various energy markets). All these constitute FSPs financially sustainable. It should be noted that the main difference between an ESP and an FSP is that the latter actor represents DERs in the DN which belong to its customers (i.e., end energy prosumers) and are possibly not compatible with existing energy market architectures. In contrast, an ESP owns assets in the TN which are compatible with the existing markets.
Finally, FLEXGRID develops functionalities necessary for TSOs and DSOs in the modern market architecture. In more detail, FLEXGRID’s architecture enables the interaction between DER investors (i.e. ESPs, FSPs) and system operators towards the design of more profitable investments through TN- and DN-level information exchange. Furthermore, FLEXGRID provides intelligence for network upgrades to DSO/TSOs according to the market and ESP/FSP needs. In this way, FLEXGRID guarantees that in this modern environment, malicious ESPs/FSPs will not be able to exercise market power and jeopardize market efficiency, while the DSOs/TSOs will be able to minimize their costs for future grid reinforcements.
2. Novel energy market architectures for integrated management of transmission and distribution network
Existing electricity markets do not consider the constraints of local distribution networks. Thus, unpredicted voltage deviations and line congestions may arise. In particular, the models used in the existing markets, do not take into account power balance, power constraints and reactive power in distribution networks. As a result, re-dispatches may be necessary, which imply that: i) expensive units will have to participate in the DN dispatch and ii) higher penetration of DERs is constrained.
Novel DN models and recent advances in modern pricing algorithms allow the efficient and stable management of the DN. By exploiting such models, FLEXGRID evolves the existing energy market architectures in order to not only enable large scale and distributed RES penetration in DN, but also to increase the efficiency and the investment opportunities in modern smart grids4,6–8.
To this end, FLEXGRID introduces the novel concept of “Distribution Level Flexibility Market - DLFM”, which is operated by an independent entity, referred to as Flexibility Market Operator (FMO). In FLEXGRID, NODES9 is the FMO that operates the proposed DLFM. In this context, FLEXGRID focuses on the development of a digital Automated Trading Platform (ATP) that facilitates FMOs to: i) operate the DLFM and interact with existing energy/balancing markets operated by MO/TSO, ii) acquire flexibility requests from DSOs, and iii) interact with ESPs and FSPs by receiving flexibility offers.
The proposed FLEXGRID energy market architectures develop, combine, and bring to interaction the following six (existing or innovative) energy markets compiled in Table I. It should be noted that FLEXGRID follows the Nord Pool paradigm currently operating in the many European countries as EU’s regulatory baseline.
FLEXGRID developments unfold around three different flexibility market architectures. FLEXGRID puts emphasis on the trade-off between: i) social welfare maximization/ market efficiency, ii) the level of compatibility of the proposed architecture with the existing markets’ architecture (i.e., day ahead, balance), iii) their efficiency for various energy sector stakeholders (e.g., ESPs, FSPs, market/system operators, etc.).
These three architectures are described below. The first architecture (A) is framed within existing markets (#1, #2, #5 according to Table 1), but offers the maximum possible smart grid efficiency by allowing market participants to facilitate the transmission or the distribution network at the same time, independently of their location. The second (B) acts reactively to the existing energy markets and in this way sacrifices efficiency (market participants in distribution network are not able to use transmission network and interact with wholesale market) for compatibility. The third one (C) acts proactively to the existing energy market.
Table I. Summary of markets assumed within FLEXGRID.
| Market #1 | Market Operator (MO) operates the day-ahead energy market at the Transmission Network (TN) level |
|---|
| Input | MO receives bids from all market participants and basic power flow constraints at the TN level. Depending on FLEXGRID’s
various energy market architectures selected from the FLEXGRID’s set of options (see below), the market clearing process
of Market #1 either ignores the DN topology, or implicitly takes it into account (output of Market #3). |
| Output | Market clearing results (TN-aware price €/MWh per TN node and Day Ahead Dispatch (DAD) |
| Market #2 | TSO operates the day-ahead reserve market at the TN level |
|---|
| Input | Receives bids from all market participants at TN level, DAD, schedules from MO, RES/demand forecasts, maintenance-
related info from assets and grid, TN model/topology and constraints
Depending on FLEXGRID’s various energy market architectures (see below), the market clearing process of Market #2
either ignores the DN topology, or implicitly takes it into account (output of Market #4) |
| Output | Reserve market clearing results at TN level (price €/MW and reserve capacity commitment per accepted market
participant) |
| Market #3 | Flexibility Market Operator (FMO) operates the day-ahead energy market at the Distribution Network (DN) level |
|---|
| Input | Receives bids from all market participants at DN level, and DN topology constraints.
DAD schedule from MO at all TSO-DSO coupling points or respective day-ahead energy market price forecasts
(depending on the selection of energy market architecture from the three FLEXGRID options below). |
| Output | Market clearing results (DN-aware price €/MWh and DAD per accepted market participant and DN node) |
| Market #4 | DSO operates the day-ahead reserve market at the DN level |
|---|
| Input | Receives bids from all market participants at DN level, DAD schedules from FMO, local RES/demand forecasts, and
maintenance-related info (if any) |
| Output | Reserve market clearing results at DN level (price €/MW per DN node and reserve capacity commitment per accepted
market participant) |
| Market #5 | TSO operates the balancing energy market at the TN level |
|---|
| Input | Receives bids from all market participants at TN level (incl. DER aggregators), updated RES/demand forecasts, updated
data from SCADA/EMS.
Depending on the energy market architecture selected from the FLEXGRID’s various options, the market clearing process
of Market #5 either ignores the DN topology, or implicitly takes it into account through the output of Market #6. |
| Output | Balancing energy market clearing results at TN level (i.e. prices €/MWh per TN node, and Up/Down activation energy
quantities per accepted market participant) |
| Market #6 | DSO operates a real time flexibility market at the DN level (which ensures congestion avoidance and
frequency control in the DN level) noted as DN level balancing market. |
|---|
| Input | Receives bids from all market participants at DN level, updated local RES/demand forecasts, and updated data from
DMS10 |
| Output | Real time flexibility market clearing results at DN level (i.e. prices €/MWh per DN node, and Up/Down activation energy
quantities per accepted market participant) |
A. A clean slate approach towards a market-based smart grid operation with optimal social welfare – Interactive DLFM (I-DLFM)
Novel smart grid architectures which are able to maximize social welfare (through efficient markets) lead to: i) energy with lower cost for consumers, ii) more revenue streams for Energy and Flexibility Service Providers (ESPs/FSPs), and iii) lower operation/management costs for network operators (i.e. TSO and DSOs). In a smart grid with high and distributed RES and high flexibility exploitation in which the distribution network faces congestion and voltage issues, an evolved energy market architecture through an advanced interaction between MO (TSO) and FMO (DSO) is needed. In this perspective, a market architecture that evolves Markets #1, #2 and #4 and is not constrained to be compatible with their existing versions can theoretically maximize social welfare.
In the proposed Interactive DLFM (I-DLFM) model, FLEXGRID considers an iterative process that takes place between the MO and FMO until they converge to an optimal dispatch schedule at both the TN and DN levels.
According to the market definitions of Table I, FLEXGRID assumes three basic markets at both TN and DN levels: A) Day-head energy markets (interactive clearing of Markets #1 and #3), B) Day-Ahead Reserve markets (interactive clearing of Markets #2 and #4), and C) near-real-time Balancing energy markets (interactive clearing of Markets #5 and #6). For example, in the day-ahead energy market context, presented in Figure 3, the MO initially runs an instance of its market clearing problem at the TN level and sends the results to the FMO. Then, the FMO takes the MO’s results as input and runs its own market clearing problem at the DN level. The respective results (e.g., Lagrange multipliers) are sent back to the MO, who runs another round of the TN-level market clearing. Of course, the dispatch schedules that are decided in each round of the algorithm’s execution are virtual and are not actuated in reality. After several algorithmic iterations (i.e. several message exchanges between MO and FMO), the process converges (through the use of optimization theory) to an overall dispatch schedule (i.e. at both TN and DN levels) that maximizes social welfare. A similar iterative process may take place for day-ahead reserve markets and near-real-time balancing markets (cf. TSO-DSO collaboration).
Figure 3. FLEXGRID’s I-DLFM architecture.
Figure 3 presents the aforementioned interactive market clearing process of a unified energy market, in which stakeholders in the DN (i.e., FSPs, ESPs) and the TN (ESPs) are able to trade energy. Briefly, the core of the proposed market architecture is a unified market clearing based on an iterative process (cf. yellow arrows) between the MO managing the TN and the FMO managing the DN.
Figure 4. I-DLFM steps.
At each iteration of this process, and according to the bids of the TN-level market stakeholders, the MO derives a time series (according to the scheduling horizon) of prices, noted as Transmission Network Locational Marginal Prices (TLMPs) for each node in the TN. These nodes include the interface nodes through which each DN exchanges power with the TN. An FMO operating in a certain DSO area takes as input: i) TLMPs that the MO derived, and ii) the bids of the DN-level market participants. In a second step, the FMO derives a time series of power flows (Distribution Network Dispatch –DND) in each node of the DN and updates the coupling point (DN-TN connection nodes) power flow time series. The termination condition of this iterative process is an identical dispatch in TN and DN in two consecutive iterations (with respect with an accuracy threshold).
According to the final dispatch, the pricing in the TN is coherent with the existing pricing policy in today’s smart grids (TLMPs) and the pricing in the DN is conducted through a clearing algorithm that the FMO executes.
The rest of this section presents in detail the necessary steps for the operation of the proposed Interactive Market Clearing Algorithm (IMCA):
Step 1: The DSO sends its distribution network data to the FMO.
Step 2: The FSPs, connected to the DSO, send their flexibility asset (e.g. ESS, DSM) bids (i.e. FlexOffers) to the FMO. Each FlexOffer includes the cost/utility function of the FSP and its operating constraints.
Step 3: The Producers (ESPs) connected to distribution network (e.g. RES, prosumers) send their bids to the FMO.
Step 4: The Consumers (ESPs) connected to the distribution network (i.e., demand aggregators) send their bids to the FMO.
Step 5: The MO generates a forecast of the TLMPs for the first iteration of the IMCA
Step 6: In each iteration k of MCA
Step 6a: Taking the TLMPk (TLMPs in kth iteration of IMCA) at the coupling point, the FMO generates a Distribution Network Dispatch denoted by DNDk (kth iteration of IMCA) for each specific DSO area through the execution of the DND Algorithm (DNDA).
Step 6b: The DLFM stakeholders are compensated for their operation according to a Distribution Network Payment Algorithm (DNPA). A detailed discussion of the DNDA and DNPA requirements and a description of the related FLEXGRID developments is included in the Section IV.
Step 6c: The TN stakeholders (i.e. generators, demand aggregators, etc) decide their dispatch based on the corresponding TLMPk. The TSO calculates its power flows based on the zonal TLMPk, which along with the TN-level stakeholders and the FMOs’ decisions formulate the TN Dispatch (TND).
Step 6d: If the DND and the TND remain the same between two consecutive iterations of IMCA, then IMCA terminates. Otherwise, TLMPk are calculated by a TLMP Update Algorithm (TLMP-UA) based on the previously computed dispatches and TLMPk). FLEXGRID develops various TLMP-UAs mainly based on duality theory10 and decomposition techniques11 to guarantee their convergence.
Step 7: The last calculation of TLMPs and the last calculation of DNPA determine the payments of participants in the TN and DN, respectively. The last TND and DND solutions/schedules determine the dispatch in the two networks.
As this market is incompatible with the existing energy market architectures, FLEXGRID implemented this market in simulations in order to realize its advantages and quantify the disadvantages (through comparisons) over the two other markets that are proposed below (reactive and proactive DLFM).
B. A wholesale market-compatible and distribution network-aware market – Reactive DLFM (R-DLFM)
In contrast to the I-DLFM, the objective of the R-DLFM architecture is to be compatible and capable to interact with the existing TN-level markets (cf. #1, #2 and #5 in Table 1). In case that the R-DFLM operates right after day ahead market, it is capable to deal with: i) congestion issues at DN level that DAD cannot capture, and ii) forecast inaccuracies in energy production and energy consumption of DN- and TN-level assets.
The drawback of this approach is the possibility of infeasibility, or the need for mandatory/forced curtailments, or financially unsustainable distribution of DAD due to costly flexibility assets. All these may lead to discontented producers/consumers. Furthermore, in cases in which DAD is modified, the spot market price (cf. market #5) at the transmission level have to be paid. Finally, the absence of joint optimization between transmission and distribution levels leads to lower market inefficiency, which in turn deteriorates the economic viability of the participating stakeholders.
The sequence and timing of the markets in the R-DLFM energy market architecture is described in Figure 5. Initially the operation of the Day Ahead (#1) and the Reserve Market (#2) take place. Then FMO takes the output of these two markets and clears the R-DLFM (#3) according to: i) DSO’s network topology and constraints (which generate FlexRequest) and ii) FSP and ESP bids (FlexOffers). A reserve market at DN level could optionally take place in this phase through the same procedure (#4). In the next phase the outputs of DAD and R-DLFM are given as input to the Balancing Market (#5). In the optional case that DSO wants to handle voltage limits due to RES in DN a balancing energy/capacity market at the DN level (#6) may also take place right after clearing the TN-level balancing energy market.
Figure 5. FLEXGRID’s R-DLFM architecture.
Figure 6 illustrates the steps of the R-DLFM market clearing process:
Figure 6. R-DLFM steps.
Step 1: The FMO takes as input the DAD that is composed from the power flows in the coupling point with the TSO and the dispatch that concerns producers and consumers in its distribution network.
Step 2: The DSO sends its DN data to the FMO.
Step 3: The FSPs and ESPs connected to the DSO send their flexibility bids to FMO.
Step 4: The FMO generates DND (execution of a DNDA).
Step 5: Flexibility assets are compensated for their operation according to a DNPA that the FMO executes.
C. Feasibility check and optimization of wholesale market bidding through a proactive distribution network-aware market - Proactive DLFM (P-DLFM)
In order to mitigate the drawback of the aforementioned architecture (namely, the difficulty to manage an infeasible (DN level) or expensive (TN level) market clearing), FLEXGRID proposes a proactive clearing of bids in the DN by the FMO before the MO clearing. In this way, it ensures an a-priori feasible dispatch of the DN-level DERs.
In order to allow the FMO to operate proactively, an accurate estimation of the TN-level market clearing prices (Markets #1, #2 and #5) at the TSO-DSO coupling point is required. An underestimation in TLMPs may result in a lower demand than calculated by TN level markets. An overestimation in TLMPs may lead to an over-supply of generation dispatch.
As shown in Figure 7, the sequence of markets starts with Market #3 operated by the FMO (DA DLFM - first row of Figure 7). In this phase, the P-DLMP is a day-ahead energy market at the DN level (ESPs and FSPs in DN level bid in this market). Right afterwards, in the next phase, the DA Energy Market (#1), second row of Figure 7, and the DA Reserve Market (#2) - third row of Figure 7 - at TN level close. The output of Day-Ahead P-DLFM acts as input to the Day-Ahead market at TN level. Finally, for near-real-time balancing markets, the DSO runs a proactive balancing market (Market #6)) -fourth row of Figure 7 - right before the established balancing capacity market operated by the TSO (Market #5)) - fifith row of Figure 7. Thus, the local congestion and voltage problems at the DN level can be directly solved by the DSO locally, at the DN level, while the “remaining bids” of “Balancing P-DLFM” (Market #6) can be used as input to the Balancing Market (Market #5). The Balancing P-DLFM acts as a balancing market at DN level and takes as input ESP and FSP bids at that level. It additionally propagates its clearing prices (output) into the TN level balancing market which may facilitate balancing at DN level in case of inadequate balancing resources.
Figure 7. FLEXGRID’s P-DLFM architecture.
Figure 8 shows the steps of the P-DLFM market clearing process and its interaction with TN level markets:
Figure 8. P-DLFM steps.
Step 1: The DSO sends to the FMO information that suffices to model its distribution network.
Step 2: The FSPs connected to the DSO send their flexibility bids to the FMO.
Step 3: The Producers (ESPs) connected to the DSO send their production bids to the FMO.
Step 4: The Retailers (ESPs) connected to the DSO send their consumption bids to the FMO.
Step 5: The FMO (or any other party) generates a forecast for the TLMPs for the coupling point at which the DSO is connected.
Step 6: The FMO generates the DND through the execution of an algorithm noted as DNDA (may reduce the quantity in the initial bids, but it does not reduce the bidding prices).
Step 7: The TLD uses DNDA results as input bids.
Step 8: After the TLD, the DNPA uses as input: i) bidding prices of flexibility providers, producers and consumers, ii) TLD, and iii) DLD in order to derive the compensations for all the stakeholders that bid in Steps 2–4.
Finally, we highlight that in case of several local DSOs which are connected to different TN nodes of one TSOs’ network the proposed architectures will be identical and functional. On the other hand, the case in which several local DSOs are connected to several TSOs is out of the scope of this work but it is a very useful research challenge.
