Plain language summary
In the future the energy we consume will increasingly be generated using renewable sources like solar and wind. These are, however, very dependent on external factors such as the weather and therefore it is not always possible to exactly match this type of supply with the current energy demand. Therefore, if we want to avoid using large batteries or non-renewable backup power plants the only solution is to adapt how much energy we are consuming at any point in time. This is called demand response.
On the other hand we are used to be able to consume as much energy as we want anytime we want to in our homes. However, most people are willing to be flexible and shift their energy use if it helps to reduce carbon emissions. This can be supported by sending messages to consumers informing them when they should be using more or less energy. The decision whom to send a message to and when needs to be guided by an understanding of the potential impact this would have on the energy grid as a whole. Since everyone is free to react or ignore a message it is important for such a system to model the likelihood this will happen. These models are what we call a behaviour digital twin (BDT).
The BDT presented in this paper is based on psychological theories of intention and motivation and aims to provide the required information by asking a set of simple questions and combining this with the actual behaviour of every participant.
Introduction
Energy is a valuable resource which has a major impact on our environment depending how it is sourced, distributed, and used. Therefore, many different approaches are currently developed and tested to decrease energy consumption and improve energy distribution1. For example, the Clean Energy Package2 recently released by the EU aims at a more efficient decarbonisation and a better integration of renewable sources into the energy system. To do so not just the production side of energy needs to be considered but also the consumer or end-user of energy and here in particular the residential sector needs to be highlighted. The end-users have a certain energy demand to which the production side needs to respond. This is called demand-response. Many consumers want to actively participate so that the demand side is in balance with the supply side, to avoid over-production, and reduce energy consumption by improving their individual energy behaviour. Manually responding to market signals such as a change in electricity prices or other financial incentives to increase/decrease/shift the timing of their electricity consumption is the classical demand-response approach3. By also including psychological parameters such as intention and motivation this concept is extended and can be defined as behavioural demand-response. To optimally steer the behaviour of residential energy users individual variables can be assessed to optimally influence participants to behave in a positive way.
To approach behavioural demand-response we propose to extend the concept of a digital twin and introduce a behavioural digital twin (BDT), which will be developed starting from well-established and empirical proven behavioural theories originating from the area of psychology and the social science suitable for applications in residential demand response. The models are operationalised in a probabilistic framework to enable the prediction of relevant behavioural and motivational parameters such as the behavioural intention4 defined in the theory of planned behaviour and the relative autonomy index5 defined in the self-determination theory. Building on these a model calibration procedure is presented that aims at refining consumer profile predictions and facilitate sending messages to the right person at the right time to support the behaviour in question in a desired way, while at the same time optimising the overall residential demand-response system.
Digital twin
To develop a BDT it is necessary to understand what the original concept of a Digital Twin (DT) refers to. A DT has been defined as "(...) a digital representation of an entity, including attributes and behaviours, sufficient to meet the requirements of a set of use cases, that is, they are virtual representations of physical assets”6. “Digital Twins consist of three major components: a physical object, a corresponding virtual object, and a continuous bi-directional data stream for transferring data from the user to the system and feedback from the system back to the user”7,8.
Artificial intelligence (AI) approaches are used in many DT applications to continuously monitor and control the physical twin and its status to influence and optimise its performance. There is a continuous exchange of data between the digital side and its physical counterpart using AI algorithms to synchronise and optimise the activities of the physical entities9. The DT continuously simulates the conditions of these physical entities, taking all relevant external environmental variables into account predicting the future status and supporting the decision-making process to trigger the right intervention at the right point in time7,10.
Up until now DT applications have shown good results particularly in the area of manufacturing. Another application domain is the area of healthcare, where Human digital twins (HDT) were developed to map and mirror humans and in particular patients’ health to support and develop optimal treatments7,8. One big shortcoming of HDT compared to the original DT concept11 is, that there is no continuous connection between the digital and physical side and therefore the data must be added manually to update the HDT. Collecting patients’ health data put a strong focus on aspects such as privacy and security as well as well-being and trust of participants which need to be considered when developing HDTs12,13.
Behaviour digital twin
Building on the concepts of the DT and HDT we propose a BDT, which by connecting the digital side with the physical counterpart aims at continuously measuring humans’ behaviour to optimise and individualise responses of processes. The BDT is not reduced to a single area of interest like the health sector but can be applied in any area where humans’ behaviour is of interest to be positively influenced. At the centre of the BDT is the behaviour of people and therefore the foundation of the BDT should be based on well established and empirical proven behavioural theories from the area of psychology and the social science. Furthermore, working with human participants requires the strict application of General Data Protection Regulation (GDPR), ethical standards, and privacy considerations to ensure a safe and trustworthy collaboration between participants and the running of the digital counterpart and all involved stakeholders. A key challenge is that the interaction between humans, their behaviour, and their environment is a dynamic system, which is constantly changing and requires from the BDT to learn and flexibly and dynamically adapt to new information and circumstances. The chosen AI models need to be able to process past information about the participants, their behaviour, and the environmental conditions with new data to be able to learn and to make accurate predictions and decisions7,14.
A research gap can be found in the HDTs where the definition and explication of behaviour relevant variables are vague or missing. So far only motivation has been discussed in more detail8, not yet considering the rich heritage of existing behavioural theories and models, which have been developed for decades in the field of behaviour science and psychology to understand, predict, and influence the human behaviour. Human behaviour is strongly determined by the concept of motivation, however, other aspects need to be considered as well, such as the external demands, the redefinition of the external demands or tasks, goal setting, the behavioural intention, feedback, and environmental influence factors. Classical theories of behaviour typically have been focusing mainly on single variables only, while at the same time representing mostly static snapshots of behaviour, thus they have been found to be not up for the task to be used in the digital world15 in their original format without being modified.
Methods
Theories of intention and motivation
Aiming to address this research gap we propose to jointly operationalise the theory of planned behaviour (TPB) and the self-determination theory (SDT), which are two quantitative theories capable of assessing and quantifying aspects of behaviour related to intention and motivation5,16. One of the key benefits of this approach is that empirically validated inventories and questionnaires are provided by these theories, enabling the measurement of relevant variables which influence the behaviour. Aspects such as intention and motivation can be measured and help categorising participants to see how high their interest in the intervention is and what kind of support they need to become and stay engaged. It will be shown how this can be operationalised into a BDT service, which can be used to assess behaviour and guide model-predictive control (MPC) of dynamic complex systems including human actors.
Theory of planned behaviour
The TPB16 qualifies as a quantitative theory because it provides inventories and a questionnaire to assess the behavioural intention (BI) as well as its contributors, the attitudes towards the behaviour (A), the subjective norms (SN), and the perceived behavioural control (PBC).
The BI can be understood as an indicator of “the likelihood that a certain behaviour towards an object is shown by an individual”17 and therefore can be defined as the “subjective probability of a person to perform the behaviour in question in respect to a given object”18. As such a MPC can utilise this as predictor of behaviour and use the BI for deciding when and to whom to send a relevant demand response message, also taking into consideration the intention-action gap which needs to also be estimated by the BDT. The latter refers to the observation that good behavioural intentions not always lead into the desired behaviour, a phenomenon called the intention-action, the intention-behaviour, the attitude-behaviour, or knowing-doing gap in the literature19–23. In the area of sustainability and energy behaviour it is quite common that actual behaviour shown, and the good intentions do not always match24.
A graphical representation of the TPB is depicted in Figure 116, showing how well-defined pre-curser variable are predicting behaviour. The TPB provides a validated method to create questionnaires which are able to evaluate the input variables of behavioural beliefs, normative beliefs, and control beliefs towards a subject area. These input variables are then aggregated on the next level of the model, leading to the intermediate variables of attitudes, subjective norms, and perceived behavioural control. Valuable information of those three variables can already be derived which is facilitated by the BDT and can be used to shape the messages provided to the participants through a user interface component. For example, someone with a high attitude value towards a topic will behave more favourably to a message pointing out the benefit of the topic in general (e.g., environmental awareness), a person with a high value in regarding social norms will respond to messages relating to relevant others in a social context (e.g., competition and/or gamification), while a person with a high perception of control will value messages most, which facilitate actionable suggestions.
Figure 1. Theory of planned behaviour.
Image used with permission from 16.
The behavioural intention is built out of all three variables summarising that “...the more favourable the attitude and subjective norm, and the greater the perceived control, the stronger should be the person’s intention to perform the behaviour in question”16. The BI is computed by the BDT and also estimates the intention-action gap given that sufficient interaction with participants is taking place through a user interface component, which leads to the possibility to refine the behavioural intention to provide a prediction of the probability that a behaviour will be carried out to the MPC.
As mentioned, the BI is calculated as a combination of the three underlying factors attitude, social pressure awareness, and perceived behavioural control. By not just considering the aggregated BI score but the three factors individually it is possible to characterise each participant depending on the combination of strength of the individual factors. Partitioning each of these three dimensions into either high (>0) or low (<0) eight different personas can be distinguished as shown in Table 1
Table 1. List of the eight different personas depending on the combination of the three influence factors attitude, social pressure awareness, and perceived behavioural control25.
| Attitude | Social
pressure
awareness | Perceived
behavioural
control |
|---|
| A | Positive | High | High |
| B | Positive | Low | High |
| C | Positive | High | Low |
| D | Positive | Low | Low |
| E | Negative | High | High |
| F | Negative | Low | High |
| G | Negative | High | Low |
| H | Negative | Low | Low |
For example, a person showing a positive attitude towards demand response energy saving but low social pressure awareness and limit perceived behavioural control over the environment will react positively to information, will show a limited reaction towards role model behaviour, and requires good action plans to be able to execute the desired behaviour. By analysing to which group someone belongs it is possible to use this approach to personalise information and interventions accordingly.
Self-determination theory
The self-determination theory (SDT)5 provides a more long-term concept of motivation, while the TPB describes the intention as the condition shortly before showing a behaviour in question16. Motivation defined in the context of the SDT can be understood as a general tendency towards a subject area, which can be visualised on a continuum from intrinsic to extrinsic motivation. To distinguish motivation and intention the following example can be given: holding a positive motivation towards behavioural change in the context of sustainable energy consumption can be considered a long-term motivation for particular behaviour, but the impact on actual more short-term behaviour is captured by more immediate intentions to act accordingly.
Motivation can be operationalised by the SDT, which provides a methodology to build questionnaires based on an empirically validated approach to assess different types of goal aspirations, which are associated with specific types of motivational regulations being external, introjected, integrated and intrinsic. According to 5 these are located on an increasing scale of self-determination and can therefore be used to describe behavioural autonomy.
Those motivational regulations are aggregated into a quantity called relative autonomy index (RAI), which reflects the autonomy of an individual regarding a topic of interest. The RAI is calculated similar to the behavioural intention as depicted in Figure 2. The SDT claims that the autonomy of an individual relates directly to the motivational type of this person as well as defining the strength of the motivational type towards a certain behaviour. For example, high values of the relative autonomy index mean a leaning towards intrinsic motivation, which describes that a person feels internally driven to behave in a positive way or purely out of curiosity and enjoyment towards a certain topic. A person who is intrinsically driven is more likely to translate this motivation into actual behaviour. Low values of autonomy indicate a leaning towards extrinsic motivation, which manifests itself in people being driven by external incentives, such as for example financial benefits only and show the behaviour in question only as long as the incentive is given to avoid punishment, or to comply with rules.
Figure 2. Visualisation of autonomy as proposed by the SDT.
Intrinsic motivation is defined as “motivation based on the satisfactions of behaving for its own sake”26, and is based on interest, enjoyment, and inherent satisfaction. In contrast “extrinsic motivation is behaviour that is instrumental – that aims toward outcomes extrinsic to the behaviour itself”26.
In summary the following concepts from the TPB as well as from the self-determination theory should be considered to improve the behaviour of participants in an intervention.
1. Attitudes towards the behaviour (TPB), “which are the predispositions of a person to respond consistently to a subject matter”17 and therefore, are particularly key drivers of sustainable behaviour. The areas of application for the TPB are manifold with plenty of empirical support27.
2. Social and subjective norms (TPB), “which are the perceived social pressures from relevant others and groups influencing the behaviour”17. Areas such as comparison and gamification use this concept, which has been extensively studied and validated by for example28,29.
3. Perceived behavioural control (TPB) describes the individual perception regarding personal abilities and environmental circumstances which can support or hinder a behaviour. The acting person gains a sense of control or non-control from it.
4. Intrinsic regulation (SDT) is understood that someone is doing things out of enjoyment and curiosity and not solely to get incentives or avoid negative consequences. Intrinsic motivation can have a positive impact in for example sustainable behaviour30.
5. Identified regulation (SDT), points out the extent to which a person has accepted a goal as their own and the commitment to act on it accordingly.
6. Introjected regulation (SDT) reflects the extent to which a person behaviour is used to avoid negative consequences. Such an avoidance strategy can be a driver of behaviour, although in this case motivation is affected negatively.
7. External regulation (SDT) explains that a desired behaviour is only shown when external incentives and rewards are given. This key driver can be addressed by monetary and financial incentives; however, there is also a correlation with negative motivation in the sense that a behaviour is only carried out as long as the incentives are given.
The points above provide a checklist of aspects which influence the intention and motivation of individuals. The presented Behavioural Digital Twin not only supports all of the above quantifiable variables, but it also enables the computation of the following two derived variables:
1. Intention (TPB) defined as the likelihood that a behaviour in question will be shown and describing the state immediately before the behaviour is executed. According to the TPB the intention is determined by the three precursor variables of Attitude towards the behaviour, Subjective Norms, and Perceived Behavioural Control (cf. points 1–3 above).
2. Motivation (SDT) defined as a more general tendency to engage or not engage with a certain topic. The level of autonomy someone displays towards an area of interest can be summarized in the motivational type. The motivational type explains which kind of feedback needs to be given to a participant in an ongoing intervention to facilitate and increase engagement and positive behaviour towards the area of interest (cf. points 4–7 above).
In the following section we will show how these concepts can be operationalised and implemented in a software service.
Operationalisation of the models
The TPB provides a framework for creating questionnaires assessing behavioural believes, normative believes, and control believes for each participant p as follows
All those are typically measured on a 5-point Likert scale and aggregated using common pre-calibrated question weights
into the attitudes towards the behaviour, subjective norms, and perceived behavioural control for each participant
enabling the computation of the behavioural intention as follows
using common but a-priori unknown mutual weights
Similarly, the SDT provides a framework for developing questionnaires to assess intrinsic, identified, introjected, and external goal aspirations for each participant
which weighted by common pre-calibrated question weights
can be aggregated into the respective regulations
The relative autonomy index is then defined as
using the common but unknown weights
We propose to combine these two well-known quantities into a single indicator for the likelihood that an action a is carried out by an individual p yielding the action indicator
which introduces the common but a-priori unknown action dependent intention-action gap IAGa and uses two pre-determined configuration parameters λBI and λRAI to control the mutual importance of long-term motivation versus short-term intention. In summary, the likelihood that a person p shows a behaviour Bap in relation to action a is assumed to be predicted by the following probability
that uses a logistic activation function ϕ[x] = (1 + e−x/γ)−1 to translate the indicator into a probability value applying a pre-determined smoothing parameter γ.
This likelihood depends on the unknown weights W = WBI ∪ WRAI and the unknown intention-action-gap IAGa, which needs to be calibrated based on actual behaviour observed by the BDT. If an action a is requested of a person Nap times and that person reacts to this action nap times then the empirical reaction probability of the behaviour is
We can calculate the mean and variance as follows
For sufficiently large Nap and assuming that actions are neither completely certain nor impossible, i.e. 0 < nap < 1, we can approximate using Normal distributions. Further assuming statistical independence, the probability of observing a behaviour given the model assumptions can be approximated as
Applying Bayes’ theorem the unknown weights and the intention action gap can be determined by maximising the log-likelihood
These weights and intention-action-gaps can be kept updated inside the BDT to synchronise the observed behaviour with the acutal behaviour always taking the assessed psychological parameters into consideration.
Implementation
To keep the log-likelihoods of the intention-action-gaps and the weights maximised we can use a simple gradient ascend method. Assuming a uniform prior for the intention-action gap and a Normal distributed prior for the weights
with mean weights assumed to be µw = 1 in accordance with the TPB and the SDT and the parameter to control the strengths of these model assumptions with respect to the empirical observed actions we minimise
which has the partial derivatives
using the per person weights
which contains the derivative of the logistic activation function .
In summary, the intention-action gaps for each action as well as the unknown weights are continuously kept updated by the BDT as follows
to accommodate newly observed empirical data throughout the operation of the system. Using these parameter estimates the BDT can provide the likelihood Pr{Bap|IAGa, WBI, WRAI} for each participant p to carry out an action a, which can be used in MPC to determine whom to target with potential requests.
Operation
A BDT as presented above is useful for model predictive control because it allows to predict for each end user if an action will be taken or not. In the absence of direct control of household energy assets, which is the typical use-case scenario for residential demand response, any control action needs to be triggered through sending a message to the participants to take some action (e.g. shifting an electrical load by choosing a different time to switch on the washing machine or other appliances). Since sending a message does not imply that the requested action is always taken an estimate of the likelihood is essential for optimising an aggregated system of residential households and selecting whom to target best and when. This estimate on an individual basis is the purpose of the BDT.
The basis for the presented approach are questionnaires based on inventories provided by the respective underlying psychological theories and models (see here for an example questionnaire to assess intention and motivation towards the topic of energy flexibility in accordance with the TPB16 and the SDT5)).
The first step of the process is always to present these questions to the end-users and to gather preliminary data to initialise the behavioural beliefs and goal aspirations for each participant (see Figure 3 for the complete workflow). This allows to calculate a first estimate of each persons likelihood to react to any requests without taking into account the intention-action gap and without any fine tuning of the internal weights.
Figure 3. Workflow and application programming interface (API) of the proposed behaviour digital twin service.
In subsequent steps the users are interacting with the system and reacting to demand response triggers sent to them. The measured actual reactions can then be used to continuously recalibrate the model parameters, keeping the system up-to-date and estimating the action specific intention-action gaps. The BDT therefore tracks the actual behaviour while at the same time incorporating the assessed intentions and motivations of each participant.
From time to time the questionnaires can be sent out again to regularly re-assess and to track changes in beliefs and goal aspirations of individuals to keep the estimates accurate.
The BDT service application programming interface (API) (software available here31) provides this use-case functionality through these four services (cf. Figure 3))
GET Questionnaire is used to request the questions to be asked through a user-interface for a particular topic. It returns the questionnaire together with the appropriate multiple-choice answers and identifiers to be used to report back the results.
POST Reply is used by the user-interface to report back the results of the questionnaire to the BDT service.
GET Profile is used to retrieve the likelihood that an individual will react to a demand response trigger for a particular action based on the models and algorithms presented above.
POST Action is used to report if a user has reacted to a demand response trigger or not in order to enable the BDT to track actual user behaviour and re-calibrate its internal state accordingly.
Conclusions
Residential demand response is a challenging and yet very relevant problem, which relies on the close integration of technical solutions with the everyday behaviour of people in their homes. While a lot of research is carried out into optimal control of assets, most energy consuming home appliances are integrated into established household practices and as such manually controlled, precluding them from being directly utilised in model predictive control use-cases that rely on automated control actions. On the other hand, people are willing to participate in behaviour demand response schemes and adapt their behaviour according to indirect messages sent to them when an action is required. However, the uncertainty of their reaction to such triggers has thus far made it difficult to implement model predictive control demand response schemes in residential settings.
On the other end of the spectrum, decades of research in psychology and the social sciences has developed methods and inventories to assess and measure quantities like intention and motivation that are directly linked with the likelihood of carrying out some behaviour. The downside of these methods is, however, that they have been too static and not operationalised in a dynamic system that can be directly used by a model predictive control optimiser.
The presented BDT tries to bridge this gap and implement a software service that based on rigorous behavioural theory measures precursor variables using well-established and tested inventories and combines these measurements with the dynamic tracking of actual behaviour. The contribution therefore makes existing behavioural models available to a wide range of practical digital applications such as the presented residential demand response scenario.
The approach is applicable to other application areas and the software including a configuration example for the use-case of energy flexibility can be downloaded at https://zenodo.org/record/812086331.
