In this section summaries of the PROCare4Life care model validation results are presented, organised according to the research questions advanced in the methods subsection above. The first subsection below focuses on assessing how the PROCare4Life solution fitted into the existing care pathway in the pilot sites. The second subsection focuses on the effects of PROCare4Life on care performance within the following dimensions: Patients’ experience of care; Equity of access; Patients’ quality of life; Caregivers’ quality of life. The third section focuses on the efficiency of PROCare4Life from the perspective of the healthcare professionals that tested the system. The fourth section focuses on the value for money analysis, operationalised as the frequency of adverse events and avoidable admissions to hospital. Finally, Scirocco best practice responses, which identify a meso-level current level of maturity for future scaling up of PROCare4Life system for each pilot site, are presented.
Effects of PROCare4Life on care performance
In this subsection we address the effects of PROCare4Life on the care performance on: people living with dementia or Parkinson’s experience of care; social and gender equity; people living with dementia or Parkinson’s quality of life; and carers’ quality of life. There is one devoted subsection for each one of the previous dimensions of validation.
As a general conclusion, it was confirmed with high consistency between questionnaires and personal interviews, that pilot representatives considered the PROCare4Life idea and underlying model of care very innovative and promising, and believed that it can certainly improve care practice through more enhanced integrated care. One of the pilot sites indicated that: “After going through several improvements, PROCare4Life positively impacted the care of patients with neurodegenerative disorders”44, confirming that: “In general, patients who have used the PROCare4Life system have shown satisfaction with it. They felt that their health was better monitored through the devices, as they had healthcare professionals looking at the information received. In addition, they were aware of the benefits of using technology to collect objective data on the evolution of their symptoms on an ongoing basis, not just through a few medical sessions per year. Special attention to serious symptoms, especially in the area of prevention or to solve problems that arise (e.g., falls).”
Additionally, “In the cases of patients whose caregiver was also involved, there was an increased sense of security, knowing that the caregiver would be regularly informed about the evolution of the patient's health”.45 However, one of the pilot sites reported not having identified any effect on patients’ experience of care (understood through its 3 main dimensions, personalisation, coordination and continuity), being their target population people living with dementia.46 Other pilot site considered that the identified positive effect might potentially be connected with the increased attention received as a result of participating in the pilot (daycare centre scenario).47
When focusing on the existing care pathways in the pilot sites, it could not be identified any effective application of the technology to improve the existing care pathways and thus make a direct impact on the model of care of each organisation, which is consistent with the prototype status of the technology developed, being most of its components around Technology Readiness Level (TRL) 7.
People living with Parkinson’s or dementia’s experience of care. The results presented in this sub-section are based on some qualitative results from the questionnaires and the analysis of several quantitative databases generated by pilot 3 research. The different databases have been explored and analysed from the perspective of this article’s objectives, assessing the consistency of their results with the qualitative responses to the questionnaires gathered among pilot site representatives.
In pilot 3 the Patient Assessment of Chronic Illness Care (PACIC+) was passed to the intervention group, the control group and the replicability study with people living with comorbidities. In total, 26 different questions were posed to participants to understand the quality of the care that they received. These aggregated responses are an adequate assessment for people living with Parkinson's or Alzheimer's regarding how much their respective systems have advanced towards people-centred, integrated care, creating a baseline for future interventions from the point of view of their users. However, it should be noted that this questionnaire did not address the specific effects of the PROCare4Life system, but the current quality of care status of their own care provision systems, as some of its questions are closely connected with integrated care. Gender analysis was not feasible for this group of variables due to lack of disaggregated information.
For the PACIC+ questionnaire, the code goes from 1, which means almost never, to 5, always. It allows the identification of a baseline on the perception of the quality of care as experienced by patients. It might be worthwhile to repeat these questionnaires once the PROCare4Life system is further developed and eventually requiring implementation change management, in the future.
The general introduction to the questionnaire is: “We would like to know what kind of help you received by the healthcare team to deal with your illness. This can include your family doctor, nurse, or medical assistant”. The questions included in PACIC+ which are more closely connected with integration of care are those intended to assess how the healthcare professionals supporting the person living with chronic conditions listen, respect them, and have their respective care plan agreed to. Another dimension of the analysis is how much the multidisciplinary coordination, and thus the required continuity, is advanced in their current healthcare services provision. The first seven PACIC+ questions have been included in the analysis, depicted in Figure 3.
Figure 3. Responses to first 7 questions PACIC+.
The responses to the first seven questions show that the most selected response is “Never” for being asked for their opinion when doing my treatment plan (question 1); Being given options to think about regarding their treatment plan (question 2); Being asked to describe the eventual problems with the medications or their side effects (question 3); Being explained how the steps I took to take care of my illness influenced my progress (question 6); Being asked questions about their goals related to the management of the illness (question 7). More positive results were received regarding being given a list of actions to adopt to improve their health (question 4) or their satisfaction with the good organization of the care provided, that had varied opinions including positive and negative ones (question 5).
No relevant differences have been identified in the responses according to condition, or according to having been control group or intervention group. The responses from PACIC+ show that the quality of care received by chronic patients has room for improvement from a person-centred care.
Social and gender equity. Improving healthcare service access, particularly for vulnerable populations like the ones PROCare4Life is targeting, is closely connected with equity with respect to the healthcare services.48 Equity refers particularly to the equality of outcomes among different population groups. In this subsection the assessment results of how PROCare4Life could be used by people coming from different social backgrounds are addressed. Also, gender differences have been found to affect equity through the PROCare4Life iterative testing.
In general terms, no relevant social determinant to using and accepting the PROCare4Life system were identified. One of the pilot site representatives has highlighted that people included in the pilots “belonged to all social groups (lower-class, middle-class, and high-class) with all types of educational backgrounds”49, thus being different profiles of citizens helping to co-design the PROCare4Life system, being proven suitable for usage for people coming from different social groups. Another pilot site confronted the challenge of supporting people that had been recently diagnosed with Parkinson’s, and that might be going through their acceptance process in anger or denial; thus “A system such as PROCare4Life solves some of these issues by allowing remote monitoring of the disease and some behaviours, as well as making communication between the people living with dementia or Parkinson’s, the caregiver and their relatives with health and social care professionals accessible”.50 Still, another pilot site considered that the effect on equity of access was none, among their participants living with dementia, because the system is still not suitable for “being considered a valuable tool to use in disease management”.51
Important gender differences have been identified, particularly regarding reduced social support for women and the over representation of women among caregivers. From the gender perspective, it has been ensured that the participation in the pilots has assured an adequate balance of both males and females: 54% of the participants were males. When considering the two main diagnoses supported by PROCare4Life, a higher presence of men among participants living with Parkinson´s and a higher presence of women among participants living with Alzheimer’s or other similar dementia should be considered when interpreting the project results.
In terms of social support, as can be seen in Figure 4, there are more women with no social support, while there are more men living with their families or enjoying the support of an informal carer. However, figures are quite close for participants institutionalized or enjoying a formal carer for both genders.52 This is an important and very worrying finding about equity in the care burden between men and women, that nevertheless confirms the already known gender disparities in care burden.
Figure 4. Social support of PROCare4Life participants by gender.
When considering the gender of carers, it is once more evident the higher burden of women in the care process (N being 126). On the one hand, women represent 72% of the carers, as depicted in Figure 5.
Figure 5. Gender of PROCare4Life Pilot 3 carers.
Additionally, female carers were much more qualified than the male carers participating in PROCare4Life pilot 3. The largest groups of carers were females with a university degree. In all educational levels, women were more numerous than men, except for the secondary education level (see Figure 6).
Figure 6. Education level of carers.
However, it must be noted that these results cannot be attributed to the PROCare4Life system, but to widespread gender social determinants.53 Future developments of the PROCare4Life system should consider these facts to make sure that the system is adapted to the differential needs according to gender in aspects such as training materials or personalised recommendations.
Other aspects such as equality of access and results will require further investigation for the future implementation of PROCare4Life, particularly the adaptation of the system for vulnerable populations, people living in remote areas, among other key targeted populations whose integration is to be the focus of the future development of the platform, according to the EU guidelines.
People living with Parkinson’s or dementia’s quality of life. This is one of the key aspects of the analysis of the PROCare4Life integrated care model validation. The increase in the quality of life of its end users was one of the key objectives of the project. The results have been positive, as the consortium was able to identify a subjective increase in the quality of life of people using it, that most likely relates to the enhanced communication with their care ecosystem, particularly their healthcare professionals. Also highlighted by people living with dementia or Parkinson’s that the constant monitoring made them feel more assured. One of the pilot sites explained it as follows: “In general, patients are aware that health monitoring can help them better understand their health status, allowing them to take preventative measures and adherence to their treatments. Furthermore, they trust that systems like PROCare4Life can detect health problems before they become more serious conditions, which could reduce their risk of complications and improve their health outcomes. Also, monitoring systems that can provide feedback to patients increase their motivation and engagement in health care, can help them feel more empowered in their health care.”54
It has been nevertheless highlighted by other pilot sites that the explanation for the improvement of the quality of life of people living with dementia or Parkinson’s might be linked to other factors responsible for these results, mainly that the healthcare professionals were seen by participants more often and their wellbeing was frequently verified both by their healthcare professionals and carers.55 Again, there was no improvement in the quality of life among some of the participants, people living with dementia in advance stages.
The previously presented qualitative results are congruent with those identified through the EUROQoL questionnaire. The quality of life of people living with dementia or Parkinson’s has been measured with EUROQoL, a standardised questionnaire. Based on 644 cases, some positive results on the quality of life of patients using the system could be identified. However, it should be noted that, due to the error level of 4% for a confidence level of 95%, some of these identified changes could be explained by statistical error. EUROQoL asks the participants to assess their own general health under a 100 points scale, and their mobility, self-care, usual activities, level of pain or discomfort and anxiety or depression under a five-point scale.
The most interesting dimension linked to our goals is the level of anxiety of the participants (Figure 7). PROCare4Life is expected to be able to reduce the level of anxiety or depression of their users, thanks to the reinforced health monitoring, direct communication with their care ecosystem (carers, healthcare professionals) and personalised recommendations. After participating for between 2 and 3 months in pilot 3, people living with dementia or Parkinson’s s confirmed that their levels of anxiety or depression improved after their participation in pilot 3, despite the advance of their chronic conditions. The best possible result for the perceived level of anxiety is zero, representing participants who were feeling good, which has increased from 18.72 to 25.54% from the entry interview to the exit interview, also representing a reduction in the number of those declaring that their anxiety levels were severe or extreme.
Figure 7. EUROQoL anxiety entry and exit interview.
If considering the different scenarios, these results are further confirmed. Clearer positive effects have been identified in the comparison between the entry and exit interviews for those using the home scenario, passing from 20% declaring zero anxiety to 28.05% when leaving the study (Figure 8).
Figure 8. EUROQoL Anxiety levels entry and exit interviews, home scenario.
Similar results were confirmed, with an almost 5% increase in those declaring zero anxiety among the participants in the rehabilitation scenario (Figure 9).
Figure 9. EUROQoL Anxiety levels entry and exit interviews, Rehabilitation scenario.
Among the participants in the Day Care scenario, the lower levels of anxiety were confirmed too, passing from 10 to 17% of those declaring zero anxiety or depression and being reduced to those having previously declared high levels of anxiety or depression (Figure 10).
Figure 10. EUROQoL anxiety entry and exit interviews, Day Care scenario.
No relevant differences were found in the self-perception of their health between entry and exit interviews among participants of pilot 3, which is coherent with the short period of participation and the natural deterioration of health linked to both chronic conditions, dementia and Parkinson’s. Additionally, it must be noted that it was not a goal of the PROCare4Life project to improve the health of the people being supported by the PROCare4Life system, but to better monitor them, to empower them and to support enhanced communication with their carers and healthcare professionals, to support the future integration of care through multidisciplinary enhanced collaboration and communication. All these changes were expected to improve their quality of life, as shown.
Finally, the satisfaction levels of the people living with dementia or Parkinson’s participating in the pilot were overall positive, obtaining a positive assessment of 0 or 1 by almost half of the participants (see Figure 11).
Figure 11. Final satisfaction level participants pilot 3.
For the future developments of PROCare4Life, it will be key to consider that there has been some positive feedback at the end of the study questionnaires, such as that the people living with dementia or Parkinson’s value highly the activity monitoring and alerts, fitbit data gathered, the exercise recommendations and its reward system. Also, users very much appreciated the large amount of health data collected and that it was shared among the care ecosystem with carers and healthcare professionals, who can monitor citizens’ health status at any time remotely.
However, it was also made evident that the system was considered complex to use for a relevant number of participants, who complained about having to be reminded to charge the devices, connectivity problems between them, the need to send data by themselves every night or, for some of them, having to wear the Fitbit all the time. Additionally, users repeatedly requested access to their own gathered data and results, since they filled in many questionnaires, but they did not have automatic access to this information. Cognitive games were loved by some but considered too easy or too difficult by others. Depth cameras were not accepted by many end users and the solution was redesigned to use cloud services instead of mini-PCs to make the system simpler, among many other adjustments to respond to the participants’ feedback.
Carer’s quality of life. When considering the anxiety of the carers (Figure 12), the positive effects of participation in the study are more evident than in the case of people living with Parkinson’s and Alzheimer’s, reflecting more than a 13% increase in those carers who do not feel anxiety or depression at all after having participated in pilot 3.
Figure 12. EUROQoL Level of anxiety/depression of carers. Entry and exit interview.
In the end of-study interview completed by carers, it was highlighted that the PROCare4Life system was valuable for them (Figure 13) because of its continuous monitoring of the people they were taking care of, most usually family members and relatives, highlighting the large amount of health and daily life data gathered by the system. Also highly valued by carers was that PROCare4Life allowed them direct contact with healthcare professionals.
Figure 13. Satisfaction with PROCare4Life system (Carers).
Among the things to be improved in the future, participants consistently highlighted the data transmission issues, the complexity of the platform usage for some of the people living with dementia or Parkinson’s, and the challenging installation and charging of the devices.
Among the final suggestions for future developments of the PROCare4Life system, the most requested have been: the automatization of data collection; increased simplicity and easier to use; less equipment; more diversity and levels of the games, again in clear alignment with the recommendations and requests posed by those they take care of. Their final satisfaction levels were positive, with more than two-thirds of carers considering that their satisfaction level with the current version of PROCare4Life was either total or high.
These results are again largely congruent with those gathered through the questionnaires completed by the pilot sites. In one of the pilot sites, it is considered that the quality of life of the care staff improved, “there is now Wi-Fi in all day care centres and mobile services. And they have been equipped with technical devices. The project participants were temporarily involved by us and needed less care and attention from the care staff during this time”.56 In another pilot site, there is an interesting reflection about cultural determinants of the care activities and care burden and its relationship to PROCare4Life: “Caring for people affected by Parkinson's disease requires a lot of effort and dedication. The main carers, at some point in the evolution of the disease (more or less prolonged), will suffer in greater or lesser intensity the following symptoms: Health problems and physical pains (help in carrying out the basic tasks of daily life); Psychological problems: depression, sadness; Social isolation (they devote all their time and effort to care).
The Mediterranean culture (generalizing) often develops a series of erroneous thoughts about care: high degree of guilt (for thinking that they do not do everything right, for thinking that they do not dedicate enough time or effort to it), care as the sole responsibility of the carer (not shared), with great difficulty in asking for help.
Systems like PROCare4Life can provide a sense of security to know that their family member is in a safe environment where they are able to consult doubts with professionals. Tranquillity, feeling of having the situation under control since they are informed periodically about their beloved activities, and specifically notified of any arising risks being aware of patient’s disease evolution.”57
This opinion seems to be also partially shared by the participants of other pilot sites: “The role of the PROCare4Life system in improving the quality of life for patients with neurodegenerative disorders was a subject of conflicting opinions among their carers. In the initial two pilots, most carers faced difficulties installing the system, making them sceptical about its usefulness. However, participants in the third pilot reported that being able to monitor their loved ones' health significantly improved their quality of life and reduced their anxiety levels”.58 Finally, one of the pilot sites considered that the technical aspects need to be further adapted for dementia participants, not having found any effect on their carers’ quality of life.59
Healthcare professionals’ efficiency. The PROCare4Life system has been designed for health data sharing among patients, carers, and healthcare professionals. This functionality is intended to promote increased multidisciplinary coordination and continuity, given that data sharing is critical for different healthcare professionals to work together on behalf of the patient’s needs and wishes, being a key for integrating care.
Among the 159 professionals responding to the end-of-study questionnaire, quite different profiles have been included: clinical doctors, nurses, nutritionists, occupational therapists, physiotherapists, psychologists, social workers, speech and language therapists, and also healthcare managers. Their overall assessment of the system is quite positive, considering almost half of the respondents (47.3%) that they would be likely or very likely to use it (Figure 14).
Figure 14. Healthcare professionals' willingness to use PROCare4Life (1 Very Unlikely to 5 Very Likely).
In the final assessment after their participation, about half of the respondents consider it easy to use (51.4%), not expecting to require assistance to use it most of them (28% would consider it likely or very likely to actually require assistance to use it), as depicted in Figure 15.
Figure 15. Healthcare professionals' opinion on easiness to use PROCare4Life.
Among the most valued functionalities of the PROCare4Life system, healthcare professionals who participated in the pilot 3 showed a great congruence in their responses about the most and least valued, by patients and carers. The most valued is to assist practitioners with data collection, which integrates information for everyone (patient, healthcare professionals, carers), integrates cognitive games that allows direct communication with patients and continuous remote monitoring.
They also considered the system to be a cost-effective way to evaluate patients, easy to navigate and use, that facilitates the work of the healthcare professionals.
Detailed and dynamic monitoring might also be suitable for personalising treatments, thus supporting healthcare professionals with innovative technologies. Again, healthcare professionals detected the same weaknesses identified before when analysing opinions of patients and carers: it was considered a complex system for the patients, time consuming devoted to charging and sending data, app and web app are slow, chat did not always work, information not always appear, difficulties installing and synchronising devices. According to this group of end users, the system needs further adaptation to the target population. These results indicate the lines for future work to refine the prototype.
How about workforce efficiency? Almost 40% of the respondents consider or strongly consider that the system will save time in clinical practice, while another 40.1% consider or strongly consider the opposite, that in fact, it will not save time in clinical practice. A similar distribution of opinions was identified when asking healthcare professionals if they considered the PROCare4Life system cost-effective: 41% agreed or strongly agreed, while 43.8% disagreed or strongly disagreed. Further research might be required to understand the reasons for this distribution of responses.
The satisfaction level with PROCare4Life (Figure 16) has an almost central normal distribution, highlighting the importance of work to adapt the system to increase the satisfaction of healthcare professionals using it in the future versions of the PROCare4Life system, making sure to keep the most valued functionalities and addressing these suggestions of improvement and satisfaction levels.
Figure 16. Healthcare professionals satisfaction level with PROCare4Life.
At this stage of development, the survey for decision-makers reflected that although the system might be able to facilitate the clinical decision-making process (62.5% completely or quite trust this result), almost half of the respondents (49%) do not know if they would pay for it.
However, 69% of the respondents expect that PROCare4Life, thanks to data sharing among different profiles of healthcare professionals, people living with Alzheimer’s, Parkinson’s, or other neurological conditions, will help to improve the healthcare provision towards a more coordinated, continued and people-centred services (the 3 main characteristics of integrated care healthcare services).
For future development of PROCare4Life, since it is still not mature enough to be brought to the market,60 it must be considered that “implementing an integrated care system can require a significant initial investment in terms of resources and time, so it is important to carefully evaluate costs and long-term benefits”. Additionally, it should be considered that “When an integrated care system improves the care experience of patients, their quality of life, allows access for different social groups, improves the quality of life for caregivers, decreases the frequency of adverse effects and hospital readmissions, and increases worker efficiency, this translates into increased system profitability in several ways: Cost reduction: The decrease in the frequency of adverse effects and hospital readmissions means a decrease in medical care costs. In addition, improved worker efficiency can also reduce administrative and personnel costs; Increased patient satisfaction: Improved care experience and quality of life for patients and caregivers can result in increased satisfaction and loyalty to the health system, which can attract more patients and increase long-term profitability; Reputation and credibility improvement: Successful implementation of an integrated care system can improve the reputation and credibility of the health system and attract more patients and medical care providers; Workload reduction and increased productivity: Improved worker efficiency can reduce workload and increase productivity, which can translate into increased long-term profitability”.61
