DAT Implementation
Here, we present implementation experiences following the four RE-AIM constructs.
(1) Reach
Participants were generally willing to use devices, with children’s caregivers more willing than adolescents (median device use: 94% [IQR 81-98%] vs 87% [IQR 50-97%], respectively). However, this willingness changed with the convenience of their use. Adolescents living with people who did not know the adolescents’ HIV status, and those whose friends/relatives had access to their phones, were less willing to use the intervention. This is because the device raised questions about its purpose and drew unwanted attention because of the sound it made when it opened. As some narrated:
“I did not accept the device because at home, there are many people around, and when they see it, they ask too many questions, and I don’t know how to answer them”. (Adolescent, FGD)
“She [his mother] told me to change the phone number so the messages would come to me. I told her I did not want the messages.” (Adolescent, IDI)
Others had concerns about the SMS contents, thinking they were not discrete enough, and could still lead to unwanted disclosure of HIV status. For example, a caregiver said:
“Its content was a little disclosing; they should have asked us how we wanted the messages to be? Then the participants could arrange what the message should be delivered in what format, so that even if someone takes my phone, they will know maybe the messages are from the SACCOS [traditional banking services] or something.” (Caregiver, IDI)
The intervention was provided free of charge to all participants. However, some factors limited the availability of the intervention at certain points during the 12 months of participation in the study. For example, reminder SMS did not reach adolescents using their caregivers' mobile phones when either of them moved. Some said:
“My mother is the one who was getting the messages. So, you find that there are days you just don’t feel like taking the pills, so I just open and don't take my medication.” (Adolescent girl, FGD)
“I had the device, but I returned it before going to school because I thought it would be a challenge; phones are not allowed in school. When you open it, it makes a certain sound, and someone might think it is a phone.” (Adolescent boy, IDI)
(2) Effectiveness
Participants perceived DAT as effective when it was convenient for addressing the adherence barriers they faced. Adolescents’ adherence barriers included: busy schedules, unfriendly environments that risk HIV status disclosure, medication boredom, and forgetfulness. For children, barriers furthermore included the absence of a caregiver or a busy caregiver, the child not being aware of their HIV status, and food insecurities. For example, a participant shared:
“I am busy in the evenings, so most of the time I get late and fail to remind him, or when I find there are some people inside, I was afraid to give him pills because the device makes some noises, so we have decided to change the time for meditation, now we take together at 20:00”. (Caregiver, IDI)
The need for DATs varied: some found specific components more useful than others (e.g., they needed only reminders), whereas others used the device as a storage container or for travel. Others clearly stated that they did not require the intervention and could maintain good adherence independently. A caregiver claimed:
“I don't see it as important; it is just a matter of someone using their mind, because someone can have a device, but still not use medications. It is simply a matter of self-determination and knowing that, when the time comes, they should take medications. Because in the past, people were taking medications without this device.” (Caregiver, IDI)
The DAT was considered a reliable tool for measuring adherence, and for caregivers, it helped them recognize when their children were not taking their medication. An adolescent explained:
“That report was helpful because you find someone who is lying and lying, and then the graph reveals the truth, and you give reasons as to why you are not taking your medication. You know it's not that easy for some people to tell the truth until they are very pressed. So, you find that the feedback report is pressing you… You’d better be honest. So, the report from the device has helped to reveal the truth and how to help this person take their medication on time.” (Adolescent, FGD)
Adolescents with an unsuppressed viral load at recruitment believed that the DAT helped them suppress viral replication. One of them said:
“The device is perfect. It can help those who take their medication monthly [unsuppressed]. It will help them take their medication. And they should be advised, and sometimes, as was the case with me, it's not that you forget but because you lose hope and are tired of taking your medication. When you accept yourself, is when you take your medication well.” (Adolescent, FGD)
The DAT motivated adolescents to take medication even when they experienced medication boredom. One explained:
“On my side, there is a time, it's not that I have forgotten, it’s just that I don’t want to take the medication, I decide that I am not taking my medication, but then when they send the messages, you feel that someone cares for you and then you don’t care about yourself… so, you just take your medication.”(Adolescent, FGD)
Participants acknowledged the effect the DAT had on their adherence. Still, they explained they could maintain optimal adherence without the pillbox when they had accepted their HIV status and understood the importance of good adherence. Regardless of the circumstances, participants made efforts to take their medications. For example, an adolescent explained:
“When I had the device, I used to leave the device in the hostel, and when I received the message, saying “The time is near,” I asked for a little bit of space, and that’s when I went to take my medication. Sometimes you find that they [colleagues] ask, “Every day you ask for space, where do you usually go?” (Adolescent, FGD)
The bi-weekly educational SMSs were less effective than the reminder SMSs because most participants ignored them. Adolescents expressed their need for education, especially concerning sexual and reproductive health, but admitted that education through SMS is not effective. They prefer that education be provided face-to-face during refill clinics or through youth seminars.
(3) Implementation fidelity
Most parts of our DAT intervention were implemented as intended and efficiently. Although we were unable to obtain SMS delivery reports, we received a Wisepill report indicating that the SMSs were sent, and none of the participants ever reported not receiving intake reminders. Participants reported not having received educational SMS. Later, we noticed they ignored them, thinking they were reminder SMS since they were both sent with the same sender ID (RMNDR). As the caregivers said:
“No, I have never received those messages [educational SMSs]; the messages that I was receiving are the reminders.” (Caregiver, IDI)
We encountered an error in the visit reminder program that caused some participants to receive SMS messages and visit CTC facilities outside their scheduled dates. Because we did not have a study team at the site every day, these participants were not attended to, and later, they didn’t trust the SMS, so they didn’t show up. To avoid confusion, and because we could not allocate additional time and personnel to accurately program clinic reminders, we ceased sending these SMS messages.
We encountered challenges related to poor network connectivity, malfunctioning devices or chargers, or drained device batteries, resulting in devices losing connectivity. Despite efforts to remind participants to charge their devices, we observed 2,458 (8.6%) days of lost device communication among children and 67,32 (19.8%) days among adolescents.
“She [the child] knows how to open and use it. Once, while travelling to Dar es Salaam, I received a call instructing me to charge the hospital device. Now, I called them [children] back home and told them “To put the charger" they had forgotten that the device needed charging.” (Caregiver, IDI)
During the study period, the standard of care changed: suppressed participants were given refills for six months. As a result, most participants missed their study visits (conducted every three months), resulting in fewer feedback sessions. Only 37/74 (50%) of children and 38/79 (48%) of adolescents had three or more feedback sessions conducted before the last study visit.
(4) Maintenance
Median device-use adherence for the first month of DAT use was 100% (IQR 96-100) and 93.3% (IQR 86-97) among children and adolescents, respectively (as shown in Figure 1). We observed a significant decrease in monthly device use over the 12 months in both groups. Comparing participants' monthly device use of each of the twelve months from the first month as baseline, mixed effects showed that adolescents had a steeper decline in device use, from a decrease of 9% in the second month to 25% in the twelfth, compared to children who declined from 5% to 12% in the same intervals (see Table: 3)
Figure 1. Monthly device use for children (left) and adolescents (right), respectively.
Table 3. Mixed effect output for children and adolescents.
| Children | Adolescents |
|---|
| % Difference
in means
from baseline | 95% CI1 | % Difference
in means
from baseline | 95% CI1 |
|---|
| Month 1 | — | — | — | — |
| Month 2 | -5 | -10, 0 | -9 | -15, -3 |
| Month 3 | -5 | -10, 0 | -9 | -15, -3 |
| Month 4 | -7 | -12, -1 | -10 | -16, -3 |
| Month 5 | -8 | -13, -3 | -13 | -19, -7 |
| Month 6 | -9 | -14, -4 | -14 | -20, -8 |
| Month 7 | -7 | -13, -2 | -17 | -23, -11 |
| Month 8 | -8 | -13, -2 | -15 | -21, -9 |
| Month 9 | -9 | -14, -3 | -21 | -27, -14 |
| Month 10 | -7 | -13, -2 | -22 | -29, -16 |
| Month 11 | -9 | -15, -4 | -23 | -29, -16 |
| Month 12 | -12 | -18, -7 | -25 | -32, -19 |
One factor that hindered consistent use of the device among adolescents was the inability to keep hiding the purpose of the device to avoid HIV status disclosure. Over time, close people around the adolescents had been exposed to the device, and at some point, participants had concealed the truth. This meant they saw the device, they would have to hide the truth repeatedly, and it would be even harder if the device were found with the pills. Some of them narrated:
“It was going well, in the beginning. Now that I've got used to it, I see it as a burden. And, after being asked so many questions like: “What is that?” “What is that?” When you open it, the alarm rings. I just started leaving it at home.” (Adolescent, IDI)
“At first, I was using it. When charging it, I would remove all the pills, then someone saw the device while it was charging, and since then. I was not putting the pills in the device because if I did, someone might see them, so it was a challenge. It was easier to take pills from the packaging container than from the Wisepill box.” (Adolescent, FGD)
Additionally, the noise the device made when opened drew unwanted attention from nearby observers, and participants couldn’t open it without being noticed. Although they admit these disadvantages are few and depend on the environment in which someone lives.
“I used to use the device at home; it helped to remind me. There were times when I would carry it, and then someone would ask what it was. You must lie to them. You tell them it's a power bank. Then they ask you where to insert the charger. It has become a problem that I must leave it at home now”. (Adolescent, FGD)
Children's caregivers used the device more consistently than adolescents due to a lower risk of disclosure. Most caregivers are living with HIV themselves, and family members are aware of the HIV status of the children, so it becomes easier to use the device when at home. However, caregivers also had concerns about the risks of disclosure, as the device was primarily kept at home and didn’t allow the child to move it.
“He [the child] has a little knowledge, so when we talk to him, he understands that these things [HIV status/ARVs] are not to be shown to everyone. So, every morning he will come to our room to take his medication.” (Caregiver, IDI)
“I can tell the reason [for not using the device], that the kid went to her grandmother’s house, so all those days we kept her medicines in a piece of paper, so we did not open the bottle for those days.” (Caregiver, IDI)
Decreased device activity over time was also due to challenges related to charging the device (i.e., not having electricity at home, failure to place the device in the socket, fearing unwanted disclosure if it was seen, and broken devices that lost their ability to stay charged for a long time.
“The challenge that I experienced was that the device was running out of charge, while the charger was already stolen. So, the charge ran out completely, but the SMS were still coming.” (Adolescent, IDI)
Adolescents advised that if the intervention were to be rolled out to the general population, time should be taken to understand participants' living conditions to determine whether it would pose challenges for them. Additionally, if possible, the device should be password-protected to ensure that only participants can access it. One said:
“In the instructions you give, you should explain that the device works like this and that. Then, after a few months, you call the person to ask if they have accepted it and still need it. You let them keep it, but if they don’t want it, then they return it.” (Adolescent, FGD)
