Skip to main content
  • Research article
  • Open access
  • Published:

Process evaluation of the systematic medical appraisal, referral and treatment (SMART) mental health project in rural India

Abstract

Background

Availability of basic mental health services is limited in rural areas of India. Health system and individual level factors such as lack of mental health professionals and infrastructure, poor awareness about mental health, stigma related to help seeking, are responsible for poor awareness and use of mental health services. We implemented a mental health services delivery model that leveraged technology and task sharing to facilitate identification and treatment of common mental disorders (CMDs) such as stress, depression, anxiety and suicide risk in rural areas of the state of Andhra Pradesh, India. The intervention was delivered by lay village health workers (Accredited Social Health Activists – ASHAs) and primary care doctors. An anti-stigma campaign was implemented prior to this activity. This paper reports the process evaluation of the intervention using mixed methods.

Methods

A mixed methods pre-post evaluation assessed the intervention using quantitative service usage analytics from the server, and qualitative interviews with different stakeholders. Barriers and facilitators in implementing the intervention were identified.

Results

Health service use increased significantly at post-intervention, ASHAs could followup 78.6% of those who had screened positive, and 78.6% of the 1243 Interactive Voice Response System calls made, were successful. Most respondents were aware of the intervention. They indicated that knowledge received through the intervention empowered them to approach ASHAs and share their mental health symptoms. ASHAs and doctors opined that EDSS was useful and easy to use. Medical camps organized in villages to increase access to the doctor were received positively by all. However, some aspects or facilitators of the intervention need to be improved, including network connectivity, booster training, anti-stigma campaigns, quality of mental health services provided by doctors, provision of psychotropic medications at primary health centers and frequency of health camps.

Conclusion

The respondents’ views helped to understand the barriers and facilitators for improving the likely effectiveness of the intervention using Andersen’s Modified Behavioral Model of Health Services Use, and identify the mechanisms by which those factors affected mental health services uptake in the community.

Trial registration

The study is registered with Clinical Trials Registry India (Applied - 16/07/14-Ref2014/07/007256; registration received - 04/10/17-CTRI/2017/10/009992).

Peer Review reports

Background

Globally, mental disorders account for 8.5% of the total Years of Life Lost due to premature death and Years Lived with Disability [1]. It is estimated that 75–85% people with mental disorders in low- and middle-income countries (LMIC) do not receive effective treatment – the ‘treatment gap’ - and mental health is often the lowest health priority in those settings [2]. Even though effective treatment exists for mental disorders, lack of trained mental health professionals, poor infrastructure, ineffective government policies, low awareness and increased stigma related to mental health are likely to be important contributors to this treatment gap [3]. This is worse in rural settings [4]. Prior research has shown that task shifting and use of electronic decision support systems (EDSS) can enable mental health services delivery [5,6,7]. Using similar principles, we conducted a project that focused on a mental health services delivery model to screen, diagnose and manage common mental disorders (CMDs) such as stress, depression, anxiety and suicide risk - the Systematic Medical Appraisal, Referral and Treatment (SMART) Mental Health Project [8].

SMART Mental Health Project was conducted across two rural sites, a Scheduled Tribe (ST) Area and a non-Scheduled Tribe Area, of Andhra Pradesh, a south-Indian state [8, 9]. Scheduled Tribe communities are identified by the administration based on their unique cultural and linguistic characteristics, which deem them to be an indigenous community, hence guaranteeing special administrative rights and status. Andhra Pradesh has about 6 million ST population, and this group has poorer health indicators such as life expectancies, under-5 mortality, and many others, compared to other rural communities in the area [10].

Overall, the project had two key objectives: 1) The development of a multifaceted intervention using training, task shifting, and mobile-based decision support to increase the screening, treatment and referral of individuals with CMDs, and 2) Evaluation of the intervention after implementation in 30 ST villages, to provide preliminary evidence of effectiveness, feasibility, acceptability and potential for scale up. The SMART Mental Health study findings indicated an improvement in awareness about mental health in the community; increased screening of common mental disorders by ASHAs; increased use of mental health care provided by primary health care doctors (increased from 0.8% at baseline to 12.6% at post-intervention); and reduction in depression and anxiety scores at post-intervention for those who were screened positive for depression and/or anxiety, at baseline [9].

This paper reports on the mixed methods process evaluation for the SMART Mental Health Project.

Methods

The purpose of the process evaluation was 1) to describe the experiences of those exposed to the intervention and 2) to identify the barriers and facilitators in implementation of the intervention.

Project plan

The project was divided into different stages as outlined in Fig. 1. The intervention had a number of facets as outlined below and shown in Fig. 1.

  1. 1.

    Mobile technology based EDSS: Two separate EDSS were developed for screening by ASHAs, and clinical diagnosis and management by primary care doctors. The tool used by ASHAs was based on the Patient Health Questionnaire (PHQ9) [11] and Generalized Anxiety Disorder questionnaire (GAD7) [12]. The diagnosis and management guidelines used by the doctors was based on the Mental Health GAP – Intervention Guide (mhGAP-IG) [13]. Both tools were developed as applications on a 7-in. Android tablet using an Open MRS platform. Both the PHQ9 and GAD7 provide diagnoses of mild/ moderate/ severe levels of depression/anxiety based on cut-off scores [14]. Only scores ≥10 on either scale, or a positive response to the question on self-harm in the PHQ9, were considered as screen positive for this project. Patients were either seen at primary health centers (PHCs) or at the health camps in the villages. Any patient with severe mental disorders or any other complications were referred to the district hospital which had trained mental health professionals. Data captured on the EDSS were shared securely between the ASHAs and doctors, and it allowed the ASHAs to monitor the progress of the cases referred by them using through a traffic light system that helped them prioritized the patients.

  2. 2.

    Interactive voice response system (IVRS): An algorithm based IVRS sent out pre-recorded messages to the screen positive individuals to visit the PHC doctor to seek care or continue treatment as advised by the ASHA or the doctor. Messages to the ASHAs and doctors reminded them to screen and follow up individuals as per guidelines.

  3. 3.

    Stigma Reduction Campaign: A campaign was conducted prior to the baseline survey to increase mental health knowledge and reduce stigma in the community for 8 weeks across all villages. It included a number of strategies: sharing brochures and posters on mental health with the community using a door-to-door campaign; showing a video of a person talking about his own mental illness and a video of a film actor talking about CMD; staging live performances of a drama on mental disorders and the help seeking in 2 villages, while showing video recordings in the other villages. Discussions followed each presentation.

Fig. 1
figure 1

Overall project plan

Total project duration and eligibility criteria

The total project duration was for 24 months from May 2014 to April 2016 including a formative phase where the interventions were tested and refined [15]. The project was conducted in 30 villages associated with two PHCs, located in an ST area of the West Godavari district of Andhra Pradesh [9]. Fifteen villages from each PHC were randomly selected and then it was ensured that each village had adequate number of ASHAs proportionate to the population. One village with a population > 3000, was replaced by another one with a smaller population, as the village did not have sufficient number of ASHAs to cater to the population. All eligible adults ≥18 years of age who gave informed consented to participate and were not suffering from any severe physical disorder that limited them from accessing mental health services, were included.

Process evaluation

The process evaluation was undertaken from January to February 2016. Quantitative data on service usage from the backend data stored on servers, was acquired throughout the intervention to understand frequency and type of services used - type and appropriateness of care provided by doctors, frequency of follow-up of screen positive cases by ASHAs, and numbers and success rate of IVRS calls. Qualitative data was collected using focus group discussions (FGDs) and in-depth interviews (IDIs) of key stakeholders. Consolidated criteria for reporting qualitative research (COREQ) guidelines for reporting qualitative research was followed [16]. A complete checklist has been uploaded as additional file (Additional file 1: Table S1).

The study participants in the process evaluation consisted of community members, ASHAs, primary care doctors, village leaders and field staff. We included all doctors and field staff in our interviews. However, community members and village leaders were selected purposively. The selection of participants ensured representations of all villages and PHCs, and we tried to ensure equal participation of both genders. We contacted all screen positive adults who had received the intervention through either PHC and divided them by gender. Since it was harvesting season, not all adults were available but we ensured that information was sent out to most. Only those community members, ASHAs, and village leaders who were available and willing to participate were included in the interviews. FGDs and IDIs were conducted either at community centers in the villages, or in the house of the interviewee.

The FGDs and IDIs guidelines comprised of a set of questions developed by the research team through discussions and were translated to Telugu (Additional file 1: Table S2). The questions were relevant to the different processes of the study and probes helped clarify some of the points. The participants were assured about the confidentiality of their responses. A trained moderator conducted the interviews after introducing the process to the participants. A note taker was responsible for audiotaping the discussion and taking notes. Each FGD and IDI lasted for about 35–40 min. Each interview was audio taped, and later transcribed and translated into English to ensure the reliability of data. No individual identifiers were stored or analysed.

Ethics approval and consent to participate

Ethics approval was received from the Independent Review Committee of the Centre for Chronic Disease Control, New Delhi, India. Informed written consent was obtained from all participants. Data were collected according to the Declaration of Helsinki on ethical research. Approval for conducting the project was obtained from the Health Department, Government of Andhra Pradesh, and the Integrated Tribal Development Agency was informed about the project. Approval was also obtained from all the local village administrative bodies.

Data analysis

Quantitative usage specific data was downloaded from the server, cleaned, and analyzed using descriptive methods. Several strategies were used to ensure that the qualitative data analyses were systematic and verifiable. Initially, the interview guide was used to identify some thematic areas which were refined subsequently as analyses progressed. Two researchers (SK and AT) coded transcripts independently and the themes identified by each of them were discussed in detail, compared, and refined, to generate the themes and sub themes based on consensus. This ensured the inter-rater reliability and reduced bias. Thematic analysis based on grounded theory was used to analyze data [17, 18]. NVIVO was used to analyse the data [19]. Barriers and facilitators were plotted onto Andersen’s Modified Behavioral Model of Health Services Use [20], which posits a set of factors – environment, population characteristics, health behavior and outcomes, which interact in a dynamic manner to determine health services use (Fig. 2).

Fig. 2
figure 2

Andersen’s modified behavioural model of health services use

Results

ASHAs were able to follow up 78.6% of all screen positive cases in the 3 month period. Altogether 1243 calls were placed to the community, ASHAs and doctors, of which 78.6% were successful (heard by them). Of these 1243 calls, 1043 were made to the community (success rate of 71.3%) and 200 were made to ASHAs and doctors (success rate of 43%). Doctors appropriately managed all cases as per the algorithm and only referred cases with moderate depression requiring antidepressants to the district hospital, due to lack of medications at the primary health center [9].

The qualitative evaluation included seven FGDs and six IDIs. Four FGDs were conducted with community members, separately for males and females (males - n = 14, aged 34–62 years; females - n = 23, aged 28–65 years). Two FGDs were conducted with ASHAs (female-n = 16, aged 29–62 years), and one FGD with the project field staff who monitored the activities of the ASHAs and doctors, which included both sexes (n = 19, aged 19–49 years). Two IDIs were conducted with the two PHC doctors and four IDIs with the community leaders of four villages (Male and female - aged 29–53 years). The results from both FGDs and IDIs are described below according to identified themes and grouped under facilitators, barriers, and further suggestions related to the overall implementation process.

Facilitators

Knowledge of the project and CMD

Most community members were aware of the different activities of the project, suggesting that the project was helpful, and that they learnt new things related to causes and treatment of mental illness. Community members shared that prior to this project they were not aware where to go and whom to approach for CMDs. Some community members highlighted that this project helped people to discuss and share their problems with others, which in turn reduced anxiety and stress in those individuals.

“We have got a lot of awareness about this project. We did not know what a mental illness is until they [project staff] came and told [us]. When they [project staff] came we explained to them about our physical pains, etc. and during a medical camp [a] doctor saw me and informed me about the treatment. (Community member, FGD5)

“This project was excellent. We learnt all about the causes of mental illnesses and what kind of treatments should be taken for such [mental disorder] problems”. (Community member, FGD-4) “If we discuss our problems with someone, it make us happy and we feel good”. (Community Member-FGD7)

ASHAs added that the increased knowledge and awareness about CMDs empowered the community members to approach ASHAs and share their mental health problems. Doctors indicated that the project was a learning experience for them as they gained knowledge about different treatments for CMDs.

“We have learnt many things from this project. Initially we did not know the importance of mental disorders [CMD] and we had been treating people for other disease conditions, and through this study we got an idea about dealing and treating people with common mental disorders” (Doctor, IDI 4).

Benefits of involving ASHA’s

Almost all the community members and doctors were aware of the role of ASHAs in the project. The majority of the community members shared that involving an ASHA was helpful as she was known to the community, making it easier for them to share their problems. This was considered to not have been possible with the involvement of any unknown person or outsider. Some community members appreciated that the ASHAs went to each house for screening.

Community members shared that ASHAs not only paid repeated visits to enquire about their health, but also tried to motivate them to visit the doctor. They believed that ASHAs were sensitive and courteous in asking questions. One community member shared,

“ASHAs know everything well. ASHAs are the right people as they keep moving around the village and are familiar with the village problems” (Community member, FGD2)

Both doctors involved in the intervention believed that ASHAs rapport with the community helped them to play a major role in implementing the intervention. They believed that this rapport helped them to motivate individuals to seek treatment. They felt that ASHAs accompanying patients to the PHC and briefing the doctor about the patient’s condition was especially helpful.

ASHAs are not new to us and our relationship with them [ASHAs] have been good. So they [ASHAs] used to give a lot of information about the patient, [at times] more than the patients. Patients [at times] did not discuss properly. As ASHAs are locals, they know what is happening in the families of the patients [and so could provide a context]”. (Doctor, IDI2)

Yes … ASHAs did a good job, they worked hard in motivating the patients. They used to accompany patients to the PHC and they used to help me understand the person’s condition. So that was very helpful”. (Doctor, IDI5)

Impact of training

The majority of participants including ASHAs, doctors and field staff reported that, overall, the training provided was useful. Training sessions helped them to operate the tablets and record observations on a regular basis. ASHAs elaborated that the training helped them to do their job and increased their confidence in approaching people to share their mental health problems. It also made them aware about mental disorders in general and CMDs specifically. They indicated that training on conducting interviews and operating the tablets was useful, even though they were initially apprehensive about participating because of their education levels.

“We have never done any interviews before so we got to know how to do it and how to approach someone and explain about the study…about mental disorders” (ASHAs, FGD1)

Doctors indicated that the training facilitated easy use of the mhGAP-IG tool.

“This training programme was very detailed and we learnt about the treatment pattern and new tool which was easy to use” … “Because of this training programme, now I know what kind of treatment is to be given to what cases and which cases have to be sent to the specialist which are beyond my ability”. (Doctor, IDI5)

The field staff mentioned that training helped ASHAs to improve their communication skills which in turn helped ASHAs persuade community members to participate in the intervention. Some field staff opined that most ASHAs who were unable to navigate a tablet initially, were able to collect data using the tablets easily as a result of the training.

“Most of them did not even know how to operate a mobile phone, and because of this training it helped them to operate the tablets and click on the various options” (Field staff, FGD4)

Organizing medical camps in community

Most community members and ASHAs appreciated organizing medical camps in villages. They felt that these camps helped people to understand their health status and facilitated access to doctors within their own village. Some community members believed that these camps not only provided health services but created an environment where the entire community were sensitized about issues related to mental disorders. A few ASHAs mentioned that in these camps, when people with similar problems interacted, they shared their problems with each other and discussed ways to overcome their problems. Village leaders felt that the camps created awareness about the importance of seeking treatment and visiting the doctor.

“Because of these medical camps we [community members] get treatment here, rather than going somewhere. So it is easier for us”. (Community member, FGD5)

“If such medical camps are conducted, people will get to know about their health condition, and will know at what stage they are, and where to seek information for getting treatment for their condition”. (Village leaders, IDI3)

Use of technology - EDSS /IVRS

Using the EDSS and receiving IVR messages were appreciated by the community members, doctors and ASHAs. The majority of ASHAs and doctors opined that EDSS was useful and was easy to use. Initially, there were some difficulties in understanding certain terminology, but these were clarified by trainers. Doctors shared that the duration of assessing a patient was initially 30–40 min due to the length of the questionnaire, but with practice could be completed within 15–20 min. The doctors believed the technology helped them to diagnose patients with CMD.

“The overall use of the app was very comfortable and now I cannot see any problems from the app perspective, overall it was a good experience”. (ASHAs-FGD4)

“Yes, definitely it was a great learning experience for me. I have used this tool for patients who came to me to get treated for some mental disorders”. (Doctor, IDI 5)

Some community members and ASHAs believed that IVRS messages were useful for patients. These messages reminded patients to visit the PHC or to follow other instructions given by doctors and ASHAs.

“Using IVR to send the messages to patients is good as it will be useful to remind people to visit the PHC or to follow instructions”. (Community members, FGD4)

Barriers

Stigma & discrimination to receive treatment

Some participants talked about the challenges faced by the community to receive treatment. Community leaders indicated that people were not fully aware of mental disorders, hence were not interested in revealing their mental health problems. They felt that those with mental health problems and even their families were often subject to stigma and discrimination which is a barrier to seeking treatment. For example, difficulties in securing matrimonial alliances for their children may be an issue if someone is diagnosed with mental disorder in the family. Some community members shared that there were some who were hesitant to visit a doctor, or disclose their problems even after visiting the doctor, because of getting “labeled as a mad person”. A few community members felt that mental illness cannot be cured by going to a doctor.

Field staff felt that the villagers did not reveal their problems completely in spite of assurance about confidentiality. They reported that community members did not visit the PHC due to stigma, and for the same reason they either did not reveal their mental health symptoms to the doctor or only mentioned their physical symptoms when visiting the PHC.

ASHAs shared that they had experienced some negative reactions from the community members while asking questions and had to spend significant time convincing people to respond. They also shared that although privacy was ensured, community members feared that their family members or neighbours might overhear or might come to know about their illness, which may complicate their social problems.

“Many people did not ‘open up’ completely. Even though we assured them [community people] about keeping all the information confidential, so I felt we should create more awareness so that they understand the problem better and come out with their problems in a better way”. (Field staff, FGD 4)

Financial livelihood and social constraints

Doctors viewed that poor economic conditions and lack of financial stability were major reasons preventing greater benefits from the intervention. Village leaders expressed that people did not go to PHCs due to financial reasons and lack of transport. They elaborated that most of the community members are daily wage earners and hence are unable to go to the PHC without financial loss.

“We would need facilities near our places [to avoid] bearing the costs of the travel expenses, and medicines.”(Community members, FGD3).

Some members mentioned that people go to PHC for physical illnesses but ignore mental illnesses. Other community members mentioned that patients did not visit the PHC as they did not trust the doctors’ abilities to treat them.

ASHAs expressed that they faced difficulty in finding people for screening as often the villagers went to the fields for agricultural work in the morning until the evening, hence making it challenging to find people during the day for interviews.

“Yes, [madam] most of us had faced the issue…where we were not able to find people for the doing the interviews. We used to start doing the interviews at 5 [o’clock] in the evening…as nobody was available during the day time…..most of them used to go for agricultural work” (ASHA’s, FGD2)

The other problem that the ASHA faced was related to faulty social perceptions because of the caste system. According to them, higher caste community members believed that mental illness occurred among ‘laborers’ who were generally from low caste communities, and was not an issue in their ‘high caste communities’, hence they were reluctant to answer the questions. Some ASHAs also shared that they faced difficulty in posing the question on suicide as most community members responded poorly to being asked such questions. ASHAs further added that problems exist but people did not express them freely as they were apprehensive about where to seek treatment and what whether they would have to leave their work to get treated.

Gaps in using technology–based applications/IVR messages

Majority of the participants including ASHAs, community members, field staff and village leaders shared that most of the households had one mobile phone in the family which is often left at home when going out to the fields, or is with another family member, hence is often not available to the patient to receive calls or IVR messages. Other reasons cited by the community members, ASHAs and field staff for not receiving or responding to IVRS were - calls from unidentified numbers, fear of losing talk time due to connecting to those calls, lack of network connectivity, and lack of awareness about handling the mobile phones.

“Usually there are some calls from the mobile companies; if we receive the calls then we were asked to click on some option; because of which some service get activated and our balance (money) get deducted for activation of those services”. (Community members, FGD 5)

Doctors shared that sometimes they found mhGAP-IG tool difficult as they were unclear how to complete options based on the patient’s response. They also added that there were some difficulties in understanding certain terminology using the app.

Field staff felt that poor network was one of the major problems in this area, as most villages were remote and lacked good connectivity.

“We are unable to use mobile phones because of network and signal [connectivity] problem. Another issue is even if we get such calls we do not pay so much of attention as we are not aware of such things”. (Community members, FGD6)

Suggestions to improve the project

Almost all participants suggested the need to create more awareness on mental health issues including help seeking. Most considered medical camps as an appropriate strategy to create awareness among community members. They suggested a need to conduct more medical camps in each village so that it can be accessed by larger number of people.

“If it [camps] is organized in each panchayat [local administrative body] it will be good. If there is a need to go to the PHC they will not go as it is very far, so if there is a camp in [a] village nearby, definitely people will come for treatment”. (Village leader, IDI1)

Some community members also recommended that the intervention should be continued for a longer period of time with support from government. Some community members said that they should be informed about the IVRS call number in advance prior to its implementation. Prior intimation of camps through IVR messages was also recommended. A few community members felt that there should be more involvement of village leaders so that they can motivate people to participate in such projects.

The project should be done by taking support from other organizations and the village leaders [and] administration, which will help in the success of the project. (Village leaders, IDI3)

Another suggestion was that when the patients visit the PHC, appropriate treatment should be provided. This would help in building the trust of the community in the health system.

Nearly all ASHAs and doctors suggested a need to conduct booster training every six months.

Discussion

The process evaluation showed that the interventions delivered as part of the project were feasible and acceptable. A number of facilitators and barriers were identified. To our knowledge, this is the first study from an ST area reporting on the process evaluation of an intervention on mental health services delivery. We reached thematic saturation, and respondents identified facilitators and barriers of the intervention that complements the quantitative data presented earlier [9]. Implementing this intervention in a specifically remote area as this, while may limit generalizability across India due to its unique logistical and geographical challenges, provides valuable insight into how to implement similar health delivery models in similar remote settings for other health conditions.

The barriers and facilitators can be framed using Andersen’s Modified Behavioral Model of Health Services Use [20] (Table 1). While the components - environmental, population characteristics, and health behavior - affect outcomes, the outcomes themselves provide a feedback loop that affects perceived need and health behavior.

Table 1 Identifying barriers and facilitators of mental health services use based on Anderson’s Modified Behavioral Model of Health Services Use [20]

Environmental factors related to healthcare system

The training provided to ASHAs and doctors about CMDs was a key facilitator for the project. Using primary healthcare staff to provide mental health services was appreciated by the community and village leaders. Task shifting of the activity - identification and treatment of CMDs - to primary healthcare workers by enhancing their capacity supports the principles of the National Mental Health Programme (http://mohfw.nic.in/sites/default/files/9903463892NMHP%20detail_0.pdf), National Mental Health Policy [21] and World Health Organization’s Mental Health Action Plan 2013–2020 [22]. An earlier review had also indicated some positive effects of task-shifting in mental health [5]. The perceived positive effect of mobile-technology enabled EDSS supports similar findings from earlier research [6, 7]. However, it was apparent that booster training was needed for ASHAs and doctors on a periodic basis, to improve performance. Even though, only 3 medical camps could be organized due to the short period of the intervention, preliminary data show that 19 out of the 30 patients who visited the doctor, did so at the camps, which underlines the value of camps. The qualitative interviews also showed that besides helping in reducing travel time and associated expenses, the camps were also a platform that increased awareness and helped those attending to share their experiences informally amongst themselves and learn coping strategies from peers.

Population characteristics (predisposing characteristics, enabling resources, need)

The whole population including health staff had poor awareness about mental health and CMDs prior to the intervention. The anti-stigma campaign and the steps taken to increase mental health awareness benefited the population, as was evident in changes in knowledge, attitude and behaviours and stigma perception related to help-seeking [9]. This was also observed in the evaluation of the campaign in another similar population within the same district [23]. However, some individuals believed that mental disorders mainly affect the lower socioeconomic status. This incorrect perception may have been a barrier to collecting information about CMDs from them.

Some key enabling factors were the involvement of ASHAS, organizing medical camps, and support of local administration. Organizing medical camps in the villages helped those seeking care, as they did not have to travel and lose out on daily wages, and it also saved time. Both expenses and extended travel time to seek care from health facilities are key barriers in rural and remote areas, and given the remoteness of the ST area and poor public transport facilities in the area, the value of the medical camps was greater in this community.

The perceived need for seeking care for CMD was enhanced by increasing awareness and the capacity of health workers to screen CMDs and treat them. Use of mobile applications and IVRS facilitated that process even further as it empowered the health workers and primary care doctors by providing them an evidence based tool, and the IVRS messages facilitated the process of follow-up. However, the lack of good quality network connectivity and non-availability of mobile phones with everyone in the family were barriers in implementing the IVRS successfully. One way to improve connectivity is by mapping hotspots in the community using online applications. Increasing the proportion of successful IVRS calls by community members who do not have access to a phone at all times, could be achieved by identifying specific time slots in the day when such individuals are more likely to have phone access. It was also clear that the IVRS number should be shared upfront so that everyone is aware of the source of the call.

Health behaviours including personal health practices and use of health services

The mental health awareness campaign and the extra effort put in by the ASHAs to allow people to respond comfortably to the screening questions helped to reduce the stigma related to mental health. ‘Suicide’ was especially viewed as being too negative and discussing it as not being culturally appropriate. However, the ASHAs were specifically trained in being sensitive while asking the ‘suicide’ related question. While talking about death is considered as a taboo in many cultures, in India, the additional burden of stigma related to mental health, and the legal implications of suicide (which was until recently an illegal activity liable for punishment) may have been additional factors that prevented people from discussing about such issues freely. More information needs to be shared with the community about the importance of psychosocial management. It may be that since medicines were not prescribed commonly, it was thought that ‘treatment was not appropriate’.

While generally the community were supportive of the role of ASHAs and doctors, some individuals were still sceptical about the skills of the doctors to manage CMDs, though the doctors themselves felt that the training enhanced their skills to manage CMDs. It was apparent that stigma was still a major factor that prevented people from accessing care, even though the need for help-seeking was present.

Outcomes related to perceived and evaluated health status and consumer satisfaction

Most community members and health workers felt that the intervention had a positive impact on the community. This was further evident in that there was increased service use and reduced scores on depression and anxiety scales following the intervention [9].

Conclusion

Process evaluations are essential in assessing the implementation of a system-level intervention. IVRS and medical camps appear to be appropriate and favoured strategies to mobilise communities and positively motivate screen positive individuals to avail mental health services. Mobile technology based applications were also deemed as beneficial, though there were some initial problems in navigating the system. Booster training for all staff and health workers, regular medical camps and better strategies to make IVRS messages more effective were identified as key enhancements needed prior to implementing the next steps to maximise the likelihood of effectiveness of the intervention. While awareness about the different intervention components including the anti-stigma campaign was present, more sustained effort is needed to reduce stigma related barriers in the community. Differential accessibility to mobile phones amongst family members is another issue that needs to be addressed by identifying strategies to overcome that barrier. Poor network connectivity should also be addressed when implementing the study in larger areas.

Abbreviations

ASHA:

Accredited Social Health Activist

CMD:

common mental disorder

COREQ:

Consolidated criteria for reporting qualitative research

EDSS:

electronic decision support systems

FGD:

focus group discussion

GAD7:

Generalized Anxiety Disorder – 7 item

IDI:

in-depth interview

IVRS:

Interactive voice response system

LMIC:

in low- and middle-income countries

mhGAP-IG:

Mental Health GAP – Intervention Guide

PHC:

primary health center

PHQ9:

Patient Health Questionnaire – 9 item

SMART:

Systematic Medical Appraisal, Referral and Treatment

ST:

Scheduled Tribe

References

  1. GBD 2013 DALYs and HALE Collaborators. Global, regional, and national disability-adjusted life years (DALYs) for 306 diseases and injuries and healthy life expectancy (HALE) for 188 countries, 1990–2013: quantifying the epidemiological transition. Lancet. 2015;386:2145–91. https://doi.org/10.1016/S0140-6736(15)61340-X.

    Article  PubMed Central  Google Scholar 

  2. Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bulletin of world health. Organization. 2004;82(11):858–66.

    Google Scholar 

  3. World Health Organization. The world health report 2001- mental health: new understanding, new hope. WHO. Switzerland: Geneva; 2001.

    Google Scholar 

  4. Armstrong G, Kermode M, Raja S, Suja S, Chandra P, Jorm AF. A mental health training program for community health workers in India: impact on knowledge and attitudes. Int J Ment Health Syst. 2011;5:17.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Joshi R, Alim M, Kengne AP, Jan S, Maulik PK, Peiris D, et al. Task shifting for non-communicable disease management in low and middle income countries - a systematic review. PLoS One. 2014;9:e103754.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Souza NM, Sebaldt RJ, Mackay JA, Prorok JC, Weise-Kelly L, Navarro T, et al. CCDSS Systematic Review Team. Computerized clinical decision support systems for primary preventive care: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci. 2011;6:87.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330:765.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Maulik PK, Devarapalli S, Kallakuri S, Praveen D, Jha V, Patel A. Systematic Medical Appraisal, Referral and Treatment (SMART) Mental Health Programme for providing innovative mental health care in rural communities in India. Global Mental Health. 2015;2:e13. doi:https://doi.org/10.1017/gmh.2015.11

  9. Maulik PK, Kallakuri S, Devarapalli S, Vadlamani VK, Jha V, Patel A. Increasing use of mental health services in remote areas using mobile technology: a pre-post evaluation of the SMART mental health project in rural India. J Glob Health. 2017;7(1):010408. https://doi.org/10.7189/jogh.07.010408.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Ministry of Tribal Affairs, Government of India. Report of the High Level Committee on Socio-economic. Health and educational status of tribal communities of India. New Delhi: Government of India; 2014.

    Google Scholar 

  11. Pfizer Inc. Patient health Questionnaire-9. 2013. Available http://www.phqscreeners.com/sites/g/files/g10016261/f/201412/PHQ-9_English.pdf . Accessed 29 September 2016.

  12. Pfizer Inc. Generalized Anxiety Disorders-7. 2013. Available http://www.phqscreeners.com/sites/g/files/g10016261/f/201412/GAD-7_English.pdf . Accessed 29 September 2016.

  13. World Health Organization. WHO mental health gap action Programme (mhGAP). WHO: Geneva, Switzerland, 2011. Available http://www.who.int/mental_health/mhgap/en/ Accessed 29 September 2016.

  14. Pfizer Inc. Instructions for Patient Health Questionnaire (PHQ) and GAD-7 measures. 2013. Available https://phqscreeners.pfizer.edrupalgardens.com/sites/g/files/g10016261/f/201412/instructions.pdf Accessed 29 September 2016.

  15. Maulik PK, Tewari A, Devarapalli S, Kallakuri S, Patel A. The Systematic Medical Appraisal, Referral and Treatment (SMART) Mental Health Project: Development and Testing of Electronic Decision Support System and Formative Research to Understand Perceptions about Mental Health in Rural India. PLoS One. 2016;11:e0164404. https://doi.org/10.1371/journal.pone.0164404.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

    Article  PubMed  Google Scholar 

  17. Morse JM, Richards L. Readme first for a User’s guide to qualitative methods. Thousand Oaks, California: Sage Publications, Inc.; 2002.

    Google Scholar 

  18. Miles MB, Huberman A. Qualitative data analysis: an expanded sourcebook.2nd ed. Beverly Hills, CA, USA: SAGE Publications, Inc; 1994.

    Google Scholar 

  19. NVivo. NVivo qualitative data analysis software. In: Version 9; QSR international Pty ltd.; 2010.

    Google Scholar 

  20. Andersen RM. Revisiting the behavioral model and access to medical care: does it matter. J Health Soc Behav. 1995;36(1):1–10. https://doi.org/10.2307/2137284.

    Article  CAS  PubMed  Google Scholar 

  21. Ministry of Health and Family Welfare Government of India. New Pathways, New Hope: National Mental Health Policy of India. India; Ministry of Health and Family Welfare Government of India, New Delhi,2014. http://mohfw.gov.in/sites/default/files/34711242651412939786%20%281%29.pdf. Accessed 20 June 2017.

  22. World Health Organization. Mental Health Action Plan 2013-2020. World Health Organization. Geneva, 2013.

  23. Maulik PK, Devarapalli S, Kallakuri S, Tewari A, Chilappagari S, Koschorke M, et al. Evaluation of an anti-stigma campaign related to common mental disorders in rural India: a mixed methods approach. Psychol Med. 2017;47(3):565–75.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgments

The authors would like to thank all field staff of the project for assisting in participant selection and data collection. The authors wish to thank all participants who took part in interviews and focus groups for this research study.

Authors’ contribution

PKM, SK, SD, AP and VJ conceived the study, provided intellectual inputs to the design of the study, and provided critical intellectual input to the development of this paper. AT, SK, SD and PKM designed and conducted the study. AT and SK analyzed the data, AT and PKM wrote the first draft of the paper and all modifications thereafter. All authors read and approved the final manuscript.

Funding

This project was supported by the Wellcome Trust / DBT India Alliance Fellowship (Grant No: IA/I/13/1/500879) awarded to Pallab Kumar Maulik (PKM), and a Grand Challenges Canada Grant (No: 0524–01-10) on which PKM was the Principal investigator and a Grand Challenges Canada Grant (No: 0524–01-10) on which PKM is the Principal investigator. The funding agencies had no role in the design, of the study, or collection, analysis and interpretation of the data, or in writing this paper.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Pallab K. Maulik.

Ethics declarations

Ethics approval and consent to participate

Ethics approval was obtained from the Independent Ethics Committee of the Centre for Chronic Disease Control, New Delhi. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. Informed written consent was obtained from all participants.

Consent for publication

Not Applicable.

Competing interests

None of the authors have any competing interest.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional file

Additional file 1:

Table S1. Consolidated criteria for reporting qualitative studies (COREQ): 32-item checklist. The Table outlines the COREQ Criteria that were fulfilled in this study. Table S2. Focus Group discussions (FGDs) and in-depth Interview (IDIs) guidelines. This Table lists the questions used in all the qualitative interviews. (DOCX 32 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tewari, A., Kallakuri, S., Devarapalli, S. et al. Process evaluation of the systematic medical appraisal, referral and treatment (SMART) mental health project in rural India. BMC Psychiatry 17, 385 (2017). https://doi.org/10.1186/s12888-017-1525-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12888-017-1525-6

Keywords