You can’t manage what you can’t measure, goes the old adage. We might adapt that to say, the Government can’t manage what you measure differently. This was a lesson we learned over the past few years at Integrate Health.
Data is critical to decision-making. This is true within our organization, at every level, and within the Ministry of Health, from clinic-based healthcare providers up through the Minister of Health. At its best, good data drives continuous quality improvement and effective budget allocation. So what makes for “good data”? To answer that, we found that we first needed to find a way to speak the same data language as the Ministry of Health.
The government of Togo uses the District Health Information Software (DHIS2) which is an open-source, web-based health management information system used by over 70 low and middle-income countries. DHIS2 allows governments to collect, manage, and analyze data while using just one system. Despite being used by government partners around the world, third-party health actors, including non-governmental and non-profit organizations, often use indicators and data capture systems which are not quite compatible with DHIS2, leaving decision-makers blind to important health system data and contributions from partners.
That was us. Our government partners could not seamlessly access our data, and even when they got it, the way we measured data did not always align with their methods and definitions. Because our indicators were not always aligned to those used by the government, we were not able to demonstrate how we were helping to achieve the health targets laid out in the national strategy. Ultimately, we were making the jobs of our colleagues in the Ministry of Health harder and struggling to demonstrate our impact in a way that was meaningful to them.
We quickly realized that if we wanted to support the Ministry of Health in achieving its goals in improving healthcare, we needed to speak their language and align our systems and indicators to theirs.
This proved to be an extensive undertaking.
Our first step was to compile a list of our own indicators, which had admittedly grown in size and scope since the initial launch of the Integrated Primary Care Program in 2015. We then cross referenced all indicators with Ministry of Health indicators as documented in both DHIS2 and the National Health Development Plan, the Ministry of Health’s national health strategy. Unfortunately, many of the definitions that the Ministry of Health used were difficult to find or interpret and documentation was often lacking. Ultimately, we managed to use the DHIS2 API to determine the calculation methods of various data points, but this required significant sleuthing.
Once our team had built a strong understanding of the DHIS2 system and the indicators contained within it, Integrate Health staff held a series of workshops to bring together various government actors working with DHIS2 to discuss challenges and find solutions. These participatory workshops covered the importance of the DHIS2 system, data quality challenges frequently encountered by our analysts, best practices for validating and using the data, and indicator definitions and calculation methods. The workshops proved to be essential in building consensus and consistency across all parties on the importance of the DHIS2 system, the definition of key indicators and practical measures that each stakeholder could take to improve data quality. After the workshops, Integrate Health teams aligned indicator definitions to be consistent with those of the Ministry of Health wherever possible.
When a home visit is not a home visit
In Togo’s DHIS2 implementation, there is an indicator “Visite a domicile” which translates in English to home-visit. Our staff quite logically understood ‘home-visit’ to measure the number of homes that a Community Health Worker (CHW) visited each month and our team dutifully reported on this every month to the Ministry of Health. However, when we reviewed the definition with the regional representatives of the Ministry of Health, we found that this wasn’t the definition at all. The Ministry of Health measures “Visite a domicile” as the number of times that a CHW leaves their home to go out and conduct home visits. On any given home visit, the CHW could visit 5–10 homes but this would still count as 1 home-visit. As a result, we now report the Visite a domicile correctly as well as a second indicator “Nombre de ménage visités / touches” (number of households visited), which counts the number of households visited during the month. By correctly aligning our definitions, we were able to show that CHWs supported by Integrate Health expand access to healthcare to thousands of households each month.
With common definitions agreed upon, we next integrated our reporting into the government reporting system so that our data would routinely appear in DHIS2, making the full scope of data from Integrate Health-supported sites completely accessible to any government health official. As a result, it is now easy for MOH decision-makers (and our staff) to compare data from Integrate Health-supported sites against other sites that have not received support. We can also quickly benchmark Integrate Health-supported site performance from before and after program launch. For example, we can now show that after launching programs, we see on average a 240% increase in the number of pediatric consultations conducted at a health facility in the first quarter after intervention compared to the same quarter in the year prior to launch. During the same time period, non-intervention sites saw very little change.
Beyond enhanced reporting and data comparison capacity, we now speak the same language as the Ministry of Health which we hope will lead to greater communication and collaboration in the future. Now, instead of turning to us for data, MOH decision-makers can pull compelling data from their own systems and eventually use it to evaluate policies and advocate for change.
When we started this process, we were surprised by how little documentation existed both on how to define and calculate indicators in Togo and we suspect that this is a common issue across many low-income countries. We see this as a major barrier to ensuring that organizations actively place the government at the center of their health information systems. To help, we created a quick tip sheet for helping organizations who use the DHIS2 API understand how indicators are calculated. While we certainly recommend the official DHIS2 Web API documentation, our team found that having a tip sheet to quickly access indicator definitions was really helpful and could be useful for team members who were unfamiliar with coding.
As always, we are interested in hearing how our experience resonates with yours. If you have tried aligning with national health information systems, please let us know. We would love to learn from you. If you use our tool, we would be interested to hear how it goes.
This post was written as part of a Data Series by Patrick Aylward, Chief Operating Officer at Integrate Health.
Biography
As Chief Operating Officer, Patrick Aylward is leading the transformation of Integrate Health’s systems to maximize the value of primary care delivery in line with Ministry of Health objectives. Patrick has over 15 years of experience working in global health in low- and middle-income countries with a focus on data-driven strategy and catalyzing markets for new products and innovations. Prior to Integrate Health, Patrick worked for Population Services International, Unitaid/WHO, the Global Fund, and Intel Corporation. He spearheaded the development of a global investment case for HIV self-testing; developed tools used to track medicine expenditure and patents in more than a hundred countries; and advised the Global Fund, the Bill and Melinda Gates Foundation, and other major donors on market dynamics strategies. He holds BA and MSE degrees from the University of Michigan and an MBA from Yale University.