Makerere Trains Policy Analysts, M&E Specialists to Assess Government Programme Impact

By Okeya John & Davidson Ndyabahika

Mr. Godfrey Mugisa Masereka, a participant in the three-day transformative training in Impact Evaluation and the Manager Planning, Monitoring, and Evaluation at the National Agriculture Advisory Services (NAADS) - a Government entity established in 2001 under the Ministry of Agriculture, Animal Industry, and Fisheries (MAAIF), tasked with effective and timely delivery of advisory services to enhance agricultural productivity and profitability for smallholder farmers in Uganda, said the training couldn't have been more timely, given its impact.

The Impact Evaluation for Evidence-based Decision Making themed training, conducted at the Makerere University School of Public Health new auditorium from 1st - 4th July 2024 by Makerere’s Schools of Public Health  (MakSPH) and Statistics and Planning (MakSSP) through the Hewlett Foundation funded Project for Impact Evaluation for Evidence-Based Decisions (IEED), attracted over 40 Monitoring and Evaluation (M&E) specialists and policy analysts from over 20 Government Ministries, Departments, and Agencies (MDAs) and Parliament of Uganda.

Aimed at strengthening evidence-based decision-making, accountability, and improvement of government programmes in Uganda, coordinated and supported by the Office of the Prime Minister (OPM), Mr. Masereka affirmed that the IEED training “provided an opportunity to refresh our minds on how we report results.”

“Most of the time we have been focusing on reporting outputs, the government has shifted to outcomes and impact, and therefore, there is a need to build a cadre of skilled M&E officers who can adequately apply rigorous skills to report results,” said the NAADS M&E Manager.

Similar to him, Ms. Fiona Auma, the Monitoring and Evaluation Officer at the Petroleum Authority of Uganda (PAU), and Mr. Tonny Kawuki, the Principal Specialist in Statistics at the Directorate of Value for Money and Specialized Audits, Office of the Auditor General (OAG), found the training invaluable in deepening their understanding of Impact Evaluation methodologies, enhancing their practical application of the frameworks, and boosting their confidence to advocate for more internal support to ensure effective monitoring and delivery of government programmes to the public.

“The capacity is there within the Government, it is the will and confidence that you have given us that it can be done - we can take steps towards the right direction, to actually collect the data and also, start to make those evaluations, with small baby steps, so that we can answer one critical question that one of the facilitators here raised, does it work?” Mr. Kawuki observed during a panel discussion on the final day of the training.

Key issues emerging during the training included insufficient budget allocations for monitoring and evaluation, which are frequently the first to face cuts. Government M&E officers and policy analysts also expressed concerns about the suboptimal use of data for evidence-based decision-making and informing top management about Government programme directions.

Participants also noted that impact evaluation was frequently left out until the last minute, with officers often excluded from the initial programme planning and design and yet still expected to evaluate at its close, undermining the effectiveness of the monitoring and evaluation function within the Government and hampering learning before new programme rollouts.

“The training has given us different methods so that we don't just jump onto evaluation but think through what evaluation is necessary for the particular kind of evaluation and the data you have, but most importantly, that you must plan for the evaluation at the time of planning,” Ms. Namirmbe Hafsa, the Principal Policy Analyst, Office of the President, said. 

This training follows a May 2023 baseline study by the Office of the Prime Minister, which identified gaps requiring training to strengthen Monitoring and Evaluation (M&E) among Government M&E and policy officers. It highlighted the need for a comprehensive training programme to enhance the capacity of the officials to conduct effective Impact Evaluations and improve the evaluation and implementation of Government programmes in Uganda, for programmes to deliver the set objectives on time and within budget.

The assessment, which included a survey and focus group discussions with over 30 officials, reported that despite recognising its importance, nearly 75 per cent of the M&E and policy officials from government MDAs assessed had never conducted an Impact Evaluation, citing a lack of skills, expertise, resources, and logistics as major constraints. Over 90 per cent of these rated themselves as either weak or inexperienced in using rigorous Impact Evaluation methodologies including cost-effectiveness analysis, randomised controlled trials, and regression discontinuity, which then informed part of the curriculum for the IEED training program.

Moreover, the results also showed that while all respondents could distinguish between monitoring and impact evaluation, only 30 per cent had prior training in Impact Evaluation, primarily from short courses. Most participants reported that they had not received any formal training but expressed a strong interest in acquiring more skills in designing, planning, implementing, managing data, analyzing, and disseminating findings of impact evaluations.

“Most of the time we are stuck on one or two methods because we lack knowledge of using a variety or alternative methods but this course provided me an opportunity to explore alternative methods of undertaking impact evaluation. I believe I'm going to influence my team, the M&E team at NAADS, to see to it that we cope with the trends of undertaking impact evaluation, especially for government programmes,” Mr. Masereka revealed after the training.

The Impact Evaluation for Evidence-Based Decisions (IEED) project is a joint initiative by academics and researchers at Makerere University’s School of Public Health and School of Statistics and Planning with support from Hewlett Foundation, working closely with the Government, the International Initiative for Impact Evaluation (3ie), Center for Effective Global Action (CEGA), regional academic institutions from Uganda and beyond, Civil Society Organisations, and the Private Sector to integrate evidence into policy-making in helping to achieve Africa's development goals, aligned with the African Union 2063 and the UN Sustainable Development Goals 2030 global agenda.

Over the past three years of implementation, IEED has enhanced the capacities of policymakers to interpret and use impact evaluation evidence for informed decision-making. These efforts have resulted in more effective, efficient, and responsive government action. At Makerere University, IEED has strengthened the skills of researchers and students through training and integration of Impact Evaluation into academic programmes, contributing to professional development among the graduates for impactful roles in evaluation.

Mr. Dunstan Balaba, the Permanent Secretary (PS), Office of the Prime Minister, represented by his colleague at OPM and Commissioner for Monitoring and Evaluation, Mr. Timothy Lubanga, noted that the training for the Government officials fits within the calendar and is part of the broader reforms in Monitoring and Evaluation in Government, revealing that: “While we (Government) have improved our M&E systems and practices, we need to strengthen even more. We have a number of reforms in M&E as part of the reforms in the National Planning Process and we have included this training as part of those reforms''

The Permanent Secretary reassured that the Government is establishing a robust cadre of M&E officers to ensure accountability, improved programme effectiveness, and informed decisions through ongoing recruitment, and establishment of M&E norms, standards, practices, and plan. The Government has also established a digital M&E system for National Development Plan (NDP) III - which is soon phasing out - and NDP IV - which is expected to be out by September 2024 awaiting implementation. He encouraged the practitioners to champion the reforms in their day-to-day work and continue to actively engage in their respective programme working groups.

Important to note, is that Uganda’s NDP IV will now focus on achieving higher household incomes and employment for sustainable socio-economic transformation, with the theme of sustainable industrialization for inclusive growth, aimed at growing the economy tenfold from USD 50 billion to USD 500 billion over the next 15 years. As Uganda prepares for the upcoming National Development Plan IV, the groundwork laid by the IEED project will be seen as instrumental in achieving the country’s socio-economic transformation objectives.

The MakSPH Dean and IEED Project Lead, Prof. Rhoda Wanyenze joined the IEED Steering Committee Chairperson and former Makerere University Deputy Vice Chancellor in Charge of Academic Affairs, Prof. Umar Kakumba in thanking all the Project partners. She commended the Government in particular, through OPM, for supporting the initiative to enhance the use of evidence in decision-making, towards the realisation of more responsive, accountable, inclusive, effective, and efficient systems to deliver public programmes.

“I hope this is the beginning of the engagements that we are having, in terms of how we plan better and be more intentional in using evidence for decision-making, because only then, can we have value for money. Money is limited and we need to invest in what works but also, make sure we are investing in inclusive programmes so that we are not leaving some people behind because when we leave people behind, it affects all of us as a society.” Prof. Wanyenze appealed.