How a new approach to monitoring and evaluation can make sense of complex projects

Building the foundation for results, sense-making and learning

‘Demonstrating real world impact can be an elusive holy grail for most NGO programme monitoring. Collaboration with Sophoi to develop the #BuildingLife RSL system has taken us a big step forward in our journey to achieving that.’

The more we draw from evidence and experience, the better we can understand and refine our next step. And when it comes to measurement and evaluation, drawing on results, sense-making and learning (RSL) is key to making informed, efficient and effective progress, and critical impact.

The #BuildingLife Initiative is a complex policy advocacy campaign co-funded by IKEA Foundation and Laudes Foundation and implemented by the World Green Building Council (WGBC). Working across 10 European countries, involving multiple stakeholders, it sets out to show how EU policymakers and the built environment sector can work together to fully decarbonise buildings and construction by 2050. 

#BuildingLife is implemented by the World Green Buildings Council and co-funded by the IKEA Foundation and Laudes Foundations. Through the initiative, 11 whole-life carbon policy advocacy campaigns seek to inform and change built environment policy in their countries. Their collective aim is to meet Paris net-zero targets and address the climate crisis.

For #BuildingLife to effectively monitor, evaluate and learn, the co-funders engaged Sophoi to collaboratively design, develop and deliver a new approach for #BuildingLife – Results, Sense-making and Learning (RSL). Here we explore our collective experience and the innovative elements this approach offers over conventional monitoring and evaluation approaches.

What is RSL in practice?

‘Sense-making is the process of, and capacity to, make sense of the world so you can act in it. Our RSL approach builds sense-making into an Initiative’s monitoring and evaluation system – enabling stakeholders to apply their experience and insights to make sense of their own evidence’. Robbie – Sophoi

An enhanced learning approach, the essence of the RSL system design is to integrate ‘evidence’ with stakeholder ‘experience’. The approach has three interlinked and uniquely insightful elements:

• Results: evidence of results is collected through frameworks that are specially designed to engage with complex contexts and diverse perspectives.
• Sense-making: a broad set of stakeholders explore and interpret evidence combined with their own experience by taking part in a series of structured and semi-structured sessions. This highlights new patterns and insights.
• Learning: insights generated from sense-making inform decision-making on course correction activities and adaptations.

BuildingLife sense-making

The RSL system runs on a yearly cycle, with evidence gathered quarterly, bi-annually and annually. Formal sense-making activities are facilitated annually to generate insights at three levels: individual-jurisdiction, inter-jurisdiction, and whole-initiative level. As #BuildingLife is delivered by teams in 10 countries across Europe (and throughout the Covid-19 pandemic), the annual sense-making activities take place online using a combination of video conferencing and virtual workspace platforms.

Level 1: Individual jurisdiction sense-making

An initial online workshop enabled each of the 11 teams (around five people per team) to review their jurisdiction-level results and reflect on overall progress – what had worked well and what hadn’t.  Insights and course corrections were documented by the teams to carry forward in their work.

Workshop output of the Finnish team in a collaborative workspace

Level 2: Inter-jurisdiction sense-making

This second-level workshop enabled the 11 jurisdiction teams to review the evidence streams of other teams and reflect on what insights they could draw about working effectively with each other across jurisdictions. Participants interpreted other teams’ evidence and used their own experiences to identify patterns, contrasts, insights, possibilities, and lessons. Participants were encouraged to embrace the complex adaptive systems approach (which is at the heart of the overall #BuildingLife Initiative design).

Workshop output from teams identifying patterns, insights and possibilities to integrate their shared work.

Level 3: Whole-Initiative sense-making

This final workshop enabled the central #BuildingLife Coordination Team, together with their funding partners IKEA Foundation and Laudes Foundation, to review the progress results of the whole initiative based on aggregation and synthesis of evidence from all 11 jurisdictions. The sense-making included using the Laudes Foundation’s rubrics and rating system initiative to track the contribution of BuildingLife to key system changes and distil lessons from that experience. The team documented insights and course corrections to inform strategic planning and funding decisions for 2022.

Workshop outputs showcasing insights highlighted by mapping 29 individual jurisdiction ‘Outcome Stories’ against corresponding BuildingLife outcome areas

What is innovative and powerful about RSL?

‘The emphasis on sense-making has helped us to really interrogate the evidence our monitoring is showing. In collaboration with our funders, the RSL has helped us to step back and really question what’s going on. The insights it has given us have helped us adapt and course correct as we go – helping increase overall impact.’  #BuildingLife Team

Feedback from the #BuildingLife teams was consistently positive. The design and development of the RSL offers significant innovations and advantages over more conventional monitoring and evaluation (M&E) approaches – from the insightful data gathered to the new processes and people it inspired. These benefits include:

• Process over product. RSL emphasises the value of a collaborative, ‘living’ process over a final written product. As the sense-making workshops happen annually, stakeholders can incorporate learnings into planning for the following year, making it possible to correct course.
• Initiative stakeholder over evaluator. RSL shifts the ownership of gathering and interpreting evidence from independent evaluators to stakeholders who are best informed and matter most within and across an initiative.
• Combines evidence with experience. The RSL process explores and interprets evidence of progress, results and contextual dynamics using stakeholder experience. This is sense-making.
• Encourages collaborative relationship forming and network building. Semi-structured, actively facilitated sense-making workshops create new and positive connections by explicitly asking stakeholders to discuss and relate to each other. ‘This is what worked for me. What worked for you and why?’
• Fully remote/virtual process if needed. Sense-making processes can be conducted remotely through video conferencing and virtual workspaces, enabling real-time collaborative exercises that produce tangible learning products.
• It’s inclusive. Based on our experience people welcome participating in sense-making – both meeting each other and sharing experiences and insights, and embracing the idea of being a ‘living’ RSL initiative.

The RSL approach is unique in that it embraces complexity, complex change processes and complicated initiatives. Stakeholders are encouraged to bring their unique understanding of their local context and contextual dynamics to the sense-making process and the insights it produces. RSL is well-suited to multi-jurisdiction / multi-country initiatives working on complex change processes, such as policy advocacy to address the threat of climate change.

Robbie Gregorowski is the founder of Sophoi where he evolves the Results-Sensemaking-Learning approach and supports its further integration into M&E practice. Zand Craig designs and facilitates multi-stakeholder sense-making and learning processes from teams working to address complex challenges linked to climate change and conflict resolution. Ly Nguyen co-leads the monitoring, learning and evaluation of climate action portfolio at IKEA Foundation. And Savi Mull is Senior Evaluation Manager in the Effective Philanthropy Team at Laudes Foundation.