Foundations are naturally concerned that their grantmaking produces meaningful results, but they often fail to take evaluation seriously, and the limited human and financial resources that they put into it reduce its potential value.

Yet, as the evaluation undertaken by the Open Society Institute’s Network Women’s Program (NWP) illustrates, evaluation can be a powerful tool for funders and grantees alike. Carried out in a participatory manner, evaluation provides valuable information on the effectiveness of an initiative. At the same time, the process itself can build the capacity of grantee partners, leaving them with specific skills to help them monitor their work long after the funder’s evaluation is complete.

This article looks at both the results of the evaluation and the impact of the process.

The Documentation and Evaluation Project

The Documentation and Evaluation Project (D&E) is a recently completed multi-site, multi-country[1] evaluation of the NWP. The NWP’s mission is to promote ‘the advancement of women’s human rights, gender equality, and empowerment as an integral part of the process of democratization’.[2] It initiates and supports women’s projects in the Soros Network of foundations, which seek to raise public awareness of gender issues, influence policymakers, and eradicate violations of women’s rights while also helping to create sustainable women’s movements and cooperation among women’s organizations.

The methodology of the D&E Project was designed to be flexible enough to reflect the range of its activities and contexts, yet rigorous enough to accurately capture the nature of its influence. In part this meant using field-based measures and including local researchers and NGO representatives in the design and implementation of the study.

The findings

The situation and status of women
The local research consultant in Montenegro voiced a theme that echoed across the entire NWP network: ‘As a consequence of a 10-year social crisis, we have witnessed an intensive process of reviving the patriarchal, which … endangers the realization of women’s human rights.’ In addition, since the transition, and with the general conditions of instability to which it has given rise, physical and psychological violence towards women has increased sharply. Data from virtually every country confirmed this.

Heavier burdens of work, too, have been imposed on women by economic crisis. The data is full of references to coping with poor health and very difficult living situations, including, in some cases, ongoing effects of conflict in the region. Finally, a country’s ratification of the UN Convention on the Rights of Women is no guarantee that these rights are actually protected. The Ukraine country report noted that, ‘As early as in 1980 Ukraine … ratified the UN Convention on “Elimination of All Forms of Women’s Discrimination” but the real mechanism of providing women’s rights in practice and control over the observance of the laws protecting women’s rights is absent both in legislation and real practice.’

These factors shape the context in which the NWP (and the women’s projects) worked, which meant that many of their efforts, at least initially, had to work against the tide.

Knowledge of women’s rights as a catalyst for change
The evaluation revealed an ‘awakening process’ as beneficiaries acquired a knowledge of women’s rights. In Central Asia, for example, participation in the NWP’s sponsored Human Rights Advanced Training led to a growing interest in legal literacy. This in turn led to the creation of a hotline, and ultimately an education centre for women and girls, giving them a sense of common identity and interests. ‘Women from these countries often do not see themselves as independent actors with individual rights, nor do they feel free to access outside resources,’ noted a Central Asian evaluation report. ‘If a woman knows that there are options – organizations for material and legal support – she becomes more active and uses that knowledge as a means of bargaining within her family, with her husband, and thus creating a different distribution of power.’

Breaking the silence
Another important impact of the NWP’s efforts was to bring the issue of domestic violence out into the open. Staff at grantee organizations, as well as participants in their programmes, repeatedly mentioned the significance of this. One staff member felt that ‘during the project period the interest in the subject increased due to our public events, radio talks, articles in newspapers, training courses … and this made it so that this problem could not be treated as a taboo, but as a problem which has to be addressed.’

Working to change policies and laws
New knowledge and skills provided by the NWP to locally based programmes led to systemic changes. The words of two interviewees capture this broader impact. ‘There were positively changed relations between representatives of law enforcement bodies and NGOs,’ said one. ‘On some issues we managed to attract the attention of power structures, enrol their representatives in seminars, roundtables, working meetings and discussions. An important result was the adoption at the government level of a Draft Law “About Equal Rights and Opportunities” … followed by a Decree of the President to increase the number of women in senior positions in government.’

‘The Human Rights Training effectively linked theoretical knowledge with the practical means to advocate and lobby for human rights … and transform them into political issues,’ said another interviewee. ‘I am actively involved in the women’s human rights movement in the country … and I have founded another national women’s non-governmental organization, Women’s Center of Democracy and Human Rights.’

Enduring changes
There were numerous instances of programmes, activities or organizations continuing (and often building on) their work after the NWP’s funding ceased, indicating that these were viable and sustainable organizations that met genuine needs. Using volunteers and seeking funds from other sources is evidence of an NGO’s strong commitment. In Estonia, a series of post-evaluation tactics emerged, including roundtables of local NGOs and debates on issues in which there was little active interest, such as violence against women. As greater public attention was garnered, grantees could look forward to increased funding possibilities for such issues.

The evaluation process as a catalyst for change

The evaluation process itself proved to be a catalyst for change, influencing grantmaker-grantee relationships in numerous ways. In Estonia, for the first time and as a result of the improved communication and understanding that developed as funders worked with grantees on the evaluation, grantees saw themselves aligned with funders as co-learners and enquirers. In the case of the Albanian Foundation’s Women’s Programme, they were able to avoid mistakes and at the same time reassess previously planned strategic steps.

The NWP was operating in formerly closed societies, and the fact that the evaluation was conducted in a confidential, transparent and cooperative manner fostered a new level of trust for the participants. As a local research consultant noted, ‘The most challenging and yet exciting element of this evaluation was that the process enabled those who were responsible to be critical of their own work and themselves but in the most friendly, genuine and human way. This was a novel experience.’

The value of ongoing evaluation

The NWP’s funding produced a range of impacts and outcomes. According to a respondent in Albania, ‘Soros has been the first donor from a gender perspective in Albania … We have affected the women and girls community … government … and media with our programme.’ As a result of NWP-funded projects, participants went from talking about issues to taking action individually and collectively.

Although the D&E project revealed the effectiveness of the NWP’s efforts, it was not only the programmes that made a critical difference on the ground, but the evaluation itself. When evaluation is an ongoing process rather than a one-off event, it keeps interest alive and fosters self-reflection. Both are critical for continued development and improvement. Additionally, involving stakeholders in evaluation from the beginning is an effective way to secure their commitment and cooperation. Grantmakers and grantees are often so involved in the day-to-day challenges of their work that they seldom step back and examine their objectives and methods. The result is that quality can decline. There are many potential benefits to incorporating research and evaluation into one’s organization. Evaluation is an effective, possibly the most effective, means of monitoring and promoting state of the art initiatives – if, and only if, the findings are put to work.

1 At the time of the D&E Project, the Network Women’s Program was operating in 26 countries in Central and Eastern Europe, the countries of the former Soviet Union and Mongolia.

2 See Bending the Bow: Targeting Women’s Human Rights and Opportunities. New York: Open Society Institute, 2002.

This article reflects the philosophy and methodology of the D&E Project itself. It is the combined effort of the evaluation consultants Betsy Brill, President of Strategic Philanthropy, Ltd and Judith Musick, PhD along with the local Soros staff and research consultants: Valdete Sala, Women’s Program Director at the Open Society Foundation-Albania; Iris Luarasi, local research consultant in Albania; and Selve Ringmaa, local researcher and subsequently the Women’s Program Coordinator at the Open Estonia Foundation.

Betsy Brill, Founder and President of Strategic Philanthropy, Ltd and Judith Musick can be reached at info@stratphilanthropy.com