english | castellano

Analytical capacities

A policy division has been created within the Ministry of Cooperation. More emphasis is being put on evaluation and research. Outcomes are awaited.

It is increasingly acknowledged that development practice needs to be embedded in research, and that linkages between researchers and policymakers need to be strengthened. There is a long tradition in some Northern European countries of Development Studies, ultimately rooted in the Colonial Studies departments that advised colonial administrations. The Spanish academic and research landscape, with its particular tradition of looking towards Latin America, has caught up lately, although there is not yet any common identity of ‘development studies’, nor any kind of active professional organization. A recent conference on “university and development” [III: Congreso Universidad y Cooperación al Desarrollo] focused in large part on the role universities have in implementing projects with their own budgets, rather than demanding a public funding scheme for development research or debating their role in improving official development cooperation practice as academics. The issue of how to foster research uptake by policy-makers has not yet gained much pertinence in Spain.

The Master Plan – although mentioning universities as actors – remains particularly vague on the issue of evidence-based policy making and research funding. Similarly the Annual Plans do not take on the issue. There have been some steps, however, towards enhancing the relation between research and development practice. The most important of them is the reform of the planning and evaluation unit of the SECI. Related to that are the new bodies’ evaluation activities, and a new instrument for research funding. Additionally, a government-controlled think tank, working on issues of aid effectiveness and social and economic development in general, has been inaugurated.

DGPOLDE: With the arrival of the new government the planning and evaluation unit of the SECI was upgraded to General Directorate. Increasing the staff from three in 2003 to over 50 in 2007, it has become the motor of reform. It has taken on the task of technically supporting the policy setting and coordination mandate given to the SECI by the 1998 law on International Development. Its capacities to participate, from a technical perspective, in international fora such as the OECD/DAC or the Development Meetings of the EC Council have increased markedly. The planning unit has developed an integrated set of planning tools, and has embarked on a furious production of documents (see below). It has increased its capacities to connect with and give direction and effectiveness to the coordination bodies of the Spanish development cooperation system – the inter-territorial and the inter-ministerial commission. Similarly, the working groups of the consultative body of Spanish development cooperation, the Development Council, can count on technical support. DGPOLDE is a young and energetic government department with dedicated leadership, striving for rapid change. Sometimes its dynamism and free-standing, organizational set-up creates tensions with other departments that are constrained in their flexibility by outdated bureaucratic regulations, most notably the Agency for Cooperation.

Evaluation: The evaluation of programmes and strategies provides an opportunity to assess their approach, structure and effectiveness. Evaluation has the dual role of controlling and ensuring accountability, and fostering learning. The Master Plan has declared systematic evaluation one key area for reform. Whereas the planning wing of DGPOLDE has been very active in producing a significant number of planning documents, the evaluation wing has been more silent. Until today, Spain has not yet revisited its evaluation methodology, which has thus not been updated with regard to planning methods or the increase in aid. DGPOLDE inherited four programme evaluations – largely conducted in a classical, project-based style – and commissioned in 2005 three further evaluations, which are supposed to follow a new, more strategic orientation.1 At the time of writing, they had still not been published. ThePlan 2006 promises three further strategic evaluations of one country strategy, one priority sector of the Master Plan, and a programmes of special importance. No terms of reference have been published. The announced mid-term evaluation of the Master Plan, in order to prepare the upcoming Master Plan 09-12, has not been heard of.

Whereas action taken on former evaluations – either in learning, strategic redirection or programme suspension – remains largely undocumented, the follow-up on the new-generation evaluations remains to be proven, given that those commissioned in 2005 have not yet been published. A revision of the evaluation methodology has been promised in the Action Plan 2006, and has been commissioned from the Institute for Regional Development [Instituto de Desarrollo Regional] IDR in Seville. No draft or results have been presented or discussed so far. For now, the methodology for evaluation from 1998 and its specification in 2001 remains valid. Meanwhile, the preparation of a guide for joint evaluation methodologies and training for decentralised cooperation has been announced. But once again, no further action has been reported. It is as yet impossible to determine whether the new-generation evaluations are actually being used to enhance poverty reduction and accountability, strengthen management practices and foster learning, since these exercises – in a imagined three-step process of design, implementation, feedback – have only entered phase two. Given this panorama, it seems that the development community will have to wait some time before harvesting the results of the new structures that are being implemented. 2

For example, the micro-credit fund – 2.2 percent of all ODA in 2005 – was widely criticised for being unclear in its poverty focus, strategically and institutionally detached from the rest of the Spanish Cooperation structure, and poorly integrated into the international microfinance development efforts gathered in the Consultative Group to Assist the Poor. 3 The pending evaluation could provide a mayor opportunity to redirect both the strategy and institutional set-up of Spanish microfinance cooperation as a complementary, internationally coordinated and outcome-oriented instrument in Spanish support to business development as a cornerstone of the fight against poverty. A similar possible outcome might result from evaluation of the relevance to poverty reduction and effectiveness of the SpanishDevelopment Credits (FAD). This, however, is widely acknowledged as being too politically sensitive.4

Evaluation should be institutionally separated from implementation, and this is being done by distinguishing between the powers of DGPOLDE and AECI. However, an internal evaluation unit of AECI would be desirable. For now, this unit does not exist and it is pending an overdue reform.

In contrast, evaluation has been enshrined in the new regulation of “convenios” [framework agreements] with NGOs 5 , in an attempt to shift from a bureaucratic mode of accounting for expenditures towards a more results-focused mode of implementation. No assessments are as yet available on this new mode of implementation and on the feedback mechanisms of evaluation, since these are mainly focused on new forms of financial accountability (reports), while much less has been done with regard to development outcomes.

The institutional design of the framework agreements, however, has been welcomed by both the large, more professional NGOs, and critical academics. The regulatory system has also been warmly welcomed by the private sector evaluation consultancies, who are expecting more business. It is still to be determined which evaluation consultancies would be eligible to carry out a high quality evaluation that satisfies both the learning needs of NGOs and AECI’s oversight mandate, as well as the will to build the capacities of its implementing partners. This could be a market for European consulting firms, bringing fresh ideas with them regarding methodologies. Until now, most big consulting firms in Spain have only dedicated a very small, usually understaffed section to development cooperation, given that Spanish development has been little more than an occasional sideline interest for the big players, concentrated in the far more lucrative Framework Contracts of the European Commission. One of the main queries relating to the near future of Spanish evaluation culture is whether government agencies are willing to invest in evaluation, and consequently take part of a competitive evaluation market, or rather maintain a precarious evaluation system that does not foster potentially painful learning and decision-making processes, but instead “says what the client wants to hear.”

Fostering development research: “convenios de investigacion”: A new instrument of DGPOLDE are research frameworks agreement with public bodies, mostly universities. Some of these contracts focus on very specific deliverables, such as the preparation of a new evaluation methodology by the IDR in Seville, a mapping exercise and the facilitation of better information exchange between municipal actors in development commissioned from the Spanish Federation of Municipalities and Provinces [FEMP], or an integrated electronic information management system for development cooperation activities commissioned from the Polytechnic University of Madrid. Others focus on more strategic issues, such as the study on aid to middle-income countries by the ICEI, on policy coherence and remittances by the Real Instituto Elcano, work on aid architecture, debt and aid effectiveness by CeALCI, and the studies on human security and local human development by HEGOA.

Thanks to these framework agreements, Spanish development policy has gained an evidence basis for its actions. There are, however, a number of constraints and shortcomings that should be tackled. Firstly, for administrative reasons, only Spanish public bodies, such as universities, are eligible for funding under the framework agreements. Secondly, the research is commissioned at the complete discretion of DGPOLDE. The commissioned research is not embedded within a research funding strategy. Thirdly, the results are not systematically published as such, nor is there any publication of who is implementing commissioned research for DGPOLDE. These parameters do not foster a critical mass of research results and informed debate.6

CEALCI: In 2004, a research centre – named Centro de Estudios para América Latina y la Cooperación Internacional CEALCI – was inaugurated under the aegis of the public-private Carolina Foundation. It replaced a centre dedicated to the promotion of cultural heritage in Latin America.7 It researches Latin America, corporate social responsibility, international economic relations, public policies and aid effectiveness. It commissions academic work and edits a number of publications. In 2007, its research agenda focused particularly on social cohesion and regional integration. It is also perceived as a training institution, and catalyst for change within the Spanish development cooperation community. In the future, political leaders will have to decide whether this institution is going to continue to serve as the intellectual face of SECI, gain academic independence, or integrate into a policy division of AECI.