World Bank, 2001–2006
Since the mid-1980s, major reform efforts had sought to improve the effectiveness of democratic institutions in Colombia. The Colombian judiciary had been subjected to increasing pressure to reform, mostly as a result of the deterioration of its credibility as an institution capable of protecting human and social rights, resolving disputes, penalizing criminal behavior, or meeting the growing demands of the population.1 To assist, the World Bank launched the Judicial Conflict Resolution Improvement Project in 2001, and the project remained in operation until 2006.
What were the overall objectives of the World Bank’s judicial reform project in Colombia? How were the program’s outputs supposed to be used? What were participants expected to learn as a result of the intervention? What were participants expected to do differently as a result of the intervention? And what conditions in Colombian society were expected to change as a result of the intervention?
The World Bank’s Judicial Conflict Resolution Improvement Project in Colombia was intended to improve the efficiency and quality of court ser vices relating to conflict resolution, leading to greater transparency and fairness for the people using the court system. At the time the project was designed, the Colombian government’s development objectives were primarily poverty reduction and peace building.2 The World Bank theorized that poor access to high-quality judicial ser vices contributed to violence, meaning that improvements to courts would contribute to peace in Colombia. At the same time, however, “judicial reform projects sponsored by the World Bank aim solely at enhancing a nation’s economic performance.”3
The judges and employees of five courts, in Bogota, Cali, Medellín, Barranquilla, and Bucaramanga, were designated “Judicial Change Teams” (JCTs). The project employed a “participatory and comprehensive organizational change strategy to bring about change in the [JCT] courts’ operations, which ensure people swifter, fairer, and more transparent conflict resolution ser vices.”4 In sum, the intended outcomes were as follows:
Intended Outputs
What did the intervention intend to produce? How were the intervention’s inputs supposed to be used? Activities: What did the intervention intend to do? Products: What did the intervention intend to create? Participants: Whom did the intervention intend to affect?
Activities. The World Bank’s court reform efforts consisted of designating a number of courts as “Judicial Change Teams” and providing them the inputs that they needed. The hope was that the limited number of JCTs would become much more efficient at processing cases, thereby reducing the national case backlog and potentially leading to positive spillover changes in the rest of the judicial system. The program was funded by the World Bank, which provided money and technical expertise, and was implemented by a new Program Coordination Unit (PCU) inside the Superior Judicial Council of Colombia (CSJ in Spanish). The donors identified a number of obstacles to the JCTs becoming fully functional and sought to address them by providing new case management software, training for judges in case management, refurbished buildings, digitization of court records, a media strategy to publicize changes, and technical training on case management.
Products. JCTs were to be equipped with new tools, facilities, and training.
Participants. The 2007 review of the project identified three direct beneficiaries: Colombian citizens seeking judicial conflict resolution ser vices; private-sector actors, who would benefit from lower risks and better contract enforcement; and judges, who would have improved work environments and better efficiency.6
Promised Inputs
What resources or capabilities was the intervention supposed to provide?
The World Bank planned to provide financing and technical assistance to the CSJ, which would implement the program, to install and operate a new case-tracking system called “Justicia XXI.”7 It also would provide new computer equipment, the inputs needed to re-model court buildings, and training for judges on case management techniques as part of a plan to change the culture in the targeted courts.8 These inputs, financed by the World Bank and using mainly World Bank technical advisors, were to include
Output Prerequisites
What resources, capabilities, or conditions, other than those produced by the intervention, would have been required for the intervention’s outputs to generate the intended outcomes?
As with any project aiming to improve institutional structure or functioning, basic security and infrastructure prerequisites would need to be in place. The transportation infrastructure would need to be good enough in the region the JCTs were located that people could physically travel to courts. The region would also need to be free enough of violence that state institutions could function: court staff and judges would need to not fear for their safety in going to work, and ordinary citizens would need to be free from threats associated with using the courts. For the JCTs to contribute to violence reduction (and then economic development), the courts would need to be able to provide fair ser vices to former or potential combatants, who in turn would need to believe that their interactions with the justice system would be fair.
As in the Cambodian case, the courts in Colombia would need to have some degree of legitimacy among the population if they are to contribute to the medium-and long-term outcomes. Does the target population respect judicial decisions? Do people trust the courts? Do the courts rely on (and have access to) force to impose their rulings? Are the courts seen as fair, accessible, and the best recourse for justice? Are laws and legal norms widely understood by members of the society? Are they understood by competent judges, who staff the court at an appropriate level? Does the society have a corpus of laws?
A key component of the Colombian project was refurbishing courts. Does a causal link between court waiting-room size and judicial quality exist? Courts were provided a computer system running newly written software. For the new software to be useful, it would require reliable access to electricity, trained and willing users, and the funds to pay for upgrades and maintenance. For the computer system to make the court system more transparent, ordinary citizens would need to have access to that information.
Finally, for the project to contribute to overall outcomes, the personnel involved in the JCTs would need to want to operate in a more efficient, fair, and transparent manner—and not be faced with countervailing pressures from within their own institution.
Input Prerequisites
What additional resources, capabilities, or conditions, other than those provided by the intervention, would have been required to produce the outputs?
The World Bank model of providing technical assistance and money but not daily project management means that local institutions bear a heavy burden for allocating resources, managing implementation, and coordinating relations between donors and beneficiaries. This model raises a typical absorptive capacity question: does the implementing institution have enough staff with the right skills, mandate, desire to participate, and financing as the program requires?
To achieve the outputs of refurbished courts and new computer systems, the program requires local labor with the necessary skills to undertake construction and maintain computers. Do such skilled workers exist in the target region? Do they have access to the necessary computer and construction materials? Or will staff and materials need to be imported? Will the program bud get support buying more costly inputs from abroad? And will doing so undermine the local capacity-building aspects of the program? Regardless of the source of materials and labor, this aspect of the project would require electricity and transportation infrastructure and a time frame that accounts for construction or coding delays.
To achieve more efficient case processing, the provided inputs of technical assistance and software would need to be useful for the judges and give them the capacity to process cases faster. Is the software compatible with existing processes and practices? Are those processes and practices entrenched, or could they be modified fairly easily? More fundamentally, the courts would need enough judges and staff, adequately paid and willing to work, in order to achieve efficiency gains, regardless of technique or tools, and judges would need to preside over courts with lawyers who understand court functioning and are able to close cases quickly.
The Colombian project ended in June 2006, after a one-year extension. Out of the $6.6 million total cost for the project, the World Bank allocated a $5 million loan, of which $3.9 million was actually disbursed. The loan instrument used for the Colombia project was a “learning and innovation loan,” intended to support small-scale pi lot projects, testing approaches for future, larger interventions. The final review noted that the Colombia program was much more expansive than a typical pi lot project, involving 37 courts in five large cities.10
The final evaluation of the World Bank’s project in Colombia rated the program’s relevance and efficacy as “negligible” and its outcome as “unsatisfactory.”11 The project did not achieve its key goal of reducing case congestion, except in Medellín.12 Counterintuitively, most of the courts that were not targeted by the intervention saw their congestion drop.13 The evaluation did not seek to assess whether the project led to improvements in Colombia’s economic or security situation, but given that the short-term outcomes were not achieved, it seems reasonable to assume not.
The government of Colombia and its executive agency showed little interest in the project. Even though important procedural reforms to the civil justice were carried out during the project term, the project staff, the government, the legislatures, and the parties that were endorsing those reforms communicated poorly with each other. The PCU was responsible for facilitating the implementation of activities as specified under the project agreement, including ensuring that contracts, disbursements, and the financial management of the project were in compliance with the norms of the World Bank and the project agreements, but high staff turnover impeded these efforts.14
Although most of the World Bank’s involvement in the project was confined to disbursing funds, delays were frequent. The Organization of Ibero-American States (OEI) was the procurement agent; however, the agreement entered into between the CSJ, PCU, and OEI entailed intricate procurement processes, which led to frequent obstacles to disbursements of funds from OEI. In addition, the accounting system was deemed inadequate and lacking in reliability, the application of Colombian contracting law was not fully compatible with the World Bank’s guidelines, and frequent turnover among CSJ authorities exacerbated poor communication.15
As the project progressed and encountered obstacles, the World Bank and CSJ did not change the component objectives to reflect the challenges.16 Thus, the implementation of the project ignored the financial crisis in Colombia, despite the fact that defaults and debt collection created much greater demand for judicial ser vices. The midterm review conducted in 2003 observed that the project’s outcome indicators were not aligned with standard CSJ indicators.17 The entire project proceeded under the assumption of constant demand for legal ser vices and a constant legal framework, even as the number of cases increased dramatically with Colombia’s financial crisis and subsequent legal changes.18
The program’s outputs were also occasionally in dispute, making final evaluations difficult but with even more troubling implications for the project’s design and implementation. For example, inside the CSJ, the implementing agency for the project, there was no consensus about whether the output of the software component was training in the new software or the new software itself.19
In addition to facing obstacles in producing the outputs, the program did not account for the factors that would prevent the limited program outputs from leading to the broader desired outcome. Although the evaluation was conducted too soon after the end of the project to definitively assess outcomes, it highlighted a number of trends that did not bode well for the long-term impact of the project.
First, for the project to be sustainable, one of the participating organizations would have needed to become the guardian of the institutional memory of the project. Because the CSJ was the implementer of the project, it would have amassed the most experience and should have absorbed the lessons of the project best. But it did not show interest in doing so: “The way in which the process concluded was inadequate, there was a gap, people were up in the air, no one knew what had been achieved what was missing, what had happened, what was going to continue.” Transferring that knowledge to a different organization would have been hampered by the “deficient” status of the program’s documentation of what went well and what went poorly with the project.20
Also troubling for the project’s greater impact, its design and priorities deviated from the basic objectives of the World Bank’s strategy for Colombia and of the government, which were to reduce violence, poverty, and the linkage between the two. The project executed in Colombia was concentrated only on civil courts, thus deviating from the idea that poor resolution of conflict was one of the main causes of violence. Even more important, the problem of access to justice was left aside, despite being a pivotal element in the diagnosis of violence.21 The project also deviated even from its own stated objective. According to the judges surveyed in the opinion poll conducted in 2005, the project focused so much on internal procedures that it mostly affected judges and court staff as opposed to users, who were initially conceived as the main benefactors of the project.22
During the project’s term, some of its objectives were achieved, but this occurred completely in dependently of the project’s efforts. The court backlog, one of the primary targets of the intervention, did indeed shrink by 50 percent between 2002 and 2005.23 However, this was due to changes in the Code of Civil Procedure and the government’s introduction of 45 “decongestion courts”— which were not the same as those courts designated as JCTs.24
How well can the donor design locally appropriate interventions?
The 2010 World Bank project review identified a number of problems with planning that contributed to the failure of the project as a whole. Planners did not have evidence of a link between the project and the World Bank’s priorities for the project: “There was no evidence, for example, of the tacitly assumed link between performance of the civil courts and poverty alleviation, or violence reduction, two key elements that would have anchored the project within the country’s priorities.”25 Specifically, the project sought to reduce court backlogs to improve economic growth and peace, but the 2010 evaluation was “unable to locate within the literature and ESW of that time any authoritative references highlighting the operation of the civil courts or the non-performing loans in the financial system as basic obstacles to growth in Colombia.”26 As the project went from inception phase to completion, Colombia experienced a financial crisis, which resulted in a surge of cases dealing with nonperforming loans. The government of Colombia responded to the increased backlog by creating the aforementioned special decongestion courts. The project’s efforts to adapt to these changing circumstances were “cursory.”27 These external changes in Colombia’s society and economy meant that the monitoring and evaluation indicators for the project “were drastically affected by factors exogenous to the project itself.”28
The evaluation did not address what processes, culture, and incentives inside the World Bank might have exacerbated poor planning. CSIS interviews suggested that the planners designing the Colombian program clearly thought about what inputs the JCTs would need in order to become functional and sought to provide them. More generally, processes are in place within the World Bank to take advantage of regional, sectorial, and technical expertise among and beyond the Bank’s personnel, and there is generally a culture that is conducive to peer review of plans and evaluations. But some personnel interviewed for this project noted that program and project designs that are inappropriate to local conditions can still get implemented if the team does not take full advantage of the opportunities for such reviews. That seems to be the case here.
Moreover, staffing and promotion procedures can exacerbate bad program design and constrain corrective actions once the loan has begun. One of the most important elements for career advancement at the World Bank is getting a loan approved.29 Project outcomes are less important for career development, because by the time final assessments are made, those responsible for the project have generally moved on to another program, area, or project.30 There is a cultural bias toward larger projects, because loans all have similar overhead and administrative burdens. But the World Bank does seem to allow for more bud get flexibility than some other donors, and processes are in place to reduce the scope of projects if absorptive capacity constraints are discovered, though whether such processes are used seems to depend on the personalities involved. Again, that seemed to be a shortcoming in this program.
1. “Project Appraisal Document on a Proposed Learning and Innovation Loan in the Amount of US $5 Million to the Republic of Colombia for a Judicial Conflict Resolution Improvement Project,” World Bank, Report No. 23184, November 8, 2001, p. 8.
2. Richard Messick, “Judicial Reform and Economic Development: A Survey of the Issues,” World Bank Research Observer 14, no. 1 (1999): 117–136 and Country Assistance Strategy Progress Report for the Republic of Colombia for the Period FY03-FY07: Report No. 32999 (Washington, D.C.: The World Bank, 2005), p. 8., qtd. in “Clustered Project Performance Assessment Report,” Independent Evaluation Group, World Bank, June 30, 2010, p. 9.
3. Richard Messick, “Judicial Reform and Economic Development: A Survey of the Issues,” World Bank Research Observer 14, no. 1 (1999): pp. 117–136, and Richard Messick, “Judicial Reform: The Why, the What and the How,” paper presented at the conference Strategies for Modernizing the Judicial Sector in the Arab World, Marrakech, Morocco, March 15–17, 2002, qtd. in “Clustered Project Performance Assessment Report,” p. 1.
4. “Project Appraisal Document on a Proposed Learning and Innovation Loan,” p. 2.
5. “Clustered Project Performance Assessment Report.”
6. “Implementation Completion and Results Report (Loan No: 70810)— Learning and Innovation Loan in the Amount of US $5 Million to the Republic of Colombia for a Judicial Conflict Resolution Improvement Project,” World Bank, June 5, 2007, p. 3.
7. Justicia XXI itself was commissioned and paid for by the CSJ separately from the World Bank project (see “Clustered Project Performance Assessment Report,” p. 29; “Implementation Completion and Results Report,” p. 5).
8. “Clustered Project Performance Assessment Report,” p. 27.
9. “Implementation Completion and Results Report,” p. 4; “Clustered Project Performance Assessment Report,” p. 5.
10. “Clustered Project Performance Assessment Report,” p. ix.
11. Ibid., 31.
12. Ibid., 28.
13. Carlos Vizcaya, Proyecto de Mejoramiento de la Resolucion de Conflictos Judiciales: Evaluacion de Resultados, Informe No. 2 Ajustado (Bogata: Universidad Externado de Colombia, Facultad de Administracion de Empresas, Direccion de Desarrollo Gerencial, August 2008), p. 125, qtd. in “Clustered Project Performance Assessment Report,” p. 28.
14. “Clustered Project Performance Assessment Report.”
15. Ibid.
16. “Implementation Completion and Results Report,” p. 6.
17. Ibid., 8.
18. “Clustered Project Performance Assessment Report,” p. 27.
19. Ibid., 29–30.
20. Ibid., 32–33.
21. Ibid, 37.
22. “Implementation Completion and Results Report,” p. 38.
23. Maria Mercedes Cuellar, ¿A la Vivienda Quien la Ronda? Situacidn y Perspectivas de la politicade vivienda en Colombia (Bogata: ICAV, Universidad Externado de Colombia, 2006), pp. 395–396, qtd. in “Clustered Project Performance Assessment Report,” p. 28.
24. Ibid.
25. “Clustered Project Performance Assessment Report,” p. 11.
26. Ibid., 12.
27. Ibid.
28. Ibid., 26
29. Author interviews, November 2012.
30. Ibid.