Assessments of Learning
APRESt (Andhra Pradesh Randomized Evaluation Studies). These are a series of large-scale randomized evaluations carried out in the Indian state of Andhra Pradesh since 2004. Cumulatively over 150,000 students have been tested in primary grades (2 to 5) on literacy and numeracy skills using an assessment designed by Education Initiatives (see below). The main researchers have been Professor Karthik Muralidharan of University of California–San Diego and Venkatesh Sundararaman of the World Bank. The research has been supported by the Azim Premji Foundation and the government of Andhra Pradesh. Because the same questions are asked in multiple grades it is possible to trace out improvement in percent answered correctly across grades. Some of the tables and figures in chapter 1 are drawn from results provided directly to me by Professor Muralidharan.
ASER. The Annual Status of Education Report is an annual report originally produced by the Indian education nongovernmental organization Pratham (www.pratham.org); it is now facilitated by Pratham and produced by the ASER Centre (www.asercentre.org). The core of the ASER report is based on a large-scale survey (more than 500,000 children) that covers rural areas in (nearly) all districts of India. The survey does an in-home (not school-based) assessment of the reading and arithmetic competencies of all children in sampled villages ages 3 to 16. The assessment is done in the child's preferred language. The ASER has been produced each year from 2005 to 2012, with the fieldwork done September to December and results available by the following January (so the ASER published in 2006 refers to results from just a few months earlier in 2005).
The ASER instrument in 2008 also included items about practical skills, such as telling time and understanding money. Chapter 1 uses the raw data provided to me by the ASER Centre for analysis of learning profiles.
DHS (Demographic and Health Surveys). The Demographic and Health Surveys are a collection of comparable survey instruments and study methods carried out since 1984 as part of the USAID-funded Monitoring and Evaluation to Assess and Use Results Demographic and Health Surveys (MEASURE DHS) project. While the main focus of the surveys is family planning and child health, the survey instrument records the education attainment and enrollment of all children in the sampled households. This, combined with information about the household such as parental education, residence, and assets, allows the construction of enrollment profiles by age (and household or child characteristic) and attainment profiles of a cohort. Using these raw data (plus others), Deon Filmer, a researcher at the World Bank, documents and analyzes differences in educational attainment around the world as part of a project, Educational Attainment and Enrollment around the World (see Filmer, “Education Attainment and Enrollment around the World: An International Database,” Washington: World Bank, 2010. http://econ.worldbank.org/projects/edattain). He provides data and graphs of enrollment and attainment profiles for a very large number of countries.
EI SLS (Education Initiatives Student Learning Study). The organization Education Initiatives (www.ei-india.com/) was founded in 2001 and specializes in the design and implementation of instruments for testing and assessing in schools. In 2009 (between January and September) with funding from Google.org, it carried out a large-scale assessment of children in grades 4, 6, and 8 testing more than 100,000 students in eighteen Indian states in both urban and rural areas. The instrument covered both language and math and tested both procedural learning and conceptual understanding in those domains.
LEAPS (Learning and Education Attainment in Punjab Schools). The LEAPS project (www.leapsproject.org/site/about/) is an effort by researchers from the World Bank, Pomona College, and Harvard University in collaboration with the government of Punjab launched in 2001. To measure learning outcomes, the LEAPS project administered detailed exams in English, math, and Urdu to students in grade 3 from 823 schools in 112 villages from 3 districts of Pakistan and then followed those same students until grade 6. By tracking the same children, the LEAPS data have learning profiles for the same students over time (in contrast to comparing results from the study of different students across grades at the same time or grades over time).
NAEP (National Assessment of Educational Progress). The NAEP (http://nces.ed.gov/nationsreportcard) is a nationally representative assessment of student skills in various subject matters and grades. Its long-term studies examine learning of students at ages 9, 13, and 17 periodically (every two to four years) in the subject areas of reading (since 1971), science (since 1969), and mathematics (since 1973). The commissioner of education statistics of the U.S. Department of Education is responsible for carrying out the NAEP.
Penn World Tables. The Penn World Tables (Alan Heston, Robert Summers, and Bettina Aten, Penn World Table Version 6.1, Center for International Comparisons of Production, Income and Prices, University of Pennsylvania, October 2002) provide time series national accounts data for a set of 189 countries, in common prices, so that real comparisons can be made both between countries and over time.
PISA (Program in Student Assessment). The OECD (Organization for Economic Cooperation and Development) launched the Program in Student Assessment (www.oecd.org/pisa/) in 1997. The PISA assesses the capabilities of 15-year-olds enrolled in school in participating countries through testing of a representative random sample. The PISA has been conducted in rounds every three years since 2000 with a different focus area of assessment in each round. In 2006 the focus area was mathematics and in 2009 reading. There was an extra round of assessments called the 2009+ that included more traditionally developing areas, including two states of India (Himachal Pradesh and Tamil Nadu).
SACMEQ (Southern and Eastern Africa Consortium for Measuring Educational Quality). The SACMEQ is a consortium of education ministries from fifteen southern and eastern African countries (www.sacmeq.org/). It assesses a nationally and regionally representative random sample of students in grade 6 in the domains of reading and mathematics. There have been three rounds: 1995, 2000, and 2007.
TIMSS (Trends in International Mathematics and Science Study) is an internationally comparable assessment of grade 8 students’ mastery of mathematics and science (http://timssandpirls.bc.edu/). TIMSS is a product of the International Association for the Evaluation of Educational Achievement (IEA). It has been carried out in participating countries every four years since 1995, and the latest data used for this book were the results from the 2007 round (the 2011 round became available in December 2012). The participating countries include both developed and developing countries and regions.
UWEZO (meaning “capability” in Kiswahili). This is an adaptation of the ASER model for testing and assessing all children aged 6 to 16 in the basics of literacy and numeracy in three east African countries (Tanzania, Uganda, and Kenya). It was carried out by the organization TAWEZA for the first time in 2009 (published in 2010), and the assessment has been done annually.
Rankings of Universities
Chapter 6 compares the position in the global rankings of American and British universities versus those in Europe. Given that any ranking of the “quality” of universities is necessarily controversial, I used four different sources of rankings to ensure the robustness of the particular finding of the dominance of U.S./U.K. (and more broadly “Anglo”) universities. This use of many different rankings was not to endorse any particular ranking or even set of rankings but rather precisely so as to not endorse any particular method(s) as superior. The key fact—the dominance of U.S./U.K. universities—emerges in all of the rankings (and in rankings such as those from Leiden, which are based purely on publications in the sciences [www.leidenranking.com/ranking]).
Annual Ranking of World Universities. “The Academic Ranking of World Universities (ARWU)” was first published in June 2003 by the Center for World-Class Universities and the Institute of Higher Education of Shanghai Jiao Tong University, China, and then updated on an annual basis. ARWU uses six objective indicators to rank world universities, including the number of alumni and staff who win Nobel Prizes and Fields Medals, number of highly cited researchers selected by Thomson Scientific, number of articles published in the journals Nature and Science, number of articles indexed in Science Citation Index—Expanded and Social Sciences Citation Index, and per capita performance with respect to the size of an institution. More than 1,000 universities are actually ranked by ARWU every year and the best 500 are published on the web (www.arwu.org/aboutARWU.jsp).
THE-QS World University Rankings (www.topuniversities.com/university-rankings/world-university-rankings/2009). The Times Higher Education Supplement and QA in 2009 ranked universities based on a composite of surveys (40 percent to academic reputation and 10 percent to employer reputation), faculty citations (20 percent), faculty-student ratios (20 percent), and 5 percent each to percent of faculty and students who are international.
Global University Rankings-Reitor. This is a ranking by Russian researchers to rate universities in a way that they felt was fairer to Russian (and former Soviet) institutions of higher education. It was carried out in 2008 and published in February 2009 and is not periodic.
Webometrics. “The Ranking Web,” or Webometrics, is the largest academic ranking of higher education institutions. Since 2004 and every six months, an independent, objective, free, open scientific exercise is performed by the Cybermetrics Lab (Spanish National Research Council, CSIC) for providing reliable, multidimensional, updated, and useful information about the performance of universities from all over the world based on their web presence and impact (www.webometrics.info/en/Methodology).