Meta-analysis: A research tool

In medical sciences (particularly, epidemiological and evidence-based medicine), meta-analysis is an accepted method. It is said that famous statisticians like Karl Pearson (to overcome power of reduced statistical power in small sample studies, 1904), RA Fisher (idea of cumulating probability values because the insingnificant results of a number of indipendent studies may give a lower picture from probabilty then one would obtain by chance, 1944), WG Cochran (in his discussion on averaging of means across independent studies laid down the statistical foundation through inverse variance weighting and homogenity testing) among others contributed towards this (see, Meta-analysis in Wikipedia and Practical Meta-Analysis by David B Wilson).

It was Gene V. Glass who first made use of the term in its statistical sense in a seminal paer Primary, secondary, and meta-analysis of research, Educational Researcher, 5 (10), 3-8, 1976, JSTOR link. To quote from this paper:

"Primary analysis is the original analysis of data in a research study. It is what one typically imagines as the application of statistical methods."

"Secondary analysis is the re-analysis of data for the purpose of answering the original research question with better statistical techniques, or answering new questions with old data."

"Meta-analysis refers to ... the statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings. It connotes a rigorous alternative to the casual, narrative discussions of research studies which typify our attempts to make sense of the rapidly expanding research literature."

I have been aware of this technique because of my research interests in the public health domain. But, what caught my attention very recently the paper by Derek D. Headey and Andrew Hodge, 'The Effect of Population Growth on Economic Growth: A Meta-Regression Analysis of the Macroeconomic Literature' Population and Development Review, 35 (2), 221-248, 2009. The study explores how differences in the method could account for differences in results.

The meta-regression technique is as follows. In the first step, one has to rely on the independent original studies on a particular theme. In these studies the regressions will have a dependent variable (like economic growth, Yi) and an independent variable (like population growth, Xi) and there may be a set of control variables (Z), which are likely to be different for different sutdies.

(1) Yi=a0+a1Xi+ajZj+ei (j>1)

To know the impact of population growth on economic growth, the relevant coefficient is a1. Its statistical significance depending on its standard error, s1; that is, if the t-stat=(a1/s1)>|2|.

In the second step, the main task for the meta-regression, one the t-stat of the k studies (t1k) on the (el, l) number of methodological dummies.

(2) tk=b0+blDl+uk

The methodological dummies could refer to alternative measures of economic growth (Gross Domestic Product (GDP) per capita, Gross National Product (GNP) per capita, GDP per worker), alternative measures of population growth (total population, young population, working population) or because of alternative techniques used (control set includes health indicator, sample is developing land-poor countries, sample is developing countries, sample is for a prticular time period, the econometric method uses weighted least squares and so on and so forth). Of course, the more number of dumies one can use would depend upon the more number of studies that already exist.

Some of the problems with meta-analysis identified in literature are the following.
The studies that one is likely to have access (read published) are the ones that would have reported significant results. In other words, unpublished studies that showed negative or null findings are difficult to find.

Another problem that comes from critiques of the approach is th 'Apples versus Oranges'. Is it proper to compare the different studies together. But, the independent studies are also comparing something. It is true that one should be careful and know what one is comparing than doing away with comparision altogether.

There is also the 'Flat Earth' criticism. That an analysis of averages may not do justice. Yes, and one should use meta-analysis judiciously to explore that also (for instance, in the use of methodological dummies). There are many others that one would come across, as one reads on more.

A good read is Meta-analysis at 25 by none other than Gene V Glass. Researchers in other disciplines should explore this method. Go head on!

Budget 2009: After Masima, Jethu's Bari

In the railway budget of 2009, Masima had her say. In Budget 2009, Jethu (Shri Pranab Mukherjee) had his say (If I was twenty years younger I would have addressed him as Dadu) while sipping water in between. The first message that has to be read between the lines is that the Indian National Congress (INC) as also Trinamool Congress are aiming at the state elections of West Bengal in 2011. There is also a medium term target of Uttar Pradesh in 2012. The Left Front, Bahujan Samaj and Samajbadi better watch out.

The budget is also a way of saying thank you to all those who voted the United Progressive Alliannce back to power. The increased expenditure under the National Rural Employment Guarantee Scheme (NREGS) and National Rural Health Mission (NRHM) is on expected lines. The National Rural Livelihood Mission (NRLM) that would restructure the existing Swarnajayati Gram Swarojgar Yojana (SGSY) and integrating with Self-help groups and other initiative are welfare programmes that show promise.

What worries most is the increasing expenditure without commensurat receipts leading to an increase in fiscal deficit. Jethu is not too much bothered about this becaue of the global financial crisis. He will wait for the fruit to ripe.

Politically, the immediate concern would be Maharashtra elections later in the year. With delayed monsoons, will the Vidarbha task force, debt waiver and urban storm water drains help. Only time will tell.

Mamtadi (Masima) Special, Railway Budget 2009

On 3rd July 2009, Kumari Mamata Banerjee (popularly known as Mamatadi (Didi), left to myself I would prefer to call her Masima-Ma jaisi or like mother) presented the first railway budget of the recently constituted government that gave United Progressive Alliance (UPA) its second term with Shri Manmohan Singh as Prime Minister. In its first term Shri Lalu Prasad Yadav was the railway minister and he is credited in turning around the Indian Railways from a loss making entity to a profit making one which continued even during the global downturn; currently, Shri Yadav is not a part of the UPA government. Given the backdrop, there are a lot of eyes on Masima. She is not new to this job. This is her third railway budget, the earlier two being presented when she was a minister in the National Democratic Alliance (NDA) government with Shri Atal Bihari Vajpayee as Prime Minister. This is the first time that a female railway minster presented a budget before a female Lok Sabha speaker and that too when the country has its first female President.

Keeping in line with the ‘inclusive growth’ agenda of the government as envisaged under the 11th five year plan, Masima wanted to identify with the common man. She came to the parliament in her own vehicle (not the official one allotted to her) braving the traffic snarls and carried the budget paper in a jhola/cloth bag (not a leather briefcase). More importantly, she wanted to emphasize on social viability over economic unviability. Thus, the inclusiveness agenda has to focus on backward areas and underprivileged people. The budget document is a good read. For some of the important highlights, see the Press Information Bureau, PIB, version or the Ministry of Railways version. My takes on some fo these are as follows.

Special trains for perishable farm produce. This should be linked with better storage as also linking transport networks from the farm gate to railway centres. It has the potential of giving farmers a better price.

Provide facilities for transportation of rural craft. The implicit thought that there is more in rural areas than just farm produce. In fact, with low returns from farm produce, it is the rural non-farm avenues that need to be identified and propagated by the larger economy (wait for the main budget).

Izzat scheme: Monthly ticket of Rs.25/- for unorganized sector/poor with income of less than Rs.1500. This will help the vendors travelling by train.

Railway tickets are to be made available through post offices and mobile vans. People without access to internet or credit card are likely to benefit from this. This calls for some integration of the optic fibre cable network of railways. The railway budget refers to setting up an expert committee headed by Shri Sam Pitroda to look into these.

Concession for accredited press persons increased to 50 per cent. They can use this to travel to hinterland areas to capture more and more people-centric stories.

There was increasing talk of the Tatkal Schme charging more money. The charges have been reduced (from Rs.150/- to Rs.100/- per ticket) as also the number of days (from five to two) before which one could avail this facility.

Some other initiatives are ladies special trains in metros (Chennai, Delhi and Kolkata; it exists in Mumbai) during office hours, Yuva trains for youth from hinterland to metros. Duronto or non-stop point-to-point long distance trains. More importantly, passenger fares and freight tariffs have not been increased.

Procedural Norms for Financial Prudence

There are certain procedural norms laid down for financial prudence. In India, a common practice is to seek quotations from at least three different firms. The purpose is to ensure that while buying with public money one ought to survey the market and go for the lowest. One has to be careful with public money.

In practice, three quotations are a formality. Some collect it from three firms whereas some others ask one firm to provide quotation from three different firms. There are many instances when the prices quoted in all the three would be much more than the normal price prevailing in the market (not necessarily the maximum retail price).

If you bargained hard and purchased at a price which is much lower than the prevailing market price but got only one quotation then you are in for trouble. On the contrary, if you purchased at a price which is much higher than the prevailing market price but this is the lowest from three quotations then you have followed the procedural norms for financial prudence.

In one occasion while coordinating a field survey for which there was provision to hire locally available vehicles. My employer suggested that I survey the market and bargain for a reasonable price but should not forget to get three quotations. The latter is required because the auditor who will come sometime in the future (it could be in the next one to five years or even more) will have no idea of the prevailing market or even if aware there are no documents to support that contention and if the person goes by personal knowledge of the market then it will be like being party to not following procedural norms.

The other problem is that if there is a prevailing market rate then how can one get three quotations in which one is the lowest. It is only possible if one asks two providers to give a slightly higher quotation.

A major problem is to ensure quality. If one is going for the lowest price then the quality of the final product will also be poor. One way out for this is to specify the quotation/bid to two parts. One technical and another financial. Ensure quality at the technical bid stage. One should have a minimum of two who qualify for the financial bid and then choose the one with the lowest price. This sounds good, but there is still a possibility that the one who looses out on the financial bid will have much lower quality.

The suggestion for this is to be very strict at the technical bid. If you fail to get two then call the bid again and if you still did not get two then call for a third time and this time you can cross the technical bid stage even if only one qualifies. But, then precious time is lost in the process. It leaves one wondering, whether these procedural norms are for financial prudence or otherwise...

Masters in Development Studies

The Central University of Bihar (CUB), which has been recently set up along with eleven other central universities by an act of parliament, The Central Universities Bill, 2009. Professor Janak Pandey, an eminent Psychologist, has been appointed Vice Chancellor. He has take the responsibility head on and set in motion to start a two-year course on Masters in Development Studies from 2009-10. Such programmes are available in the European Universities (ISS, LSE, Machester and Sussex among others). The programme offered by CUB shows promise and young minds interested in development studies should apply for the same. For details, see their brochure. A summarized form is as folows.

Applications are invited to Masters in Development Studies by CUB. The minimum eligibility shall be of 55% (50% for SC/ST candidates) of marks in the bachelor degree examination. There will be a written test on August 1, 2009 in five centres spread across the country (Kolkata, Mumbai, Mysore, New Delhi and Patna). Selected candidates wil be called for Group Discussion and Interview to take place during August 24-25, 2009. Application fee for all candidates is Rs.500/- (only through crossed Demand Drafts to be drawn in favour of 'Central University of Bihar' payable at Patna). Submission Of completed applications begins from July 06, 2009 and should reach (Central University of Bihar Counter (CMIP campus), Phaneeswarnath Renu Hindi Bhawan, Chajjubagh, Patna -800001) before July 24, 2009.

All the best to all those who take this plunge.