The Major Characteristics of Meta-Analysis Application in Economy and Management Research in China and Abroad:International Comparison,Current Situations,and Pitfalls

Meta analysis is a research process that applies statistical method to analyze and summarize relevant empirical research findings from previous research toobtain the average effects in a quantitative form.In a typical Meta analysis,researchers usually take the research findings from many independent but highly correlatedstudies as the research object,and then conduct a comprehensive analysis of these research findings in a systematic,quantitative and objective way based onscientific and strict research design.It has the advantages of increasing the credibility and generalizability of relevant research findings by combining various population and samples and expanding the sample size to solve inconsistent results from prior studies.As a result,Meta-analysis,as a new method of literature review,has been widely applied in academic research both in China and Abroad.However,only recently,it attracts more attention from Chinese researchers in the fields of economy and management.This paper focuses on the application of Meta-analysis method and identifies the problems and pitfalls in the application in the fields of economy and management research in China and Abroad.In doing so,this study makes a systematic study of relevant empirical papers in the field of economy and management that appliedMeta analysis method,which were published between 1990 and 2010.Firstly,we search all the papers that were published by CNKI(Chinese) and the Elsevier(English) databases during such a period mentioned above with the keyword of Meta analysis.And then proper criteria were applied to make sure whether a paper actually applied Meta analysis method for quantitative study or not,and all papers that were found in these two databases were carefully screened.This yielded 22 papers in Chinese and 95 papers in English.For each paper,totally six methodological characteristics in the application of Meta analysis were coded systematically.Then data were analyzed based on the six major aspects as follows: sample size,effect size,heterogeneity test,statistical model,statistical software,and publication bias analysis.Our intensive study reveals that Meta analysis method has been widely applied in international studies in the fields of economy and management.However,we also find that only a few publications in the same fields in China adopt this method and most of them were published only recently.In general,we find that some of domestic and international researchers in China showed their ignorance of how to apply this method in a more scientific way.Furthermore,based on a systematic analysis and comparison of the few publications in China with international research,it is concluded that the application of meta analysis method in China is relatively much less than what it should be,and most of them failed to abide by the scientific rules strictly,which results in serious problems in normative procedures and many doubts about their conclusions.And even for some international studies that applied Meta analysis has the potential to misuse this method.For example,1) 18% of Chinese papers failed to meet the minimum criterionof sample size,i.e.,above 30,while 26% of international papers had the same problems;2) 45% of Chinese papers and international studies did not include theheterogeneity test,which is necessary for Meta analysis technique;3) 50% of Chinese papers and 78% international papers failed to report which kind of software for Meta analysis was used in their respective studies,which makes it almost impossible to replicate what they conclude and see the generalization of theirresearch findings;4) 68.2% of Chinese papers failed to conduct publication bias analysis while 35.5% of international papers made the same mistake,which is also a necessary procedure for Meta analysis.In sum,this paper provides a quantitative,state-of-the-art comparison of the application of Meta analysis method in domestic and international publications in the field of economy and management,and identifies potential problems for Meta analysis application in the fields of economy and management in China,and offers useful suggestions on how to conduct a study more scientifically by using Meta analysis method.