AN EMPIRICAL ANALYSIS FOR STUDY OF INFLUENCE FACTORS TO EFFECTS ON P-VALUES AND TESTS CRITERIA
DOI:
https://doi.org/10.62533/ekhq6b12Abstract
This paper aims to identify the key statistical factors that influence the threshold and interpretation of p-value significance levels. Our empirical analysis provides evidence to better understand the range of high significance, moderate significance, and non-significance levels. We thoroughly discuss several contributing factors, including sample size (large vs. small datasets), standard error, the quality and utility of statistical tests, statistical power, effect size, sampling procedures, data collection quality, and the use of both qualitative and quantitative research data. Consideration is also given to challenges presented by big data. Additionally, we review contributions from prominent researchers regarding factors that influence p-values. To support our analysis, we generate sets of random sample data (n = 10, 100, 500, 1000, 5000, and 10000) from well-known continuous and discrete distributions, along with observed secondary data, to examine the behavior of p-values under different conditions. This study also investigates the question: “Is p-value a reliable measure of unknown population characteristics?” We conclude with findings, recommendations, and possible remedies based on our empirical results. All analyses were conducted using the Statistical Package for Social Sciences (SPSS, version 24).
Downloads
References
Badenes, R. L., Frias, N. D., Lotti, B., Bonilla, C. A., & Longobardi, C. (2016).Misconceptions of the P-value among Chilean and Italian academic psychologists. Frontiers in Psychology, 7, 1247.
Barnett, M. L., & Mathisen, A. (1997). Tyranny of the p-value: The conflict between statistical significance and common sense (Editorial). Journal of Dental Research, 76(2), 152–154.
Clarue, G. M., & Kempson, R. E. (1997). Introduction to the design and analysis of experiments. Arnold.
Concato, J., & Hartigan, J. A. (2016). P-values: From suggestion to superstition. Journal of Investigative Medicine, 64(11), 1166–1170.
Dahiru, T. (2008). P-value, a true test of statistical significance? A cautionary note. Annals of Ibadan Postgraduate Medicine, 6(1), 21–26.
Demidenko, E. (2016). The p-value you can't buy. The American Statistician, 7(1), 33–38.
Feinstein, A. R. (1998). P-values and confidence intervals: Two sides of the same unsatisfactory coin. Journal of Clinical Epidemiology, 51(3), 355–360.
Gelman, A., & Carlin, J. (2017). Some natural solutions to the *p*-value communication problem—and why they won’t work. Journal of the American Statistical Association, 112(519), 899–901.
Ashfaq, A., Imran, H., Nehal, N., Shamim, M., Asif, A., Ikhlaque, M., & Katper, N. (2023). Interest of Women in the Tourism Sector and How to Empower Them. In Women’s Empowerment Within the Tourism Industry (pp. 427-441). IGI Global Scientific Publishing.
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P-values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337–350.
Hubbard, R., & Lindsay, R. M. (2008). Why P-values are not a useful measure of evidence in statistical significance testing. Theory & Psychology, 18(1), 69–88.
Jamali, K. A., Katper, N., Khaskhelly, F. Z., & Khaskhelly, Y. (2024). University Incubators: Cultivating Innovation for a Flourishing Entrepreneurial Society. International Research Journal of Management and Social Sciences, 5(3), 151-165.
Katper, N., S. M. Salman, K. Mohy-ud-Din, and M. Hassan. 2025. “ Mapping Interference in Green Finance, Environmental, and Social Innovation: A New Insight From Quantile-on-Quantile Approach.” Environmental Quality Management 35, no. 1: e70163. https://doi.org/10.1002/tqem.70163
Kim, J., & Bang, H. (2016). Three common misuses of P-values. Dental Hypotheses, 7(3), 73–80.
Kwak, S. (2023). Are only *p*-values less than 0.05 significant? A *p*-value greater than 0.05 is also significant! Journal of Lipid and Atherosclerosis, 12(2), 89–95.
Leo, D. G., & Sardanelli, F. (2020). Statistical significance: *p* value, 0.05 threshold, and applications to radiomics—reasons for a conservative approach. European Radiology Experimental, 4(1), 2–8.
Lin, M., Lucas, H. C., & Shmueli, G. (2013). Too big to fail: Large samples and the *p*-value problem. Information Systems Research, 24(4), 906– 917.
Marasini, D., Quatto, P., & Ripamonti, E. (2016). The use of *p*-values in applied research: Interpretation and new trends. Statistica, 76(4), 315–325.
Nahm, F. S. (2017). What the P-values really tell us. Korean Journal of Pain, 30(4), 241–242.
Ramp, W. K., & Yancey, J. M. (1991). P-values and their problems. Bone and Mineral, 13, 163–165.
Wasserstein, R. L., & Lazar, N. A. (2016). The ASA's statement on *p*-values: Context, process, and purpose. The American Statistician, 70(2), 129-133.
Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019). Moving to a world beyond "p < 0.05". The American Statistician, 73(Suppl. 1), 1–19.
