AN EMPIRICAL ANALYSIS FOR STUDY OF INFLUENCE FACTORS TO EFFECTS ON P-VALUES AND TESTS CRITERIA

Abstract

This paper aims to identify the key statistical factors that influence the threshold and interpretation of p-value significance levels. Our empirical analysis provides evidence to better understand the range of high significance, moderate significance, and non-significance levels. We thoroughly discuss several contributing factors, including sample size (large vs. small datasets), standard error, the quality and utility of statistical tests, statistical power, effect size, sampling procedures, data collection quality, and the use of both qualitative and quantitative research data. Consideration is also given to challenges presented by big data. Additionally, we review contributions from prominent researchers regarding factors that influence p-values. To support our analysis, we generate sets of random sample data (n = 10, 100, 500, 1000, 5000, and 10000) from well-known continuous and discrete distributions, along with observed secondary data, to examine the behavior of p-values under different conditions. This study also investigates the question: “Is p-value a reliable measure of unknown population characteristics?” We conclude with findings, recommendations, and possible remedies based on our empirical results. All analyses were conducted using the Statistical Package for Social Sciences (SPSS, version 24).