Comparing Regression, PLS, and LISREL Using a Monte Carlo Simulation

This study uses Monte Carlo simulation to compare three relatively popular techniques, with findings that run counter to extant suggestions in MIS literature. We compare multiple regression, Partial Least Squares and LISREL under varying sample sizes (N = 40, 90, 150, and 200) and varying effect sizes (large, medium, small and no effect). The focus of the analysis was on determining how frequently Type I errors (non-existent effect erroneously detected) and Type II errors (true effect not detected) occur for each combination of technique, sample size, and effect size. In addition, we tested the efficacy of bootstrapping versus jackknifing for the test of statistical significance of parameter estimates using PLS. Initial results, using very simple models and nicely behaved and normally distributed data, suggest that: (1) PLS with jackknifing (to test statistical significance) has unacceptably low power (too many Type II errors) except with a strong effect size; and (2) at small sample size and medium or small effect size, all techniques (including PLS, which is reputed to be strong in this realm) have unacceptably low power.