When the test of mediation is more powerful than the test of the total effect

Abstract

Although previous research has studied power in mediation models, the extent to which the inclusion of a mediator will increase power has not been investigated. To address this deficit, in a first study we compared the analytical power values of the mediated effect and the total effect in a single-mediator model, to identify the situations in which the inclusion of one mediator increased statistical power. The results from this first study indicated that including a mediator increased statistical power in small samples with large coefficients and in large samples with small coefficients, and when coefficients were nonzero and equal across models. Next, we identified conditions under which power was greater for the test of the total mediated effect than for the test of the total effect in the parallel two-mediator model. These results indicated that including two mediators increased power in small samples with large coefficients and in large samples with small coefficients, the same pattern of results that had been found in the first study. Finally, we assessed the analytical power for a sequential (three-path) two-mediator model and compared the power to detect the three-path mediated effect to the power to detect both the test of the total effect and the test of the mediated effect for the single-mediator model. The results indicated that the three-path mediated effect had more power than the mediated effect from the single-mediator model and the test of the total effect. Practical implications of these results for researchers are then discussed.

Appendixes

Appendix 1

For the single-mediator model described in Eqs. 2 and 3, and assuming uncorrelated errors across equations, normally distributed variables, and a linear system of relations (McDonald, 1997), the true covariances among X, M, and Y for the single-mediator model can be used to obtain expected values of the regression coefficients and standard errors of those coefficients, to compute predicted power for the test of a, b, and c at any combination of effect size and sample size (MacKinnon, 2008). The expected ${\widehat{a}}_T$ coefficient is given in Eq. A.1. The subscript T is used to indicate that this value is the true or theoretical value, given the covariance matrix and sample size.

$$ {\widehat{a}}_T=\frac{ Cov\left(X,M\right)}{ Var(X)} $$

(A.1)

The true or theoretical (true is again represented by the T in the subscript) variance of the estimator $\widehat{a}$, ${\sigma}_{\widehat{a}T}^2$ (standard error is the square root of the variance), is equal to

$$ {\sigma}_{\widehat{a}T}^2=\frac{\sigma_{e3}^2}{\left(N-2\right) Var\left[\mathrm{X}\right]} $$

(A.2)

In Eq. A.2, σ ² _e3 is the true error variance from Eq. 3, which predicts M from X. An estimator of the true error variance represented in two ways is shown in Eqs. A.3 and A.4.

$$ {\widehat{\sigma}}_{e3T}^2=\frac{{\displaystyle \sum {e}^2}}{N-\mathrm{p}-1} $$

(A.3)

$$ {\widehat{\sigma}}_{e3T}^2={\widehat{\sigma}}_M^2-{a}^2{\widehat{\sigma}}_X^2 $$

(A.4)

In Eq. A.3, p is the number of X variables (which is equal to 1 here, for the one-predictor case), and ∑ e ² is the sum of the squared differences between the predicted and observed M scores. Equation A.4 involves the variances of X and M and the $\widehat{a}$ coefficient. Similar formulas are used to determine the expected values of $\widehat{b}$ and $\widehat{c}^{\prime }$, as is shown in Eqs. A.5 and A.6.

$$ {\widehat{b}}_T=\frac{ Var\left[\mathrm{X}\right] Cov\left[\mathrm{M},\mathrm{Y}\right]- Cov\left[\mathrm{X},\mathrm{M}\right] Cov\left[\mathrm{X},\mathrm{Y}\right]}{ Var\left[\mathrm{X}\right] Var\left[M\right]- Cov{\left[\mathrm{X},\mathrm{M}\right]}^2} $$

(A.5)

$$ {\widehat{c}}_T^{\prime }=\frac{ Var\left[\mathrm{M}\right] Cov\left[\mathrm{X},\mathrm{Y}\right]- Cov\left[\mathrm{X},\mathrm{M}\right] Cov\left[\mathrm{M},\mathrm{Y}\right]}{ Var\left[\mathrm{X}\right] Var\left[M\right]- Cov{\left[\mathrm{X},\mathrm{M}\right]}^2} $$

(A.6)

$$ {\sigma}_{\widehat{b}T}^2=\frac{\sigma_{e2}^2}{N-3}\frac{\frac{1}{ Var\left[\mathrm{M}\right]}}{1-{\mathrm{r}}_{\mathrm{XM}}^2} $$

(A.7)

$$ {\sigma}_{\widehat{c}\hbox{'}T}^2=\frac{\sigma_{e2}^2}{N-3}\frac{\frac{1}{ Var\left[\mathrm{X}\right]}}{{1\hbox{-} \mathrm{r}}_{\mathrm{XM}}^2} $$

(A.8)

$$ {\widehat{\sigma}}_{e2T}^2={\widehat{\sigma}}_Y^2-{b}^2{\widehat{\sigma}}_M^2-c{\prime}^2{\widehat{\sigma}}_X^2+ bc^{\prime }{\widehat{\sigma}}_{MX}^2 $$

(A.9)

Equations A.7 and A.8 show the true variances of ${\widehat{b}}_T$ and ${\widehat{c}}_T^{\prime }$ respectively, where N is the number of participants and ${\widehat{\sigma}}_{e2T}^2$ is the expected true error variance from Eq. 2. Equation A.9 shows an estimator of the true error variance for Eq. 2, predicting Y from X and M, and is analogous to the error variance formula in Eq. A.4, but with p = 2 predictors (X and M).

The covariance matrix in Table 5 produces analogous formulas to determine the expected values of ${\widehat{c}}_T$, ${\sigma}_{\widehat{c}T}^2$, and ${\widehat{\sigma}}_{e1T}^2$ from Eq. 1, shown in Eqs. A.10, A.11, and A.12, respectively.

$$ {\widehat{c}}_T=\frac{ Cov\left[\mathrm{X},\mathrm{Y}\right]}{ Var\left[\mathrm{X}\right]} $$

(A.10)

$$ {\sigma}_{\widehat{c}T}^2=\frac{\sigma_{e1}^2}{\left(N-2\right) Var\left[\mathrm{X}\right]} $$

(A.11)

$$ {\widehat{\sigma}}_{e1T}^2={\widehat{\sigma}}_Y^2-{c}^2{\widehat{\sigma}}_X^2 $$

(A.12)

Statistical power to detect the mediated effect

Statistical power is determined using these expected values, as is shown in Eqs. A.13, A.14, and A.15, where ϕ _t is the probability from the cumulative t distribution. The statistical power of the joint significance test for mediation consists of the power to detect a nonzero a path and the power to detect a nonzero b path. If both paths are statistically significant, the hypothesis of no mediated effect is rejected (MacKinnon et al., 2002).

$$ P\left({\widehat{a}}_T\right)=1-{\phi}_t\left({\widehat{a}}_T/{\sigma}_{\widehat{a}T}-{t}_{1-\alpha /2,N-2}\right) $$

(A.13)

$$ P\left({\widehat{b}}_T\right)=1-{\phi}_t\left({\widehat{b}}_T/{\sigma}_{\widehat{b}T}-{t}_{1-\alpha /2,N-3}\right) $$

(A.14)

$$ P\left({\widehat{c}}_T\right)=1-{\phi}_t\left({\widehat{c}}_T/{\sigma}_{\widehat{c}T}-{t}_{1-\alpha /2,N-2}\right) $$

(A.15)

So, the case in which the power to detect the mediated effect is greater than the power to detect the total effect is given by the following inequality:

$$ P\left({\widehat{a}}_T\right)P\left({\widehat{b}}_T\right)>P\left({\widehat{c}}_T\right) $$

(A.16)

All power values were computed analytically with an SAS program available from the authors. These analytical power values were checked empirically in a statistical simulation study for the sample sizes and parameter values described in this article. Note that when both a and b are zero, for the joint significance test of a and b, the Type I error rate is .0025.

Appendix 2: Effect size formulas for the parallel two-mediator model

$$ \begin{array}{l}{\rho}_{{\mathrm{M}}_1{\mathrm{M}}_2}=\frac{\sigma_{{\mathrm{M}}_1{\mathrm{M}}_2}}{\sqrt{\sigma_{{\mathrm{M}}_1}^2}\sqrt{\sigma_{{\mathrm{M}}_2}^2}}\hfill \\ {}{\rho}_{{\mathrm{M}}_1\mathrm{Y}}=\frac{\sigma_{{\mathrm{M}}_1\mathrm{Y}}}{\sqrt{\sigma_{{\mathrm{M}}_1}^2}\sqrt{\sigma_{\mathrm{Y}}^2}}\hfill \\ {}{\rho}_{{\mathrm{M}}_2\mathrm{Y}}=\frac{\sigma_{{\mathrm{M}}_2\mathrm{Y}}}{\sqrt{\sigma_{{\mathrm{M}}_2}^2}\sqrt{\sigma_{\mathrm{Y}}^2}}\hfill \\ {}{\rho}_{\mathrm{X}\mathrm{Y}}=\frac{\sigma_{\mathrm{X}\mathrm{Y}}}{\sqrt{\sigma_{\mathrm{X}}^2}\sqrt{\sigma_{\mathrm{Y}}^2}}\hfill \end{array} $$

For a ₁:

$$ {\rho}_{{\mathrm{X}\mathrm{M}}_1}=\frac{\sigma_{{\mathrm{X}\mathrm{M}}_1}}{\sqrt{\sigma_{\mathrm{X}}^2}\sqrt{\sigma_{{\mathrm{M}}_1}^2}} $$

For a ₂:

$$ {\rho}_{{\mathrm{X}\mathrm{M}}_2}=\frac{\sigma_{{\mathrm{X}\mathrm{M}}_2}}{\sqrt{\sigma_{\mathrm{X}}^2}\sqrt{\sigma_{{\mathrm{M}}_2}^2}} $$

For b ₁:

$$ \begin{array}{l}{\rho}_{{\mathrm{M}}_1\mathrm{Y}.{\mathrm{XM}}_2}=\frac{\rho_{{\mathrm{M}}_1\mathrm{Y}.\mathrm{X}}-{\rho}_{{\mathrm{M}}_1{\mathrm{M}}_2.\mathrm{X}}{\rho}_{{\mathrm{M}}_2\mathrm{Y}.\mathrm{X}}}{\sqrt{1-{\rho}_{{\mathrm{M}}_1{\mathrm{M}}_2.\mathrm{X}}^2}\sqrt{1-{\rho}_{{\mathrm{M}}_2\mathrm{Y}.\mathrm{X}}^2}},\kern1em \mathrm{where}\hfill \\ {}{\rho}_{{\mathrm{M}}_1\mathrm{Y}.\mathrm{X}}=\frac{\rho_{{\mathrm{M}}_1\mathrm{Y}}-{\rho}_{{\mathrm{XM}}_1}{\rho}_{\mathrm{XY}}}{\sqrt{1-{\rho}_{{\mathrm{XM}}_1}^2}\sqrt{1-{\rho}_{\mathrm{XY}}^2}},\hfill \\ {}{\rho}_{{\mathrm{M}}_1{\mathrm{M}}_2.\mathrm{X}}=\frac{\rho_{{\mathrm{M}}_1{\mathrm{M}}_2}-{\rho}_{{\mathrm{XM}}_1}{\rho}_{{\mathrm{XM}}_2}}{\sqrt{1-{\rho}_{{\mathrm{XM}}_1}^2}\sqrt{1-{\rho}_{{\mathrm{XM}}_2}^2}},\hfill \\ {}{\rho}_{{\mathrm{M}}_2\mathrm{Y}.\mathrm{X}}=\frac{\rho_{{\mathrm{M}}_2\mathrm{Y}}-{\rho}_{{\mathrm{XM}}_2}{\rho}_{\mathrm{XY}}}{\sqrt{1-{\rho}_{{\mathrm{XM}}_2}^2}\sqrt{1-{\rho}_{\mathrm{XY}}^2}}\hfill \end{array} $$

For b ₂:

$$ \begin{array}{l}{\rho}_{{\mathrm{M}}_2\mathrm{Y}.{\mathrm{XM}}_1}=\frac{\rho_{{\mathrm{M}}_2\mathrm{Y}.\mathrm{X}}-{\rho}_{{\mathrm{M}}_1{\mathrm{M}}_2.\mathrm{X}}{\rho}_{{\mathrm{M}}_1\mathrm{Y}.\mathrm{X}}}{\sqrt{1-{\rho}_{{\mathrm{M}}_1{\mathrm{M}}_2.\mathrm{X}}^2}\sqrt{1-{\rho}_{{\mathrm{M}}_1\mathrm{Y}.\mathrm{X}}^2}},\kern1em \mathrm{where}\hfill \\ {}{\rho}_{{\mathrm{M}}_2\mathrm{Y}.\mathrm{X}}=\frac{\rho_{{\mathrm{M}}_2\mathrm{Y}}-{\rho}_{{\mathrm{XM}}_2}{\rho}_{\mathrm{XY}}}{\sqrt{1-{\rho}_{{\mathrm{XM}}_2}^2}\sqrt{1-{\rho}_{\mathrm{XY}}^2}},\hfill \\ {}{\rho}_{{\mathrm{M}}_1{\mathrm{M}}_2.\mathrm{X}}=\frac{\rho_{{\mathrm{M}}_1{\mathrm{M}}_2}-{\rho}_{{\mathrm{XM}}_1}{\rho}_{{\mathrm{XM}}_2}}{\sqrt{1-{\rho}_{{\mathrm{XM}}_1}^2}\sqrt{1-{\rho}_{{\mathrm{XM}}_2}^2}},\hfill \\ {}{\rho}_{{\mathrm{M}}_1\mathrm{Y}.\mathrm{X}}=\frac{\rho_{{\mathrm{M}}_1\mathrm{Y}}-{\rho}_{{\mathrm{XM}}_1}{\rho}_{\mathrm{XY}}}{\sqrt{1-{\rho}_{{\mathrm{XM}}_1}^2}\sqrt{1-{\rho}_{\mathrm{XY}}^2}}\hfill \end{array} $$

For c′:

$$ \begin{array}{l}{\rho}_{\mathrm{XY}.{\mathrm{M}}_1{\mathrm{M}}_2}=\frac{\rho_{\mathrm{XY}.{\mathrm{M}}_1}-{\rho}_{{\mathrm{XM}}_2.{\mathrm{M}}_1}{\rho}_{{\mathrm{YM}}_2.{\mathrm{M}}_1}}{\sqrt{1-{\rho}_{{\mathrm{XM}}_2.{\mathrm{M}}_1}^2}\sqrt{1-{\rho}_{{\mathrm{YM}}_2.{\mathrm{M}}_1}^2}},\mathrm{where}\hfill \\ {}{\rho}_{\mathrm{XY}.{\mathrm{M}}_1}=\frac{\rho_{\mathrm{XY}}-{\rho}_{{\mathrm{XM}}_1}{\rho}_{{\mathrm{M}}_1\mathrm{Y}}}{\sqrt{1-{\rho}_{{\mathrm{XM}}_1}^2}\sqrt{1-{\rho}_{{\mathrm{M}}_1\mathrm{Y}}^2}},\hfill \\ {}{\rho}_{{\mathrm{XM}}_2.{\mathrm{M}}_1}=\frac{\rho_{{\mathrm{XM}}_2}-{\rho}_{{\mathrm{XM}}_1}{\rho}_{{\mathrm{M}}_1{\mathrm{M}}_2}}{\sqrt{1-{\rho}_{{\mathrm{XM}}_1}^2}\sqrt{1-{\rho}_{{\mathrm{M}}_1{\mathrm{M}}_2}^2}},\hfill \\ {}{\rho}_{{\mathrm{YM}}_2.{\mathrm{M}}_1}=\frac{\rho_{{\mathrm{M}}_2\mathrm{Y}}-{\rho}_{{\mathrm{M}}_1\mathrm{Y}}{\rho}_{{\mathrm{M}}_1{\mathrm{M}}_2}}{\sqrt{1-{\rho}_{{\mathrm{M}}_1\mathrm{Y}}^2}\sqrt{1-{\rho}_{{\mathrm{M}}_1{\mathrm{M}}_2}^2}}\hfill \end{array} $$