Interaction analyses — How large a sample do I need? (part 3)

pwr.r.test(r=0.25,sig.level = 0.05,power = 0.8)
approximate correlation power calculation (arctangh transformation)
n = 122.4466
r = 0.25
sig.level = 0.05
power = 0.8
alternative = two.sided
(log(2)+1)*.25
[1] 0.4232868
pwr.r.test(r=0.125,sig.level = 0.05,power = 0.8)
approximate correlation power calculation (arctangh transformation)
n = 499.1926
r = 0.125
sig.level = 0.05
power = 0.8
alternative = two.sided

What about all those published interactions with small samples?

This isn’t to say that published interactions with small samples are false positives. But it is important to keep in mind that small samples will tend to over estimate an effect, proportional to how under-powered the sample is to detect the true effect. This is also why it isn’t a good idea to plan the sample size for a study based on the effect size in a pilot-study, because the pilot-study is nearly guaranteed to overestimate the effect of interest.

So how large a sample do I need?

To return to our original hypothetical, if there is a main effect of r=0.25, and I want to test an interaction, the sample should be large enough to detect plausible and meaningful effects. It’s up to you to decide what that means, but keep in mind that if the sample is only barely large enough to detect the main effect, then the only interactions that can reliably observed are quite large, and may not be plausible, depending on what the research question is.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store