19. Power

Question 1

For this question we have φ = 2/3 = .667 with α = .05. From Figure 19.3 we see the required sample size is around n=20.

We can refine this estimate. For n=20 we need T₁₉ ≥ 1.729 for significance at the 5% level. We expect a t value of φ√n. To get a power of 90% we want

P(T₁₉ ≥ 1.729 - φ√n) = .90.

The value of t₁₉ with 90% to the right is -1.328. Thus we want 1.729 - φ√n = -1.328, so n = ((1.729 + 1.328)/φ)². With φ=.667 this gives n=21.

We could repeat this calculation, starting with n=21 instead (and so using the T₂₀ values). This gives an estimate of n=20.9, so we still need 21 subjects to get a power of 90%.

(Note that this is based on an estimate for σ. Since there is probably some uncertainly in this you should only think of 21 as an estimate for the desired power. Note also that this iteration for finding n is only necessary for small sample sizes. For larger samples the critical t values don't change much and are close to the Normal values.)

 

Question 2

Assuming X has a Binomial(n, .5) distribution, we want to find the smallest value of n such that there is an x that gives

P(X ≥ x) < .025

(since this is a two-sided test). For example, suppose we toss a coin 5 times. From Table 11.4, the probability of getting all heads is .031 and so the strongest two-sided P-value we could obtain is .062. Thus with 5 tosses we can never get evidence of bias at the 5% level.

However for n=6 we find P(X ≥ 6) = .016, giving a two-sided P-value from observing all heads or all tails or .032. So we can get significant evidence from 6 coin tosses.

To determine the power for a particular probability of heads, p, we just need to know how likely it is to get all heads or tails (since that is the only way we can get evidence with 6 coin tosses). The probability of 6 heads is p⁶ while the probability of 6 tails is (1-p)⁶, giving the total probability

P(evidence) = (1-p)⁶ + p⁶.

For values of p well above 0.5 the probability of all tails is neglibible so we can approximate this by p⁶. We want this value, the probability of finding evidence, to be .8. Taking logarithms gives

6 log(p) = log(.8) ⇒ p = exp(log(.8)/6) = .9635

so in fact the only bias we could detect with 80% power is a probability of heads of at least 96%! This is not surprising - allowing only 6 coin tosses means we can only really detect if the coin has heads (or tails) on both sides, for example.