Sign Test
Menu location: Analysis_Exact_Sign.
In a sample of n observations, if r out of n show a change in one particular direction then the sign test can be used to assess the significance of this change. The value of interest is the proportion r/n.
The binomial distribution is used to evaluate the probability that r/n exceeds an expected value of 0.5 (i.e. 50:50, the chance of heads when tossing a coin). If you want to use an expected value other than 0.5 then please see the single proportion test (binomial test).
Null hypothesis: observed proportion is not different from 0.5
StatsDirect gives you one and two sided cumulative probabilities from a binomial distribution (based on an expected proportion of 0.5) for the null hypothesis. A normal approximation (with continuity correction) is also given. You are also given an exact confidence interval for the proportion r/n (Conover, 1999; Altman, 1991; Vollset, 1993).
Example
From Altman (1991 p. 186).
Out of a group of 11 women investigated 9 were found to have a food energy intake below the daily average and 2 above. We want to quantify the impact of 9 out of 11, i.e. how much evidence have we got that these women are different from the norm?
To analyse these data in StatsDirect you must select the sign test from the exact tests section of the analysis menu. Then choose the default 95% two sided confidence interval.
For this example:
For 11 observations with 9 on one side:
Cumulative probability
Two sided P = 0.0654
One sided P = 0.0327
Normal approximate z = 1.809068
Two sided P = 0.0704
One sided P = 0.0352
Exact (Clopper-Pearson) 95% confidence interval for the proportion:
Lower Limit = 0.482244
Proportion = 0.818182
Upper Limit = 0.977169
If we were confident that this group could only realistically be expected to have a lower caloric intake and we would not be interested in higher caloric intakes then we could make inference from the one sided P value. We do not, however, have evidence for such an assumption so we can not reject the null hypothesis that the proportion is not significantly different from 0.5. We can say with 95% confidence that the true population value of the proportion lies somewhere between 0.48 and 0.98. The most sensible response to these results would be to go back and collect more data.
R code
This R code reproduces the example above. It needs no packages and was checked with R 4.6.1. Paste it into R, or save it as a script and run it.
# Sign test: the StatsDirect help example (Altman 1991, p. 186: 9 of 11 women with a
# food energy intake below the daily average) in R
n <- 11
r <- 9
# R's standard test. With p = 0.5 the binomial distribution is symmetric, so R's
# two sided P (0.06543) is twice the one sided tail (0.032715), the report's two P
# values. Its confidence interval is the exact (Clopper-Pearson) interval.
print(binom.test(r, n, p = 0.5, conf.level = 0.95))
six <- function(x) formatC(x, digits = 6, format = "f", drop0trailing = TRUE)
pv <- function(p) {
if (p < 0.0001) "P < 0.0001" else
paste("P =", formatC(p, digits = 4, format = "f", drop0trailing = TRUE))
}
# Exact cumulative probabilities: the one sided P is the probability of r or fewer
# on the less frequent side (2 or fewer of 11 here); the two sided P is twice that,
# capped at 1.
p1 <- pbinom(min(r, n - r), n, 0.5)
cat("Cumulative probability: two sided", pv(min(1, 2 * p1)), " one sided", pv(p1), "\n")
# Normal approximation with a continuity correction of a half, as the report prints
# it: |n/2 - r| - 1/2 over the standard deviation sqrt(n/4); z is 0 when r is within
# half an observation of n/2.
z <- max(0, abs(n / 2 - r) - 0.5) / sqrt(n / 4)
cat("Normal approximate z =", six(z), "\n")
cat("Two sided", pv(2 * pnorm(-z)), " One sided", pv(pnorm(-z)), "\n")
# Exact (Clopper-Pearson) confidence limits, as binom.test gives them, from the beta
# distribution: 0 or 1 when r is 0 or n
ci <- binom.test(r, n, conf.level = 0.95)$conf.int
cat("Lower Limit =", six(ci[1]), "\n")
cat("Proportion =", six(r / n), "\n")
cat("Upper Limit =", six(ci[2]), "\n")