T-Test Calculator

What is the t-test formula?

A t-test divides a difference between means by its standard error. For one sample, t = (x̄ − μ₀)/(s/√n), with n − 1 degrees of freedom. For two independent samples, Welch’s test uses t = (x̄₁ − x̄₂)/√(s₁²/n₁ + s₂²/n₂). The p-value is the probability of a t this far from zero if the null hypothesis is true.

Which t-test to use

• One sample compares a sample’s mean with a value fixed before the data were collected, such as a stated fill volume or a theoretical value. It has n − 1 degrees of freedom.
• Paired is for two measurements on each subject: before and after a treatment, two drugs given to the same patients, or the left and right sides of the same body. It subtracts each pair and runs a one-sample test on the differences against zero, with one fewer degree of freedom than there are pairs.
• Two samples is for two separate groups of individuals. Welch’s test, the default here, lets each group keep its own variance and takes its degrees of freedom from the Welch-Satterthwaite equation, so they are usually not a whole number. The pooled test, Student’s original, assumes the two populations share one variance and has n₁ + n₂ − 2 degrees of freedom.

Every version needs the same three figures from each sample, its mean, its standard deviation and its size, which is why the calculator works from summary statistics as well as from raw data. For the spread of a single column, with the standard error and a confidence interval for its mean, use the standard deviation calculator.

Worked example: two groups of plants

The calculator opens on the dried weights of ten plants grown as a control and ten grown under a treatment (Dobson 1983, An Introduction to Statistical Modelling, distributed with R as PlantGrowth): 4.17, 5.58, 5.18, 6.11, 4.50, 4.61, 5.17, 4.53, 5.33, 5.14 for the control, and 6.31, 5.12, 5.54, 5.50, 5.37, 5.29, 4.92, 6.15, 5.80, 5.26 under the treatment.

• Control: x̄₁ = 5.032, s₁ = 0.5831. Treatment: x̄₂ = 5.526, s₂ = 0.4426.
• Standard error of the difference: √(s₁²/10 + s₂²/10) = 0.2315.
• t = (5.032 − 5.526) / 0.2315 = −2.134, on 16.786 degrees of freedom from the Welch-Satterthwaite equation.
• The two-tailed critical value at α = 0.05 is 2.112, and the p-value is 0.0479.

So the difference is significant at 0.05, though only just, and a report would read t(16.79) = −2.13, p = .048. The 95% confidence interval for μ₁ − μ₂ runs from −0.9829 to −0.005128. It excludes zero, as it must when the two-tailed test is significant, but only barely, so the data are also consistent with a very small difference.

The pooled test gives the same t here, because the groups are the same size, but its 18 degrees of freedom give a p-value of 0.0469. With equal group sizes the two tests rarely disagree by much. With unequal sizes and unequal spreads they can, and it is the pooled test whose false-positive rate then drifts away from α, which is why Welch’s test is the default (Ruxton 2006, Behavioral Ecology 17: 688 to 690).

Worked example: why pairing matters

Student’s 1908 paper analyzed the extra hours of sleep that ten patients gained, against a control night, on each of two drugs (Cushny and Peebles 1905). Every patient took both drugs, so the data are paired by patient. Choose Paired in the calculator and it opens on them:

• differences, d = measurement 1 − measurement 2: −1.2, −2.4, −1.3, −1.3, 0, −1, −1.8, −0.8, −4.6, −1.4
• n = 10, d̄ = −1.58, s_d = 1.23
• SE = s_d / √n = 1.23 / √10 = 0.38896
• t = (d̄ − μ₀) / SE = (−1.58 − 0) / 0.38896 = −4.0621
• df = n − 1 = 9
• p, two-tailed = P(|T| ≥ 4.0621) on 9 degrees of freedom = 0.00283
• critical t at α = 0.05, two-tailed = 2.262
• 95% confidence interval = −1.58 ± 2.2622 × 0.38896 = −2.4599 to −0.70011
• |t| = 4.062 ≥ 2.262, so p ≤ 0.05: significant at α = 0.05

Enter the same twenty numbers as two independent groups and Welch’s test gives t = −1.861 and p = 0.0794, not significant. Patients who slept well on one drug tended to sleep well on the other, so much of the spread within each column belongs to the patients rather than to the drugs. The paired test works on the differences, which removes it: the standard error falls from 0.8491 hours to 0.389. The pairing has to come from how the data were collected, though. Matching values that were never measured on the same subject is as wrong as ignoring a real pairing.

One-tailed or two-tailed?

A two-tailed test asks whether the means differ in either direction. A one-tailed test asks about one direction only, and it is valid only when that direction was stated before the data were seen and a difference the other way would count the same as none. On the plant weights, a prediction that the treated plants are heavier, μ₁ < μ₂, gives a one-tailed p-value of 0.0239, half the two-tailed 0.0479, against a one-tailed critical value of −1.741. Choosing the tail after seeing which way the data went doubles the false-positive rate, because either tail could have been the one chosen.

What the test assumes

• Independent observations. Each value, or each pair, comes from a different subject, and none influences another. Repeated measurements on one subject entered as separate values break this, and no choice of t-test repairs it.
• Normal populations, or samples large enough not to need them. The t distribution is exact for normally distributed data, and by the central limit theorem it becomes a good approximation for other shapes as the samples grow. With small samples, strong skew or outliers it can mislead, and a rank-based test such as the Mann-Whitney U test or the Wilcoxon signed-rank test is the usual alternative.
• Equal variances, for the pooled test only. Welch’s test drops this assumption, and loses very little when the variances do turn out to be equal.

Critical values of the t distribution

A two-tailed test at level α is significant when |t| is at or above the value in the column for that two-tailed α. A one-tailed test is significant when t is at or beyond the value in the column for its one-tailed α, in the predicted direction. The table runs to 120 degrees of freedom; the calculator gives the value for any number, including Welch’s fractional ones.

Every figure is computed by the function the calculator uses, which the tests check against the NIST/SEMATECH e-Handbook of Statistical Methods and against Fisher and Yates’ tables.

Common mistakes

• Entering the standard error as the standard deviation. A paper that reports mean ± SEM has already divided the standard deviation by √n, so multiply back: SD = SEM × √n. Entered as it stands, the SEM makes t √n times too large.
• A two-sample test on paired data. On the sleep data it turns a p-value of 0.00283 into 0.0794.
• Reading “not significant” as “no difference”. A p-value above α says the data are consistent with no difference, not that the means are equal. A small sample can miss a real difference, and the confidence interval shows how large a difference the data still allow.
• Running a t-test on every pair of three or more groups. Each comparison carries its own chance of a false positive, and the chances add up across the comparisons. A one-way analysis of variance tests all the groups at once.
• Testing counts in categories. A t-test compares the means of measurements. Counts, such as offspring of each phenotype, need the chi-square calculator.

What this calculator leaves out

It runs the one-sample, paired and two-sample t-tests and nothing else: not the analysis of variance for three or more groups, the rank-based tests for data far from normal, equivalence tests, or power and sample-size calculations. To ask whether two measured variables rise and fall together, rather than whether two means differ, the Spearman’s rank correlation calculator tests an association between ranks, and the linear regression calculator fits a line.