Student's t-distribution (Fisher's distribution)

Navigation:  Probability distributions > Continuous univariate distributions >

Student's t-distribution (Fisher's distribution)

Previous pageReturn to chapter overviewNext page

"Students" t-distribution is a family of curves depending on a single parameter, ν (the degrees of freedom). The distribution converges to the standard Normal distribution, N(0,1), as the parameter ν→∞ (see graphs below). The t-distribution is used in place of the standard Normal for small samples, typically where n<50, when the population variance, σ2, is unknown. It is of particular use when seeking confidence intervals for a single mean value, or when comparing two mean values, and in some areas of regression analysis. More specifically, if z=(x-μ)/σ ~N(0,1) then the t distribution with ν=(n-1) degrees of freedom is used in place of z for samples of size n (typically <50) when σ2 is estimated from the sample.

W S Gosset derived the t distribution whilst working at the Guinness brewery, but was not permitted by his employer to publish under his own name so he decided to use the pseudonym "Student" for his published work. When deriving the distribution in 1908 Gosset [GOS1] assumed that the population distribution was Normal, but observed (using various examples) that even if this criteria does not strictly hold, the t distribution still provides a good model for use in inference regarding the population mean value or the difference between two means. Note that this distribution is symmetric, like the Normal, but the frequency distribution displays lower values near the population mean and higher values further away, reflecting the greater variability of small samples. As per the Normal distribution, the t-distribution has points of inflection, which are located at:

which rapidly approaches the Normal distribution value of 1 as ν increases. The graphs below demonstrate that for ν>20 the differences between the t distribution and the Normal are very small, and indeed up until Gosset's work the Normal was used for such tests.

Student's t-distribution - plots of 1,2,3,4,5,10,20,50 DF and Normal (limiting distribution)

t-dist

It is not immediately obvious how this distribution was obtained, but it fairly straightforward (see further, Mood and Graybill, 1950). The starting point is a random sample of size n, {x1,x2,x3... xn}, drawn from a Normal distribution with unknown mean, μ, and variance, σ2. We know that mean value of such a sample, suitably transformed, is distributed as a unit Normal, N(0,1) or U, but we only know the sample variance, s2, not the population variance, σ2. If we knew the population variance we could determine upper and lower confidence intervals on the sample mean by using the percentage points of the Normal distribution, and for larger samples this is perfectly acceptable. For smaller samples we need the distribution of an expression of the form:

where

is the sample variance. Substituting in the previous expression and ignoring the divisor (n-1) at present we have:

and this is the expression for which Gosset derived the distribution and published brief tables of the cumulative distribution function.A more complete table is provided in the Resources topic, Distribution tables section

Now the distribution of the mean and the distribution of the variance of a sample are independent, with the distribution of the mean being Normal in this case (one of our initial assumptions about the sample data). The distribution of the sample variance is the distribution of summed squared Normal variates, which we know is a Chi-square distribution. Thus the ratio, z, is simply the joint distribution of a unit Normal and the square root of a Chi-square variable with n-1 degrees of freedom.

The expression that Gosset derived above can also be written as:

so now we can see the sample variance, s, in the expression. If we remove the final square root term by dividing through by this value, we produce the form now known as the t distribution (or Fisher's distribution, as it was Fisher in 1925 who suggested this change). Thus we have the t distribution defined as:

The probability density functions of the unit Normal and the Chi-square distributions are:

and

so their joint probability density function is simply the product of these two distributions. If we then apply this with our ratio, and let

we can substitute into the joint pdf and integrate out u, giving the final result which is the pdf of the t distribution with ν degrees of freedom:

We have adopted this description of the genesis of the t distribution because it highlights the reason it came into use and remains widely applied today, principally as a method of determining confidence intervals for the population mean from sample mean values in small samples (see further, our section on t tests). Clearly it also applies more generally to any variable that can be expressed as the ratio of a Normal to the square root of a Chi-square variable.  A brief comparison of the t-distribution and Normal distribution is provided in the table below. As the degrees of freedom increases the probability levels can be seen to approach those of the Normal.

Table of the t-distribution with ν df and Normal distribution cumulative probabilities (one tail)

 

t-distribution

 

Normal

ν=1

ν=5

ν=10

ν=20

u or t

0.999

0.8976

0.9850

0.9933

0.9965

3

0.975

0.8498

0.9464

0.9608

0.9680

1.9600

0.950

0.8261

0.9195

0.9345

0.9422

1.6449

Key measures for the t-distribution are provided in the table below:

Item

Value

Mean/Median/Mode

0

Variance

ν/(ν-2), ν>2

Skewness

0, for ν>3

Kurtosis

6/(ν-4), ν>4

MGF

not defined

References

[GOS1] Gosset W S (1908) On the probable error of the mean. Biometrika, 6, 1-25.

[JOH1] Johnson N L, Kotz S (1970) Continuous Univariate Distributions, II. Houghton Mifflin/J Wiley & Sons, New York

[MOO1] Mood A M, Graybill F A (1950) Introduction to the Theory of Statistics. McGraw-Hill, New York

Mathworld: Weisstein E W: Student's t-distribution: http://mathworld.wolfram.com/Studentst-Distribution.html

Wikipedia: Student's t-distribution: http://en.wikipedia.org/wiki/T-distribution