Standard error

Revision as of 09:56, 28 October 2013

The standard error is a measure of the variability of estimation and defines the Confidence interval; it is the square root of the error variance. Error variance and standard error can be estimated for any estimated statistic; here, we refer to the estimated mean.

An empirical illustration of the standard error is: if we take all possible samples (for a defined sampling design) then we will produce many estimated means. These means follow a distribution which has, in the case of an unbiased estimator, a mean value which is equal to the true population mean. The variance of this distribution of means is the error variance of the mean, and the standard error is the standard deviation of the means. This standard error is also called standard error of the mean.

The estimated error variance is denoted as \(S_\bar y^2\) or as \(\hat{var}(\bar y)\) and the standard error as \(S_\bar y\) or \(SE\). The standard error of the mean can either be given in absolute or in relative terms; the latter is the ratio of the standard error of the mean and the mean and is frequently denoted as \(SE%\).

For a random sample of size \(n\), the parametric standard error is calculated from

\[\sigma_\hat {y}=\sqrt{var(\bar{y})}=\sqrt{\frac{\sigma^2}{n}}\] and the sample based standard error of the mean is estimated from

\[S_\hat {y}=\sqrt{\hat{var}(\bar{y})}=\sqrt{\frac{S^2}{n}}=\frac{S}{\sqrt{n}}\]

This estimator holds for sampling with replacement and for small samples or large populations, that is when only a small portion of the population comes into the sample (5% or less, say).

Finite population correction

For sampling without replacement, however, we would expect that the standard error is 0 when \(n = N\). But this is obviously not the case for the above estimator: as s is always greater 0, the expression cannot become 0. In order to make sure that this property holds, we must introduce the finite population correction (fpc) into the standard error estimator for sampling without replacement.

\[fpc=\frac{N-n}{n}=1-\frac{N}{n}\]

Then, for finite populations (or relatively large samples) the estimated standard error is

\[S_\bar{y}=\sqrt{\hat{var}*fpc}=\frac{S}{\sqrt{n}}*\sqrt{1-\frac{N}{n}}\]

which obviously becomes 0 if sample size \(n=N\).

If the parametric standard error is to be calculated, the finite population correction is

\[fpc=\frac{N-n}{N-1}\]

Relation of standard error and sample size

In order to make the standard error smaller, that is the estimation more precise one may increase the sample size. In the expression \(\frac{S}{\sqrt{n}}\), the sample size appears as square root; that means: if we wish to increase precision by the factor \(f\) (that is reduce the standard error by \(1/f\)), we need to multiply the sample size by \(f^2\)

Example:

If we wish to reduce the standard error to a half (that is: doubling precision), we must take 4 times as many sample. It is important to observe that the sampling intensity has no effect on the standard error!