S

TO

CK

HO

LMS+UN

IVE

RS

IT

ET Internal Report SUF{PFY/96{01Stockholm, 11 December 19961st revision, 31 October 1998last modi�cation 22 May 2001

Hand-book onSTATISTICALDISTRIBUTIONSforexperimentalistsbyChristian WalckParticle Physics GroupFysikumUniversity of Stockholm(e-mail: walck@physto.se)

Contents1 Introduction 11.1 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12 Probability Density Functions 32.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 32.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 32.2.1 Errors of Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 42.3 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 42.4 Probability Generating Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : 52.5 Cumulants : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 62.6 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 72.6.1 Cumulative Technique : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 72.6.2 Accept-Reject technique : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 72.6.3 Composition Techniques : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 82.7 Multivariate Distributions : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 92.7.1 Multivariate Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 92.7.2 Errors of Bivariate Moments : : : : : : : : : : : : : : : : : : : : : : : : : : 102.7.3 Joint Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : 102.7.4 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : 113 Bernoulli Distribution 123.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 123.2 Relation to Other Distributions : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 124 Beta distribution 134.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 134.2 Derivation of the Beta Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : 134.3 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 144.4 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 144.5 Probability Content : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 144.6 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 155 Binomial Distribution 165.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 165.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 165.3 Probability Generating Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : 165.4 Cumulative Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 175.5 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 175.6 Estimation of Parameters : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 175.7 Probability Content : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 186 Binormal Distribution 206.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 206.2 Conditional Probability Density : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 216.3 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 216.4 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 216.5 Box-Muller Transformation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 22i

6.6 Probability Content : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 236.7 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 247 Cauchy Distribution 267.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 267.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 267.3 Normalization : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 277.4 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 277.5 Location and Scale Parameters : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 277.6 Breit-Wigner Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 287.7 Comparison to Other Distributions : : : : : : : : : : : : : : : : : : : : : : : : : : : 287.8 Truncation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 297.9 Sum and Average of Cauchy Variables : : : : : : : : : : : : : : : : : : : : : : : : : 307.10 Estimation of the Median : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 317.11 Estimation of the HWHM : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 317.12 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 337.13 Physical Picture : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 347.14 Ratio Between Two Standard Normal Variables : : : : : : : : : : : : : : : : : : : : 358 Chi-square Distribution 378.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 378.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 378.3 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 398.4 Cumulative Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 398.5 Origin of the Chi-square Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : 398.6 Approximations : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 408.7 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 418.8 Con�dence Intervals for the Variance : : : : : : : : : : : : : : : : : : : : : : : : : : 418.9 Hypothesis Testing : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 418.10 Probability Content : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 428.11 Even Number of Degrees of Freedom : : : : : : : : : : : : : : : : : : : : : : : : : : 438.12 Odd Number of Degrees of Freedom : : : : : : : : : : : : : : : : : : : : : : : : : : 438.13 Final Algorithm : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 448.14 Chi Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 449 Compound Poisson Distribution 469.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 469.2 Branching Process : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 469.3 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 469.4 Probability Generating Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : 479.5 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4710 Double-Exponential Distribution 4810.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4810.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4810.3 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4810.4 Cumulative Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4910.5 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 49ii

11 Doubly Non-Central F -Distribution 5011.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5011.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5011.3 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5111.4 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5112 Doubly Non-Central t-Distribution 5212.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5212.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5212.3 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5312.4 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5313 Error Function 5413.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5413.2 Probability Density Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5414 Exponential Distribution 5514.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5514.2 Cumulative Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5514.3 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5514.4 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5614.5 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5614.5.1 Method by von Neumann : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5614.5.2 Method by Marsaglia : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5614.5.3 Method by Ahrens : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5715 Extreme Value Distribution 5815.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5815.2 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5915.3 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5915.4 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5915.5 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6116 F-distribution 6216.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6216.2 Relations to Other Distributions : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6316.3 1/F : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6316.4 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6316.5 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6316.6 F-ratio : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6416.7 Variance Ratio : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6516.8 Analysis of Variance : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6616.9 Calculation of Probability Content : : : : : : : : : : : : : : : : : : : : : : : : : : : 6616.9.1 The Incomplete Beta function : : : : : : : : : : : : : : : : : : : : : : : : : : 6716.9.2 Final Formul� : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6816.10 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 69iii

17 Gamma Distribution 7017.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7017.2 Derivation of the Gamma Distribution : : : : : : : : : : : : : : : : : : : : : : : : : 7017.3 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7117.4 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7117.5 Probability Content : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7217.6 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7217.6.1 Erlangian distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7217.6.2 General case : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7217.6.3 Asymptotic Approximation : : : : : : : : : : : : : : : : : : : : : : : : : : : 7318 Generalized Gamma Distribution 7418.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7418.2 Cumulative Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7418.3 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7518.4 Relation to Other Distributions : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7519 Geometric Distribution 7619.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7619.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7619.3 Probability Generating Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7619.4 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7620 Hyperexponential Distribution 7820.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7820.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7820.3 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7820.4 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 7921 Hypergeometric Distribution 8021.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8021.2 Probability Generating Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8021.3 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8021.4 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8122 Logarithmic Distribution 8222.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8222.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8222.3 Probability Generating Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8222.4 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8323 Logistic Distribution 8423.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8423.2 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8423.3 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8523.4 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8523.5 Random numbers : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 86iv

24 Log-normal Distribution 8724.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8724.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8724.3 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8824.4 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8825 Maxwell Distribution 8925.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8925.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 8925.3 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9025.4 Kinetic Theory : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9025.5 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9126 Moyal Distribution 9226.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9226.2 Normalization : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9326.3 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9326.4 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9326.5 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9426.6 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9427 Multinomial Distribution 9627.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9627.2 Histogram : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9627.3 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9627.4 Probability Generating Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9727.5 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9727.6 Signi�cance Levels : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9727.7 Equal Group Probabilities : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 9728 Multinormal Distribution 10028.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10028.2 Conditional Probability Density : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10028.3 Probability Content : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10028.4 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10129 Negative Binomial Distribution 10329.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10329.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10329.3 Probability Generating Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10429.4 Relations to Other Distributions : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10429.4.1 Poisson Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10429.4.2 Gamma Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10529.4.3 Logarithmic Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10629.4.4 Branching Process : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10729.4.5 Poisson and Gamma Distributions : : : : : : : : : : : : : : : : : : : : : : : 10729.5 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 108v

30 Non-central Beta-distribution 10930.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10930.2 Derivation of distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 10930.3 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11030.4 Cumulative distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11030.5 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11031 Non-central Chi-square Distribution 11131.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11131.2 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11131.3 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11231.4 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11231.5 Approximations : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11231.6 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11332 Non-central F -Distribution 11432.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11432.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11532.3 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11532.4 Approximations : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11632.5 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11633 Non-central t-Distribution 11733.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11733.2 Derivation of distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11833.3 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11833.4 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11833.5 Approximation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11933.6 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 11934 Normal Distribution 12034.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12034.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12034.3 Cumulative Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12134.4 Characteristic Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12134.5 Addition Theorem : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12234.6 Independence of x and s2 : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12234.7 Probability Content : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12334.8 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12534.8.1 Central Limit Theory Approach : : : : : : : : : : : : : : : : : : : : : : : : 12534.8.2 Exact Transformation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12534.8.3 Polar Method : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12534.8.4 Trapezoidal Method : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12634.8.5 Center-tail method : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12734.8.6 Composition-rejection Methods : : : : : : : : : : : : : : : : : : : : : : : : : 12834.8.7 Method by Marsaglia : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12834.8.8 Histogram Technique : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 12934.8.9 Ratio of Uniform Deviates : : : : : : : : : : : : : : : : : : : : : : : : : : : : 130vi

34.8.10Comparison of random number generators : : : : : : : : : : : : : : : : : : : 13234.9 Tests on Parameters of a Normal Distribution : : : : : : : : : : : : : : : : : : : : : 13335 Pareto Distribution 13435.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13435.2 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13435.3 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13435.4 Random Numbers : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13436 Poisson Distribution 13536.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13536.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13536.3 Probability Generating Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13636.4 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13636.5 Addition Theorem : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13636.6 Derivation of the Poisson Distribution : : : : : : : : : : : : : : : : : : : : : : : : : 13736.7 Histogram : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13736.8 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13837 Rayleigh Distribution 13937.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13937.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 13937.3 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14037.4 Two-dimensional Kinetic Theory : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14037.5 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14138 Student's t-distribution 14238.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14238.2 History : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14238.3 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14338.4 Cumulative Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14438.5 Relations to Other Distributions : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14438.6 t-ratio : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14538.7 One Normal Sample : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14638.8 Two Normal Samples : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14638.9 Paired Data : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14638.10 Con�dence Levels : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14738.11 Testing Hypotheses : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 14738.12 Calculation of Probability Content : : : : : : : : : : : : : : : : : : : : : : : : : : : 14838.12.1 Even number of degrees of freedom : : : : : : : : : : : : : : : : : : : : : : 14838.12.2 Odd number of degrees of freedom : : : : : : : : : : : : : : : : : : : : : : : 14938.12.3 Final algorithm : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15038.13 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15139 Triangular Distribution 15239.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15239.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15239.3 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 152vii

40 Uniform Distribution 15340.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15340.2 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15340.3 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15341 Weibull Distribution 15441.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15441.2 Cumulative Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15541.3 Moments : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15541.4 Random Number Generation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15542 Appendix A: The Gamma and Beta Functions 15642.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15642.2 The Gamma Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15642.2.1 Numerical Calculation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15742.2.2 Formul� : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15842.3 Digamma Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 15942.4 Polygamma Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16042.5 The Incomplete Gamma Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16142.5.1 Numerical Calculation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16142.5.2 Formul� : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16242.5.3 Special Cases : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16242.6 The Beta Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16342.7 The Incomplete Beta Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16342.7.1 Numerical Calculation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16442.7.2 Approximation : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16542.8 Relations to Probability Density Functions : : : : : : : : : : : : : : : : : : : : : : 16542.8.1 The Beta Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16542.8.2 The Binomial Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16542.8.3 The Chi-squared Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : 16642.8.4 The F -distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16642.8.5 The Gamma Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16642.8.6 The Negative Binomial Distribution : : : : : : : : : : : : : : : : : : : : : : 16642.8.7 The Normal Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16742.8.8 The Poisson Distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16742.8.9 Student's t-distribution : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16842.8.10 Summary : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16843 Appendix B: Hypergeometric Functions 16943.1 Introduction : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16943.2 Hypergeometric Function : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 16943.3 Con uent Hypergeometric Function : : : : : : : : : : : : : : : : : : : : : : : : : : 170viii

Mathematical Constants : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 182Errata et Addenda : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 184References : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :187Index : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 190List of Tables1 Percentage points of the chi-square distribution : : : : : : : : : : : : : : : : : : : : 1732 Extreme con�dence levels for the chi-square distribution : : : : : : : : : : : : : : : 1743 Extreme con�dence levels for the chi-square distribution (as �2/d.f. values) : : : : 1754 Exact and approximate values for the Bernoulli numbers : : : : : : : : : : : : : : : 1765 Percentage points of the F -distribution : : : : : : : : : : : : : : : : : : : : : : : : : 1776 Probability content from �z to z of Gauss distribution in % : : : : : : : : : : : : : 1787 Standard normal distribution z-values for a speci�c probability content : : : : : : : 1798 Percentage points of the t-distribution : : : : : : : : : : : : : : : : : : : : : : : : : 1809 Expressions for the Beta function B(m;n) for integer and half-integer arguments : 181

ix

x

1 IntroductionIn experimental work e.g. in physics one often encounters problems where a standardstatistical probability density function is applicable. It is often of great help to be ableto handle these in di�erent ways such as calculating probability contents or generatingrandom numbers.For these purposes there are excellent text-books in statistics e.g. the classical work ofMaurice G. Kendall and Alan Stuart [1,2] or more modern text-books as [3] and others.Some books are particularly aimed at experimental physics or even speci�cally at particlephysics [4{8]. Concerning numerical methods a valuable references worth mentioning is [9]which has been surpassed by a new edition [10]. Also hand-books, especially [11], has beenof great help throughout.However, when it comes to actual applications it often turns out to be hard to �nd de-tailed explanations in the literature ready for implementation. This work has been collectedover many years in parallel with actual experimental work. In this way some material maybe \historical" and sometimes be na��ve and have somewhat clumsy solutions not alwaysmade in the mathematically most stringent may. We apologize for this but still hope thatit will be of interest and help for people who is struggling to �nd methods to solve theirstatistical problems in making real applications and not only learning statistics as a course.Even if one has the skill and may be able to �nd solutions it seems worthwhile to haveeasy and fast access to formul� ready for application. Similar books and reports exist e.g.[12,13] but we hope the present work may compete in describing more distributions, beingmore complete, and including more explanations on relations given.The material could most probably have been divided in a more logical way but wehave chosen to present the distributions in alphabetic order. In this way it is more of ahand-book than a proper text-book.After the �rst release the report has been modestly changed. Minor changes to cor-rect misprints is made whenever found. In a few cases subsections and tables have beenadded. These alterations are described on page 184. In October 1998 the �rst somewhatbigger revision was made where in particular a lot of material on the non-central samplingdistributions were added.1.1 Random Number GenerationIn modern computing Monte Carlo simulations are of vital importance and we give meth-ods to achieve random numbers from the distributions. An earlier report dealt entirelywith these matters [14]. Not all text-books on statistics include information on this subjectwhich we �nd extremely useful. Large simulations are common in particle physics as well asin other areas but often it is also useful to make small \toy Monte Carlo programs" to inves-tigate and study analysis tools developed on ideal, but statistically sound, random samples.A related and important �eld which we will only mention brie y here, is how to getgood basic generators for achieving random numbers uniformly distributed between zero1

and one. Those are the basis for all the methods described in order to get random numbersfrom speci�c distributions in this document. For a review see e.g. [15].From older methods often using so called multiplicative congruential method or shift-generators G. Marsaglia et al [16] introduced in 1989 a new \universal generator" whichbecame the new standard in many �elds. We implemented this in our experiments atCERN and also made a package of routines for general use [17].This method is still a very good choice but later alternatives, claimed to be even better,have turned up. These are based on on the same type of lagged Fibonacci sequences asis used in the universal generator and was originally proposed by the same authors [18].An implementations of this method was proposed by F. James [15] and this version wasfurther developed by M. L�uscher [19]. A similar package of routine as was prepared for theuniversal generator has been implemented for this method [20].

2

2 Probability Density Functions2.1 IntroductionProbability density functions in one, discrete or continuous, variable are denoted p(r) andf(x), respectively. They are assumed to be properly normalized such thatXr p(r) = 1 and 1Z�1 f(x)dx = 1where the sum or the integral are taken over all relevant values for which the probabilitydensity function is de�ned.Statisticians often use the distribution function or as physicists more often call it thecumulative function which is de�ned asP (r) = rXi=�1 p(i) and F (x) = xZ�1 f(t)dt2.2 MomentsAlgebraic moments of order r are de�ned as the expectation value�0r = E(xr) =Xk krp(k) or 1Z�1 xrf(x)dxObviously �00 = 1 from the normalization condition and �01 is equal to the mean, sometimescalled the expectation value, of the distribution.Central moments of order r are de�ned as�r = E((k �E(k))r) or E((x� E(x))r)of which the most commonly used is �2 which is the variance of the distribution.Instead of using the third and fourth central moments one often de�nes the coe�cientsof skewness 1 and kurtosis1 2 by 1 = �3� 322 and 2 = �4�22 � 3where the shift by 3 units in 2 assures that both measures are zero for a normal distribution.Distributions with positive kurtosis are called leptokurtic, those with kurtosis around zeromesokurtic and those with negative kurtosis platykurtic. Leptokurtic distributions arenormally more peaked than the normal distribution while platykurtic distributions aremore at topped.1From greek kyrtosis = curvature from kyrt(�os) = curved, arched, round, swelling, bulging. Sometimes,especially in older literature, 2 is called the coe�cient of excess.3

2.2.1 Errors of MomentsFor a thorough presentation of how to estimate errors on moments we refer to the classicalbooks by M. G. Kendall and A. Stuart [1] (pp 228{245). Below only a brief description isgiven. For a sample with n observations x1; x2; : : : ; xn we de�ne the moment-statistics forthe algebraic and central moments m0r and mr asm0r = 1n nXr=0xr and mr = 1n nXr=0 (x�m01)rThe notation m0r and mr are thus used for the statistics (sample values) while we denotethe true, population, values by �0r and �r.The mean value of the r:th and the sampling covariance between the q:th and r:thmoment-statistic are given by. E(m0r) = �0rCov(m0q;m0r) = 1n ��0q+r � �0q�0r�These formula are exact. Formul� for moments about the mean are not as simple sincethe mean itself is subject to sampling uctuations.E(mr) = �rCov(mq;mr) = 1n (�q+r � �q�r + rq�2�r�1�q�1 � r�r�1�q+1 � q�r+1�q�1)to order 1=pn and 1=n, respectively. The covariance between an algebraic and a centralmoment is given by Cov(mr;m0q) = 1n(�q+r � �q�r � r�q+1�r�1)to order 1=n. Note especially thatV (m0r) = 1n ��02r � �02r �V (mr) = 1n ��2r � �2r + r2�2�2r�1 � 2r�r�1�r+1�Cov(m01;mr) = 1n (�r+1 � r�2�r�1)2.3 Characteristic FunctionFor a distribution in a continuous variable x the Fourier transform of the probability densityfunction �(t) = E(e{xt) = 1Z�1 e{xtf(x)dx4

is called the characteristic function. It has the properties that �(0) = 1 and j�(t)j � 1for all t. If the cumulative, distribution, function F (x) is continuous everywhere anddF (x) = f(x)dx then we reverse the transform such thatf(x) = 12� 1Z�1 �(t)e�{xtdtThe characteristic function is related to the moments of the distribution by�x(t) = E(e{tx) = 1Xn=0 ({t)nE(xn)n! = 1Xn=0 ({t)n�0nn!e.g. algebraic moments may be found by�0r = 1{r ddt!r �(t)�����t=0To �nd central moments (about the mean �) use�x��(t) = E �e{t(x��)� = e�{t��x(t)and thus �r = 1{r ddt!r e�{t��(t)�����t=0A very useful property of the characteristic function is that for independent variables xand y �x+y(t) = �x(t) � �y(t)As an example regard the sum P aizi where the zi's are distributed according to normaldistributions with means �i and variances �2i . Then the linear combination will also bedistributed according to the normal distribution with mean P ai�i and variance P a2i�2i .To show that the characteristic function in two variables factorizes is the best way toshow independence between two variables. Remember that a vanishing correlation coe�-cient does not imply independence while the reversed is true.2.4 Probability Generating FunctionIn the case of a distribution in a discrete variable r the characteristic function is given by�(t) = E(e{tr) =X p(r)e{trIn this case it is often convenient to write z = e{t and de�ne the probability generatingfunction as G(z) = E(zr) =X p(r)zr5

Derivatives of G(z) evaluated at z = 1 are related to factorial moments of the distribu-tion G(1) = 1 (normalization)G1(1) = ddzG(z)�����z=1 = E(r)G2(1) = d2dz2G(z)�����z=1 = E(r(r � 1))G3(1) = d3dz3G(z)�����z=1 = E(r(r � 1)(r � 2))Gk(1) = dkdzkG(z)�����z=1 = E(r(r � 1)(r � 2) � � � (r � k + 1))Lower order algebraic moments are then given by�01 = G1(1)�02 = G2(1) + G1(1)�03 = G3(1) + 3G2(1) +G1(1)�04 = G4(1) + 6G3(1) + 7G2(1) +G1(1)while expression for central moments become more complicated.A useful property of the probability generating function is for a branching process in nsteps where G(z) = G1(G2(: : :Gn�1(Gn(z)) : : :))with Gk(z) the probability generating function for the distribution in the k:th step. As anexample see section 29.4.4 on page 107.2.5 CumulantsAlthough not much used in physics the cumulants, �r, are of statistical interest. Onereason for this is that they have some useful properties such as being invariant for a shiftin scale (except the �rst cumulant which is equal to the mean and is shifted along withthe scale). Multiplying the x-scale by a constant a has the same e�ect as for algebraicmoments namely to multiply �r by ar.As the algebraic moment �0n is the coe�cient of ({t)n=n! in the expansion of �(t) the cu-mulant �n is the coe�cient of ({t)n=n! in the expansion of the logarithm of �(t) (sometimescalled the cumulant generating function) i.e.ln�(t) = 1Xn=1 ({t)nn! �nand thus �r = 1{r ddt!r ln�(t)�����t=0Relations between cumulants and central moments for some lower orders are as follows6

�1 = �01�2 = �2 �2 = �2�3 = �3 �3 = �3�4 = �4 � 3�22 �4 = �4 + 3�22�5 = �5 � 10�3�2 �5 = �5 + 10�3�2�6 = �6 � 15�4�2 � 10�23 + 30�32 �6 = �6 + 15�4�2 + 10�23 + 15�32�7 = �7 � 21�5�2 � 35�4�3 + 210�3�22 �7 = �7 + 21�5�2 + 35�4�3 + 105�3�22�8 = �8 � 28�6�2 � 56�5�3 � 35�24+ �8 = �8 + 28�6�2 + 56�5�3 + 35�24++420�4�22 + 560�23�2 � 630�42 +210�4�22 + 280�23�2 + 105�422.6 Random Number GenerationWhen generating random numbers from di�erent distribution it is assumed that a goodgenerator for uniform pseudorandom numbers between zero and one exist (normally theend-points are excluded).2.6.1 Cumulative TechniqueThe most direct technique to obtain random numbers from a continuous probability densityfunction f(x) with a limited range from xmin to xmax is to solve for x in the equation� = F (x)� F (xmin)F (xmax)� F (xmin)where � is uniformly distributed between zero and one and F (x) is the cumulative dis-tribution (or as statisticians say the distribution function). For a properly normalizedprobability density function thus x = F�1(�)The technique is sometimes also of use in the discrete case if the cumulative sum maybe expressed in analytical form as e.g. for the geometric distribution.Also for general cases, discrete or continuous, e.g. from an arbitrary histogram thecumulative method is convenient and often faster than more elaborate methods. In thiscase the task is to construct a cumulative vector and assign a random number according tothe value of a uniform random number (interpolating within bins in the continuous case).2.6.2 Accept-Reject techniqueA useful technique is the acceptance-rejection, or hit-miss, method where we choose fmax tobe greater than or equal to f(x) in the entire interval between xmin and xmax and proceedas followsi Generate a pair of uniform pseudorandom numbers �1 and �2.ii Determine x = xmin + �1 � (xmax� xmin).iii Determine y = fmax � �2. 7

iv If y�f(x) > 0 reject and go to i else accept x as a pseudorandom number from f(x).The e�ciency of this method depends on the average value of f(x)=fmax over the in-terval. If this value is close to one the method is e�cient. On the other hand, if thisaverage is close to zero, the method is extremely ine�cient. If � is the fraction of the areafmax � (xmax�xmin) covered by the function the average number of rejects in step iv is 1��1and 2� uniform pseudorandom numbers are required on average.The e�ciency of this method can be increased if we are able to choose a function h(x),from which random numbers are more easily obtained, such that f(x) � �h(x) = g(x) overthe entire interval under consideration (where � is a constant). A random sample fromf(x) is obtained byi Generate in x a random number from h(x).ii Generate a uniform random number �.iii If � � f(x)=g(x) go back to i else accept x as a pseudorandom number from f(x).Yet another situation is when a function g(x), from which fast generation may beobtained, can be inscribed in such a way that a big proportion (f) of the area under thefunction is covered (as an example see the trapezoidal method for the normal distribution).Then proceed as follows:i Generate a uniform random number �.ii If � < f then generate a random number from g(x).iii Else use the acceptance/rejection technique for h(x) = f(x)� g(x) (in subintervals ifmore e�cient).2.6.3 Composition TechniquesIf f(x) may be written in the formf(x) = 1Z�1 gz(x)dH(z)where we know how to sample random numbers from the p.d.f. g(x) and the distributionfunction H(z). A random number from f(x) is then obtained byi Generate two uniform random numbers �1 and �2.ii Determine z = H�1(�1).iii Determine x = G�1z (�2) where Gz is the distribution function corresponding to thep.d.f. gz(x). 8

For more detailed information on the Composition technique see [21] or [22].A combination of the composition and the rejection method has been proposed byJ. C. Butcher [23]. If f(x) can be writtenf(x) = nXi=0 �ifi(x)gi(x)where �i are positive constants, fi(x) p.d.f.'s for which we know how to sample a randomnumber and gi(x) are functions taking values between zero and one. The method is thenas follows:i Generate uniform random numbers �1 and �2.ii Determine an integer k from the discrete distribution pi = �i=(�1 + �2 + ::: + �n)using �1.iii Generate a random number x from fk(x).iv Determine gk(x) and if �2 > gk(x) then go to i.v Accept x as a random number from f(x).2.7 Multivariate DistributionsJoint probability density functions in several variables are denoted by f(x1; x2; : : : ; xn) andp(r1; r2; : : : ; rn) for continuous and discrete variables, respectively. It is assumed that theyare properly normalized i.e. integrated (or summed) over all variables the result is unity.2.7.1 Multivariate MomentsThe generalization of algebraic and central moments to multivariate distributions is straight-forward. As an example we take a bivariate distribution f(x; y) in two continuous variablesx and y and de�ne algebraic and central bivariate moments of order k; ` as�0k` � E(xky`) = ZZ xky`f(x; y)dxdy�k` � E((x� �x)k(y � �y)`) = ZZ (x� �x)k(y � �y)`f(x; y)dxdywhere �x and �y are the mean values of x and y. The covariance is a central bivariatemoment of order 1; 1 i.e. Cov(x; y) = �11. Similarly one easily de�nes multivariate momentsfor distribution in discrete variables. 9

2.7.2 Errors of Bivariate MomentsAlgebraic (m0rs) and central (mrs) bivariate moments are de�ned by:m0rs = 1n nXi=1 xriysi and mrs = 1n nXi=1(xi �m010)r(yi �m001)sWhen there is a risk of ambiguity we write mr;s instead of mrs.The notations m0rs and mrs are used for the statistics (sample values) while we write�0rs and �rs for the population values. The errors of bivariate moments are given byCov(m0rs;m0uv) = 1n(�0r+u;s+v � �0rs�0uv)Cov(mrs;muv) = 1n(�r+u;s+v � �rs�uv + ru�20�r�1;s�u�1;v + sv�02�r;s�1�u;v�1+rv�11�r�1;s�u;v�1 + su�11�r;s�1�u�1;v � u�r+1;s�u�1;v�v�r;s+1�u;v�1 � r�r�1;s�u+1;v � s�r;s�1�u;v+1)especially V (m0rs) = 1n(�02r;2s � �02rs)V (mrs) = 1n(�2r;2s � �2rs + r2�20�2r�1;s + s2�02�2r;s�1+2rs�11�r�1;s�r;s�1 � 2r�r+1;s�r�1;s � 2s�r;s+1�r;s�1)For the covariance (m11) we get by error propagationV (m11) = 1n (�22 � �211)Cov(m11;m010) = �21nCov(m11;m20) = 1n (�31 � �20�11)For the correlation coe�cient (denoted by � = �11=p�20�02 for the population value andby r for the sample value) we getV (r) = �2n (�22�211 + 14 "�40�220 + �04�202 + 2�22�20�02# � 1�11 "�31�20 + �13�02#)Beware, however, that the sampling distribution of r tends to normality very slowly.2.7.3 Joint Characteristic FunctionThe joint characteristic function is de�ned by�(t1; t2; : : : ; tn) = E(e{t1x1+{t2x2+:::tnxn) == 1Z�1 1Z�1 : : : 1Z�1 e{t1x1+{t2x2+:::+{tnxnf(x1; x2; : : : ; xn)dx1dx2 : : : dxn10

From this function multivariate moments may be obtained e.g. for a bivariate distributionalgebraic bivariate moments are given by�0rs = E(xr1xs2) = @r+s�(t1; t2)@({t1)r@({t2)s �����t1=t2=02.7.4 Random Number GenerationRandom sampling from a many dimensional distribution with a joint probability densityfunction f(x1; x2; :::; xn) can be made by the following method:� De�ne the marginal distributionsgm(x1; x2; :::; xm) = Z f(x1; :::; xn)dxm+1dxm+2:::dxn = Z gm+1(x1; :::; xm+1)dxm+1� Consider the conditional density function hm given byhm(xmjx1; x2; :::xm�1) � gm(x1; x2; :::; xm)=gm�1(x1; x2; :::; xm�1)� We see that gn = f and thatZ hm(xmjx1; x2; :::; xm�1)dxm = 1from the de�nitions. Thus hm is the conditional distribution in xm given �xed valuesfor x1; x2; :::; xm�1.� We can now factorize f asf(x1; x2; :::; xn) = h1(x1)h2(x2jx1) : : : hn(xnjx1; x2; :::; xn�1)� We sample values for x1; x2; :::; xn from the joint probability density function f by:{ Generate a value for x1 from h1(x1).{ Use x1 and sample x2 from h2(x2jx1).{ Proceed step by step and use previously sampled values for x1; x2; :::; xm toobtain a value for xm+1 from hm+1(xm+1jx1; x2; :::; xm).{ Continue until all xi:s have been sampled.� If all xi:s are independent the conditional densities will equal the marginal densitiesand the variables can be sampled in any order.11

3 Bernoulli Distribution3.1 IntroductionThe Bernoulli distribution, named after the swiss mathematician Jacques Bernoulli (1654{1705), describes a probabilistic experiment where a trial has two possible outcomes, asuccess or a failure.The parameter p is the probability for a success in a single trial, the probability for afailure thus being 1 � p (often denoted by q). Both p and q is limited to the interval fromzero to one. The distribution has the simple formp(r; p) = � 1 � p = q if r = 0 (failure)p if r = 1 (success)and zero elsewhere. The work of J. Bernoulli, which constitutes a foundation of probabilitytheory, was published posthumously in Ars Conjectandi (1713) [24].The probability generating function is G(z) = q+pz and the distribution function givenby P (0) = q and P (1) = 1. A random numbers are easily obtained by using a uniformrandom number variate � and putting r = 1 (success) if � � p and r = 0 else (failure).3.2 Relation to Other DistributionsFrom the Bernoulli distribution we may deduce several probability density functions de-scribed in this document all of which are based on series of independent Bernoulli trials:� Binomial distribution: expresses the probability for r successes in an experimentwith n trials (0 � r � n).� Geometric distribution: expresses the probability of having to wait exactly r trialsbefore the �rst successful event (r � 1).� Negative Binomial distribution: expresses the probability of having to wait ex-actly r trials until k successes have occurred (r � k). This form is sometimes referredto as the Pascal distribution.Sometimes this distribution is expressed as the number of failures n occurring whilewaiting for k successes (n � 0).12

4 Beta distribution4.1 IntroductionThe Beta distribution is given byf(x; p; q) = 1B(p; q)xp�1(1� x)q�1where the parameters p and q are positive real quantities and the variable x satis�es 0 �x � 1. The quantity B(p; q) is the Beta function de�ned in terms of the more commonGamma function as B(p; q) = �(p)�(q)�(p + q)For p = q = 1 the Beta distribution simply becomes a uniform distribution betweenzero and one. For p = 1 and q = 2 or vise versa we get triangular shaped distributions,f(x) = 2 � 2x and f(x) = 2x. For p = q = 2 we obtain a distribution of parabolic shape,f(x) = 6x(1�x). More generally, if p and q both are greater than one the distribution hasa unique mode at x = (p � 1)=(p + q � 2) and is zero at the end-points. If p and/or q isless than one f(0)!1 and/or f(1) !1 and the distribution is said to be J-shaped. In�gure 1 below we show the Beta distribution for two cases: p = q = 2 and p = 6; q = 3.Figure 1: Examples of Beta distributions4.2 Derivation of the Beta DistributionIf ym and yn are two independent variables distributed according to the chi-squared distri-bution with m and n degrees of freedom, respectively, then the ratio ym=(ym + yn) followsa Beta distribution with parameters p = m2 and q = n2 .13

To show this we make a change of variables to x = ym=(ym+yn) and y = ym+yn whichimplies that ym = xy and yn = y(1� x). We obtainf(x; y) = ���������� @ym@x @ym@y@yn@x @yn@y ���������� f(ym; yn) == ���� y x�y 1 � x ���� 8><>:�ym2 �m2 �1 e� ym22� �m2 � 9>=>;8><>:�yn2 �n2�1 e� yn22� �n2� 9>=>; == 8<: � �m+n2 �� �m2 �� �n2�xm2 �1(1� x)n2�19=;8><>:�y2�m2 +n2�1 e� y22� �m+n2 � 9>=>;which we recognize as a product of a Beta distribution in the variable x and a chi-squareddistribution with m + n degrees of freedom in the variable y (as expected for the sum oftwo independent chi-square variables).4.3 Characteristic FunctionThe characteristic function of the Beta distribution may be expressed in terms of thecon uent hypergeometric function (see section 43.3) as�(t) =M(p; p + q; {t)4.4 MomentsThe expectation value, variance, third and fourth central moment are given byE(x) = pp + qV (x) = pq(p + q)2(p+ q + 1)�3 = 2pq(q � p)(p + q)3(p+ q + 1)(p + q + 2)�4 = 3pq(2(p + q)2 + pq(p+ q � 6))(p + q)4(p+ q + 1)(p + q + 2)(p + q + 3)More generally algebraic moments are given in terms of the Beta function by�0k = B(p+ k; q)B(p; q)4.5 Probability ContentIn order to �nd the probability content for a Beta distribution we form the cumulativedistribution F (x) = 1B(p; q) xZ0 tp�1(1� t)q�1dt = Bx(p; q)B(p; q) = Ix(p; q)14

where both Bx and Ix seems to be called the incomplete Beta function in the literature.The incomplete Beta function Ix is connected to the binomial distribution for integervalues of a by1 � Ix(a; b) = I1�x(b; a) = (1� x)a+b�1 a�1Xi=0 a+ b� 1i !� x1� x�ior expressed in the opposite directionnXs=a ns!ps(1� p)n�s = Ip(a; n� a+ 1)Also to the negative binomial distribution there is a connection by the relationnXs=a n+ s� 1s !pnqs = Iq(a; n)The incomplete Beta function is also connected to the probability content of Student'st-distribution and the F -distribution. See further section 42.7 for more information on Ix.4.6 Random Number GenerationIn order to obtain random numbers from a Beta distribution we �rst single out a few specialcases.For p = 1 and/or q = 1 we may easily solve the equation F (x) = � where F (x) is thecumulative function and � a uniform random number between zero and one. In these casesp = 1 ) x = 1 � �1=qq = 1 ) x = �1=pFor p and q half-integers we may use the relation to the chi-square distribution byforming the ratio ymym + ynwith ym and yn two independent random numbers from chi-square distributions with m =2p and n = 2q degrees of freedom, respectively.Yet another way of obtaining random numbers from a Beta distribution valid when pand q are both integers is to take the `:th out of k (1 � ` � k) independent uniform randomnumbers between zero and one (sorted in ascending order). Doing this we obtain a Betadistribution with parameters p = ` and q = k + 1 � `. Conversely, if we want to generaterandom numbers from a Beta distribution with integer parameters p and q we could usethis technique with ` = p and k = p+q�1. This last technique implies that for low integervalues of p and q simple code may be used, e.g. for p = 2 and q = 1 we may simply takemax(�1; �2) i.e. the maximum of two uniform random numbers.15

5 Binomial Distribution5.1 IntroductionThe Binomial distribution is given byp(r;N; p) = Nr !pr(1� p)N�rwhere the variable r with 0 � r � N and the parameter N (N > 0) are integers and theparameter p (0 � p � 1) is a real quantity.The distribution describes the probability of exactly r successes in N trials if the prob-ability of a success in a single trial is p (we sometimes also use q = 1 � p, the probabilityfor a failure, for convenience). It was �rst presented by Jacques Bernoulli in a work whichwas posthumously published [24].5.2 MomentsThe expectation value, variance, third and fourth moment are given byE(r) = NpV (r) = Np(1 � p) = Npq�3 = Np(1 � p)(1 � 2p) = Npq(q � p)�4 = Np(1 � p) [1 + 3p(1 � p)(N � 2)] = Npq [1 + 3pq(N � 2)]Central moments of higher orders may be obtained by the recursive formula�r+1 = pq (Nr�r�1 + @�r@p )starting with �0 = 1 and �1 = 0.The coe�cients of skewness and kurtosis are given by 1 = q � ppNpq and 2 = 1 � 6pqNpq5.3 Probability Generating FunctionThe probability generating function is given byG(z) = E(zr) = NXr=0 zr Nr !pr(1� p)N�r = (pz + q)Nand the characteristic function thus by�(t) = G(e{t) = �q + pe{t�N16

5.4 Cumulative FunctionFor �xed N and p one may easily construct the cumulative function P (r) by a recursiveformula, see section on random numbers below.However, an interesting and useful relation exist between P (r) and the incomplete Betafunction Ix namely P (k) = kXr=0 p(r;N; p) = I1�p(N � k; k + 1)For further information on Ix see section 42.7.5.5 Random Number GenerationIn order to achieve random numbers from a binomial distribution we may either� Generate N uniform random numbers and accumulate the number of such that areless or equal to p, or� Use the cumulative technique, i.e. construct the cumulative, distribution, functionand by use of this and one uniform random number obtain the required randomnumber, or� for larger values of N , say N > 100, use an approximation to the normal distributionwith mean Np and variance Npq.Except for very small values of N and very high values of p the cumulative technique is thefastest for numerical calculations. This is especially true if we proceed by constructing thecumulative vector once for all2 (as opposed to making this at each call) using the recursiveformula p(i) = p(i� 1) pq N + 1� iifor i = 1; 2; : : : ; N starting with p(0) = qN .However, using the relation given in the previous section with a well optimized codefor the incomplete Beta function (see [10] or section 42.7) turns out to be a numericallymore stable way of creating the cumulative distribution than a simple loop adding up theindividual probabilities.5.6 Estimation of ParametersExperimentally the quantity rN , the relative number of successes in N trials, often is of moreinterest than r itself. This variable has expectation E( rN ) = p and variance V ( rN ) = pqN .The estimated value for p in an experiment giving r successes in N trials is p̂ = rN .2This is possible only if we require random numbers from one and the same binomial distribution with�xed values of N and p. 17

If p is unknown a unbiased estimate of the variance of a binomial distribution is givenby V (r) = NN � 1N � rN ��1� rN � = NN � 1Np̂(1� p̂)To �nd lower and upper con�dence levels for p we proceed as follows.� For lower limits �nd a plow such thatNXr=k Nr !prlow(1� plow)N�r = 1 � �or expressed in terms of the incomplete Beta function 1� I1�p(N � k+1; k) = 1��� for upper limits �nd a pup such thatkXr=0 Nr !prup(1� pup)N�r = 1 � �which is equivalent to I1�p(N � k; k + 1) = 1 � � i.e. Ip(k + 1; N � k) = �.As an example we take an experiment with N = 10 where a certain number of successes0 � k � N have been observed. The con�dence levels corresponding to 90%, 95%, 99%as well as the levels corresponding to one, two and three standard deviations for a normaldistribution (84.13%, 97.72% and 99.87% probability content) are given below.Lower con�dence levels Upper con�dence levelsk �3� 99% �2� 95% 90% �� p̂ �� 90% 95% �2� 99% �3�0 0.00 0.17 0.21 0.26 0.31 0.37 0.481 0.00 0.00 0.00 0.01 0.01 0.02 0.10 0.29 0.34 0.39 0.45 0.50 0.612 0.01 0.02 0.02 0.04 0.05 0.07 0.20 0.41 0.45 0.51 0.56 0.61 0.713 0.02 0.05 0.06 0.09 0.12 0.14 0.30 0.51 0.55 0.61 0.66 0.70 0.794 0.05 0.09 0.12 0.15 0.19 0.22 0.40 0.60 0.65 0.70 0.74 0.78 0.855 0.10 0.15 0.18 0.22 0.27 0.30 0.50 0.70 0.73 0.78 0.82 0.85 0.906 0.15 0.22 0.26 0.30 0.35 0.40 0.60 0.78 0.81 0.85 0.88 0.91 0.957 0.21 0.30 0.34 0.39 0.45 0.49 0.70 0.86 0.88 0.91 0.94 0.95 0.988 0.29 0.39 0.44 0.49 0.55 0.59 0.80 0.93 0.95 0.96 0.98 0.98 0.999 0.39 0.50 0.55 0.61 0.66 0.71 0.90 0.98 0.99 0.99 1.00 1.00 1.0010 0.52 0.63 0.69 0.74 0.79 0.83 1.005.7 Probability ContentIt is sometimes of interest to judge the signi�cance level of a certain outcome given thehypothesis that p = 12 . If N trials are made and we �nd k successes (let's say k < N=2 elseuse N � k instead of k) we want to estimate the probability to have k or fewer successesplus the probability for N � k or more successes. Since the assumption is that p = 12 wewant the two-tailed probability content.To calculate this either sum the individual probabilities or use the relation to the in-complete beta function. The former may seem more straightforward but the latter may be18

computationally easier given a routine for the incomplete beta function. If k = N=2 wewatch up not to add the central term twice (in this case the requested probability is 100%anyway). In the table below we show such con�dence levels in % for values of N rangingfrom 1 to 20. E.g. the probability to observe 3 successes (or failures) or less and 12 failures(or successes) or more for n = 15 is 3.52%. kN 0 1 2 3 4 5 6 7 8 9 101 100.002 50.00 100.003 25.00 100.004 12.50 62.50 100.005 6.25 37.50 100.006 3.13 21.88 68.75 100.007 1.56 12.50 45.31 100.008 0.78 7.03 28.91 72.66 100.009 0.39 3.91 17.97 50.78 100.0010 0.20 2.15 10.94 34.38 75.39 100.0011 0.10 1.17 6.54 22.66 54.88 100.0012 0.05 0.63 3.86 14.60 38.77 77.44 100.0013 0.02 0.34 2.25 9.23 26.68 58.11 100.0014 0.01 0.18 1.29 5.74 17.96 42.40 79.05 100.0015 0.01 0.10 0.74 3.52 11.85 30.18 60.72 100.0016 0.00 0.05 0.42 2.13 7.68 21.01 45.45 80.36 100.0017 0.00 0.03 0.23 1.27 4.90 14.35 33.23 62.91 100.0018 0.00 0.01 0.13 0.75 3.09 9.63 23.79 48.07 81.45 100.0019 0.00 0.01 0.07 0.44 1.92 6.36 16.71 35.93 64.76 100.0020 0.00 0.00 0.04 0.26 1.18 4.14 11.53 26.32 50.34 82.38 100.00

19

6 Binormal Distribution6.1 IntroductionAs a generalization of the normal or Gauss distribution to two dimensions we de�ne thebinormal distribution asf(x1; x2) = 12��1�2p1� �2 � e� 12(1��2)��x1��1�1 �2+� x2��2�2 �2�2��x1��1�1 �x2��2�2 �where �1 and �2 are the expectation values of x1 and x2, �1 and �2 their standard deviationsand � the correlation coe�cient between them. Putting � = 0 we see that the distributionbecomes the product of two one-dimensional Gauss distributions.

x1x2

-4 -3 -2 -1 0 1 2 3 4-4-3-2-101234

p p pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp ppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppFigure 2: Binormal distributionIn �gure 2 we show contours for a standardized Binormal distribution i.e putting �1 =�2 = 0 and �1 = �2 = 1 (these parameters are anyway shift- and scale-parameters only).In the example shown � = 0:5. Using standardized variables the contours range from aperfect circle for � = 0 to gradually thinner ellipses in the �45� direction as � ! �1.The contours shown correspond to the one, two, and three standard deviation levels. Seesection on probability content below for details.20

6.2 Conditional Probability DensityThe conditional density of the binormal distribution is given byf(xjy) = f(x; y)=f(y) == 1p2��xp1� �2 exp8<:� 12�2x(1� �2) "x� �x + ��x�y (y � �y)!#29=; == N �x + ��x�y (y � �y); �2x(1� �2)!which is seen to be a normal distribution which for � = 0 is, as expected, given by N(�x; �2x)but generally has a mean shifted from �x and a variance which is smaller than �2x.6.3 Characteristic FunctionThe characteristic function of the binormal distribution is given by�(t1; t2) = E(e{t1x1+{t2x2) = 1Z�1 1Z�1 e{t1x1+{t2x2f(x1; x2)dx1dx2 == expn{t1�1 + {t2�2 + 12 h({t1)2�21 + ({t2)2�22 + 2({t1)({t2)��1�2iowhich shows that if the correlation coe�cient � is zero then the characteristic functionfactorizes i.e. the variables are independent. This is a unique property of the normaldistribution since in general � = 0 does not imply independence.6.4 MomentsTo �nd bivariate moments of the binormal distribution the simplest, but still quite tedious,way is to use the characteristic function given above (see section 2.7.3).Algebraic bivariate moments for the binormal distribution becomes somewhat compli-cated but normally they are of less interest than the central ones. Algebraic moments ofthe type �00k and �0k0 are, of course, equal to moments of the marginal one-dimensionalnormal distribution e.g. �010 = �1, �020 = �21 + �21, and �030 = �1(2�21 + �21) (for �00k simplyexchange the subscripts on � and �). Some other lower order algebraic bivariate momentsare given by �011 = �1�2 + ��1�2�012 = 2��1�2�2 + �22�1 + �22�1�022 = �21�22 + �21�22 + �22�21 + �21�22 + 2�2�21�22 + 4��1�2�1�2Beware of the somewhat confusing notation where � with two subscripts denotes bivariatemoments while � with one subscript denotes expectation values.Lower order central bivariate moments �k`, arranged in matrix form, are given by21

` = 0 ` = 1 ` = 2 ` = 3 ` = 4k = 0 1 0 �22 0 3�42k = 1 0 ��1�2 0 3��1�32 0k = 2 �21 0 �21�22(2�2 + 1) 0 3�21�42(4�2 + 1)k = 3 0 3��31�2 0 3��31�32(2�2 + 3) 0k = 4 3�41 0 3�41�22(4�2 + 1) 0 3�41�42(8�4 + 24�2 + 3)6.5 Box-Muller TransformationRecall that if we have a distribution in one set of variables fx1; x2; :::; xng and want tochange variables to another set fy1; y2; :::; yng the distribution in the new variables aregiven by f(y1; y2; :::; yn) = ���������������������������� @x1@y1 @x1@y2 : : : @x1@yn@x2@y1 @x2@y2 : : : @x2@yn... ... . . . ...@xn@y1 @xn@y2 : : : @xn@yn ���������������������������� f(x1; x2; :::; xn)where the symbol jjJ jj denotes the absolute value of the determinant of the Jacobian J .Let x1 and x2 be two independent stochastic variables from a uniform distributionbetween zero and one and de�ney1 = q�2 lnx1 sin 2�x2y2 = q�2 lnx1 cos 2�x2Note that with the de�nition above �1 < y1 < 1 and �1 < y2 < 1. In order toobtain the joint probability density function in y1 and y2 we need to calculate the Jacobianmatrix @(x1; x2)@(y1; y2) = @x1@y1 @x1@y2@x2@y1 @x2@y2 !In order to obtain these partial derivatives we express x1 and x2 in y1 and y2 by rewritingthe original equations. y21 + y22 = �2 ln x1y1y2 = tan 2�x2which implies x1 = e� 12 (y21+y22)x2 = 12� arctan y1y2!Then the Jacobian matrix becomes@(x1; x2)@(y1; y2) = �y1e� 12 (y21+y22) �y2e� 12 (y21+y22)12�y2 cos2 arctan �y1y2 � � y12�y22 cos2 arctan �y1y2 �!22

The distribution f(y1; y2) is given byf(y1; y2) = ����������@(x1; x2)@(y1; y2) ���������� f(x1; x2)where f(x1; x2) is the uniform distribution in x1 and x2. Now f(x1; x2) = 1 in the interval0 � x1 � 1 and 0 � x2 � 1 and zero outside this region. and the absolute value of thedeterminant of the Jacobian is����������@(x1; x2)@(y1; y2) ���������� = 12�e� 12 (y21+y22) y21y22 + 1! cos2 arctan y1y2!but y21y22 + 1! cos2 arctan y1y2! = (tan2 2�x2 + 1) cos2 2�x2 = 1and thus f(y1; y2) = 12�e� 12 (y21+y22) = 1p2�e� y212 1p2�e� y222i.e. the product of two standard normal distributions.Thus the result is that y1 and y2 are distributed as two independent standard normalvariables. This is a well known method, often called the Box-Muller transformation, usedin order to achieve pseudorandom numbers from the standard normal distribution givena uniform pseudorandom number generator (see below). The method was introduced byG. E. P. Box and M. E. Muller [25].6.6 Probability ContentIn �gure 2 contours corresponding to one, two, and three standard deviations were shown.The projection on each axis for e.g. the one standard deviation contour covers the range�1 � xi � 1 and contains a probability content of 68.3% which is well known from theone-dimensional case.More generally, for a contour corresponding to z standard deviations the contour hasthe equation (x1 + x2)21 + � + (x1 � x2)21 � � = 2z2i.e. the major and minor semi-axes are zp1 + � and zp1 � �, respectively. The functionvalue at the contour is given byf(x1; x2) = 12�p1 � �2 exp(�z22 )Expressed in polar coordinates (r; �) the contour is described byr2 = z2(1� �2)1 � 2� sin � cos�23

While the projected probability contents follow the usual �gures for one-dimensionalnormal distributions the joint probability content within each ellipse is smaller. For theone, two, and three standard deviation contours the probability content, regardless of thecorrelation coe�cient �, inside the ellipse is approximately 39.3%, 86.5%, and 98.9%. Ifwe would like to �nd the ellipse with a joint probability content of 68.3% we must chosez � 1:5 (for a content of 95.5% use z � 2:5 and for 99.7% use z � 3:4). Se further discussionon probability content for a multinormal distribution in section 28.3.6.7 Random Number GenerationThe joint distribution of y1 and y2 in section 6.5 above is a binormal distribution having� = 0. For arbitrary correlation coe�cients � the binormal distribution is given byf(x1; x2) = 12��1�2p1� �2 � e� 12(1��2)��x1��1�1 �2+� x2��2�2 �2�2��x1��1�1 �x2��2�2 �where �1 and �2 are the expectation values of x1 and x2, �1 and �2 their standard deviationsand � the correlation coe�cient between them.Variables distributed according to the binormal distribution may be obtained by trans-forming the two independent numbers y1 and y2 found in the section 6.5 either asz1 = �1 + �1 �y1q1 � �2 + y2��z2 = �2 + �2y2or as z1 = �1 + �1p2 �y1q1 + �+ y2q1 � ��z2 = �2 + �2p2 �y1q1 + �� y2q1 � ��which can be proved by expressing y1 and y2 as functions of z1 and z2 and evaluatef(z1; z2) = ����������@(y1; y2)@(z1; z2) ���������� f(y1; y2) = ���������� @y1@z1 @y1@z2@y2@z1 @y2@z2 ���������� f(y1; y2)In the �rst case y1 = 1p1 � �2 �z1 � �1�1 � �z2 � �2�2 �y2 = z2 � �2�2and in the second case y1 = p22p1 + � �z1 � �1�1 + z2 � �2�2 �y2 = p22p1� � �z1 � �1�1 � z2 � �2�2 �24

In both cases the absolute value of the determinant of the Jacobian is 1=�1�2p1� �2 andwe get f(z1; z2) = 1�1�2p1� �2 � 1p2�e� y212 � 1p2�e� y222 = 12��1�2p1� �2 � e� 12 (y21+y22)Inserting the relations expressing y1 and y2 in z1 and z2 in the exponent we �nally obtainthe binormal distribution in both cases.Thus we have found methods which given two independent uniform pseudorandom num-bers between zero and one supplies us with a pair of numbers from a binormal distributionwith arbitrary means, standard deviations and correlation coe�cient.

25

7 Cauchy Distribution7.1 IntroductionThe Cauchy distribution is given byf(x) = 1� � 11 + x2and is de�ned for �1 < x < 1. It is a symmetric unimodal distribution as is shown in�gure 3.Figure 3: Graph of the Cauchy distributionThe distribution is named after the famous frenchmathematicianAugustin Louis Cauchy(1789-1857) who was a professor at �Ecole Polytechnique in Paris from 1816. He was one ofthe most productive mathematicians which have ever existed.7.2 MomentsThis probability density function is peculiar inasmuch as it has unde�ned expectation valueand all higher moments diverge. For the expectation value the integralE(x) = 1� 1Z�1 x1 + x2dxis not completely convergent, i.e. lima!1;b!1 1� bZ�a x1 + x2dx26

does not exist. However, the principal valuelima!1 1� aZ�a x1 + x2dxdoes exist and is equal to zero. Anyway the convention is to regard the expectation valueof the Cauchy distribution as unde�ned.Other measures of location and dispersion which are useful in the case of the Cauchydistribution is the median and the mode which are at x = 0 and the half-width at half-maximum which is 1 (half-maxima at x = �1).7.3 NormalizationIn spite of the somewhat awkward property of not having any moments the distribution atleast ful�l the normalization requirement for a proper probability density function i.e.N = 1Z�1 f(x)dx = 1� 1Z�1 11 + x2dx = 1� �=2Z��=2 11 + tan2 � � d�cos2 � = 1where we have made the substitution tan� = x in order to simplify the integration.7.4 Characteristic FunctionThe characteristic function for the Cauchy distribution is given by�(t) = 1Z�1 e{txf(x)dx = 1� 1Z�1 cos tx+ { sin tx1 + x2 dx == 1� 0@ 1Z0 cos tx1 + x2dx+ 0Z�1 cos tx1 + x2dx+ 1Z0 { sin tx1 + x2 dx+ 0Z�1 { sin tx1 + x2 dx1A == 2� 1Z0 cos tx1 + x2dx = e�jtjwhere we have used that the two sine integrals are equal but with opposite sign whereasthe two cosine integrals are equal. The �nal integral we have taken from standard integraltables. Note that the characteristic function has no derivatives at t = 0 once again tellingus that the distribution has no moments.7.5 Location and Scale ParametersIn the form given above the Cauchy distribution has no parameters. It is useful, however,to introduce location (x0) and scale (� > 0) parameters writingf(x;x0;�) = 1� � ��2 + (x� x0)227

where x0 is the mode of the distribution and � the half-width at half-maximum (HWHM).Including these two parameters the characteristic function is modi�ed to�(t) = eitx0��jtj7.6 Breit-Wigner DistributionIn this last form we recognize the Breit-Wigner formula, named after the two physicistsGregory Breit and Eugene Wigner, which arises in physics e.g. in the description of thecross section dependence on energy (mass) for two-body resonance scattering. Resonanceslike e.g. the �++ in �+p scattering or the � in �� scattering can be quite well describedin terms of the Cauchy distribution. This is the reason why the Cauchy distribution inphysics often is referred to as the Breit-Wigner distribution. However, in more elaboratephysics calculations the width may be energy-dependent in which case things become morecomplicated.7.7 Comparison to Other DistributionsThe Cauchy distribution is often compared to the normal (or Gaussian) distribution withmean � and standard deviation � > 0f(x;�; �) = 1�p2�e� 12(x��� )2and the double-exponential distribution with mean � and slope parameter � > 0f(x;�; �) = �2 e��jx��jThese are also examples of symmetric unimodal distributions. The Cauchy distribution haslonger tails than the double-exponential distribution which in turn has longer tails thanthe normal distribution. In �gure 4 we compare the Cauchy distribution with the standardnormal (� = 0 and � = 1) and the double-exponential distributions (� = 1) for x > 0.The normal and double-exponential distributions have well de�ned moments. Sincethey are symmetric all central moments of odd order vanish while central moments of evenorder are given by �2n = (2n)!�2n=2nn! (for n � 0) for the normal and by �n = n!=�n (foreven n) for the double-exponential distribution. E.g. the variances are �2 and 2=�2 and thefourth central moments 3�4 and 24=�4, respectively.The Cauchy distribution is related to Student's t-distribution with n degrees of freedom(with n a positive integer)f(t;n) = � �n+12 �pn�� �n2� 1 + t2n!�n+12 = �1 + t2n ��n+12pnB �12 ; n2�where �(x) is the Euler gamma-function not no be mixed up with the width parameter forthe Cauchy distribution used elsewhere in this section. B is the beta-function de�ned in28

Figure 4: Comparison between the Cauchy distribution, the standard normal distribution,and the double-exponential distributionterms of the �-function as B(p; q) = �(p)�(q)�(p+q) . As can be seen the Cauchy distribution arisesas the special case where n = 1. If we change variable to x = t=pn and put m = n+12 theStudent's t-distribution becomesf(x;m) = k(1 + x2)m with k = �(m)� � 12�� �m� 12� = 1B �m� 12 ; 12�where k is simply a normalization constant. Here it is easier to see the more general formof this distribution which for m = 1 gives the Cauchy distribution. The requirement n � 1corresponds to m being a half-integer � 1 but we could even allow for m being a realnumber.As for the Cauchy distribution the Student's t-distribution have problems with divergentmoments and moments of order � n does not exist. Below this limit odd central momentsare zero (the distribution is symmetric) and even central moments are given by�2r = nr� �r + 12�� �n2 � r�� � 12�� �n2� = nrB �r + 12 ; n2 � r�B �12 ; n2�for r a positive integer (2r < n). More speci�cally the expectation value is E(t) = 0, thevariance V (t) = nn�2 and the fourth central moment is given by �4 = 3n2(n�2)(n�4) when theyexist. As n!1 the Student's t-distribution approaches a standard normal distribution.7.8 TruncationIn order to avoid the long tails of the distribution one sometimes introduces a truncation.This, of course, also cures the problem with the unde�ned mean and divergent higher29

moments. For a symmetric truncation �X � x � X we obtain the renormalized probabilitydensity function f(x) = 12 arctanX � 11 + x2which has expectation value E(x) = 0, variance V (x) = XarctanX � 1, third central moment�3 = 0 and fourth central moment �4 = XarctanX �X23 � 1� + 1. The fraction of the originalCauchy distribution within the symmetric interval is f = 2� arctanX. We will, however,not make any truncation of the Cauchy distribution in the considerations made in thisnote.7.9 Sum and Average of Cauchy VariablesIn most cases one would expect the sum and average of many variables drawn from thesame population to approach a normal distribution. This follows from the famous CentralLimit Theorem. However, due to the divergent variance of the Cauchy distribution therequirements for this theorem to hold is not ful�lled and thus this is not the case here. Wede�ne Sn = nXi=1 xi and Sn = 1nSnwith xi independent variables from a Cauchy distribution.The characteristic function of a sum of independent random variables is equal to theproduct of the individual characteristic functions and hence�(t) = �(t)n = e�njtjfor Sn. Turning this into a probability density function we get (putting x=Sn for conve-nience)f(x) = 12� 1Z�1 �(t)e�{xtdt = 12� 1Z�1 e�({xt+njtj)dt = 12� 0@ 0Z�1 ent�{xtdt+ 1Z0 e�{xt�ntdt1A == 12� 0@"et(n�{x)n � {x #0�1 + "e�t({x+n)�n� {x#10 1A = 12� � 1n� {x + 1n+ {x� = 1� � nn2 + x2This we recognize as a Cauchy distribution with scale parameter �=n and thus for eachadditional Cauchy variable the HWHM increases by one unit.Moreover, the probability density function of Sn is given byf(Sn) = dSndSn f(Sn) = 1� � 11 + S2ni.e. the somewhat amazing result is that the average of any number of independent randomvariables from a Cauchy distribution is also distributed according to the Cauchy distribu-tion. 30

7.10 Estimation of the MedianFor the Cauchy distribution the sample mean is not a consistent estimator of the medianof the distribution. In fact, as we saw in the previous section, the sample mean is itselfdistributed according to the Cauchy distribution and therefore has divergent variance.However, the sample median for a sample of n independent observations from a Cauchydistribution is a consistent estimator of the true median.In the table below we give the expectations and variances of the sample mean andsample median estimators for the normal, double-exponential and Cauchy distributions(see above for de�nitions of distributions). Sorting all the observations the median is takenas the value for the central observation for odd n and as the average of the two centralvalues for even n. The variance of the sample mean is simply the variance of the distributiondivided by the sample size n. For large n the variance of the sample median m is given byV (m) = 1=4nf2 where f is the function value at the median.Distribution E(x) V (x) E(m) V (m)Normal � �2n � ��22nDouble-exponential � 2n�2 � 1n�2Cauchy undef. 1 x0 �2�24nFor a normal distribution the sample mean is superior to the median as an estimator ofthe mean (i.e. it has the smaller variance). However, the double-exponential distributionis an example of a distribution where the sample median is the best estimator of the meanof the distribution. In the case of the Cauchy distribution only the median works of theabove alternatives but even better is a proper Maximum Likelihood estimator. In the caseof the normal and double-exponential the mean and median, respectively, are identical tothe maximum likelihood estimators but for the Cauchy distribution such an estimator maynot be expressed in a simple way.The large n approximation for the variance of the sample median gives conservativeestimates for lower values of n in the case of the normal distribution. Beware, however,that for the Cauchy and the double-exponential distributions it is not conservative butgives too small values. Calculating the standard deviation this is within 10% of the truevalue already at n = 5 for the normal distribution whereas for the Cauchy distribution thisis true at about n = 20 and for the double-exponential distribution only at about n = 60.7.11 Estimation of the HWHMTo �nd an estimator for the half-width at half-maximum is not trivial. It implies binningthe data, �nding the maximum and then locating the positions where the curve is at half-31

maximum. Often it is preferable to �t the probability density function to the observationsin such a case.However, it turns out that another measure of dispersion the so called semi-interquar-tile range can be used as an estimator. The semi-interquartile range is de�ned as half thedi�erence between the upper and the lower quartiles. The quartiles are the values whichdivide the probability density function into four parts with equal probability content, i.e.25% each. The second quartile is thus identical to the median. The de�nition is thusS = 12(Q3 �Q1) = �where Q1 is the lower and Q3 the upper quartile which for the Cauchy distribution is equalto x0 � � and x0 + �, respectively. As is seen S = HWHM = � and thus this estimatormay be used in order to estimate �.We gave above the large n approximation for the variance of the median. The medianand the quartiles are examples of the more general concept quantiles. Generally the largen approximation for the variance of a quantile Q is given by V (Q) = pq=nf2 where f is theordinate at the quantile and p and q = 1� p are the probability contents above and belowthe quantile, respectively. The covariance between two quantilesQ1 and Q2 is, with similarnotations, given by Cov(Q1;Q2) = p2q1=nf1f2 where Q1 should be the leftmost quantile.For large n the variance of the semi-interquartile range for a sample of size n is thusfound by error propagation inserting the formul� aboveV (S) = 14 (V (Q1) + V (Q3) � 2Cov(Q1; Q3)) = 164n 3f21 + 3f23 � 2f1f3! = 116nf21 = �2�24nwhere f1 and f3 are the function values at the lower and upper quartile which are bothequal to 1=2��. This turns out to be exactly the same as the variance we found for themedian in the previous section.After sorting the sample the quartiles are determined by extrapolation between the twoobservations closest to the quartile. In the case where n+2 is a multiple of 4 i.e. the seriesn = 2; 6; 10::: the lower quartile is exactly at the n+24 :th observation and the upper quartileat the 3n+24 :th observation. In the table below we give the expectations and variances ofthe estimator of S as well as the variance estimator s2 for the normal, double-exponentialand Cauchy distributions. The variance estimator s2 and its variance are given bys2 = 1n� 1 nXi=1(xi � x)2 and V (s2) = �4 � �22n + 2�22n(n� 1)with �2 and �4 the second and fourth central moments. The expectation value of s2 isequal to the variance and thus it is a unbiased estimator.32

Distribution HWHM E(s2) V (s2) E(S) V (S)Normal �p2 ln 2 �2 2�4n�1 0:6745� 116nf(Q1)2Double-exponential ln 2� 2�2 20n�4� ln 2� 1n�2Cauchy � 1 1 � �2�24nIn this table � = 1 + 0:4n�1 if we include the second term in the expression of V (s2) aboveand � = 1 otherwise. It can be seen that the double-exponential distribution also hasHWHM = S but for the normal distribution HWHM � 1:1774� as compared to S �0:6745�.For the three distributions tested the semi-interquartile range estimator is biased. Inthe case of the normal distribution the values are approaching the true value from belowwhile for the Cauchy and double-exponential distributions from above. The large n ap-proximation for V (S) is conservative for the normal and double-exponential distributionbut not conservative for the Cauchy distribution. In the latter case the standard deviationof S is within 10% of the true value for n > 50 but for small values of n it is substantiallylarger than given by the formula. The estimated value for � is less than 10% too big forn > 25.7.12 Random Number GenerationIn order to generate pseudorandom numbers from a Cauchy distribution we may solve theequation F (x) = � where F (x) is the cumulative distribution function and � is a uniformpseudorandom number between 0 and 1. This means solving for x in the equationF (x) = �� xZ�1 1�2 + (t� x0)2dt = �If we make the substitution tan � = (t� x0)=� using that d�= cos2 � = dt=� we obtain1� arctan�x� x0� �+ 12 = �which �nally gives x = x0 + � tan �� �� � 12��as a pseudorandom number from a Cauchy distribution. One may easily see that it isequivalent to use x = x0 + � tan(2��)which is a somewhat simpler expression. 33

An alternative method (see also below) to achieve random numbers from a Cauchydistribution would be to use x = x0 + �z1z2where z1 and z2 are two independent random numbers from a standard normal distribution.However, if the standard normal random numbers are achieved through the Box-Mullertransformation then z1=z2 = tan 2�� and we are back to the previous method.In generating pseudorandom numbers one may, if pro�table, avoid the tangent bya Generate in u and v two random numbers from a uniform distribution between -1and 1.b If u2 + v2 > 1 (outside circle with radius one in uv-plane) go back to a.c Obtain x = x0 + �uv as a random number from a Cauchy distribution.7.13 Physical PictureA physical picture giving rise to the Cauchy distribution is as follows: Regard a plane inwhich there is a point source which emits particles isotropically in the plane (either in thefull 2� region or in one hemisphere � radians wide). The source is at the x-coordinate x0and the particles are detected in a detector extending along a line � length units from thesource. This scenario is depicted in �gure 5� -?��� @@@R -@@@@@@@@@@@@ x-axisSOURCEu

DETECTOR AXIS� = �2� = ��2� = ��4 � = �4�� = 0 x0 x�Figure 5: Physical scenario leading to a Cauchy distributionThe distribution in the variable x along the detector will then follow the Cauchy distri-bution. As can be seen by pure geometrical considerations this is in accordance with theresult above where pseudorandom numbers from a Cauchy distribution could be obtainedby x = x0 + � tan �, i.e. tan � = x�x0� , with � uniformly distributed between ��2 and �2 .34

To prove this let us start with the distribution in �f(�) = 1� for � �2 � � � �2To change variables from � to x requires the derivative d�=dx which is given byd�dx = cos2 �� = 1� cos2 arctan�x� x0� �Note that the interval from ��2 to �2 in � maps onto the interval �1 < x <1. We getf(x) = �����d�dx ����� f(�) = 1�� cos2 arctan�x� x0� � = 1�� cos2 � == 1�� � �2�2 + (x� x0)2 = 1� � ��2 + (x� x0)2i.e. the Cauchy distribution.It is just as easy to make the proof in the reversed direction, i.e. given a Cauchydistribution in x one may show that the �-distribution is uniform between ��2 and �2 .7.14 Ratio Between Two Standard Normal VariablesAs mentioned above the Cauchy distribution also arises if we take the ratio between twostandard normal variables z1 and z2, viz.x = x0 + �z1z2 :In order to deduce the distribution in x we �rst introduce a dummy variable y which wesimply take as z2 itself. We then make a change of variables from z1 and z2 to x and y.The transformation is given by x = x0 + �z1z2y = z2or if we express z1 and z2 in x and yz1 = y(x� x0)=�z2 = yThe distribution in x and y is given byf(x; y) = ����������@(z1; z2)@(x; y) ���������� f(z1; z2)where the absolute value of the determinant of the Jacobian is equal to y=� and f(z1; z2)is the product of two independent standard normal distributions. We getf(x; y) = y� � 12�e� 12 (z21+z22) = y2��e� 12� y2(x�x0)2�2 +y2�35

In order to obtain the marginal distribution in x we integrate over yf(x) = 1Z�1 f(x; y)dy = 12�� 1Z�1 ye��y2dywhere we have put � = 12 �x� x0� �2 + 1!for convenience. If we make the substitution z = y2 we getf(x) = 2 12�� 1Z0 e��z dz2 = 12���Note that the �rst factor of 2 comes from the fact that the region �1 < y < 1 mapstwice onto the region 0 < z <1. Finallyf(x) = 12��� = 12�� � 2�x�x0� �2 + 1 = 1� � �(x� x0)2 + �2i.e. a Cauchy distribution.

36

8 Chi-square Distribution8.1 IntroductionThe chi-square distribution is given byf(x;n) = �x2�n2�1 e�x22� �n2�where the variable x � 0 and the parameter n, the number of degrees of freedom, is apositive integer. In �gure 6 the distribution is shown for n-values of 1, 2, 5 and 10. Forn � 2 the distribution has a maximum at n�2.Figure 6: Graph of chi-square distribution for some values of n8.2 MomentsAlgebraic moments of order k are given by�0k = E(xk) = 12� �n2� 1Z0 xk �xn�n2�1 e�x22 dx = 2k� �n2� 1Z0 y n2�1+ke�ydy = 2k� �n2 + k�� �n2� == 2k � n2 (n2 + 1) � � � (n2 + k � 2)(n2 + k � 1) = n(n+ 2)(n + 4) � � � (n+ 2k � 2)e.g. the �rst algebraic moment which is the expectation value is equal to n. A recursiveformula to calculate algebraic moments is thus given by�0k = �0k�1 � (n+ 2k � 2)37

where we may start with �00 = 1 to �nd the expectation value �01 = n, �02 = n(n+ 1) etc.From this we may calculate the central moments which for the lowest orders become�2 = 2n; �3 = 8n; �4 = 12n(n + 4); �5 = 32n(5n + 12) and �6 = 40n(3n2 + 52n + 96).The coe�cients of skewness and kurtosis thus becomes 1 = 2q2=n and 2 = 12=n.The fact that the expectation value of a chi-square distribution equals the number ofdegrees of freedom has led to a bad habit to give the ratio between a found chi-squarevalue and the number of degrees of freedom. This is, however, not a very good variableand it may be misleading. We strongly recommend that one always should give both thechi-square value and degrees of freedom e.g. as �2/n.d.f.=9.7/5.To judge the quality of the �t we want a better measure. Since the exact samplingdistribution is known one should stick to the chi-square probability as calculated from anintegral of the tail i.e. given a speci�c chi-square value for a certain number of degrees offreedom we integrate from this value to in�nity (see below).As an illustration we show in �gure 7 the chi-square probability for constant ratios of�2=n.d.f.Figure 7: Chi-square probability for constant ratios of �2=n.d.f.Note e.g. that for few degrees of freedom we may have an acceptable chi-square valueeven for larger ratios. 38

8.3 Characteristic FunctionThe characteristic function for a chi-square distribution with n degrees of freedom is givenby�(t) = E(e{tx) = 12� �n2� 1Z0 �x2�n2�1 e�( 12�{t)xdx = 12� �n2� 1Z0 � y1� 2{t�n2�1 e�y dy12 � {t == 1� �n2� (1� 2{t)n2 1Z0 y n2�1e�ydy = (1� 2{t)�n28.4 Cumulative FunctionThe cumulative, or distribution, function for a chi-square distribution with n degrees offreedom is given byF (x) = 12� �n2� xZ0 �x2�n2�1 e�x2 dx = 12� �n2� x2Z0 y n2�1e�y2dy == �n2 ; x2�� �n2� = P �n2 ; x2�where P �n2 ; x2� is the incomplete Gamma function (see section 42.5). In this calculationwe have made the simple substitution y = x=2 in simplifying the integral.8.5 Origin of the Chi-square DistributionIf z1; z2; :::; zn are n independent standard normal random variables then nPi=1 z2i is distributedas a chi-square variable with n degrees of freedom.In order to prove this �rst regard the characteristic function for the square of a standardnormal variableE(e{tz2) = 1p2� 1Z�1 e� z22 (1�2{t)dz = 1p2� 1Z�1 e� y22 dyp1� 2{t = 1p1� 2{twhere we made the substitution y = zp1� 2{t.For a sum of n such independent variables the characteristic function is then given by�(t) = (1 � 2it)�n2which we recognize as the characteristic function for a chi-square distribution with n degreesof freedom.This property implies that if x and y are independently distributed according to the chi-square distribution with n and m degrees of freedom, respectively, then x+y is distributedas a chi-square variable with m+ n degrees of freedom.39

Indeed the requirement that all z's come from a standard normal distribution is morethan what is needed. The result is the same if all observations xi come from di�erent normalpopulations with means �i and variance �2i if we in each case calculate a standardizedvariable by subtracting the mean and dividing with the standard deviation i.e. takingzi = (xi � �i)=�i.8.6 ApproximationsFor large number of degrees of freedom n the chi-square distribution may be approximatedby a normal distribution. There are at least three di�erent approximations. Firstly wemay na��vely construct a standardized variablez1 = x� E(x)qV (x) = x� np2nwhich would tend to normality as n increases. Secondly an approximation, due to R. A. Fisher,is that the quantity z2 = p2x�p2n � 1approaches a standard normal distribution faster than the standardized variable. Thirdlya transformation, due to E. B. Wilson and M. M. Hilferty, is that the cubic root of x=n isclosely distributed as a standard normal distribution usingz3 = �xn� 13 � �1� 29n�q 29nThe second approximation is probably the most well known but the latter is approachingnormality even faster. In fact there are even correction factors which may be applied to z3to give an even more accurate approximation (see e.g. [26])z4 = z3 + hn = z3 + 60n h60with h60 given for values of z2 from {3.5 to 3.5 in steps of 0.5 (in this order the values ofh60 are {0.0118, {0.0067, {0.0033, {0.0010, 0.0001, 0.0006, 0.0006, 0.0002, {0.0003, {0.0006,{0.0005, 0.0002, 0.0017, 0.0043, and 0.0082).To compare the quality of all these approximations we calculate the maximumdeviationbetween the cumulative function for the true chi-square distribution and each of theseapproximations for n=30 and n=100. The results are shown in the table below. Normallyone accepts z2 for n > 100 while z3, and certainly z4, are even better already for n > 30.Approximation n = 30 n = 100z1 0.034 0.019z2 0.0085 0.0047z3 0.00039 0.00011z4 0.000044 0.00003540

8.7 Random Number GenerationAs we saw above the sum of n independent standard normal random variables gave achi-square distribution with n degrees of freedom. This may be used as a technique toproduce pseudorandom numbers from a chi-square distribution. This required a generatorfor standard normal random numbers and may be quite slow. However, if we make use ofthe Box-Muller transformation in order to obtain the standard normal random numberswe may simplify the calculations.First we recall the Box-Muller transformation which given two pseudorandom numbersuniformly distributed between zero and one through the transformationz1 = q�2 ln �1 cos 2��2z2 = q�2 ln �1 sin 2��2gives, in z1 and z2, two independent pseudorandom numbers from a standard normal dis-tribution.Adding n such squared random numbers implies thaty2k = �2 ln(�1 � �2 � � � �k)y2k+1 = �2 ln(�1 � �2 � � � �k)� 2 ln �k+1 cos2 2��k+2for k a positive integer will be distributed as chi-square variable with even or odd numberof degrees of freedom. In this manner a lot of unnecessary operations are avoided.Since the chi-square distribution is a special case of the Gamma distribution we mayalso use a generator for this distribution.8.8 Con�dence Intervals for the VarianceIf x1; x2; :::; xn are independent normal random variables from a N(�; �2) distribution then(n�1)s2�2 is distributed according to the chi-square distribution with n�1 degrees of freedom.A 1� � con�dence interval for the variance is then given by(n � 1)s2�21��=2;n�1 � �2 � (n� 1)s2�2�=2;n�1where ��;n is the chi-square value for a distribution with n degrees of freedom for which theprobability to be greater or equal to this value is given by �. See also below for calculationsof the probability content of the chi-square distribution.8.9 Hypothesis TestingLet x1; x2; :::; xn be n independent normal random variables distributed according to aN(�; �2) distribution. To test the null hypothesis H0: �2 = �20 versus H1: �2 6= �20 at the �level of signi�cance, we would reject the null hypothesis if (n�1)s2=�20 is less than �2�=2;n�1or greater than �21��=2;n�1. 41

8.10 Probability ContentIn testing hypotheses using the chi-square distribution we de�ne x� = �2�;n fromF (x�) = x�Z0 f(x;n)dx = 1� �i.e. � is the probability that a variable distributed according to the chi-square distributionwith n degrees of freedom exceeds x�.This formula can be used in order to determine con�dence levels for certain values of �.This is what is done in producing the tables which is common in all statistics text-books.However, more often the equation is used in order to calculate the con�dence level � givenan experimentally determined chi-square value x�.In calculating the probability content of a chi-square distribution we di�er between thecase with even and odd number of degrees of freedom. This is described in the two followingsubsections.Note that one may argue that it is as unlikely to obtain a very small chi-square valueas a very big one. It is customary, however, to use only the upper tail in calculation ofsigni�cance levels. A too small chi-square value is regarded as not a big problem. However,in such a case one should be somewhat critical since it indicates that one either is cheating,are using selected (biased) data or has (undeliberately) overestimated measurement errors(e.g. included systematic errors).To proceed in calculating the cumulative function we write1 � � = F (x�) = 12� �n2� x�Z0 �x2�n2�1 e�x2 dx = 1� �n2� x�=2Z0 z n2�1e�zdz = P �n2 ; x�2 �where we have made the substitution z = x=2. From this we see that we may use theincomplete Gamma function P (see section 42.5) in evaluating probability contents but forhistorical reasons we have solved the problem by considering the cases with even and odddegrees of freedom separately as is shown in the next two subsections.Although we prefer exact routines to calculate the probability in each speci�c case aclassical table may sometimes be useful. In table 1 on page 173 we show percentage points,i.e. points where the cumulative probability is 1��, for di�erent degrees of freedom.It is sometimes of interest e.g. when rejecting a hypothesis using a chi-square test toscrutinize extremely small con�dence levels. In table 2 on page 174 we show this forcon�dence levels down to 10�12 as chi-square values. In table 3 on page 175 we show thesame thing in terms of chi-square over degrees of freedom ratios (reluctantly since we donot like such ratios). As discussed in section 8.2 we see, perhaps even more clearly, thatfor few degrees of freedom the ratios may be very high while for large number of degreesof freedom this is not the case for the same con�dence level.42

8.11 Even Number of Degrees of FreedomWith even n the power of z in the last integral in the formula for F (x�) above is an integer.From standard integral tables we �ndZ xmeaxdx = eax mXr=0(�1)r m!xm�r(m� r)!ar+1where, in our case, a = �1. Putting m = n2 � 1 and using this integral we obtain1 � � = 1� �n2� x�=2Z0 z n2�1e�zdz = " 1m!e�z mXr=0(�1)r m!zm�r(m� r)!(�1)r+1#x�20= 1� e�x�2 mXr=0 xm�r�2m�r(m� r)! = 1 � e�x�2 n2�1Xr=0 xr�2rr!a result which indeed is identical to the formula for P (n; x) for integer n given on page 162.8.12 Odd Number of Degrees of FreedomIn the case of odd number of degrees of freedom we make the substitution z2 = x yielding1 � � = F (x�) = 12� �n2� x�Z0 �x2�n2�1 e�x2 dx = 12n2 � �n2� px�Z0 �z2�n2�1 e� z22 2zdz == 12n2�1� �n2� px�Z0 �z2�n�12 e� z22 dz = 12m� 12� �m+ 12� px�Z0 z2me� z22 dzwhere we have put m = n�12 which for odd n is an integer. By partial integration in msteps Z z2me� z22 dz = Z z2m�1ze� z22 dz = �z2m�1e� z22 + (2m� 1) Z z2m�2e� z22 dzZ z2m�2e� z22 dz = �z2m�3e� z22 + (2m� 3) Z z2m�4e� z22 dz...Z z4e� z22 dz = �z3e� z22 + 3 Z z2e� z22 dzZ z2e� z22 dz = �ze� z22 + Z e� z22 dzwe obtain Z z2me� z22 = (2m� 1)!! Z e� z22 dz � m�1Xr=0 (2m� 1)!!(2r + 1)!! z2r+1e� z2243

Applying this to our case gives1 � � = 12m� 12� �m+ 12� 0B@(2m � 1)!! px�Z0 e� z22 dz � "m�1Xr=0 (2m� 1)!!(2r + 1)!! z2r+1e� z22 #px�0 1CA == s2� 0B@ px�Z0 e� z22 dz � "m�1Xr=0 1(2r + 1)!!z2r+1e� z22 #px�0 1CA == 2G(px�)� 1 �s2x�� e�x�2 m�1Xr=0 xr�(2r + 1)!!where G(z) is the integral of the standard normal distribution from �1 to z. Here wehave used � �m+ 12� = (2m�1)!!2m p� in order to simplify the coe�cient. This result may becompared to the formula given on page 163 for the incomplete Gamma function when the�rst argument is a half-integer.8.13 Final AlgorithmThe �nal algorithm to evaluate the probability content from �1 to x for a chi-squaredistribution with n degrees of freedom is� For n even:� Put m = n2 � 1.� Set u0 = 1; s = 0 and i = 0.� For i = 0; 1; :::;m set s = s + ui; i = i+ 1 and ui = ui�1 � x2i .� � = s � e�x2 .� For n odd:� Put m = n�12 .� Set u0 = 1; s = 0 and i = 0.� For i = 0; 1; :::;m� 1 set s = s+ ui; i = i+ 1 and ui = ui�1 � x2i+1 .� � = 2 � 2G(px) +q2x� e�x2 � s.8.14 Chi DistributionSometimes, but less often, the chi distribution i.e. the distribution of y = px is used. Bya simple change of variables this distribution is given byf(y) = �����dxdy ����� f(y2) = 2y � 12 y22 !n2�1 e� y22� �n2� = yn�1 �12�n2�1 e� y22� �n2�44

Figure 8: Graph of chi distribution for some values of nIn �gure 8 the chi distribution is shown for n-values of 1, 2, 5, and 10. The mode of thedistribution is at pn� 1.The cumulative function for the chi distribution becomesF (y) = �12�n2�1� �n2� yZ0 xn�1e�x22 dx = P n2 ; y22 !and algebraic moments are given by�0k = �12�n2�1� �n2� 1Z0 ykyn�1e� y22 dy = 2 k2� �n2 + k2�� �n2�45

9 Compound Poisson Distribution9.1 IntroductionThe compound Poisson distribution describes the branching process for Poisson variablesand is given by p(r;�; �) = 1Xn=0 (n�)re�n�r! �ne��n!where the integer variable r � 0 and the parameters � and � are positive real quantities.9.2 Branching ProcessThe distribution describes the branching of n Poisson variables ni all with mean � wheren is also distributed according to the Poisson distribution with mean � i.e.r = nXi=1 ni with p(ni) = �nie��ni! and p(n) = �ne��n!and thus p(r) = 1Xn=0 p(rjn)p(n)Due to the so called addition theorem (see page 122) for Poisson variables with mean � thesum of n such variables are distributed as a Poisson variable with mean n� and thus thedistribution given above results.9.3 MomentsThe expectation value and variance of the Compound Poisson distribution are given byE(r) = �� and V (r) = ��(1 + �)while higher moments gets slightly more complicated:�3 = ��n�+ (� + 1)2o�4 = ��n�3 + 6�2 + 7�+ 1 + 3��(1 + �)2o�5 = ��n�4 + 10�3 + 25�2 + 15� + 1 + 10��(� + 1)(�+ (1 + �)2)o�6 = ��n�5 + 15�4 + 65�3 + 90�2 + 31� + 1+ 5�� �5�4 + 33�3 + 61�2 + 36� + 5�+ 15�2�2(� + 1)3o46

9.4 Probability Generating FunctionThe probability generating function of the compound Poisson distribution is given byG(z) = expn��+ �e��+�zoThis is easily found by using the rules for branching processes where the probabilitygenerating function (p.g.f.) is given byG(z) = GP (GP (z))where GP (z) is the p.g.f. for the Poisson distribution.9.5 Random Number GenerationUsing the basic de�nition we may proceed by �rst generate a random number n from aPoisson distribution with mean � and then another one with mean n�.For �xed � and � it is, however, normally much faster to prepare a cumulative vectorfor values ranging from zero up to the point where computer precision gives unity and thenuse this vector for random number generation. Using a binary search technique one mayallow for quite long vectors giving good precision without much loss in e�ciency.

47

10 Double-Exponential Distribution10.1 IntroductionThe Double-exponential distribution is given byf(x;�; �) = �2 e��jx��jwhere the variable x is a real number as is the location parameter � while the parameter� is a real positive number.The distribution is sometimes called the Laplace distribution after the french astronomer,mathematician and physicist marquis Pierre Simon de Laplace (1749{1827). It is a sym-metric distribution whose tails fall o� less sharply than the Gaussian distribution but fasterthan the Cauchy distribution. It has a cusp, discontinuous �rst derivative, at x = �.The distribution has an interesting feature inasmuch as the best estimator for the mean� is the median and not the sample mean. See further the discussion in section 7 on theCauchy distribution where the Double-exponential distribution is discussed in some detail.10.2 MomentsFor the Double-exponential distribution central moments are more easy to determine thanalgebraic moments (the mean is �01 = �). They are given by�n = 1Z�1 (x� �)nf(x)dx = �2 8<: �Z�1 (x� �)ne��(��x) + 1Z� (x� �)ne��(x��)9=; == 12 8<: 0Z�1 ��y��n e�ydy + 1Z0 �y��n e�ydy9=; = n!2�n + (�1)n n!2�ni.e. odd moments vanish as they should due to the symmetry of the distribution and evenmoments are given by the simple relation �n = n!=�n. From this one easily �nds that thecoe�cient of skewness is zero and the coe�cient of kurtosis 3.If required algebraic moments may be calculated from the central moments especiallythe lowest order algebraic moments become�01 = �; �02 = 2�2 + �2; �03 = 6��2 + �3; and �04 = 24�4 + 12�2�2 + �4but more generally �0n = nXr=0 nr!�r�n�r10.3 Characteristic FunctionThe characteristic function which generates central moments is given by�x��(t) = �2�2 + t248

from which we may �nd the characteristic function which generates algebraic moments�x(t) = E(e{tx) = e{t�E(e{t(x��)) = e{t��x��(t) = e{t� �2�2 + t2Sometimes an alternative which generates the sequence �01; �2; �3; : : : is given as�(t) = {t�+ �2�2 + t210.4 Cumulative FunctionThe cumulative function, or distribution function, for the Double-exponential distributionis given by F (x) = ( 12e��(��x) if x � �1� 12e��(x��) if x > �From this we see not only the obvious that the median is at x = � but also that the lowerand upper quartile is located at �� ln 2=�.10.5 Random Number GenerationGiven a uniform random number between zero and one in � a random number from aDouble-exponential distribution is given by solving the equation F (x) = � for x givingFor � � 12 x = �+ ln(2�)=�for � > 12 x = �� ln(2� 2�)=�

49

11 Doubly Non-Central F -Distribution11.1 IntroductionIf x1 and x2 are independently distributed according to two non-central chi-square distribu-tions with n1 and n2 degrees of freedom and non-central parameters �1 and �2, respectively,then the variable F 0 = x1=n1x2=n2is said to have a doubly non-central F -distribution with n1; n2 degrees of freedom (positiveintegers) and non-centrality parameters �1; �2 (both � 0).This distribution may be written asf(x;n1; n2; �1; �2) = n1n2 e��2 1Xr=0 1Xs=0 ��12 �rr! ��22 �ss! �n1xn2 �n12 +r�1�1 + n1xn2 �n2+r+s 1B �n12 + r; n22 + s�where we have put n = n1 + n2 and � = �1 + �2. For �2 = 0 we obtain the (singly)non-central F -distribution (see section 32) and if also �1 = 0 we are back to the ordinaryvariance ratio, or F -, distribution (see section 16).With four parameters a variety of shapes are possible. As an example �gure 9 shows thedoubly non-central F -distribution for the case with n1 = 10; n2 = 5 and �1 = 10 varying�2 from zero (an ordinary non-central F -distribution) to �ve.Figure 9: Examples of doubly non-central F -distributions11.2 MomentsAlgebraic moments of this distributions becomeE(xk) = �n2n1�k e��2 1Xr=0 ��12 �rr! � �n12 + r + k�� �n12 + r� 1Xs=0 ��22 �ss! � �n22 + s� k�� �n22 + s� =50

= �n2n1�k e��2 1Xr=0 ��12 �rr! �n12 + r + k � 1� � � � �n12 + r� �� 1Xs=0 ��22 �ss! �n22 + s � 1��1 � � � �n22 + s� k��1The r-sum involved, with a polynomial in the numerator, is quite easily solvable giv-ing similar expressions as for the (singly) non-central F -distribution. The s-sums, how-ever, with polynomials in the denominator give rise to con uent hypergeometric functionsM(a; b;x) (see appendix B). Lower order algebraic moments are given byE(x) = e��22 nm � m+ �n� 2M �n�22 ; n2 ; �22 �E(x2) = e��22 � nm�2 �2 + (2� +m)(m+ 2)(n� 2)(n � 4) M �n�42 ; n2 ; �22 �E(x3) = e��22 � nm�3 �3 + 3(m + 4)�2 + (3� +m)(m+ 2)(m+ 4)(n� 2)(n� 4)(n� 6) M �n�62 ; n2 ; �22 �E(x4) = e��22 � nm�4 �4 + (m+ 6) f4�3 + (m+ 4) [6�2 + (4� +m)(m+ 2)]g(n � 2)(n � 4)(n� 6)(n � 8) ��M �n�82 ; n2 ; �22 �11.3 Cumulative DistributionThe cumulative, or distribution, function may be deduced by simple integrationF (x) = n1n2 e��2 1Xr=0 1Xs=0 ��12 �rr! ��22 �ss! 1B �n12 + r; n22 + s� xZ0 �n1un2 �n12 +r�1�1 + n1un2 �n2+r+s du == e��2 1Xr=0 1Xs=0 ��12 �rr! ��22 �ss! Bq �n12 + r; n22 + s�B �n12 + r; n22 + s� == e��2 1Xr=0 1Xs=0 ��12 �rr! ��22 �ss! Iq �n12 + r; n22 + s�with q = n1xn21 + n1xn211.4 Random Number GenerationRandom numbers from a doubly non-central F -distribution is easily obtained using thede�nition in terms of the ratio between two independent random numbers from non-centralchi-square distributions. This ought to be su�cient for most applications but if neededmoree�cient techniques may easily be developed e.g. using more general techniques.51

12 Doubly Non-Central t-Distribution12.1 IntroductionIf x and y are independent and x is normally distributed with mean � and unit variancewhile y is distributed according a non-central chi-square distribution with n degrees offreedom and non-centrality parameter � then the variablet = x=qy=nis said to have a doubly non-central t-distribution with n degrees of freedom (positiveinteger) and non-centrality parameters � and � (with � � 0).This distribution may be expressed asf(t;n; �; �) = e� �22 e��2pn� 1Xr=0 ��2�rr! 1� �n2 + r� 1Xs=0 (t�)ss! �n2� s2 1 + t2n !�(n+s+12 +r) � �n+s+12 + r�For � = 0 we obtain the (singly) non-central t-distribution (see section 33) and if also � = 0we are back to the ordinary t-distribution (see section 38).Examples of doubly non-central t-distributions are shown in �gure 9 for the case withn = 10 and �2 = 5 varying � from zero (an ordinary non-central t-distribution) to ten.Figure 10: Examples of doubly non-central t-distributions12.2 MomentsAlgebraic moments may be deduced from the expressionE(tk) = e� �22 e��2pn� 1Xr=0 ��2�rr! 1� �n2 + r� 1Xs=0 �ss!n s2 2 s2� �n+s+12 + r� 1Z�1 ts+k�1 + t2n �n+s+12 +r dt == e� �22 e��2p� 1Xr=0 ��2�rr! 1� �n2 + r� 1Xs=0 �ss! 2 s2n k2� � s+k+12 �� �n�k2 + r�52

where the sum should be taken for even values of s+ k i.e. for even (odd) orders sum onlyover even (odd) s-values.Di�ering between moments of even and odd order the following expressions for lowerorder algebraic moments of the doubly non-central t-distribution may be expressed in termsof the con uent hypergeometric function M(a; b;x) (see appendix B for details) asE(t) = �rn2 e��2 � �n�12 �� �n2� M �n�12 ; n2 ; �2�E(t2) = ne��2 (�2 + 1)n� 2 M �n�22 ; n2 ; �2�E(t3) = �(�2+ 3)sn38 e��2 � �n�32 �� �n2� M �n�32 ; n2 ; �2�E(t4) = n2(n� 2)(n� 4) ��4 + 6�2 + 3� e��2M �n�42 ; n2 ; �2�12.3 Cumulative DistributionThe cumulative, distribution, function is given byF (t) = e� �22 e��2pn� 1Xr=0 ��2�rr! 1� �n2 + r� 1Xs=0 �ss! �n2� s2 � �n+s+12 + r� tZ�1 us�1 + u2n �n+s+12 +r du == e� �22 e��2p� 1Xr=0 ��2�rr! 1Xs=0 �ss! 2 s2�1� � s+12 � ns1 + s2Iq � s+12 ; n2 + r�owhere q = (t2=n)=(1 + t2=n) and s1, s2 are signs di�ering between cases with positive ornegative t as well as odd or even s in the summation. More speci�c, the sign s1 is �1 if sis odd and +1 if it is even while s2 is +1 unless t < 0 and s is even in which case it is �1.12.4 Random Number GenerationRandom numbers from a doubly non-central t-distribution is easily obtained with the def-inition given above using random numbers from a normal distribution and a non-centralchi-square distribution. This ought to be su�cient for most applications but if needed moree�cient techniques may easily be developed e.g. using more general techniques.53

13 Error Function13.1 IntroductionA function, related to the probability content of the normal distribution, which often isreferred to is the error function erf z = 2p� zZ0 e�t2dtand its complement erfc z = 2p� 1Zz e�t2dt = 1� erf zThese functions may be de�ned for complex arguments, for many relations concerning theerror function see [27], but here we are mainly interested in the function for real positivevalues of z. However, sometimes one may still want to de�ne the function values for negativereal values of z using symmetry relationserf(�z) = �erf(z)erfc(�z) = 1 � erf(�z) = 1 + erf(z)13.2 Probability Density FunctionAs is seen the error function erf is a distribution (or cumulative) function and the corre-sponding probability density function is given byf(z) = 2p�e�z2If we make the transformation z = (x��)=�p2 we obtain a folded normal distributionf(x;�; �) = 1�s2�e�12(x��� )2where the function is de�ned for x > � corresponding to z > 0, � may be any real numberwhile � > 0.This implies that erf(z=p2) is equal to the symmetric integral of a standard normaldistribution between �z and z.The error function may also be expressed in terms of the incomplete Gamma functionerf x = 2p� xZ0 e�t2dt = P �12 ; x2�de�ned for x � 0. 54

14 Exponential Distribution14.1 IntroductionThe exponential distribution is given byf(x;�) = 1�e� x�where the variable x as well as the parameter � is positive real quantities.The exponential distribution occur in many di�erent connections such as the radioactiveor particle decays or the time between events in a Poisson process where events happen ata constant rate.14.2 Cumulative FunctionThe cumulative (distribution) function isF (x) = xZ0 f(x)dx = 1 � e� x�and it is thus straightforward to calculate the probability content in any given situation.E.g. we �nd that the median and the lower and upper quartiles are atM = � ln 2 � 0:693�; Q1 = �� ln 34 � 0:288�; and Q3 = � ln 4 � 1:386�14.3 MomentsThe expectation value, variance, and lowest order central moments are given byE(x) = �; V (x) = �2; �3 = 2�3; �4 = 9�4;�5 = 44�5; �6 = 265�6; �7 = 1854�7; and �8 = 14833�8More generally algebraic moments are given by�0n = �nn!Central moments thereby becomes�n = �nn! nXm=0 (�1)mm! ! �nn!e = �0ne when n!1the approximation is, in fact, quite good already for n = 5 where the absolute error is0:146�5 and the relative error 0.3%. 55

14.4 Characteristic FunctionThe characteristic function of the exponential distribution is given by�(t) = E(e{tx) = 1� 1Z0 e({t� 1� )xdx = 11 � {t�14.5 Random Number GenerationThe most common way to achieve random numbers from an exponential distribution is touse the inverse to the cumulative distribution such thatx = F�1(�) = �� ln(1 � �) = �� ln �0where � is a uniform random number between zero and one (aware not to include exactlyzero in the range) and so is, of course, also �0 = 1 � �.There are, however, alternatives some of which may be of some interest and useful ifthe penalty of using the logarithm would be big on any system [28].14.5.1 Method by von NeumannThe �rst of these is due to J. von Neumann [29] and is as follows (with di�erent �'s denotingindependent uniform random numbers between zero and one)i Set a = 0.ii Generate � and put �0 = �.iii Generate �� and if �� < � then go to vi.iv Generate � and if � < �� then go to iii.v Put a = a+ 1 and go to ii.vi Put x = �(a+ �0) as a random number from an exponential distribution.14.5.2 Method by MarsagliaThe second technique is attributed to G. Marsaglia [30].� Prepare pn = 1 � e�n and qn = 1e� 1 � 11! + 12! + � � �+ 1n!�for n = 1; 2; : : : until the largest representable fraction below one is exceeded in bothvectors.i Put i = 0 and generate �.ii If � > pi+1 put i = i+ 1 and perform this step again.56

iii Put k = 1, generate � and ��, and set �min = ��.iv If � � qk then go to vi else set k = k + 1.v Generate a new �� and if �� < �min set �min = �� and go to iv.vi Put x = �(i+ �min) as an exponentially distributed random number.14.5.3 Method by AhrensThe third method is due to J. H. Ahrens [28]� Prepare qn = ln 21! + (ln 2)22! + � � �+ (ln 2)nn!for n = 1; 2; : : : until the largest representable fraction less than one is exceeded.i Put a = 0 and generate �.ii If � < 12 set a = a+ ln 2 = a+ q1, � = 2� and perform this step again.iii Set � = 2� � 1 and if � � ln 2 = q1 then exit with x = �(a + �) else put i = 2 andgenerate �min.iv Generate � and put �min = � if � < �min then if � > qi put i = i+ 1 and perform thisstep again else exit with x = �(a+ q1�min).Of these three methods the method by Ahrens is the fastest. This is much due to thefact that the average number of uniform random numbers consumed by the three methodsis 1.69 for Ahrens, 3.58 for Marsaglia, and 4.31 for von Neumann. The method by Ahrensis often as fast as the direct logarithm method on many computers.57

15 Extreme Value Distribution15.1 IntroductionThe extreme value distribution is given byf(x;�; �) = 1� exp��x� �� � e�x��� �where the upper sign is for the maximum and the lower sign for the minimum (oftenonly the maximum is considered). The variable x and the parameter � (the mode) arereal numbers while � is a positive real number. The distribution is sometimes referredto as the Fisher-Tippett distribution (type I), the log-Weibull distribution, or the Gumbeldistribution after E. J. Gumbel (1891{1966).The extreme value distribution gives the limiting distribution for the largest or small-est elements of a set of independent observations from a distribution of exponential type(normal, gamma, exponential, etc.).A normalized form, useful to simplify calculations, is obtained by making the substitu-tion to the variable z = �x��� which has the distributiong(z) = e�z�e�zIn �gure 11 we show the distribution in this normalized form. The shape corresponds tothe case for the maximum value while the distribution for the minimum value would bemirrored in z = 0.Figure 11: The normalized Extreme Value Distribution58

15.2 Cumulative DistributionThe cumulative distribution for the extreme value distribution is given byF (x) = xZ�1 f(u)du = �x���Z�1 g(z)dz = G(�x� �� )where G(z) is the cumulative function of g(z) which is given byG(z) = zZ�1 e�u�e�udu = 1Ze�z e�ydy = e�e�zwhere we have made the substitution y = e�u in simplifying the integral. From this, andusing x = �� �z, we �nd the position of the median and the lower and upper quartile asM = �� � ln ln 2 � � � 0:367�;Q1 = �� � ln ln 4 � � � 0:327�; andQ3 = �� � ln ln 43 � � � 1:246�15.3 Characteristic FunctionThe characteristic function of the extreme value distribution is given by�(t) = E �e{tx� = 1Z�1 e{tx 1� exp��x� �� � e�x��� � dx == � 1Z0 e{t(��� ln z) ze�z ��dzz ! = e{t� 1Z0 z�{t�e�zdz = e{t��(1 � {t�)where we have made the substitution z = exp (�(x� �)=�) i.e. x = � � � ln z and thusdx = ��dz=z to achieve an integral which could be expressed in terms of the Gammafunction (see section 42.2). As a check we may calculate the �rst algebraic moment, themean, by �01 = 1{ d�(t)dt �����t=0 = 1{ [{��(1) + �(1) (1)(�{�)] = �� � Here (1) = � is the digamma function, see section 42.3, and is Euler's constant.Similarly higher moments could be obtained from the characteristic function or, perhapseven easier, we may �nd cumulants from the cumulant generating function ln�(t). In thesection below, however, moments are determined by a more direct technique.15.4 MomentsAlgebraic moments for f(x) are given byE(xn) = 1Z�1 xnf(x)dx = 1Z�1 (�� �z)ng(z)dz59

which are related to moments of g(z)E(zn) = 1Z�1 zne�z�e�zdz = 1Z0 (� ln y)ne�ydyThe �rst six such integrals, for n values from 1 to 6, are given by1Z0 (� lnx)e�xdx = 1Z0 (� lnx)2e�xdx = 2 + �261Z0 (� lnx)3e�xdx = 3 + �22 + 2�31Z0 (� lnx)4e�xdx = 4 + 2�2 + 3�420 + 8 �31Z0 (� lnx)5e�xdx = 5 + 5 3�23 + 3 �44 + 20 2�3 + 10�2�33 + 24�51Z0 (� lnx)6e�xdx = 6 + 5 4�22 + 9 2�44 + 61�6168 + 40 3�3 ++20 �2�3 + 40�23 + 144 �5corresponding to the six �rst algebraic moments of g(z). Here is Euler's (or Euler-Mascheroni) constant = limn!1 nXk=1 1k � lnn! = 0:57721 56649 01532 86060 65120 ...and �n is a short hand notation for Riemann's zeta-function �(n) given by�(z) = 1Xk=1 1kz = 1�(z) 1Z0 xz�1ex � 1dx for z > 1(see also [31]). For z an even integer we may use�(2n) = 22n�1�2njB2nj(2n)! for n = 1; 2; :::where B2n are the Bernoulli numbers given by B2 = 16 ; B4 = � 130; B6 = 142 ; B8 = � 130etc (see table 4 on page 176 for an extensive table of the Bernoulli numbers). This implies�2 = �26 ; �4 = �490 ; �6 = �6945 etc.For odd integer arguments no similar relation as for even integers exists but evaluatingthe sum of reciprocal powers the two numbers needed in the calculations above are given60

by �3 =1.20205 69031 59594 28540 ... and �5 =1.03692 77551 43369 92633 .... The number �3 issometimes referred to as Ap�ery's constant after the person who in 1979 showed that it isan irrational number (but sofar it is not known if it is also transcendental) [32].Using the algebraic moments of g(z) as given above we may �nd the low order centralmoments of g(z) as �2 = �26 = �2�3 = 2�3�4 = 3�4=20�5 = 10�2�33 + 24�5�6 = 61�6168 + 40�23and thus the coe�cients of skewness 1 and kurtosis 2 are given by 1 = �3=� 322 = 12p6�3=�3 � 1:13955 2 = �4=�22 � 3 = 2:4Algebraic moments of f(x) may be found from this with some e�ort. Central moments aresimpler being connected to those for g(z) through the relation �n(x) = (�1)n�n�n(z).In particular the expectation value and the variance of f(x) are given byE(x) = �� �E(z) = � � � V (x) = �2V (z) = �2�26The coe�cients of skewness (except for a sign �1) and kurtosis are the same as for g(z).15.5 Random Number GenerationUsing the expression for the cumulative distribution we may use a random number �,uniformly distributed between zero and one, to obtain a random number from the extremevalue distribution by G(z) = e�e�z = � ) z = � ln(� ln �)which gives a random number from the normalized function g(z). A random number fromf(x) is then easily obtained by x = � � �z. 61

16 F-distribution16.1 IntroductionThe F -distribution is given byf(F ;m;n) = mm2 nn2 � �m+n2 �� �m2 �� �n2� � F m2 �1(mF + n)m+n2 = mm2 nn2B �m2 ; n2� � F m2 �1(mF + n)m+n2where the parameters m and n are positive integers, degrees of freedom and the variableF is a positive real number. The functions � and B are the usual Gamma and Betafunctions. The distribution is often called the Fisher F -distribution, after the famous britishstatistician Sir Ronald Aylmer Fisher (1890-1962), sometimes the Snedecor F -distributionand sometimes the Fisher-Snedecor F -distribution. In �gure 12 we show the F -distributionfor low values of m and n.Figure 12: The F -distribution (a) for m = 10 and n = 1; 2; : : : ; 10 and (b) for m =1; 2; : : : ; 10 and n = 10For m � 2 the distribution has its maximum at F = 0 and is monotonically decreasing.Otherwise the distribution has the mode atFmode = m� 2m � nn+ 2This distribution is also known as the variance-ratio distribution since it, as will beshown below, describes the distribution of the ratio of the estimated variances from twoindependent samples from normal distributions with equal variance.62

16.2 Relations to Other DistributionsForm = 1 we obtain a t2-distribution, the distribution of the square of a variable distributedaccording to Student's t-distribution. As n!1 the quantity mF approaches a chi-squaredistribution with m degrees of freedom.For large values of m and n the F -distribution tends to a normal distribution. There areseveral approximations found in the literature all of which are better than a simplemindedstandardized variable. One is z1 = p2n � 1 mFn �p2m� 1q1 + mFnand an even better choice is z2 = F 13 �1� 29n�� �1� 29m�q 29m + F 23 � 29nFor large values of m and n also the distribution in the variable z = lnF2 , the distri-bution of which is known as the Fisher z-distribution, is approximately normal with mean12 � 1n � 1m� and variance 12 � 1m + 1n�. This approximation is, however, not as good as z2above.16.3 1/FIf F is distributed according to the F -distribution with m and n degrees of freedom then 1Fhas the F -distribution with n and m degrees of freedom. This is easily veri�ed by a changeof variables. Putting G = 1F we havef(G) = �����dFdG ����� f(F ) = 1G2 � mm2 nn2B(m2 ; n2 ) � � 1G�m2 �1�mG + n�m+n2 = mm2 nn2B(m2 ; n2 ) � Gn2�1(m+ nG)m+n2which is seen to be identical to a F -distribution with n andm degrees of freedom for G = 1F .16.4 Characteristic FunctionThe characteristic function for the F -distribution may be expressed in terms of the con uenthypergeometric function M (see section 43.3) as�(t) = E(e{F t) =M �m2 ;�n2 ;� nm {t�16.5 MomentsAlgebraic moments are given by�0r = mm2 nn2B(m2 ; n2 ) 1Z0 F m2 �1+r(mF + n)m+n2 dF = �mn �m2 1B(m2 ; n2 ) 1Z0 F m2 �1+r�mFn + 1�m+n2 dF =63

= �mn �m2 1B(m2 ; n2 ) 1Z0 �unm �m2 �1+r(u+ 1)m+n2 nmdu = � nm�r B(m2 + r; n2 � r)B(m2 ; n2 ) == � nm�r � � �m2 + r�� �n2 � r�� �m2 �� �n2�and are de�ned for r < n2 . This may be written�0r = � nm�r � m2 (m2 + 1) � � � (m2 + r � 1)(n2 � r)(n2 � r + 1) � � � (n2 � 1)a form which may be more convenient for computations especially when m or n are large.A recursive formula to obtain the algebraic moments would thus be�0r = �0r�1 � � nm� � m2 + r � 1n2 � rstarting with �00 = 1.The �rst algebraic moment, the mean, becomesE(F ) = nn� 2 for n > 2and the variance is given byV (F ) = 2n2(m+ n � 2)m(n� 2)2(n� 4) for n > 416.6 F-ratioRegard F = u=mv=n where u and v are two independent variables distributed according to thechi-square distribution with m and n degrees of freedom, respectively.The independence implies that the joint probability function in u and v is given by theproduct of the two chi-square distributionsf(u; v;m;n) = 0B@�u2�m2 �1 e�u22� �m2 � 1CA0B@�v2�n2�1 e� v22� �n2� 1CAIf we change variables to x = u=mv=n and y = v the distribution in x and y becomesf(x; y;m;n) = ����������@(u; v)@(x; y)���������� f(u; v;m;n)The determinant of the Jacobian of the transformation is ymn and thus we havef(x; y;m;n) = ymn 0B@�xymn �m2 �1 e�xym2n2m2 � �m2 � 1CA0@y n2�1e� y22n2� �n2�1A64

Finally, since we are interested in the marginal distribution in x we integrate over yf(x;m;n) = 1Z0 f(x; y;m;n)dy = �mn �m2 xm2 �12m+n2 � �m2 �� �n2� 1Z0 ym+n2 �1e� y2 (xmn +1)dy == �mn �m2 xm2 �12m+n2 � �m2 �� �n2� � 2m+n2 � �m+n2 ��xmn + 1�m+n2 = �mn �m2B �m2 ; n2� � xm2 �1�xmn + 1�m+n2which with x = F is the F -distribution with m and n degrees of freedom. Here we usedthe integral 1Z0 tz�1e��tdt = �(z)�zin simplifying the expression.16.7 Variance RatioA practical example where the F -distribution is applicable is when estimates of the variancefor two independent samples from normal distributionss21 = mXi=1 (xi � x)2m� 1 and s22 = nXi=1 (yi � y)2n� 1have been made. In this case s21 and s22 are so called normal theory estimates of �21 and �22i.e. (m� 1)s21=�21 and (n� 1)s22=�22 are distributed according to the chi-square distributionwith m� 1 and n� 1 degrees of freedom, respectively.In this case the quantity F = s21�21 � �22s22is distributed according to the F -distribution with m � 1 and n � 1 degrees of freedom.If the true variances of the two populations are indeed the same then the variance ratios21=s22 have the F -distribution. We may thus use this ratio to test the null hypothesisH0 : �21 = �22 versus the alternative H1 : �21 6= �22 using the F -distribution. We would rejectthe null hypotheses at the � con�dence level if the F -ratio is less than F1��=2;m�1;n�1 orgreater than F�=2;m�1;n�1 where F�;m;n is de�ned byF�;m;nZ0 f(F ;m;n)dF = 1 � �i.e. � is the probability content of the distribution above the value F�;m�1;n�1. Note that thefollowing relation between F -values corresponding to the same upper and lower con�dencelevels is valid F1��;m;n = 1F�;n;m65

16.8 Analysis of VarianceAs a simple example, which is often called analysis of variance, we regard n observationsof a dependent variable x with overall mean x divided into k classes on an independentvariable. The mean in each class is denoted xj for j = 1; 2; :::; k. In each of the k classesthere are nj observations together adding up to n, the total number of observations. Belowwe denote by xji the i:th observation in class j.Rewrite the total sum of squares of the deviations from the meanSSx = kXj=1 njXi=1(xji � x)2 = kXj=1 njXi=1 ((xji � xj) + (xj � x))2 == kXj=1 njXi=1 h(xji � xj)2 + (xj � x)2 + 2(xji � xj)(xj � x)i == kXj=1 njXi=1(xji � xj)2 + kXj=1 njXi=1(xj � x)2 + 2 kXj=1(xj � x) njXi=1(xji � xj) == kXj=1 njXi=1(xji � xj)2 + kXj=1nj(xj � x)2 = SSwithin + SSbetweeni.e. the total sum of squares is the sum of the sum of squares within classes and the sumof squares between classes. Expressed in terms of variancesnV (x) = kXj=1 njVj(x) + kXj=1 nj(xj � x)2If the variable x is independent on the classi�cation then the variance within groups andthe variance between groups are both estimates of the same true variance. The quantityF = SSbetween=(k � 1)SSwithin=(n� k)is then distributed according to the F -distribution with k�1 and n�k degrees of freedom.This may then be used in order to test the hypothesis of no dependence. A too high F -valuewould be unlikely and thus we can choose a con�dence level at which we would reject thehypothesis of no dependence of x on the classi�cation.Sometimes one also de�nes �2 = SSbetween=SSx, the proportion of variance explained,as a measure of the strength of the e�ects of classes on the variable x.16.9 Calculation of Probability ContentIn order to set con�dence levels for the F -distribution we need to evaluate the cumulativefunction i.e. the integral 1� � = F�Z0 f(F ;m;n)dF66

where we have used the notation F� instead of F�;m;n for convenience.1 � � = mm2 nn2B(m2 ; n2 ) F�Z0 F m2 �1(mF + n)m+n2 dF = �mn �m2B(m2 ; n2 ) F�Z0 F m2 �1�mFn + 1�m+n2 dF == �mn �m2B(m2 ; n2 ) mF�nZ0 �unm �m2 �1(u+ 1)m+n2 nmdu = 1B(m2 ; n2 ) mF�nZ0 um2 �1(1 + u)m+n2 duwhere we made the substitution u = mFn . The last integral we recognize as the incompleteBeta function Bx de�ned for 0 � x � 1 asBx(p; q) = xZ0 tp�1(1� t)q�1dt = x1�xZ0 up�1(1 + u)p+q duwhere we made the substitution u = t1�t i.e. t = u1+u . We thus obtain1 � � = Bx(m2 ; n2 )B(m2 ; n2 ) = Ix(m2 ; n2 )with x1�x = mF�n i.e. x = mF�n+mF� . The variable x thus has a Beta distribution. Note thatalso Ix(a; b) is called the incomplete Beta function (for historical reasons discussed belowbut see also section 42.7).16.9.1 The Incomplete Beta functionIn order to evaluate the incomplete Beta function we may use the serial expansionBx(p; q) = xp "1p + 1� qp + 1x+ (1 � q)(2� q)2!(p + 2) x2 + : : :+ (1 � q)(2� q) � � � (n� q)n!(p+ n) xn + : : :#For integer values of q corresponding to even values of n the sum may be stopped atn = q � 1 since all remaining terms will be identical to zero in this case.We may express the sum with successive terms expressed recursively in the previoustermBx(p; q) = xp 1Xr=0 tr with tr = tr�1 � x(r � q)(p+ r � 1)r(p + r) starting with t0 = 1pThe sum normally converges quite fast but beware that e.g. for p = q = 12 (m =n = 1) the convergence is very slow. Also some cases with q very big but p small seempathological since in these cases big terms with alternate signs cancel each other causingroundo� problems. It seems preferable to keep q < p to assure faster convergence. Thismay be done by using the relationBx(p; q) = B1(q; p)�B1�x(q; p)67

which if inserted in the formula for 1� � gives1 � � = B1(n2 ; m2 ) �B1�x(n2 ; m2 )B(m2 ; n2 ) ) � = B1�x(n2 ; m2 )B(m2 ; n2 ) = I1�x(n2 ; m2 )since B1(p; q) = B(p; q) = B(q; p).A numerically better way to evaluate the incomplete Beta function Ix(a; b) is by thecontinued fraction formula [10]Ix(a; b) = xa(1 � x)baB(a; b) " 11+ d11+ d21+ � � �#Here d2m+1 = � (a+m)(a+ b+m)x(a+ 2m)(a+ 2m + 1) and d2m = m(b�m)x(a+ 2m� 1)(a+ 2m)and the formula converges rapidly for x < (a+1)=(a+ b+1). For other x-values the sameformula may be used after applying the symmetry relationIx(a; b) = 1 � I1�x(b; a)16.9.2 Final Formul�Using the serial expression for Bx given in the previous subsection the probability contentof the F-distribution may be calculated. The numerical situation is, however, not ideal. Forinteger a- or b-values3 the following relation to the binomial distribution valid for integervalues of a is useful1� Ix(a; b) = I1�x(b; a) = a�1Xi=0 a+ b� 1i !xi(1� x)a+b�1�iOur �nal formul� are taken from [26], using x = nn+mF (note that this is one minus ourprevious de�nition of x),� Even m: 1� � = xn2 � "1 + n2 (1 � x) + n(n+ 1)2 � 4 (1� x)2 + : : :: : :+ n(n+ 2) : : : (m+ n� 4)2 � 4 : : : (m� 2) (1 � x)m�22 #� Even n: 1 � � = 1� (1 � x)m2 "1 + m2 x+ m(m+ 2)2 � 4 x2 + : : :: : :+ m(m+ 2) : : : (m+ n � 4)2 � 4 : : : (n� 2) xn�22 #3If only b is an integer use the relation Ix(a; b) = 1� I1�x(b; a).68

� Odd m and n:1� � = 1 �A+ � withA = 2� �� + sin � �cos � + 23 cos3 � + : : :: : :+ 2 � 4 : : : (n� 3)1 � 3 : : : (n� 2) cosn�2 �!# for n > 1 and� = 2p� � � �n+12 �� �n2� � sin � � cosn � � �1 + n+ 13 sin2 � + : : :: : :+ (n+ 1)(n+ 3) : : : (m+ n� 4)3 � 5 : : : (n� 2) sinm�3 �# for m > 1 where� = arctansnFm and � �n+12 �� �n2� = (n� 1)!!(n� 2)!! � 1p�If n = 1 then A = 2�=� and if m = 1 then � = 0.� For large values of m and n we use an approximation using the standard normaldistribution where z = F 13 �1 � 29n�� �1� 29m�q 29m + F 23 � 29nis approximately distributed according to the standard normal distribution. Con�-dence levels are obtained by 1� � = 1p2� 1Zz e�x22 dxIn table 5 on page 177 we show some percentage points for the F -distribution. Here nis the degrees of freedom of the greater mean square and m the degrees of freedom for thelesser mean square. The values express the values of F which would be exceeded by purechance in 10%, 5% and 1% of the cases, respectively.16.10 Random Number GenerationFollowing the de�nition the quantity F = ym=myn=nwhere yn and ym are two variables distributed according to the chi-square distribution withn and mx degrees of freedom respectively follows the F-distribution. We may thus use thisrelation inserting random numbers from chi-square distributions (see section 8.7).69

17 Gamma Distribution17.1 IntroductionThe Gamma distribution is given byf(x; a; b) = a(ax)b�1e�ax=�(b)where the parameters a and b are positive real quantities as is the variable x. Note thatthe parameter a is simply a scale factor.For b � 1 the distribution is J-shaped and for b > 1 it is unimodal with its maximumat x = b�1a .In the special case where b is a positive integer this distribution is often referred to asthe Erlangian distribution.For b = 1 we obtain the exponential distribution and with a = 12 and b = n2 with n aninteger we obtain the chi-squared distribution with n degrees of freedom.In �gure 13 we show the Gamma distribution for b-values of 2 and 5.Figure 13: Examples of Gamma distributions17.2 Derivation of the Gamma DistributionFor integer values of b, i.e. for Erlangian distributions, we may derive the Gamma distri-bution from the Poisson assumptions. For a Poisson process where events happen at a rateof � the number of events in a time interval t is given by Poisson distributionP (r) = (�t)re��tr!70

The probability that the k:th event occur at time t is then given byk�1Xr=0 P (r) = k�1Xr=0 (�t)re��tr!i.e. the probability that there are at least k events in the time t is given byF (t) = 1Xr=k P (r) = 1 � k�1Xr=0 (�t)re��tr! = �tZ0 zk�1e�z(k � 1)!dz = tZ0 �kzk�1e��z(k � 1)! dzwhere the sum has been replaced by an integral (no proof given here) and the substitutionz = �z made at the end. This is the cumulative Gamma distribution with a = � and b = k,i.e. the time distribution for the k:th event follows a Gamma distribution. In particularwe may note that the time distribution for the occurrence of the �rst event follows anexponential distribution.The Erlangian distribution thus describes the time distribution for exponentially dis-tributed events occurring in a series. For exponential processes in parallel the appropriatedistribution is the hyperexponential distribution.17.3 MomentsThe distribution has expectation value, variance, third and fourth central moments givenby E(x) = ba; V (x) = ba2 ; �3 = 2ba3 ; and �4 = 3b(2 + b)a4The coe�cients of skewness and kurtosis is given by 1 = 2pb and 2 = 6bMore generally algebraic moments are given by�0n = 1Z0 xnf(x)dx = ab�(b) 1Z0 xn+b�1e�axdx == ab�(b) 1Z0 �ya�n+b�1 e�y dya = �(n+ b)an�(b) == b(b+ 1) � � � (b+ n � 1)anwhere we have made the substitution y = ax in simplifying the integral.17.4 Characteristic FunctionThe characteristic function is�(t) = E(e{tx) = ab�(b) 1Z0 xb�1e�x(a�{t)dx =71

= ab�(b) � 1(a� {t)b 1Z0 yb�1e�ydy = �1 � {ta��bwhere we made the transformation y = x(a� {t) in evaluating the integral.17.5 Probability ContentIn order to calculate the probability content for a Gamma distribution we need the cumu-lative (or distribution) functionF (x) = xZ0 f(x)dx = ab�(b) xZ0 ub�1e�audu == ab�(b) axZ0 �va�b�1 e�v dva = 1�(b) axZ0 vb�1e�vdv = (b; ax)�(b)where (b; ax) denotes the incomplete gamma function4.17.6 Random Number Generation17.6.1 Erlangian distributionIn the case of an Erlangian distribution (b a positive integer) we obtain a random numberby adding b independent random numbers from an exponential distribution i.e.x = � ln(�1 � �2 � : : : � �b)=awhere all the �i are uniform random numbers in the interval from zero to one. Note thatcare must be taken if b is large in which case the product of uniform random numbers maybecome zero due to machine precision. In such cases simply divide the product in piecesand add the logarithms afterwards.17.6.2 General caseIn a more general case we use the so called Johnk's algorithmi Denote the integer part of b with i and the fractional part with f and put r = 0. Let� denote uniform random numbers in the interval from zero to one.ii If i > 0 then put r = � ln(�1 � �2 � : : : � �i).iii If f = 0 then go to vii.iv Calculate w1 = �1=fi+1 and w2 = �1=(1�f)i+2 .4When integrated from zero to x the incomplete gamma function is often denoted by (a; x) while for thecomplement, integrated from x to in�nity, it is denoted �(a; x). Sometimes the ratio P (a; x) = (a; x)=�(a)is called the incomplete Gamma function. 72

v If w1 + w2 > 1 then go back to iv.vi Put r = r � ln(�i+3) � w1w1+w2 .vii Quit with r = r=a.17.6.3 Asymptotic ApproximationFor b big, say b > 15, we may use the Wilson-Hilferty approximation:i Calculate q = 1 + 19b + z3pb where z is a random number from a standard normaldistribution.ii Calculate r = b � q3.iii If r < 0 then go back to i.iv Quit with r = r=a.

73

18 Generalized Gamma Distribution18.1 IntroductionThe Gamma distribution is often used to describe variables bounded on one side. An evenmore exible version of this distribution is obtained by adding a third parameter givingthe so called generalized Gamma distributionf(x; a; b; c) = ac(ax)bc�1e�(ax)c=�(b)where a (a scale parameter) and b are the same real positive parameters as is used for theGamma distribution but a third parameter c has been added (c = 1 for the ordinary Gammadistribution). This new parameter may in principle take any real value but normally weconsider the case where c > 0 or even c � 1. Put jcj in the normalization for f(x) if c < 0.According to Hegyi [33] this density function �rst appeared in 1925 when L. Amorosoused it in analyzing the distribution of economic income. Later it has been used to describethe sizes of grains produced in comminution and drop size distributions in sprays etc.In �gure 14 we show the generalized Gamma distribution for di�erent values of c forthe case a = 1 and b = 2.Figure 14: Examples of generalized Gamma distributions18.2 Cumulative FunctionThe cumulative function is given byF (x) = � (b; (ax)c) =�(b) = P (b; (ax)c) if c > 0� (b; (ax)c) =�(b) = 1 � P (b; (ax)c) if c < 0where P is the incomplete Gamma function.74

18.3 MomentsAlgebraic moments are given by �0n = 1an � � �b+ nc ��(b)For negative values of c the moments are �nite for ranks n satisfying n=c > �b (or evenjust avoiding the singularities 1a + nc 6= 0;�1;�2 : : :).18.4 Relation to Other DistributionsThe generalized Gamma distribution is a general form which for certain parameter com-binations gives many other distributions as special cases. In the table below we indicatesome such relations. For notations see the corresponding section.Distribution a b c SectionGeneralized gamma a b c 18Gamma a b 1 17Chi-squared 12 n2 1 8Exponential 1� 1 1 14Weibull 1� 1 � 41Rayleigh 1�p2 1 2 37Maxwell 1�p2 32 2 25Standard normal (folded) 1p2 12 2 34In reference [33], where this distribution is used in the description of multiplicity distri-butions in high energy particle collisions, more examples on special cases as well as moredetails regarding the distribution are given.75

19 Geometric Distribution19.1 IntroductionThe geometric distribution is given byp(r; p) = p(1 � p)r�1where the integer variable r � 1 and the parameter 0 < p < 1 (no need to include limitssince this give trivial special cases). It expresses the probability of having to wait exactlyr trials before the �rst successful event if the probability of a success in a single trial is p(probability of failure q = 1 � p). It is a special case of the negative binomial distribution(with k = 1).19.2 MomentsThe expectation value, variance, third and fourth moment are given byE(r) = 1p V (r) = 1 � pp2 �3 = (1 � p)(2 � p)p3 �4 = (1� p)(p2 � 9p + 9)p4The coe�cients of skewness and kurtosis is thus 1 = 2 � pp1 � p and 2 = p2 � 6p + 61� p19.3 Probability Generating FunctionThe probability generating function isG(z) = E(zr) = 1Xr=1 zrp(1 � p)r�1 = pz1 � qz19.4 Random Number GenerationThe cumulative distribution may be writtenP (k) = kXr=1 p(r) = 1 � qk with q = 1 � pwhich can be used in order to obtain a random number from a geometric distribution bygenerating uniform random numbers between zero and one until such a number (the k:th)is above qk.A more straightforward technique is to generate uniform random numbers �i until we�nd a success where �k � p.These two methods are both very ine�cient for low values of p. However, the �rsttechnique may be solved explicitlykXr=1P (r) = � ) k = ln �ln q76

which implies taking the largest integer less than k+1 as a random number from a geometricdistribution. This method is quite independent of the value of p and we found [14] thata reasonable breakpoint below which to use this technique is p = 0:07 and use the �rstmethod mentioned above this limit. With such a method we do not gain by creating acumulative vector for the random number generation as we do for many other discretedistributions.

77

20 Hyperexponential Distribution20.1 IntroductionThe hyperexponential distribution describes exponential processes in parallel and is givenby f(x; p; �1; �2) = p�1e��1x + q�2e��2xwhere the variable x and the parameters �1 and �2 are positive real quantities and 0 � p � 1is the proportion for the �rst process and q = 1 � p the proportion of the second.The distribution describes the time between events in a process where the events aregenerated from two independent exponential distributions. For exponential processes inseries we obtain the Erlangian distribution (a special case of the Gamma distribution).The hyperexponential distribution is easily generalized to the case with k exponentialprocesses in parallel f(x) = kXi=1 pi�ie��ixwhere �i is the slope and pi the proportion for each process (with the constraint thatP pi = 1).The cumulative (distribution) function isF (x) = p �1� e��1x�+ q �1 � e��2x�and it is thus straightforward to calculate the probability content in any given situation.20.2 MomentsAlgebraic moments are given by �0n = n! p�n1 + q�n2 !Central moments becomes somewhat complicated but the second central moment, thevariance of the distribution, is given by�2 = V (x) = p�21 + q�22 + pq � 1�1 � 1�2�220.3 Characteristic FunctionThe characteristic function of the hyperexponential distribution is given by�(t) = p1 � {t�1 + q1 � {t�278

20.4 Random Number GenerationGenerating two uniform random numbers between zero and one, �1 and �2, we obtain arandom number from a hyperexponential distribution by� If �1 � p then put x = � ln �2�1 .� If �1 > p then put x = � ln �2�2 .i.e. using �1 we choose which of the two processes to use and with �2 we generate anexponential random number for this process. The same technique is easily generalized tothe case with k processes.

79

21 Hypergeometric Distribution21.1 IntroductionThe Hypergeometric distribution is given byp(r;n;N;M) = �Mr ��N�Mn�r ��Nn�where the discrete variable r has limits from max(0; n�N +M) to min(n;M) (inclusive).The parameters n (1 � n � N), N (N � 1) and M (M � 1) are all integers.This distribution describes the experiment where elements are picked at random withoutreplacement. More precisely, suppose that we haveN elements out of whichM has a certainattribute (and N �M has not). If we pick n elements at random without replacement p(r)is the probability that exactly r of the selected elements come from the group with theattribute.If N � n this distribution approaches a binomial distribution with p = MN .If instead of two groups there are k groups with di�erent attributes the generalizedhypergeometric distribution p(r;n;N;M ) = kQi=1 �Miri ��Nn�where, as before, N is the total number of elements, n the number of elements picked andM a vector with the number of elements of each attribute (whose sum should equal N).Here n =P ri and the limits for each rk is given by max(0; n�N+Mk) � rk � min(n;Mk).21.2 Probability Generating FunctionThe Hypergeometric distribution is closely related to the hypergeometric function, seeappendix B on page 169, and the probability generating function is given byG(z) = �N�Mn ��Nn� 2F1(�n;�M ;N�M�n+1; z)21.3 MomentsWith the notation p = MN and q = 1� p, i.e. the proportions of elements with and withoutthe attribute, the expectation value, variance, third and fourth central moments are givenbyE(r) = npV (r) = npqN � nN � 1�3 = npq(q � p)(N � n)(N � 2n)(N � 1)(N � 2) 80

�4 = npq(N � n)N(N + 1) � 6n(N � n) + 3pq(N2(n� 2)�Nn2 + 6n(N � n))(N � 1)(N � 2)(N � 3)For the generalized hypergeometric distribution using pi = Mi=N and qi = 1 � pi we�nd moments of ri using the formul� above regarding the group i as having an attributeand all other groups as not having the attribute. the covariances are given byCov(ri; rj) = npipjN � nN � 121.4 Random Number GenerationTo generate random numbers from a hypergeometric distribution one may construct aroutine which follow the recipe above by picking elements at random. The same techniquemay be applied for the generalized hypergeometric distribution. Such techniques may besu�cient for many purposes but become quite slow.For the hypergeometric distribution a better choice is to construct the cumulative func-tion by adding up the individual probabilities using the recursive formulap(r) = (M � r + 1)(n� r + 1)r(N �M � n+ r) p(r � 1)for the appropriate r-range (see above) starting with p(rmin). With the cumulative vectorand one single uniform random number one may easily make a fast algorithm in order toobtain the required random number.

81

22 Logarithmic Distribution22.1 IntroductionThe logarithmic distribution is given byp(r; p) = �(1� p)rr ln pwhere the variable r � 1 is an integer and the parameter 0 < p < 1 is a real quantity.It is a limiting form of the negative binomial distribution when the zero class has beenomitted and the parameter k ! 0 (see section 29.4.3).22.2 MomentsThe expectation value and variance are given byE(r) = ��qp and V (r) = ��q(1 + �q)p2where we have introduced q = 1� p and � = 1= ln p for convenience. The third and fourthcentral moments are given by�3 = ��qp3 �1 + q + 3�q + 2�2q2��4 = ��qp4 �1 + 4q + q2 + 4�q(1 + q) + 6�2q2 + 3�3q3�More generally factorial moments are easily found using the probability generatingfunction E(r(r � 1) � � � (r � k + 1)) = dkdzkG(z)�����z=1 = �(n� 1)!�qkpkFrom these moments ordinary algebraic and central moments may be found by straightfor-ward but somewhat tedious algebra.22.3 Probability Generating FunctionThe probability generating function is given byG(z) = E(zr) = 1Xr=0�zr(1� p)rr ln p = � 1ln p 1Xr=0 (zq)rr = ln(1� zq)ln(1 � q)where q = 1� p and sinceln(1� x) = � x+ x22 + x33 + x44 + : : :! for � 1 � x < 182

22.4 Random Number GenerationThe most straightforward way to obtain random numbers from a logarithmic distributionis to use the cumulative technique. If p is �xed the most e�cient way is to prepare acumulative vector starting with p(1) = ��q and subsequent elements by the recursiveformula p(i) = p(i � 1)q=i. The cumulative vector may, however, become very long forsmall values of p. Ideally it should extend until the cumulative vector element is exactlyone due to computer precision. It p is not �xed the same procedure has to be made at eachgeneration.

83

23 Logistic Distribution23.1 IntroductionThe Logistic distribution is given byf(x; a; k) = ezk(1 + ez)2 with z = x� akwhere the variable x is a real quantity, the parameter a a real location parameter (themode, median, and mean) and k a positive real scale parameter (related to the standarddeviation). In �gure 15 the logistic distribution with parameters a=0 and k=1 (i.e. z=x)is shown.Figure 15: Graph of logistic distribution for a = 0 and k = 123.2 Cumulative DistributionThe distribution function is given byF (x) = 1� 11 + ez = 11 + e�z = 11 + e�x�akThe inverse function is found by solving F (x)=� givingx = F�1(�) = a� k ln�1 � �� �from which we may �nd e.g. the median asM=a. Similarly the lower and upper quartilesare given by Q1;2=a� k ln 3. 84

23.3 Characteristic FunctionThe characteristic function is given by�(t) = E(e{tx) = 1Z�1 e{tx ex�akk �1 + ex�ak �2dx = e{ta 1Z�1 e{tzkezk(1 + ez)2kdz == e{ta 1Z0 y{tky(1 + y)2 � dyy = e{taB(1+{tk; 1�{tk) == e{ta�(1+{tk)�(1�{tk)�(2) = e{ta{tk�({tk)�(1�{tk) = e{ta {tk�sin �{tkwhere we have used the transformations z=(x�a)=k and y=ez in simplifying the integral,at the end identifying the beta function, and using relation of this in terms of Gammafunctions and their properties (see appendix A in section 42).23.4 MomentsThe characteristic function is slightly awkward to use in determining the algebraic momentsby taking partial derivatives in t. However, usingln�(t) = {ta+ ln�(1+{tk) + ln�(1�{tk)we may determine the cumulants of the distributions. In the process we take derivatives ofln�(t) which involves polygamma functions (see section 42.4) but all of them with argument1 when inserting t=0 a case which may be explicitly written in terms of Riemann's zeta-functions with even real argument (see page 60). It is quite easily found that all cumulantsof odd order except �1 = a vanish and that for even orders�2n = 2k2n (2n�1)(1) = 2(2n � 1)!k2n�(2n) = 2(2n � 1)!k2n22n�1�2njB2nj(2n)!for n = 1; 2; : : : and where B2n are the Bernoulli numbers (see table 4 on page 176).Using this formula lower order moments and the coe�cients of skewness and kurtosisis found to be �01 = E(x) = �1 = a�2 = V (x) = �2 = k2�2=3�3 = 0�4 = �4 + 3�22 = 2k4�415 + k4�43 = 7k4�415�5 = 0�6 = �6 + 15�4�2 + 10�23 + 15�32 == 16k6�663 + 2k6�63 + 15k6�627 = 31k6�611 1 = 0 2 = 1:2 (exact) 85

23.5 Random numbersUsing the inverse cumulative function one easily obtains a random number from a logisticdistribution by x = a+ k ln �1� �!with � a uniform random number between zero and one (limits not included).

86

24 Log-normal Distribution24.1 IntroductionThe log-normal distribution or is given byf(x;�; �) = 1x�p2�e� 12( ln x��� )2where the variable x > 0 and the parameters � and � > 0 all are real numbers. It issometimes denoted �(�; �2) in the same spirit as we often denote a normally distributedvariable by N(�; �2).If u is distributed as N(�; �2) and u = lnx then x is distributed according to thelog-normal distribution.Note also that if x has the distribution �(�; �2) then y = eaxb is distributed as �(a+b�; b2�2).In �gure 16 we show the log-normal distribution for the basic form, with � = 0 and� = 1.Figure 16: Log-normal distributionThe log-normal distribution is sometimes used as a �rst approximation to the Landaudistribution describing the energy loss by ionization of a heavy charged particle (cf also theMoyal distribution in section 26).24.2 MomentsThe expectation value and the variance of the distribution are given byE(x) = e�+�22 and V (x) = e2�+�2 �e�2 � 1�87

and the coe�cients of skewness and kurtosis becomes 1 = qe�2 � 1 �e�2 + 2� and 2 = �e�2 � 1� �e3�2 + 3e2�2 + 6e�2 + 6�More generally algebraic moments of the log-normal distribution are given by�0k = E(xk) = 1�p2� 1Z0 xk�1e� 12( ln x��� )2dx = 1�p2� 1Z�1 eyke� 12( y��� )2dy == 1�p2�ek�+ k2�22 1Z�1 e� 12� y���k�2� �2dy = ek�+ k2�22where we have used the transformation y = lnx in simplifying the integral.24.3 Cumulative DistributionThe cumulative distribution, or distribution function, for the log-normal distribution isgiven by F (x) = 1�p2� xZ0 1t e� 12( ln t��� )2dt = 1�p2� lnxZ0 e� 12( y��� )2dy == 12 � 12P 12 ; z22 !where we have put z = (lnx��)=� and the positive sign is valid for z � 0 and the negativesign for z < 0.24.4 Random Number GenerationThe most straightforward way of achieving random numbers from a log-normal distributionis to generate a random number u from a normal distribution with mean � and standarddeviation � and construct r = eu.88

25 Maxwell Distribution25.1 IntroductionThe Maxwell distribution is given byf(x;�) = 1�3s2�x2e� x22�2where the variable x with x � 0 and the parameter � with � > 0 are real quantities. It isnamed after the famous scottish physicist James Clerk Maxwell (1831{1879).The parameter � is simply a scale factor and the variable y = x=� has the simpli�eddistribution g(y) = s2�y2e� y22Figure 17: The Maxwell distributionThe distribution, shown in �gure 17, has a mode at x = � and is positively skewed.25.2 MomentsAlgebraic moments are given byE(xn) = 1Z0 xnf(x)dx = 12�3s2� 1Z�1 jxjn+2e�x2=2�2i.e. we have a connection to the absolute moments of the Gauss distribution. Using these(see section on the normal distribution) the result isE(xn) = (q 2�2kk!�2k�1 for n = 2k � 1(n+ 1)!!�n for n even89

Speci�cally we note that the expectation value, variance, and the third and fourthcentral moments are given byE(x) = 2�s2�; V (x) = �2 �3 � 8�� ; �3 = 2�3 �16� � 5�s2� ; and �4 = �4 �15 � 8��The coe�cients of skewness and kurtosis is thus 1 = 2 �16� � 5�q 2��3� 8�� 32 � 0:48569 and 2 = 15 � 8��3 � 8��2 � 3 � 0:1081825.3 Cumulative DistributionThe cumulative distribution, or the distribution function, is given byF (x) = xZ0 f(y)dy = 1a3s2� xZ0 y2e� y22�2 dy = 2p� x22�2Z0 pze�zdz = �32 ; x22�2�� �32� = P 32 ; x22�2!where we have made the substitution z = y22�2 in order to simplify the integration. HereP (a; x) is the incomplete Gamma function.Using the above relation we may estimate the median M and the lower and upperquartile, Q1 and Q3, as Q1 = �qP�1(32; 12) � 1:10115 �M = �qP�1(32; 12) � 1:53817 �Q3 = �qP�1(32; 12) � 2:02691 �where P�1(a; p) denotes the inverse of the incomplete Gamma function i.e. the value x forwhich P (a; x) = p.25.4 Kinetic TheoryThe following is taken from kinetic theory, see e.g. [34]. Let v = (vx; vy; vz) be the velocityvector of a particle where each component is distributed independently according to normaldistributions with zero mean and the same variance �2.First construct w = v2�2 = v2x�2 + v2y�2 + v2z�2Since vx=�, vy=�, and vz=� are distributed as standard normal variables the sum of theirsquares has the chi-squared distribution with 3 degrees of freedom i.e. g(w) = q w2�e�w=2which leads to f(v) = g(w) �����dwdv ����� = g v2�2! 2v�2 = 1�3s2�v2e� v22�290

which we recognize as a Maxwell distribution with � = �.In kinetic theory � = kT=m, where k is Boltzmann's constant, T the temperature, andm the mass of the particles, and we thus havef(v) = s 2m3�k3T 3 v2e�mv22kTThe distribution in kinetic energy E = mv2=2 becomesg(E) = s 4E�k3T 3e� EkTwhich is a Gamma distribution with parameters a = 1=kT and b = 32.25.5 Random Number GenerationTo obtain random numbers from the Maxwell distribution we �rst make the transformationy = x2=2�2 a variable which follow the Gamma distribution g(y) = pye�y=� � 32�.A random number from this distribution may be obtained using the so called Johnk'salgorithmwhich in this particular case becomes (denoting independent pseudorandom num-bers from a uniform distribution from zero to one by �i)i Put r = � ln �1 i.e. a random number from an exponential distribution.ii Calculate w1 = �22 and w2 = �23 (with new uniform random numbers �2 and �3 eachiteration, of course).iii If w = w1 + w2 > 1 then go back to ii above.iv Put r = r � w1w ln �4v Finally construct ap2r as a random number from the Maxwell distribution withparameter r.Following the examples given above we may also use three independent random numbersfrom a standard normal distribution, z1, z2, and z3, and constructr = 1�qz21 + z22 + z23However, this technique is not as e�cient as the one outlined above.As a third alternative we could also use the cumulative distribution puttingF (x) = � ) P �32; x22�2� = � ) x = �r2P�1 �32; ��where P�1(a; p), as above, denotes the value x where P (a; x) = p. This technique is,however, much slower than the alternatives given above.The �rst technique described above is not very fast but still the best alternative pre-sented here. Also it is less dependent on numerical algorithms (such as those to �nd theinverse of the incomplete Gamma function) which may a�ect the precision of the method.91

26 Moyal Distribution26.1 IntroductionThe Moyal distribution is given byf(z) = 1p2� expn�12 �z + e�z�ofor real values of z. A scale shift and a scale factor is introduced by making the standardizedvariable z = (x� �)=� and hence the distribution in the variable x is given byg(x) = 1�f �x� �� �Without loss of generality we treat the Moyal distribution in its simpler form, f(z), in thisdocument. Properties for g(x) are easily obtained from these results which is sometimesindicated.The Moyal distribution is a universal form for(a) the energy loss by ionization for a fast charged particle and(b) the number of ion pairs produced in this process.It was proposed by J. E. Moyal [35] as a good approximation to the Landau distribution.It was also shown that it remains valid taking into account quantum resonance e�ects anddetails of atomic structure of the absorber.Figure 18: The Moyal distributionThe distribution, shown in �gure 18, has a mode at z = 0 and is positively skewed.This implies that the mode of the x�distribution, g(x), is equal to the parameter �.92

26.2 NormalizationMaking the transformation x = e�z we �nd that1Z�1 f(z)dz = 1Z0 1p2� expn�12 (� lnx+ x)o dxx = 1p2� 1Z0 e�x2px dx == 1p2� 1Z0 e�yp2y2dy = 1p� 1Z0 e�ypy dy = 1p�� �12� = 1where we have made the simple substitution y = x=2 in order to clearly recognize theGamma function at the end. The distribution is thus properly normalized.26.3 Characteristic FunctionThe characteristic function for the Moyal distribution becomes�(t) = E(e{tz) = 1p2� 1Z�1 e{tze� 12(z+e�z)dz = 1p2� 1Z0 (2x) 12 (1�2{t)e�xdxx == 2 12 (1�2{t)p2� 1Z0 x� 12 (1+2{t)e�xdx = 2�{tp� � �12 � {t�where we made the substitution x = e�z=2 in simplifying the integral. The last relation tothe Gamma function with complex argument is valid when the real part of the argumentis positive which indeed is true in the case at hand.26.4 MomentsAs in some other cases the most convenient way to �nd the moments of the distribution isvia its cumulants (see section 2.5). We �nd that�1 = � ln 2 � (12) = ln 2 + �n = (�1)n (n�1)(12) = (n� 1)!(2n � 1)�n for n � 2with � 0:5772156649 Euler's constant, (n) polygamma functions (see section 42.4) and� Riemann's zeta-function (see page 60). Using the cumulants we �nd the lower ordermoments and the coe�cients of skewness and kurtosis to be�01 = E(z) = �1 = ln 2 + � 1:27036�2 = V (z) = �2 = (1)(12) = �22 � 4:93480�3 = �3 = � (2)(12) = 14�3�4 = �4 + 3�22 = (3)(12) + 3 (1)(12)2 = 7�44 1 = 28p2�3�3 � 1:53514 2 = 4 93

For the distribution g(x) we have E(x) = �E(z)+ �, V (x) = �2V (z) or more generallycentral moments are obtained by �n(x) = �n�n(z) for n � 2 while 1 and 2 are identical.26.5 Cumulative DistributionUsing the same transformations as was used above in evaluating the normalization of thedistribution we write the cumulative (or distribution) function asF (Z) = ZZ�1 f(z)dz = 1p2� ZZ�1 expn�12 �z + e�z�o = 1p2� 1Ze�Z e�x2px dx == 1p� 1Ze�Z=2 e�ypy dy = 1p�� 12 ; e�Z2 ! = � �12 ; e�Z2 �� � 12� = 1� P 12 ; e�Z2 !where P is the incomplete Gamma function.Using the inverse of the cumulative function we �nd the medianM � 0:78760 and thelower and upper quartiles Q1 � -0:28013 and Q3 � 2:28739.26.6 Random Number GenerationTo obtain random numbers from the Moyal distribution we may either make use of theinverse to the incomplete Gamma function such that given a pseudorandom number � weget a random number by solving the equation1� P 12 ; e�z2 ! = �for z. If P�1(a; p) denotes the value x where P (a; x) = p thenz = � lnn2P�1 �12 ; 1� ��ois a random number from a Moyal distribution.This is, however, a very slow method and one may instead use a straightforward reject-accept (or hit-miss) method. To do this we prefer to transform the distribution to get itinto a �nite interval. For this purpose we make the transformation tan y = x givingh(y) = f(tan y) 1cos2 y = 1p2� 1cos2 y exp��12 �tan y + e� tany��This distribution, shown in �gure 19, has a maximum of about 0.911 and is limited to theinterval ��2 � y � �2 .A simple algorithm to get random numbers from a Moyal distribution, either f(z) org(x), using the reject-accept technique is as follows:a Get into �1 and �2 two uniform random numbers uniformly distributed between zeroand one using a good basic pseudorandom number generator.94

Figure 19: Transformed Moyal distributionb Calculate uniformly distributed variables along the horizontal and vertical directionby y = ��1 � �2 and h = �2hmax where hmax = 0:912 is chosen slightly larger than themaximum value of the function.c Calculate z = tan y and the function value h(y).d If h � h(y) then accept z as a random number from the Moyal distribution f(z) elsego back to point a above.e If required then scale and shift the result by x = z�+ � in order to obtain a randomnumber from g(x).This method is easily improved e.g. by making a more tight envelope to the distributionthan a uniform distribution. The e�ciency of the reject-accept technique outlined here isonly 1=0:912� � 0:35 (the ratio between the area of the curve and the uniform distribution).The method seems, however, fast enough for most applications.95

27 Multinomial Distribution27.1 IntroductionThe Multinomial distribution is given byp(r;N; k; p) = N !r1!r2! � � � rk!pr11 pr22 � � � prkk = N ! kYi=1 priiri!where the variable r is a vector with k integer elements for which 0 � ri � N andP ri = N .The parameters N > 0 and k > 2 are integers and p is a vector with elements 0 � pi � 1with the constraint that P pi = 1.The distribution is a generalization of the Binomial distribution (k = 2) to many di-mensions where, instead of two groups, the N elements are divided into k groups each witha probability pi with i ranging from 1 to k. A common example is a histogram with Nentries in k bins.27.2 HistogramThe histogram example is valid when the total number of events N is regarded as a �xednumber. The variance in each bin then becomes, see also below, V (ri) = Npi(1� p1) � riif pi � 1 which normally is the case for a histogram with many bins.If, however, we may regard the total number of events N as a random variable dis-tributed according to the Poisson distribution we �nd: Given a multinomial distribution,here denoted M(r;N; p), for the distribution of events into bins for �xed N and a Poissondistribution, denoted P (N ; �), for the distribution of N we write the joint distributionP(r;N) = M(r;N; p)P (N ; �) = N !r1!r2! : : : rk!pr11 pr22 : : : prkk ! �Ne��N ! ! == � 1r1! (�p1)r1e��p1�� 1r2! (�p2)r2e��p2� : : :� 1rk! (�pk)rke��pk�where we have used that kXi=1 pi = 1 and kXi=1 ri = Ni.e. we get a product of independent Poisson distributions with means �pi for each individualbin.As seen, in both cases, we �nd justi�cation for the normal rule of thumb to assign thesquare root of the bin contents as the error in a certain bin. Note, however, that in principlewe should insert the true value of ri for this error. Since this normally is unknown we usethe observed number of events in accordance with the law of large numbers. This meansthat caution must be taken in bins with few entries.27.3 MomentsFor each speci�c ri we may obtain moments using the Binomial distribution with qi = 1�piE(ri) = Npi and V (ri) = Npi(1 � pi) = Npiqi96

The covariance between two groups are given byCov(ri; rj) = �Npipj for i 6= j27.4 Probability Generating FunctionThe probability generating function for the multinomial distribution is given byG(z) = kXi=1 pizi!N27.5 Random Number GenerationThe straightforward but time consuming way to generate random numbers from a multi-nomial distribution is to follow the de�nition and generate N uniform random numberswhich are assigned to speci�c bins according to the cumulative value of the p-vector.27.6 Signi�cance LevelsTo determine a signi�cance level for a certain outcome from a multinomial distribution onemay add all outcomes which are as likely or less likely than the probability of the observedoutcome. This may be a non-trivial calculation for large values of N since the number ofpossible outcomes grows very fast. An alternative, although quite clumsy, is to generatea number of multinomial random numbers and evaluate how often these outcomes are aslikely or less likely than the observed one.If we as an example observe the outcome r = (4; 1; 0; 0; 0; 0) for a case with 5 obser-vations in 6 groups (N = 5 and k = 6) and the probability for all groups are the samepi = 1=k = 1=6 we obtain a probability of p � 0:02. This includes all orderings of the sameoutcome since these are all equally probable but also all less likely outcomes of the typep = (5; 0; 0; 0; 0; 0).If a probability calculated in this manner is too small one may conclude that the nullhypothesis that all probabilities are equal is wrong. Thus if our con�dence level is presetto 95% this conclusion would be drawn in the above example. Of course, the conclusionwould be wrong in 2% of all cases.27.7 Equal Group ProbabilitiesA common case or null hypothesis for a multinomial distribution is that the probability ofthe k groups is the same i.e. p = 1=k. In this case the multinomial distribution is simpli�edand since ordering become insigni�cant much fewer unique outcomes are possible.Take as an example a game where �ve dices are thrown. The probabilities for di�erentoutcomes may quite readily be evaluated from basic probability theory properly accountingfor the 65 = 7776 possible outcomes. But one may also use the multinomial distributionwith k = 6 and N = 5 to �nd probabilities for di�erent outcomes. If we properly takecare of combinatorial coe�cients for each outcome we obtain (with zeros for empty groupssuppressed) 97

name outcome # combinations probabilityone doublet 2,1,1,1 3600 0.46296two doublets 2,2,1 1800 0.23148triplets 3,1,1 1200 0.15432nothing 1,1,1,1,1 720 0.09259full house 3,2 300 0.03858quadruplets 4,1 150 0.01929quintuplets 5 6 0.00077total 7776 1.00000The experienced dice player may note that the \nothing" group includes 240 combina-tions giving straights (1 to 5 or 2 to 6). From this table we may verify the statement fromthe previous subsection that the probability to get an outcome with quadruplets or lesslikely outcomes is given by 0.02006.Generally we have for N < k that the two extremes of either all observations in separategroups psep or all observations in one group pallpsep = k!kN (k �N)! = kk � k � 1k � � � k �N + 1kpall = 1kN�1which we could have concluded directly from a quite simple probability calculation.The �rst case is the formula which shows the quite well known fact that if 23 people ormore are gathered the probability that at least two have the same birthday, i.e. 1� psep, isgreater than 50% (using N = 23 and k = 365 and not bothering about leap-years or possibledeviations from the hypothesis of equal probabilities for each day). This somewhat non-intuitive result becomes even more pronounced for higher values of k and the level abovewhich psep < 0:5 is approximately given byN � 1:2pkFor higher signi�cance levels we may note that in the case with k = 365 the probability1 � psep becomes greater than 90% at N = 41, greater than 99% at N = 57 and greaterthan 99.9% at N = 70 i.e. already for N << k a bet would be almost certain.In Fig.20 we show, in linear scale to the left and logarithmic scale to the right, thelower limit on N for which the probability to have 1 � psep above 50%, 90%, 99% and99.9.% for k-values ranging up to 1000. By use of the gamma function the problem hasbeen generalized to real numbers. Note that the curves start at certain values where k = Nsince for N > k it is impossible to have all events in separate groups5.5This limit is at N = k = 2 for the 50%-curve, 3.92659 for 90%, 6.47061 for 99% and 8.93077 for 99.9%98

Figure 20: Limits for N at several con�dence levels as a function of k (linear scale to theleft and logarithmic scale to the right).

99

28 Multinormal Distribution28.1 IntroductionAs a generalization of the normal or Gauss distribution to many dimensions we de�ne themultinormal distribution.A multinormal distribution in x = fx1; x2; : : : ; xng with parameters � (mean vector)and V (variance matrix) is given byf(xj�; V ) = e� 12 (x��)V�1(x��)T(2�)n2qjV jThe variance matrix V has to be a positive semi-de�nite matrix in order for f to be a properprobability density function (necessary in order that the normalization integral R f(x)dxshould converge).If x is normal and V non-singular then (x��)V �1(x��)T is called the covariance formof x and has a �2-distribution with n degrees of freedom. Note that the distribution hasconstant probability density for constant values of the covariance form.The characteristic function is given by�(t) = e{t�� 12 tTV twhere t is a vector of length n.28.2 Conditional Probability DensityThe conditional density for a �xed value of any xi is given by a multinormal density withn � 1 dimensions where the new variance matrix is obtained by deleting the i:th row andcolumn of V �1 and inverting the resulting matrix.This may be compared to the case where we instead just want to neglect one of thevariables xi. In this case the remaining variables has a multinormal distribution with n�1dimensions with a variance matrix obtained by deleting the i:th row and column of V .28.3 Probability ContentAs discussed in section 6.6 on the binormal distribution the joint probability content of amultidimensional normal distribution is di�erent, and smaller, than the corresponding wellknown �gures for the one-dimensional normal distribution. In the case of the binormaldistribution the ellipse (see �gure 2 on page 20) corresponding to one standard deviationhas a joint probability content of 39.3%.The same is even more true for the probability content within the hyperellipsoid in thecase of a multinormal distribution. In the table below we show, for di�erent dimensions n,the probability content for the one (denoted z = 1), two and three standard deviation con-tours. We also give z-values z1, z2, and z3 adjusted to give a probability content within thehyperellipsoid corresponding to the one-dimensional one, two, and three standard deviation100

contents ( 68.3%, 95.5%, and 99.7%). Finally z-value corresponding to joint probabilitycontents of 90%, 95% and 99% in z90, z95, and z99, respectively, are given. Note that theseprobability contents are independent of the variance matrix which only has the e�ect tochange the shape of the hyperellipsoid from a perfect hypersphere with radius z when allvariables are uncorrelated to e.g. cigar shapes when correlations are large.Note that this has implications on errors estimated from a chi-square or a maximumlikelihood �t. If a multiparameter con�dence limit is requested and the chi-square minimumis at �2min or the logarithmic likelihood maximum at lnLmax, one should look for the errorcontour at �2min + z2 or lnLmax � z2=2 using a z-value from the right-hand side of thetable below. The probability content for a n-dimensional multinormal distribution as givenbelow may be expressed in terms of the incomplete Gamma function byp = P �n2 ; z22 �as may be deduced by integrating a standard multinormal distribution out to a radius z.Special formul� for the incomplete Gamma function P (a; x) for integer and half-integer aare given in section 42.5.3.Probability content in % Adjusted z-valuesn z = 1 z = 2 z = 3 z1 z2 z3 z90 z95 z991 68.27 95.45 99.73 1.000 2.000 3.000 1.645 1.960 2.5762 39.35 86.47 98.89 1.515 2.486 3.439 2.146 2.448 3.0353 19.87 73.85 97.07 1.878 2.833 3.763 2.500 2.795 3.3684 9.020 59.40 93.89 2.172 3.117 4.031 2.789 3.080 3.6445 3.743 45.06 89.09 2.426 3.364 4.267 3.039 3.327 3.8846 1.439 32.33 82.64 2.653 3.585 4.479 3.263 3.548 4.1007 0.517 22.02 74.73 2.859 3.786 4.674 3.467 3.751 4.2988 0.175 14.29 65.77 3.050 3.974 4.855 3.655 3.938 4.4829 0.0562 8.859 56.27 3.229 4.149 5.026 3.832 4.113 4.65510 0.0172 5.265 46.79 3.396 4.314 5.187 3.998 4.279 4.81811 0.00504 3.008 37.81 3.556 4.471 5.340 4.156 4.436 4.97212 0.00142 1.656 29.71 3.707 4.620 5.486 4.307 4.585 5.12013 0.00038 0.881 22.71 3.853 4.764 5.626 4.451 4.729 5.26214 0.00010 0.453 16.89 3.992 4.902 5.762 4.590 4.867 5.39815 0.00003 0.226 12.25 4.126 5.034 5.892 4.723 5.000 5.53016 0.00001 0.1097 8.659 4.256 5.163 6.018 4.852 5.128 5.65717 � 0 0.0517 5.974 4.382 5.287 6.140 4.977 5.252 5.78018 � 0 0.0237 4.026 4.503 5.408 6.259 5.098 5.373 5.90019 � 0 0.0106 2.652 4.622 5.525 6.374 5.216 5.490 6.01620 � 0 0.00465 1.709 4.737 5.639 6.487 5.330 5.605 6.12925 � 0 0.00005 0.1404 5.272 6.170 7.012 5.864 6.136 6.65730 � 0 � 0 0.0074 5.755 6.650 7.486 6.345 6.616 7.13428.4 Random Number GenerationIn order to obtain random numbers from a multinormal distribution we proceed as follows:101

� If x = fx1; x2; : : : ; xng is distributed multinormally with mean 0 (zero vector) andvariance matrix I (unity matrix) then each xi (i = 1; 2; : : : ; n) can be found indepen-dently from a standard normal distribution.� If x is multinormally distributed with mean � and variance matrix V then any linearcombination y = Sx is also multinormally distributed with mean S� and variancematrix SV ST ,� If we want to generate vectors, y, from a multinormal distribution with mean � andvariance matrix V we may make a so called Cholesky decomposition of V , i.e. we �nda triangular matrix S such that V = SST . We then calculate y = Sx + � with thecomponents of x generated independently from a standard normal distribution.Thus we have found a quite nice way of generating multinormally distributed randomnumbers which is important in many simulations where correlations between variables maynot be ignored. If many random numbers are to be generated for multinormal variablesfrom the same distribution it is bene�cial to make the Cholesky decomposition once andstore the matrix S for further usage.

102

29 Negative Binomial Distribution29.1 IntroductionThe Negative Binomial distribution is given byp(r; k; p) = r � 1k � 1!pk(1� p)r�kwhere the variable r � k and the parameter k > 0 are integers and the parameter p(0 � p � 1) is a real number.The distribution expresses the probability of having to wait exactly r trials until ksuccesses have occurred if the probability of a success in a single trial is p (probability offailure q = 1 � p).The above form of the Negative Binomial distribution is often referred to as the Pascaldistribution after the french mathematician, physicist and philosopher Blaise Pascal (1623{1662).The distribution is sometimes expressed in terms of the number of failures occurringwhile waiting for k successes, n = r � k, in which case we writep(n; k; p) = n + k � 1n !pk(1� p)nwhere the new variable n � 0.Changing variables, for this last form, to n and k instead of p and k we sometimes usep(n;n; k) = n+ k � 1n ! nnkk(n+ k)n+k = n + k � 1n !� nn+ k�n kn+ k!kThe distribution may also be generalized to real values of k, although this may seemobscure from the above probability view-point (\fractional success"), writing the binomialcoe�cient as (n + k � 1)(n+ k � 2) � � � (k + 1)k=n!.29.2 MomentsIn the �rst form given above the expectation value, variance, third and fourth centralmoments of the distribution areE(r) = kp ; V (r) = kqp2 ; �3 = kq(2� p)p3 ; and �4 = kq(p2 � 6p + 6 + 3kq)p4The coe�cients of skewness and kurtosis are 1 = 2 � ppkq and 2 = p2 � 6p + 6kqIn the second formulation above, p(n), the only di�erence is that the expectation valuebecomes E(n) = E(r)� k = k(1� p)p = kqp103

while higher moments remain unchanged as they should since we have only shifted the scaleby a �xed amount.In the last form given, using the parameters n and k, the expectation value and thevariance are E(n) = n and V (n) = n+ n2k29.3 Probability Generating FunctionThe probability generating function is given byG(z) = pz1 � zq!kin the �rst case (p(r)) andG(z) = p1 � zq!k = 11 + (1� z)nk !kin the second case (p(n)) for the two di�erent parameterizations.29.4 Relations to Other DistributionsThere are several interesting connections between the Negative Binomial distribution andother standard statistical distributions. In the following subsections we brie y addresssome of these connections.29.4.1 Poisson DistributionRegard the negative binomial distribution in the formp(n;n; k) = n+ k � 1n ! 11 + n=k!k n=k1 + n=k!nwhere n � 0, k > 0 and n > 0.As k !1 the three terms become n+ k � 1n ! = (n+ k � 1)(n + k � 2) : : : kn! ! knn! ; 11 + n=k!k = 1� knk + k(k + 1)2 �nk�2 � k(k + 1)(k + 2)6 �nk�3 + : : :! e�n andkn n=k1 + n=k!n ! nnwhere, for the last term we have incorporated the factor kn from the �rst term.104

Thus we have shown that limk!1 p(n;n; k) = nne�nn!i.e. a Poisson distribution.This \proof" could perhaps better be made using the probability generating functionof the negative binomial distributionG(z) = p1 � zq!k = 11 � (z � 1)n=k!kMaking a Taylor expansion of this for (z � 1)n=k � 1 we getG(z) = 1 + (z � 1)n+ k + 1k (z � 1)2n22 + (k + 1)(k + 2)k2 (z � 1)3n36 + : : :! e(z�1)nas k ! 1. This result we recognize as the probability generating function of the Poissondistribution.29.4.2 Gamma DistributionRegard the negative binomial distribution in the formp(n; k; p) = n+ k � 1n !pkqnwhere n � 0, k > 0 and 0 � p � 1 and where we have introduced q = 1� p. If we changeparameters from k and p to k and n = kq=p this may be writtenp(n;n; k) = n+ k � 1n ! 11 + n=k!k n=k1 + n=k!nChanging variable from n to z = n=n we get (dn=dz = n)p(z;n; k) = p(n;n; k)dndz = n zn+ k � 1zn ! 11 + n=k!k n=k1 + n=k!zn == n (zn+ k � 1)(zn+ k � 2) : : : (zn+ 1)�(k) kk � 1k + n�k 1k=n + 1!zn !! n kk (zn)k�1�(k) � 1k + n�k 1k=n+ 1!zn == zk�1kk�(k) � nk + n�k 1k=n+ 1!zn ! zk�1kke�kz�(k)where we have used that for k � n!1� nk + n�k ! 1 and105

1k=n + 1!zn = 1 � znkn + zn(zn+ 1)2 kn!2 � zn(zn+ 1)(zn+ 2)6 kn!3 + : : :!! 1 � zk + z2k22 � z3k36 + : : : = e�kzas n!1.Thus we have \shown" that as n! 1 and n� k we obtain a gamma distribution inthe variable z = n=n.29.4.3 Logarithmic DistributionRegard the negative binomial distribution in the formp(n; k; p) = n+ k � 1n !pkqnwhere n � 0, k > 0 and 0 � p � 1 and where we have introduced q = 1� p.The probabilities for n = 0; 1; 2; 3::: are given byf p(0); p(1); p(2); p(3); : : :g = pk ( 1; kq; k(k + 1)2! q2; k(k + 1)(k + 2)3! q3; :::)if we omit the zero class (n=0) and renormalize we getkpk1� pk ( 0; q; k + 12! q2; (k + 1)(k + 2)3! q3; :::)and if we let k ! 0 we �nally obtain� 1ln p ( 0; q; q22 ; q33 ; :::)where we have used that limk!0 kp�k � 1 = � 1lnpwhich is easily realized expanding p�k = e�k ln p into a power series.This we recognize as the logarithmic distributionp(n; p) = � 1lnp (1� p)nnthus we have shown that omitting the zero class and letting k ! 0 the negative binomialdistribution becomes the logarithmic distribution.106

29.4.4 Branching ProcessIn a process where a branching occurs from a Poisson to a logarithmic distribution the mostelegant way to determine the resulting distribution is by use of the probability generatingfunction. The probability generating functions for a Poisson distribution with parameter(mean) � and for a logarithmic distribution with parameter p (q = 1� p) are given byGP (z) = e�(z�1) and GL(z) = ln(1 � zq)= ln(1� q) = � ln(1 � zq)where � > 0, 0 � q � 1 and � = 1= ln p.For a branching process in n stepsG(z) = G1(G2(: : :Gn�1(Gn(z)) : : :))where Gk(z) is the probability generating function in the k:th step. In the above case thisgives G(z) = GP (GL(z)) = exp f�(� ln(1� zq)� 1)g == expf�� ln(1� zq)� �g = (1 � zq)��e�� == (1� zq)�k(1� q)k = pk=(1 � zq)kwhere we have put k = ���. This we recognize as the probability generating function ofa negative binomial distribution with parameters k and p.We have thus shown that a Poisson distribution with mean � branching into a loga-rithmic distribution with parameter p gives rise to a negative binomial distribution withparameters k = ��� = ��= ln p and p (or n = kq=p).Conversely a negative binomial distribution with parameters k and p or n could arisefrom the combination of a Poisson distribution with parameter � = �k ln p = k ln(1 + nk )and a logarithmic distribution with parameter p and mean n=�.A particle physics example would be a charged multiplicity distribution arising from theproduction of independent clusters subsequently decaying into charged particles accordingto a logarithmic distribution. The UA5 experiment [36] found on the SppS collider atCERN that at a centre of mass energy of 540 GeV a negative binomial distribution withn = 28:3 and k = 3:69 �tted the data well. With the above scenario this would correspondto � 8 clusters being independently produced (Poisson distribution with � = 7:97) each onedecaying, according to a logarithmic distribution, into 3.55 charged particles on average.29.4.5 Poisson and Gamma DistributionsIf a Poisson distribution with mean � > 0p(n;�) = e���nn! for n � 0is weighted by a gamma distribution with parameters a > 0 and b > 0f(x; a; b) = a(ax)b�1e�ax�(b) for x > 0107

we obtainP(n) = 1Z0 p(n;�)f(�; a; b)d� = 1Z0 e���nn! a(a�)b�1e�a��(b) d� == abn!�(b) 1Z0 �n+b�1e��(a+1)d� = abn!(b� 1)!(n+ b� 1)!(a+ 1)�(n+b) == n + b� 1n !� aa+ 1�b � 1a+ 1�nwhich is a negative binomial distribution with parameters p = aa+1, i.e. q = 1 � p = 1a+1,and k = b. If we aim at a negative binomial distribution with parameters n and k we shouldthus weight a Poisson distribution with a gamma distribution with parameters a = k=nand b = k. This is the same as superimposing Poisson distributions with means comingfrom a gamma distribution with mean n.In the calculation above we have made use of integral tables for the integral1Z0 xne��xdx = n!��(n+1)29.5 Random Number GenerationIn order to obtain random numbers from a Negative Binomial distribution we may use therecursive formulap(r + 1) = p(r) qrr + 1� k or p(n + 1) = p(n)q(k + n)n+ 1for r = k; k + 1; : : : and n = 0; 1; : : : in the two cases starting with the �rst term (p(k) orp(0)) being equal to pk. This technique may be speeded up considerably, if p and k areconstants, by preparing a cumulative vector once for all.One may also use some of the relations described above such as the branching of aPoisson to a Logarithmic distribution6 or a Poisson distribution weighted by a Gammadistribution7. This, however, will always be less e�cient than the straightforward cumula-tive technique.6Generating random numbers from a Poisson distribution with mean � = �k ln p branching to a Log-arithmic distribution with parameter p will give a Negative Binomial distribution with parameters k andp. 7Taking a Poisson distribution with a mean distributed according to a Gamma distribution with pa-rameters a = k=n and b = k. 108

30 Non-central Beta-distribution30.1 IntroductionThe non-central Beta-distribution is given byf(x; p; q) = 1Xr=0 e��2 ��2�rr! xp+r�1(1 � x)q�1B (p + r; q)where p and q are positive real quantities and the non-centrality parameter � � 0.In �gure 21 we show examples of a non-central Beta distribution with p = 32 and q = 3varying the non-central parameter � from zero (an ordinary Beta distribution) to ten insteps of two.Figure 21: Graph of non-central Beta-distribution for p = 32, q = 3 and some values of �30.2 Derivation of distributionIf ym and yn are two independent variables distributed according to the chi-squared distri-bution with m and n degrees of freedom, respectively, then the ratio ym=(ym + yn) followsa Beta distribution with parameters p = m2 and q = n2 . If instead ym follows a non-centralchi-square distribution we may proceed in a similar way as was done for the derivation ofthe Beta-distribution (see section 4.2).We make a change of variables to x = ym=(ym+yn) and y = ym+yn which implies thatym = xy and yn = y(1� x) obtainingf(x; y) = ���������� @ym@x @ym@y@yn@x @yn@y ���������� f(ym; yn) == ���� y x�y 1 � x ���� 8><>: 1Xr=0 e��2 ��2�rr! �ym2 �m2 +r�1 e� ym22� �m2 + r� 9>=>;8><>:�yn2 �n2�1 e� yn22� �n2� 9>=>; =109

= y8><>: 1Xr=0 e��2 ��2�rr! �xy2 �m2 +r�1 e�xy22� �m2 + r� 9>=>;8><>:�y(1�x)2 �n2�1 e� y(1�x)22� �n2� 9>=>; == 1Xr=0 e��2 ��2�rr! xm2 +r�1(1 � x)n2�1B �m2 + r; n2� 8>><>>:�y2�m+n2 +r�1 e� y22� �m+n2 + r� 9>>=>>;In the last braces we see a chi-square distribution in y with m + n + 2r degrees offreedom and integrating f(x; y) over y in order to get the marginal distribution in x givesus the non-central Beta-distribution as given above with p = m=2 and q = n=2.If instead yn were distributed as a non-central chi-square distribution we would geta very similar expression (not amazing since ym=(ym + yn) = 1 � yn=(ym + yn)) but it'sthe form obtained when ym is non-central, that is normally referred to as the non-centralBeta-distribution.30.3 MomentsAlgebraic moments of the non-central Beta-distribution are given in terms of the hyperge-ometric function 2F2 asE(xk) = 1Z0 xkf(x; p; q)dx = 1Z0 1Xr=0 e��2 ��2�rr! xp+r+k�1(1 � x)q�1B (p+ r; q) dx == 1Xr=0 e��2 ��2�rr! B (p+ r + k; q)B (p+ r; q) = 1Xr=0 e��2 ��2�rr! � (p + r + k)� (p+ r) � (p + r + q)� (p+ r + q + k) == 1Xr=0 e��2 ��2�rr! (p + r + k � 1) � � � (p + r + 1)(p + r)(p+ q + r + k � 1) � � � (p + q + r + 1)(p + q + r) == e��2 � �(p + k)�(p) � �(p + q)�(p + q + k) � 2F2 �p + q; p+ k; p; p + q + k; �2�However, to evaluate the hypergeometric function involves a summation so it is more e�-cient to directly use the penultimate expression above.30.4 Cumulative distributionThe cumulative distribution is found by straightforward integrationF (x) = xZ0 1Xr=0 e��2 ��2�rr! up+r�1(1 � u)q�1B (p + r; q) du = 1Xr=0 e��2 ��2�rr! Ix (p+ r; q)30.5 Random Number GenerationRandom numbers from a non-central Beta-distribution with integer or half-integer p� andq�values is easily obtained using the de�nition above i.e. by using a random number froma non-central chi-square distribution and another from a (central) chi-square distribution.110

31 Non-central Chi-square Distribution31.1 IntroductionIf we instead of adding squares of n independent standard normal, N(0; 1), variables,giving rise to the chi-square distribution with n degrees of freedom, add squares of N(�i; 1)variables we obtain the non-central chi-square distributionf(x;n; �) = 1Xr=0 e��2 ��2�rr! f(x;n+ 2r) = 12n2 � � 12�xn2�1e� 12 (x+�) 1Xr=0 (�x)r(2r)! � � 12 + r�� �n2 + r�where � = P�2i is the non-central parameter and f(x;n) the ordinary chi-square distri-bution. As for the latter the variable x � 0 and the parameter n a positive integer. Theadditional parameter � � 0 and in the limit � = 0 we retain the ordinary chi-square dis-tribution. According to [2] pp 227{229 the non-central chi-square distribution was �rstintroduced by R. A. Fisher in 1928. In �gure 22 we show the distribution for n = 5 andnon-central parameter � = 0; 1; 2; 3; 4; 5 (zero corresponding to the ordinary chi-squareddistribution).Figure 22: Graph of non-central chi-square distribution for n = 5 and some values of �31.2 Characteristic FunctionThe characteristic function for the non-central chi-square distribution is given by�(t) = exp � {t�1�2{t�(1 � 2{t)n2but even more useful in determining moments isln�(t) = {t�1� 2{t � n2 ln(1 � 2{t)111

from which cumulants may be determined in a similar manner as we normally obtainalgebraic moments from �(t) (see below).By looking at the characteristic function one sees that the sum of two non-central chi-square variates has the same distribution with degrees of freedoms as well as non-centralparameters being the sum of the corresponding parameters for the individual distributions.31.3 MomentsTo use the characteristic function to obtain algebraic moments is not trivial but the cumu-lants (see section 2.5) are easily found to be given by the formula�r = 2r�1(r � 1)!(n+ r�) for r � 1from which we may �nd the lower order algebraic and central moments (with a = n + �and b = �=a) as�01 = �1 = a = n+ ��2 = �2 = 2a(1 + b) = 2(n + 2�)�3 = �3 = 8a(1 + 2b) = 8(n+ 3�)�4 = �4 + 3�22 = 48(n + 4�) + 12(n + 2�)2�5 = �5 + 10�3�2 = 384(n + 5�) + 160(n + 2�)(n + 3�)�6 = �6 + 15�4�2 + 10�23 + 15�32 == 3840(n + 6�) + 1440(n + 2�)(n + 4�) + 640(n + 3�)2 + 120(n + 2�)3 1 = � 21 + b�32 � 1 + 2bpa = 8(n+ 3�)[2(n + 2�)] 32 2 = 12a � 1 + 3b(1 + b)2 = 12(n + 4�)(n+ 2�)231.4 Cumulative DistributionThe cumulative, or distribution, function may be found byF (x) = 12n2� � 12�e��2 1Xr=0 �r(2r)! � � 12 + r�� �n2 + r� xZ0 un2+r�1e�u2 du == 12n2� � 12�e��2 1Xr=0 �r(2r)! � � 12 + r�� �n2 + r�2n2+r �n2 + r; x2� == e��2 1Xr=0 ��2�rr! P �n2 + r; x2�31.5 ApproximationsAn approximation to a chi-square distribution is found by equating the �rst two cumulantsof a non-central chi-square distribution with those of � times a chi-square distribution.112

Here � is a constant to be determined. The result is that with� = n+ 2�n+ � = 1 + �n+ � and n� = (n+ �)2n+ 2� = n+ �2n+ 2�we may approximate a non-central chi-square distribution f(x;n; �) with a (central) chi-square distribution in x=� with n� degrees of freedom (n� in general being fractional).Approximations to the standard normal distribution are given usingz = s 2x1 + b �s 2a1 + b � 1 or z = �xa� 13 � h1 � 29 � 1+ba iq29 � 1+ba31.6 Random Number GenerationRandom numbers from a non-central chi-square distribution is easily obtained using thede�nition above by e.g.� Put � = q�=n� Sum n random numbers from a normal distribution with mean � and variance unity.Note that this is not a unique choice. The only requirement is that � =P �2i .� Return the sum as a random number from a non-central chi-square distribution withn degrees of freedom and non-central parameter �.This ought to be su�cient for most applications but if needed more e�cient techniquesmay easily be developed e.g. using more general techniques.

113

32 Non-central F -Distribution32.1 IntroductionIf x1 is distributed according to a non-central chi-square distribution with m degrees offreedom and non-central parameter � and x2 according to a (central) chi-square distributionwith n degrees of freedom then, provided x1 and x2 are independent, the variableF 0 = x1=mx2=nis said to have a non-central F -distribution with m;n degrees of freedom (positive integers)and non-central parameter � � 0. As the non-central chi-square distribution it was �rstdiscussed by R. A. Fisher in 1928.This distribution in F 0 may be writtenf(F 0;m;n; �) = e��2 1Xr=0 1r! �2!r � �m+n2 + r�� �m2 + r�� �n2� �mn �m2 + r (F 0)m2 � 1 + r�1 + mF 0n � 12 (m+n)+rIn �gure 23 we show the non-central F -distribution for the case with m = 10 and n = 5varying � from zero (an ordinary, central, F -distribution) to �ve.Figure 23: Graph of non-central F -distribution for m = 10, n = 5 and some values of �When m = 1 the non-central F -distribution reduces to a non-central t2-distributionwith �2 = �. As n!1 then nF 0 approaches a non-central chi-square distribution with mdegrees of freedom and non-central parameter �.114

32.2 MomentsAlgebraic moments of the non-central F -distribution may be achieved by straightforward,but somewhat tedious, algebra asE(F 0k) = 1Z0 xkf(x;m;n; �)dx == e��2 � nm�k � �n2 � k�� �n2� 1Xr=0 1r! �2!r � �m2 + r + k�� �m2 + r�an expression which may be used to �nd lower order moments (de�ned for n > 2k)E(F 0) = nm � m+ �n� 2E(F 02) = � nm�2 1(n� 2)(n � 4) n�2 + (2� +m)(m+ 2)oE(F 03) = � nm�3 1(n� 2)(n � 4)(n� 6) � n�3 + 3(m+ 4)�2 + (3� +m)(m+ 4)(m+ 2)oE(F 04) = � nm�4 1(n� 2)(n � 4)(n� 6)(n� 8) � n�4 + 4(m + 6)�3 + 6(m+ 6)(m+ 4)�2++(4� +m)(m+ 6)(m+ 4)(m + 2)gV (F 0) = � nm�2 2(n� 2)(n � 4) ( (�+m)2n� 2 + 2�+m)32.3 Cumulative DistributionThe cumulative, or distribution, function may be found byF (x) = xZ0 ukf(u;m;n; �)du == e��2 1Xr=0 1r! �2!r � �m+n2 + r�� �m2 + r�� �n2� �mn �m2 +r xZ0 um2 �1+r+k�1 + mun �m+n2 +r du == e��2 1Xr=0 1r! �2!r Bq �m2 + r; n2�B �m2 + r; n2� = e��2 1Xr=0 ��2�rr! Iq �m2 + r; n2�with q = mxn1 + mxn115

32.4 ApproximationsUsing the approximation of a non-central chi-square distribution to a (central) chi-squaredistribution given in the previous section we see thatmm+ �F 0is approximately distributed according to a (central) F -distribution with m� = m+ �2m+2�and n degrees of freedom.Approximations to the standard normal distribution is achieved withz1 = F 0 � E(F 0)qV (F 0) = F 0 � n(m+�)m(n�2)nm h 2(n�2)(n�4) n(m+�)2n�2 +m+ 2�oi12or z2 = � mF 0m+�� 13 �1 � 29n�� �1� 29 � m+2�(m+�)2��29 � m+2�(m+�)2 + 29n � � mF 0m+�� 23 � 1232.5 Random Number GenerationRandom numbers from a non-central chi-square distribution is easily obtained using thede�nition above i.e. by using a random number from a non-central chi-square distributionand another from a (central) chi-square distribution.

116

33 Non-central t-Distribution33.1 IntroductionIf x is distributed according to a normal distribution with mean � and variance 1 and yaccording to a chi-square distribution with n degrees of freedom (independent of x) thent0 = xqy=nhas a non-central t-distribution with n degrees of freedom (positive integer) and non-centralparameter � (real).We may also write t0 = z + �qw=nwhere z is a standard normal variate and w is distributed as a chi-square variable with ndegrees of freedom.The distribution is given by (see comments on derivation in section below)f(t0;n; �) = e� �22pn�� �n2� 1Xr=0 (t0�)rr!n r2 1 + t02n !�n+r+12 2 r2� �n+r+12 �In �gure 24 we show the non-central t-distribution for the case with n = 10 varying �from zero (an ordinary t-distribution) to �ve.Figure 24: Graph of non-central t-distribution for n = 10 and some values of �This distribution is of importance in hypotheses testing if we are interested in theprobability of committing a Type II error implying that we would accept an hypothesisalthough it was wrong, see discussion in section 38.11 on page 147.117

33.2 Derivation of distributionNot many text-books include a formula for the non-central t-distribution and some turnsout to give erroneous expressions. A non-central F -distribution with m = 1 becomes anon-central t2-distribution which then may be transformed to a non-central t-distribution.However, with this approach one easily gets into trouble for t0 < 0. Instead we adopta technique very similar to what is used in section 38.6 to obtain the normal (central)t-distribution from a t-ratio.The di�erence in the non-central case is the presence of the �-parameter which intro-duces two new exponential terms in the equations to be solved. One is simply exp(��2=2)but another factor we treat by a serial expansion leading to the p.d.f. above. This may notbe the `best' possible expression but empirically it works quite well.33.3 MomentsWith some e�ort the p.d.f. above may be used to calculate algebraic moments of thedistribution yielding E(t0k) = e� �22p�� �n2�� �n�k2 �n k2 1Xr=0 �r2 r2r! � � r+k+12 �where the sum should be made for odd (even) values of r if k is odd (even). This gives forlow orders �01 = rn2 � �n�12 �� �n2� ��02 = n� �n�22 �2� �n2� �1 + �2� = nn � 2 �1 + �2��03 = n 32p2� �n�32 �4� �n2� � �3 + �2��04 = n2� �n�42 �4� �n2� ��4 + 6�2 + 3� = n2(n� 2)(n� 4) ��4 + 6�2 + 3�from which expressions for central moments may be found e.g. the variance�2 = V (t0) = n2 8><>:�1 + �2� 2n � 2 � �20@� �n�12 �� �n2� 1A29>=>;33.4 Cumulative DistributionThe cumulative, or distribution, function may be found byF (t) = e� �22pn�� �n2� 1Xr=0 �rr!n r2 2 r2� �n+r+12 � tZ�1 ur�1 + u2n �n+r+12 du =118

= e� �22p�� �n2� 1Xr=0 �rr! 2 r2�1� �n+r+12 � ns1B � r+12 ; n2�+ s2Bq � r+12 ; n2�o == e� �22p� 1Xr=0 �rr! 2 r2�1� � r+12 � ns1 + s2Iq � r+12 ; n2�owhere s1 are and s2 are signs di�ering between cases with positive or negative t as well asodd or even r in the summation. The sign s1 is �1 if r is odd and +1 if it is even while s2is +1 unless t < 0 and r is even in which case it is �1.33.5 ApproximationAn approximation is given by z = t0 �1 � 14n�� �q1 + t022nwhich is asymptotically distributed as a standard normal variable.33.6 Random Number GenerationRandom numbers from a non-central t-distribution is easily obtained using the de�nitionabove i.e. by using a random number from a normal distribution and another from a chi-square distribution. This ought to be su�cient for most applications but if needed moree�cient techniques may easily be developed e.g. using more general techniques.

119

34 Normal Distribution34.1 IntroductionThe normal distribution or, as it is often called, the Gauss distribution is the most impor-tant distribution in statistics. The distribution is given byf(x;�; �2) = 1�p2�e� 12(x��� )2where � is a location parameter, equal to the mean, and � the standard deviation. For� = 0 and � = 1 we refer to this distribution as the standard normal distribution. In manyconnections it is su�cient to use this simpler form since � and � simply may be regardedas a shift and scale parameter, respectively. In �gure 25 we show the standard normaldistribution.Figure 25: Standard normal distributionBelow we give some useful information in connection with the normal distribution.Note, however, that this is only a minor collection since there is no limit on important andinteresting statistical connections to this distribution.34.2 MomentsThe expectation value of the distribution is E(x) = � and the variance V (x) = �2.Generally odd central moments vanish due to the symmetry of the distribution andeven central moments are given by�2r = (2r)!2rr! �2r = (2r � 1)!!�2r120

for r � 1.It is sometimes also useful to evaluate absolute moments E(jxjn) for the normal distri-bution. To do this we make use of the integral1Z�1 e�ax2dx = r�awhich if di�erentiated k times with respect to a yields1Z�1 x2ke�ax2dx = (2k � 1)!!2k r �a2k+1In our case a = 1=2�2 and since even absolute moments are identical to the algebraicmoments it is enough to evaluate odd absolute moments for which we getE(jxj2k+1) = 2�p2� 1Z0 x2k+1e� x22�2 dx = s2� (2�2)k+12� 1Z0 yke�ydyThe last integral we recognize as being equal to k! and we �nally obtain the absolutemoments of the normal distribution asE(jxjn) = ( (n� 1)!!�n for n = 2kq 2�2kk!�2k+1 for n = 2k + 1The half-width at half-height of the normal distribution is given by p2 ln 2� � 1:177�which may be useful to remember when estimating � using a ruler.34.3 Cumulative FunctionThe distribution function, or cumulative function, may be expressed in term of the incom-plete gamma function P asF (z) = 8<: 12 + 12P �12 ; z22 � if z � 012 � 12P �12 ; z22 � if z < 0or we may use the error function erf(z=p2) in place of the incomplete gamma function.34.4 Characteristic FunctionThe characteristic function for the normal distribution is easily found from the generalde�nition �(t) = E �e{tx� = expn�{t� 12�2t2o121

34.5 Addition TheoremThe so called Addition theorem for normally distributed variables states that any linearcombination of independent normally distributed random variables xi (i = 1; 2; : : : ; n) isalso distributed according to the normal distribution.If each xi is drawn from a normal distribution with mean �i and variance �2i then regardthe linear combination S = nXi=1 aixiwhere ai are real coe�cients. Each term aixi has characteristic function�aixi(t) = expn(ai�i){t� 12(a2i�2i )t2oand thus S has characteristic function�S(t) = nYi=1�aixi(t) = exp( nXi=1 ai�i! {t� 12 nXi=1 a2i�2i! t2)which is seen to be a normal distribution with mean P ai�i and variance P a2i�2i .34.6 Independence of x and s2A unique property of the normal distribution is the independence of the sample statisticsx and s2, estimates of the mean and variance of the distribution. Recall that the de�nitionof these quantities are x = 1n nXi=1 xi and s2 = 1n� 1 nXi=1(xi � x)2where x is an estimator of the true mean � and s2 is the usual unbiased estimator for thetrue variance �2.For a population of n events from a normal distribution x has the distributionN(�; �2=n)and (n� 1)s2=�2 is distributed according to a chi-square distribution with n� 1 degrees offreedom. Using the relationnXi=1 �xi � �� �2 = (n� 1)s2�2 + x� ��=pn!2and creating the joint characteristic function for the variables (n � 1)s2=�2 and (pn(x ��)=�2)2 one may show that this function factorizes thus implying independence of thesequantities and thus also of x and s2.In summary the \independence theorem" states that given n independent random vari-ables with identical normal distributions the two statistics x and s2 are independent. Alsoconversely it holds that if the mean x and the variance s2 of a random sample are indepen-dent then the population is normal. 122

34.7 Probability ContentThe probability content of the normal distribution is often referred to in statistics. Whenthe term one standard deviation is mentioned one immediately thinks in terms of a proba-bility content of 68.3% within the symmetric interval from the value given.Without loss of generality we may treat the standard normal distribution only sincethe transformation from a more general case is straightforward putting z = (x� �)=�. Indi�erent situation one may want to �nd� the probability content, two-side or one-sided, to exceed a certain number of standarddeviations, or� the number of standard deviations corresponding to a certain probability content.In calculating this we need to evaluate integrals like� = Z 1p2�e� t22 dtThere are no explicit solution to this integral but it is related to the error function (seesection 13) as well as the incomplete gamma function (see section 42).1p2� zZ�z e� z22 = erf zp2! = P 12 ; z22 !These relations may be used to calculate the probability content. Especially the errorfunction is often available as a system function on di�erent computers. Beware, however,that it seems to be implemented such that erf(z) is the symmetric integral from �z to zand thus the p2 factor should not be supplied. Besides from the above relations there arealso excellent approximations to the integral which may be used.In the tables below we give the probability content for exact z-values (left-hand table)as well as z-values for exact probability contents (right-hand table).z zR�1 zR�z 1Rz0.0 0.50000 0.00000 0.500000.5 0.69146 0.38292 0.308541.0 0.84134 0.68269 0.158661.5 0.93319 0.86639 0.066812.0 0.97725 0.95450 0.022752.5 0.99379 0.98758 6:210 � 10�33.0 0.99865 0.99730 1:350 � 10�33.5 0.99977 0.99953 2:326 � 10�44.0 0.99997 0.99994 3:167 � 10�54.5 1.00000 0.99999 3:398 � 10�65.0 1.00000 1.00000 2:867 � 10�76.0 1.00000 1.00000 9:866 � 10�107.0 1.00000 1.00000 1:280 � 10�128.0 1.00000 1.00000 6:221 � 10�16z zR�1 zR�z 1Rz0.00000 0.5 0.0 0.50.25335 0.6 0.2 0.40.67449 0.75 0.5 0.250.84162 0.8 0.6 0.21.28155 0.9 0.8 0.11.64485 0.95 0.9 0.051.95996 0.975 0.95 0.0252.32635 0.99 0.98 0.012.57583 0.995 0.99 0.0053.09023 0.999 0.998 0.0013.29053 0.9995 0.999 0.00053.71902 0.9999 0.9998 0.00013.89059 0.99995 0.9999 0.000054.26489 0.99999 0.99998 0.00001123

It is sometimes of interest to scrutinize extreme signi�cance levels which implies inte-grating the far tails of a normal distribution. In the table below we give the number ofstandard deviations, z, required in order to achieve a one-tailed probability content of 10�n.z-values for which 1p2� 1Rz e�z2=2dz = 10�n for n = 1; 2; : : : ; 23n z n z n z n z n z1 1.28155 6 4.75342 11 6.70602 16 8.22208 21 9.505022 2.32635 7 5.19934 12 7.03448 17 8.49379 22 9.741793 3.09023 8 5.61200 13 7.34880 18 8.75729 23 9.973054 3.71902 9 5.99781 14 7.65063 19 9.013275 4.26489 10 6.36134 15 7.94135 20 9.26234Below are also given the one-tailed probability content for a standard normal distribu-tion in the region from z to 1 (or �1 to �z). The information in the previous as well asthis table is taken from [26].Probability content Q(z) = 1p2� 1Rz e�z2=2dz for z = 1; 2; : : : ; 50; 60; : : : ; 100; 150; : : : ; 500z � logQ(z) z � logQ(z) z � logQ(z) z � logQ(z) z � logQ(z)1 0.79955 14 44.10827 27 160.13139 40 349.43701 80 1392.044592 1.64302 15 50.43522 28 172.09024 41 367.03664 90 1761.246043 2.86970 16 57.19458 29 184.48283 42 385.07032 100 2173.871544 4.49934 17 64.38658 30 197.30921 43 403.53804 150 4888.388125 6.54265 18 72.01140 31 210.56940 44 422.43983 200 8688.589776 9.00586 19 80.06919 32 224.26344 45 441.77568 250 13574.499607 11.89285 20 88.56010 33 238.39135 46 461.54561 300 19546.127908 15.20614 21 97.48422 34 252.95315 47 481.74964 350 26603.480189 18.94746 22 106.84167 35 267.94888 48 502.38776 400 34746.5597010 23.11805 23 116.63253 36 283.37855 49 523.45999 450 43975.3686011 27.71882 24 126.85686 37 299.24218 50 544.96634 500 54289.9083012 32.75044 25 137.51475 38 315.53979 60 783.9074313 38.21345 26 148.60624 39 332.27139 70 1066.26576Beware, however, that extreme signi�cance levels are purely theoretical and that oneseldom or never should trust experimental limits at these levels. In an experimental situa-tions one rarely ful�lls the statistical laws to such detail and any bias or background mayheavily a�ect statements on extremely small probabilities.Although one normally would use a routine to �nd the probability content for a normaldistribution it is sometimes convenient to have a \classical" table available. In table 6 onpage 178 we give probability contents for a symmetric region from �z to z for z-valuesranging from 0.00 to 3.99 in steps of 0.01. Conversely we give in table 7 on page 179 thez-values corresponding to speci�c probability contents from 0.000 to 0.998 in steps of 0.002.124

34.8 Random Number GenerationThere are many di�erent methods to obtain random numbers from a normal distributionsome of which are reviewed below. It is enough to consider the case of a standard normaldistribution since given such a random number z we may easily obtain one from a generalnormal distribution by making the transformation x = � + �z.Below f(x) denotes the standard normal distribution and if not explicitly stated allvariables denoted by � are uniform random numbers in the range from zero to one.34.8.1 Central Limit Theory ApproachThe sum of n independent random numbers from a uniform distribution between zero andone, Rn, has expectation value E(Rn) = n=2 and variance V (Rn) = n=12. By the centrallimit theorem the quantity zn = Rn � E(Rn)qV (Rn) = Rn � n2q n12approaches the standard normal distribution as n!1. A practical choice is n = 12 sincethis expression simpli�es to z12 = R12 � 6 which could be taken as a random number froma standard normal distribution. Note, however, that this method is neither accurate norfast.34.8.2 Exact TransformationThe Box-Muller transformation used to �nd random numbers from the binormal distribu-tion (see section 6.5 on page 22), using two uniform random numbers between zero andone in �1 and �2, z1 = q�2 ln �1 sin 2��2z2 = q�2 ln �1 cos 2��2may be used to obtain two independent random numbers from a standard normal distri-bution.34.8.3 Polar MethodThe above method may be altered in order to avoid the cosine and sine byi Generate u and v as two uniformly distributed random numbers in the range from -1to 1 by u = 2�1 � 1 and v = 2�2 � 1.ii Calculate w = u2 + v2 and if w > 1 then go back to i.iii Return x = uz and y = vz with z = q�2 lnw=w as two independent random numbersfrom a standard normal distribution. 125

This method is often faster than the previous since it eliminates the sine and cosineat the slight expense of 1 � �=4 � 21% rejection in step iii and a few more arithmeticoperations. As is easily seen u=pw and v=pw plays the role of the cosine and the sine inthe previous method.34.8.4 Trapezoidal MethodThe maximum trapezoid that may be inscribed under the standard normal curve covers anarea of 91.95% of the total area. Random numbers from a trapezoid is easily obtained by alinear combination of two uniform random numbers. In the remaining cases a tail-techniqueand accept-reject techniques, as described in �gure 26, are used.Figure 26: Trapezoidal methodBelow we describe, in some detail, a slightly modi�ed version of what is presented in[28]. For more exact values of the constants used see this reference.i Generate two uniform random numbers between zero and one � and �0ii If � < 0:9195 generate a random number from the trapezoid by x = 2:404�0+1:984��2:114 and exitiii Else if � < 0:9541 (3.45% of all cases) generate a random number from the tail x > 2:114a Generate two uniform random numbers �1 and �2b Put x = 2:1142 � 2 ln �1 and if x�22 > 2:1142 then go back to ac Put x = px and go to viiiv Else if � < 0:9782 ( 2.41% of all cases) generate a random number from the region0:290 < x < 1:840 between the normal curve and the trapezoida Generate two uniform random numbers �1 and �2b Put x = 0:290+ 1:551�1 and if f(x)� 0:443+ 0:210x < 0:016�2 then go to a126

c Go to viiv Else if � < 0:9937 ( 1.55% of all cases) generate a random number from the region1:840 < � < 2:114 between the normal curve and the trapezoida Generate two uniform random numbers �1 and �2b Put x = 1:840+ 0:274�1 and if f(x)� 0:443+ 0:210x < 0:043�2 then go to ac Go to viivi Else, in 0.63% of all cases, generate a random number from the region 0 < x < 0:290between the normal curve and the trapezoid bya Generate two uniform random numbers �1 and �2b Put x = 0:290�1 and if f(x)� 0:383 < 0:016�2 then go back to avii Assign a minus sign to x if �0 � 1234.8.5 Center-tail methodAhrens and Dieter [28] also proposes a so called center-tail method. In their article theytreat the tails outside jzj > p2 with a special tail method which avoids the logarithm.However, it turns out that using the same tail method as in the previous method is evenfaster. The method is as follows:i Generate a uniform random number � and use the �rst bit after the decimal point asa sign bit s i.e. for � � 12 put � = 2� and s = �1 and for � > 12 put � = 2� � 1 ands = 1ii If � > 0:842700792949715 (the area for �p2 < z < p2) go to vi.iii Center method: Generate �0 and set � = � + 0iv Generate �1 and �2 and set �� = max(�1; �2).If � < �� calculate y = �0p2 and go to viiiv Generate �1 and �2 and set � = max(�1; �2)If � < �� go to iv else go to iiivi Tail method: Generate �1 and set y = 1 � ln �1vii Generate �2 and if y�22 > 1 go to vi else put y = pyviii Set x = syp2. 127

34.8.6 Composition-rejection MethodsIn reference [21] two methods using the composition-rejection method is proposed. The�rst one, attributed to Butcher [23] and Kahn, uses only one term in the sum and has� = q2e=�, f(x) = exp f�xg and g(x) = exp f�(x� 1)2=2g. The algorithm is as follows:i Generate �1 and �2ii Determine x = � ln �1, i.e. a random number from f(x)iii Determine g(x) = exp f�(x� 1)2=2giv If �2 > g(x) then go to iv Decide a sign either by generating a new random number, or by using �2 for which0 < �2 � g(x) here, and exit with x with this sign.The second method is originally proposed by J. C. Butcher [23] and uses two terms�1 = q 2� f1(x) = 1 g1(x) = e�x22 for 0 � x � 1�2 = 1=p2� f2(x) = 2e�2(x�1) g2(x) = e� (x�2)22 for x > 1i Generate �1 and �2ii If �1 � 23 > 0 then determine x = 1� 12 ln(3�1 � 2) and z = 12(x� 2)2 else determinex = 3�1=2 and z = x2=2iii Determine g = e�ziv If �2 > g the go to iv Determine the sign of �2 � g=2 and exit with x with this sign.34.8.7 Method by MarsagliaA nice method proposed by G. Marsaglia is based on inscribing a spline function beneath thestandard normal curve and subsequently a triangular distribution beneath the remainingdi�erence. See �gure 27 for a graphical presentation of the method. The algorithm used isdescribed below.� The sum of three uniform random numbers �1, �2, and �3 follow a parabolic splinefunction. Using x = 2(�1 + �2 + �3 � 32) we obtain a distributionf1(x) = 8><>: (3� x2)=8 if jxj � 1(3� jxj)2=16 if 1 < jxj � 30 if jxj > 3Maximizing �1 with the constraint f(x) � �1f1(x) � 0 in the full interval jxj � 3gives �1 = 16e�2=p2� � 0:8638554 i.e. in about 86% of all cases such a combinationis made. 128

Figure 27: Marsaglia method� Moreover, a triangular distribution given by making the combination x = 32(�1+�2�1)leading to a function f2(x) = 49 �32 � jxj� for jxj < 32and zero elsewhere. This function may be inscribed under the remaining curvef(x) � f1(x) maximizing �2 such that f3(x) = f(x) � �1f1(x) � �2f2(x) � 0 inthe interval jxj � 32 . This leads to a value �2 � 0:1108 i.e. in about 11% of all casesthis combination is used� The maximum value of f3(x) in the region jxj � 3 is 0.0081 and here we use astraightforward reject-accept technique. This is done in about 2.26% of all cases.� Finally, the tails outside jxj > 3, covering about 0.27% of the total area is dealt withwith a standard tail-method wherea Put x = 9 � 2 ln �1b If x�22 > 9 then go to ac Else generate a sign s = +1 or s = �1 with equal probability and exit withx = spx34.8.8 Histogram TechniqueYet another method due to G. Marsaglia and collaborators [37] is one where a histogramwith k bins and bin-width c is inscribed under the (folded) normal curve. The di�erencebetween the normal curve and the histogram is treated with a combination of triangulardistributions and accept-reject techniques as well as the usual technique for the tails. Tryingto optimize fast generation we found k = 9 and c = 13 to be a fair choice. This may, however,not be true on all computers. See �gure 28 for a graphical presentation of the method.The algorithm is as follows: 129

Figure 28: Histogram methodi Generate �1 and chose which region to generate from. This is done e.g. with asequential search in a cumulative vector where the areas of the regions have beensorted in descending order. The number of elements in this vector is 2k+ 1c +1 whichfor the parameters mentioned above becomes 22.ii If a histogram bin i (i = 1; 2; : : : ; k) is selected then determine x = (�2 + i� 1)c andgo to vii.iii If an inscribed triangle i (i = 1; 2; : : : ; 1c ) then determine x = (min(�2; �3) + 1 � i)cand go to vii.iv If subscribed triangle i (i = 1c+1; : : : ; k) then determine x = (min(�2; �3)+i�1)c andaccept this value with a probability equal to the ratio between the normal curve andthe triangle at this x-value (histogram subtracted in both cases) else iterate. Whena value is accepted then go to vii.v For the remaining 1c regions between the inscribed triangles and the normal curve forx < 1 use a standard reject accept method in each bin and then go to vii.vi If the tail region is selected then use a standard technique e.g. (a) x = (kc)2�2 ln �2,(b) if x�23 > (kc)2 then go to a else use x = px.vii Attach a random sign to x and exit. This is done by either generating a new uniformrandom number or by saving the �rst bit of �1 in step i. The latter is faster and thedegradation in precision is negligible.34.8.9 Ratio of Uniform DeviatesA technique using the ratio of two uniform deviates was propose by A. J. Kinderman andJ. F. Monahan in 1977 [38]. It is based on selecting an acceptance region such that the ratioof two uniform pseudorandom numbers follow the standard normal distribution. With u130

and v uniform random numbers, u between 0 and 1 and v between �q2=e and q2=e, sucha region is de�ned by v2 < �4 u2 lnuas is shown in the left-hand side of �gure 29.

Figure 29: Method using ratio between two uniform deviatesNote that it is enough to consider the upper part (v > 0) of the acceptance limit due tothe symmetry of the problem. In order to avoid taking the logarithm, which may slow thealgorithm down, simpler boundary curves were designed. An improvement to the originalproposal was made by Joseph L. Leva in 1992 [39,40] choosing the same quadratic form forboth the lower and the upper boundary namelyQ(u; v) = (u� s)2 � b(u� s)(v � t) + (a� v)2Here (s; t) = (0:449871; -0:386595) is the center of the ellipses and a = 0:196 and b = 0:25472are suitable constants to obtain tight boundaries. In the right-hand side of �gure 29 weshow the value of the quadratic form at the acceptance limitQ(u; 2up� lnu) as a functionof u. It may be deduced that only in the interval r1 < Q < r2 with r1 = 0:27597 andr2 = 0:27846 we still have to evaluate the logarithm.The algorithm is as follows:i Generate uniform random numbers u = �1 and v = 2q2=e(�2 � 12).ii Evaluate the quadratic form Q = x2 + y(ay � bx) with x = u� s and y = jvj � t.131

iii Accept if inside inner boundary, i.e. if Q < r1, then go to vi.iv Reject if outside upper boundary, i.e. if Q > r2, then go to i.v Reject if outside acceptance region, i.e. if v2 > �4u2 lnu, then go to i.vi Return the ratio v=u as a pseudorandom number from a standard normal distribution.On average 2:738 uniform random numbers are consumed and 0:012 logarithms are com-puted per each standard normal random number obtained by this algorithm. As a com-parison the number of logarithmic evaluations without cutting on the boundaries, skippingsteps ii through iv above, would be 1:369. The penalty when using logarithms on moderncomputers is not as severe as it used to be but still some e�ciency is gained by using theproposed algorithm.34.8.10 Comparison of random number generatorsAbove we described several methods to achieve pseudorandom numbers from a standardnormal distribution. Which one is the most e�cient may vary depending on the actualimplementation and the computer it is used at. To give a rough idea we found the followingtimes per random number8 (in the table are also given the average number of uniformpseudorandom numbers consumed per random number in our implementations)Method section �s/r.n. N�/r.n. commentTrapezoidal method 34.8.4 0.39 2.246Polar method 34.8.3 0.41 1.273 pairHistogram method 34.8.8 0.42 2.121Box-Muller transformation 34.8.2 0.44 1.000 pairSpline functions 34.8.7 0.46 3.055Ratio of two uniform deviates 34.8.9 0.55 2.738Composition-rejection, two terms 34.8.6 0.68 2.394Center-tail method 34.8.5 0.88 5.844Composition-rejection, one term 34.8.6 0.90 2.631Central limit method approach 34.8.1 1.16 12.000 inaccurateThe trapezoidal method is thus fastest but the di�erence is not great as compared tosome of the others. The central limit theoremmethod is slow as well as inaccurate althoughit might be the easiest to remember. The other methods are all exact except for possiblenumerical problems. "Pair" indicates that these generators give two random numbers at atime which may implies that either one is not used or one is left pending for the next call(as is the case in our implementations).8The timing was done on a Digital Personal Workstation 433au workstation running Unix version 4.0Dand all methods were programmed in standard Fortran as functions giving one random number at eachcall. 132

34.9 Tests on Parameters of a Normal DistributionFor observations from a normal sample di�erent statistical distributions are applicable indi�erent situations when it comes to estimating one or both of the parameters � and �. Inthe table below we try to summarize this in a condensed form.TESTS OF MEAN AND VARIANCE OF NORMAL DISTRIBUTIONH0 Condition Statistic Distribution� = �0 �2 known x��0�=pn N(0; 1)�2 unknown x��0s=pn tn�1�2 = �20 � known (n�1)s2�20 = nPi=1 (xi��)2�20 �2n� unknown (n�1)s2�20 = nPi=1 (xi�x)2�20 �2n�1�1 = �2 = � �21 = �22 = �2 known x�y�p 1n+ 1m N(0; 1)�21 6= �22 known x�yr�21n +�22m N(0; 1)�21 = �22 = �2 unknown x�ysp 1n+ 1m tn+m�2s = (n�1)s21+(m�1)s22n+m�2�21 6= �22 unknown x�yr s21n + s22m � N(0; 1)�21 = �22 �1 6= �2 known s21s22 = 1n�1 nPi=1(xi��1)21m�1 mPi=1(yi��2)2 Fn;m�1 6= �2 unknown s21s22 = 1n�1 nPi=1(xi�x)21m�1 mPi=1(yi�y)2 Fn�1;m�1133

35 Pareto Distribution35.1 IntroductionThe Pareto distribution is given byf(x;�; k) = �k�=x�+1where the variable x � k and the parameter � > 0 are real numbers. As is seen k is onlya scale factor.The distribution has its name after its inventor the italian Vilfredo Pareto (1848{1923)who worked in the �elds of national economy and sociology (professor in Lausanne, Switzer-land). It was introduced in order to explain the distribution of wages in society.35.2 Cumulative DistributionThe cumulative distribution is given byF (x) = xZk f(u)du = 1 � kx!�35.3 MomentsAlgebraic moments are given byE(xn) = 1Zk xnf(x) = 1Zk xn �k�x�+1 = "� �k�x��n+1#1k = �k��� nwhich is de�ned for � > n.Especially the expectation value and variance are given byE(x) = �k� � 1 for � > 1V (x) = �k2(� � 2)(�� 1)2 for � > 235.4 Random NumbersTo obtain a random number from a Pareto distribution we use the straightforward way ofsolving the equation F (x) = � with � a random number uniformly distributed between zeroand one. This gives F (x) = 1 � kx!� = � ) x = k(1 � �) 1�134

36 Poisson Distribution36.1 IntroductionThe Poisson distribution is given by p(r;�) = �re��r!where the variable r is an integer (r � 0) and the parameter � is a real positive quantity.It is named after the french mathematician Sim�eon Denis Poisson (1781{1840) who wasthe �rst to present this distribution in 1837 (implicitly the distribution was known alreadyin the beginning of the 18th century).As is easily seen by comparing two subsequent r-values the distribution increases up tor + 1 < � and then declines to zero. For low values of � it is very skewed (for � < 1 it isJ-shaped).The Poisson distribution describes the probability to �nd exactly r events in a givenlength of time if the events occur independently at a constant rate �. An unbiased ande�cient estimator of the Poisson parameter � for a sample with n observations xi is �̂ = �x,the sample mean, with variance V (�̂) = �=n.For � ! 1 the distribution tends to a normal distribution with mean � and variance�. The Poisson distribution is one of the most important distributions in statistics withmany applications. Along with the properties of the distribution we give a few exampleshere but for a more thorough description we refer to standard text-books.36.2 MomentsThe expectation value, variance, third and fourth central moments of the Poisson distribu-tion are E(r) = �V (r) = ��3 = ��4 = �(1 + 3�)The coe�cients of skewness and kurtosis are 1 = 1=p� and 2 = 1=� respectively, i.e.they tend to zero as � ! 1 in accordance with the distribution becoming approximatelynormally distributed for large values of �.Algebraic moments may be found by the recursive formula�0k+1 = �(�0k + d�0kd� )and central moments by a similar formula�k+1 = �(k�k�1 + d�kd� )135

For a Poisson distribution one may note that factorial moments gk (cf page 6) andcumulants �k (see section 2.5) become especially simplegk = E(r(r � 1) � � � (r � k + 1)) = �k�r = � for all r � 136.3 Probability Generating FunctionThe probability generating function is given byG(z) = E(zr) = 1Xr=0 zr�re��r! = e�� 1Xr=0 (�z)r! = e�(z�1)Although we mostly use the probability generating function in the case of a discrete distri-bution we may also de�ne the characteristic function�(t) = E(e{tr) = e�� 1Xr=0 e{tr�rr! = expn� �e{t � 1�oa result which could have been given directly since �(t) = G(e{t).36.4 Cumulative DistributionWhen calculating the probability content of a Poisson distribution we need the cumulative,or distribution, function. This is easily obtained by �nding the individual probabilities e.g.by the recursive formula p(r) = p(r � 1)�r starting with p(0) = e��.There is, however, also an interesting connection to the incomplete Gamma function[10] P (r) = rXk=0 �ke��k! = 1� P (r + 1; �)with P (a; x) the incomplete Gamma function not to be confused with P (r).Since the cumulative chi-square distribution also has a relation to the incompleteGamma function one may obtain a relation between these cumulative distributions namelyP (r) = rXk=0 �ke��k! = 1 � 2�Z0 f(x; � = 2r + 2)dxwhere f(x; � = 2r + 2) denotes the chi-square distribution with � degrees of freedom.36.5 Addition TheoremThe so called addition theorem states that the sum of any number of independent Poisson-distributed variables is also distributed according to a Poisson distribution.For n variables each distributed according to the Poisson distribution with parameters(means) �i we �nd characteristic function�(r1 + r2 + : : :+ rn) = nYi=1 expn�i �e{t � 1�o = exp( nXi=1 �i �e{t � 1�)which is the characteristic function for a Poisson variable with parameter � =P �i.136

36.6 Derivation of the Poisson DistributionFor a binomial distribution the rate of \success" p may be very small but in a long series oftrials the total number of successes may still be a considerable number. In the limit p! 0and N !1 but with Np = � a �nite constant we �ndp(r) = Nr !pr(1� p)N�r � 1r! p2�NNNe�Nq2�(N � r)(N � r)N�re�(N�r) � �N �r �1� �N �N�r == 1r!s NN � r 1�1� rN �N e�r�r �1 � �N �N�r ! �re��r!as N !1 and where we have used that limn!1(1� xn )n = e�x and Stirling's formula (sesection 42.2) for the factorial of a large number n! � p2�n nn e�n.It was this approximation to the binomial distribution which S. D. Poisson presentedin his book in 1837.36.7 HistogramIn a histogram of events we would regard the distribution of the bin contents as multi-nomially distributed if the total number of events N were regarded as a �xed number.If, however, we would regard the total number of events not as �xed but as distributedaccording to a Poisson distribution with mean � we obtain (with k bins in the histogramand the multinomial probabilities for each bin in the vector p)Given a multinomial distribution, denotedM(r;N; p), for the distribution of events intobins for �xed N and a Poisson distribution, denoted P (N ; �), for the distribution of N wewrite the joint distributionP(r;N) = M(r;N; p)P (N ; �) = N !r1!r2! : : : rk!pr11 pr22 : : : prkk ! �Ne��N ! ! == � 1r1! (�p1)r1e��p1�� 1r2! (�p2)r2e��p2� : : :� 1rk! (�pk)rke��pk�where we have used that kXi=1 pi = 1 and kXi=1 ri = Ni.e. we get a product of independent Poisson distributions with means �pi for each individualbin. A simpler case leading to the same result would be the classi�cation into only twogroups using a binomial and a Poisson distribution.The assumption of independent Poisson distributions for the number events in each binis behind the usual rule of using pN as the standard deviation in a bin with N entries andneglecting correlations between bins in a histogram.137

36.8 Random Number GenerationBy use of the cumulative technique e.g. forming the cumulative distribution by startingwith P (0) = e�� and using the recursive formulaP (r) = P (r � 1)�ra random number from a Poisson distribution is easily obtained using one uniform randomnumber between zero and one. If � is a constant the by far fastest generation is obtainedif the cumulative vector is prepared once for all.An alternative is to obtain, in �, a random number from a Poisson distribution bymultiplying independent uniform random numbers �i until�Yi=0 �i � e��For large values of � use the normal approximation but beware of the fact that thePoisson distribution is a function in a discrete variable.

138

37 Rayleigh Distribution37.1 IntroductionThe Rayleigh distribution is given byf(x;�) = x�2 e� x22�2for real positive values of the variable x and a real positive parameter �. It is named afterthe british physicist Lord Rayleigh (1842{1919), also known as Baron John William StruttRayleigh of Terling Place and Nobel prize winner in physics 1904.Note that the parameter � is simply a scale factor and that the variable y = x=� hasthe simpli�ed distribution g(y) = ye�y2=2.Figure 30: The Rayleigh distributionThe distribution, shown in �gure 30, has a mode at x = � and is positively skewed.37.2 MomentsAlgebraic moments are given byE(xn) = 1Z0 xnf(x)dx = 12�2 1Z�1 jxjn+1e�x2=2�2i.e. we have a connection to the absolute moments of the Gauss distribution. Using these(see section 34 on the normal distribution) the result isE(xn) = (q�2n!!�n for n odd2kk!�2k for n = 2k139

Speci�cally we note that the expectation value, variance, and the third and fourthcentral moments are given byE(x) = �r�2 ; V (x) = �2 �2 � �2� ; �3 = �3(� � 3)r�2 ; and �4 = �4 8 � 3�24 !The coe�cients of skewness and kurtosis is thus 1 = (� � 3)q�2�2� �2� 32 � 0:63111 and 2 = 8 � 3�24�2 � �2�2 � 3 � 0:2450937.3 Cumulative DistributionThe cumulative distribution, or the distribution function, is given byF (x) = xZ0 f(y)dy = 1a2 xZ0 ye� y22�2 dy = x22�2Z0 e�zdz = 1 � e� x22�2where we have made the substitution z = y22�2 in order to simplify the integration. As itshould we see that F (0) = 0 and F (1) = 1.Using this we may estimate the medianM byF (M) = 12 )M = �p2 ln 2 � 1:17741�and the lower and upper quartiles becomesQ1 = �q�2 ln 34 � 0:75853� and Q3 = �p2 ln 4 � 1:66511�and the same technique is useful when generating random numbers from the Rayleighdistribution as is described below.37.4 Two-dimensional Kinetic TheoryGiven two independent coordinates x and y from normal distributions with zero mean andthe same variance �2 the distance z = px2 + y2 is distributed according to the Rayleighdistribution. The x and y may e.g. be regarded as the velocity components of a particlemoving in a plane.To realize this we �rst write w = z2�2 = x2�2 + y2�2Since x=� and y=� are distributed as standard normal variables the sum of their squareshas the chi-squared distribution with 2 degrees of freedom i.e. g(w) = e�w=2=2 from whichwe �nd f(z) = g(w) �����dwdz ����� = g z2�2! 2z�2 = z�2e� z22�2which we recognize as the Rayleigh distribution. This may be compared to the three-dimensional case where we end up with the Maxwell distribution.140

37.5 Random Number GenerationTo obtain random numbers from the Rayleigh distribution in an e�cient way we make thetransformation y = x2=2�2 a variable which follow the exponential distribution g(y) = e�y.A random number from this distribution is easily obtained by taking minus the naturallogarithm of a uniform random number. We may thus �nd a random number r from aRayleigh distribution by the expressionr = �q�2 ln �where � is a random number uniformly distributed between zero and one.This could have been found at once using the cumulative distribution puttingF (x) = � ) 1� e� x22�2 = � ) x = �q�2 ln(1 � �)a result which is identical since if � is uniformly distributed between zero and one so is1� �.Following the examples given above we may also have used two independent randomnumbers from a standard normal distribution, z1 and z2, and constructr = 1�qz21 + z22However, this technique is not as e�cient as the one outlined above.

141

38 Student's t-distribution38.1 IntroductionThe Student's t-distribution is given byf(t;n) = � �n+12 �pn�� �n2� 1 + t2n!�n+12 = �1 + t2n ��n+12pnB �12 ; n2�where the parameter n is a positive integer and the variable t is a real number. Thefunctions � and B are the usual Gamma and Beta functions. In �gure 31 we show thet-distribution for n values of 1 (lowest maxima), 2, 5 and 1 (fully drawn and identical tothe standard normal distribution).Figure 31: Graph of t-distribution for some values of nIf we change variable to x = t=pn and put m = n+12 the Student's t-distributionbecomes f(x;m) = k(1 + x2)m with k = �(m)� � 12�� �m� 12� = 1B �12 ;m� 12�where k is simply a normalization constant and m is a positive half-integer.38.2 HistoryA brief history behind this distribution and its name is the following. William Sealy Gosset(1876-1937) had a degree in mathematics and chemistry from Oxford when he in 1899 beganworking for Messrs. Guinness brewery in Dublin. In his work at the brewery he developed142

a small-sample theory of statistics which he needed in making small-scale experiments.Due to company policy it was forbidden for employees to publish scienti�c papers and hiswork on the t-ratio was published under the pseudonym \Student". It is a very importantcontribution to statistical theory.38.3 MomentsThe Student's t-distribution is symmetrical around t = 0 and thus all odd central momentsvanish. In calculating even moments (note that algebraic and central moments are equal)we make use of the somewhat simpler f(x;m) form given above with x = tpn which impliesthe following relation between expectation values E(t2r) = nrE(x2r). Central moments ofeven order are given by, with r an integer � 0,�2r(x) = 1Z�1 f(x;m)dx = k 1Z�1 x2r(1 + x2)mdx = 2k 1Z0 x2r(1 + x2)m dxIf we make the substitution y = x21+x2 implying 11+x2 = 1 � y and x = q y1�y then dy =2x(1+x2)2dx and we obtain�2r(x) = 2k 1Z0 x2r(1 + x2)m � (1 + x2)22x dy = k 1Z0 x2r�1(1 + x2)m�2dy == k 1Z0 (1 � y)m�2 s y1 � y!2r�1 dy = k 1Z0 (1 � y)m�r� 32yr� 12dy == kB(r + 12 ;m� r � 12) = B(r + 12 ;m� r � 12)B(12;m� 12)The normalization constant k was given above and we may now verify this expression bylooking at �0 = 1 giving k = 1=B(12;m� 12) and thus �nally, including the nr factor givingmoments in t we have�2r(t) = nr�2r(x) = nrB(r + 12 ;m� r � 12)B(12;m� 12) = nrB(r + 12 ; n2 � r)B(12; n2 )As can be seen from this expression we get into problems for 2r � n and indeed thosemoments are unde�ned or divergent9. The formula is thus valid only for 2r < n. A recursiveformula to obtain even algebraic moments of the t-distribution is�02r = �02r�2 � n � r � 12n2 � rstarting with �00 = 1.9See e.g. the discussion in the description of the moments for the Cauchy distribution which is thespecial case where m = n = 1. 143

Especially we note that, when n is big enough so that these moments are de�ned,the second central moment (i.e. the variance) is �2 = V (t) = nn�2 and the fourth centralmoment is given by �4 = 3n2(n�2)(n�4) . The coe�cients of skewness and kurtosis are given by 1 = 0 and 2 = 6n�4 , respectively.38.4 Cumulative FunctionIn calculating the cumulative function for the t-distribution it turns out to be simplifyingto �rst estimate the integral for a symmetric regiontZ�t f(u)du = 1pnB �12 ; n2� tZ�t 1 + u2n !�n+12 du == 2pnB �12 ; n2� tZ0 1 + u2n !�n+12 du == �2pnB �12 ; n2� nn+t2Z1 xn+12 npx2x2pnp1� xdx == 1B �12 ; n2� 1Znn+t2 (1� x)� 12xn2�1dx == 1B �12 ; n2� �B �n2 ; 12��B nn+t2 �n2 ; 12�� == 1� I nn+t2 �n2 ; 12� = I t2n+t2 �12; n2�where we have made the substitution x = n=(n + u2) in order to simplify the integration.From this we �nd the cumulative function asF (t) = 8><>: 12 � 12I t2n+t2 �12 ; n2� for �1 < x < 012 + 12I x2n+x2 �12 ; n2� for 0 � x <138.5 Relations to Other DistributionsThe distribution in F = t2 is given byf(F ) = ����� dtdF ����� f(t) = 12pF � �1 + Fn ��n+12pnB(12; n2 ) = nn2F� 12B(12 ; n2 )(F + n)n+12which we recognize as a F -distribution with 1 and n degrees of freedom.As n ! 1 the Student's t-distribution approaches the standard normal distribution.However, a better approximation than to create a simpleminded standardized variable,144

dividing by the square root of the variance, is to usez = t �1 � 14n�q1 + t22nwhich is more closely distributed according to the standard normal distribution.38.6 t-ratioRegard t = x=qyn where x and y are independent variables distributed according to thestandard normal and the chi-square distribution with n degrees of freedom, respectively.The independence implies that the joint probability function in x and y is given byf(x; y;n) = 1p2�e�x22 !0@y n2�1e� y22n2 � �n2�1Awhere �1 < x < 1 and y > 0. If we change variables to t = x=q yn and u = y thedistribution in t and u, with �1 < t <1 and u > 0, becomesf(t; u;n) = ����������@(x; y)@(t; u) ���������� f(x; y;n)The determinant is qun and thus we havef(t; u;n) = run 1p2�e�ut22n !0@un2�1e�u22n2� �n2�1A = u 12 (n+1)�1e�u2�1+ t2n �pn�� �n2� 2n+12Finally, since we are interested in the marginal distribution in t we integrate over uf(t;n) = 1Z0 f(t; u;n)du = 1pn�� �n2� 2n+12 1Z0 un+12 �1e�u2�1+ t2n �du == 1pn�� �n2� 2n+12 1Z0 2v1 + t2n !n+12 �1 e�v dv12 �1 + t2n � == �1 + t2n ��n+12pn�� �n2� 1Z0 v n+12 �1e�vdv = �1 + t2n ��n+12pn�� �n2� � �n+12 � = �1 + t2n ��n+12pnB �12; n2�where we made the substitution v = u2 �1 + t2n � in order to simplify the integral which inthe last step is recognized as being equal to � �n+12 �.145

38.7 One Normal SampleRegard a sample from a normal population N(�; �2) where the mean value x is distributedas N(�; �2n ) and (n�1)s2�2 is distributed according to the chi-square distribution with n � 1degrees of freedom. Here s2 is the usual unbiased variance estimator s2 = 1n�1 nPi=1(xi � x)2which in the case of a normal distribution is independent of x. This implies thatt = x���=pnq (n�1)s2�2 =(n � 1) = x� �s=pnis distributed according to Student's t-distribution with n� 1 degrees of freedom. We maythus use Student's t-distribution to test the hypothesis that x = � (see below).38.8 Two Normal SamplesRegard two samples fx1; x2; :::; xmg and fy1; y2; :::; yng from normal distributions havingthe same variance �2 but possibly di�erent means �x and �y, respectively. Then thequantity (x� y)� (�x � �y) has a normal distribution with zero mean and variance equalto �2 � 1m + 1n�. Furthermore the pooled variance estimates2 = (m� 1)s2x + (n� 1)s2ym+ n� 2 = mPi=1(xi � x)2 + nPi=1(yi � y)2m+ n� 2is a normal theory estimate of �2 with m+ n� 2 degrees of freedom10.Since s2 is independent of x for normal populations the variablet = (x� y)� (�x � �y)sq 1m + 1nhas the t-distribution with m + n � 2 degrees of freedom. We may thus use Student'st-distribution to test the hypotheses that x�y is consistent with � = �x��y. In particularwe may test if � = 0 i.e. if the two samples originate from population having the samemeans as well as variances.38.9 Paired DataIf observations are made in pairs (xi; yi) for i = 1; 2; :::; n the appropriate test statistic ist = dsd = dsd=pn = dvuut nPi=1(di�d)2n(n�1)10If y is a normal theory estimate of �2 with k degrees of freedom then ky=�2 is distributed accordingto the chi-square distribution with k degrees of freedom.146

where di = xi � yi and d = x� y. This quantity has a t-distribution with n� 1 degrees offreedom. We may also write this t-ratio ast = pn � dqs2x + s2y � 2Cxywhere s2x and s2y are the estimated variances of x and y and Cxy is the covariance betweenthem. If we would not pair the data the covariance term would be zero but the number ofdegrees of freedom 2n� 2 i.e. twice as large. The smaller number of degrees of freedom inthe paired case is, however, often compensated for by the inclusion of the covariance.38.10 Con�dence LevelsIn determining con�dence levels or testing hypotheses using the t-distribution we de�nethe quantity t�;n from F (t�;n) = t�;nZ�1 f(t;n)dt = 1� �i.e. � is the probability that a variable distributed according to the t-distribution withn degrees of freedom exceeds t�;n. Note that due to the symmetry about zero of thet-distribution t�;n = �t1��;n.In the case of one normal sample described above we may set a 1�� con�dence intervalfor � x� spnt�=2;n�1 � � � x+ spnt�=2;n�1Note that in the case where �2 is known we would not use the t-distribution. Theappropriate distribution to use in order to set con�dence levels in this case would be thenormal distribution.38.11 Testing HypothesesAs indicated above we may use the t-statistics in order to test hypotheses regarding themeans of populations from normal distributions.In the case of one sample the null hypotheses would be H0: � = �0 and the alternativehypothesis H1: � 6= �0. We would then use t = x��0s=pn as outlined above and reject H0 atthe � con�dence level of signi�cance if jtj > spnt�=2;n�1. This test is two-tailed since wedo not assume any a priori knowledge of in which direction an eventual di�erence wouldbe. If the alternate hypothesis would be e.g. H1 : � > �0 then a one-tailed test would beappropriate.The probability to reject the hypothesis H0 if it is indeed true is thus �. This is a socalled Type I error. However, we might also be interested in the probability of committinga Type II error implying that we would accept the hypothesis although it was wrong andthe distribution instead had a mean �1. In addressing this question the t-distributioncould be modi�ed yielding the non-central t-distribution. The probability content � ofthis distribution in the con�dence interval used would then be the probability of wrongly147

accepting the hypothesis. This calculation would depend on the choice of � as well as onthe number of observations n. However, we do not describe details about this here.In the two sample case we may want to test the null hypothesis H0: �x = �y ascompared to H1: �x 6= �y. Once again we would reject H0 if the absolute value of thequantity t = (x� y)=sq 1m + 1n would exceed t�=2;n+m�2.38.12 Calculation of Probability ContentIn order to �nd con�dence intervals or to test hypotheses we must be able to calculateintegrals of the probability density function over certain regions. We recall the formulaF (t�;n) = t�;nZ�1 f(t;n)dt = 1� �which de�nes the quantity t�;n for a speci�ed con�dence level �. The probability to get avalue equal to t�;n or higher is thus �.Classically all text-books in statistics are equipped with tables giving values of t�;n forspeci�c �-values. This is sometimes useful and in table 8 on page 180 we show such atable giving points where the distribution has a cumulative probability content of 1�� fordi�erent number of degrees of freedom.However, it is often preferable to calculate directly the exact probability that one wouldobserve the actual t-value or worse. To calculate the integral on the left-hand side we di�erbetween the case where the number of degrees of freedom is an odd or even integer. Theequation above may either be adjusted such that a required � is obtained or we mayreplace t�;n with the actual t-value found in order to calculate the probability for thepresent outcome of the experiment.The algorithm proposed for calculating the probability content of the t-distribution isdescribed in the following subsections.38.12.1 Even number of degrees of freedomFor even n we have putting m = n2 and making the substitution x = tpn1 � � = t�;nZ�1 f(t;n)dt = � �n+12 �pn�� �n2� t�;nZ�1 dt�1 + t2n �n+12 = � �m+ 12�p��(m) t�;n=pnZ�1 dx(1 + x2)m+ 12For convenience (or maybe it is rather laziness) we make use of standard integral tableswhere we �nd the integralZ dx(ax2 + c)m+ 12 = xpax2 + c m�1Xr=0 22m�2r�1(m� 1)!m!(2r)!(2m)!(r!)2cm�r (ax2 + c)rwhere in our case a = c = 1. Introducing x� = t�;n=pn for convenience this gives1� � = � �m+ 12� (m� 1)!m!22mp��(m)(2m)! � 24 x�2q1 + x2� m�1Xr=0 (2r)!22r(r!)2 (1 + x2�)r + 1235148

The last term inside the brackets is the value of the integrand at �1 which is seen to equal�12. Looking at the factor outside the brackets using that �(n) = (n� 1)! for n a positiveinteger, � �m+ 12� = (2m�1)!!2m p�, and rewriting (2m)! = (2m)!!(2m�1)!! = 2mm!(2m�1)!!we �nd that it in fact is equal to one. We thus have1 � � = x�2q1 + x2� m�1Xr=0 (2r)!22r(r!)2 (1 + x2�)r + 12In evaluating the sum it is useful to look at the individual terms. Denoting these by ur we�nd the recurrence relationur = ur�1 � 2r(2r � 1)r222(1 + x2�) = ur�1 � 1 � 12r1 + x2�where we start with u0 = 1.To summarize: in order to determine the probability � to observe a value t or biggerfrom a t-distribution with an even number of degrees of freedom n we calculate1 � � = tpn2q1 + t2n m�1Xr=0 ur + 12where u0 = 1 and ur = ur�1 � 1� 12r1+t2=n .38.12.2 Odd number of degrees of freedomFor odd n we have putting m = n�12 and making the substitution x = tpn1 � � = t�;nZ�1 f(t;n)dt = � �n+12 �pn�� �n2� t�;nZ�1 dt�1 + t2n �n+12 = �(m+ 1)p�� �m+ 12� x�Z�1 dx(1 + x2)m+1where we again have introduced x� = t�;n=pn. Once again we make use of standardintegral tables where we �nd the integralZ dx(a+ bx2)m+1 = (2m)!(m!)2 " x2a mXr=1 r!(r � 1)!(4a)m�r(2r)! (a+ bx2)r + 1(4a)m Z dxa+ bx2#where in our case a = b = 1. We obtain1 � � = �(m+ 1)(2m)!p�� �m+ 12�m!24m "x�2 mXr=1 4rr!(r � 1)!(2r)! (1 + x2�)r + arctan x� + �2 #where the last term inside the brackets is the value of the integrand at �1. The factoroutside the brackets is equal to 1� which is found using that �(n) = (n�1)! for n a positive149

integer, � �m+ 12� = (2m�1)!!2m p�, and (2m)! = (2m)!!(2m � 1)!! = 2m(m)!(2m � 1)!!. Weget 1� � = 1� "x�2 mXr=1 4rr!(r � 1)!(2r)! (1 + x2�)r + arctan x� + �2 # == 1� " x�1 + x2� mXr=1 22r�1r!(r � 1)!(2r)!(1 + x2�)r�1 + arctan x� + �2 #To compute the sum we denote the terms by vr and �nd the recurrence relationvr = vr�1 4r(r � 1)2r(2r � 1)(1 + x2�) = vr�1�1 � 12r�1�(1 + x2�)starting with v1 = 1.To summarize: in order to determine the probability � to observe a value t or biggerfrom a t-distribution with an odd number of degrees of freedom n we calculate1 � � = 1� 264 tpn1 + t2n n�12Xr=1 vr + arctan tpn375+ 12where v1 = 1 and vr = vr�1 � 1� 12r�11+t2=n .38.12.3 Final algorithmThe �nal algorithm to evaluate the probability content from �1 to t for a t-distributionwith n degrees of freedom is� Calculate x = tpn� For n even:� Put m = n2� Set u0 = 1; s = 0 and i = 0.� For i = 0; 1; 2; :::;m� 1 set s = s+ ui, i = i+ 1 and ui = ui�1 1� 12i1+x2 .� � = 12 � 12 � xp1+x2 s.� For n odd:� Put m = n�12 .� Set v1 = 1; s = 0 and i = 1.� For i = 1; 2; :::;m set s = s + vi, i = i+ 1 and vi = vi�1 � 1� 12i�11+x2 .� � = 12 � 1� ( x1+x2 � s+ arctan x). 150

38.13 Random Number GenerationFollowing the de�nition we may de�ne a random number t from a t-distribution, usingrandom numbers from a normal and a chi-square distribution, ast = zqyn=nwhere z is a standard normal and yn a chi-squared variable with n degrees of freedom. Toobtain random numbers from these distributions see the appropriate sections.

151

39 Triangular Distribution39.1 IntroductionThe triangular distribution is given byf(x;�;�) = �jx� �j�2 + 1�where the variable x is bounded to the interval � � � � x � � + � and the location andscale parameters � and � (� > 0) all are real numbers.39.2 MomentsThe expectation value of the distribution is E(x) = �. Due to the symmetry of thedistribution odd central moments vanishes while even moments are given by�n = 2�n(n+ 1)(n + 2)for even values of n. In particular the variance V (x) = �2 = �2=6 and the fourth centralmoment �4 = �4=15. The coe�cient of skewness is zero and the coe�cient of kurtosis 2 = �0:6.39.3 Random Number GenerationThe sum of two pseudorandom numbers uniformly distributed between (� � �)=2 and(�+�)=2 is distributed according to the triangular distribution. If �1 and �2 are uniformlydistributed between zero and one thenx = �+ (�1 + �2 � 1)� or x = � + (�1 � �2)�follow the triangular distribution.Note that this is a special case of a combinationx = (a+ b)�1 + (b� a)�2 � bwith b > a � 0 which gives a random number from a symmetric trapezoidal distributionwith vertices at (�b; 0) and (�a; 1a+b).152

40 Uniform Distribution40.1 IntroductionThe uniform distribution is, of course, a very simple case withf(x; a; b) = 1b� a for a � x � bThe cumulative, distribution, function is thus given byF (x; a; b) = 8<: 0 if x � ax�ab�a if a � x � b1 if b � x40.2 MomentsThe uniform distribution has expectation value E(x) = (a + b)=2, variance V (x) = (b �a)2=12, �3 = 0, �4 = (b� a)4=80, coe�cient of skewness 1 = 0 and coe�cient of kurtosis 2 = �1:2. More generally all odd central moments vanish and for n an even integer�n = (b� a)n2n(n+ 1)40.3 Random Number GenerationSince we assume the presence of a pseudorandom number generator giving random numbers� between zero and one a random number from the uniform distribution is simply given byx = (b� a)� + a153

41 Weibull Distribution41.1 IntroductionThe Weibull distribution is given byf(x; �; �) = �� �x����1 e�(x�)�where the variable x and the parameters � and � all are positive real numbers. Thedistribution is named after the swedish physicist Waloddi Weibull (1887{1979) a professorat the Technical Highschool in Stockholm 1924{1953.The parameter � is simply a scale parameter and the variable y = x=� has the distri-bution g(y) = � y��1 e�y�In �gure 32 we show the distribution for a few values of �. For � < 1 the distribution hasits mode at y = 0, at � = 1 it is identical to the exponential distribution, and for � > 1the distribution has a mode at x = � � 1� ! 1�which approaches x = 1 as � increases (at the same time the distribution gets more sym-metric and narrow).Figure 32: The Weibull distribution154

41.2 Cumulative DistributionThe cumulative distribution is given ,byF (x) = xZ0 f(u)du = xZ0 �� �u����1 e�(u� )�du = (x=�)�Z0 e�ydy = 1 � e�(x�)�where we have made the substitution y = (u=�)� in order to simplify the integration.41.3 MomentsAlgebraic moments are given byE(xk) = 1Z0 xkf(x)dx = �k 1Z0 y k� e�ydy = �k� k� + 1!where we have made the same substitution as was used when evaluating the cumulativedistribution above.Especially the expectation value and the variance are given byE(x) = �� 1� + 1! and V (x) = �28<:� 2� + 1! � � 1� + 1!29=;41.4 Random Number GenerationTo obtain random numbers fromWeibull's distribution using �, a random number uniformlydistributed from zero to one, we may solve the equation F (x) = � to obtain a randomnumber in x. F (x) = 1� e�( x� )� = � ) x = �(� ln �) 1�155

42 Appendix A: The Gamma and Beta Functions42.1 IntroductionIn statistical calculations for standard statistical distributions such as the normal (or Gaus-sian) distribution, the Student's t-distribution, the chi-squared distribution, and the F -distribution one often encounters the so called Gamma and Beta functions. More speci�-cally in calculating the probability content for these distributions the incomplete Gammaand Beta functions occur. In the following we brie y de�ne these functions and give nu-merical methods on how to calculate them. Also connections to the di�erent statisticaldistributions are given. The main references for this has been [41{43] for the formalismand [10] for the numerical methods.42.2 The Gamma FunctionThe Gamma function is normally de�ned as�(z) = 1Z0 tz�1e�tdtwhere z is a complex variable with Re(z) > 0. This is the so called Euler's integral form forthe Gamma function. There are, however, two other de�nitions worth mentioning. FirstlyEuler's in�nite limit form�(z) = limn!1 1 � 2 � 3 � � � nz(z + 1)(z + 2) � � � (z + n) nz z 6= 0;�1;�2; : : :and secondly the in�nite product form sometimes attributed to Euler and sometimes toWeierstrass 1�(z) = z z 1Yn=1�1 + zn� e� zn jzj <1where � 0:5772156649 is Euler's constant.In �gure 33 we show the Gamma function for real arguments from �5 to 5. Note thesingularities at x = 0;�1;�2; : : :.For z a positive real integer n we have the well known relation to the factorial functionn! = �(n+ 1)and, as the factorial function, the Gamma function satis�es the recurrence relation�(z + 1) = z�(z)In the complex plane �(z) has a pole at z = 0 and at all negative integer values of z.The re ection formula �(1 � z) = ��(z) sin(�z) = �z�(z + 1) sin(�z)156

Figure 33: The Gamma functionmay be used in order to get function values for Re(z) < 1 from values for Re(z) > 1.A well known approximation to the Gamma function is Stirling's formula�(z) = zze�zs2�z �1 + 112z + 1288z2 � 13951840z3 � 5712488320z4 + : : :�for j arg zj < � and jzj ! 1 and where often only the �rst term (1) in the series expansionis kept in approximate calculations. For the faculty of a positive integer n one often usesthe approximation n! � p2�n nn e�nwhich has the same origin and also is called Stirling's formula.42.2.1 Numerical CalculationThere are several methods to calculate the Gamma function in terms of series expansionsetc. For numerical calculations, however, the formula by Lanczos is very useful [10]�(z + 1) = �z + + 12�z+ 12 e�(z+ + 12)p2� �c0 + c1z + 1 + c2z + 2 + � � �+ cnz + n + ��for z > 0 and an optimal choice of the parameters , n, and c0 to cn. For = 5,n = 6 and a certain set of c's the error is smaller than j�j < 2 � 10�10. This bound istrue for all complex z in the half complex plane Re(z) > 0. The coe�cients normallyused are c0 = 1, c1 = 76:18009173, c2 = -86:50532033, c3 = 24:01409822, c4 = -1:231739516,c5 = 0:00120858003, and c6 = -0:00000536382. Use the re ection formula given above to ob-tain results for Re(z) < 1 e.g. for negative real arguments. Beware, however, to avoid thesingularities. While implementing routines for the Gamma function it is recommendable157

to evaluate the natural logarithm in order to avoid numerical over ow.An alternative way of evaluating ln �(z) is given in references [44,45], giving formul�which also are used in order to evaluate the Digamma function below. The expressionsused for ln �(z) areln �(z) = 8>>>>><>>>>>:�z � 12� ln z � z + 12 ln 2� + z KPk=1 B2k2k(2k�1)z�2k +RK(z) for 0 < x0 � xln �(z + n)� ln n�1Qk=0(z + k) for 0 � x < x0ln� + ln�(1 � z)� ln sin�z for x < 0Here n = [x0]� [x] (the di�erence of integer parts, where x is the real part of z = x+ {y)and e.g. K = 10 and x0 = 7:0 gives excellent accuracy i.e. small RK . Note that K�olbig [45]gives the wrong sign on the (third) constant term in the �rst case above.42.2.2 Formul�Below we list some useful relations concerning the Gamma function, faculties and semi-faculties (denoted by two exclamation marks here). For a more complete list consult e.g.[42]. �(z) = 1Z0 �ln 1t�z�1 dt�(z + 1) = z�(z) = z!�(z) = �z 1Z0 tz�1e��tdt for Re(z) > 0; Re(�) > 0�(k) = (k � 1)! for k � 1 (integer; 0! = 1)z! = �(z + 1) = 1Z0 e�ttzdt for Re(z) > �1��12� = p���n+ 12� = (2n � 1)!!2n p��(z)�(1 � z) = �sin �zz!(�z)! = �zsin �z(2m)!! = 2 � 4 � 6 � � � 2m = 2mm!(2m� 1)!! = 1 � 3 � 5 � � � (2m� 1)(2m)! = (2m)!!(2m� 1)!! = 2mm!(2m� 1)!!158

42.3 Digamma FunctionIt is often convenient to work with the logarithm of the Gamma function in order to avoidnumerical over ow in the calculations. The �rst derivatives of this function (z) = ddz ln �(z) = 1�(z) d�(z)dzis known as the Digamma, or Psi, function. A series expansion of this function is given by (z + 1) = � � 1Xn=1� 1z + n � 1n� for z 6= 0;�1;�2;�3; : : :where � 0:5772156649 is Euler's constant which is seen to be equal to � (1). If thederivative of the Gamma function itself is required we may thus simply use d�(z)=dz =�(z) � (z). Note that some authors write (z) = ddz ln �(z + 1) = ddz z! for the Digammafunction, and similarly for the polygamma functions below, thus shifting the argument byone unit.In �gure 34 we show the Digamma function for real arguments from �5 to 5. Note thesingularities at x = 0;�1;�2; : : :.Figure 34: The Digamma, or Psi, functionFor integer values of z we may write (n) = � + n�1Xm=1 1mwhich is e�cient enough for numerical calculations for not too large values of n. Similarlyfor half-integer values we have �n + 12� = � � 2 ln 2 + 2 nXm=1 12m� 1159

However, for arbitrary arguments the series expansion above is unusable. Following therecipe given in an article by K. S. K�olbig [45] we use (z) = 8>>>>><>>>>>: ln z � 12z � KPk=1 B2k2k z�2k +RK(z) for 0 < x0 � x (z + n) � n�1Pk=0 1z+k for 0 � x < x0 (�z) + 1z + � cot�z for x < 0Here n = [x0]� [x] (the di�erence of integer parts, where x is the real part of z = x+ {y)and we have chosen K = 10 and x0 = 7:0 which gives a very good accuracy (i.e. smallRK, typically less than 10�15) for double precision calculations. The main interest instatistical calculations is normally function values for (x) for real positive argumentsbut the formul� above are valid for any complex argument except for the singularitiesalong the real axis at z = 0;�1;�2;�3; : : :. The B2k are Bernoulli numbers given byB0 = 1; B1 = �12 ; B2 = 16 ; B4 = � 130; B6 = 142 ; B8 = � 130; B10 = 566; B12 = � 6912730; B14 =76 ; B16 = �3617510 ; B18 = 43867798 ; B20 = �174611330 : : :42.4 Polygamma FunctionHigher order derivatives of ln �(z) are called Polygamma functions11 (n)(z) = dndzn (z) = dn+1dzn+1 ln �(z) for n = 1; 2; 3; : : :Here a series expansion is given by (n)(z) = (�1)n+1n! 1Xk=0 1(z + k)n+1 for z 6= 0;�1;�2; : : :For numerical calculations we have adopted a technique similar to what was used toevaluate ln �(z) and (z). (n)(z) = 8>>><>>>: (�1)n�1 "t1 + n!2zn+1 + KPk=1B2k (2k+n�1)!(2k)!z2k+n +RK(z)# for 0 < x0 � x (n)(z +m)� (�1)nn!m�1Pk=0 1(z+k)n+1 for 0 � x < x0where t1 = � ln z for n = 0 and t1 = (n � 1)!=zn for n > 0. Here m = [x0] � [x] i.e.the di�erence of integer parts, where x is the real part of z = x + {y. We treat primarilythe case for real positive arguments x and if complex arguments are required one oughtto add a third re ection formula as was done in the previous cases. Without any specialoptimization we have chosen K = 14 and x0 = 7:0 which gives a very good accuracy, i.e.11Sometimes the more speci�c notation tri-, tetra-, penta- and hexagamma functions are used for 0, 00, (3) and (4), respectively. 160

small RK , typically less than 10�15, even for double precision calculations except for higherorders and low values of x where the function value itself gets large.12For more relations on the Polygamma (and the Digamma) functions see e.g. [42]. Twouseful relations used in this document in �nding cumulants for some distributions are (n)(1) = (�1)n+1n!�(n+ 1) (n)(12) = (�1)n+1n!(2n+1 � 1)�(n + 1) = (2n+1 � 1) (n)(1)where � is Riemann's zeta function (see page 60 and [31]).42.5 The Incomplete Gamma FunctionFor the incomplete Gamma function there seem to be several de�nitions in the literature.De�ning the two integrals (a; x) = xZ0 ta�1e�tdt and �(a; x) = 1Zx ta�1e�tdtwith Re(a) > 0 the incomplete Gamma function is normally de�ned asP (a; x) = (a; x)�(a)but sometimes also (a; x) and �(a; x) is referred to under the same name as well as thecomplement to P (a; x) Q(a; x) = 1 � P (a; x) = �(a; x)�(a)Note that, by de�nition, (a; x) + �(a; x) = �(a).In �gure 35 the incomplete Gamma function P (a; x) is shown for a few a-values (0.5,1, 5 and 10).42.5.1 Numerical CalculationFor numerical evaluations of P two formul� are useful [10]. For values x < a+1 the series (a; x) = e�xxa 1Xn=0 �(a)�(a+ n+ 1)xnconverges rapidly while for x � a+ 1 the continued fraction�(a; x) = e�xxa� 1x+ 1� a1+ 1x+ 2� a1+ 2x+ � � ��is a better choice.12For this calculation we need a few more Bernoulli numbers not given on page 160 above namelyB22 = 854513138 ; B24 = �2363640912730 ; B26 = 85531036 ; and B28 = �23749461029870161

Figure 35: The incomplete Gamma function42.5.2 Formul�Below we list some relations concerning the incomplete Gamma function. For a morecomplete list consult e.g. [42].�(a) = (a; x) + �(a; x) (a; x) = xZ0 e�tta�1dt for Re(a) > 0 (a+ 1; x) = a (a; x)� xae�x (n; x) = (n� 1)! "1� e�x n�1Xr=0 xrr! #�(a; x) = 1Zx e�tta�1dt�(a+ 1; x) = a�(a; x)� xae�x�(n; x) = (n� 1)!e�x n�1Xr=0 xrr! n = 1; 2; : : :42.5.3 Special CasesThe usage of the incomplete Gamma function P (a; x) in calculations made in this documentoften involves integer or half-integer values for a. These cases may be solved by the followingformul� P (n; x) = 1� e�x n�1Xk=0 xkk! 162

P �12 ; x� = erfpxP (a+ 1; x) = P (a; x)� xae�x�(a+ 1) = P (a; x)� xae�xa�(a)P �n2 ; x� = erfpx� n�12Xk=1 x 2k�12 e�x� � 2k+12 � = erfpx� 2e�xrx� n�12Xk=1 (2x)k�1(2k � 1)!!the last formula for odd values of n.42.6 The Beta FunctionThe Beta function is de�ned through the integral formulaB(a; b) = B(b; a) = 1Z0 ta�1(1 � t)b�1dtand is related to the Gamma function byB(a; b) = �(a)�(b)�(a+ b)The most straightforward way to calculate the Beta function is by using this last expressionand a well optimized routine for the Gamma function. In table 9 on page 181 expressionsfor the Beta function for low integer and half-integer arguments are given.Another integral, obtained by the substitution x = t=(1� t), yielding the Beta functionis B(a; b) = 1Z0 xa�1(1 + x)a+bdx42.7 The Incomplete Beta FunctionThe incomplete Beta function is de�ned asIx(a; b) = Bx(a; b)B(a; b) = 1B(a; b) xZ0 ta�1(1� t)b�1dtfor a; b > 0 and 0 � x � 1.The function Bx(a; b), often also called the incomplete Beta function, satis�es the fol-lowing formula Bx(a; b) = x1�xZ0 ua�1(1 + u)a+bdu = B1(b; a)�B1�x(b; a) == xa "1a + 1� ba+ 1x+ (1� b)(2� b)2!(a+ 2) x2+� � �+ (1 � b)(2� b) � � � (n� b)n!(a+ n) xn + � � �#163

In �gure 36 the incomplete Beta function is shown for a few (a; b)-values. Note that bysymmetry the (1; 5) and (5; 1) curves are re ected around the diagonal. For large values ofa and b the curve rises sharply from near zero to near one around x = a=(a+ b).Figure 36: The incomplete Beta function42.7.1 Numerical CalculationIn order to obtain Ix(a; b) the series expansionIx(a; b) = xa(1� x)baB(a; b) "1 + 1Xn=0 B(a+ 1; n+ 1)B(a+ b; n+ 1)xn+1#is not the most useful formula for computations. The continued fraction formulaIx(a; b) = xa(1 � x)baB(a; b) " 11+ d11+ d21+ � � �#turns out to be a better choice [10]. Hered2m+1 = � (a+m)(a+ b+m)x(a+ 2m)(a+ 2m + 1) and d2m = m(b�m)x(a+ 2m� 1)(a+ 2m)and the formula converges rapidly for x < (a+1)=(a+ b+1). For other x-values the sameformula may be used after applying the symmetry relationIx(a; b) = 1 � I1�x(b; a)164

42.7.2 ApproximationFor higher values of a and b, well already from a + b > 6, the incomplete Beta functionmay be approximated by� For (a+ b+ 1)(1� x) � 0:8 using an approximation to the chi-square distribution inthe variable �2 = (a+ b � 1)(1 � x)(3 � x)� (1 � x)(b � 1) with n = 2b degrees offreedom.� For (a+b+1)(1�x) � 0:8 using an approximation to the standard normal distributionin the variable z = 3 hw1 �1� 19b�� w2 �1 � 19a�iqw21b + w22awhere w1 = 3pbx and w2 = 3qa(1� x)In both cases the maximum di�erence to the true cumulative distribution is below 0.005all way down to the limit where a+ b = 6 [26].42.8 Relations to Probability Density FunctionsThe incomplete Gamma and Beta functions, P (a; x) and Ix(a; b) are related to many stan-dard probability density functions or rather to their cumulative (distribution) functions.We give very brief examples here. For more details on these distributions consult any bookin statistics.42.8.1 The Beta DistributionThe cumulative distribution for the Beta distribution with parameters p and q is given byF (x) = 1B(p; q) xZ0 tp�1(1� t)q�1dt = Bx(p; q)B(p; q) = Ix(p; q)i.e. simply the incomplete Beta function.42.8.2 The Binomial DistributionFor the binomial distribution with parameters n and pnXj=k nj!pj(1 � p)n�j = Ip(k; n�k+1)i.e. the cumulative distribution may be obtained byP (k) = kXi=0 ni!pi(1� p)n�i = I1�p(n�k; k+1)However, be careful to evaluate P (n), which obviously is unity, using the incomplete Betafunction since this is not de�ned for arguments which are less or equal to zero.165

42.8.3 The Chi-squared DistributionThe cumulative chi-squared distribution for n degrees of freedom is given byF (x) = 12� �n2� xZ0 �x2�n2�1 e�x2 dx = 12� �n2� x2Z0 y n2�1e�y2dy == �n2 ; x2�� �n2� = P �n2 ; x2�where x is the chi-squared value sometimes denoted �2. In this calculation we made thesimple substitution y = x=2 in simplifying the integral.42.8.4 The F -distributionThe cumulative F -distribution with m and n degrees of freedom is given byF (x) = 1B �m2 ; n2� xZ0 mm2 nn2 F m2 �1(mF + n)m+n2 dF = 1B �m2 ; n2� xZ0 � mFmF + n�m2 � nmF + n�n2 dFF == 1B �m2 ; n2� mxmx+nZ0 ym2 (1 � y)n2 dyy(1� y) = 1B �m2 ; n2� mxmx+nZ0 ym2 �1(1� y)n2�1dy == Bz �m2 ; n2�B �m2 ; n2� = Iz �m2 ; n2�with z = mx=(n +mx). Here we have made the substitution y = mF=(mF + n), leadingto dF=F = dy=y(1� y), in simplifying the integral.42.8.5 The Gamma DistributionNot surprisingly the cumulative distribution for the Gamma distribution with parametersa and b is given by an incomplete Gamma function.F (x) = xZ0 f(x)dx = ab�(b) xZ0 ub�1e�audu = ab�(b) axZ0 �va�b�1 e�v dva == 1�(b) axZ0 vb�1e�vdv = (b; ax)�(b) = P (b; ax)42.8.6 The Negative Binomial DistributionThe negative binomial distribution with parameters n and p is related to the incompleteBeta function via the relationnXs=a n+ s� 1s !pn(1 � p)s = I1�p(a; n)166

Also the geometric distribution, a special case of the negative binomial distribution, isconnected to the incomplete Beta function, see summary below.42.8.7 The Normal DistributionThe cumulative normal, or Gaussian, distribution is given by13F (x) = 8<: 12 + 12P �12 ; x22 � if x � 012 � 12P �12 ; x22 � if x < 0where P �12 ; x22 � is the incomplete Gamma function occurring as twice the integral of thestandard normal curve from 0 to x since1p2� xZ0 e� t22 dt = 12p� x22Z0 e�upudu = 12� � 12� x22Z0 u� 12 e�udu == �12 ; x22 �2� � 12� = 12P �12; x22 �The so called error function may be expressed in terms of the incomplete Gammafunction erf x = 2p� xZ0 e�t2dt = P �12 ; x2�as is the case for the complementary error functionerfc x = 1� erf x = 2p� 1Zx e�t2dt = 1� P �12 ; x2�de�ned for x � 0, for x < 0 use erf(�x) = �erf(x) and erfc(�x) = 1 + erf(x). See alsosection 13.There are also other series expansions for erf x likeerf x = 2p� "x� x33 � 1! + x55 � 2! � x77 � 3! + : : :# == 1 � e�x2p�x "1� 12x2 + 1 � 3(2x2)2 � 1 � 3 � 5(2x2)3 + : : :#42.8.8 The Poisson DistributionAlthough the Poisson distribution is a probability density function in a discrete variablethe cumulative distribution may be expressed in terms of the incomplete Gamma function.The probability for outcomes from zero to k � 1 inclusive for a Poisson distribution withparameter (mean) � isP�(< k) = k�1Xn=0 �ne��n! = 1 � P (k; �) for k = 1; 2; : : :13Without loss of generality it is enough to regard the standard normal density.167

42.8.9 Student's t-distributionThe symmetric integral of the t-distribution with n degrees of freedom, often denotedA(tjn), is given byA(tjn) = 1pnB �12; n2� tZ�t 1 + x2n !�n+12 dx = 2pnB �12 ; n2� tZ0 � nn+ x2�n+12 dx == 2pnB �12; n2� t2n+t2Z0 (1 � y)n+12 n2 11� y!2s1 � yny dy == 1B �12 ; n2� t2n+t2Z0 y� 12 (1 � y)n2�1dy = Bz �12; n2�B �12 ; n2� = Iz �12 ; n2�with z = t2=(n+ t2).42.8.10 SummaryThe following table summarizes the relations between the cumulative, distribution, func-tions of some standard probability density functions and the incomplete Gamma and Betafunctions.Distribution Parameters Cumulative distribution RangeBeta p, q F (x) = Ix(p; q) 0 � x � 1Binomial n, p P (k) = I1�p(n�k; k+1) k = 0; 1; : : : ; nChi-squared n F (x) = P�n2 ; x2� x � 0F m, n F (x) = I mxn+mx �m2 ; n2� x � 0Gamma a, b F (x) = P (b; ax) x � 0Geometric p P (k) = Ip(1; k) k = 1; 2; : : :Negative binomial n, p P (k) = Ip(n; k+1) k = 0; 1; : : :Standard normal F (x) = 12 � 12P�12; x22 � �1 < x < 0F (x) = 12 + 12P�12; x22 � 0 � x <1Poisson � P (k) = 1� P (k+1; �) k = 0; 1; : : :Student n F (x) = 12 � 12I x2n+x2 �12 ; n2� �1 < x < 0F (x) = 12 + 12I x2n+x2 �12 ; n2� 0 � x <1168

43 Appendix B: Hypergeometric Functions43.1 IntroductionThe hypergeometric and the con uent hypergeometric functions has a central role inasmuchas many standard functions may be expressed in terms of them. This appendix is based oninformation from [41,46,47] in which muchmore detailed information on the hypergeometricand con uent hypergeometric function may be found.43.2 Hypergeometric FunctionThe hypergeometric function, sometimes called Gauss's di�erential equation, is given by[41,46] x(1� x)@2f(x)@x2 + [c� (a+ b+ 1)x] @f(x)@x � abf(x) = 0One solution isf(x) = 2F1(a; b; c;x) = 1 + abc x1! + a(a+ 1)b(b+ 1)c(c+ 1) x22! + � � � c 6= 0;�1;�2;�3; : : :The range of convergence is jxj < 1 and x = 1, for c > a+ b, and x = �1, for c > a+ b� 1.Using the so called Pochhammer symbol(a)n = a(a+ 1)(a+ 2) � � � (a+ n� 1) = (a+ n� 1)!(a� 1)! = �(a+ n)�(a)with (a)0 = 1 this solution may be written14 as2F1(a; b; c;x) = 1Xn=0 (a)n(b)n(c)n xnn! = �(c)�(a)�(b) 1Xn=0 �(a+ n)�(b + n)�(c+ n) xnn!By symmetry 2F1(a; b; c;x) = 2F1(b; a; c;x) and sometimes the indices are dropped andwhen the risk for confusion is negligible one simply writes F (a; b; c;x).Another independent solution to the hypergeometric equation isf(x) = x1�c 2F1(a+1�c; b+1�c; 2�c;x) c 6= 2; 3; 4; : : :The n:th derivative of the hypergeometric function is given bydndxn 2F1(a; b; c;x) = (a)n(b)n(c)n 2F1(a+n; b+n; c+n;x)and 2F1(a; b; c;x) = (1 � x)c�a�b 2F1(c�a; c�b; c;x)Several common mathematical function may be expressed in terms of the hypergeomet-ric function such as, the incomplete Beta function Bx(a; b), the complete elliptical integrals14The notation 2F1 indicates the presence of two Pochhammer symbols in the numerator and one in thedenominator. 169

K and E, the Gegenbauer functions T �n (x), the Legendre functions Pn(x), Pmn (x) and Q�(x)(second kind), and the Chebyshev functions Tn(x), Un(x) and Vn(x)(1 � z)�a = 2F1(a; b; b; z)ln(1 + z) = x � 2F1(1; 1; 2;�z)arctan z = z � 2F1�12 ; 1; 32;�z2�arcsin z = z � 2F1�12 ; 12; 32 ; z2� = zp1� z2 2F1�1; 1; 32 ; z2�Bx(a; b) = xaa 2F1(a; 1�b; a+1;x)K = �2Z0 (1� k2 sin2 �)� 12d� = �2 2F1�12 ; 12; 1; k2�E = �2Z0 (1� k2 sin2 �) 12d� = �2 2F1�12 ;�12; 1; k2�T �n (x) = (n + 2�)!2�n!�! 2F1��n; n+2�+1; 1+�; 1�x2 �Pn(x) = 2F1��n; n+ 1; 1; 1�x2 �Pmn (x) = (n+m)!(n �m)! (1� x2)m22mm! 2F1�m�n;m+n+1;m+1; 1�x2 �P2n(x) = (�1)n (2n � 1)!!(2n)!! 2F1��n; n+ 12 ; 12;x2�P2n+1(x) = (�1)n (2n + 1)!!(2n)!! x � 2F1��n; n+ 32 ; 32 ;x2�Q�(x) = p��!�� + 12�!(2x)�+1 2F1��+12 ; �2+1; �+32 ; 1x2�Tn(x) = 2F1��n; n; 12 ; 1�x2 �Un(x) = (n + 1) � 2F1��n; n+2; 32 ; 1�x2 �Vn(x) = np1 � x2 2F1��n+1; n+1; 32 ; 1�x2 �for Q�(x) the conditions are jxj > 1; j arg xj < �; and � 6= �1;�2;�3; : : :. See [46] formany more similar and additional formul�.43.3 Con uent Hypergeometric FunctionThe con uent hypergeometric equation, or Kummer's equation as it is often called, is givenby [41,47] x@2f(x)@x2 + (c� x)@f(x)@x � af(x) = 0170

One solution to this equation isf(x) = 1F1(a; c;x) =M(a; c;x) = 1Xn=0 (a)n(c)n xnn! c 6= 0;�1;�2; : : :This solution is convergent for all �nite real x (or complex z). Another solution is given byf(x) = x1�cM(a+1�c; 2�c;x) c 6= 2; 3; 4; : : :Often a linear combination of the �rst and second solution is usedU(a; c;x) = �sin �c " M(a; c;x)(a�c)!(c�1)! � x1�cM(a+1�c; 2�c;x)(a�1)!(1�c)! #The con uent hypergeometric functionsM and U may be expressed in integral form asM(a; c;x) = �(c)�(a)�(c � a) 1Z0 extta�1(1� t)c�a�1dt Re c > 0; Re a > 0U(a; c;x) = 1�(a) 1Z0 e�xtta�1(1 + t)c�a�1dt Re x > 0; Re a > 0Useful formul� are the Kummer transformationsM(a; c;x) = exM(c�a; c;�x)U(a; c;x) = x1�cU(a�c+1; 2�c;x)The n:th derivatives of the con uent hypergeometric functions are given bydndznM(a; b; z) = (a)n(b)nM(a+n; b+n; z)dndznU(a; b; z) = (�1)n(a)nU(a+n; b+n; z)Several common mathematical function may be expressed in terms of the hypergeomet-ric function such as the error function, the incomplete Gamma function (a; x), Bessel func-tions J�(x), modi�ed Bessel functions of the �rst kind I�(x), Hermite functions Hn(x), La-guerre functions Ln(x), associated Laguerre functions Lmn (x), Whittaker functions Mk�(x)and Wk�(x), Fresnel integrals C(x) and S(x), modi�ed Bessel function of the second kindK�(x) ez = M(a; a; z)erf(x) = 2p�xM�12 ; 32 ;�x2� = 2p�xe�x2M�1; 32;x2� (a; x) = xaa M(a; a+1;�x) Re a > 0J�(x) = e�{x�! �x2�� M��+ 12; 2�+1; 2{x�171

I�(x) = e�x�! �x2�� M��+ 12 ; 2�+1; 2x�H2n(x) = (�1)n (2n)!n! M��n; 12;x2�H2n+1(x) = (�1)n 2(2n + 1)!n! xM��n; 32 ;x2�Ln(x) = M(�n; 1;x)Lmn (x) = (�1)m @m@xmLn+m(x) = (n+m)!n!m! M(�n;m+1;x)Mk�(x) = e�x2 x�+12M���k+ 12 ; 2�+1;x�Wk�(x) = e�x2 x�+12U���k+ 12 ; 2�+1;x�C(x) + {S(x) = xM 1x; 32; {�x22 !K� (x) = p�e�x(2x)�U��+ 12; 2�+1; 2x�See [47] for many more similar and additional formul�.

172

Table 1: Percentage points of the chi-square distribution1� �n 0.5000 0.8000 0.9000 0.9500 0.9750 0.9900 0.9950 0.99901 0.4549 1.6424 2.7055 3.8415 5.0239 6.6349 7.8794 10.8282 1.3863 3.2189 4.6052 5.9915 7.3778 9.2103 10.597 13.8163 2.3660 4.6416 6.2514 7.8147 9.3484 11.345 12.838 16.2664 3.3567 5.9886 7.7794 9.4877 11.143 13.277 14.860 18.4675 4.3515 7.2893 9.2364 11.070 12.833 15.086 16.750 20.5156 5.3481 8.5581 10.645 12.592 14.449 16.812 18.548 22.4587 6.3458 9.8032 12.017 14.067 16.013 18.475 20.278 24.3228 7.3441 11.030 13.362 15.507 17.535 20.090 21.955 26.1249 8.3428 12.242 14.684 16.919 19.023 21.666 23.589 27.87710 9.3418 13.442 15.987 18.307 20.483 23.209 25.188 29.58811 10.341 14.631 17.275 19.675 21.920 24.725 26.757 31.26412 11.340 15.812 18.549 21.026 23.337 26.217 28.300 32.90913 12.340 16.985 19.812 22.362 24.736 27.688 29.819 34.52814 13.339 18.151 21.064 23.685 26.119 29.141 31.319 36.12315 14.339 19.311 22.307 24.996 27.488 30.578 32.801 37.69716 15.338 20.465 23.542 26.296 28.845 32.000 34.267 39.25217 16.338 21.615 24.769 27.587 30.191 33.409 35.718 40.79018 17.338 22.760 25.989 28.869 31.526 34.805 37.156 42.31219 18.338 23.900 27.204 30.144 32.852 36.191 38.582 43.82020 19.337 25.038 28.412 31.410 34.170 37.566 39.997 45.31521 20.337 26.171 29.615 32.671 35.479 38.932 41.401 46.79722 21.337 27.301 30.813 33.924 36.781 40.289 42.796 48.26823 22.337 28.429 32.007 35.172 38.076 41.638 44.181 49.72824 23.337 29.553 33.196 36.415 39.364 42.980 45.559 51.17925 24.337 30.675 34.382 37.652 40.646 44.314 46.928 52.62026 25.336 31.795 35.563 38.885 41.923 45.642 48.290 54.05227 26.336 32.912 36.741 40.113 43.195 46.963 49.645 55.47628 27.336 34.027 37.916 41.337 44.461 48.278 50.993 56.89229 28.336 35.139 39.087 42.557 45.722 49.588 52.336 58.30130 29.336 36.250 40.256 43.773 46.979 50.892 53.672 59.70340 39.335 47.269 51.805 55.758 59.342 63.691 66.766 73.40250 49.335 58.164 63.167 67.505 71.420 76.154 79.490 86.66160 59.335 68.972 74.397 79.082 83.298 88.379 91.952 99.60770 69.334 79.715 85.527 90.531 95.023 100.43 104.21 112.3280 79.334 90.405 96.578 101.88 106.63 112.33 116.32 124.8490 89.334 101.05 107.57 113.15 118.14 124.12 128.30 137.21100 99.334 111.67 118.50 124.34 129.56 135.81 140.17 149.45173

Table 2: Extreme con�dence levels for the chi-square distributionChi-square Con�dence Levels (as �2 values)d.f. 0.1 0.01 10�3 10�4 10�5 10�6 10�7 10�8 10�9 10�10 10�11 10�121 2.71 6.63 10.8 15.1 19.5 23.9 28.4 32.8 37.3 41.8 46.3 50.82 4.61 9.21 13.8 18.4 23.0 27.6 32.2 36.8 41.4 46.1 50.7 55.33 6.25 11.3 16.3 21.1 25.9 30.7 35.4 40.1 44.8 49.5 54.2 58.94 7.78 13.3 18.5 23.5 28.5 33.4 38.2 43.1 47.9 52.7 57.4 62.25 9.24 15.1 20.5 25.7 30.9 35.9 40.9 45.8 50.7 55.6 60.4 65.26 10.6 16.8 22.5 27.9 33.1 38.3 43.3 48.4 53.3 58.3 63.2 68.17 12.0 18.5 24.3 29.9 35.3 40.5 45.7 50.8 55.9 60.9 65.9 70.88 13.4 20.1 26.1 31.8 37.3 42.7 48.0 53.2 58.3 63.4 68.4 73.59 14.7 21.7 27.9 33.7 39.3 44.8 50.2 55.4 60.7 65.8 70.9 76.010 16.0 23.2 29.6 35.6 41.3 46.9 52.3 57.7 62.9 68.2 73.3 78.511 17.3 24.7 31.3 37.4 43.2 48.9 54.4 59.8 65.2 70.5 75.7 80.912 18.5 26.2 32.9 39.1 45.1 50.8 56.4 61.9 67.3 72.7 78.0 83.213 19.8 27.7 34.5 40.9 46.9 52.7 58.4 64.0 69.5 74.9 80.2 85.514 21.1 29.1 36.1 42.6 48.7 54.6 60.4 66.0 71.6 77.0 82.4 87.815 22.3 30.6 37.7 44.3 50.5 56.5 62.3 68.0 73.6 79.1 84.6 90.016 23.5 32.0 39.3 45.9 52.2 58.3 64.2 70.0 75.7 81.2 86.7 92.217 24.8 33.4 40.8 47.6 54.0 60.1 66.1 71.9 77.6 83.3 88.8 94.318 26.0 34.8 42.3 49.2 55.7 61.9 68.0 73.8 79.6 85.3 90.9 96.419 27.2 36.2 43.8 50.8 57.4 63.7 69.8 75.7 81.6 87.3 92.9 98.520 28.4 37.6 45.3 52.4 59.0 65.4 71.6 77.6 83.5 89.3 94.9 10125 34.4 44.3 52.6 60.1 67.2 73.9 80.4 86.6 92.8 98.8 105 11130 40.3 50.9 59.7 67.6 75.0 82.0 88.8 95.3 102 108 114 12035 46.1 57.3 66.6 74.9 82.6 89.9 97.0 104 110 117 123 12940 51.8 63.7 73.4 82.1 90.1 97.7 105 112 119 125 132 13845 57.5 70.0 80.1 89.1 97.4 105 113 120 127 134 140 14750 63.2 76.2 86.7 96.0 105 113 120 128 135 142 149 15560 74.4 88.4 99.6 110 119 127 135 143 150 158 165 17270 85.5 100 112 123 132 141 150 158 166 173 181 18880 96.6 112 125 136 146 155 164 172 180 188 196 20490 108 124 137 149 159 169 178 187 195 203 211 219100 118 136 149 161 172 182 192 201 209 218 226 234120 140 159 174 186 198 209 219 228 237 246 255 263150 173 193 209 223 236 247 258 268 278 288 297 306200 226 249 268 283 297 310 322 333 344 355 365 374300 332 360 381 400 416 431 445 458 471 483 495 506400 437 469 493 514 532 549 565 580 594 607 620 632500 541 576 603 626 646 665 682 698 714 728 742 756600 645 684 713 737 759 779 798 815 832 847 862 877800 852 896 929 957 982 1005 1026 1045 1064 1081 1098 11141000 1058 1107 1144 1175 1202 1227 1250 1272 1292 1311 1330 1348174

Table 3: Extreme con�dence levels for the chi-square distribution (as �2/d.f. values)Chi-square Con�dence Levels (as �2/d.f. values)d.f. 0.1 0.01 10�3 10�4 10�5 10�6 10�7 10�8 10�9 10�10 10�11 10�121 2.71 6.63 10.83 15.14 19.51 23.93 28.37 32.84 37.32 41.82 46.33 50.842 2.30 4.61 6.91 9.21 11.51 13.82 16.12 18.42 20.72 23.03 25.33 27.633 2.08 3.78 5.42 7.04 8.63 10.22 11.80 13.38 14.95 16.51 18.08 19.644 1.94 3.32 4.62 5.88 7.12 8.34 9.56 10.77 11.97 13.17 14.36 15.555 1.85 3.02 4.10 5.15 6.17 7.18 8.17 9.16 10.14 11.11 12.08 13.056 1.77 2.80 3.74 4.64 5.52 6.38 7.22 8.06 8.89 9.72 10.54 11.357 1.72 2.64 3.47 4.27 5.04 5.79 6.53 7.26 7.98 8.70 9.41 10.128 1.67 2.51 3.27 3.98 4.67 5.34 6.00 6.65 7.29 7.92 8.56 9.189 1.63 2.41 3.10 3.75 4.37 4.98 5.57 6.16 6.74 7.31 7.88 8.4510 1.60 2.32 2.96 3.56 4.13 4.69 5.23 5.77 6.29 6.82 7.33 7.8511 1.57 2.25 2.84 3.40 3.93 4.44 4.94 5.44 5.92 6.41 6.88 7.3512 1.55 2.18 2.74 3.26 3.76 4.24 4.70 5.16 5.61 6.06 6.50 6.9313 1.52 2.13 2.66 3.14 3.61 4.06 4.49 4.92 5.34 5.76 6.17 6.5814 1.50 2.08 2.58 3.04 3.48 3.90 4.31 4.72 5.11 5.50 5.89 6.2715 1.49 2.04 2.51 2.95 3.37 3.77 4.16 4.54 4.91 5.28 5.64 6.0016 1.47 2.00 2.45 2.87 3.27 3.65 4.01 4.37 4.73 5.08 5.42 5.7617 1.46 1.97 2.40 2.80 3.17 3.54 3.89 4.23 4.57 4.90 5.22 5.5518 1.44 1.93 2.35 2.73 3.09 3.44 3.78 4.10 4.42 4.74 5.05 5.3619 1.43 1.90 2.31 2.67 3.02 3.35 3.67 3.99 4.29 4.59 4.89 5.1820 1.42 1.88 2.27 2.62 2.95 3.27 3.58 3.88 4.17 4.46 4.75 5.0325 1.38 1.77 2.10 2.41 2.69 2.96 3.21 3.47 3.71 3.95 4.19 4.4230 1.34 1.70 1.99 2.25 2.50 2.73 2.96 3.18 3.39 3.60 3.80 4.0035 1.32 1.64 1.90 2.14 2.36 2.57 2.77 2.96 3.15 3.34 3.52 3.6940 1.30 1.59 1.84 2.05 2.25 2.44 2.62 2.80 2.97 3.13 3.29 3.4545 1.28 1.55 1.78 1.98 2.16 2.34 2.50 2.66 2.82 2.97 3.12 3.2650 1.26 1.52 1.73 1.92 2.09 2.25 2.41 2.55 2.70 2.84 2.97 3.1160 1.24 1.47 1.66 1.83 1.98 2.12 2.25 2.38 2.51 2.63 2.75 2.8670 1.22 1.43 1.60 1.75 1.89 2.02 2.14 2.25 2.37 2.48 2.58 2.6880 1.21 1.40 1.56 1.70 1.82 1.94 2.05 2.15 2.26 2.35 2.45 2.5490 1.20 1.38 1.52 1.65 1.77 1.87 1.98 2.07 2.17 2.26 2.35 2.43100 1.18 1.36 1.49 1.61 1.72 1.82 1.92 2.01 2.09 2.18 2.26 2.34120 1.17 1.32 1.45 1.55 1.65 1.74 1.82 1.90 1.98 2.05 2.12 2.19150 1.15 1.29 1.40 1.49 1.57 1.65 1.72 1.79 1.85 1.92 1.98 2.04200 1.13 1.25 1.34 1.42 1.48 1.55 1.61 1.67 1.72 1.77 1.82 1.87300 1.11 1.20 1.27 1.33 1.39 1.44 1.48 1.53 1.57 1.61 1.65 1.69400 1.09 1.17 1.23 1.28 1.33 1.37 1.41 1.45 1.48 1.52 1.55 1.58500 1.08 1.15 1.21 1.25 1.29 1.33 1.36 1.40 1.43 1.46 1.48 1.51600 1.07 1.14 1.19 1.23 1.27 1.30 1.33 1.36 1.39 1.41 1.44 1.46800 1.06 1.12 1.16 1.20 1.23 1.26 1.28 1.31 1.33 1.35 1.37 1.391000 1.06 1.11 1.14 1.17 1.20 1.23 1.25 1.27 1.29 1.31 1.33 1.35175

Table 4: Exact and approximate values for the Bernoulli numbersBernoulli numbersn N/D = Bn=10k k0 1/1 = 1.00000 00000 01 {1/2 = { 5.00000 00000 {12 1/6 = 1.66666 66667 {14 {1/30 = { 3.33333 33333 {26 1/42 = 2.38095 23810 {28 {1/30 = { 3.33333 33333 {210 5/66 = 7.57575 75758 {212 {691/2730 = { 2.53113 55311 {114 7/6 = 1.16666 66667 016 {3617/510 = { 7.09215 68627 018 43867/798 = 5.49711 77945 120 {174611/330 = { 5.29124 24242 222 854513/138 = 6.19212 31884 324 {236364091/2 730 = { 8.65802 53114 426 8 553103/6 = 1.42551 71667 628 {23749461029/870 = { 2.72982 31068 730 8 615841276005/14 322 = 6.01580 87390 832 {7 709321041217/510 = { 1.51163 15767 1034 2 577687858367/6 = 4.29614 64306 1136 {26315271 553053477373/1 919190 = { 1.37116 55205 1338 2929 993913841559/6 = 4.88332 31897 1440 {261082718 496449122051/13 530 = { 1.92965 79342 1642 1 520097643 918070802691/1 806 = 8.41693 04757 1744 {27833269579 301024235023/690 = { 4.03380 71854 1946 596451111593 912163277961/282 = 2.11507 48638 2148 {5 609403368997817 686249127547/46 410 = { 1.20866 26522 2350 495057205241079 648212477525/66 = 7.50086 67461 2452 {801165718135489957 347924991853/1 590 = { 5.03877 81015 2654 29 149963634884862421 418123812691/798 = 3.65287 76485 2856 {2479 392929313226753685 415739663229/870 = { 2.84987 69302 3058 84483 613348880041862046 775994036021/354 = 2.38654 27500 3260 {1 215233140483 755572040304994079 820246041491/56 786730 = { 2.13999 49257 3462 12300585 434086858541953039 857403386151/6 = 2.05009 75723 3664 {106783830147 866529886385444979 142647942017/510 = { 2.09380 05911 3866 1 472600022126335 654051619428551932 342241899101/64 722 = 2.27526 96488 4068 {78773130858718 728141909149208474 606244347001/30 = { 2.62577 10286 4270 1505 381347333367003 803076567377857208 511438160235/4 686 = 3.21250 82103 44176

Table 5: Percentage points of the F -distribution�=0.10 nm 1 2 3 4 5 10 20 50 100 11 39.86 49.50 53.59 55.83 57.24 60.19 61.74 62.69 63.01 63.332 8.526 9.000 9.162 9.243 9.293 9.392 9.441 9.471 9.481 9.4913 5.538 5.462 5.391 5.343 5.309 5.230 5.184 5.155 5.144 5.1344 4.545 4.325 4.191 4.107 4.051 3.920 3.844 3.795 3.778 3.7615 4.060 3.780 3.619 3.520 3.453 3.297 3.207 3.147 3.126 3.10510 3.285 2.924 2.728 2.605 2.522 2.323 2.201 2.117 2.087 2.05520 2.975 2.589 2.380 2.249 2.158 1.937 1.794 1.690 1.650 1.60750 2.809 2.412 2.197 2.061 1.966 1.729 1.568 1.441 1.388 1.327100 2.756 2.356 2.139 2.002 1.906 1.663 1.494 1.355 1.293 1.2141 2.706 2.303 2.084 1.945 1.847 1.599 1.421 1.263 1.185 1.000�=0.05 nm 1 2 3 4 5 10 20 50 100 11 161.4 199.5 215.7 224.6 230.2 241.9 248.0 251.8 253.0 254.32 18.51 19.00 19.16 19.25 19.30 19.40 19.45 19.48 19.49 19.503 10.13 9.552 9.277 9.117 9.013 8.786 8.660 8.581 8.554 8.5264 7.709 6.944 6.591 6.388 6.256 5.964 5.803 5.699 5.664 5.6285 6.608 5.786 5.409 5.192 5.050 4.735 4.558 4.444 4.405 4.36510 4.965 4.103 3.708 3.478 3.326 2.978 2.774 2.637 2.588 2.53820 4.351 3.493 3.098 2.866 2.711 2.348 2.124 1.966 1.907 1.84350 4.034 3.183 2.790 2.557 2.400 2.026 1.784 1.599 1.525 1.438100 3.936 3.087 2.696 2.463 2.305 1.927 1.676 1.477 1.392 1.2831 3.841 2.996 2.605 2.372 2.214 1.831 1.571 1.350 1.243 1.000�=0.01 nm 1 2 3 4 5 10 20 50 100 11 4052 5000 5403 5625 5764 6056 6209 6303 6334 63662 98.50 99.00 99.17 99.25 99.30 99.40 99.45 99.48 99.49 99.503 34.12 30.82 29.46 28.71 28.24 27.23 26.69 26.35 26.24 26.134 21.20 18.00 16.69 15.98 15.52 14.55 14.02 13.69 13.58 13.465 16.26 13.27 12.06 11.39 10.97 10.05 9.553 9.238 9.130 9.02010 10.04 7.559 6.552 5.994 5.636 4.849 4.405 4.115 4.014 3.90920 8.096 5.849 4.938 4.431 4.103 3.368 2.938 2.643 2.535 2.42150 7.171 5.057 4.199 3.720 3.408 2.698 2.265 1.949 1.825 1.683100 6.895 4.824 3.984 3.513 3.206 2.503 2.067 1.735 1.598 1.4271 6.635 4.605 3.782 3.319 3.017 2.321 1.878 1.523 1.358 1.000177

Table 6: Probability content from �z to z of Gauss distribution in %z 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.090.0 0.00 0.80 1.60 2.39 3.19 3.99 4.78 5.58 6.38 7.170.1 7.97 8.76 9.55 10.34 11.13 11.92 12.71 13.50 14.28 15.070.2 15.85 16.63 17.41 18.19 18.97 19.74 20.51 21.28 22.05 22.820.3 23.58 24.34 25.10 25.86 26.61 27.37 28.12 28.86 29.61 30.350.4 31.08 31.82 32.55 33.28 34.01 34.73 35.45 36.16 36.88 37.590.5 38.29 38.99 39.69 40.39 41.08 41.77 42.45 43.13 43.81 44.480.6 45.15 45.81 46.47 47.13 47.78 48.43 49.07 49.71 50.35 50.980.7 51.61 52.23 52.85 53.46 54.07 54.67 55.27 55.87 56.46 57.050.8 57.63 58.21 58.78 59.35 59.91 60.47 61.02 61.57 62.11 62.650.9 63.19 63.72 64.24 64.76 65.28 65.79 66.29 66.80 67.29 67.781.0 68.27 68.75 69.23 69.70 70.17 70.63 71.09 71.54 71.99 72.431.1 72.87 73.30 73.73 74.15 74.57 74.99 75.40 75.80 76.20 76.601.2 76.99 77.37 77.75 78.13 78.50 78.87 79.23 79.59 79.95 80.291.3 80.64 80.98 81.32 81.65 81.98 82.30 82.62 82.93 83.24 83.551.4 83.85 84.15 84.44 84.73 85.01 85.29 85.57 85.84 86.11 86.381.5 86.64 86.90 87.15 87.40 87.64 87.89 88.12 88.36 88.59 88.821.6 89.04 89.26 89.48 89.69 89.90 90.11 90.31 90.51 90.70 90.901.7 91.09 91.27 91.46 91.64 91.81 91.99 92.16 92.33 92.49 92.651.8 92.81 92.97 93.12 93.27 93.42 93.57 93.71 93.85 93.99 94.121.9 94.26 94.39 94.51 94.64 94.76 94.88 95.00 95.12 95.23 95.342.0 95.45 95.56 95.66 95.76 95.86 95.96 96.06 96.15 96.25 96.342.1 96.43 96.51 96.60 96.68 96.76 96.84 96.92 97.00 97.07 97.152.2 97.22 97.29 97.36 97.43 97.49 97.56 97.62 97.68 97.74 97.802.3 97.86 97.91 97.97 98.02 98.07 98.12 98.17 98.22 98.27 98.322.4 98.36 98.40 98.45 98.49 98.53 98.57 98.61 98.65 98.69 98.722.5 98.76 98.79 98.83 98.86 98.89 98.92 98.95 98.98 99.01 99.042.6 99.07 99.09 99.12 99.15 99.17 99.20 99.22 99.24 99.26 99.292.7 99.31 99.33 99.35 99.37 99.39 99.40 99.42 99.44 99.46 99.472.8 99.49 99.50 99.52 99.53 99.55 99.56 99.58 99.59 99.60 99.612.9 99.63 99.64 99.65 99.66 99.67 99.68 99.69 99.70 99.71 99.723.0 99.73 99.74 99.75 99.76 99.76 99.77 99.78 99.79 99.79 99.803.1 99.81 99.81 99.82 99.83 99.83 99.84 99.84 99.85 99.85 99.863.2 99.86 99.87 99.87 99.88 99.88 99.88 99.89 99.89 99.90 99.903.3 99.90 99.91 99.91 99.91 99.92 99.92 99.92 99.92 99.93 99.933.4 99.93 99.94 99.94 99.94 99.94 99.94 99.95 99.95 99.95 99.953.5 99.95 99.96 99.96 99.96 99.96 99.96 99.96 99.96 99.97 99.973.6 99.97 99.97 99.97 99.97 99.97 99.97 99.97 99.98 99.98 99.983.7 99.98 99.98 99.98 99.98 99.98 99.98 99.98 99.98 99.98 99.983.8 99.99 99.99 99.99 99.99 99.99 99.99 99.99 99.99 99.99 99.993.9 99.99 99.99 99.99 99.99 99.99 99.99 99.99 99.99 99.99 99.99178

Table 7: Standard normal distribution z-values for a speci�c probability content from �zto z. Read column-wise and add marginal column and row z. Read column-wise and addmarginal column and row �gures to �nd probabilities.Prob. 0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.900.000 0.000 0.125 0.253 0.385 0.524 0.674 0.841 1.036 1.282 1.6450.002 0.002 0.128 0.256 0.388 0.527 0.677 0.845 1.041 1.287 1.6550.004 0.005 0.130 0.258 0.390 0.530 0.681 0.849 1.045 1.293 1.6650.006 0.007 0.133 0.261 0.393 0.533 0.684 0.852 1.049 1.299 1.6750.008 0.010 0.135 0.263 0.396 0.536 0.687 0.856 1.054 1.305 1.6850.010 0.012 0.138 0.266 0.398 0.538 0.690 0.859 1.058 1.311 1.6960.012 0.015 0.141 0.268 0.401 0.541 0.693 0.863 1.063 1.317 1.7060.014 0.017 0.143 0.271 0.404 0.544 0.696 0.867 1.067 1.323 1.7170.016 0.020 0.146 0.274 0.407 0.547 0.700 0.870 1.071 1.329 1.7280.018 0.022 0.148 0.276 0.409 0.550 0.703 0.874 1.076 1.335 1.7400.020 0.025 0.151 0.279 0.412 0.553 0.706 0.878 1.080 1.341 1.7510.022 0.027 0.153 0.281 0.415 0.556 0.709 0.881 1.085 1.347 1.7630.024 0.030 0.156 0.284 0.417 0.559 0.712 0.885 1.089 1.353 1.7750.026 0.033 0.158 0.287 0.420 0.562 0.716 0.889 1.094 1.360 1.7870.028 0.035 0.161 0.289 0.423 0.565 0.719 0.893 1.099 1.366 1.8000.030 0.038 0.163 0.292 0.426 0.568 0.722 0.896 1.103 1.372 1.8120.032 0.040 0.166 0.295 0.428 0.571 0.725 0.900 1.108 1.379 1.8250.034 0.043 0.168 0.297 0.431 0.574 0.729 0.904 1.112 1.385 1.8390.036 0.045 0.171 0.300 0.434 0.577 0.732 0.908 1.117 1.392 1.8530.038 0.048 0.173 0.302 0.437 0.580 0.735 0.911 1.122 1.399 1.8670.040 0.050 0.176 0.305 0.439 0.582 0.739 0.915 1.126 1.405 1.8810.042 0.053 0.179 0.308 0.442 0.585 0.742 0.919 1.131 1.412 1.8960.044 0.055 0.181 0.310 0.445 0.588 0.745 0.923 1.136 1.419 1.9110.046 0.058 0.184 0.313 0.448 0.591 0.749 0.927 1.141 1.426 1.9270.048 0.060 0.186 0.316 0.451 0.594 0.752 0.931 1.146 1.433 1.9440.050 0.063 0.189 0.318 0.453 0.597 0.755 0.935 1.150 1.440 1.9600.052 0.065 0.191 0.321 0.456 0.600 0.759 0.938 1.155 1.447 1.9780.054 0.068 0.194 0.323 0.459 0.603 0.762 0.942 1.160 1.454 1.9960.056 0.070 0.196 0.326 0.462 0.606 0.765 0.946 1.165 1.461 2.0150.058 0.073 0.199 0.329 0.464 0.609 0.769 0.950 1.170 1.469 2.0340.060 0.075 0.202 0.331 0.467 0.612 0.772 0.954 1.175 1.476 2.0540.062 0.078 0.204 0.334 0.470 0.615 0.775 0.958 1.180 1.484 2.0750.064 0.080 0.207 0.337 0.473 0.619 0.779 0.962 1.185 1.491 2.0970.066 0.083 0.209 0.339 0.476 0.622 0.782 0.966 1.190 1.499 2.1210.068 0.085 0.212 0.342 0.478 0.625 0.786 0.970 1.195 1.507 2.1450.070 0.088 0.214 0.345 0.481 0.628 0.789 0.974 1.200 1.514 2.1710.072 0.090 0.217 0.347 0.484 0.631 0.792 0.978 1.206 1.522 2.1980.074 0.093 0.219 0.350 0.487 0.634 0.796 0.982 1.211 1.530 2.2270.076 0.095 0.222 0.353 0.490 0.637 0.799 0.986 1.216 1.539 2.2580.078 0.098 0.225 0.355 0.493 0.640 0.803 0.990 1.221 1.547 2.2910.080 0.100 0.227 0.358 0.495 0.643 0.806 0.994 1.227 1.555 2.3270.082 0.103 0.230 0.361 0.498 0.646 0.810 0.999 1.232 1.564 2.3660.084 0.105 0.232 0.363 0.501 0.649 0.813 1.003 1.237 1.572 2.4090.086 0.108 0.235 0.366 0.504 0.652 0.817 1.007 1.243 1.581 2.4580.088 0.110 0.237 0.369 0.507 0.655 0.820 1.011 1.248 1.590 2.5130.090 0.113 0.240 0.371 0.510 0.659 0.824 1.015 1.254 1.599 2.5760.092 0.115 0.243 0.374 0.513 0.662 0.827 1.019 1.259 1.608 2.6520.094 0.118 0.245 0.377 0.515 0.665 0.831 1.024 1.265 1.617 2.7480.096 0.120 0.248 0.379 0.518 0.668 0.834 1.028 1.270 1.626 2.8790.098 0.123 0.250 0.382 0.521 0.671 0.838 1.032 1.276 1.636 3.091179

Table 8: Percentage points of the t-distribution1� �n 0.60 0.70 0.80 0.90 0.95 0.975 0.990 0.995 0.999 0.99951 0.325 0.727 1.376 3.078 6.314 12.71 31.82 63.66 318.3 636.62 0.289 0.617 1.061 1.886 2.920 4.303 6.965 9.925 22.33 31.603 0.277 0.584 0.978 1.638 2.353 3.182 4.541 5.841 10.21 12.924 0.271 0.569 0.941 1.533 2.132 2.776 3.747 4.604 7.173 8.6105 0.267 0.559 0.920 1.476 2.015 2.571 3.365 4.032 5.893 6.8696 0.265 0.553 0.906 1.440 1.943 2.447 3.143 3.707 5.208 5.9597 0.263 0.549 0.896 1.415 1.895 2.365 2.998 3.499 4.785 5.4088 0.262 0.546 0.889 1.397 1.860 2.306 2.896 3.355 4.501 5.0419 0.261 0.543 0.883 1.383 1.833 2.262 2.821 3.250 4.297 4.78110 0.260 0.542 0.879 1.372 1.812 2.228 2.764 3.169 4.144 4.58711 0.260 0.540 0.876 1.363 1.796 2.201 2.718 3.106 4.025 4.43712 0.259 0.539 0.873 1.356 1.782 2.179 2.681 3.055 3.930 4.31813 0.259 0.538 0.870 1.350 1.771 2.160 2.650 3.012 3.852 4.22114 0.258 0.537 0.868 1.345 1.761 2.145 2.624 2.977 3.787 4.14015 0.258 0.536 0.866 1.341 1.753 2.131 2.602 2.947 3.733 4.07316 0.258 0.535 0.865 1.337 1.746 2.120 2.583 2.921 3.686 4.01517 0.257 0.534 0.863 1.333 1.740 2.110 2.567 2.898 3.646 3.96518 0.257 0.534 0.862 1.330 1.734 2.101 2.552 2.878 3.610 3.92219 0.257 0.533 0.861 1.328 1.729 2.093 2.539 2.861 3.579 3.88320 0.257 0.533 0.860 1.325 1.725 2.086 2.528 2.845 3.552 3.85021 0.257 0.532 0.859 1.323 1.721 2.080 2.518 2.831 3.527 3.81922 0.256 0.532 0.858 1.321 1.717 2.074 2.508 2.819 3.505 3.79223 0.256 0.532 0.858 1.319 1.714 2.069 2.500 2.807 3.485 3.76824 0.256 0.531 0.857 1.318 1.711 2.064 2.492 2.797 3.467 3.74525 0.256 0.531 0.856 1.316 1.708 2.060 2.485 2.787 3.450 3.72526 0.256 0.531 0.856 1.315 1.706 2.056 2.479 2.779 3.435 3.70727 0.256 0.531 0.855 1.314 1.703 2.052 2.473 2.771 3.421 3.69028 0.256 0.530 0.855 1.313 1.701 2.048 2.467 2.763 3.408 3.67429 0.256 0.530 0.854 1.311 1.699 2.045 2.462 2.756 3.396 3.65930 0.256 0.530 0.854 1.310 1.697 2.042 2.457 2.750 3.385 3.64640 0.255 0.529 0.851 1.303 1.684 2.021 2.423 2.704 3.307 3.55150 0.255 0.528 0.849 1.299 1.676 2.009 2.403 2.678 3.261 3.49660 0.254 0.527 0.848 1.296 1.671 2.000 2.390 2.660 3.232 3.46070 0.254 0.527 0.847 1.294 1.667 1.994 2.381 2.648 3.211 3.43580 0.254 0.526 0.846 1.292 1.664 1.990 2.374 2.639 3.195 3.41690 0.254 0.526 0.846 1.291 1.662 1.987 2.368 2.632 3.183 3.402100 0.254 0.526 0.845 1.290 1.660 1.984 2.364 2.626 3.174 3.390110 0.254 0.526 0.845 1.289 1.659 1.982 2.361 2.621 3.166 3.381120 0.254 0.526 0.845 1.289 1.658 1.980 2.358 2.617 3.160 3.3731 0.253 0.524 0.842 1.282 1.645 1.960 2.326 2.576 3.090 3.291180

Table 9: Expressions for the Beta function B(m;n) for integer and half-integer argumentsn! 12 1 32 2 52 3 72 4 92 5m #12 �1 2 132 12� 23 18�2 43 12 415 1652 38� 25 116� 435 3128�3 1615 13 16105 112 16315 13072 516� 27 5128� 463 3256� 16693 51024�4 3235 14 32315 120 321155 160 323003 114092 35128� 29 7256� 499 71024� 161287 52048� 326435 3532768�5 256315 15 2563465 130 25615015 1105 25645045 1280 256109395 1630112 63256� 211 211024� 4143 92048� 162145 4532768� 3212155 3565536� 2562309456 512693 16 5129009 142 51245045 1168 512153153 1504 512415701 11260132 2311024� 213 332048� 4195 9932768� 163315 5565536� 3220995 77262144� 2564408957 20483003 17 204845045 156 2048255255 1252 2048969969 1840 20482909907 12310152 4292048� 215 42932768� 4255 14365536 164845 143262144� 3233915 91524288� 256780045n! 112 6 132 7 152m #112 63262144�6 512969969 12772132 63524288� 5122028117 2314194304�7 20487436429 15544 204816900975 112012152 2734194304� 5123900225 2318388608� 204835102025 42933554432�181

Mathematical ConstantsIntroductionIt is handy to have available the values of di�er-ent mathematical constants appearing in many ex-pressions in statistical calculations. In this sectionwe list, with high precision, many of those whichmay be needed. In some cases we give, after the ta-bles, basic expressions which may be nice to recall.Note, however, that this is not full explanations andconsult the main text or other sources for details.Some Basic Constantsexact approx.� 3.14159 26535 89793 23846e 2.71828 18284 59045 23536 0.57721 56649 01532 86061p� 1.77245 38509 05516 027301=p2� 0.39894 22804 01432 67794e = 1Xn=0 1n! = limn!1�1 + 1n�n = limn!1 nXk=1 1k � lnn!Gamma Functionexact approx.�(12) p� 1.77245 38509 05516 02730�(32) 12p� 0.88622 69254 52758 01365�(52) 34p� 1.32934 03881 79137 02047�(72) 158 p� 3.32335 09704 47842 55118�(92) 10516 p� 11.63172 83965 67448 92914�(z) = 1Z0 tz�1e�tdtn! = �(n+ 1) = n�(n)� �n + 12� = (2n� 1)!!2n � �12� == (2n)!22nn!p�See further section 42.2 on page 156 and reference[42] for more details.

Beta FunctionFor exact expressions for the Beta function for half-integer and integer values see table 9 on page 181.B(a; b) = �(a)�(b)�(a+ b) == 1Z0 xa�1(1� x)b�1dx == 1Z0 xa�1(1 + x)a+b dxSee further section 42.6 on page 163.Digamma Functionexact approx. (12) � � 2 ln2 {1.96351 00260 21423 47944 (32) (12 ) + 2 0.03648 99739 78576 52056 (52) (12 ) + 83 0.70315 66406 45243 18723 (72) (12 ) + 4615 1.10315 66406 45243 18723 (92) (12 ) + 352105 1.38887 09263 59528 90151 (112 ) (92 ) + 29 1.61109 31485 81751 12373 (132 ) (112 ) + 211 1.79291 13303 99932 94192 (152 ) (132 ) + 213 1.94675 74842 46086 78807 (172 ) (152 ) + 215 2.08009 08175 94201 21402 (192 ) (172 ) + 217 2.19773 78764 02949 53317 (1) � {0.57721 56649 01532 86061 (2) 1� 0.42278 43350 98467 13939 (3) 32 � 0.92278 43350 98467 13939 (4) 116 � 1.25611 76684 31800 47273 (5) 2512 � 1.50611 76684 31800 47273 (6) 13760 � 1.70611 76684 31800 47273 (7) 4920 � 1.87278 43350 98467 13939 (8) 363140 � 2.01564 14779 55609 99654 (9) 761280 � 2.14064 14779 55609 99654 (10) 71292520 � 2.25175 25890 66721 10765 (z) = ddz ln �(z) = 1�(z) d�(z)dz (z + 1) = (z) + 1z (n) = � + n�1Xm=1 1m (n + 12) = � � 2 ln 2 + 2 nXm=1 12m � 1182

See further section 42.3 on page 159.Polygamma Functionexact approx. (1)(12 ) �2=2 4.93480 22005 44679 30942 (2)(12 ) �14�3 {16.82879 66442 34319 99560 (3)(12 ) �4 97.40909 10340 02437 23644 (4)(12 ) �744�5 {771.47424 98266 67225 19054 (5)(12 ) 8�6 7691.11354 86024 35496 24176 (1)(1) �2 1.64493 40668 48226 43647 (2)(1) �2�3 {2.40411 38063 19188 57080 (3)(1) 6�4 6.49393 94022 66829 14910 (4)(1) �24�5 {24.88626 61234 40878 23195 (5)(1) 120�6 122.08116 74381 33896 76574 (n)(z) = dndzn (z) = dn+1dzn+1 ln �(z) (n)(z) = (�1)n+1n! 1Xk=0 1(z + k)n+1 (n)(1) = (�1)n+1n!�n+1 (n)(12 ) = (2n+1 � 1) (n)(1) (m)(n + 1) = (�1)mm!���m+1 + 1 ++ 12m+1 + : : :+ 1nm+1 � == (m)(n) + (�1)mm! 1nm+1See further section 42.4 on page 160.Bernoulli NumbersSee table 4 on page 176.Riemann's Zeta-functionexact approx.�0 -1�1 1�2 �2=6 1.64493 40668 48226 43647�3 1.20205 69031 59594 28540�4 �4=90 1.08232 32337 11138 19152�5 1.03692 77551 43369 92633�6 �6=945 1.01734 30619 84449 13971�7 1.00834 92773 81922 82684�8 �8=9450 1.00407 73561 97944 33938�9 1.00200 83928 26082 21442�10 �10=93555 1.00099 45751 27818 08534

�n = 1Xk=1 1kn�2n = 22n�1�2njB2nj(2n)!See also page 60 and for details reference [31].Sum of PowersIn many calculations involving discrete distributionssums of powers are needed. A general formula forthis is given bynXk=1ki = iXj=0(�1)jBj�ij� ni�j+1i � j + 1where Bj denotes the Bernoulli numbers (see page176). More speci�callynXk=1k = n(n + 1)=2nXk=1k2 = n(n + 1)(2n+ 1)=6nXk=1k3 = n2(n+ 1)2=4 = nXk=1k!2nXk=1k4 = n(n + 1)(2n+ 1)(3n2 + 3n� 1)=30nXk=1k5 = n2(n+ 1)2(2n2 + 2n� 1)=12nXk=1k6 = n(n + 1)(2n+ 1)(3n4 + 6n3 � 3n+ 1)=42183

ERRATA et ADDENDAErrors in this report are corrected as they are found but for those who have printed anearly version of the hand-book we list here errata. These are thus already obsolete in thiscopy. Minor errors in language etc are not listed. Note, however, that a few additions(subsections and tables) have been made to the original report (see below).� Contents part now having roman page numbers thus shifting arabic page numbersfor the main text.� A new section 6.2 on conditional probability density for binormal distribution hasbeen added after the �rst edition� Section 42.6, formula, line 2, � changed into � givingf(x;�; �) = �2 e��jx��j� Section 10.3, formula 2, line 4 has been corrected�x(t) = E(e{tx) = e{t�E(e{t(x��)) = e{t��x��(t) = e{t� �2�2 + t2� Section 14.4, formula, line 2 changed to�(t) = E(e{tx) = 1� 1Z0 e({t� 1� )xdx = 11 � {t�� Section 18.1, �gure 14 was erroneous in early editions and should look as is nowshown in �gure 74.� Section 27.2, line 12: change �ri to �pi.� Section 27.6 on signi�cance levels for the multinomial distribution has been addedafter the �rst edition.� Section 27.7 on the case with equal group probabilities for a multinomial distributionhas been added after the �rst edition.� A small paragraph added to section 28.1 introducing the multinormal distribution.� A new section 28.2 on conditional probability density for the multinormal distributionhas been added after the �rst edition.� Section 36.4, �rst formula, line 5, should read:P (r) = rXk=0 �ke��k! = 1� P (r + 1; �)184

� Section 36.4, second formula, line 9, should read:P (r) = rXk=0 �ke��k! = 1 � 2�Z0 f(x; � = 2r + 2)dx� and in the next line it should read f(x; � = 2r + 2).� Section 42.5.2, formula 3, line 6, should read:�(z) = �z 1Z0 tz�1e��tdt for Re(z) > 0; Re(�) > 0� Section 42.6, line 6: a reference to table 9 has been added (cf below).� Table 9 on page 181, on the Beta function B(m;n) for integer and half-integer argu-ments, has been added after the �rst version of the paper.These were, mostly minor, changes up to the 18th of March 1998 in order of apperance.In October 1998 the �rst somewhat larger revision was made:� Some text concerning the coe�cient of kurtosis added in section 2.2.� Figure 6 for the chi-square distribution corrected for a normalization error for then = 10 curve.� Added �gure 8 for the chi distribution on page 45.� Added section 11 for the doubly non-central F -distribution and section 12 for thedoubly non-central t-distribution.� Added �gure 12 for the F -distribution on page 62.� Added section 30 on the non-central Beta-distribution on page 109.� For the non-central chi-square distribution we have added �gure 22 and subsections31.4 and 31.6 for the cumulative distribution and random number generation, respec-tively.� For the non-central F -distribution �gure 23 has been added on page 114. Errors inthe formul� for f(F 0;m;n; �) in the introduction and z1 in the section on approxi-mations have been corrected. Subsections 32.2 on moments, 32.3 for the cumulativedistribution, and 32.5 for random number generation have been added.� For the non-central t-distribution �gure 24 has been added on page 117, some textaltered in the �rst subsection, and an error corrected in the denominator of theapproximation formula in subsection 33.5. Subsections 33.2 on the derivation of thedistribution, 33.3 on its moments, 33.4 on the cumulative distribution, and 33.6 onrandom number generation have been added.185

� A new subsection 34.8.9 has been added on yet another method, using a ratio betweentwo uniform deviates, to achieve standard normal random numbers. With this changethree new references [38{40] were introduced.� A comparison of the e�ciency for di�erent algorithms to obtain standard normalrandom numbers have been introduced as subsection 34.8.10.� Added a comment on factorial moments and cumulants for a Poisson distribution insection 36.2.� This list of \Errata et Addenda" for past versions of the hand-book has been addedon page 184 and onwards.� Table 2 on page 174 and table 3 on page 175 for extreme signi�cance levels of thechi-square distribution have been added thus shifting the numbers of several othertables. This also slightly a�ected the text in section 8.10.� The Bernoulli numbers used in section 15.4 now follow the same convention used e.g.in section 42.3. This change also a�ected the formula for �2n in section 23.4. Table4 on page 176 on Bernoulli numbers was introduced at the same time shifting thenumbers of several other tables.� A list of some mathematical constants which are useful in statistical calculations havebeen introduced on page 182.Minor changes afterwards include:� Added a \proof" for the formula for algebraic moments of the log-normal distributionin section 24.2 and added a section for the cumulative distribution as section 24.3.� Added formula also for c < 0 for F (x) of a Generalized Gamma distribution in section18.2.� Corrected bug in �rst formula in section 6.6.� Replaced table for multinormal con�dence levels on page 101 with a more precise onebased on an analytical formula.� New section on sums of powers on page 183.186

References[1] The Advanced Theory of Statistics by M. G. Kendall and A. Stuart, Vol. 1, Charles Gri�n& Company Limited, London 1958.[2] The Advanced Theory of Statistics by M. G. Kendall and A. Stuart, Vol. 2, Charles Gri�n& Company Limited, London 1961.[3] An Introduction to Mathematical Statistics and Its Applications by Richard J. Larsen andMorris L. Marx, Prentice-Hall International, Inc. (1986).[4] Statistical Methods in Experimental Physics byW. T. Eadie, D. Drijard, F. E. James, M. Roosand B. Sadoulet, North-Holland Publishing Company, Amsterdam-London (1971).[5] Probability and Statistics in Particle Physics by A. G. Frodesen, O. Skjeggestad and H. T�fte,Universitetsforlaget, Bergen-Oslo-Troms� (1979).[6] Statistics for Nuclear and Particle Physics by Louis Lyons, Cambridge University Press(1986).[7] Statistics { A Guide to the Use of Statistical Methods in the Physical Sciences by RogerJ. Barlow, John Wiley & Sons Ltd., 1989.[8] Statistical Data Analysis by Glen Cowan, Oxford University Press, 1998.[9] Numerical Recipes (The Art of Scienti�c Computing) by William H. Press, Brian P. Flannery,Saul A. Teukolsky and William T. Vetterling, Cambridge University Press, 1986.[10] Numerical Recipes in Fortran (The Art of Scienti�c Computing), second edition by WilliamH. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery, Cambridge Uni-versity Press, 1992.[11] Handbook of Mathematical Functions { with Formulas, Graphs, and Mathematical Tables,edited by Milton Abramowitz and Irene A. Stegun, Dover Publications, Inc., New York,1965.[12] Statistical Distributions by N. A. J. Hastings and J. B. Peacock, Butterworth & Co (Pub-lishers) Ltd, 1975.[13] A Monte Carlo Sampler by C. J. Everett and E. D. Cashwell, LA-5061-MS Informal Report,October 1972, Los Alamos Scienti�c Laboratory of the University of California, New Mexico.[14] Random Number Generation by Christian Walck, USIP Report 87-15, Stockholm University,December 1987.[15] A Review of Pseudorandom Number Generators by F. James, Computer Physics Communi-cations 60 (1990) 329{344.[16] Toward a Universal Random Number Generator by George Marsaglia, Arif Zaman and WaiWan Tsang, Statistics & Probability Letters 9 (1990) 35{39.[17] Implementation of a New Uniform Random Number Generator (including benchmark tests)by Christian Walck, Internal Note SUF{PFY/89{01, Particle Physics Group, Fysikum,Stockholm University, 21 December 1989.187

[18] A Random Number Generator for PC's by George Marsaglia, B. Narasimhan and Arif Zaman,Computer Physics Communications 60 (1990) 345{349.[19] A Portable High-Quality Random Number Generator for Lattice Field Theory Simulationsby Martin L�uscher, Computer Physics Communications 79 (1994) 100{110.[20] Implementation of Yet Another Uniform Random Number Generator by Christian Walck,Internal Note SUF{PFY/94{01, Particle Physics Group, Fysikum, Stockholm University, 17February 1994.[21] Random number generation by Birger Jansson, Victor Pettersons Bokindustri AB, Stock-holm, 1966.[22] J. W. Butler, Symp. on Monte Carlo Methods, Whiley, New York (1956) 249{264.[23] J. C. Butcher, Comp. J. 3 (1961) 251{253.[24] Ars Conjectandi by Jacques Bernoulli, published posthumously in 1713.[25] G. E. P. Box and M. E. Muller in Annals of Math. Stat. 29 (1958) 610{611.[26] Probability Functions by M. Zelen and N. C. Severo in Handbook of Mathematical Functions,ed. M. Abramowitz and I. A. Stegun, Dover Publications, Inc., New York, 1965, 925.[27] Error Function and Fresnel Integrals by Walter Gautschi in Handbook of Mathematical Func-tions, ed. M. Abramowitz and I. A. Stegun, Dover Publications, Inc., New York, 1965, 295.[28] Computer Methods for Sampling From the Exponential and Normal Distributions byJ. H. Ahrens and U. Dieter, Communications of the ACM 15 (1972) 873.[29] J. von Neumann, Nat. Bureau Standards, AMS 12 (1951) 36.[30] G. Marsaglia, Ann. Math. Stat. 32 (1961) 899.[31] Bernoulli and Euler Polynomials | Riemann Zeta Function by Emilie V. Haynsworth andKarl Goldberg in Handbook of Mathematical Functions, ed. M. Abramowitz and I. A. Stegun,Dover Publications, Inc., New York, 1965, 803.[32] Irrationalit�e de �(2) et �(3) by R. Ap�ery, Ast�erisque 61 (1979) 11{13.[33] Multiplicity Distributions in Strong Interactions: A Generalized Negative Binomial Model byS. Hegyi, Phys. Lett. B387 (1996) 642.[34] Probability, Random Variables and Stochastic Processes by Athanasios Papoulis, McGraw-Hill book company (1965).[35] Theory of Ionization Fluctuation by J. E. Moyal, Phil. Mag. 46 (1955) 263.[36] A New Empirical Regularity for Multiplicity Distributions in Place of KNO Scaling by theUA5 Collaboration: G. J. Alner et al., Phys. Lett. B160 (1985) 199.[37] G. Marsaglia, M. D. MacLaren and T. A. Bray. Comm. ACM 7 (1964) 4{10.188

[38] Computer Generation of Random Variables Using the Ratio of Uniform Deviates byA. J. Kinderman and John F. Monahan, ACM Transactions on Mathematical Software 3(1977) 257{260.[39] A Fast Normal Random Number Generator by Joseph L. Leva, ACM Transactions on Math-ematical Software 18 (1992) 449{453.[40] Algorithm 712: A Normal Random Number Generator by Joseph L. Leva, ACM Transactionson Mathematical Software 18 (1992) 454{455.[41] Mathematical Methods for Physicists by George Arfken, Academic Press, 1970.[42] Gamma Function and Related Functions by Philip J. Davis in Handbook of MathematicalFunctions, ed. M. Abramowitz and I. A. Stegun, Dover Publications, Inc., New York, 1965,253.[43] Tables of Integrals, Series, and Products by I. S. Gradshteyn and I. M. Ryzhik, FourthEdition, Academic Press, New York and London, 1965.[44] The Special Functions and their Approximations by Yudell L. Luke, Volume 1, AcademicPress, New York and London, 1969.[45] Programs for Calculating the Logarithm of the Gamma Function, and the Digamma Function,for Complex Argument by K. S. K�olbig, Computer Physics Communications 4 (1972) 221{226.[46] Hypergeometric Functions by Fritz Oberhettinger in Handbook of Mathematical Functions,ed. M. Abramowitz and I. A. Stegun, Dover Publications, Inc., New York, 1965, 555.[47] Con uent Hypergeometric Functions by Lucy Joan Slater in Handbook of Mathematical Func-tions, ed. M. Abramowitz and I. A. Stegun, Dover Publications, Inc., New York, 1965, 503.189

IndexAAbsolute moments : : : : : : : : : : : : : : : : : : : : : 121Accept-reject technique : : : : : : : : : : : : : : : : : : 7Addition theorem (Normal) : : : : : : : : : : : : 122Addition theorem (Poisson) : : : : : : : : : : : : 136Algebraic moments : : : : : : : : : : : : : : : : : : : : : : :3Analysis of variance : : : : : : : : : : : : : : : : : : : : :66Ap�ery's number : : : : : : : : : : : : : : : : : : : : : : : : :61BBernoulli, Jacques : : : : : : : : : : : : : : : : : : : : : : 12Bernoulli distribution : : : : : : : : : : : : : : : : : : : 12Bernoulli numbers : : : : : : : : : : : : : : : : : : 60,160Table : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :176Beta distribution : : : : : : : : : : : : : : : : : : : : : : : :13Non-central : : : : : : : : : : : : : : : : : : : : : : : :109Beta function : : : : : : : : : : : : : : : : : : : : : : : : : : 163Table : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :181Binomial distribution : : : : : : : : : : : : : : : : : : : 16Binormal distribution : : : : : : : : : : : : : : : : : : : 20Bivariate moments : : : : : : : : : : : : : : : : : : : : : : : 9Box-Muller transformation : : : : : : : : : : : : : : 22Branching process : : : : : : : : : : : : : : : : : : : :6,107Breit, Gregory : : : : : : : : : : : : : : : : : : : : : : : : : : 28Breit-Wigner distribution : : : : : : : : : : : : : : : 28CCauchy, Augustin Louis : : : : : : : : : : : : : : : : : 26Cauchy distribution : : : : : : : : : : : : : : : : : : : : :26Central moments : : : : : : : : : : : : : : : : : : : : : : : : :3Characteristic function : : : : : : : : : : : : : : : : : : : 4Chi distribution : : : : : : : : : : : : : : : : : : : : : : : : :44Chi-square distribution : : : : : : : : : : : : : : : : : :37Extreme con�dence levels : : : : : : 174,175Non-central : : : : : : : : : : : : : : : : : : : : : : : :111Percentage points : : : : : : : : : : : : : : : : : :173Cholesky decomposition : : : : : : : : : : : : : : : :102Compound Poisson distribution : : : : : : : : : 46Con uent hypergeometric function : : : : : 170Constants : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 182Correlation coe�cient : : : : : : : : : : : : : : : : : : :10Covariance : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :9Covariance form : : : : : : : : : : : : : : : : : : : : : : : 100Cumulant generating function : : : : : : : : : : : : 6

Cumulants : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 6Logistic distribution : : : : : : : : : : : : : : : : 85Moyal distribution : : : : : : : : : : : : : : : : : : 93Non-central chi-square distribution :112Poisson distribution : : : : : : : : : : : : : : : :136Cumulative function : : : : : : : : : : : : : : : : : : : : : 3DDigamma function : : : : : : : : : : : : : : : : : : : : : 159Distribution function : : : : : : : : : : : : : : : : : : : : :3Double-Exponential distribution : : : : : : : : :48Doubly non-central F -distribution : : : : : : :50Doubly non-central t-distribution : : : : : : : :52EErlangian distribution : : : : : : : : : : : : : : : : : : :70Error function : : : : : : : : : : : : : : : : : : : : : : : : : : 54Euler gamma-function : : : : : : : : : : : : : : : : : : 28Euler's constant : : : : : : : : : : : : : : : : : : : : : : : : :60Euler-Mascheroni constant : : : : : : : : : : : : : : 60Excess, coe�cient of : : : : : : : : : : : : : : : : : : : : : 3Expectation value : : : : : : : : : : : : : : : : : : : : : : : :3Exponential distribution : : : : : : : : : : : : : : : : 55Extreme value distribution : : : : : : : : : : : : : : 58FFactorial : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :156Factorial moments : : : : : : : : : : : : : : : : : : : : : : : 6Poisson distribution : : : : : : : : : : : : : : : :136F -distribution : : : : : : : : : : : : : : : : : : : : : : : : : : 62Doubly non-central : : : : : : : : : : : : : : : : : 50Non-central : : : : : : : : : : : : : : : : : : : : : : : :114Percentage points : : : : : : : : : : : : : : : : : :177F -ratio : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 64Fisher, Sir Ronald Aylmer : : : : : : : : : : : 40,62Fisher-Tippett distribution : : : : : : : : : : : : : :58Fisher z-distribution : : : : : : : : : : : : : : : : : : : : 63GGamma distribution : : : : : : : : : : : : : : : : : : : : 70Gamma function : : : : : : : : : : : : : : : : : : : : : : :156Gauss distribution : : : : : : : : : : : : : : : : : : : : : 120Gauss's di�erential equation : : : : : : : : : : : 169Generalized Gamma distribution : : : : : : : : 74190

Geometric distribution : : : : : : : : : : : : : : : : : : 76Gosset, William Sealy : : : : : : : : : : : : : : : : : :142Guinness brewery, Dublin : : : : : : : : : : : : : : 142Gumbel, E. J. : : : : : : : : : : : : : : : : : : : : : : : : : : :58Gumbel distribution : : : : : : : : : : : : : : : : : : : : 58HHilferty, M. M. : : : : : : : : : : : : : : : : : : : : : : :40,73Hit-miss technique : : : : : : : : : : : : : : : : : : : : : : : 7Hyperexponential distribution : : : : : : : : : : :78Hypergeometric distribution : : : : : : : : : : : : :80Hypergeometric function : : : : : : : : : : : : : : : 169IIncomplete Beta function : : : : : : : : : : : : : : 163Incomplete Gamma distribution : : : : : : : :161Independence : : : : : : : : : : : : : : : : : : : : : : : : : : : : 5Independence theorem (normal) : : : : : : : :122JJohnk's algorithm : : : : : : : : : : : : : : : : : : : :72,91KKendall, Maurice G. : : : : : : : : : : : : : : : : : : : : : 1Kinetic theory : : : : : : : : : : : : : : : : : : : : : : : : : : 90Kinetic theory, 2-dim. : : : : : : : : : : : : : : : : : 140Kurtosis, Coe�cient of : : : : : : : : : : : : : : : : : : : 3Kummer's equation : : : : : : : : : : : : : : : : : : : : 170Kummer transformations : : : : : : : : : : : : : : 171LLanczos formula : : : : : : : : : : : : : : : : : : : : : : : 157Laplace, Pierre Simon de : : : : : : : : : : : : : : : :48Laplace distribution : : : : : : : : : : : : : : : : : : : : :48Leptokurtic : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 3Logarithmic distribution : : : : : : : : : : : : : : : : 82Logistic distribution : : : : : : : : : : : : : : : : : : : : 84Log-normal distribution : : : : : : : : : : : : : : : : :87Log-Weibull distribution : : : : : : : : : : : : : : : : 58MMathematical constants : : : : : : : : : : : : : : : :182Maxwell, James Clerk : : : : : : : : : : : : : : : : : : :89Maxwell distribution : : : : : : : : : : : : : : : : : : : : 89Kinetic theory : : : : : : : : : : : : : : : : : : : : : : 90Median : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 27,31Mesokurtic : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :3

Mode : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 27Moyal distribution : : : : : : : : : : : : : : : : : : : : : : 92Multinomial distribution : : : : : : : : : : : : : : : : 96Multinormal distribution : : : : : : : : : : : : : : :100Multivariate moments : : : : : : : : : : : : : : : : : : : :9NNegative binomial distribution : : : : : : : : : 103Non-central Beta-distribution : : : : : : : : : : 109Non-central chi-square distribution : : : : : 111Non-central F -distribution : : : : : : : : : : : : : 114Non-central t-distribution : : : : : : : : : : : : : : 117Normal distribution : : : : : : : : : : : : : : : : : : : :120Addition theorem : : : : : : : : : : : : : : : : : :122Independence theorem : : : : : : : : : : : : : 122Tables : : : : : : : : : : : : : : : : : : : : : : : : : 178,179PPareto. Vilfredo : : : : : : : : : : : : : : : : : : : : : : : 134Pareto distribution : : : : : : : : : : : : : : : : : : : : :134Pascal, Blaise : : : : : : : : : : : : : : : : : : : : : : : : : : 103Pascal distribution : : : : : : : : : : : : : : : : : : : : :103Platykurtic : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 3Pochhammer symbol : : : : : : : : : : : : : : : : : : :169Poisson, Sim�eon Denis : : : : : : : : : : : : : : : : : 135Poisson distribution : : : : : : : : : : : : : : : : : : : :135Addition theorem : : : : : : : : : : : : : : : : : :136Polygamma function : : : : : : : : : : : : : : : : : : : 160Probability generating function : : : : : : : : : : :5Branching process : : : : : : : : : : : : : : : : : 107Psi function : : : : : : : : : : : : : : : : : : : : : : : : : : : 159QQuantile : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 32Quartile : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 32RRayleigh, Lord : : : : : : : : : : : : : : : : : : : : : : : : :139Rayleigh distribution : : : : : : : : : : : : : : : : : : 139Kinetic theory, 2-dim. : : : : : : : : : : : : : 140Riemann's zeta-function : : : : : : : : : : : : : : : : :60SSemi-faculty : : : : : : : : : : : : : : : : : : : : : : : : : : : 158Semi-interquartile range : : : : : : : : : : : : : : : : :32Skewness, Coe�cient of : : : : : : : : : : : : : : : : : : 3Standard normal distribution : : : : : : : : : : 120191

Stirling's formula : : : : : : : : : : : : : : : : : : : : : : 157Stuart, Alan : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 1Student's t-distribution : : : : : : : : : : : : : : : : 142Tt-distribution : : : : : : : : : : : : : : : : : : : : : : : : : : 142Doubly non-central : : : : : : : : : : : : : : : : : 52Non-central : : : : : : : : : : : : : : : : : : : : : : : :117Percentage points : : : : : : : : : : : : : : : : : :180t-ratio : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 145Trapezoidal distribution : : : : : : : : : : : : : : : 152Triangular distribution : : : : : : : : : : : : : : : : :152UUniform distribution : : : : : : : : : : : : : : : : : : : 153VVariance : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 3Variance-ratio distribution : : : : : : : : : : : : : : 62WWeibull, Waloddi : : : : : : : : : : : : : : : : : : : : : : 154Weibull distribution : : : : : : : : : : : : : : : : : : : :154Wigner, Eugene : : : : : : : : : : : : : : : : : : : : : : : : :28Wilson, E. B. : : : : : : : : : : : : : : : : : : : : : : : : 40,73ZZeta-function, Riemann's : : : : : : : : : : : : : : : :60192