Lesson #8: Statistical Inference

In this lesson, we will almost exclusivly discuss estimation of a parameter in a parametric model.

Let (Subscript[X, 1]),(Subscript[X, 2]),...,(Subscript[X, n]) be a "random i.i.d. sample" from distribution f((Subscript[x, i])|θ).

We are interested in using the observations (Subscript[X, 1]),(Subscript[X, 2]),...,(Subscript[X, n]) to "guess" what is g(θ), often just θ.

Example:
Let (Subscript[X, i])~Gamma(α,λ).
Here θ = {α,λ}.  We may be interested in estimating g(θ)=(α/λ), the mean. Or,  g(θ)={θ}. Or g(θ)=(α/(λ^2)), the variance, etc..

An estimator, (OverBar[θ]) (I wanted hat)   is a function of (Subscript[X, 1]),(Subscript[X, 2]),...,(Subscript[X, n]).  It is thus a random variable.

Maximum Likelihood Estimation

Example - Gamma(α,β)

PDF[GammaDistribution[α,β],x]

Mean[GammaDistribution[α,β]]

like[α_,β_,xx_]:=Apply[Times,Table[(((E^(-((xx[[i]])/β))) ((xx[[i]])^(-1+α)) (β^(-α)))/(Gamma[α])),{i,1,Length[xx]}]]

plotSample[xx_]:=Plot3D[like[α,β,xx],{α,0,10},{β,0,10},PlotRange->All,AxesLabel->{α,β},PlotPoints->50]

sample=RandomReal[GammaDistribution[2,2],10];
plotSample[sample]

NMaximize[like[α,β,sample],{α,β}]

NMaximize[Log[like[α,β,sample]],{α,β}]

Now lets make a "serious" demonstration of this ...

Demonstration During Class

logLike[α_,β_,xx_]:=Log[Apply[Times,Table[(((E^(-((xx[[i]])/β))) ((xx[[i]])^(-1+α)) (β^(-α)))/(Gamma[α])),{i,1,Length[xx]}]]]

testAlpha=1;testBetta=1;
repeats=30;
data=Table[
Map[{α,β} /.NMaximize[#,{α,β}][[2]]&,
Table[logLike[α,β,RandomReal[GammaDistribution[testAlpha,testBetta],sampleSize]],
{30}]
],{sampleSize,100,1000,100}];

Manipulate[ListPlot[data[[i]],AxesOrigin->{0,0},PlotRange->{{0,2},{0,2}},Epilog->{Red,PointSize[Large],Point[{1,1}]}]
,
{i,1,10,1}]