Chapter 6: penalized ratio of gaussian likelihoods

In the ‘balanced’ example the two model likelihoods were weighted equally, as neither model allowed a search over the Binomial parameter space. Here, the situation is different, in that the second model searches a large prior range of values to find the best fit to the data. This ability to search the parameter space for a good fit downweights in the evidence calculation; i.e., it defines an Occam penalty.

%%% Model likelihoods and evidence

dat=[1.1,2.75,-.12,-1.23,3.84]; dat0=dat-mean(dat); Nx=length(dat);

sig=2; murange=[-3*sig 3*sig]; Dmu=diff(murange);

mu1=0;

x=linspace(murange(1),murange(2),1201);

L=ones(size(x));

for n=1:Nx, L=L.*npdf(dat(n),x,sig); end

figure(3); clf; hold on;

plot(x,L,'k-','LineWidth',2.2);

L1=nan(Nx,1); L2=nan(Nx,1); EV=nan(Nx,1);

Nx=31; xlist=linspace(-3,3,Nx);

for i=1:Nx; xnow=xlist(i);

dnow=dat0+xnow; mu2=mean(dnow);

L1(i)=prod(npdf(dnow,mu1,sig)); %uncertain model

L2(i)=prod(npdf(dnow,mu2,sig)); %single-mu model

%%% compute evidence

%Eraw(i)=10*log10(

EV(i)=10*(log10(L2(i)./(Dmu*sqrt(Nx)))-log10(L1(i)./(sig*sqrt(2*pi)))); end

%%% plotting

COL={.4*[1 1 1],.8*[1 1 1]};

ind=findnearestN(xlist,mean(dat),1);

figure(3);

stem(mu1,L1(ind),'k--','LineWidth',1.2); plot(mu1,L1(ind),'ko','MarkerFaceColor',COL{1},'MarkerSize',12,'LineWidth',1);

stem(mean(dat),L2(ind),'k--','LineWidth',1.2); plot(mean(dat),L2,'ko','MarkerFaceColor',COL{2},'MarkerSize',12,'LineWidth',1);

xlabel(['location parameter (\mu)'],'FontName','Arial','FontSize',15)

ylabel('likelihood','FontName','Arial','FontSize',15)

figure(4); clf; hold on

plot(xlist,L2./L1,'ko','MarkerFaceColor',.7*[1 1 1],'MarkerSize',8)

plot(xlist([1 end]),[1 1]./(sig*sqrt(2*pi)/(Dmu*sqrt(Nx))),'k--','LineWidth',2)

figure(5); clf; hold on

for n=1:Nx,

plot(xlist(n),EV(n),'ko','MarkerFaceColor',COL{(EV(n)>0)+1},'MarkerSize',9,'LineWidth',1); end

plot(xlist([1 end]),[0 0],'k--','LineWidth',2)

xlabel(['data mean'],'FontSize',15)

ylabel('evidence (db)','FontName','Arial','FontSize',15)