entropy

 function H = entropy(var,min,max,laplace)

 entropy: Calculates the entropy for a discrete variables with positive states from min to max.
 INPUT:
 min: The minimum value
 max: The maximum value
 var: Array with the values for the discrete variable
 laplace: Determines wether or not Laplace correction is used in the computation
 of the probabilities. Laplace=1 (It is used), otherwise  it is not.
 OUTPUT:
 H: The entropy for a given variable.

 Last version 5/11/08. Carlos Echegoyen (carlos.echegoyen@ehu.es)

0001 function H = entropy(var,min,max,laplace)
0002 % function H = entropy(var,min,max,laplace)
0003 %
0004 % entropy: Calculates the entropy for a discrete variables with positive states from min to max.
0005 % INPUT:
0006 % min: The minimum value
0007 % max: The maximum value
0008 % var: Array with the values for the discrete variable
0009 % laplace: Determines wether or not Laplace correction is used in the computation
0010 % of the probabilities. Laplace=1 (It is used), otherwise  it is not.
0011 % OUTPUT:
0012 % H: The entropy for a given variable.
0013 %
0014 % Last version 5/11/08. Carlos Echegoyen (carlos.echegoyen@ehu.es)
0015 
0016 
0017 if size(var,2) == 1 
0018     var = var';
0019 end
0020 
0021 H = 0;
0022 N = size(var,2); 
0023 
0024 % One counter for each state
0025 nstates = max-min+1;
0026 cont=zeros(1,nstates);
0027 c=0;
0028 
0029 for j=min:max % sumo 1 por si hay ceros. Recorro los estados
0030     c = c + 1;
0031     cont(c) = sum(var==j);
0032     
0033     % Calculating probabilities
0034     if (laplace==1)
0035      Probs(c) = (cont(c)+1) / (N+nstates);
0036     else 
0037       Probs(c) = (cont(c)) / (nstates);   
0038     end
0039     
0040     % Calculating entropy
0041     if Probs(c)>0 
0042       H = H + Probs(c) * log2(Probs(c));
0043     end,
0044 end
0045 
0046 H = -H;
0047 
0048