Statistics of the Information Component in Bayesian Neural Networks

Abstract

In previous work approximate solutions have been used for expectation and variance of the {\em information component (IC)}. This report presents an analytical approach to calculate exact expressions for the expectation and variance of the information component (IC). The IC is used in a Bayesian neural network [ref TRITA-NA-P9325] as a weight between neurons representing discrete events. The IC relates the information possessed about one state of one variable with one state of another variable, and is used for calculation of a posterior probability distribution conditioned on a set of given input events. It is used as a measure of disproportionality in data mining
Explanation of data mining methods .

The mutual information between two variables, as defined in information theory [ref Shannon 1948, A Mathematical Theory of Communication], can in its discrete form be regarded as a weighted sum of ICs. The expectation of the IC provides a measure of the strength of an association between two states and its variance a measure of the uncertainty, which is essential for low counter values.


Timo Koski

Roland Orre
Last modified: Mon Feb 17 03:38:06 CET 2003