Value of Information in Statistical Environments

Title: Value of Information in Statistical Environments (w/ Roman V. Belavkin)

Abstract: The Value of Information (VoI) framework, as developed by Ruslan Stratonovich, bridges Claude Shannon’s information theory with economics, particularly utility and decision theory. VoI employs a Bayesian utility perspective, incorporating prior and posterior distributions to quantify the expected value of information in decision-making. This paper revisits the VoI concept within the Boolean setting of hypothesis testing, reviewing foundational aspects and applying them to classical frequentist tests such as the chi^2 test, where decisions are based on test statistics and critical values under the null hypothesis.

By relating mutual information to empirical test accuracy, we interpret frequentist performance measures through the lens of expected utility. This allows VoI to quantify the maximal achievable decision performance under equal priors, using the identity utility matrix. The VoI function, which can sometimes be derived analytically, serves as a theoretical upper bound on information-based accuracy.

This comparison reveals that empirical test accuracies often lie below the theoretical Value of Information curve, which indicates suboptimal use of the available information. Since the Value of Information curve represents the maximum achievable accuracy for a given level of mutual information, any consistent gap suggests that alternative statistical procedures or different bin sizes could extract decisions more effectively from the same data, highlighting the potential for improved test design or model selection. Moreover, this gap can serve as a proxy for the potential gains from increasing mutual information, implying that larger sample sizes or enhanced data collection may be worthwhile if they help reduce this gap. Finally, our analysis is extended beyond the Boolean case by exploring higher-dimensional utility structures, hinting at richer and more complex decision frameworks.