Gradients of the fundamental information measures: Theory and applications

Mahboobeh Sedighizad | Babak Seyfe

---

URL :   http://research.shahed.ac.ir/WSR/WebPages/Report/PaperView.aspx?PaperID=116611
Date :  2019/06/19
Publish in :    Signal Processing

Link :  http://dx.doi.org/10.1016/j.sigpro.2019.04.019
Keywords :measures, Theory

Abstract :

---

Gradient of the information measures can be considered as the building block of the fundamental re- lations between information and estimation theories established in the literature. This paper introduces the gradients of two fundamental information measures, i.e., differential entropy and mutual information in some general cases. Our approach is based on the utilizing a general functional model to describe the stochastic systems. This approach enables us to give a simple new proof for De Bruijn’s identity. We also derive a general expression for the gradient of the mutual information in systems with both additive and multiplicative noises. Systems with additive noise, as the most common model for the com- munication channels, is considered as well, where in the Gaussian case some of the available results in the literature are recovered. Moreover, the aforementioned general functional model allows us to study the sensitivity of the input–output mutual information to the system parameters in some other practi- cal scenarios. Specifically, Orthogonal Frequency Division Multiplexing (OFDM) systems with High Power Amplifier (HPA) are considered and an information theoretic analysis for the behavior of these systems is given.