glitinosim3

2022-06-30

I am trying to derive the entropy in normal distribution. Let $p\left(x\right)$ to be the probability density function of uniform normal distribution
$p\left(x\right)=\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{{x}^{2}}{2{\sigma }^{2}}}$
hence, by using integration by parts, we have
${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{x}^{2}p\left(x\right)dx={x}^{2}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}p\left(x\right)dx-{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}2x\left({\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}p\left(x\right)dx\right)dx$
Because
${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}p\left(x\right)dx=100\mathrm{%}$
we have
${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{x}^{2}p\left(x\right)dx={x}^{2}-{x}^{2}+C=C$
However, lots of relevant proofs online says that
${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{x}^{2}p\left(x\right)dx={\sigma }^{2}$
Does anyone know the reason?

Kathryn Moody

Expert

By definition, if $X$ has density $p\left(x\right)$ then $E{X}^{2}={\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{x}^{2}p\left(x\right)\phantom{\rule{thinmathspace}{0ex}}dx$. So here we have
${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{x}^{2}p\left(x\right)\phantom{\rule{thinmathspace}{0ex}}dx=E{X}^{2}=Var\left(X\right)+\left(EX{\right)}^{2}=Var\left(X\right)={\sigma }^{2}.$
If you really need to use integration method, here is one.
$\begin{array}{rl}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{x}^{2}p\left(x\right)\phantom{\rule{thinmathspace}{0ex}}dx& =\frac{1}{\sqrt{2\pi }\sigma }{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{x}^{2}{e}^{-\frac{{x}^{2}}{2{\sigma }^{2}}}\phantom{\rule{thinmathspace}{0ex}}dx\\ & =\frac{1}{\sqrt{2\pi }\sigma }{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}-{\sigma }^{2}x\phantom{\rule{thinmathspace}{0ex}}d\left({e}^{-\frac{{x}^{2}}{2{\sigma }^{2}}}\right)\\ & =\frac{1}{\sqrt{2\pi }\sigma }\left(-{\sigma }^{2}x{e}^{-\frac{{x}^{2}}{2{\sigma }^{2}}}{\mid }_{-\mathrm{\infty }}^{\mathrm{\infty }}+{\sigma }^{2}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{e}^{-\frac{{x}^{2}}{2{\sigma }^{2}}}\phantom{\rule{thinmathspace}{0ex}}dx\right)\\ & ={\sigma }^{2}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}\frac{1}{\sqrt{2\pi }\sigma }{e}^{-\frac{{x}^{2}}{2{\sigma }^{2}}}\phantom{\rule{thinmathspace}{0ex}}dx\\ & ={\sigma }^{2}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}p\left(x\right)\phantom{\rule{thinmathspace}{0ex}}dx\\ & ={\sigma }^{2}.\end{array}$
Finally, the entropy.
$\begin{array}{rl}H& =-{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}p\left(x\right)\mathrm{log}p\left(x\right)\phantom{\rule{thinmathspace}{0ex}}dx\\ & =\frac{1}{2{\sigma }^{2}}{\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}{x}^{2}p\left(x\right)\phantom{\rule{thinmathspace}{0ex}}dx+\mathrm{log}\left(\sqrt{2\pi }\sigma \right){\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}p\left(x\right)\phantom{\rule{thinmathspace}{0ex}}dx\\ & =\frac{1}{2}+\mathrm{log}\left(\sqrt{2\pi }\sigma \right)\\ & =\mathrm{log}\left(\sqrt{2\pi e}\sigma \right).\end{array}$

Do you have a similar question?