I am trying to derive the entropy in normal distribution. Let p ( x )...

glitinosim3

glitinosim3

Answered

2022-06-30

I am trying to derive the entropy in normal distribution. Let p ( x ) to be the probability density function of uniform normal distribution
p ( x ) = 1 2 π σ e x 2 2 σ 2
hence, by using integration by parts, we have
x 2 p ( x ) d x = x 2 p ( x ) d x 2 x ( p ( x ) d x ) d x
Because
p ( x ) d x = 100 %
we have
x 2 p ( x ) d x = x 2 x 2 + C = C
However, lots of relevant proofs online says that
x 2 p ( x ) d x = σ 2
Does anyone know the reason?

Answer & Explanation

Kathryn Moody

Kathryn Moody

Expert

2022-07-01Added 10 answers

By definition, if X has density p ( x ) then E X 2 = x 2 p ( x ) d x. So here we have
x 2 p ( x ) d x = E X 2 = V a r ( X ) + ( E X ) 2 = V a r ( X ) = σ 2 .
If you really need to use integration method, here is one.
x 2 p ( x ) d x = 1 2 π σ x 2 e x 2 2 σ 2 d x = 1 2 π σ σ 2 x d ( e x 2 2 σ 2 ) = 1 2 π σ ( σ 2 x e x 2 2 σ 2 + σ 2 e x 2 2 σ 2 d x ) = σ 2 1 2 π σ e x 2 2 σ 2 d x = σ 2 p ( x ) d x = σ 2 .
Finally, the entropy.
H = p ( x ) log p ( x ) d x = 1 2 σ 2 x 2 p ( x ) d x + log ( 2 π σ ) p ( x ) d x = 1 2 + log ( 2 π σ ) = log ( 2 π e σ ) .

Do you have a similar question?

Recalculate according to your conditions!

Ask your question.
Get your answer.

Let our experts help you. Answer in as fast as 15 minutes.

Didn't find what you were looking for?