I am wondering how to derive the entropy of a geometric random variable? Or where I can find some proof/derivation? I tried to search online, but seems not much resources is available. Here is the probability density function of geometric distribution: (1-p)^{k-1}p. Here is the entropy of a geometric distribution: (-(1-p)log_2(1-p)-p log_2p)/p. Where p is the probability for the event to occur during each single experiment.

Aydin Jarvis

Aydin Jarvis

Answered question

2022-10-13

Entropy of geometric random variable?
I am wondering how to derive the entropy of a geometric random variable? Or where I can find some proof/derivation? I tried to search online, but seems not much resources is available.
Here is the probability density function of geometric distribution: ( 1 p ) k 1 p
Here is the entropy of a geometric distribution: ( 1 p ) log 2 ( 1 p ) p log 2 p p
Where p is the probability for the event to occur during each single experiment.

Answer & Explanation

Dobricap

Dobricap

Beginner2022-10-14Added 14 answers

Step 1
Assume P ( X = k ) = ( 1 p ) k 1 p, where k Z + , then the entropy is
Step 2
E n t r o p y ( X ) = k = 1 + ( 1 p ) k 1 p log 2 ( ( 1 p ) k 1 p ) = p log 2 ( p ) k = 1 + ( 1 p ) k 1 p log 2 ( 1 p ) k = 1 + ( k 1 ) ( 1 p ) k 1 = log 2 ( p ) ( 1 p ) log 2 ( 1 p ) p
Mattie Monroe

Mattie Monroe

Beginner2022-10-15Added 3 answers

Step 1
It may be faster to derive the above entropy using the memoryless property of the Geometric distribution. Let Y be an auxiliary binary random variable such that Y = 0 indicates X > 1. Then H ( X ) = H B ( p ) / p follows from the following:
H ( X , Y ) = H ( X | Y ) + H ( Y ) = H ( Y | X ) + H ( X )
Step 2
Note that H ( X | Y ) = H ( X | Y = 1 ) P ( Y = 1 ) + H ( X | Y = 0 ) P ( Y = 0 ) = H ( X | Y = 0 ) ( 1 p ) = ( 1 p ) H ( X ) H ( Y | X ) = 0 H ( Y ) = H B ( p ) ,
where the memoryless property indicates H ( X | Y = 0 ) = H ( X ), and H B ( p ) is the binary entropy function.

Do you have a similar question?

Recalculate according to your conditions!

New Questions in High school geometry

Ask your question.
Get an expert answer.

Let our experts help you. Answer in as fast as 15 minutes.

Didn't find what you were looking for?