detineerlf

Answered

2022-07-18

what's the relationship with log(sum) and sum(log)?

hi I'm a little confused about the log(sum) function and sum(log) function. In special, what's the relationship between these two terms?

$-\mathrm{log}\sum _{i}{a}_{i}\sum _{i}{b}_{i}$

$-\sum _{i}\mathrm{log}({a}_{i}+{b}_{i})$

given a negative log-likelihood of an observation set:

$\mathbf{L}=-\sum _{i,j}\mathrm{log}({\pi}_{a}{M}_{i,j}+{\pi}_{b}{N}_{i,j})$

where C is the constant parameter. ${\pi}_{a}$+${\pi}_{b}$=1 are proportion of the two component, given the instance ${O}_{ij}$

Lemma1

$-\mathrm{log}\sum _{k=1}^{K}{f}_{k}(x)=\underset{\mathrm{\Phi}(x)\in {\mathrm{\Delta}}_{+}}{min}\sum _{k=1}^{K}\{{\mathrm{\Phi}}_{k}(x)[\mathrm{log}{\mathrm{\Phi}}_{k}(x)-log({f}_{k}(x)]\}\phantom{\rule{0ex}{0ex}}s.t.\sum {\mathrm{\Phi}}_{k}(x)=1,{\mathrm{\Phi}}_{k}(x)\in (0,1)$

proof

$RHS=\sum _{k=1}^{K}{\mathrm{\Phi}}_{k}(x)\mathrm{log}\frac{{\mathrm{\Phi}}_{k}(x)}{{f}_{k}(x)}\phantom{\rule{0ex}{0ex}}>=\sum _{k=1}^{K}{\mathrm{\Phi}}_{k}(x)\mathrm{log}\frac{\sum _{k=1}^{K}{\mathrm{\Phi}}_{k}(x)}{\sum _{k=1}^{K}{f}_{k}(x)}(log-sum\text{}inequality)\phantom{\rule{0ex}{0ex}}=-\mathrm{log}\sum _{k=1}^{K}{f}_{k}(x)(\sum {\mathrm{\Phi}}_{k}(x)=1)$

Let:

$\mathit{C}=\sum _{i,j}{\mathrm{\Phi}}_{a}^{i,j}(\mathrm{log}{\mathrm{\Phi}}_{a}^{i,j}-\mathrm{log}({\pi}_{a}{M}_{i,j}))+{\mathrm{\Phi}}_{b}^{i,j}(\mathrm{log}{\mathrm{\Phi}}_{B}^{i,j}-\mathrm{log}({\pi}_{b}{N}_{i,j}))$

given the constraint, that for each $(i,j)$, ${\mathrm{\Phi}}_{a}^{i,j}+{\mathrm{\Phi}}_{b}^{i,j}=1$

Then how to prove:

Minimize $C$ equals minimize $L$?

we have

$minC=-\mathrm{log}\sum ({\pi}_{a}{M}_{i,j})-\mathrm{log}\sum ({\pi}_{b}{N}_{i,j})$

then the next step is how to prove the relationship between $minC$ and $L$?

hi I'm a little confused about the log(sum) function and sum(log) function. In special, what's the relationship between these two terms?

$-\mathrm{log}\sum _{i}{a}_{i}\sum _{i}{b}_{i}$

$-\sum _{i}\mathrm{log}({a}_{i}+{b}_{i})$

given a negative log-likelihood of an observation set:

$\mathbf{L}=-\sum _{i,j}\mathrm{log}({\pi}_{a}{M}_{i,j}+{\pi}_{b}{N}_{i,j})$

where C is the constant parameter. ${\pi}_{a}$+${\pi}_{b}$=1 are proportion of the two component, given the instance ${O}_{ij}$

Lemma1

$-\mathrm{log}\sum _{k=1}^{K}{f}_{k}(x)=\underset{\mathrm{\Phi}(x)\in {\mathrm{\Delta}}_{+}}{min}\sum _{k=1}^{K}\{{\mathrm{\Phi}}_{k}(x)[\mathrm{log}{\mathrm{\Phi}}_{k}(x)-log({f}_{k}(x)]\}\phantom{\rule{0ex}{0ex}}s.t.\sum {\mathrm{\Phi}}_{k}(x)=1,{\mathrm{\Phi}}_{k}(x)\in (0,1)$

proof

$RHS=\sum _{k=1}^{K}{\mathrm{\Phi}}_{k}(x)\mathrm{log}\frac{{\mathrm{\Phi}}_{k}(x)}{{f}_{k}(x)}\phantom{\rule{0ex}{0ex}}>=\sum _{k=1}^{K}{\mathrm{\Phi}}_{k}(x)\mathrm{log}\frac{\sum _{k=1}^{K}{\mathrm{\Phi}}_{k}(x)}{\sum _{k=1}^{K}{f}_{k}(x)}(log-sum\text{}inequality)\phantom{\rule{0ex}{0ex}}=-\mathrm{log}\sum _{k=1}^{K}{f}_{k}(x)(\sum {\mathrm{\Phi}}_{k}(x)=1)$

Let:

$\mathit{C}=\sum _{i,j}{\mathrm{\Phi}}_{a}^{i,j}(\mathrm{log}{\mathrm{\Phi}}_{a}^{i,j}-\mathrm{log}({\pi}_{a}{M}_{i,j}))+{\mathrm{\Phi}}_{b}^{i,j}(\mathrm{log}{\mathrm{\Phi}}_{B}^{i,j}-\mathrm{log}({\pi}_{b}{N}_{i,j}))$

given the constraint, that for each $(i,j)$, ${\mathrm{\Phi}}_{a}^{i,j}+{\mathrm{\Phi}}_{b}^{i,j}=1$

Then how to prove:

Minimize $C$ equals minimize $L$?

we have

$minC=-\mathrm{log}\sum ({\pi}_{a}{M}_{i,j})-\mathrm{log}\sum ({\pi}_{b}{N}_{i,j})$

then the next step is how to prove the relationship between $minC$ and $L$?

Answer & Explanation

eri1ti0m

Expert

2022-07-19Added 11 answers

sorry, I forgot one constraint, that is, for each i,j, we have ${\mathrm{\Phi}}_{a}^{i,j}+{\mathrm{\Phi}}_{b}^{i,j}=1$. So this should be straightforward, i.e.,

for each coordinate $(i,j)$, ${\mathrm{\Phi}}_{a}^{i,j}+{\mathrm{\Phi}}_{b}^{i,j}=1$, then,

${C}_{i,j}={\mathrm{\Phi}}_{a}^{i,j}(\mathrm{log}{\mathrm{\Phi}}_{a}^{i,j}-\mathrm{log}({\pi}_{a}{M}_{i,j}))+{\mathrm{\Phi}}_{b}^{i,j}(\mathrm{log}{\mathrm{\Phi}}_{b}^{i,j}-\mathrm{log}({\pi}_{b}{N}_{i,j}))$

e.g., ${\pi}_{a}={\pi}^{g},{\pi}_{b}={\pi}^{u},{M}_{i,j}=Norma{l}_{i,j}({O}_{i,j}|\theta ),{N}_{i,j}=\frac{1}{256}$

Apply $\mathbf{L}\mathbf{e}\mathbf{m}\mathbf{m}\mathbf{a}\mathbf{\text{}}\mathbf{1}$

$min{C}_{i,j}=-\mathrm{log}({\pi}_{a}{M}_{i,j}+{\pi}_{b}{N}_{i,j})$

integrating out RHS of $C$,

$minC=\sum _{i,j}min{C}_{i,j}=-\sum _{i,j}\mathrm{log}({\pi}_{a}{M}_{i,j}+{\pi}_{b}{N}_{i,j})=L$

for each coordinate $(i,j)$, ${\mathrm{\Phi}}_{a}^{i,j}+{\mathrm{\Phi}}_{b}^{i,j}=1$, then,

${C}_{i,j}={\mathrm{\Phi}}_{a}^{i,j}(\mathrm{log}{\mathrm{\Phi}}_{a}^{i,j}-\mathrm{log}({\pi}_{a}{M}_{i,j}))+{\mathrm{\Phi}}_{b}^{i,j}(\mathrm{log}{\mathrm{\Phi}}_{b}^{i,j}-\mathrm{log}({\pi}_{b}{N}_{i,j}))$

e.g., ${\pi}_{a}={\pi}^{g},{\pi}_{b}={\pi}^{u},{M}_{i,j}=Norma{l}_{i,j}({O}_{i,j}|\theta ),{N}_{i,j}=\frac{1}{256}$

Apply $\mathbf{L}\mathbf{e}\mathbf{m}\mathbf{m}\mathbf{a}\mathbf{\text{}}\mathbf{1}$

$min{C}_{i,j}=-\mathrm{log}({\pi}_{a}{M}_{i,j}+{\pi}_{b}{N}_{i,j})$

integrating out RHS of $C$,

$minC=\sum _{i,j}min{C}_{i,j}=-\sum _{i,j}\mathrm{log}({\pi}_{a}{M}_{i,j}+{\pi}_{b}{N}_{i,j})=L$

Most Popular Questions