detineerlf

2022-07-18

what's the relationship with log(sum) and sum(log)?
hi I'm a little confused about the log(sum) function and sum(log) function. In special, what's the relationship between these two terms?
$-\mathrm{log}\sum _{i}{a}_{i}\sum _{i}{b}_{i}$
$-\sum _{i}\mathrm{log}\left({a}_{i}+{b}_{i}\right)$
given a negative log-likelihood of an observation set:
$\mathbf{L}=-\sum _{i,j}\mathrm{log}\left({\pi }_{a}{M}_{i,j}+{\pi }_{b}{N}_{i,j}\right)$
where C is the constant parameter. ${\pi }_{a}$+${\pi }_{b}$=1 are proportion of the two component, given the instance ${O}_{ij}$
Lemma1
$-\mathrm{log}\sum _{k=1}^{K}{f}_{k}\left(x\right)=\underset{\mathrm{\Phi }\left(x\right)\in {\mathrm{\Delta }}_{+}}{min}\sum _{k=1}^{K}\left\{{\mathrm{\Phi }}_{k}\left(x\right)\left[\mathrm{log}{\mathrm{\Phi }}_{k}\left(x\right)-log\left({f}_{k}\left(x\right)\right]\right\}\phantom{\rule{0ex}{0ex}}s.t.\sum {\mathrm{\Phi }}_{k}\left(x\right)=1,{\mathrm{\Phi }}_{k}\left(x\right)\in \left(0,1\right)$
proof

Let:
$\mathit{C}=\sum _{i,j}{\mathrm{\Phi }}_{a}^{i,j}\left(\mathrm{log}{\mathrm{\Phi }}_{a}^{i,j}-\mathrm{log}\left({\pi }_{a}{M}_{i,j}\right)\right)+{\mathrm{\Phi }}_{b}^{i,j}\left(\mathrm{log}{\mathrm{\Phi }}_{B}^{i,j}-\mathrm{log}\left({\pi }_{b}{N}_{i,j}\right)\right)$
given the constraint, that for each $\left(i,j\right)$, ${\mathrm{\Phi }}_{a}^{i,j}+{\mathrm{\Phi }}_{b}^{i,j}=1$
Then how to prove:
Minimize $C$ equals minimize $L$?
we have
$minC=-\mathrm{log}\sum \left({\pi }_{a}{M}_{i,j}\right)-\mathrm{log}\sum \left({\pi }_{b}{N}_{i,j}\right)$
then the next step is how to prove the relationship between $minC$ and $L$?

eri1ti0m

Expert

sorry, I forgot one constraint, that is, for each i,j, we have ${\mathrm{\Phi }}_{a}^{i,j}+{\mathrm{\Phi }}_{b}^{i,j}=1$. So this should be straightforward, i.e.,
for each coordinate $\left(i,j\right)$, ${\mathrm{\Phi }}_{a}^{i,j}+{\mathrm{\Phi }}_{b}^{i,j}=1$, then,
${C}_{i,j}={\mathrm{\Phi }}_{a}^{i,j}\left(\mathrm{log}{\mathrm{\Phi }}_{a}^{i,j}-\mathrm{log}\left({\pi }_{a}{M}_{i,j}\right)\right)+{\mathrm{\Phi }}_{b}^{i,j}\left(\mathrm{log}{\mathrm{\Phi }}_{b}^{i,j}-\mathrm{log}\left({\pi }_{b}{N}_{i,j}\right)\right)$
e.g., ${\pi }_{a}={\pi }^{g},{\pi }_{b}={\pi }^{u},{M}_{i,j}=Norma{l}_{i,j}\left({O}_{i,j}|\theta \right),{N}_{i,j}=\frac{1}{256}$
Apply
$min{C}_{i,j}=-\mathrm{log}\left({\pi }_{a}{M}_{i,j}+{\pi }_{b}{N}_{i,j}\right)$
integrating out RHS of $C$,
$minC=\sum _{i,j}min{C}_{i,j}=-\sum _{i,j}\mathrm{log}\left({\pi }_{a}{M}_{i,j}+{\pi }_{b}{N}_{i,j}\right)=L$

Do you have a similar question?