 # Let <mo fence="false" stretchy="false">{ X 1 </msub> , &#x2026;<!-- … --> ttyme411gl 2022-07-09 Answered
Let $\left\{{X}_{1},\dots ,{X}_{K}\right\}$ is a set of random matrices, where ${X}_{k}\in {\mathbb{R}}^{M×N},k=1,\dots ,K$, and $U\in {\mathbb{R}}^{M×r}$ and $V\in {\mathbb{R}}^{N×r}$ are two matrices containing orthogonal columns (i.e., ${U}^{\mathrm{\top }}U=I,{V}^{\mathrm{\top }}V=I$). I was wondering, if the following question has a analytical solution:
$\underset{U,V}{max}\sum _{k=1}^{K}‖{U}^{\mathrm{\top }}{X}_{k}V{‖}_{F}^{2}$
If not, how should I solve it? Alternating optimization?

(At first, I thought it may be related to the SVD of the sum of the matrices $\left\{{X}_{k}\right\}$, but so far I have no hint to prove it.)
You can still ask an expert for help

• Live experts 24/7
• Questions are typically answered in as fast as 30 minutes
• Personalized clear answers

Solve your problem for the price of one coffee

• Math expert for every subject
• Pay only if we can solve it Nicolas Calhoun
$\begin{array}{ll}\text{maximize}& \sum _{k=1}^{K}‖{\mathrm{U}}^{\mathrm{\top }}{\mathrm{X}}_{k}\mathrm{V}{‖}_{\text{F}}^{2}\\ \text{subject to}& {\mathrm{U}}^{\mathrm{\top }}\mathrm{U}=\mathrm{I}\\ & {\mathrm{V}}^{\mathrm{\top }}\mathrm{V}=\mathrm{I}\end{array}$
Let
${f}_{k}\left(\mathrm{U},\mathrm{V}\right):=‖{\mathrm{U}}^{\mathrm{\top }}{\mathrm{X}}_{k}\mathrm{V}{‖}_{\text{F}}^{2}=\text{tr}\left({\mathrm{U}}^{\mathrm{\top }}{\mathrm{X}}_{k}\mathrm{V}{\mathrm{V}}^{\mathrm{\top }}{\mathrm{X}}_{k}^{\mathrm{\top }}\mathrm{U}\right)=\text{tr}\left({\mathrm{V}}^{\mathrm{\top }}{\mathrm{X}}_{k}^{\mathrm{\top }}\mathrm{U}{\mathrm{U}}^{\mathrm{\top }}{\mathrm{X}}_{k}\mathrm{V}\right)$
Hence,
${\mathrm{\partial }}_{\mathrm{U}}\phantom{\rule{thinmathspace}{0ex}}{f}_{k}\left(\mathrm{U},\mathrm{V}\right)=2\phantom{\rule{thinmathspace}{0ex}}{\mathrm{X}}_{k}\mathrm{V}{\mathrm{V}}^{\mathrm{\top }}{\mathrm{X}}_{k}^{\mathrm{\top }}\mathrm{U}$
${\mathrm{\partial }}_{\mathrm{V}}\phantom{\rule{thinmathspace}{0ex}}{f}_{k}\left(\mathrm{U},\mathrm{V}\right)=2\phantom{\rule{thinmathspace}{0ex}}{\mathrm{X}}_{k}^{\mathrm{\top }}\mathrm{U}{\mathrm{U}}^{\mathrm{\top }}{\mathrm{X}}_{k}\mathrm{V}$
Let the Lagrangian be
$\mathcal{L}\left(\mathrm{U},\mathrm{V},{\mathrm{\Lambda }}_{1},{\mathrm{\Lambda }}_{2}\right):=\sum _{k=1}^{K}{f}_{k}\left(\mathrm{U},\mathrm{V}\right)-⟨{\mathrm{\Lambda }}_{1},{\mathrm{U}}^{\mathrm{\top }}\mathrm{U}-\mathrm{I}⟩-⟨{\mathrm{\Lambda }}_{2},{\mathrm{V}}^{\mathrm{\top }}\mathrm{V}-\mathrm{I}⟩$
where the Lagrange multipliers ${\mathrm{\Lambda }}_{1}$ and ${\mathrm{\Lambda }}_{2}$ are symmetric matrices. Taking the partial derivatives with respect to $\mathrm{U}$ and $\mathrm{V}$,
${\mathrm{\partial }}_{\mathrm{U}}\mathcal{L}\left(\mathrm{U},\mathrm{V},{\mathrm{\Lambda }}_{1},{\mathrm{\Lambda }}_{2}\right)=2\sum _{k=1}^{K}{\mathrm{X}}_{k}\mathrm{V}{\mathrm{V}}^{\mathrm{\top }}{\mathrm{X}}_{k}^{\mathrm{\top }}\mathrm{U}-2\mathrm{U}{\mathrm{\Lambda }}_{1}$
${\mathrm{\partial }}_{\mathrm{V}}\mathcal{L}\left(\mathrm{U},\mathrm{V},{\mathrm{\Lambda }}_{1},{\mathrm{\Lambda }}_{2}\right)=2\sum _{k=1}^{K}{\mathrm{X}}_{k}^{\mathrm{\top }}\mathrm{U}{\mathrm{U}}^{\mathrm{\top }}{\mathrm{X}}_{k}\mathrm{V}-2\mathrm{V}{\mathrm{\Lambda }}_{2}$
Finding where the partial derivatives vanish, we obtain two cubic matrix equations in $\mathrm{U},\mathrm{V},{\mathrm{\Lambda }}_{1},{\mathrm{\Lambda }}_{2}$ and two quadratic matrix equations in $\mathrm{U}$ and $\mathrm{V}$
$\overline{)\begin{array}{rl}\sum _{k=1}^{K}{\mathrm{X}}_{k}\mathrm{V}{\mathrm{V}}^{\mathrm{\top }}{\mathrm{X}}_{k}^{\mathrm{\top }}\mathrm{U}& =\mathrm{U}{\mathrm{\Lambda }}_{1}\\ \sum _{k=1}^{K}{\mathrm{X}}_{k}^{\mathrm{\top }}\mathrm{U}{\mathrm{U}}^{\mathrm{\top }}{\mathrm{X}}_{k}\mathrm{V}& =\mathrm{V}{\mathrm{\Lambda }}_{2}\\ {\mathrm{U}}^{\mathrm{\top }}\mathrm{U}& =\mathrm{I}\\ {\mathrm{V}}^{\mathrm{\top }}\mathrm{V}& =\mathrm{I}\end{array}}$
How can one solve these matrix equations? I do not know.

We have step-by-step solutions for your answer! Ximena Skinner
Define the variables
$\begin{array}{rll}Y& =U{U}^{T}& \phantom{\rule{thickmathspace}{0ex}}⟹\phantom{\rule{thickmathspace}{0ex}}y=\mathrm{v}\mathrm{e}\mathrm{c}\left(Y\right)\\ Z& =V{V}^{T}& \phantom{\rule{thickmathspace}{0ex}}⟹\phantom{\rule{thickmathspace}{0ex}}z=\mathrm{v}\mathrm{e}\mathrm{c}\left(Z\right)\\ S& =\sum _{k}{X}_{k}\otimes {X}_{k}\end{array}$
Note that $\left(Y,Z\right)$ are not orthogonal but they are ortho-projectors
$\begin{array}{rl}{Y}^{2}& =Y={Y}^{T}\\ {Z}^{2}& =Z={Z}^{T}\end{array}$
Similarly, the vectors $\left(y,z\right)$ are not unit vectors but they satisfy simple normalization conditions
$\begin{array}{rl}{y}^{T}y& =M\\ {z}^{T}z& =N\end{array}$
Write the objective function as
$\begin{array}{rl}f& =\sum _{k}\phantom{\rule{thinmathspace}{0ex}}‖{U}^{T}{X}_{k}V{‖}_{F}^{2}\\ & =\sum _{k}\left({U}^{T}{X}_{k}V\right):\left({U}^{T}{X}_{k}V\right)\\ & =\sum _{k}\left(Y{X}_{k}\right):\left({X}_{k}Z\right)\\ & =\sum _{k}\mathrm{v}\mathrm{e}\mathrm{c}\left(Y{X}_{k}{\right)}^{T}\mathrm{v}\mathrm{e}\mathrm{c}\left({X}_{k}Z\right)\\ & =\sum _{k}{y}^{T}\left({X}_{k}\otimes {I}_{M}\right)\left({I}_{N}\otimes {X}_{k}\right)z\\ & ={y}^{T}Sz\end{array}$
where colon denotes the inner/Frobenius product.

The main result of these manipulations was to recast the problem as a single matrix-vector equation. The vector gradients are then
$\begin{array}{rll}\frac{\mathrm{\partial }f}{\mathrm{\partial }y}& =Sz,\phantom{\rule{thinmathspace}{0ex}}\phantom{\rule{thinmathspace}{0ex}}\phantom{\rule{thinmathspace}{0ex}}\phantom{\rule{thinmathspace}{0ex}}\phantom{\rule{thinmathspace}{0ex}}\frac{\mathrm{\partial }f}{\mathrm{\partial }z}& ={S}^{T}y\\ \end{array}$
This still isn't a solution to the problem, just another way of thinking about it.

We have step-by-step solutions for your answer!