Benefits of factoring matrix multiplication into two matrix multiplications?Assuming we have a linear transformation for...
Benefits of factoring matrix multiplication into two matrix multiplications?
Assuming we have a linear transformation for vectors to vectors denoted y=Wx and that the goal is to learn the values of W (or at least the values that produce minimum loss) from data.
My question is as follows:
Is there any benefit from splitting the linear transformation to two matrices and learning the factors instead of learning the original one, i.e., if W=AB, then learn A and B instead of learning W? Are there any mathematical properties of such factoring that make it useful?
Unfortunately, I am unable to find a mathematical term for this matrix factorization (if this is a proper term to use), so it would be great if someone can tell me what should I be reading about.
Also, the only benefit I could find is that if and q<n, the number of parameters needed would be (n+m)q instead of nm.