Caleb Proctor

2022-07-10

I have an overdetemined system of linear equation and want to minimize overall error. Up to now, not a problem, I could use least squares. The problem is that I know that some equations in my system are more uncertain, while others are exact. Actually, I have a number of equations with different confidence levels ("low confidence","medium confidence", "high confidence" and so on). In a AX=B system, the solution should take this into account and keep unchanged the B coefficients of the "high confidence" equations, while the B coefficients of "low confidence" equations could be changed more drastically than the B coefficients of "mid confidence" equations.

Elianna Wilkinson

Expert

an efficient way to give more or less weight to equations in an overdetermined system is to rescale them; that is, multiply both $A$ and $b$ by a diagonal matrix $W$ on the left. Here is an example:
$\left\{\begin{array}{ll}x+y& =10\\ x+2y& =14\\ x+3y& =40\end{array}$
Solve this system using least squares,
$A=\left(\begin{array}{cc}1& 1\\ 1& 2\\ 1& 3\end{array}\right),\phantom{\rule{1em}{0ex}}b=\left(\begin{array}{c}10\\ 14\\ 30\end{array}\right)$
and got $x=-2$, $y=10$. The right hand side for this solution is $\left(8,18,28{\right)}^{T}$.
But suppose the first equation is very important / certain, while the last one is the least important. If I let $W$ be the diagonal matrix with entries $\left(4,1,1/2\right)$, the result is $x=2.84$, $y=7.07$. The right hand side for this solution is $\left(9.91,16.98,24.05{\right)}^{T}$. So, the first equation is satisfied almost exactly, while the last equation is pretty far from target.

Do you have a similar question?