Suppose that there is a positive definite matrix <mrow class="MJX-TeXAtom-ORD"> <mi mathvar

Rebecca Villa 2022-07-10 Answered
Suppose that there is a positive definite matrix A R n × n , and a vector b R n , then minimization of quadratic functions with linear terms can be done in closed form as
arg min x R n ( 1 2 x T A x b T x ) = A 1 b

I met this in a machine learning book. However, the book didn't provide a proof. I wonder why this can be well-formed. Hope that someone can help me with it. I find that many machine learning books like to skip all of the proofs, which made me uncomfortable.
You can still ask an expert for help

Expert Community at Your Service

  • Live experts 24/7
  • Questions are typically answered in as fast as 30 minutes
  • Personalized clear answers
Learn more

Solve your problem for the price of one coffee

  • Available 24/7
  • Math expert for every subject
  • Pay only if we can solve it
Ask Question

Answers (1)

Melina Richard
Answered 2022-07-11 Author has 14 answers
One way of proving this is to "complete the square":
1 2 x A x b x = 1 2 ( ( x A 1 b ) A ( x A 1 b ) b A 1 b )   .
.Because   A   is positive definite this is never less than   1 2 b A 1 b  , and it attains that value when   x = A 1 b  .
Not exactly what you’re looking for?
Ask My Question

Expert Community at Your Service

  • Live experts 24/7
  • Questions are typically answered in as fast as 30 minutes
  • Personalized clear answers
Learn more

You might be interested in

asked 2021-11-20
(a) In a regression analysis, the sum of squares for the predicted scores is 100 and the sum of squares error is 200, what is R2?
(b) In a different regression analysis, 40% of the variance was explained. The sum of squares total is 1000. What is the sum of squares of the predicted values?
asked 2021-09-05

Exercise No. 10 (D.E. with coefficient linear in two variables)
Find the general / solution of the following D.E.
4. (2x+3y5)dx+(3xy2)dy=0

asked 2022-01-25
Write a multivariable function (i.e z=f(x,y)) for a linear function that contains the points (−1,0),(0,2),(1,−1). Ive
asked 2021-02-25
Let F(x,y)=4cos(y),2sin(y). Compute the flux Fnds of F across the boundary of the rectangle 0x5,0yπ2 using the vector form of Greens
asked 2022-05-03
Solve for x in the equation 5x+4=4-7n.
asked 2022-06-23
I have a very simple linear problem:
min x   x 2 s.t.    a 1 x 1 + a 2 x 2 = b
Suppose I want to write this problem equivalently as in Find the equivalent linear program. Unlike the problem in the link, I have equality. Can I write it equivalently as:
min x , α , β   x 2 s.t.    a 1 x 1 = α b ,   a 2 x 2 = β b ,   α + β = 1.
The converse is intuitive: Given { x , α , β } feasible for the second problem, adding the first and second constraints gives the constraint of the first problem. But the forward part is not clear, especially because I have never seen an equality constraint written like this. Any help would be highly appreciated.
asked 2021-09-16
Requires Uploaded Supporting Analysis Find dydx by implicit differentiation. y22x=4y