I am a web developer who is trying to understand Machine learning.Solving a set of linear equations is a fundamental problem in maths. I understand that there exist efficient matrix based algorithms to compute the solution.Now, to solve a set of non-linear equations is tough it seems and there aren't any algorithms to solve them.My question is why is non-linearity such a big hazard in mathematics? Is it because obtaining a closed form solution of non-linear equation is not possible? (I am also vague about what closed form means, I think closed form is anything for which we can exactly write a formula.)In particular, how is non-linearity connected to optimization problems and why can't we just take the derivative of the equation and solve it; in all cases. I think the answer to this lies in my previous question, i.e. we can't actually solve the non-linear equations we get by taking the derivative and setting it to zero.

Question

I am a web developer who is trying to understand Machine learning.Solving a set of linear equations is a fundamental problem in maths. I understand that there exist efficient matrix based algorithms to compute the solution.Now, to solve a set of non-linear equations is tough it seems and there aren&#039;t any algorithms to solve them.My question is why is non-linearity such a big hazard in mathematics? Is it because obtaining a closed form solution of non-linear equation is not possible? (I am also vague about what closed form means, I think closed form is anything for which we can exactly write a formula.)In particular, how is non-linearity connected to optimization problems and why can&#039;t we just take the derivative of the equation and solve it; in all cases. I think the answer to this lies in my previous question, i.e. we can&#039;t actually solve the non-linear equations we get by taking the derivative and setting it to zero.

robegarj · Accepted Answer

The following assumes that all functions under consideration are differentiable, which is not always a reasonable assumption in practice.Only for linear functions will taking derivatives give a priori global information. You can think of differentiation as local linear approximation of a function. Thus if the function is already linear, this &quot;local approximation&quot; actually leads to statements which are globally true.For non-linear functions, taking derivatives in general gives only a priori local information. If one assumes additionally convexity then one can often say more. However, for general non-linear objective functions the derivative gives a priori only local information, which is one reason why non-convex optimization is so difficult.Note that this is related to the reason why the study of arbitrary manifolds is so much more difficult than the study of Euclidean spaces -- the &quot;flatness&quot; of Euclidean spaces and the global vector space structure which can be imposed on them is much easier to work with in general than the curvature and tangent bundles of arbitrary Riemannian manifolds.

I am a web developer who is trying to understand Machine learning. Solving a set of linear equatio

Answered question

Answer & Explanation

New Questions in Algebra I