I am a web developer who is trying to understand Machine learning.
Solving a set of linear equations is a fundamental problem in maths. I understand that there exist efficient matrix based algorithms to compute the solution.
Now, to solve a set of non-linear equations is tough it seems and there aren't any algorithms to solve them.
My question is why is non-linearity such a big hazard in mathematics? Is it because obtaining a closed form solution of non-linear equation is not possible? (I am also vague about what closed form means, I think closed form is anything for which we can exactly write a formula.)
In particular, how is non-linearity connected to optimization problems and why can't we just take the derivative of the equation and solve it; in all cases. I think the answer to this lies in my previous question, i.e. we can't actually solve the non-linear equations we get by taking the derivative and setting it to zero.