Ratio test:

Let the series is \(\sum a_n\) then,

\(\lim_{n\to\infty}\frac{a_{n+1}}{a_n}=L\)

If \(L<1\), series is convergent.

If \(L>1\), series is divergent.

If \(L=1\), may be convergent or divergent.

Given that,

\(\sum_{n=1}^\infty\frac{n^2}{(n+1)(n^2+2)}\)

Here

\(a_n=\frac{n^2}{(n+1)(n^2+2)}\) so

\(a_{n+1}=\frac{(n+1)^2}{(n+2)((n+1)^2+2)}\)

Now apply the ratio test,

\(\lim_{n\to\infty}\frac{a_{n+1}}{a_n}=\lim_{n\to\infty}(\frac{\frac{(n+1)^2}{(n+2)((n+1)^2+2)}}{\frac{n^2}{(n+1)(n^2+2)}})\)

\(\lim_{n\to\infty}\left(\frac{(n+1)^2}{(n+2)((n+1)^2+2)}\times\frac{(n+1)(n^2+2)}{n^2}\right)\)

\(\lim_{n\to\infty}\left(\frac{n^2(1+\frac{1}{n})^2}{n^3(1+\frac{2}{n})((1+\frac{1}{n})^2+\frac{2}{n^2})}\times\frac{n^3(1+\frac{1}{n})(1+\frac{2}{n^2})}{n^2}\right)\)

\(\lim_{n\to\infty}\left(\frac{(1+\frac{1}{n})^2}{(1+\frac{2}{n})((1+\frac{1}{n})^2+\frac{2}{n^2})}\times\frac{(1+\frac{1}{n})(1+\frac{2}{n^2})}{1}\right)\)

=1

Since \(\lim_{n\to\infty}\frac{a_{n+1}}{a_n}=L\), where \(L=1\). So ratio test fails.

Comparison test:

Let the series is \(\sum a_n\) then,

\(\lim_{n\to\infty}\frac{a_n}{b_n}=L\)

Where L is positive and finite then \(\sum a_n\) and \(\sum b_n\) are both convergent or divergent.

Given that,

\(\sum_{n=1}^\infty\frac{n^2}{(n+1)(n^2+2)}\)

Here \(a_n=\frac{n^2}{(n+1)(n^2+2)}\) so

\(b_n=\frac{n^2}{n^3}=\frac{1}{n}\) Now using the comparison test,

\(\lim_{n\to\infty}\frac{a_n}{b_n}=\lim_{n\to\infty}\left(\frac{\frac{n^2}{(n+1)(n^2+2)}}{\frac{1}{n}}\right)\)

\(=\lim_{n\to\infty}\left(\frac{n^2}{(n+1)(n^2+2)}\times n\right)\)

\(=\lim_{n\to\infty}\left(\frac{n^2}{n^3(1+\frac{1}{n})(1+\frac{2}{n^2})}\times n\right)\)

\(=\lim_{n\to\infty}\left(\frac{1}{(1+\frac{1}{n})(1+\frac{2}{n^2})}\right)\)

\(=1\)

Since \(=\lim_{n\to\infty}\frac{a_n}{b_n}=1\), which is positive and finite.

Since \(b_n=\frac{1}{n}\), so using p series test

[\(\sum_{n=1}^\infty\frac{1}{n^p}\) is convergent if p>1 or divergent if \(p\leq1\)], \(b_n=\frac{1}{n}\) is divergent.

Since \(\lim_{n\to\infty}\frac{a_n}{b_n}=1\), which is positive and finite and \(\sum b_n\) is divergent, so using comparison test \(\sum a_n\) is also divergent

Hence \(\sum_{n=1}^\infty\frac{n^2}{(n+1)(n^2+2)}\) is divergent.