a) \(\Rightarrow\) Suppose that AB is symmetric.

This means that \((AB)^{T} = AB\)

Since \((AB)^{T} = B^{T} A^{T} = BA\)

(because A and B are symmetric), we get \(AB = BA\)

as required

\(\Leftarrow\)

Suppose that \(Ab = BA\)

To prove that AB is symmetric, we will prove that \(AB = (AB)^{T}.\)

Since \((AB)^{T} = B^{T} A^{T} = BA = AB\)

where we used that A and B are symmetric in the second equality and the assumption \(Ba = Ab\)

in the last equation and the assumption \(BA = BA\) in the last equality,

we get that \((AB)T = AB\)

as required

b) Let

\(A=\begin{bmatrix}1 & 1 \\1 & 0 \end{bmatrix}\ and\ B=\begin{bmatrix}0 & 0 \\0 & 1 \end{bmatrix}\)

Then \(AB=\begin{bmatrix}1 & 1 \\1 & 0 \end{bmatrix}\begin{bmatrix}0 & 0 \\0 & 1 \end{bmatrix}=\begin{bmatrix}0 & 1 \\0 & 0 \end{bmatrix}\)

and \(AB=\begin{bmatrix}0 & 0 \\0 & 1 \end{bmatrix}\begin{bmatrix}1 & 1 \\1 & 0 \end{bmatrix}=\begin{bmatrix}0 & 0 \\1 & 0 \end{bmatrix}\)

so \(AB \neq BA\)

This means that \((AB)^{T} = AB\)

Since \((AB)^{T} = B^{T} A^{T} = BA\)

(because A and B are symmetric), we get \(AB = BA\)

as required

\(\Leftarrow\)

Suppose that \(Ab = BA\)

To prove that AB is symmetric, we will prove that \(AB = (AB)^{T}.\)

Since \((AB)^{T} = B^{T} A^{T} = BA = AB\)

where we used that A and B are symmetric in the second equality and the assumption \(Ba = Ab\)

in the last equation and the assumption \(BA = BA\) in the last equality,

we get that \((AB)T = AB\)

as required

b) Let

\(A=\begin{bmatrix}1 & 1 \\1 & 0 \end{bmatrix}\ and\ B=\begin{bmatrix}0 & 0 \\0 & 1 \end{bmatrix}\)

Then \(AB=\begin{bmatrix}1 & 1 \\1 & 0 \end{bmatrix}\begin{bmatrix}0 & 0 \\0 & 1 \end{bmatrix}=\begin{bmatrix}0 & 1 \\0 & 0 \end{bmatrix}\)

and \(AB=\begin{bmatrix}0 & 0 \\0 & 1 \end{bmatrix}\begin{bmatrix}1 & 1 \\1 & 0 \end{bmatrix}=\begin{bmatrix}0 & 0 \\1 & 0 \end{bmatrix}\)

so \(AB \neq BA\)