suppose that the combined area of two squares

Let x represents the side of the smaller square.
Given that each side of the larger square is 5 times as long as a side of the smaller square.
That is the side length of larger square is 5x.
Now the area of the smaller square is ${\left(x\right)}^2 [\sin ce area=side\times side=x\times x={\left(x\right)}^2$
Area of the larger square is ${\left(5x\right)}^2 [\sin ce area=side\times side=5x\times 5x={\left(5x\right)}^2$

The combined area of two squares is 104 square inches.
Then the area ${\left(x\right)}^2+{\left(5x\right)}^2=104$

$x^2+25x^2=104 [\sin ce {\left(5x\right)}^2=5^2{x}^2=25x^2$

To simplify the equation we will add -104 on both sides of the equation.

$26x^2-104=104-104$
That is $26x^2-104=0$

Here take 26 common as the two terms are multiples of 26. so it can be written outside the parenthesis in the following form

$26(x^2-4)=0$
That is $x^2-4=0$

Now solve the equation $x^2-4=0$ can be factored using difference of squares pattern. So the equation can be written in the following form.

$x^2-2^2=0$

$(x-2)(x+2)=0 [\sin ce a^2-b^2=(a-b\left)\right(a+b\left)\right]$

By zero product rule we have (x-2)=0 or (x+2)=0 [since ab=0 if and only if a=0 or b=0]. Now take the equation x-2=0. Add 2 on both sides of the equation x-2+2=0+2

On simplification $x-\cancel{2}+\cancel{2}=0+2$  x=2. Now take the equation x+2=0. Add -2 on both sides of the equation x+2-2=0-2.

Then $x+\cancel{2}-\cancel{2}=0-2$  x=-2. Now the solutions are x=2 or x=-2. But the side of a square is always positive. So the side length of the smaller square is x=2 inches The length of the larger square is 5x=10 inches [since substitute x=2 in 5x=5*2=10]

Therefore of the smaller square is 2 inches by 2 inches and the size of the larger square is 10 inches by 10 inches.
Hence the answer is

smaller square is 2 by 2 square inches, lerger square is 10 by 10 square inches.