Does a non-invertible matrix transformation "really" not have an inverse?

There are no inverse linear transformations. And in linear algebra we rarely, if ever, bother with functions that aren't linear transformations. So we say it's not invertible for that reason.
However, your function isn't injective. So regardless of whether we limit ourself to linear transformations or to general functions, there is no inverse to your function. Most points on your line segment is the image of an entire line segment from your square, and you can't reverse that mapping with our modern conventional understanding of what a function is.

There is no logical implication from
There exists a map from A to B that has an inverse.
to
Every map from A to B has an inverse.