Matrix Inverse in Terms of Geometry:
If a matrix works on a set of vectors by rotating and scaling the vectors, then the matrix's inverse will undo the rotations and scalings and return the original vectors.
If the first linear transformation is not unique, there are several ways to do the transformation and you cannot determine that path you need to take to reverse the transformation. In terms of geometry that means that the vectors you're scaling/rotating are in some sense so alike that you can reproduce a specific result by combining the vectors in more than one way. I believe in terms of statistics we'd refer to that as multicollinearity. If the transformation is not unique then you have a singular matrix, and you need to apply specific rules governing how you interpret the transformation in order to generate the inverse.