Think of the Identity Matrix as the number 1 in the world of matrix multiplication. Multiplying any matrix by the Identity matrix (of the right size) leaves the original matrix unchanged.
The Identity Matrix, denoted as \(\mathbf{I}\) or \(\mathbf{I}_n\) (where \(n\) is the dimension), is a square matrix (\(n \times n\)) with 1s on the main diagonal (from top-left to bottom-right) and 0s everywhere else.
Structure:\((\mathbf{I}_n)_{ij} = 1\) if \(i=j\) (on the diagonal), and \((\mathbf{I}_n)_{ij} = 0\) if \(i \neq j\) (off-diagonal). Using Kronecker delta: \((\mathbf{I}_n)_{ij} = \delta_{ij}\).
For any \(m \times n\) matrix \(\mathbf{A}\):
$$ \mathbf{I}_m \mathbf{A} = \mathbf{A} $$
For any \(n \times p\) matrix \(\mathbf{B}\):
$$ \mathbf{B} \mathbf{I}_p = \mathbf{B} $$
Requirement: The Identity matrix must have compatible dimensions for the matrix multiplication to be defined. \(\mathbf{I}_n\) multiplies \(n \times p\) matrices on the left, \(\mathbf{I}_n\) multiplies \(m \times n\) matrices on the right.
As a transformation, the Identity matrix represents "doing nothing". Applying \(\mathbf{I}\) to a vector\(\mathbf{x}\) results in the same vector \(\mathbf{x}\): \(\mathbf{Ix = x}\).