1. Basic of Linear Algebra
1. 키워드
- Scalar(스칼라), Vector(벡터), Matrix(행렬)
- Row Vector(열 벡터)와 Column Vector(행 벡터)
- 벡터와 행렬의 연산
2. Scalar, Vector, and Matrix
Scalar: a single number \(s \in \mathbb{R}\) (lower case), e.g., \(3.8\)
Vector: an ordered list of numbers, e.g., \(\mathbf{x}=\left[\begin{array}{c}x_{1} \\ x_{2} \\ \vdots \\ x_{n}\end{array}\right] \in \mathbb{R}^{n}\) (boldface, lower case)
Example
Matrix: a two-dimensional array of numbers, e.g., \(A=\left[\begin{array}{ll}1 & 6 \\ 3 & 4 \\ 5 & 2\end{array}\right] \in \mathbb{R}^{3 \times 2}\) (capital letter)
- Matrix size: \(3 \times 2\) means 3 rows and 2 columns
- Row vector: a horizontal vector
- Column vector: a vertical vector
3. Column Vector and Row Vector
A vector of \(n\)-dimension is usually a column vector, i.e., a matrix of the size \(n \times 1\) \(\mathbf{x}=\left[\begin{array}{c}x_{1} \\ x_{2} \\ \vdots \\ x_{n}\end{array}\right] \in \mathbb{R}^{n}=\mathbb{R}^{n \times 1}\)
Thus, a row vector is usually written as its transpose, i.e., \(\mathbf{x}^{T}=\left[\begin{array}{c}x_{1} \\x_{2} \\\vdots \\x_{n}\end{array}\right]^{T}=\left[\begin{array}{llll}x_{1} & x_{2} & \cdots & x_{n}\end{array}\right] \in \mathbb{R}^{1 \times n}\)
4. Matrix Notations
\(A \in \mathbb{R}^{n \times n}\): Square matrix (#rows = #columns)
Example
\(A \in \mathbb{R}^{m \times n}\): Rectangular matrix (#rows ≠ #columns)
Example
\(A^{T}\): Transpose of matrix (mirroring across the main diagonal)
Example
\(A_{i j}\): \((i, j)\)-th component of \(A\)
Example
\(A_{i,:}\): \(i\)-th row vector of \(A\)
Example
\(A_{:, i}\): \(i\)-th column vector of \(A\)
Example
5. Vector/Matrix Additions and Multiplications
\(C=A+B\): Element-wise addition, i.e., \(C_{i j}=A_{i j}+B_{i j}\)
- \(A, B, C\) should have the same size, i.e., \(A, B, C \in \mathbb{R}^{m \times n}\)
\(c\mathbf{a}\), \(c A\): Scalar multiple of vector/matrix
Example
\(C=A B\): Matrix-matrix multiplication, i.e., \(C_{i j}=\sum_{k} A_{i, k} B_{k, j}\)
Example
6. Matrix multiplication is NOT commutative
\(A B \neq B A\): Matrix multiplication is NOT commutative, e.g., Given \(A \in \mathbb{R}^{2 \times {3}}\) and \(B \in \mathbb{R}^{{3} \times 5}, A B\) is defined, but \(B A\) is not even defined.
What if \(B A\) is defined, e.g., \(A \in \mathbb{R}^{{2} \times {3}}\) and \(B \in \mathbb{R}^{{3} \times {2}}\)? Still, the sizes of \(A B \in \mathbb{R}^{{2} \times {2}}\) and \(B A \in \mathbb{R}^{{3} \times {3}}\) does not match, so \(A B \neq B A\).
What if the sizes of \(A B\) match, e.g., \(A \in \mathbb{R}^{2 \times 2}\) and \(B \in \mathbb{R}^{2 \times 2}\)? Still, in this case, generally, \(A B \neq B A\).
Example
7. Other Properties
\(A(B+C)=A B+A C\): Distributive
\(A(B C)=(A B) C\): Associative
\((A B)^{T}=B^{T} A^{T}\): Property of transpose