2. Linear Equation and Linear System
1. 키워드
- Linear Equation(선형방정식): 최고 차수의 항의 차수가 1을 넘지 않는 다항 방정식
- Linear System(선형시스템): 동일한 변수를 포함하는 하나 이상의 선형방정식의 모음
- Identity Matrix(단위 행렬): 대각선 항목이 모두 1이고 다른 모든 항목이 0인 정방 행렬
- Inverse Matrix(역행렬): 어떤 행렬과 곱했을 때 곱셈에 대한 단위 행렬이 나오게 하는 행렬
2. Linear Equation
A linear equation in the variables \(x_{1}, \cdots, x_{n}\) is an equation that can be written in the form \(a_{1} x_{1}+a_{2} x_{2}+\cdots+a_{n} x_{n}=b\), where \(b\) and the coefficients \(a_{1}, \cdots, a_{n}\) are real or complex numbers that are usually known in advance.
The above equation can be written as \(\mathbf{a}^{T} \mathbf{x}=b\) where \(\mathbf{a}=\left[\begin{array}{c}a_{1} \\a_{2} \\\vdots \\a_{n}\end{array}\right]\) and \(\mathbf{X}=\left[\begin{array}{c}x_{1} \\x_{2} \\\vdots \\x_{n}\end{array}\right]\).
3. Linear System: Set of Equations
A System of linear equations (or a linear system) is a collection of one or more linear equations involving the same variables - say, \(x_{1}, \cdots, x_{n}\).
4. Linear System Example
Suppose we collected persons' weight, height, and life-span (e.g., how long s/he lived)
| Person ID | Weight | Height | Is_smoking | Life-span |
|---|---|---|---|---|
| 1 | 60kg | 5.5ft | Yes (=1) | 66 |
| 2 | 65kg | 5.0ft | No (=0) | 74 |
| 3 | 55kg | 6.0ft | Yes (=1) | 78 |
We want to set up the following linear system:
Once we solve for \(x_{1}\), \(x_{2}\), and \(x_{3}\), given a new person with his/her weight, height, and is_smoking, we can predict his/her life-span.
The essential information of a linear system can be written compactly using a matrix.
In the following set of equations:
Let’s collect all the coefficients on the left and form a matrix:
Also, let’s form two vectors:
5. From Multiple Equations to Single Matrix Equation
Multiple equations can be converted into a single matrix equations:
How can we solve for \(\mathbf{x}\)?
6. Identity Matrix
Definition: An identity matrix is a square matrix whose diagonal entries are all 1's, and all the other entries are zeros. Often, we denote it as \(I_{n} \in \mathbb{R}^{n \times n}\).
Example
An identity matrix \(I_{n}\) preserves any vector \(\mathbf{x} \in \mathbb{R}^{n}\) after multiplying \(\mathbf{x}\) by \(I_{n}\):
7. Inverse Matrix
Definition: For a square matrix \(A \in \mathbb{R}^{{n \times n}}\), its inverse matrix \(A^{-1}\) is defined such that \(A^{-1} A=A A^{-1}=I_{n}\) .
For a \(2 \times 2\) matrix \(A=\left[\begin{array}{ll}a & b \\ c & d\end{array}\right]\), its inverse matrix \(A^{-1}\) is defined as \(A^{-1}=\frac{1}{a d-b c}\left[\begin{array}{cc}d & -b \\-c & a\end{array}\right]\).
8. Solving Linear System via Inverse Matrix
We can now solve \(A \mathbf{x}=\mathbf{b}\) as follows:
Example
One can verify \(A^{-1} A=A A^{-1}=I_{n}\) .
\(\mathbf{x}=A^{-1} \mathbf{b}=\left[\begin{array}{ccc}0.0870 & 0.0087 & -0.0870 \\-1.1304 & 0.0870 & 1.1314 \\2.0000 & -1.0000 & -1.0000\end{array}\right]\left[\begin{array}{l}66 \\74 \\78\end{array}\right]=\left[\begin{array}{c}-0.4 \\20 \\-20\end{array}\right]\)
Now, the life-span can be written as \((\text { life-span })=-0.4 \times(\text { weight })+20 \times(\text { height })-20 \times(\text { is\_smoking })\).
9. Non-Invertible Matrix \(A\) for \(A \mathbf{x}=\mathbf{b}\)
Note that if \(A\) is invertible, the solution is uniquely obtained as \(\mathbf{x}=A^{-1} \mathbf{b}\).
What if \(A\) is non-invertible, i.e., the inverse does not exist?
E.g., For \(A=\left[\begin{array}{ll}1 & 2 \\ 2 & 4\end{array}\right]\), in \(A^{-1}=\frac{1}{a d-b c}\left[\begin{array}{cc}d & -b \\ -c & a\end{array}\right]\), the denominator \(a d-b c =0\), so \(A\) is not invertible.
For \(A=\left[\begin{array}{ll}a & b \\ c & d\end{array}\right]\), \(a d-b c\) is called the determinant of \(A\), or \(\operatorname{det}A\).
10. Does a Matrix Have an Inverse Matrix?
\(\operatorname{det}A\) determines whether \(A\) is invertible (when \(\operatorname{det}A \neq 0\)) or not (when \(\operatorname{det}A = 0\)).
11. Non-Invertible Matrix \(A\) for \(A \mathbf{x}=\mathbf{b}\)
Back to the linear system, if \(A\) is non-invertible, \(A \mathbf{x}=\mathbf{b}\) will have either no solution or infinitely many solutions.
12. Rectangular Matrix \(A\) in \(A \mathbf{x}=\mathbf{b}\)
What if \(A\) is a rectangular matrix, e.g., \(A \in \mathbb{R}^{m \times n}\), where \(m \neq n\)?
\(\left[\begin{array}{lll}60 & 5.5 & 1 \\65 & 5.0 & 0 \\55 & 6.0 & 1\end{array}\right]\left[\begin{array}{l}x_{1} \\x_{2} \\x_{3}\end{array}\right]=\left[\begin{array}{l}66 \\74 \\78\end{array}\right]\)
Recall \(m=\) #equations and \(n=\) #variables.
\(m<n\): more variables than equations
- Usually infinitely many solutions exist (under-determined system).
\(m>n\): more equations than variables
- Usually no solution exists (over-determined system).