Math►Linear Algebra

Transpose

Inverse of Matrix Production

$$
\forall \text{invertible} A,B , \text{what is} (AB)^{-1}?
$$

$$
B^{-1}(A^{-1}A)B = I
$$

$$
\therefore (AB)^{-1} = B^{-1}A^{-1}
$$

Don’t forget to reverse order.

Transpose of Matrix Production

$$
\forall A,B,\text{what is } (AB)^{T} ?
$$

Transpose exchange rows and columns.

$(AB)^{T}$ is a combination of $B$’s rows specified by $A$’s rows, and finally transposed

$A^T$ specify a combination of columns

$B^T$ change $B$’s rows into columns

So for column selection, the order is $B^TA^T$

$$
\therefore (AB)^T = B^TA^T
$$

Transpose and inverse

$$
\forall \text{square and invertible} A, \text{what is } (A^{-1})^T?
$$

$$
\because AA^{-1}=I
$$

$$
\therefore (AA^{-1})^T = I^T = I
$$

$$
\therefore (A^{-1})^T A^T = I
$$

$$
\therefore (A^{-1})^T = (A^T)^{-1}
$$

LD Decomposition

Extra work in Gaussian Elimination

All invertible square matrix can be decomposed into a product of P,L,U

1
2
3
"P" : "Permutaion Matrix (row exchange of identity matrix)",
"L" : "Lower Trangular Matrix",
"U" : "Upper Trangular Matrix"

We need $P$ in case of row exchange scipy.linalg.lu(MatLab also do this)

Let’s do a Gaussian Elimination

$$
\because E_{nn-1}…E_{32}E_{31}E_{21}A = U
$$

$$
\therefore A = (E_{nn-1}…E_{32}E_{31}E_{21})^{-1} U
$$

$$
\because (E_{nn-1}…E_{32}E_{31}E_{21})^{-1} = E_{21}^{-1}E_{31}^{-1}E_{32}^{-1}…E_{nn-1}^{-1}=L
$$

$$
\therefore A = LU
$$

If there is 0-pivot, we can apply a Permutation Matrix $P$ to do a row exchange.

Example

$$
A = \begin{bmatrix}
2 & 4 & 5\
1 & 3 & 2\
4 & 2 & 1
\end{bmatrix}
$$

$$
E_{21} = \begin{bmatrix}
1 & 0 & 0\
-\frac{1}{2} & 1 & 0\
0 & 0 & 1
\end{bmatrix}
$$

$$
E_{31} = \begin{bmatrix}
1 & 0 & 0\
0 & 1 & 0\
-2 & 0 & 1
\end{bmatrix}
$$

$$
E_{31}E_{21}A = \begin{bmatrix}
2 & 4 & 5\
0 & 1 & -\frac{1}{2}\
0 & -6 & -9
\end{bmatrix}
$$

$$
E_{32} = \begin{bmatrix}
1 & 0 & 0\
0 & 1 & 0\
0 & 6 & 1
\end{bmatrix}
$$

$$
E_{32}E_{31}E_{21}A = \begin{bmatrix}
2 & 4 & 5\
0 & 1 & -\frac{1}{2}\
0 & 0 & -12
\end{bmatrix} = U
$$

$$
L = E_{21}^{-1}E_{31}^{-1}E_{32}^{-1}
$$

To inverse a Gaussian Elimination Matrix $E$, we only need to undo the elimination, flipping the sign

$$
= \begin{bmatrix}
1 & 0 & 0\
\frac{1}{2} & 1 & 0\
0 & 0 & 1
\end{bmatrix}
\cdot
\begin{bmatrix}
1 & 0 & 0\
0 & 1 & 0\
2 & 0 & 1
\end{bmatrix}
\cdot
\begin{bmatrix}
1 & 0 & 0\
0 & 1 & 0\
0 & -6 & 1
\end{bmatrix}
$$

And for lower triangular matrix like this(a variant of identity matrix), we can do production by simply adding numbers in corresponding position

$$
L = \begin{bmatrix}
1 & 0 & 0\
\frac{1}{2} & 1 & 0\
2 & -6 & 1
\end{bmatrix}
$$

Verify
1
2
3
4
5
6
7
>>> L = array([ [1,0,0] , [1/2 , 1, 0] , [2,-6,1] ])
>>> U = array([ [2,4,5] , [0,1,-1/2] , [0,0,-12] ])
>>> matmul(L,U)
array([[2., 4., 5.],
       [1., 3., 2.],
       [4., 2., 1.]])
# LU == A

$$
\forall \text{invertible} A , A=PLU
$$

1
2
3
4
5
6
"If we have permutation in the process"
>>> p,l,u = scipy.linalg.lu(A)
>>> matmul( p , matmul(l,u) ) == A
array([[ True,  True,  True],
       [ True,  True,  True],
       [ True,  True,  True]])

Another good example

Permutation Matrixes

matrixes only do row exchanges

Example : permutation which exchanges row 1 and 2

$$
\begin{bmatrix}
0 & 1 & 0\
1 & 0 & 0\
0 & 0 & 1
\end{bmatrix}
$$

  • Permutation matrixes are all invertible, since the operation can be undo

  • For a $n\times n$ matrix there are all $C_n^1 C_{n-1}^1…C_1^1 = n(n-1)(n-2)…1=n!$ possible permutation matrixes.

  • And for permutation matrixes, $P^T=P^{-1}$,prove

Transpose

rows become columns and columns become rows

1
transposed(A)[i][j] == A[j][i]
1
2
3
4
5
>>> A = array([ [1,2,3] , [4,5,6] ])
>>> A.T
array([[1, 4],
       [2, 5],
       [3, 6]])

Symmetric matrix

This is easy obvious and easy to identify they

$$
A^T = A
$$

And by this we can say that any matrix can produce a symmetric matrix !

$$
\forall A,A\text{ has transpose} A^T
$$

$$
(AA^T)^T=(A^T)^TA^T=AA^T
$$

If we view $AA^T$ as an entity $B$, we can see that $B^T=B$

So $AA^T$ is definitely symmetric