SymPy demo 3: linear maps

Demo by Christian Mikkelstrup, Hans Henrik Hermansen, Karl Johan Måstrup Kristiansen and Magnus Troen. Revised 22-10-26 by shsp.

from sympy import *
init_printing()

Kernel and column space

Let

\[\begin{split} \mathbf{B} = \left[\begin{matrix}5 & 4 i & 3 + 5 i\\3 i & 2 & -3 + 4 i\\3 & 8 & 3 + 7 i\end{matrix}\right]. \end{split}\]

B = Matrix([[5, 4*I, 3+5*I], [3*I, 2, -3+4*I],[3, 8, 3+7*I]])
B

\[\begin{split}\displaystyle \left[\begin{matrix}5 & 4 i & 3 + 5 i\\3 i & 2 & -3 + 4 i\\3 & 8 & 3 + 7 i\end{matrix}\right]\end{split}\]

We determine the kernel \(\operatorname{ker}\mathbf B\) using

B.nullspace()

\[\begin{split}\displaystyle \left[ \left[\begin{matrix}-1 - i\\- \frac{i}{2}\\1\end{matrix}\right]\right]\end{split}\]

Hence \(\operatorname{ker}\mathbf B = \operatorname{span}\left(\begin{bmatrix} -1-i \\ -\frac{i}{2} \\ 1 \end{bmatrix}\right).\)

The column space and row space, \(\operatorname{colsp}\mathbf B\) and \(\operatorname{rowsp}\mathbf B\), can be computed by

B.columnspace(), B.rowspace()

\[\begin{split}\displaystyle \left( \left[ \left[\begin{matrix}5\\3 i\\3\end{matrix}\right], \ \left[\begin{matrix}4 i\\2\\8\end{matrix}\right]\right], \ \left[ \left[\begin{matrix}5 & 4 i & 3 + 5 i\end{matrix}\right], \ \left[\begin{matrix}0 & 22 & 11 i\end{matrix}\right]\right]\right)\end{split}\]

We see that we have:

\[\begin{split} \operatorname{colsp}\mathbf B = \operatorname{span}\left(\left[\begin{matrix}5\\3 i\\3\end{matrix}\right], \ \left[\begin{matrix}4 i\\2\\8\end{matrix}\right]\right) \end{split}\]

\[ \operatorname{rowsp}\mathbf B = \operatorname{span}\left(\left[\begin{matrix}5 & 4 i & 3 + 5 i\end{matrix}\right], \ \left[\begin{matrix}0 & 22 & 11 i\end{matrix}\right] \right). \]

Note: the dimensions of these spaces will be identical and equal to the rank of \(\mathbf B\):

\[\operatorname{dim}(\operatorname{colsp}\mathbf B) = \operatorname{dim}(\operatorname{rowsp}\mathbf B) = \rho (\mathbf B).\]

Linearity of maps

We want to investigate whether a map is linear. To do this, we will examine the map \(\boldsymbol f:\mathbb{R}^2 \to \mathbb{R}^4\) given by

\[\boldsymbol f(x_1,x_2)=(2\,x_1+x_2,\;3\,x_1+2\,x_2,\;x_1+x_2,\;2\,x_1+3\,x_2).\]

def f(x):
    x1 = x[0]
    x2 = x[1]
    return Matrix([2*x1+x2, 3*x1+2*x2, x1+x2, 2*x1+3*x2])

Method 1 - Checking the linearity requirements directly

A map is linear if it, for two arbitrary vectors \(\mathbf u\) and \(\mathbf v\) from its domain as well as an arbitrary constant \(k\) from the field, satisfies the following two requirements:

\[\begin{split} \begin{aligned} & \boldsymbol f(\mathbf u+\mathbf v)=\boldsymbol f(\mathbf u)+\boldsymbol f(\mathbf v)\\ & \boldsymbol f(k\,\mathbf u)=k\,\boldsymbol f(\mathbf u). \end{aligned} \end{split}\]

Find the relevant theorem in the course textbook where this is stated to confirm. To check whether these two requirements are fulfilled, we first define arbitrary vectors and a constant to work with:

u1,u2,v1,v2,k = symbols('u_1,u_2,v_1,v_2,k')
u = Matrix([u1,u2])
v = Matrix([v1,v2])

u, v, k

\[\begin{split}\displaystyle \left( \left[\begin{matrix}u_{1}\\u_{2}\end{matrix}\right], \ \left[\begin{matrix}v_{1}\\v_{2}\end{matrix}\right], \ k\right)\end{split}\]

Try inserting these arbitrary vectors into the map to ensure that everything is properly defined:

f(u), f(v)

\[\begin{split}\displaystyle \left( \left[\begin{matrix}2 u_{1} + u_{2}\\3 u_{1} + 2 u_{2}\\u_{1} + u_{2}\\2 u_{1} + 3 u_{2}\end{matrix}\right], \ \left[\begin{matrix}2 v_{1} + v_{2}\\3 v_{1} + 2 v_{2}\\v_{1} + v_{2}\\2 v_{1} + 3 v_{2}\end{matrix}\right]\right)\end{split}\]

Now we check the linearity requirements:

# Linearity requirement no. 1
f(u+v) - (f(u)+f(v))

\[\begin{split}\displaystyle \left[\begin{matrix}0\\0\\0\\0\end{matrix}\right]\end{split}\]

# Linearity requirement no. 2
simplify(f(k*u)-k*f(u))

\[\begin{split}\displaystyle \left[\begin{matrix}0\\0\\0\\0\end{matrix}\right]\end{split}\]

We see clearly that \(\boldsymbol f\) satisfies the criteria, so we conclude that \(\boldsymbol f\) is linear.

Method 2 - Finding the mapping matrix

If we can write the map \(\boldsymbol f(\mathbf x)\) on matrix form like this:

\[ \boldsymbol f(\mathbf x)=\mathbf F\,\mathbf x \]

for some matrix \(\mathbf F\in \mathbb{R}^{m\times n}\), then \(\boldsymbol f\) is linear (see the relevant theorem in the course textbook). The matrix \(\mathbf F\) is what we often refer to as the mapping matrix - if a mapping matrix exists, then the map is linear.

We observe that \(\boldsymbol f\) has the following matrix representation (with respect to the ordered standard bases for \(\mathbb{R}^2\) and \(\mathbb{R}^4\)):

\[\begin{split} \begin{aligned} \boldsymbol f(\mathbf x) = & \mathbf F\,\mathbf x\\ \begin{bmatrix}2\,x_1+x_2\\ 3\,x_1+2\,x_2\\ x_1+x_2\\ 2\,x_1+3\,x_2\end{bmatrix} = & \begin{bmatrix}2&1\\3&2\\1&1\\2&3\end{bmatrix} \begin{bmatrix}x_1\\x_2\end{bmatrix}. \end{aligned} \end{split}\]

So, we have found a matrix \(\mathbf F \in \mathbb{R}^{4\times 2}\) to be

\[\begin{split} \mathbf F = \begin{bmatrix}2&1\\3&2\\1&1\\2&3\end{bmatrix}, \end{split}\]

which is a matrix representation (a mapping matrix) of the map with respect to the ordered standard bases.

As a side note, observe that the columns are the images of the basis vectors from the domain \(\mathbb R^2\), as they should be:

f([1,0]),f([0,1])

\[\begin{split}\displaystyle \left( \left[\begin{matrix}2\\3\\1\\2\end{matrix}\right], \ \left[\begin{matrix}1\\2\\1\\3\end{matrix}\right]\right)\end{split}\]

We did all the above manually, but when using the standard basis then you can automate the process with the following, where we utilise the identity matrix eye - note that if a linear map is given, then it is always possible to find a matrix representation in this way:

n = 2
m = 4
V = eye(n)
F = zeros(m,n)
for k in range(n):
    F[:,k] = f(V.col(k))
F

\[\begin{split}\displaystyle \left[\begin{matrix}2 & 1\\3 & 2\\1 & 1\\2 & 3\end{matrix}\right]\end{split}\]

Typical scenarios for using the mapping matrix

Consider the map \(\boldsymbol g:\mathbb{R}^3\to\mathbb{R}^4\), which with respect to the ordered standard bases for \(\mathbb{R}^3\) and \(\mathbb{R}^4\) is given by its mapping matrix:

G = Matrix([[1,3,1],[2,4,0],[1,1,-1],[-3,-1,5]])
G

\[\begin{split}\displaystyle \left[\begin{matrix}1 & 3 & 1\\2 & 4 & 0\\1 & 1 & -1\\-3 & -1 & 5\end{matrix}\right]\end{split}\]

Here follow five examples on how a problem can be solved using this mapping matrix.

Example 1 - Finding the image of a given vector

We wish to find the image \(\boldsymbol g(\mathbf v)\) of a given vector \(\mathbf v\):

v = Matrix([1,-2,5])

Since we have the mapping matrix, this is easily done with a simple matrix multiplication, because \(\boldsymbol g(\mathbf v)=\mathbf G\mathbf v\):

G*v

\[\begin{split}\displaystyle \left[\begin{matrix}0\\-6\\-6\\24\end{matrix}\right]\end{split}\]

Hence \(\boldsymbol g(\mathbf v) = (0,-6,-6,24)\).

Example 2 - Investigating whether a vector belongs to the kernel

Does the following vector \(\mathbf u\) belong to the kernel of \(\boldsymbol g\)?

u = Matrix([-6,3,-3])

If so, then \(\boldsymbol g(\mathbf u)\) is the zero vector (see the definition of the kernel in the course textbook). Again this is done as a simple mapping of the vector:

G*u

\[\begin{split}\displaystyle \left[\begin{matrix}0\\0\\0\\0\end{matrix}\right]\end{split}\]

Hence \(\mathbf u=(-6,3,-3)\in\operatorname{ker}\boldsymbol g\).

Example 3 - Finding the kernel of a map

As we know from the course textbook, \(\mathbf x\in \operatorname{ker}\boldsymbol g\) holds true if and only if \(\boldsymbol g(\mathbf x)=\mathbf 0\). So the kernel of our map is found by solving the equation

\[ \mathbf G\,\mathbf x=\mathbf{0}. \]

This can be done by reducing the mapping matrix:

G.rref()

\[\begin{split}\displaystyle \left( \left[\begin{matrix}1 & 0 & -2\\0 & 1 & 1\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right], \ \left( 0, \ 1\right)\right)\end{split}\]

From here we can read the kernel to be

\[ \text{ker}\,\boldsymbol g=\text{span}((2,-1,1)). \]

This agrees with the rank-nullity theorem, which states that

\[\begin{split} \begin{aligned} \text{dim}(\text{ker}\,\boldsymbol g) &= \text{dim}(\mathbb{R}^3)-\rho(\mathbf G)\\ &= 3-2 \\ &= 1. \end{aligned} \end{split}\]

Another way to show this is

G.gauss_jordan_solve(Matrix([0,0,0,0]))

\[\begin{split}\displaystyle \left( \left[\begin{matrix}2 \tau_{0}\\- \tau_{0}\\\tau_{0}\end{matrix}\right], \ \left[\begin{matrix}\tau_{0}\end{matrix}\right]\right)\end{split}\]

Check: This gives us the same vector as when using the direct command:

G.nullspace()

\[\begin{split}\displaystyle \left[ \left[\begin{matrix}2\\-1\\1\end{matrix}\right]\right]\end{split}\]

Example 4 - Determining the image space

We know from the reduced matrix above that \(\rho(\mathbf G)=\text{dim}(\boldsymbol g(\mathbb{R}^3))=2\). Therefore a basis for \(\boldsymbol g(\mathbb R^3)\) - that is, a basis for the image space, \(\operatorname{image}\boldsymbol g\) - needs to consist of two linearly independent vectors. As we know from the course textbook, these vectors can be chosen among the columns of \(\mathbf G\).

From the result of G.rref() above, which we will repeat here for the sake of the overview:

G.rref()

\[\begin{split}\displaystyle \left( \left[\begin{matrix}1 & 0 & -2\\0 & 1 & 1\\0 & 0 & 0\\0 & 0 & 0\end{matrix}\right], \ \left( 0, \ 1\right)\right)\end{split}\]

we see that the first two columns in \(\mathbf G\) are linearly independent, as that is where we find the pivots. Thus, we pluck out these first two columns from \(\mathbf G\), and these constitute a basis for the image space:

G

\[\begin{split}\displaystyle \left[\begin{matrix}1 & 3 & 1\\2 & 4 & 0\\1 & 1 & -1\\-3 & -1 & 5\end{matrix}\right]\end{split}\]

G.col(0), G.col(1)

\[\begin{split}\displaystyle \left( \left[\begin{matrix}1\\2\\1\\-3\end{matrix}\right], \ \left[\begin{matrix}3\\4\\1\\-1\end{matrix}\right]\right)\end{split}\]

\[ \operatorname{image}\boldsymbol g=\boldsymbol g(\mathbb{R}^3) = \text{span}((1,2,1,-3),(3,4,1,-1)). \]

The built-in command can also be used here:

G.columnspace()

\[\begin{split}\displaystyle \left[ \left[\begin{matrix}1\\2\\1\\-3\end{matrix}\right], \ \left[\begin{matrix}3\\4\\1\\-1\end{matrix}\right]\right]\end{split}\]

This SymPy command might return different vectors than those we plucked out from \(\mathbf G\) above, but they will nevertheless span the same space!

Example 5 - Determining whether a vector belongs to the image space

If a given vector \(\mathbf b\) belongs to the image space, meaning \(\mathbf b \in \boldsymbol g(\mathbb{R}^3)\), then there must exist a vector \(\mathbf x \in \mathbb{R}^3\) such that \(\boldsymbol g(\mathbf x)=\mathbf b\). To find out if such a vector \(\mathbf x\) exists, we need to solve:

\[ \mathbf G\,\mathbf x=\mathbf b. \]

First, we are given the vector \(\mathbf b_1\):

b1 = Matrix([1,2,1,-4])

Does it belong to the image of \(\boldsymbol g\)?

T1 = Matrix.hstack(G,b1)

T1.rref()

\[\begin{split}\displaystyle \left( \left[\begin{matrix}1 & 0 & -2 & 0\\0 & 1 & 1 & 0\\0 & 0 & 0 & 1\\0 & 0 & 0 & 0\end{matrix}\right], \ \left( 0, \ 1, \ 3\right)\right)\end{split}\]

This system clearly has no solution as the rref shows an inconsistent equation. Indeed, if we ask SymPy for a solution, we are given an empty set (or an error if you are using G.gauss_jordan_solve(b1)):

linsolve((G,b1))

So, \(\mathbf b_1\) does not belong to the image of \(\boldsymbol g\).

Next, we are given the vector \(\mathbf b_2\):

b2 = Matrix([3,0,-3,15])

Does this one belong to the image of \(\boldsymbol g\)?

T2 = Matrix.hstack(G,b2)

T2.rref()

\[\begin{split}\displaystyle \left( \left[\begin{matrix}1 & 0 & -2 & -6\\0 & 1 & 1 & 3\\0 & 0 & 0 & 0\\0 & 0 & 0 & 0\end{matrix}\right], \ \left( 0, \ 1\right)\right)\end{split}\]

Yes! The system has solutions, so \(\mathbf b_2\) belongs to the image, and we can write \(\mathbf b_2\in \boldsymbol g(\mathbb{R}^3)\). We can even see that \(\mathbf b_2\) has the coordinates \((-6,3)\) w.r.t. the ordered basis \(((1,2,1,-3),(3,4,1,-1))\) for \(\boldsymbol g(\mathbb R^3)\). We can also see the infinitely many vectors that are mapped to \(\mathbf b_2\). These vectors have the form:

\[ (-6,3,0)+t\,(2,-1,1), \quad t\in\mathbb{R}. \]

Now we can use .gauss_jordan_solve() without any error messages (or just linsolve as before):

G.gauss_jordan_solve(b2), linsolve((G,b2))

\[\begin{split}\displaystyle \left( \left( \left[\begin{matrix}2 \tau_{0} - 6\\3 - \tau_{0}\\\tau_{0}\end{matrix}\right], \ \left[\begin{matrix}\tau_{0}\end{matrix}\right]\right), \ \left\{\left( 2 \tau_{0} - 6, \ 3 - \tau_{0}, \ \tau_{0}\right)\right\}\right)\end{split}\]

Changing mapping matrix via a change of basis

In the following, two examples on how to change the matrix representation of a map (so, the mapping matrix) by changing the basis are shown.

Example 1 - Changing matrix representation from given basis to standard basis

An ordered basis \(\gamma=((5,3),(-2,-1))\) is given. W.r.t. this, the linear map \(\boldsymbol M: \mathbb{R}^2\to \mathbb{R}^2\) is defined by a matrix:

\[\begin{split} _\gamma[\boldsymbol M]_\gamma=\begin{bmatrix}1&2\\3&4\end{bmatrix}. \end{split}\]

yMy = Matrix([[1,2],[3,4]])
yMy

\[\begin{split}\displaystyle \left[\begin{matrix}1 & 2\\3 & 4\end{matrix}\right]\end{split}\]

What is the matrix representation of \(\boldsymbol M\) with respect to the ordered standard basis?

We can denote the ordered standard basis for \(\mathbb R^2\) by \(e = ((1,0),(0,1))\), and what we want to find is thus \(_e[\boldsymbol M]_e\). For that we need two change-of-basis matrices; one that changes from \(\gamma\) basis to \(e\) basis, and another that changes from \(e\) to \(\gamma\).

The change-of-basis matrix that changes from \(\gamma\) basis to \(e\) basis is denoted by \(_e[\text{id}_V]_\gamma\). We are able to write up this matrix right away since it simply consists of the two \(\gamma\) basis vectors as columns:

eIy = Matrix([[5,-2],[3,-1]])
eIy

\[\begin{split}\displaystyle \left[\begin{matrix}5 & -2\\3 & -1\end{matrix}\right]\end{split}\]

So,

\[\begin{split} _e[\text{id}_V]_\gamma=\begin{bmatrix}5&-2\\3&-1\end{bmatrix}. \end{split}\]

Conversely, the change-of-basis matrix switching from \(e\) basis to the \(\gamma\) basis is found as the inverse of the above (see the course textbook for the details):

\[ _\gamma[\text{id}_V]_e=_e[\text{id}_V]_\gamma^{-1}. \]

yIe = eIy**-1
yIe

\[\begin{split}\displaystyle \left[\begin{matrix}-1 & 2\\-3 & 5\end{matrix}\right]\end{split}\]

So,

\[\begin{split} _\gamma[\text{id}_V]_e=\begin{bmatrix}-1&2\\-3&5\end{bmatrix}. \end{split}\]

We now have all the tools prepared, and we can find \(_e[\boldsymbol M]_e\) as follows:

\[ _e[\boldsymbol M]_e = _e[\text{id}_V]_\gamma \ _\gamma [\boldsymbol M]_\gamma \ _\gamma [\text{id}_V]_e. \]

eMe = eIy * yMy * yIe
eMe

\[\begin{split}\displaystyle \left[\begin{matrix}-5 & 8\\-6 & 10\end{matrix}\right]\end{split}\]

So,

\[\begin{split} _e[M]_e = \left[\begin{matrix}-5 & 8\\-6 & 10\end{matrix}\right]. \end{split}\]

Example 2 - Changing mapping matrix from standard basis to another basis

We will here reuse the ordered basis \(\gamma\) from above. In the ordered standard basis \(e\) the linear map \(\boldsymbol K:\mathbb{R}^2\to \mathbb{R}^2\) is defined by the matrix representation

\[\begin{split} _e[\boldsymbol K]_e=\begin{bmatrix}-2&0\\1&3\end{bmatrix}. \end{split}\]

eKe = Matrix([[-2,0],[1,3]])

What is the matrix representation with respect to the \(\gamma\) basis?

What we need to find is \(_\gamma[\boldsymbol K]_\gamma\). Since we from the previous example already know the change-of-basis matrices between the \(\gamma\) and \(e\) bases, the matrix representation for \(\boldsymbol K\) with respect to basis \(\gamma\) is quickly found:

yKy = yIe*eKe*eIy
yKy

\[\begin{split}\displaystyle \left[\begin{matrix}38 & -14\\100 & -37\end{matrix}\right]\end{split}\]