Linear algebra part 3 - Scalar multiplication and linear combinations

Scalar is the mathematical term for something that only has a single dimension, i.e. a number. In this lesson we will learn how to multiply one-dimensional scalars with multi-dimensional vectors.

When we multiple a vectors by a scalar, we multiply each element of the vector by the scalar. The result is a vector with the same number of dimensions as the original vector. Multiplying the scalar cc with vector v\mathbf{v} looks like this.

c v=c[v1v2v3⋮vn]=[c v1c v2c v3⋮c vn] c \, \mathbf{v} = c \begin{bmatrix} v_1 \\ v_2 \\ v_3 \\ \vdots \\ v_n \end{bmatrix} = \begin{bmatrix} c \, v_1 \\ c \, v_2 \\ c \, v_3 \\ \vdots \\ c \, v_n \end{bmatrix}

To make that a bit more tangible, let us put some real numbers to it.

c=2,v=[170]c = 2, \quad \mathbf{v} = \begin{bmatrix} 1 \\ 7 \\ 0 \end{bmatrix}
c v=2[170]=[2⋅12⋅72⋅0]=[2140] c \, \mathbf{v} = 2 \begin{bmatrix} 1 \\ 7 \\ 0 \end{bmatrix} = \begin{bmatrix} 2 \cdot 1 \\ 2 \cdot 7 \\ 2 \cdot 0 \end{bmatrix} = \begin{bmatrix} 2 \\ 14 \\ 0 \end{bmatrix}

If you have not come across the dot notation for multiplication before, it is no different to the normal times symbol. In the equation above, treat 2⋅12 \cdot 1 exactly like 2×12 \times 1.

Linear combinations of vectors

We learnt how to add vectors together in the first linear algebra post. We can combine our knowlege of adding and subtracting vectors which scalar multiplication to create linear combinations of vectors. If we have vectors v\mathbf{v} and w\mathbf{w}, and scalars cc and dd, the general form of the linear combination looks like this:

c v+d w=c[v1v2v3⋮vn]+d[w1w2w3⋮wn] c \, \mathbf{v} + d \, \mathbf{w} = c \begin{bmatrix} v_1 \\ v_2 \\ v_3 \\ \vdots \\ v_n \end{bmatrix} + d \begin{bmatrix} w_1 \\ w_2 \\ w_3 \\ \vdots \\ w_n \end{bmatrix}
c v+d w=c[v1v2v3⋮vn]+d[w1w2w3⋮wn]=[c v1+d w1c v2+d w2c v3+d w3⋮c vn+d wn] c \, \mathbf{v} + d \, \mathbf{w} = c \begin{bmatrix} v_1 \\ v_2 \\ v_3 \\ \vdots \\ v_n \end{bmatrix} + d \begin{bmatrix} w_1 \\ w_2 \\ w_3 \\ \vdots \\ w_n \end{bmatrix} = \begin{bmatrix} c \, v_1 + d \, w_1 \\ c \, v_2 + d \, w_2 \\ c \, v_3 + d \, w_3 \\ \vdots \\ c \, v_n + d \, w_n \end{bmatrix}
=[c v1+d w1c v2+d w2c v3+d w3⋮c vn+d wn] = \begin{bmatrix} c \, v_1 + d \, w_1 \\ c \, v_2 + d \, w_2 \\ c \, v_3 + d \, w_3 \\ \vdots \\ c \, v_n + d \, w_n \end{bmatrix}

You can create linear combinations of as many vectors as you like, with as many dimensions as you like. Let us look at a numeric example with three vectors and three scalars.

c=2,d=7,e=4c = 2, \quad d = 7, \quad e = 4
v=[170],w=[331],x=[962] \mathbf{v} = \begin{bmatrix} 1 \\ 7 \\ 0 \end{bmatrix}, \quad \mathbf{w} = \begin{bmatrix} 3 \\ 3 \\ 1 \end{bmatrix}, \quad \mathbf{x} = \begin{bmatrix} 9 \\ 6 \\ 2 \end{bmatrix}
c v+d w+e xc \, \mathbf{v} + d \, \mathbf{w} + e \, \mathbf{x}
c v+d w+e x=2[170]+7[331]+4[962]=[2⋅1+7⋅3+4⋅92⋅7+7⋅3+4⋅62⋅0+7⋅1+4⋅2]=[595915] c \, \mathbf{v} + d \, \mathbf{w} + e \, \mathbf{x} = 2 \begin{bmatrix} 1 \\ 7 \\ 0 \end{bmatrix} + 7 \begin{bmatrix} 3 \\ 3 \\ 1 \end{bmatrix} + 4 \begin{bmatrix} 9 \\ 6 \\ 2 \end{bmatrix} = \begin{bmatrix} 2 \cdot 1 + 7 \cdot 3 + 4 \cdot 9 \\ 2 \cdot 7 + 7 \cdot 3 + 4 \cdot 6 \\ 2 \cdot 0 + 7 \cdot 1 + 4 \cdot 2 \end{bmatrix} = \begin{bmatrix} 59 \\ 59 \\ 15 \end{bmatrix}
=2[170]+7[331]+4[962] = 2 \begin{bmatrix} 1 \\ 7 \\ 0 \end{bmatrix} + 7 \begin{bmatrix} 3 \\ 3 \\ 1 \end{bmatrix} + 4 \begin{bmatrix} 9 \\ 6 \\ 2 \end{bmatrix}
=[2⋅1+7⋅3+4⋅92⋅7+7⋅3+4⋅62⋅0+7⋅1+4⋅2] = \begin{bmatrix} 2 \cdot 1 + 7 \cdot 3 + 4 \cdot 9 \\ 2 \cdot 7 + 7 \cdot 3 + 4 \cdot 6 \\ 2 \cdot 0 + 7 \cdot 1 + 4 \cdot 2 \end{bmatrix}
=[595915]= \begin{bmatrix} 59 \\ 59 \\ 15 \end{bmatrix}

All combinations of c vc \,\mathbf{v} fill a line

An interesting question we can now ask is what happens when we consider all linear combinations of linear combinations. Take the scalar multiplication of c vc \,\mathbf{v} and consider what that looks like for different values of cc. Let us define v\mathbf{v} as a two dimensional vector.

v=[17]\mathbf{v} = \begin{bmatrix} 1 \\ 7 \end{bmatrix}

Now let us calculate and plot c vc \,\mathbf{v} for different values of cc, ranging from −3-3 to 33. If you are unfamiliar with plotting vectors review the last post.

c=−3→c v=−3[17]=[−3−21] c = -3 \quad \rightarrow \quad c \,\mathbf{v} = -3 \begin{bmatrix} 1 \\ 7 \end{bmatrix} = \begin{bmatrix} -3 \\ -21 \end{bmatrix}
c=−2→c v=−2[17]=[−2−14] c = -2 \quad \rightarrow \quad c \,\mathbf{v} = -2 \begin{bmatrix} 1 \\ 7 \end{bmatrix} = \begin{bmatrix} -2 \\ -14 \end{bmatrix}
c=−1→c v=−3[17]=[−1−7] c = -1 \quad \rightarrow \quad c \,\mathbf{v} = -3 \begin{bmatrix} 1 \\ 7 \end{bmatrix} = \begin{bmatrix} -1 \\ -7 \end{bmatrix}
c=0→c v=0[17]=[00] c = 0 \quad \rightarrow \quad c \,\mathbf{v} = 0 \begin{bmatrix} 1 \\ 7 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}
c=1→c v=1[17]=[17] c = 1 \quad \rightarrow \quad c \,\mathbf{v} = 1 \begin{bmatrix} 1 \\ 7 \end{bmatrix} = \begin{bmatrix} 1 \\ 7 \end{bmatrix}
c=2→c v=2[17]=[214] c = 2 \quad \rightarrow \quad c \,\mathbf{v} = 2 \begin{bmatrix} 1 \\ 7 \end{bmatrix} = \begin{bmatrix} 2 \\ 14 \end{bmatrix}
c=3→c v=−3[17]=[321] c =3 \quad \rightarrow \quad c \,\mathbf{v} = -3 \begin{bmatrix} 1 \\ 7 \end{bmatrix} = \begin{bmatrix} 3 \\ 21 \end{bmatrix}

Plotting those points on a graph you can see they all fall along a straight line.

Consider what would happen if you plotted every value of cc. All the way from −∞-\infty to ∞\infty. Not just integers, but all the decemal value of cc: c=1.1c = 1.1, c=1.2c = 1.2, c=1.2345...c = 1.2345... and so on. If you imagine plotting c vc \, \mathbf{v} for every value of cc it would completely fill that line, from −∞-\infty to ∞\infty.

In math notation we call the space occupied by an infinitely long line R\mathbb{R}. The space occupied by an infinitely large two-dimensional plane is R2\mathbb{R^2}. An infinitely large three-dimensional space is R3\mathbb{R^3}, and so on. We can therefore say that c vc \, \mathbf{v} for every value of cc fills R\mathbb{R}.

The number of components in the vector v\mathbf{v} does not affect the fact that c vc \, \mathbf{v} for every value of cc fills R\mathbb{R}. If v\mathbf{v} has one dimension, a number, then c vc \, \mathbf{v} for every value of cc would fill a single line. This is the number line we are all familiar with. If v\mathbf{v} has two dimension then we need two axis to plot the line it fills, as we saw in the example above. If v\mathbf{v} has three dimensions, we would need three axis to plot the line, but c vc \, \mathbf{v} for every value of cc will still fill a single line.

All combinations of c v+d wc \,\mathbf{v} + d \, \mathbf{w} fill a plane

A linear combination of two vectors and two scalars will fill a plane, R2\mathbb{R^2}. We have shown that c vc \, \mathbf{v} for every value of cc will fill a line. If we take the linear combination of a different vector, w\mathbf{w}, and scalar, dd, then d wd \, \mathbf{w} for every value of dd will also fill a line.

Think of these two lines created by c vc \, \mathbf{v} and d wd \, \mathbf{w} as the two axis of a graph. specifying values on either one only allows us to move along that axis. But if we specify values on both at once, we can specify a point anywhere in that two dimensional space. If we take every value of cc and dd, it will fill that two-dimensional plane.

Linear combinations of more vectors

If all linear combinations of a single vector fill a one-dimensional line, and all linear combinations of two vectors fill a two-dimensional plane, it is not a big leap to say that all linear combinations of three vectors will fill a three-dimensional space, R3\mathbb{R^3}. Think of the third vector as the third axis of a graph. Consider all linear combinations of c v+d w+e xc \,\mathbf{v} + d \, \mathbf{w} + e \, \mathbf{x}: All the linear combinations of c vc \,\mathbf{v} contain every point on one axis. All linear combinations of d wd \, \mathbf{w} contain every point on the second axis. All linear combinations of e xe \, \mathbf{x} contain every point on the third axis. Combining all three will contain every point in that three dimensional space.

This logic continues for linear combinations of as many vectors as you have. All linear combinations of four vectors will fill four dimensional space, R4\mathbb{R^4}, all linear combinations of five vectors will fill five dimensional space, R5\mathbb{R^5}, and so on. (But good luck trying to picture those!)

Exceptions to the rule

If any of the vectors you are add together are in the same plane as each other, their linear combinations will not behave as expected. If you are adding the linear combinations of two vectors, and those two vectors are parallel, combinations of the two vectors do not create a two-dimensional space. If you consider the linear combinations of three vectors, if the third vector sits in the two dimensional plane created by the first two, it will not add a third dimension. When thinking about how many dimensions the linear combinations of vectors fills, it is important to consider if any of them are in the same plane.

Problems

If you can solve these problems you have understood scalar multiplication and linear combinations of vectors.

  1. Calculate the scalar multiplication c vc \,\mathbf{v}, if:
c=3,v=[251]c = 3, \quad \mathbf{v} = \begin{bmatrix} 2 \\ 5 \\ 1 \end{bmatrix}
  1. Calculate the linear combination c v+d wc \,\mathbf{v} + d \, \mathbf{w}, if:
d=2,w=[139]d = 2, \quad \mathbf{w} = \begin{bmatrix} 1 \\ 3 \\ 9 \end{bmatrix}
  1. Will all linear combinations of c v+d wc \,\mathbf{v} + d \, \mathbf{w} fill a line, plane or space?

Solutions

  1. To multiply a vector by a scalar, multiply each component of the vector by the scalar, creating a new vector with the same number of components.
c v=3[251]=[3⋅23⋅53⋅1]=[6153] c \, \mathbf{v} = 3 \begin{bmatrix} 2 \\ 5 \\ 1 \end{bmatrix} = \begin{bmatrix} 3 \cdot 2 \\ 3 \cdot 5 \\ 3 \cdot 1 \end{bmatrix} = \begin{bmatrix} 6 \\ 15 \\ 3 \end{bmatrix}
  1. To find linear combination c v+d wc \,\mathbf{v} + d \, \mathbf{w} first calculate the individual scalar multiplications, then sum the components:
c v+d w=3[251]+2[139] c \, \mathbf{v} + d \, \mathbf{w} = 3 \begin{bmatrix} 2 \\ 5 \\ 1 \end{bmatrix} + 2 \begin{bmatrix} 1 \\ 3 \\ 9 \end{bmatrix}
c v+d w=3[251]+2[139]=[3⋅2+2⋅13⋅5+2⋅33⋅1+2⋅9]=[82121] c \, \mathbf{v} + d \, \mathbf{w} = 3 \begin{bmatrix} 2 \\ 5 \\ 1 \end{bmatrix} + 2 \begin{bmatrix} 1 \\ 3 \\ 9 \end{bmatrix} = \begin{bmatrix} 3 \cdot 2 + 2 \cdot 1 \\ 3 \cdot 5 + 2 \cdot 3 \\ 3 \cdot 1 + 2 \cdot 9 \end{bmatrix} = \begin{bmatrix} 8 \\ 21 \\ 21 \end{bmatrix}
=[3⋅2+2⋅13⋅5+2⋅33⋅1+2⋅9] = \begin{bmatrix} 3 \cdot 2 + 2 \cdot 1 \\ 3 \cdot 5 + 2 \cdot 3 \\ 3 \cdot 1 + 2 \cdot 9 \end{bmatrix}
=[82121] = \begin{bmatrix} 8 \\ 21 \\ 21 \end{bmatrix}
  1. All linear combinations of two vectors fill a plane. The only exception would be if the two vectors are parallel. You can prove that the vectors in this example are not parallel by plotting them. If you need a refresher on how to plot vectors have a look at this post.
#Linear algebra

Read Next

Linear algebra part 4 - Dot products

In this post we will learn about the dot product of vectors and how you can use them to calculate portfolio values.

Linear algebra part 2 - Plotting vectors

Plotting vectors on a graph is a great way to gain an intuative understanding of what they are showing you. In this post we will learn how to plot two and three dimensional vectors.

Subscribe to 15 Minute Finance

Be the first to know when new posts are published

Buy me a coffee

If you have learnt something today, please consider supporting me