Dot Product

The dot product (aka inner product or scalar product) of two vectors \(\mathbf{a}\) and \(\mathbf{b}\) expresses how similar two vectors are and is defined as a scalar number expressing the angular relationship given by the product of the vector lengths times the cosine of the angle \(\theta\) between the vectors:

\[\mathbf{a}\cdot\mathbf{b} = |\mathbf{a}|\cdot|\mathbf{b}|\cos\theta\]

When we draw a triangle to apply the definition of the dot product

we can read off the Law of cosines:

\[|\mathbf{b}-\mathbf{a}|^2 = |\mathbf{a}|^2 + |\mathbf{b}|^2 - 2|\mathbf{a}||\mathbf{b}|\cos\theta\]

which we can rewrite to

\[- 2|\mathbf{a}||\mathbf{b}|\cos\theta = |\mathbf{b}-\mathbf{a}|^2 - |\mathbf{a}|^2 - |\mathbf{b}|^2\]

When substituting the length of the vector definitions we get

\[\begin{array}{rl}- 2|\mathbf{a}||\mathbf{b}|\cos\theta =& (a_1 - b_1)^2 + (a_2 - b_2)^2 + \dots - (a_1^2 + a_2^2 + \dots) - (b_1^2 + b_2^2 + \dots)\\=& -2a_1b_1-2a_2b_2-\dots\end{array}\]

From which follows the algebraic view on the dot product as the sum of the products of their corresponding coordinates

\[\mathbf{a}\cdot\mathbf{b} = \sum\limits_{i=1}^d a_ib_i= \mathbf{a}^T\mathbf{b}\]

Proof:

Let \(\mathbf e_{i}\) be the orthonormal basis vector,

\[\begin{array}{rl}\mathbf{a}\cdot\mathbf{b} =& \mathbf {a} \cdot \sum \limits_{i=1}^d {b}_{i}\mathbf {e} _{i}\\=&\sum \limits_{i=1}^d {b}_{i}(\mathbf {a} \cdot \mathbf {e} _{i})\\=&\sum \limits_{i=1}^d {b}_{i}(|\mathbf {a}| \cdot |\mathbf {e} _{i}|\cos\theta)\\=&\sum \limits_{i=1}^d {b}_{i}(|\mathbf {a}| \cos\theta)\\=&\sum \limits_{i=1}^d {b}_{i} {a}_{i}\\=&\sum \limits_{i=1}^d {a}_{i} {b}_{i}\\\end{array}\]

Properties of Dot Product

If both vectors \(\mathbf{a}\) and \(\mathbf{b}\) are orthogonal \(\left(\theta=90^°\right)\) they are called normal to each other and it follows that

\[\mathbf{a}\cdot\mathbf{b} = 0\]

since \(\cos(90^\circ)=0\). We prefer the term orthogonal over perpendicular to indicate that the dot product of two vectors is zero, since perpendicularity is only possible for non-zero vectors.

If \(|\theta|<90^°\) it follows that

\[\mathbf{a}\cdot\mathbf{b} > 0\]

If \(|\theta|>90^°\) it follows that

\[\mathbf{a}\cdot\mathbf{b} < 0\]

If the vectors are contradirectional or parallel with opposite direction \(\left(\theta=180^°\right)\) it follows that

\[\mathbf{a}\cdot\mathbf{b} = -|\mathbf{a}|\cdot|\mathbf{b}|\]

If the vectors are codirectional or parallel with same direction \(\left(\theta=0^°\right)\) it follows that

\[\mathbf{a}\cdot\mathbf{b} = |\mathbf{a}|\cdot|\mathbf{b}|\]

We can use this property to see that the dot product with itself is

\[\mathbf{a}\cdot\mathbf{a} = |\mathbf{a}|\cdot|\mathbf{a}|=|\mathbf{a}|^2\]

It turns out the vector length or norm can therefore be defined as

\[|\mathbf{a}|=\sqrt{\mathbf{a}\cdot\mathbf{a}}\]

Commutative law

The dot product is symmtric:

\[\mathbf{a}\cdot\mathbf{b}=\mathbf{b}\cdot\mathbf{a}\]

Proof: \[\mathbf{a}\cdot\mathbf{b}=|\mathbf{a}|\cdot|\mathbf{b}|\cos\theta=|\mathbf{b}|\cdot|\mathbf{a}|\cos\theta=\mathbf{b}\cdot\mathbf{a}\]

Distributive law

The dot product is additive distributive

\[\mathbf{a}\cdot(\mathbf{b}+\mathbf{c})=(\mathbf{a}\cdot\mathbf{b})+(\mathbf{a}\cdot\mathbf{c})\]

Proof:

\[\begin{array}{rl}\mathbf{a}\cdot(\mathbf{b}+\mathbf{c}) &= [\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot([\mathbf{b}_1, \mathbf{b}_2, ..., \mathbf{b}_d]+[\mathbf{c}_1, \mathbf{c}_2, ..., \mathbf{c}_d])\\&= [\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot[\mathbf{b}_1+\mathbf{c}_1, \mathbf{b}_2+\mathbf{c}_2, ..., \mathbf{b}_d+\mathbf{c}_d]\\&= [\mathbf{a}_1(\mathbf{b}_1+\mathbf{c}_1), \mathbf{a}_2(\mathbf{b}_2+\mathbf{c}_2), ..., \mathbf{a}_d(\mathbf{b}_d+\mathbf{c}_d)]\\&= [\mathbf{a}_1\mathbf{b}_1+\mathbf{a}_1\mathbf{c}_1, \mathbf{a}_2\mathbf{b}_2+\mathbf{a}_2\mathbf{c}_2, ..., \mathbf{a}_d\mathbf{b}_d+\mathbf{a}_d\mathbf{c}_d]\\&= [\mathbf{a}_1\mathbf{b}_1, \mathbf{a}_2\mathbf{b}_2, ..., \mathbf{a}_d\mathbf{b}_d]+[\mathbf{a}_1\mathbf{c}_1,\mathbf{a}_2\mathbf{c}_2, ..., \mathbf{a}_d\mathbf{c}_d]\\&= [\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot[\mathbf{b}_1, \mathbf{b}_2, ..., \mathbf{b}_d]+[\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot(\mathbf{c}_1, \mathbf{c}_2, ..., \mathbf{c}_d]\\&=\mathbf{a}\cdot\mathbf{b}+\mathbf{a}\cdot\mathbf{c}\end{array}\]

From distributive law follows that we can expand an expression like for example

\[(\mathbf{a}+\mathbf{b})\cdot(\mathbf{c}+\mathbf{d}) = \mathbf{a}\cdot\mathbf{c} + \mathbf{a}\cdot\mathbf{d} + \mathbf{b}\cdot\mathbf{c} + \mathbf{b}\cdot\mathbf{d}\]

Mixed associative law

The dot product has a homogeneity

\[\alpha (\mathbf{a}\cdot\mathbf{b}) = (\alpha\mathbf{a})\cdot \mathbf{b} = \mathbf{a}\cdot(\alpha\mathbf{b})\]

Bilinear

Often the mixed associative law and distributive law are combined to show that the dot product is bilinear:

\[\mathbf{a}\cdot(\alpha\mathbf{b}+\mathbf{c}) = \alpha(\mathbf{a}\cdot\mathbf{b})+(\mathbf{a}\cdot\mathbf{c})\]

Scalar Multiplication

\[(\alpha_1\mathbf{a})\cdot(\alpha_2\mathbf{b})=\alpha_1\alpha_2(\mathbf{a}\cdot\mathbf{b})\]

Dot product is homogenious under scaling \[\alpha(\mathbf{a}\cdot\mathbf{b}) = (\alpha\mathbf{a})\cdot\mathbf{b} = \mathbf{a}\cdot(\alpha\mathbf{b})\]

Cauchy-Schwarz Inequality

\[|\mathbf{a}\cdot\mathbf{b}|\leq|\mathbf{a}||\mathbf{b}|\]

No cancellation

If \(\mathbf{a}\cdot\mathbf{b}=\mathbf{a}\cdot\mathbf{c}\) and \(\mathbf{a}\neq \mathbf{0}\), we can write \(\mathbf{a}\cdot(\mathbf{b}-\mathbf{c})=0\) by distributive law, which means that \(\mathbf{a}\perp(\mathbf{b}-\mathbf{c})\), which allows \((\mathbf{b}-\mathbf{c})\neq\mathbf{0}\) and therefore \(\mathbf{b}\neq\mathbf{c}\).

Applications

Fast calculation of Cosine

The definition of the dot-product is used extensively in computer graphics since it speeds up computations in many circumstances by avoiding inefficient trigonometric functions. Additionally, working with normalized vectors gives the cosine between these two vectors by taking the dot product of them:

\[\cos\theta = \frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{a}||\mathbf{b}|} = \hat{\mathbf{a}}\cdot\hat{\mathbf{b}}\]

Making perpendicular vectors

Since two vectors are perpendicular when \(\mathbf{a}\cdot\mathbf{b} = 0\), we can easily construct a perpendicular vector by zeroing all components except 2, flipping those two and reversing the sign of one of them. For example with \(\mathbf{a} = (\mathbf{a}_1, \mathbf{a}_2, \dots, \mathbf{a}_n)\) the vectors \(\mathbf{a}' = (-\mathbf{a}_2, \mathbf{a}_1, 0, \dots, 0)\) or \(\mathbf{a}'' = (0, -\mathbf{a}_3, \mathbf{a}_2, 0, \dots, 0)\), etc are all perpendcular to \(\mathbf{a}\), that means that the dot product of any of these with the original vector \(\mathbf{a}\) is zero.

Vector Projection

The dot product can be used to determine the projection of one vector onto another, like for shadows. Given two vectors \(\mathbf{a}\) and \(\mathbf{b}\) we want to determine the projection of \(\mathbf{a}\) onto \(\mathbf{b}\), which is the vector \(\mathbf{a}'\) of length \(a'\) parallel to the vector \(\mathbf{b}\).

The length of the projection of \(\mathbf{a}\) on \(\mathbf{b}\) here \(a'\) is called the Scalar Projection or Vector Component and is determined by

\[a' = |\mathbf{a}|\cos\theta = |\mathbf{a}|\underbrace{\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{a}|\cdot|\mathbf{b}|}}_{\cos\theta}=\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|}=\mathbf{a}\cdot\frac{\mathbf{b}}{|\mathbf{b}|}=\mathbf{a}\cdot\hat{\mathbf{b}}=:\text{comp}_{\mathbf{b}}(\mathbf{a})\]

Having the Scalar Projection, we can then calculate the Vector Projection of \(\mathbf{a}\) on \(\mathbf{b}\), highlighted in blue:

\[\mathbf{a}' = a'\cdot\hat{\mathbf{b}} = (\mathbf{a}\cdot\hat{\mathbf{b}})\cdot\hat{\mathbf{b}}=\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|}\cdot\frac{\mathbf{b}}{|\mathbf{b}|}=\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|^2}\mathbf{b}=\frac{\mathbf{a}\cdot\mathbf{b}}{\mathbf{b}\cdot\mathbf{b}}\mathbf{b}=:\text{proj}_{\mathbf{b}}(\mathbf{a})\]

Scalar Projection Percentage: The length of the distance \(a'\) to the total length of \(\mathbf{b}\) can be expressed in percent as follows:

\[\frac{a'}{|\mathbf{b}|} = \frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|\cdot |\mathbf{b}|} =\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|^2}=\frac{\mathbf{a}\cdot\mathbf{b}}{\mathbf{b}\cdot\mathbf{b}}\]

Which is interesting. It says that the Vector Projection is just a scaling of vector \(\mathbf{b}\) by the proportion of \(\mathbf{a}'\) to \(\mathbf{b}\).

Distance between Line and Point

Given a line as \(y=mx+b\) or in form as a line segment between two points \({P}_1\) and \({P}_2\) as well as a point \({P}\), we only need to find a normal \(\mathbf{n}\) to that line and a point \({Q}\) on the line. Given the a segment, a possible point \({Q}={P}_1\) and given the line formula, a possible point is \({Q}=\left(\begin{array}{c}0\\b\end{array}\right)\).

The normal \(\mathbf{n}\) is basically \(\mathbf{b}^\perp\), the perpendicular vector of \(\mathbf{b}={P}_2-{P}_1\) between \({P}_1\) and \({P}_2\). Given the formula, we know that if we multiply the slopes of two perpendicular lines, we get \(−1\). The slope must therefore be \(-\frac{1}{m}\). If we want the line pass point \(\mathbf{Q}\), we have \(y = -\frac{1}{m}x+b\). Therefore \(\mathbf{n}=\left(\begin{array}{c}1\\-\frac{1}{m}\end{array}\right)\).

The distance between the line and point \({P}\) is thus the vector projection from vector \(\mathbf{a}={P}-{Q}\) between point \({Q}\) and \({P}\) onto \(\mathbf{n}\).

\[d = \frac{|\mathbf{n}\cdot \mathbf{a}|}{|\mathbf{n}|}\]

For the case of a line segment, the expression can then be further simplified using the 2D Perp product:

\[d=|\hat{\mathbf{b}}^\perp\cdot\mathbf{a}|\]

Direction Cosines

In three dimensional space, a vector \(\mathbf{a}\) will form the angle \(\alpha\) with the x-axis, \(\beta\) with the y-axis and \(\gamma\) with the z-axis. These angles are called direction angles and the cosine of these angles are called direction cosines.

PLOT

When we use the standard basis vectors for the dimension \(\mathbf{i}, \mathbf{j}\) and \(\mathbf{k}\), the direction cosines are:

\[\begin{array}{rl}\cos\alpha =& \frac{\mathbf{a}\cdot\mathbf{i}}{|\mathbf{a}|\cdot|\mathbf{i}|} = \frac{\mathbf{a}_1}{|\mathbf{a}|}\\\cos\beta =& \frac{\mathbf{a}\cdot\mathbf{j}}{|\mathbf{a}|\cdot|\mathbf{j}|} = \frac{\mathbf{a}_2}{|\mathbf{a}|}\\\cos\gamma =& \frac{\mathbf{a}\cdot\mathbf{k}}{|\mathbf{a}|\cdot|\mathbf{k}|} = \frac{\mathbf{a}_3}{|\mathbf{a}|}\\\end{array}\]

As such, the following properties hold:

  1. The vector \(\mathbf{b}=[\cos\alpha, \cos\beta, \cos\gamma]^T\) is a unit vector.
  2. \(\cos^2\alpha+ \cos^2\beta+ \cos^2\gamma=1\)
  3. \(\mathbf{a} = |\mathbf{a}|[\cos\alpha, \cos\beta, \cos\gamma]^T\)

Law of Cosines

Interestingly, the well known law of cosines can be derived using the dot-product:

\[{\begin{array}{rl}\mathbf {c} \cdot \mathbf {c} &=(\mathbf {a} -\mathbf {b} )\cdot (\mathbf {a} -\mathbf {b} )\\&=\mathbf {a} \cdot \mathbf {a} -\mathbf {a} \cdot \mathbf {b} -\mathbf {b} \cdot \mathbf {a} +\mathbf {b} \cdot \mathbf {b} \\&=a^{2}-\mathbf {a} \cdot \mathbf {b} -\mathbf {a} \cdot \mathbf {b} +b^{2}\\&=a^{2}-2(\mathbf {a} \cdot \mathbf {b}) +b^{2}\\c^{2}&=a^{2}+b^{2}-2ab\cos \theta \\\end{array}}\]

Linear Combination

A very common use-case of the dot product are linear combinations of single variables \(\mathbf{a}\) and \(\mathbf{b}\). This idea is continued with matrix multiplication, where the dot product is the inner ingredient.

As an example, let \(\mathbf{a}=(20, 30, 40)\) be the number of products in our shop and \(\mathbf{b}=(\$5, \$6, \$7)\) the prices per product. How much is the inventory worth? Obviously \(\$5\cdot 20+\$6\cdot 30+\$7\cdot 40 = \mathbf{a}\cdot\mathbf{b}\)

« Back to Book Overview