The dot product (aka inner product or scalar product) of two vectors \(\mathbf{a}\) and \(\mathbf{b}\) expresses how similar two vectors are and is defined as a scalar number expressing the angular relationship given by the product of the vector lengths times the cosine of the angle \(\theta\) between the vectors:
\[\mathbf{a}\cdot\mathbf{b} = |\mathbf{a}| |\mathbf{b}|\cos\theta\]
When we draw a triangle to apply the definition of the dot product
we can read off the Law of Cosines:
\[\begin{array}{rrl} &|\mathbf{b} - \mathbf{a}|^2 &= |\mathbf{a}|^2 + |\mathbf{b}|^2 - 2|\mathbf{a}||\mathbf{b}|\cos\theta\\ \Leftrightarrow &|\mathbf{c}|^2&=|\mathbf{a}|^2 + |\mathbf{b}|^2 - 2(\mathbf{a}\cdot\mathbf{b})\\ \Leftrightarrow &\mathbf{c}\cdot\mathbf{c}&=\mathbf{a}\cdot\mathbf{a} + \mathbf{b}\cdot\mathbf{b}-2\mathbf{a}\cdot\mathbf{b}\\ & &= (\mathbf{a} - \mathbf{b})\cdot(\mathbf{a}-\mathbf{b}) \end{array} \]
But we can also rewrite the first line to
\[- 2|\mathbf{a}||\mathbf{b}|\cos\theta = |\mathbf{b}-\mathbf{a}|^2 - |\mathbf{a}|^2 - |\mathbf{b}|^2\]
When substituting the length of the vector definitions we get
\[\begin{array}{rl} - 2|\mathbf{a}||\mathbf{b}|\cos\theta =& (a_1 - b_1)^2 + (a_2 - b_2)^2 + \dots - (a_1^2 + a_2^2 + \dots) - (b_1^2 + b_2^2 + \dots)\\ =& -2a_1b_1-2a_2b_2-\dots \end{array}\]
From which follows the algebraic view on the dot product as the sum of the products of their corresponding coordinates
\[\mathbf{a}\cdot\mathbf{b} = \sum\limits_{i=1}^d a_ib_i= \mathbf{a}^T\mathbf{b}\]
Proof:
Let \(\mathbf e_{i}\) be the orthonormal basis vector,
\[\begin{array}{rl} \mathbf{a}\cdot\mathbf{b} =& \mathbf {a} \cdot \sum \limits_{i=1}^d {b}_{i}\mathbf {e} _{i}\\ =&\sum \limits_{i=1}^d {b}_{i}(\mathbf {a} \cdot \mathbf {e} _{i})\\ =&\sum \limits_{i=1}^d {b}_{i}(|\mathbf {a}| \cdot |\mathbf {e} _{i}|\cos\theta)\\ =&\sum \limits_{i=1}^d {b}_{i}(|\mathbf {a}| \cos\theta)\\ =&\sum \limits_{i=1}^d {b}_{i} {a}_{i}\\ =&\sum \limits_{i=1}^d {a}_{i} {b}_{i}\\ \end{array}\]
Properties of Dot Product
If both vectors \(\mathbf{a}\) and \(\mathbf{b}\) are orthogonal\(\left(\theta=90^°\right)\) they are called normal to each other and it follows that
\[\mathbf{a}\cdot\mathbf{b} =|\mathbf{a}||\mathbf{b}|\cos\frac{\pi}{2}= 0\]
since \(\cos(90^\circ)=0\). We prefer the term orthogonal over perpendicular to indicate that the dot product of two vectors is zero, since perpendicularity is only possible for non-zero vectors.
If \(|\theta|<90^°\) it follows that
\[\mathbf{a}\cdot\mathbf{b} > 0\]
If \(|\theta|>90^°\) it follows that
\[\mathbf{a}\cdot\mathbf{b} < 0\]
If the vectors are contradirectional or parallel with opposite direction \(\left(\theta=180^°\right)\) it follows that
\[\mathbf{a}\cdot\mathbf{b} = -|\mathbf{a}|\cdot|\mathbf{b}|\]
If the vectors are codirectional or parallel with same direction \(\left(\theta=0^°\right)\) it follows that
\[\mathbf{a}\cdot\mathbf{b} = |\mathbf{a}|\cdot|\mathbf{b}|\]
We can use this property to see that the dot product with itself is
\[\mathbf{a}\cdot\mathbf{a} = |\mathbf{a}|\cdot|\mathbf{a}|=|\mathbf{a}|^2\]
It turns out the vector length or norm can therefore be defined as
\[|\mathbf{a}|=\sqrt{\mathbf{a}\cdot\mathbf{a}}\]
The length of the sum of two vectors \(\mathbf{a}+\mathbf{b}\) can be expressed in terms of inner products as follows
\[\begin{array}{rl} |\mathbf{a}+\mathbf{b}|^2 &= (\mathbf{a}+\mathbf{b})\cdot(\mathbf{a}+\mathbf{b})\\ &= \mathbf{a}\cdot\mathbf{a}+\mathbf{a}\cdot\mathbf{b}+\mathbf{b}\cdot\mathbf{a}+\mathbf{b}\cdot\mathbf{b}\\ &= |\mathbf{a}|^2+2\mathbf{a}\cdot\mathbf{b}+|\mathbf{b}|^2 \end{array}\]
Commutative law
The dot product is symmtric:
\[\mathbf{a}\cdot\mathbf{b}=\mathbf{b}\cdot\mathbf{a}\]
Proof:\[\mathbf{a}\cdot\mathbf{b}=|\mathbf{a}|\cdot|\mathbf{b}|\cos\theta=|\mathbf{b}|\cdot|\mathbf{a}|\cos\theta=\mathbf{b}\cdot\mathbf{a}\]
Distributive law
The dot product is additive distributive
\[\mathbf{a}\cdot(\mathbf{b}+\mathbf{c})=(\mathbf{a}\cdot\mathbf{b})+(\mathbf{a}\cdot\mathbf{c})\]
Proof:
\[\begin{array}{rl} \mathbf{a}\cdot(\mathbf{b}+\mathbf{c}) &= [\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot([\mathbf{b}_1, \mathbf{b}_2, ..., \mathbf{b}_d]+[\mathbf{c}_1, \mathbf{c}_2, ..., \mathbf{c}_d])\\ &= [\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot[\mathbf{b}_1+\mathbf{c}_1, \mathbf{b}_2+\mathbf{c}_2, ..., \mathbf{b}_d+\mathbf{c}_d]\\ &= [\mathbf{a}_1(\mathbf{b}_1+\mathbf{c}_1), \mathbf{a}_2(\mathbf{b}_2+\mathbf{c}_2), ..., \mathbf{a}_d(\mathbf{b}_d+\mathbf{c}_d)]\\ &= [\mathbf{a}_1\mathbf{b}_1+\mathbf{a}_1\mathbf{c}_1, \mathbf{a}_2\mathbf{b}_2+\mathbf{a}_2\mathbf{c}_2, ..., \mathbf{a}_d\mathbf{b}_d+\mathbf{a}_d\mathbf{c}_d]\\ &= [\mathbf{a}_1\mathbf{b}_1, \mathbf{a}_2\mathbf{b}_2, ..., \mathbf{a}_d\mathbf{b}_d]+[\mathbf{a}_1\mathbf{c}_1,\mathbf{a}_2\mathbf{c}_2, ..., \mathbf{a}_d\mathbf{c}_d]\\ &= [\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot[\mathbf{b}_1, \mathbf{b}_2, ..., \mathbf{b}_d]+[\mathbf{a}_1, \mathbf{a}_2, ..., \mathbf{a}_d]\cdot(\mathbf{c}_1, \mathbf{c}_2, ..., \mathbf{c}_d]\\ &=\mathbf{a}\cdot\mathbf{b}+\mathbf{a}\cdot\mathbf{c} \end{array}\]
From distributive law follows that we can expand an expression like for example
\[(\mathbf{a}+\mathbf{b})\cdot(\mathbf{c}+\mathbf{d}) = \mathbf{a}\cdot\mathbf{c} + \mathbf{a}\cdot\mathbf{d} + \mathbf{b}\cdot\mathbf{c} + \mathbf{b}\cdot\mathbf{d}\]
Mixed associative law
The dot product has a homogeneity
\[\alpha (\mathbf{a}\cdot\mathbf{b}) = (\alpha\mathbf{a})\cdot \mathbf{b} = \mathbf{a}\cdot(\alpha\mathbf{b})\]
Bilinear
Often the mixed associative law and distributive law are combined to show that the dot product is bilinear:
\[\mathbf{a}\cdot(\alpha\mathbf{b}+\mathbf{c}) = \alpha(\mathbf{a}\cdot\mathbf{b})+(\mathbf{a}\cdot\mathbf{c})\]
Scalar Multiplication
\[(\alpha_1\mathbf{a})\cdot(\alpha_2\mathbf{b})=\alpha_1\alpha_2(\mathbf{a}\cdot\mathbf{b})\]
Dot product is homogenious under scaling \[\alpha(\mathbf{a}\cdot\mathbf{b}) = (\alpha\mathbf{a})\cdot\mathbf{b} = \mathbf{a}\cdot(\alpha\mathbf{b})\]
Cauchy-Schwarz Inequality
\[|\mathbf{a}\cdot\mathbf{b}|\leq|\mathbf{a}||\mathbf{b}|\]
No cancellation
If \(\mathbf{a}\cdot\mathbf{b}=\mathbf{a}\cdot\mathbf{c}\) and \(\mathbf{a}\neq \mathbf{0}\), we can write \(\mathbf{a}\cdot(\mathbf{b}-\mathbf{c})=0\) by distributive law, which means that \(\mathbf{a}\perp(\mathbf{b}-\mathbf{c})\), which allows \((\mathbf{b}-\mathbf{c})\neq\mathbf{0}\) and therefore \(\mathbf{b}\neq\mathbf{c}\).
Applications
Fast calculation of Cosine
The definition of the dot-product is used extensively in computer graphics since it speeds up computations in many circumstances by avoiding inefficient trigonometric functions. Additionally, working with normalized vectors gives the cosine between these two vectors by taking the dot product of them:
\[\cos\theta = \frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{a}||\mathbf{b}|} = \hat{\mathbf{a}}\cdot\hat{\mathbf{b}}\]
Making perpendicular vectors
Since two vectors are perpendicular when \(\mathbf{a}\cdot\mathbf{b} = 0\), we can easily construct a perpendicular vector by zeroing all components except 2, flipping those two and reversing the sign of one of them. For example with \(\mathbf{a} = (\mathbf{a}_1, \mathbf{a}_2, \dots, \mathbf{a}_n)\) the vectors \(\mathbf{a}' = (-\mathbf{a}_2, \mathbf{a}_1, 0, \dots, 0)\) or \(\mathbf{a}'' = (0, -\mathbf{a}_3, \mathbf{a}_2, 0, \dots, 0)\), etc are all perpendcular to \(\mathbf{a}\), that means that the dot product of any of these with the original vector \(\mathbf{a}\) is zero.
Vector Projection
The dot product can be used to determine the projection of one vector onto another, like for shadows. Given two vectors \(\mathbf{a}\) and \(\mathbf{b}\) we want to determine the projection of \(\mathbf{a}\) onto \(\mathbf{b}\), which is the vector \(\mathbf{a}'\) of length \(a'\) parallel to the vector \(\mathbf{b}\).
The length of the projection of \(\mathbf{a}\) on \(\mathbf{b}\) here \(a'\) is called the Scalar Projection or Vector Component and is determined by
\[a' = |\mathbf{a}|\cos\theta = |\mathbf{a}|\underbrace{\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{a}|\cdot|\mathbf{b}|}}_{\cos\theta}=\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|}=\mathbf{a}\cdot\frac{\mathbf{b}}{|\mathbf{b}|}=\mathbf{a}\cdot\hat{\mathbf{b}}=:\text{comp}_{\mathbf{b}}(\mathbf{a})\]
Having the Scalar Projection, we can then calculate the Vector Projection of \(\mathbf{a}\) onto \(\mathbf{b}\), highlighted in blue:
\[\mathbf{a}' = a'\cdot\hat{\mathbf{b}} = (\mathbf{a}\cdot\hat{\mathbf{b}})\cdot\hat{\mathbf{b}}=\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|}\cdot\frac{\mathbf{b}}{|\mathbf{b}|}=\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|^2}\mathbf{b}=\frac{\mathbf{a}\cdot\mathbf{b}}{\mathbf{b}\cdot\mathbf{b}}\mathbf{b}=:\text{proj}_{\mathbf{b}}(\mathbf{a})\]
Scalar Projection Percentage: The length of the distance \(a'\) to the total length of \(\mathbf{b}\) can be expressed in percent as follows:
\[\frac{a'}{|\mathbf{b}|} = \frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|\cdot |\mathbf{b}|} =\frac{\mathbf{a}\cdot\mathbf{b}}{|\mathbf{b}|^2}=\frac{\mathbf{a}\cdot\mathbf{b}}{\mathbf{b}\cdot\mathbf{b}}\]
Which is interesting. It says that the Vector Projection is just a scaling of vector \(\mathbf{b}\) by the proportion of \({a}'\) to \(\mathbf{b}\).
Even more interesting though, when we multiply \({a}'=|\mathbf{a}|\cos\theta\) with the length of \(\mathbf{b}\), we get that the length of the projection times the length of the vector projected onto, IS the dot product!
\[{a}'|\mathbf{b}| = |\mathbf{a}||\mathbf{b}|\cos\theta = \mathbf{a}\cdot\mathbf{b}\]
Distance between Line and Point
Given a line as \(y=mx+b\) or in form as a line segment between two points \({P}_1\) and \({P}_2\) as well as a point \({P}\), we only need to find a normal \(\mathbf{n}\) to that line and a point \({Q}\) on the line. Given the a segment, a possible point \({Q}={P}_1\) and given the line formula, a possible point is \({Q}=\left(\begin{array}{c}0\\b\end{array}\right)\).
The normal \(\mathbf{n}\) is basically \(\mathbf{b}^\perp\), the perpendicular vector of \(\mathbf{b}={P}_2-{P}_1\) between \({P}_1\) and \({P}_2\). Given the formula, we know that if we multiply the slopes of two perpendicular lines, we get \(−1\). The slope must therefore be \(-\frac{1}{m}\). If we want the line pass point \(\mathbf{Q}\), we have \(y = -\frac{1}{m}x+b\). Therefore \(\mathbf{n}=\left(\begin{array}{c}1\\-\frac{1}{m}\end{array}\right)\).
The distance between the line and point \({P}\) is thus the vector projection from vector \(\mathbf{a}={P}-{Q}\) between point \({Q}\) and \({P}\) onto \(\mathbf{n}\).
\[d = \frac{|\mathbf{n}\cdot \mathbf{a}|}{|\mathbf{n}|}\]
For the case of a line segment, the expression can then be further simplified using the 2D Perp product:
\[d=|\hat{\mathbf{b}}^\perp\cdot\mathbf{a}|\]
Direction Cosines
In three dimensional space, a vector \(\mathbf{a}\) will form the angle \(\alpha\) with the x-axis, \(\beta\) with the y-axis and \(\gamma\) with the z-axis. These angles are called direction angles and the cosine of these angles are called direction cosines.
PLOT
When we use the standard basis vectors for the dimension \(\mathbf{i}, \mathbf{j}\) and \(\mathbf{k}\), the direction cosines are:
\[\begin{array}{rl} \cos\alpha =& \frac{\mathbf{a}\cdot\mathbf{i}}{|\mathbf{a}|\cdot|\mathbf{i}|} = \frac{\mathbf{a}_1}{|\mathbf{a}|}\\ \cos\beta =& \frac{\mathbf{a}\cdot\mathbf{j}}{|\mathbf{a}|\cdot|\mathbf{j}|} = \frac{\mathbf{a}_2}{|\mathbf{a}|}\\ \cos\gamma =& \frac{\mathbf{a}\cdot\mathbf{k}}{|\mathbf{a}|\cdot|\mathbf{k}|} = \frac{\mathbf{a}_3}{|\mathbf{a}|}\\ \end{array}\]
As such, the following properties hold:
- The vector \(\mathbf{b}=[\cos\alpha, \cos\beta, \cos\gamma]^T\) is a unit vector.
- \(\cos^2\alpha+ \cos^2\beta+ \cos^2\gamma=1\)
- \(\mathbf{a} = |\mathbf{a}|[\cos\alpha, \cos\beta, \cos\gamma]^T\)
Linear Combination
A very common use-case of the dot product are linear combinations of single variables \(\mathbf{a}\) and \(\mathbf{b}\). This idea is continued with matrix multiplication, where the dot product is the inner ingredient.
As an example, let \(\mathbf{a}=(20, 30, 40)\) be the number of products in our shop and \(\mathbf{b}=(\$5, \$6, \$7)\) the prices per product. How much is the inventory worth? Obviously \(\$5\cdot 20+\$6\cdot 30+\$7\cdot 40 = \mathbf{a}\cdot\mathbf{b}\)
Equation of a line
The equation of a line in 3D space in the form \(ax+by+cz=d\) can be written as a dot product
\[\mathbf{a}\cdot\mathbf{x}=d\]
where \(\mathbf{a}=(a, b, c)\) and \(\mathbf{x}=(x, y, z)\). Similarly, for 2D lines and any other space.