Inverse Perspective Mapping of Images Without OpenCV

Introduction

With the development of autonomous driving technology, more and more people are being exposed to newer technologies, and are increasingly curious about how the computer world actually achieves autonomous driving. They are also curious about how some of the features that come with autonomous driving systems are implemented. This time, we will explain and practice the algorithm logic behind the "360° reversing camera" in the system.

Inverse Perspective Mapping

When capturing the image, the vehicle calls multiple cameras and stitches them into a "360° panoramic photo".

To form the top-down view of the "360° reversing camera", a mathematical operation is required, which is Inverse Perspective Mapping, or IPM for short.

In this field, there are many IPM transformation methods, such as the "corresponding point homography transformation method" and the "simplified camera model inverse perspective transformation", but they all use matrix transformation rules.

Corresponding Point Homography Transformation Method

This transformation method is relatively simple, so we won't describe it in too much detail.

Input at least four corresponding point pairs, with no three or more points being collinear. No information about the camera parameters or the plane position is needed. Using the point pairs, solve for the perspective transformation matrix, where the matrix is a third-order square matrix, so a linear equation can be constructed to solve it. If there are more than four points, the $ransac$ method can be used to solve it, and the point selection method is usually manual, typically by selecting vanishing points.

\begin{bmatrix} t_i x'_i \\ t_i y'_i \\ t_i \end{bmatrix} = map\_matrix \cdot \begin{bmatrix} x_i \\ y_i \\ 1 \end{bmatrix}

dst(i) = (x'_i, y'_i), src(i) = (x_i, y_i), i = 0, 1, 2, 3

This transformation is relatively simple to implement in code, and the IPM transformation can be implemented relatively easily. We won't go into too much detail here, and we won't provide a code example either.

Simplified Camera Model IPM Method

This is the transformation method we will focus on this time. The essence of this algorithm is to use the conversion relationships between various coordinates during the camera imaging process, then abstract and simplify it, finally obtaining the world coordinates.

Then, establish the correspondence between world coordinates and image coordinates, and use this relationship to perform the mathematical transformation.

Unlike some complex and lengthy calculation formulas, we still use coordinate operations here. For this IPM calculation method, we need to first measure the actual camera parameters.

Here, the elevation angle $θ$ is $23°$ , the center height $H$ is $37 cm$ , and the distance from the viewpoint to the view plane $d$ is $87 cm$ . Then, we find the world coordinate $P_W$ .

Assume the camera image coordinate is $\boldsymbol{P_G} = (x, y, z, 1)$ , and establish the matrix equation based on the relationship between world coordinates and image coordinates,

\boldsymbol{P_G} = \boldsymbol{P_W} \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & -H & 1 \end{bmatrix} \begin{bmatrix} \cos\theta & 0 & -\sin\theta & 0 \\ 0 & 1 & 0 & 0 \\ \sin\theta & 0 & \cos\theta & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix} \cdot \begin{bmatrix} 1 & 0 & 0 & \frac{1}{d} \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}. \tag{1}

Substitute the image coordinates into equation $(1)$ to find the matrix of world coordinates, i.e.,

\boldsymbol{P_W} = \left[ \frac{x\cos\theta - z\sin\theta}{1 - \frac{x}{d}} \quad \frac{y}{1 - \frac{x}{d}} \quad \frac{x\sin\theta + z\cos\theta - \frac{H}{d}}{1 - \frac{x}{d}} \quad 1 \right]. \tag{2}

Let $A = H\cos\theta$ , $B = -d$ , $C = d\sin\theta - \frac{H}{d}$ , $D = \cos\theta$ , and $E = d\sin\theta$ . From the geometric relationship, we know that $x\sin\theta + z\cos\theta - \frac{H}{d} = 0$ . The simplest form of $\boldsymbol{P_W}$ is

\boldsymbol{P_W} = \left[ \frac{A + Bz}{C + Dz} \quad \frac{Ey}{C + Dz} \quad 0 \quad 1 \right]. \tag{3}

Finally, process the image. Since the image being processed is a two-dimensional plane image, the image depth is always 0. According to $(3)$ , we only need to substitute the horizontal and vertical coordinates of the array to obtain the coordinate values in world coordinates, which is the top-down view after IPM.

CodeBlock Loading...

Introduction