/pleɪn ˈtræk.ɪŋ/

Plane tracking

noun · AR technology

What it is

Before an app can make a reference image "sit" on your paper, it has to find the paper. Plane tracking is how it does that. As the camera moves, the app spots clusters of feature points that all lie on the same flat surface, recognises them as a plane, and works out where that plane is in space and how big it is. Once found, the plane is tracked — kept in place — so anything anchored to it stays put even as you shift the phone.

This is the foundation the rest of AR tracing is built on. Plane tracking finds and holds the surface; homography then warps the reference image so it lies flat on that plane; and occlusion can make real objects correctly cover the overlay. Together they are the working parts of augmented reality on a phone. A flat, evenly lit sheet with a little visible texture tracks best; a blank glossy surface in dim light gives the app less to lock onto.

Etymology

"Plane" comes from the Latin planum, "a flat surface"; "tracking" here means following something continuously over time. The specific engineering term "plane detection" was popularised by the two frameworks that brought markerless AR to ordinary phones — Apple's ARKit and Google's ARCore, both released in 2017 — whose developer documentation describes detecting horizontal and vertical planes from the camera feed.^1,2

Examples in use

A shopping app uses plane tracking to place a virtual rug flat on your real floor. A game drops characters onto your kitchen table. For artists, plane tracking is what lets an app treat your drawing surface as a fixed canvas, so a scaled-up reference holds still while you sketch during AR tracing. The free web tool has no camera and no plane tracking; it simply draws overlays on an uploaded image in the browser.