How Robots Understand Places

Strip away the hardware and every autonomous vehicle is solving the same four problems. Whether it is a campus shuttle, a warehouse tug, or a security robot roaming Harbor Yard at 2 a.m., the work breaks down the same way. Get these four questions straight and the rest of HD mapping stops feeling like a pile of formats. Each format suddenly has an obvious job.

The four questions

A robot moving through a place has to answer, over and over, in real time:

1. Where am I? (Localization)
2. What is around me? (Perception)
3. Where can I go? (Planning and routing)
4. What am I allowed to do? (Rules and access)

Notice the boundary between the last two. The third question is about possibility, the fourth about permission: can versus may. A flatbed parked across half the loop lane is a path the tug may take but cannot. The wrong-way end of the one-way loop is a path it can take but may not. Most of the map’s complexity lives on that boundary, because the planner needs both answers at once: a path that is possible but not permitted is not a plan.

That is the whole game. Four questions, answered fast enough and often enough that the vehicle stays safe and useful.¹

Why this model beats memorizing formats

A beginner tries to learn HD mapping by collecting formats. OSM, Lanelet2, OpenDRIVE, USD. The names blur together, none of them stick, and it feels like memorizing vocabulary without a grammar.

The four questions are the grammar.

The map’s job, restated

An HD map is the shared answer to all four questions, written down once so the robot does not have to re-derive them on the fly.

• Localization compares the map to live sensors to answer where am I?
• Perception layers live sensing on top of the map to answer what is around me?
• Planning reads the map’s connectivity to answer where can I go?
• The rules baked into the map answer what am I allowed to do?

Where the answers live

Each question maps to a part of this book, and to a layer of the map you will build.

• Where am I? and what is around me? are sensing questions. The data they read is captured in Part III, Scanning Your Site.
• Where can I go? and what am I allowed to do? are representation questions. They are encoded in Part V, Rules of the Road.
• Part VII is where the map meets the robot: the vehicle answers all four live, reading your map against its sensors as it drives.

Coordinates (Part II) are the address system underneath all four. Without a shared way to say “where,” none of the questions can be answered.

A note on scope

This book focuses on controlled environments: campuses, industrial yards, depots like Harbor Yard, the places where map-centric autonomy works reliably today. The same four questions apply on a public highway. The difficulty on a highway is the world, not the model. Controlled environments are where you can actually ship.

Try this

Before you go on, spend five minutes with a machine you already trust to move on its own, even a simple one: a robot vacuum, a moving walkway, an elevator.

[5 min] Watch it work and name, out loud, which of the four questions it is solving at each moment. Where does it think it is? What is it sensing? Where is it deciding to go? Which rule just stopped it? You will find that even a machine with no map at all is quietly answering all four, badly. The rest of this book is about answering them well.

What comes next

Part II starts the foundations: coordinates, the address system underneath everything. Because before a robot can answer “where am I,” you first have to decide what where even means.

Footnotes

1. “Fast enough and often enough” is doing real work in that sentence. Localization loops often run at 10 to 100 Hz, and planning fires whenever the world changes. We get into the numbers in Part VII. Those rates quietly depend on Part II: a projected coordinate system puts the whole site in a local frame of small, meter-valued numbers, so each loop does its arithmetic with plain subtraction instead of trigonometry against the curvature of the earth. ↩