A chip you can hold between your fingers that maps the world in real time
For decades, the dream of small autonomous machines that could truly understand their surroundings was constrained by a quiet but stubborn paradox: the more a device needed to know about its environment, the faster it would exhaust the energy keeping it alive. Researchers at MIT have now resolved that tension with Gleanmer, a system-on-a-chip that constructs detailed three-dimensional maps of the world in real time using only six milliwatts of power — less than a single LED draws. By rethinking the very geometry of how space is represented, the team has opened a door through which drones, augmented reality glasses, and autonomous robots may finally pass without sacrificing their independence to the demands of a battery.
- Battery-limited robots and AR headsets have long been trapped between two impossible demands: map the world in detail, or stay powered long enough to matter.
- MIT's Gleanmer chip shatters that constraint, consuming only 2.5 percent of the power required by the best existing 3D mapping solutions — a gap so large it redefines what small devices can attempt.
- The key disruption is geometric: replacing rigid, memory-hungry voxel cubes with flexible Gaussian ellipsoids that stretch to fit curved surfaces, collapsing the data a chip must carry at any moment to just a handful of pixels.
- A single-pass algorithm and on-chip fast memory work in concert to eliminate the costly back-and-forth with distant storage that has always made real-time mapping so power-expensive.
- Tested against live iPhone video and pre-recorded environments, the chip delivered accurate real-time maps — and the team is already moving toward sensor-adjacent processing and AI blueprint reasoning as the next frontier.
A small drone threading through an industrial ventilation shaft and an AR headset mapping a room in real time share a problem that has quietly blocked autonomous machines for years: building a detailed 3D picture of the world demands more power and memory than battery-limited devices can afford. MIT researchers have now built a chip, called Gleanmer, that consumes only about six milliwatts — roughly what a single LED draws — while constructing those maps in real time.
The breakthrough begins with geometry. Conventional 3D mapping systems carve space into voxels, rigid cubes that multiply quickly when representing anything curved. The MIT team, led by professor Vivienne Sze, replaced voxels with Gaussian ellipsoids — flexible, stretchable shapes that can represent a curved pipe or a rounded corner in a single blob where dozens of cubes would otherwise be needed. The map becomes dramatically more compact.
Compressing the representation was only the first step. Graduate students Zih-Sing Fu and Peter Zhi Xuan Li helped develop an algorithm that builds these Gaussians in a single pass through image data, comparing each pixel only to its immediate neighbors rather than to every other pixel in the frame. The chip never needs to hold an entire image in memory at once. When overlapping Gaussians appear as the robot moves, the system merges them directly — no return trip to raw pixel data required.
The hardware was designed around these algorithmic gains. Because the Gaussian maps are so compact, the data being actively processed fits into small, fast memory units built directly onto the chip, eliminating the power-hungry journeys to off-chip storage that traditional systems demand. In testing, Gleanmer used roughly 2.5 percent of the power of the best existing mapping chip, and only about 20 percent of the energy conventional path-planning requires.
The chip was validated against pre-recorded environments and live video from an iPhone, producing detailed real-time maps in both cases. Presented at the IEEE Very Large-Scale Integrated Circuits Symposium, the work is a deliberate exercise in co-design — algorithm and hardware shaped together, each sharpening the other. The team is now exploring sensor-adjacent processing to reduce power further, and whether Gaussians might help AI systems reason about architectural blueprints. For now, Gleanmer stands as evidence that small, battery-powered machines no longer have to choose between understanding their world and surviving long enough to act in it.
A tiny drone navigating the cramped passages of an industrial ventilation system needs to know what's around the next corner. So does an augmented reality headset trying to map a room in real time. Both face the same problem: building a detailed 3D map of their surroundings demands so much power and memory that battery-limited devices can't do it. MIT researchers have now built a chip that changes that equation.
Gleanmer, a new system-on-a-chip developed at MIT, can construct detailed 3D maps of a robot's environment in real time while consuming only about 6 milliwatts of power—roughly what a single LED uses. The advance opens a path for small autonomous devices to navigate complex spaces, avoid obstacles, and plan safe routes without draining their batteries in minutes. A drone inspecting a gas leak in an HVAC system, a pair of AR glasses worn for hours during medical training, or a robot moving through an unfamiliar warehouse could all benefit from this capability.
The breakthrough rests on a fundamental rethinking of how to represent space. Traditional 3D mapping systems store and process images as voxels—rigid, cube-shaped units that tile space like blocks. Representing a curved pipe or a rounded corner requires many voxels, and comparing every pixel in an image to every other pixel to build these representations consumes enormous amounts of memory and power. The MIT team, led by Vivienne Sze, a professor of electrical engineering and computer science, took a different approach. Instead of voxels, they use ellipsoid shapes called Gaussians to represent obstacles. Because these ellipsoids can be stretched and shaped to fit curved geometry, a single elongated blob can represent a region that would require dozens of rigid cubes. The map becomes far more compact.
But compressing the representation was only half the problem. The researchers, including graduate students Zih-Sing Fu and Peter Zhi Xuan Li, also had to make the process of generating these maps efficient enough for a chip to handle. Typically, building Gaussians from depth images requires loading and reprocessing each image multiple times, comparing every pixel to every other pixel. The MIT team invented a technique that generates accurate Gaussians in a single pass through the image data. Rather than comparing each pixel globally, their algorithm assumes nearby pixels belong to the same object and compares each pixel only to its neighbors. This means the chip never needs to store an entire image at once—just a few pixels at a time. As the robot moves and sees the same object from different angles, overlapping Gaussians emerge. Rather than going back to the original pixel data to fuse them, the researchers developed a method to merge overlapping Gaussians directly, keeping all computations on the compact representations rather than the raw images.
The hardware design amplifies these algorithmic gains. Because the Gaussian maps are so compact, the researchers could fit the data the chip is actively processing into small, fast memory units built directly onto the chip, right next to the computational units. This eliminates the power-hungry trips to distant off-chip storage that traditional systems require. The Gaussians needed for the next few frames sit waiting in fast local memory, ready to be accessed instantly. The result is a chip that operates at a fraction of the power of existing solutions. In testing, Gleanmer consumed only about 2.5 percent of the power that the best existing chip for 3D map construction would require. When the chip uses the compact Gaussians to help a robot plan a collision-free path, it needs only about 20 percent of the energy that traditional path-planning would demand.
The researchers tested Gleanmer by reconstructing diverse 3D environments from pre-recorded data and from live video streamed from an iPhone camera. The chip generated detailed, real-time maps in all cases. Sertac Karaman, a professor of aeronautics and astronautics and director of the MIT Laboratory for Information and Decision Systems, sees the implications clearly: "Real-time 3D mapping has been the missing piece for small autonomous systems. A drone inspecting a pipeline or a pair of AR glasses navigating a room both need to understand the space around them—instantly, continuously, and at almost no power cost. Gleanmer makes that possible for the first time in a chip you can hold between your fingers."
The work, presented at the IEEE Very Large-Scale Integrated Circuits Symposium, represents a textbook example of what researchers call co-design—building an algorithm and the hardware to run it in tandem, each optimized for the other. The team plans to push further, moving processing units closer to the sensors that gather environmental data to reduce power even more. They are also exploring whether Gaussians could represent architectural blueprints, helping AI systems reason about complex designs more efficiently. For now, the chip stands as proof that tiny, battery-powered devices no longer have to choose between understanding their world and staying alive.
Citações Notáveis
Real-time 3D mapping has been the missing piece for small autonomous systems. A drone inspecting a pipeline or a pair of AR glasses navigating a room both need to understand the space around them—instantly, continuously, and at almost no power cost.— Sertac Karaman, professor of aeronautics and astronautics
By having dedicated memory that just stores the objects you've seen in the previous few frames, you can access the data much more efficiently.— Zih-Sing Fu, MIT graduate student
A Conversa do Hearth Outra perspectiva sobre a história
Why does a tiny robot need to build a 3D map at all? Can't it just react to what it sees?
A robot reacting frame by frame would crash into things it didn't see coming. A map lets it plan a safe path before it moves. But building that map usually kills the battery.
So the problem is power, not the mapping itself.
Exactly. Every existing system that makes detailed maps burns through energy because it has to store and reprocess images over and over. The researchers found a way to do it in one pass.
By using these Gaussian shapes instead of cubes.
Right. A curved object needs dozens of cubes to approximate. One elongated ellipsoid captures it perfectly. Less data to store, less power to move it around.
But what happens when the robot sees the same object twice from different angles?
You get overlapping Gaussians. Normally you'd go back to the original images to merge them. These researchers merge the Gaussians directly, never touching the raw data again.
That's the real trick, isn't it—keeping everything in the compact form.
Exactly. And they put the memory holding those Gaussians right next to the processor, so it doesn't have to fetch data from far away. That's where most of the power savings come from.
So this is as much about chip design as it is about the algorithm.
It's inseparable. The algorithm had to be designed so the hardware could be designed this way. Neither works without the other.