Explainers / How They Work / The AI Stack

The AI Stack: Policies, Imitation, On-Robot vs Cloud

Pillar B: How They Work · Last updated Nov 7, 2025

Bottom Line

  • Behavior policies are AI models that map sensor inputs to robot actions—trained via imitation learning from human demonstrations
  • On-device inference enables fast, private responses but limits model complexity; cloud processing allows bigger models at the cost of latency and privacy
  • Real home robots use hybrid approaches: simple tasks on-device, complex reasoning in the cloud

What Is a Behavior Policy?

A behavior policy is an AI model that tells the robot what to do given what it sees. Think of it as the brain's decision-making layer:

  • Input: Camera images, depth maps, joint positions, force sensor readings
  • Output: Motor commands (move arm 5cm left, grasp with 2N force, rotate wrist 15°)
  • Goal: Complete a task (load dishwasher, fold shirt, fetch remote)

Instead of programming every "if-then" rule manually, modern robots learn policies from data—specifically, by watching humans do tasks.

Imitation Learning: Learning from Humans

Imitation learning (also called learning from demonstrations) is how most home robots are trained:

The Data Collection Loop

  1. Human demonstrates: A teleoperator controls the robot to complete a task (e.g., loading a plate into the dishwasher)
  2. Robot records everything: Camera feeds, joint angles, gripper force, success/failure
  3. Repeat thousands of times: Different homes, different plates, different dishwashers
  4. Train the model: AI learns the pattern: "When I see a plate at this angle, grasp here, move there"
  5. Deploy autonomously: Robot attempts task on its own; when uncertain, calls teleop for help (which adds more training data)
Why this works: Instead of programming "if plate diameter > 25cm then grasp at edge," the model learns from seeing 10,000 plates. It generalizes better to new objects.

Types of Imitation Learning

Behavioral cloning: Simplest form—train a model to mimic human actions given the same observations. Fails when robot encounters situations not in training data.

DAgger (Dataset Aggregation): Robot runs autonomously, but when it deviates from training, human corrects it. These corrections get added to training data, improving the policy over time.

Inverse reinforcement learning: Instead of copying actions, learn the goal/reward. More robust but harder to implement.

The AI Stack: Layers of Processing

Typical Humanoid Robot AI Architecture

Layer 1: Perception Convert raw sensor data (RGB images, depth) into structured understanding (object locations, scene geometry). Uses computer vision models (e.g., segmentation networks).
Layer 2: World Model (Optional) Predict how the environment will change based on robot actions. Enables planning: "If I grasp here, the plate will tilt—bad. Try different grasp point."
Layer 3: Behavior Policy Map current state to action. This is where imitation learning lives. Outputs high-level commands like "grasp plate" or "move to dishwasher."
Layer 4: Motion Planning Convert high-level commands to joint trajectories. Ensures smooth, collision-free motion. Uses optimization algorithms (not learned).
Layer 5: Low-Level Control Execute joint commands while stabilizing balance, adjusting for disturbances. Runs at 100–1000 Hz (very fast).

On-Device vs Cloud: The Tradeoff

Where should the AI models run? This is a critical architectural decision.

On-Device Inference (Edge Computing)

Pros:

  • Fast response time (1–10ms latency)
  • Works offline—no internet required
  • Private—data stays on robot
  • No per-request cloud costs

Cons:

  • Limited compute (robot CPUs/GPUs are power-constrained)
  • Smaller models—less capable of handling edge cases
  • Harder to update (need OTA firmware updates)

Cloud Inference

Pros:

  • Massive compute—can run huge models (GPT-scale reasoning)
  • Easy to update models centrally
  • Aggregate learning across all robots in fleet

Cons:

  • High latency (50–300ms typical; unusable for balance/grasping)
  • Privacy concerns (video/sensor data sent to cloud)
  • Requires internet—fails if WiFi drops
  • Ongoing cloud costs (API fees, server time)

The Hybrid Approach (What Real Robots Use)

Modern home robots split the workload:

  • On-device: Perception, low-level control, simple grasping policies, navigation
  • Cloud: Complex reasoning ("which room did the user say?"), novel object handling, language understanding, fleet-wide learning
  • Teleop (cloud-based human): Fallback when both fail
Example: Picking up a familiar mug = on-device (5ms decision). Understanding "bring me the mug I used yesterday" = cloud reasoning (200ms). Novel object never seen before = teleop assist.

Data: The Hidden Bottleneck

AI quality depends on training data. Here's what makes good training data for home robots:

Data Requirements

  • Diversity: Many homes, layouts, objects, lighting conditions
  • Volume: Thousands of demonstrations per task
  • Quality: Successful completions, not just attempts
  • Edge cases: Failures, unusual objects, cluttered scenes
  • Labeling: Clear task boundaries, object annotations

Why This Is Hard

Unlike training ChatGPT on internet text, robot data requires:

  • Physical access to diverse homes
  • Teleoperators with specialized training
  • Time (each demo takes minutes, not milliseconds)
  • Safety protocols (can't break user's dishes during data collection)

Industry secret: Companies deploying home robots prioritize data collection over immediate reliability. Every pilot deployment = more training data = better future models.

Model Architectures in Practice

Vision Transformers (ViT): Used for perception; process images in patches. Pretrained on millions of internet images, then fine-tuned for home scenes.

Diffusion models: Emerging approach for trajectory generation. Model learns the "shape" of successful grasps/motions and generates new ones by denoising random inputs.

Multimodal models: Combine vision + language (e.g., "put the red cup on the top shelf"). Requires massive pretraining (GPT-4V scale).

What This Means for You

Early robots will improve rapidly: As more homes participate in pilots, data accumulates, and models get better. Your robot in Month 12 will be smarter than Month 1.

Privacy tradeoffs: Cloud-based models mean some sensor data leaves your home. Review privacy policies—can you opt for on-device-only mode?

Offline capability matters: If your home has spotty WiFi, prefer robots with strong on-device inference.

You're part of the training: Every teleop assist, every correction, every task completion improves the next software update.

→ Learn how teleop fits into the AI loop

What to Read Next