Skip to main content

Weeks 1-2: Sensor Systems Overview

Introduction to Robot Sensors​

Robots perceive the world through sensors—devices that convert physical phenomena into digital signals. For humanoid robots, sensor systems provide the foundational data for navigation, manipulation, and interaction.

Core Sensor Types​

1. LIDAR (Light Detection and Ranging)​

Purpose: 3D environment mapping and distance measurement

How it works:

  • Emits laser pulses and measures time-of-flight
  • Generates point clouds representing 3D space
  • Provides accurate distance measurements up to 100+ meters

Applications in Robotics:

  • Obstacle detection and avoidance
  • SLAM (Simultaneous Localization and Mapping)
  • Navigation in unknown environments
  • Object detection and classification

Common LIDAR Types:

  • 2D LIDAR: Scans in a single plane (e.g., Hokuyo, SICK)
  • 3D LIDAR: Full 360° scanning (e.g., Velodyne, Ouster)

Advantages:

  • High accuracy and range
  • Works in various lighting conditions
  • Provides precise distance measurements

Limitations:

  • Expensive
  • Limited performance with reflective or transparent surfaces
  • Computational overhead for processing point clouds

2. Cameras (RGB and Depth)​

Purpose: Visual perception, object recognition, and scene understanding

RGB Cameras​

How it works:

  • Captures color images like human vision
  • Provides rich visual information

Applications:

  • Object detection and recognition
  • Visual servoing for manipulation
  • Human gesture recognition
  • Scene segmentation

Common Types:

  • Monocular cameras (single lens)
  • Stereo cameras (two lenses for depth estimation)

Depth Cameras​

How it works:

  • Provides both color and depth information
  • Uses structured light (Kinect) or time-of-flight (RealSense)

Applications:

  • 3D object reconstruction
  • Hand tracking and gesture recognition
  • Collision avoidance
  • Indoor navigation

Popular Depth Cameras:

  • Intel RealSense series
  • Microsoft Kinect Azure
  • Orbbec Astra
  • ZED cameras

Advantages:

  • Rich visual information
  • Cost-effective compared to LIDAR
  • Enables object recognition using computer vision

Limitations:

  • Sensitive to lighting conditions
  • Limited range compared to LIDAR
  • Requires significant computation for processing

3. IMU (Inertial Measurement Unit)​

Purpose: Measuring acceleration, angular velocity, and orientation

How it works:

  • Accelerometer: Measures linear acceleration in 3 axes
  • Gyroscope: Measures angular velocity (rotation rate)
  • Magnetometer: Measures magnetic field (compass direction)

Applications in Robotics:

  • Balance control for bipedal walking
  • Orientation estimation
  • Fall detection
  • Motion tracking
  • Dead reckoning navigation

Common IMUs:

  • MPU-6050 (6-axis: accelerometer + gyroscope)
  • MPU-9250 (9-axis: adds magnetometer)
  • BNO055 (fusion-enabled IMU)
  • Xsens MTi series (high-precision)

Advantages:

  • High update rate (100-1000 Hz)
  • Compact and lightweight
  • Low cost
  • Works in all conditions

Limitations:

  • Drift over time (especially gyroscopes)
  • Requires sensor fusion with other sensors
  • Sensitive to vibrations

4. Force/Torque Sensors​

Purpose: Measuring contact forces during manipulation

How it works:

  • Strain gauges or capacitive sensors measure deformation
  • Provides 6-axis force/torque measurements

Applications:

  • Compliant robot control
  • Object grasping force control
  • Assembly tasks
  • Human-robot physical interaction

Advantages:

  • Essential for safe interaction
  • Enables delicate manipulation
  • Improves assembly precision

Limitations:

  • Expensive
  • Requires careful calibration
  • Sensitive to temperature changes

Sensor Fusion​

Modern robots don't rely on a single sensor type. Sensor fusion combines data from multiple sensors to create a more accurate and robust perception system:

Example Fusion Strategies​

  1. Visual-Inertial Odometry (VIO):

    • Combines camera and IMU data
    • Provides robust pose estimation
    • Used in drones and mobile robots

  2. LIDAR + Camera Fusion:

    • LIDAR provides accurate distances
    • Camera provides visual features
    • Enables semantic SLAM and object detection

  3. Multi-sensor Localization:

    • GPS + IMU + wheel odometry
    • Provides accurate positioning
    • Handles GPS outages

Sensor Integration in ROS 2​

Throughout this course, you'll work with these sensors in ROS 2:

  • sensor_msgs: Standard message types for sensor data
  • tf2: Transform library for coordinate frames
  • robot_localization: Sensor fusion package
  • perception pipelines: Combining multiple sensors

Learning Objectives​

By the end of this overview, you should be able to:

  • Identify the main sensor types used in robotics
  • Explain how LIDAR, cameras, and IMUs work
  • Understand the strengths and limitations of each sensor
  • Recognize when to use which sensor type
  • Appreciate the importance of sensor fusion

Hands-On in Later Modules​

You'll gain practical experience with these sensors in:

  • Module 2 (Weeks 6-7): Simulating sensors in Gazebo
  • Module 3 (Weeks 8-10): Using NVIDIA Isaac perception stack
  • Module 4 (Weeks 11-13): Integrating sensors for humanoid control

Additional Resources​

Next Steps​

Now that you understand sensor systems, you'll move on to learning ROS 2, where you'll integrate and process sensor data for robot control.