Ros2 Perception

ROS 2 Perception & Computer Vision Instructions (Ubuntu 24.04 LTS & ROS 2 Jazzy)

1. Core Principles & Architecture

• Target OS & ROS Distro: Ubuntu 24.04 LTS & ROS 2 Jazzy Jalisco.
• Perception Pipeline: Handles image streams, OpenCV conversions (cv_bridge), 2D/3D object detections (vision_msgs), depth image processing (depth_image_proc), LiDAR scan projection (laser_geometry), 3D-to-2D scan conversion (pointcloud_to_laserscan), and Point Cloud Library processing (pcl_ros).
• Zero-Hallucination Policy: Always verify C++ and Python class names, message fields, and plugin encodings against official ROS 2 Jazzy documentation.

2. Official Documentation Catalog

A. Master Documentation & Package Indices

• image_transport Package: https://docs.ros.org/en/jazzy/p/image_transport/
• cv_bridge Package: https://docs.ros.org/en/jazzy/p/cv_bridge/
• vision_msgs Package: https://docs.ros.org/en/jazzy/p/vision_msgs/
• depthimageproc Package: https://docs.ros.org/en/jazzy/p/depth_image_proc/
• pointcloudtolaserscan Package: https://docs.ros.org/en/jazzy/p/pointcloud_to_laserscan/
• laser_geometry Package: https://index.ros.org/p/laser_geometry/
• pcl_ros Package: https://docs.ros.org/en/jazzy/p/pcl_ros/

3. Key Concepts & Patterns

A. cv_bridge OpenCV Conversion (C++)

#include 
#include 
#include 
#include 
#include 

void process_image(const sensor_msgs::msg::Image::ConstSharedPtr & msg) {
  cv_bridge::CvImagePtr cv_ptr = cv_bridge::toCvCopy(msg, sensor_msgs::image_encodings::BGR8);
  cv::circle(cv_ptr->image, cv::Point(50, 50), 10, CV_RGB(255, 0, 0), 2);
  sensor_msgs::msg::Image::SharedPtr out_msg = cv_ptr->toImageMsg();
}

B. pcl_ros PointCloud Conversion (C++)

#include 
#include 
#include 
#include 

void filter_cloud(const sensor_msgs::msg::PointCloud2::SharedPtr msg) {
  pcl::PointCloud::Ptr pcl_cloud(new pcl::PointCloud);
  pcl::fromROSMsg(*msg, *pcl_cloud);

  pcl::PointCloud::Ptr filtered(new pcl::PointCloud);
  pcl::VoxelGrid sor;
  sor.setInputCloud(pcl_cloud);
  sor.setLeafSize(0.05f, 0.05f, 0.05f);
  sor.filter(*filtered);

  sensor_msgs::msg::PointCloud2 output_msg;
  pcl::toROSMsg(*filtered, output_msg);
}

4. Symptom -> Root Cause -> Action

| Symptom | Likely root cause | Action | | :--- | :--- | :--- | | Image topic listed but callback never fires | QoS mismatch: camera drivers publish BestEffort, subscriber defaults Reliable | Subscribe with SensorDataQoS; confirm with ros2 topic info -v | | cv_bridge throws encoding exception | Requested encoding doesn't match source (bgr8 vs rgb8, 16UC1 depth) | Use toCvCopy(msg, msg->encoding) (passthrough) or convert explicitly; never assume bgr8 for depth | | Depth values look 1000x off or all ~0 | 16UC1 is millimeters, 32FC1 is meters — unit confusion | Check msg->encoding before scaling; divide 16UC1 by 1000.0 for meters | | Point cloud misaligned with the RGB image | Depth not registered into the color optical frame | Use the depth_registered topic or depth_image_proc register node; verify both frame_ids | | pointcloud_to_laserscan outputs empty scans | min_height/max_height band excludes all points, or target_frame transform missing | Widen the height band around the sensor's actual Z; check TF to target_frame | | Detection boxes drawn at wrong image positions | Processing the rectified topic but projecting with the raw camera matrix (or vice versa) | Pair image_rect with P (projection) matrix, raw image with K; don't mix | | High CPU from image subscribers | Subscribing raw full-resolution images over the network | Use image_transport compressed transport, or throttle/downscale before processing |

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: Leehyunbin0131
• Source: Leehyunbin0131/claude-ros2-skills
• License: MIT

Install and usage instructions live in the source repository linked above.