autonomous mobile robot that follows a target

As part of the European Project Semester, we developed a software solution for an AMR to follow a person while avoiding static and dynamic obstacles.

Developed in collaboration with three other ESP students and the ENIT Engineering Department.
École Nationale D'Ingénieurs de Tarbes

Overview

As part of the European Project Semester at ENIT (École Nationale d’Ingénieurs de Tarbes), I collaborated with an international team to design and implement a software solution enabling the Pioneer LX mobile robot to autonomously track and follow a human while avoiding static and dynamic obstacles.

The system combined Microsoft Kinect for human detection with the robot’s embedded sensors for navigation and obstacle avoidance, using the potential field method to ensure safe, adaptive movement in real time.

The project delivered a working prototype demonstrating reliable human-following behavior, documented in multiple software versions, and was validated by the client as a showcase of ENIT’s mobile robotics research capabilities.

Project Requirements

The project goal was to develop a software solution enabling the Pioneer LX mobile robot to track and follow a person indoors while avoiding static and dynamic obstacles. Core requirements included:

  • Reliable human tracking using Microsoft Kinect.
  • Safe following distance with <1s reaction time.
  • Obstacle avoidance (static and dynamic) within defined proximity limits.
  • Gesture-based commands for start/stop identification.
  • Delivery of software, user and designer manuals, and demonstration material

Industrial Autonomous Mobile Robot

Robot used: Pioneer LX Mobile Robot from Omron Adept®.

The system was implemented on the Pioneer LX mobile robot (Adept®), a general-purpose differential-drive platform equipped with a laser rangefinder, bumpers, and ultrasonic sensors. It runs an embedded Intel PC with Ubuntu 12.04. The development was done in C++ using Adept’s ARIA library for robot control, with simulation support through MobileSim.

Example simulation with MobileSim software and testing our obstacle avoidance algorithm implementation.

Solution Design and Implementation

Work was split into three parallel branches:

  • Robot Control: low-level motor commands, sensor data handling, ARIA actions.
  • Navigation: algorithms for path planning and obstacle avoidance.
  • Target Tracking: human position detection using Microsoft Kinect and OpenNI/NITE libraries for skeleton tracking
1) The three main development areas. 2) Functional diagram of the solution.
Class UML diagram of the code

Human tracking was implemented using the Microsoft Kinect v1 as the primary sensor. The Kinect provided both RGB and depth data, which was processed with OpenNI and NITE libraries to detect a person’s skeleton and extract their position relative to the robot.

The software established a communication protocol between the Kinect and the Pioneer LX’s embedded controller, allowing the robot to continuously update the target’s coordinates. Gesture-based recognition was added to initialize and stop tracking, ensuring controlled user interaction.

This tracking module provided the input for the navigation system, enabling the robot to follow a designated human in real time with an accuracy of ~10% of the desired following distance

The robot was programmed to identify the defined posed (right image) to start the tracking and to stop tracking when that pose is detected again.
Tracking as input for controlling the robot in the simulator.

The navigation strategy was based on the Potential Field Method, where attractive forces guide the robot toward the human target, while repulsive forces push it away from obstacles detected by onboard sensors. This approach allowed real-time adaptation to both static and limited dynamic obstacles

Representation of potential field method for the robot to decide next move given current sensor readings.

Results

  • Version 1: Basic person-following without obstacle avoidance.
  • Version 2: Human following with static obstacle avoidance, extended to handle some dynamic obstacles.
  • Achieved safe and stable human-following behavior in indoor tests.
  • Delivered a working prototype validated by ENIT as a demonstration of autonomous mobile robotics capabilities
Plots of measurements during live tests of the robot following a target.
Tests in real world and their respective project requirements.

What did we miss?

Several improvements were identified to extend the system’s robustness and performance:

  • State Estimation with Kalman Filter: Incorporating a Kalman filter could improve tracking stability by fusing Kinect depth data with robot odometry, reducing noise and handling temporary occlusions.
  • Mitigating Target Loss: Since the Kinect sensor is fixed to the robot chassis, target loss occurs when the person exits its field of view. A possible enhancement is to mount the sensor on a pan-tilt mechanism or integrate sensor fusion with the laser scanner for wider coverage.
  • Smoother Control: Current motion relies on discrete responses to position updates, which can result in jerky trajectories. Implementing continuous controllers (e.g., PID or fuzzy logic) on top of the navigation layer would enable smoother, more natural robot motion.