Drone Feedback Controller

Adaptive Q-learning reinforcement learning agent with a nested PID actuation layer for UAV position stabilisation and waypoint navigation.

RoboticsRLControl Systems
# overview

Problem to result

Who had the problem, what I built, the hardest part, and what it achieved.

01 · The problem

The coursework drone had to reach each waypoint within 10 seconds and stop on it; my first controller overshot by 6.8%, against a target of under 1%.

02 · What I built

A drone controller that learns which of 16 tuning profiles to use as it closes in on each waypoint.

03 · The hardest part, and how I solved it

Removing the overshoot without breaking the time limit. The integral term wound up during the climb and pushed the drone past the target, and the learning agent kept picking fast profiles near it. I zeroed the integral term on approach, stopped exploration within 0.5 m, made overshoot costly in the reward, and blocked the slow profiles far from the target so it still arrives in time.

04 · The results

Overshoot cut from 6.8% to 0.20% / 0.29% / 0.37% (X / Y / Z); arrival in 8.6 s against a 10 s limit; control loop at 1000 Hz.

# simulation

Adaptive RL Control Simulation

Real-time Q-learning simulation with PID gain scheduling.

Drone Simulatorv6
T = 0.0 sDrag / Touch · Pinch zoom
Simulation
Fly to Coordinates
X (±10)
Y (±10)
Z (0–8)
Yaw (±π)
Name (optional)
Velocity
Vx0.00m/s
Vy0.00m/s
Vz0.00m/s
X0.000 m
Y0.000 m
Z0.000 m
Dist · Home
0.000 m
Queue
Auto
Home
(0.0, 0.0, 0.0)
Motor RPM
M15000.00
M25000.00
M35000.00
M45000.00
Avg RPM
5000
# videos

Project Walkthroughs

Click any video to play.

Drone Week-1 trail
3:15

Drone Week-1 trail

Initial drone feedback control session and flight testing.

Drone Week-2 trail
Live

Drone Week-2 trail

UAV adaptive trajectory tracking and simulation demonstration.

Drone Week-3 trail
Live

Drone Week-3 trail

Trajectory tracking and stabilisation testing during week 3.

Technical Evaluvation
Live

Technical Evaluvation

UAV adaptive position stabilisation and robust waypoint tracking tests.

# challenges

Problems & Fixes

What went wrong while building it, and how each problem was solved.

Overshoot tuning (25 April)01 / 17

6.8% XY and 3.5% Z overshoot

Cause: Too little damping and too much speed on the approach

How it was fixed

Raised kd_xy by more than 250% and lowered kp. Reduced the speed caps on the mid-range profiles. Added an overshoot penalty to the RL reward.

# github

GitHub

GitHub repositories for this project.

Drone Feedback Controller Repository

Access the complete source code on GitHub.

Quick Start
$ git clone https://github.com/prathapselvakumar/AMR-Assignment-3
$ cd AMR-Assignment-3
$ pip install -r requirements.txt
$ python run.py