Reinforcement Learning Approach to Autonomous PID Tuning

doi:10.23919/ACC53348.2022.9867687

Paper

Reinforcement Learning Approach to Autonomous PID Tuning

Published Mar 1, 2022 · Oguzhan Dogru, Kirubakaran Velswamy, Fadi Ibrahim

2022 American Control Conference (ACC)

UNKNOWN SJR score

73

Citations

1

Influential Citations

Full textSemantic Scholar

Abstract

Many industrial processes utilize proportional-integral-derivative (PID) controllers due to their practicality and often satisfactory performance. The proper controller parameters depend highly on the operational conditions and process uncertainties. This dependence brings the necessity of frequent tuning for real-time control problems due to process drifts and operational condition changes. This study combines the recent developments in computer sciences and control theory to address the tuning problem. It formulates the PID tuning problem as a reinforcement learning task with constraints. The proposed scheme identifies an initial approximate step-response model and lets the agent learn dynamics off-line from the model with minimal effort. After achieving a satisfactory training performance on the model, the agent is fine-tuned on-line on the actual process to adapt to the real dynamics, thereby minimizing the training time on the real process and avoiding unnecessary wear, which can be beneficial for industrial applications. This sample efficient method is applied to a pilot-scale multi-modal tank system. The performance of the method is demonstrated by setpoint tracking and disturbance regulatory experiments.

Highly Cited

Study Snapshot

This study proposes a reinforcement learning approach for autonomous PID tuning, minimizing training time and reducing wear in industrial processes.

PopulationOlder adults (50-71 years)

Sample size24

MethodsObservational

OutcomesBody Mass Index projections

ResultsSocial networks mitigate obesity in older groups.

Sign up to use Study Snapshot

Consensus is limited without an account. Create an account or sign in to get more searches and use the Study Snapshot.

Create an account

Paper

Reinforcement Learning Approach to Autonomous PID Tuning

Sign up to use Study Snapshot

References

Reinforcement Learning With Constrained Uncertain Reward Function Through Particle Filtering

The proposed particle filter improves deep reinforcement learning performance by reducing the impact of noise in reward functions, enhancing process efficiency in industrial separation processes.

Deep Reinforcement Learning with Shallow Controllers: An Experimental Application to PID Tuning

Deep reinforcement learning can be successfully implemented on real physical systems using a PID controller, offering simplicity, ease of implementation, and a well-tuned control strategy.

Actor–Critic Reinforcement Learning and Application in Developing Computer-Vision-Based Interface Tracking

Actor-Critic reinforcement learning effectively tracks interfaces between liquids in the oil sands industry, requiring fewer parameters and being robust to environmental uncertainties.

Online reinforcement learning for a continuous space system with experimental validation

This study successfully implements reinforcement learning for real-time control of a multivariable, multi-modal, hybrid three-tank process using a socket connection and various learning approaches.

Iterative Feedback Tuning of Cascade Control of Two-Inertia System

The proposed iterative feedback tuning method optimizes cascade control in two-inertia systems under various plant conditions, achieving high-performance robust control.

Citations

Theoretical and experimental study of an automated kill procedure based on a neural network-PID controller

PIDNN controllers can effectively control casing pressure in automated well kill procedures, reducing control time and increasing safety.

Efficient safe learning for controller tuning with experimental validation

The proposed safe learning algorithm for controller tuning achieves 30% better tracking at sub-micrometer precision and reduces computational cost by seven times compared to exhaustive grid search methods.

Leveraging Deep Reinforcement Learning for Effective PI Controller Tuning in Industrial Water Tank Systems

The Deep Deterministic Policy Gradient (DDPG) algorithm effectively tunes PI controller parameters in industrial water tank systems, eliminating manual tuning and ensuring robust performance under uncertainties and disturbances.

Deep Deterministic Policy Gradient Reinforcement Learning Based Temperature Control of a Fermentation Bioreactor for Ethanol Production

Deep deterministic policy gradient reinforcement learning (DDPG) effectively controls temperature in fermentation bioreactors for ethanol production, outperforming deep Q-learning network (DQN) in terms of integral squared error.

Optimizing olefin purification: An artificial intelligence-based process-conscious PI controller tuning for double dividing wall column distillation

The Deep Deterministic Policy Gradient (DDPG) method significantly enhances control stability and accuracy in light olefin separation, reducing error by factors of 11.9, 2.3, and 1.6 compared to traditional methods.