Description

Logically-Constrained Reinforcement Learning (LCRL) is a model-free reinforcement learning framework to synthesise policies for unknown, continuous-state Markov Decision Processes (MDPs) under a given Linear Temporal Logic (LTL) property. LCRL automatically shapes a synchronous reward function on-the-fly. This enables any off-the-shelf RL algorithm to synthesise policies that yield traces which probabilistically satisfy the LTL property. This probability is calculated in parallel with the learning process when the MDP state space is finite. LCRL produces policies that are certified with respect to the LTL property.

Code Repository

LCRL code can be found here: github.com/grockious/lcrl

Publications

• • Hasanbeig, M. , Kroening, D. and Abate, A., "Deep Reinforcement Learning with Temporal Logics", International Conference on Formal Modeling and Analysis of Timed Systems, 2020. [Bib ] [PDF ]
@inproceedings{hasanbeig2020deep, title={Deep Reinforcement Learning with Temporal Logics}, author={Hasanbeig, Mohammadhosein and Kroening, Daniel and Abate, Alessandro}, booktitle={International Conference on Formal Modeling and Analysis of Timed Systems}, pages={1--22}, year={2020}, organization={Springer} }
• • Hasanbeig, M. , Abate, A. and Kroening, D., "Cautious Reinforcement Learning with Logical Constraints", International Conference on Autonomous Agents and Multi-agent Systems, 2020. [Bib ] [PDF ]
@article{cautiousRL, title={Cautious Reinforcement Learning with Logical Constraints}, author={Hasanbeig, Mohammadhosein and Abate, Alessandro and Kroening, Daniel}, booktitle={Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems}, year={2020}, organization={International Foundation for Autonomous Agents and Multiagent Systems} }
• • Hasanbeig, M. , Jeppu, N. Y., Abate, A., Melham, T., Kroening, D., "DeepSynth: Program Synthesis for Automatic Task Segmentation in Deep Reinforcement Learning", CoRR abs/1911.10244, 2019. [Bib ] [PDF ]
@article{deepsynth, title={DeepSynth: Program Synthesis for Automatic Task Segmentation in Deep Reinforcement Learning}, author={Hasanbeig, Mohammadhosein and Yogananda Jeppu, Natasha and Abate, Alessandro and Melham, Tom and Kroening, Daniel}, journal={arXiv preprint arXiv:1911.10244}, year={2019} }
• • Hasanbeig, M. , Kantaros, Y., Abate, A., Kroening, D., Pappas, G. J., and Lee, I., "Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guarantees", IEEE Conference on Decision and Control, 2019. [Bib ] [PDF ]
@article{plmdp, title={Reinforcement Learning for Temporal Logic Control Synthesis with Probabilistic Satisfaction Guarantees}, author={Hasanbeig, Mohammadhosein and Kantaros, Yiannis and Abate, Alessandro and Kroening, Daniel and Pappas, George J. and Lee, Insup}, journal={arXiv preprint arXiv:1909.05304}, year={2019} }
• • Lim Zun Yuan, Hasanbeig, M. , Abate, A. and Kroening, D., "Modular Deep Reinforcement Learning with Temporal Logic Specifications", CoRR abs/1909.11591, 2019. [Bib ] [PDF ]
@article{deeplcrl, title={Modular Deep Reinforcement Learning with Temporal Logic Specifications}, author={Yuan, Lim Zun and Hasanbeig, Mohammadhosein and Abate, Alessandro and Kroening, Daniel}, journal={arXiv preprint arXiv:1909.11591}, year={2019} }
• • Hasanbeig, M. , Abate, A. and Kroening, D., "Certified Reinforcement Learning with Logic Guidance", CoRR abs/1902.00778, 2019. [Bib ] [PDF ]
@article{lcrl_j, title={Certified reinforcement learning with logic guidance}, author={Hasanbeig, Mohammadhosein and Abate, Alessandro and Kroening, Daniel}, journal={arXiv preprint arXiv:1902.00778}, year={2019} }
• • Hasanbeig, M. , Abate, A. and Kroening, D., "Logically-Constrained Neural Fitted Q-Iteration", International Conference on Autonomous Agents and Multi-agent Systems, 2019. [Bib ] [PDF ]
@inproceedings{lcnfq, title={Logically-Constrained Neural Fitted Q-Iteration}, author={Hasanbeig, Mohammadhosein and Abate, Alessandro and Kroening, Daniel}, booktitle={Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems}, pages={2012--2014}, year={2019}, organization={International Foundation for Autonomous Agents and Multiagent Systems} }
• • Hasanbeig, M. , Abate, A. and Kroening, D., "Logically-Constrained Reinforcement Learning", CoRR abs/1801.08099, 2018. [Bib ] [PDF ]
@article{lcrl, title={Logically-constrained reinforcement learning}, author={Hasanbeig, Mohammadhosein and Abate, Alessandro and Kroening, Daniel}, journal={arXiv preprint arXiv:1801.08099}, year={2018} }