Verifiable Autonomous Systems Using Rational Agents to Provide Assurance about Decisions Made by Machines

How can we provide guarantees of behaviours for autonomous systems such as driverless cars? This tutorial text, for professionals, researchers and graduate students, explains how autonomous systems, from intelligent robots to driverless cars, can be programmed in ways that make them amenable to formal verification. The authors review specific definitions, applications and the unique future potential of autonomous systems, along with their impact on safer decisions and ethical behaviour. Topics discussed include the use of rational cognitive agent programming from the Beliefs-Desires-Intentions paradigm to control autonomous systems and the role model-checking in verifying the properties of this decision-making component. Several case studies concerning both the verification of autonomous systems and extensions to the framework beyond the model-checking of agent decision-makers are included, along with complete tutorials for the use of the freely-available verifiable cognitive agent toolkit Gwendolen, written in Java.

1. Introduction; Part I. Foundations: 2. Autonomous systems architectures; 3. Agent decision maker; 4. Formal agent verification; 5. Verifying autonomous systems; 6. Agent-based autonomous system verification; Part II. Applications: 7. Multi-agent auctions; 8. Autonomous satellite control; 9. Certification of unmanned air systems; 10. Ethical decision making; Part III. Extensions: 11. Compositional verification – widening our view beyond the agent; 12. Runtime verification – recognising abstraction violations; 13. Utilising external model-checkers; Part IV. Concluding Remarks: 14. Verifiable autonomous systems; 15. The future; Appendix A. Gwendolen documentation; Appendix B. AIL toolkit documentation; Appendix C. AJPF documentation; Bibliography; Index.

Dr Louise Dennis is leader of the Autonomy and Verification research group at the University of Manchester and conference coordinator for the ACM Special Interest Group for Artificial Intelligence. She studied Mathematics and Philosophy at the University of Oxford and received her PhD from the University of Edinburgh in using AI techniques to prove mathematical theorems; her interest in the overlap between Mathematics, Philosophy and Artificial Intelligence has continued ever since. Her current research encompasses the programming of autonomous systems, the development of agent programming languages, reasoning about systems and programs via formal mathematical techniques, and the ethical implications of Artificial Intelligence. Beyond the university setting, Dr Dennis is active in public engagement and spends a lot of time taking Lego Robots into schools to introduce robotics programming to children.
Dr Michael Fisher is a Professor of Computer Science at the University of Manchester. He holds a Royal Academy of Engineering Chair in Emerging Technologies and is a fellow of both the British Computer Society and the Institution of Engineering and Technology. He was previously a Professor of Logic and Computation in the Department of Computing & Mathematics at the Manchester Metropolitan University and a Professor of Computer Science at the University of Liverpool. Dr Fisher's research concerns autonomous systems, particularly software engineering, formal verification, safety, responsibility, and trustworthiness. He has been involved in over 200 journal and conference papers and authored the book An Introduction to Practical Formal Methods using Temporal Logic (Wiley) in 2011.