Mathematical models and algorithms for sequential decisions under uncertainty

Sequential decisions under uncertainty arise in many fields including energy, healthcare, finance, transportation, and logistics, yet accessible treatments linking foundational theory to computational practice remain scarce. Decision Analytics: Mathematical Models and Algorithms for Sequential Decision-Making, written by Brian T. Denton, a past President of INFORMS, presents a structured progression from core concepts through advanced methods, pairing rigorous mathematics with implementable Python code.

Across ten chapters, Decision Analytics covers decision trees, Monte Carlo simulation, Markov chains, Markov decision processes, partially observable Markov decision processes, and constrained optimization models, including stochastic programs. Dedicated chapters on reinforcement learning and multi-agent learning introduce model-free approaches for finding optimal or near-optimal solutions. The final chapter covers approximate dynamic programming for decision-making at scale. Real-world examples, exercises, and an instructor's solution manual support classroom adoption.

Readers will also find:

• Coverage of artificial intelligence techniques applied to sequential decision-making problems
• Monte Carlo simulation methods used to analyse decision trees, Markov decision processes, and stochastic programming formulations
• Python code examples throughout the text enabling direct implementation and experimentation with each model and algorithm presented
• Practice exercises with solutions and an instructor's manual designed to support both self-study and classroom-based teaching
• A concept-first pedagogical approach that explains foundational principles before demonstrating how they solve applied problems

Designed for undergraduate and graduate students in industrial engineering, operations research, and related STEM disciplines with introductory knowledge of mathematics, probability, and statistics, this book also serves researchers and professionals who require unified treatment of sequential decision-making methods.

Brian T. Denton, PhD, is the Stephen M. Pollock Collegiate Professor of Industrial and Operations Engineering at the University of Michigan. A past President of INFORMS, his research focuses on data-driven sequential decision making and optimization under uncertainty. His work has earned the National Science Foundation Career Award, INFORMS Daniel H. Wagner Prize, and the George E. Kimball medal. He is a Fellow of INFORMS and the Institute of Industrial and Systems Engineers.