SimilarBooks.org
Cover of Markov Decision Processes

Books like this one

Books like Markov Decision Processes

0.0 (0 community ratings)

The past decade has seen considerable theoretical and applied research on Markov decision processes, as well as the growing use of these models in ecology, economics, communications engineering, and other fields where outcomes are uncertain and sequential decision-making processes are needed. A timely response to this increased activity, Martin L. Puterman's new work provides a uniquely up-to-date, unified, and rigorous treatment of the theoretical, computational, and applied research on Markov decision process models. It discusses all major research directions in the field, highlights many significant applications of Markov decision processes models, and explores numerous important topics that have previously been neglected or given cursory coverage in the literature. Markov Decision Processes focuses primarily on infinite horizon discrete time models and models with discrete time spaces while also examining models with arbitrary state spaces, finite horizon models, and continuous-time discrete state models. The book is organized around optimality criteria, using a common framework centered on the optimality (Bellman) equation for presenting results. The results are presented in a "theorem-proof" format and elaborated on through both discussion and examples, including results that are not available in any other book. A two-state Markov decision process model, presented in Chapter 3, is analyzed repeatedly throughout the book and demonstrates many results and algorithms. Markov Decision Processes covers recent research advances in such areas as countable state space models with average reward criterion, constrained models, and models with risk sensitive optimality criteria. It also explores several topics that have received little or no attention in other books, including modified policy iteration, multichain models with average reward criterion, and sensitive optimality. In addition, a Bibliographic Remarks section in each chapter comments on relevant historical references in the book's extensive, up-to-date bibliography...numerous figures illustrate examples, algorithms, results, and computations...a biographical sketch highlights the life and work of A. A. Markov...an afterword discusses partially observed models and other key topics...and appendices examine Markov chains, normed linear spaces, semi-continuous functions, and linear programming. Markov Decision Processes will prove to be invaluable to researchers in operations research, management science, and control theory. Its applied emphasis will serve the needs of researchers in communications and control engineering, economics, statistics, mathematics, computer science, and mathematical ecology. Moreover, its conceptual development from simple to complex models, numerous applications in text and problems, and background coverage of relevant mathematics will make it a highly useful textbook in courses on dynamic programming and stochastic control.

First published: 1994

Discover your next read

Books similar to Markov Decision Processes

Compare books other readers have recommended. Vote to help improve these suggestions.

4 books

Looking for more books like Markov Decision Processes by Martin L. Puterman? This list brings together suggested alternatives and feedback from SimilarBooks readers. Ratings reflect what readers think of each individual book, while the Yes and No votes show whether visitors consider it a good match for this title. Compare the recommendations below and suggest another book to help fellow readers discover their next read.

Cover of Bioinformatics

Bioinformatics

Pierre Baldi

Pierre Baldi and Soren Brunak present the key machine learning approaches and apply them to the computational problems encountered in the analysi...

Is this book similar?

Subjects and themes

Stochastic processes, Linear programming, Markov processes, Statistical decision, Entscheidungstheorie

How our recommendations work

Community ratings describe each book independently. Similarity votes measure whether readers consider a recommendation a good match for this particular title; results can change as more visitors share feedback.

Have a recommendation?

Suggest another book below, or vote Yes or No on any existing recommendation.

Share your suggestion →

More related books

Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto
Approximate Dynamic Programming: Solving the curses of dimensionality by Warren B. Powell
Bellman Equation and Dynamic Programming by Richard Bellman
Markov Decision Processes in Artificial Intelligence by Lars P. Reinhold and Michael J. Pazzani
Decision Processes: An Introduction to Markov Decision Processes by Martin L. Puterman
Planning and Control in Robotics and Automation by Jianing Liu
Dynamic Optimization: The Calculus of Variations and Optimal Control in Economics and Management by M. L. Puterman
Introduction to Stochastic Control by Kostantinos Spiliopoulos
Stochastic Processes: Theory for Applications by Robert G. Gallager

Readers make discovery better

Know a similar book?

Share a recommendation and help other readers discover their next great read.

Please sign in to suggest a book.