Partially observed Markov decision processes : from filtering to controlled sensing / Vikram Krishnamurthy, University of British Columbia.

Author
Krishnamurthy, V. (Vikram) [Browse]
Format
Book
Language
English
Published/​Created
Cambridge : Cambridge University Press, 2016.
Description
xiii, 476 pages : illustrations ; 26 cm

Availability

Copies in the Library

Location Call Number Status Location Service Notes
Engineering Library - Stacks QA274.7 .K75 2016 Browse related items Request

    Details

    Subject(s)
    Library of Congress genre(s)
    Summary note
    This book covers formulation, algorithms, and structural results of partially observed Markov decision processes, linking theory to real-world applications in controlled sensing.
    Bibliographic references
    Includes bibliographical references (pages 455-470) and index.
    Contents
    Preface; 1. Introduction; Part I. Stochastic Models and Bayesian Filtering: 2. Stochastic state-space models; 3. Optimal filtering; 4. Algorithms for maximum likelihood parameter estimation; 5. Multi-agent sensing: social learning and data incest; Part II. Partially Observed Markov Decision Processes. Models and Algorithms: 6. Fully observed Markov decision processes; 7. Partially observed Markov decision processes (POMDPs); 8. POMDPs in controlled sensing and sensor scheduling; Part III. Partially Observed Markov Decision Processes: 9. Structural results for Markov decision processes; 10. Structural results for optimal filters; 11. Monotonicity of value function for POMPDs; 12. Structural results for stopping time POMPDs; 13. Stopping time POMPDs for quickest change detection; 14. Myopic policy bounds for POMPDs and sensitivity to model parameters; Part IV. Stochastic Approximation and Reinforcement Learning: 15. Stochastic optimization and gradient estimation; 16. Reinforcement learning; 17. Stochastic approximation algorithms: examples; 18. Summary of algorithms for solving POMPDs; Appendix A. Short primer on stochastic simulation; Appendix B. Continuous-time HMM filters; Appendix C. Markov processes; Appendix D. Some limit theorems; Bibliography; Index.
    ISBN
    • 9781107134607 ((hbk.))
    • 1107134609 ((hbk.))
    LCCN
    2015047142
    OCLC
    933719599
    Statement on language in description
    Princeton University Library aims to describe library materials in a manner that is respectful to the individuals and communities who create, use, and are represented in the collections we manage. Read more...
    Other views
    Staff view

    Supplementary Information