爱学术-智能检索

Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes -- Application to Reliability

模拟与优化分段确定的马尔可夫过程的计算方法

Wiley Benoite de Saporta Francois Dufour

ISBN：9781848218390，出版年：2015，中图分类号：O1 被引 5次

Mark H.A. Davis introduced the Piecewise-Deterministic Markov Process (PDMP) class of stochastic hybrid models in an article in 1984. Today it is used to model a variety of complex systems in the fields of engineering, economics, management sciences, biology, Internet traffic, networks and many more. Yet, despite this, there is very little in the way of literature devoted to the development of numerical methods for PDMDs to solve problems of practical importance, or the computational control of PDMPs. This book therefore presents a collection of mathematical tools that have been recently developed to tackle such problems. It begins by doing so through examples in several application domains such as reliability. The second part is devoted to the study and simulation of expectations of functionals of PDMPs. Finally, the third part introduces the development of numerical techniques for optimal control problems such as stopping and impulse control problems.

Time reversibility Markov renewal process Markov chain Markov model Markov process Markov kernel Piecewise Variable-order Markov model Mathematical optimization Computer science

Markov Chains and Decision Processes for Engineers and Managers

工程师和经理的马尔可夫链和决策过程

CRC Press Sheskin, Theodore J

ISBN：9781420051117，出版年：2016，中图分类号：F2 被引 26次

Markov Chain Structure and Models Historical Note States and Transitions Model of the Weather Random Walks Estimating Transition Probabilities Multiple-Step Transition Probabilities State Probabilities after Multiple Steps Classification of States Markov Chain Structure Markov Chain Models Problems References Regular Markov Chains Steady State Probabilities First Passage to a Target State Problems References Reducible Markov Chains Canonical Form of the Transition Matrix The Fundamental Matrix Passage to a Target State Eventual Passage to a Closed Set Within a Reducible Multichain Limiting Transition Probability Matrix Problems References A Markov Chain with Rewards (MCR) Rewards Undiscounted Rewards Discounted Rewards Problems References A Markov Decision Process (MDP) An Undiscounted MDP A Discounted MDP Problems References Special Topics: State Reduction and Hidden Markov Chains State Reduction An Introduction to Hidden Markov Problems References Index

Markov chain Examples of Markov chains Markov model Markov kernel Markov renewal process Markov chain mixing time Partially observable Markov decision process Variable-order Markov model Mathematical optimization Mathematics