Further Topics on Discrete-Time Markov Control Processes
Onesimo Hernandez-Lerma and Jean B. Lasserre,
Springer-Verlag, New York, 1999.
ISBN: 0-387-98694-4.
PREFACE
CHAPTER 7. Ergodicity and Poisson Equation
7.1. Introduction
7.2. Weighted Norms and Signed Kernels
A. Weighted norm spaces
B. Signed kernels
C. Contraction maps
7.3. Recurrence Concepts
A. Irreducibility and recurrence
B. Invariant measures
C. Conditions for irreducibility and recurence
D. w-Geometric ergodicity
7.4. Examples on w-Geometric Ergodicity
7.4. Poisson's Equation
A. The multichain case
B. The unichain case
C. Examples
CHAPTER 8. Discounted Dynamic Programming with Weighted Norms
8.1. Introduction
8.2. The Control Model and Control Policies
8.3. The Optimality Equation
A. Assumptions
B. The discounted-cost optimality equation
C. The dynamic programming operator
D. Proof of Theorem 8.3.6.
8.4. Further Analysis of Value Iteration
A. Asymptotic discount optimality
B. Estimates of VI convergence
C. Rolling horizon procedures
D. Forecast horizon and elimination of non-optimal actions
8.5. The Weakly Continuous Case
8.6. Examples
8.7. Further Remarks
CHAPTER 9. The Expected Total Cost Criterion
9.1. Introduction
9.2. Preliminaries
A. Extended real numbers
B. Integrability
9.3. The Expected Total Cost
9.4. Occupation Measures
A. Expected occupation measures
B. The sufficiency problem
9.5. The Optimality Equation
A. The optimality equation
B. Optimality criteria
C. Deterministic stationary policies
9.6. The Transient Case
A. Transient models
B. Optikmality conditions
C. Reduction to deterministic policies
D. The policy iteration algorithm
CHAPTER 10. Undiscounted-Cost Criteria
10.1. Introduction
A. Undiscounted criteria
B. AC-criteria
C. Outline of the chapter
10.2. Preliminaries
A. Assumptions
B. Corollaries
C. Discussion
10.3. From AC-Optimality to Undiscounted Criteria
A. The AC-optimality inequality
B. The AC-optimality equation
C. Uniqueness of the ACOE
D. Bias-optimal policies
E. Undiscounted criteria
10.4. Proof of Theorem 10.3.1
A. Preliminary lemmas
B. Completion of the proof
10.5. Proof of Theorem 10.3.6.
A. Proof of part (a)
B. Proof of part (b)
C. Policy iteration
10.6. Proof of Theorem 10.3.6.
10.7. Proof of Theorem 10.3.10.
10.8. Proof of Theorem 10.3.11.
10.9. Examples
CHAPTER 11. Sample-Path Average Cost
11.1. Introduction
A. Definitions
B. Outline of the Chapter
11.2. Preliminaries
A. Positive Harris Recurrence
B. Limiting average variance
11.3. The w-Geometrically Ergodic Case
A. Optimality in PI_DS
B. Optimality in PI
C. Variance Minimization
D. Proof of Theorem 11.3.5
E. Proof of Theorem 11.3.8
11.4. Strictly Unbounded Costs
11.5. Examples
CHAPTER 12. The Linear Programming Approach
12.1. Introduction
A. Outline of the chapter
12.2. Preliminaries
A. Dual pairs of vector spaces
B. Infinite linear programming,br>
C. Approximations of linear programs
D. Tightness and invariant measures
12.3. Linear Programs for the AC Problem
A. The linear programs
B. Solvability of (P)
C. Absence of duality gap
D. The Farkas alternative
12.4. Approximating sequences and strong duality
A. Minimizing sequences for (P)
B. Maximizing sequences for (P*)
12.5. Finite LP Approximations
A. Aggregation
B. Aggregation-relaxation
C. Aggregation-relaxation-inner approximations
12.6. Proofs of Theorems 12.5.3., 12.5.5., 12.5.7
REFERENCES
ABBREVIATIONS
GLOSSARY OF NOTATION
INDEX