Aggregation-disaggregation algorithm for ε²-singularly perturbed limiting average Markov control problems

Finite state and action Markov decision processes (MDPs) are dynamic, stochastic systems controlled by a controller. These models are usually referred to as Markovian control problems (MCPs). The authors consider a singular perturbation of order 2 for a Markov decision process with the limiting average reward criterion. They define a singular perturbation of order 2 in the following sense: it is assumed that the underlying process is composed of n separate irreducible processes, and that a small ε-perturbation is such that it unites these processes into m separate irreducible processes. Then another small ε²-pertubration is such that it unites these latter processes into a single irreducible process. The singular perturbation of order 2 is formulated. The limit MCP that is entirely different from the original unperturbed MDP, which forms an appropriate asymptotic approximation to a whole family of perturbed problems, is given explicitly. Thus, only the single limit MCP needs to be solved. An aggregation-disaggregation algorithm is constructed for solving the limit MCP.