Informatics and Applications
2022, Volume 16, Issue 2, pp 19-26
LINEAR OUTPUT CONTROL OF MARKOV CHAIN BY SQUARE CRITERION. COMPLETE INFORMATION CASE
Abstract
The problem of optimal control of the linear output of a stochastic differential system, formed by an additive jumping input, was solved. The goal of optimization is set by a quadratic criterion of a special type which allows one to formalize the tasks of tracking an abruptly changing target and stabilizing the system near the directions determined by the input. The problem is solved under the assumption that there is complete information,
i. e., the known state of the input Markov chain. This statement complements the previously obtained solution of the problem with incomplete information, in which the control and estimation problems are separated, provided by the optimal in this case Wonham filter. The result obtained in the article, in addition to its independent significance, also provides a reference solution for analyzing the quality of control under conditions of indirect observations.
The solution of the problem under consideration, as in the statement with incomplete information, is provided by the direct application of the dynamic programming method. The Bellman's equation is refined for a given input model - a martingale representation of the chain and a range of values limited by unit coordinate vectors are used. A numerical experiment was carried out, the results of which illustrate the efficiency of the obtained control algorithms in both settings, with complete information and indirect observations.
[+] References (12)
- Bosov, A. V. 2021. Upravlenie lineynym vykhodom markovskoy tsepi po kvadratichnomu kriteriyu [Linear output control of Markov chains by the quadratic criterion]. Informatika i ee Primeneniya - Inform. Appl. 15(2):3-11.
- Elliott, R. J., L. Aggoun, and J. B. Moore. 1995. Hidden Markov models: Estimation and control. New York, NY: Springer-Verlag. 382 p.
- Bosov, A. V. 2021. The problem of controlling the linear output of a nonlinear uncontrollable stochastic differential system by the square criterion. J. Comput. Sys. Sc. Int. 60(5):719-739.
- Bosov, A.V. 2021. O nekotorykh chastnykh sluchayakh v zadache upravleniya vykhodom stokhasticheskoy differentsial'noy sistemy po kvadratichnomu kriteriyu [On
some special cases in the problem of stochastic differential system output control by the quadratic criterion]. Informatika i ee Primeneniya - Inform. Appl. 15(1):11-17.
- Fleming, W H., and R. W. Rishel. 1975. Deterministic and stochastic optimal control. New York, NY: Springer-Verlag. 222 p.
- Bertsekas, D. P. 2017. Dynamic programming and optimal control. Cambridge, MA: Athena Scientific. Vol. 1. 576 p.
- Benes, V. 1991. Quadratic approximation by linear systems controlled from partial observations. Stochastic analysis. Eds. E. Mayer-Wolf, E. Merzbach, and A. Shwartz. 39-50.
- Helmes, K., and R. Rishel. 1992. The solution of a partially observed stochastic optimal control problem in terms of predictedmiss. IEEE T. Automat. Contr. 37(9):1462-1464.
- Rishel, R. 1994. A strong separation principle for stochas-tic control systems driven by a hidden Markov model. SIAM J. Control Optim. 32(4):1008-1020.
- Liptser, R. S., and A. N. Shiryaev. 2001. Statistics of random processes II. Applications. Berlin: Springer-Verlag. 402 p.
- Liptser, R., and A. Shiryaev. 1989. Theory of martingales. Dortrecht: Springer. 792 p.
- Athans, M., and P. L. Falb. 2007. Optimal control: An introduction to the theory and its applications. New York, NY Dover Publications. 879 p.
[+] About this article
Title
LINEAR OUTPUT CONTROL OF MARKOV CHAIN BY SQUARE CRITERION. COMPLETE INFORMATION CASE
Journal
Informatics and Applications
2022, Volume 16, Issue 2, pp 19-26
Cover Date
2022-07-25
DOI
10.14357/19922264220203
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
Markov jump process; linear stochastic differential system; optimal control; quadratic criterion; dynamic programming
Authors
A. V. Bosov 
Author Affiliations
Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44-2 Vavilov Str., Moscow 119333, Russian Federation
|