Informatics and Applications
2019, Volume 13, Issue 1, pp 9-15
STOCHASTIC DIFFERENTIAL SYSTEM OUTPUT CONTROL BY THE QUADRATIC CRITERION. II. DYNAMIC PROGRAMMING EQUATIONS NUMERICAL SOLUTION
- A. V. Bosov
- A. I. Stefanovich
Abstract
The second part of the optimal control problem investigation for the Ito diffusion process and the controlled linear output is presented. Optimal control for output dzt = atyt dt + btzt dt + ctut dt + at dwt of the stochastic differential system dyt = At(yt) dt + >t(yt) dvt and quadratic quality criterion defined by Bellman function having form Vt(y, z) = atz2 + @t(y)z + Yt(y) is determined numerically by an approximate solution to the grid methods of differential equations for the coefficients at, j3t(y), and 7 t(y). A model experiment based on a simple differential presentation for the RTT (Round-Trip Time) parameter of the TCP (Transmission Control Protocol) network protocol is considered in detail. The results of numerical simulation are given and allow one to assess the difficulties in the practical implementation of the optimal solution and define the tasks of further research.
[+] References (10)
- Bosov, A.V., and A.I. Stefanovich. 2018. Upravlenie vykhodom stokhasticheskoy differentsial'noy sistemy po kvadratichnomu kriteriyu. I. Optimal'noe reshenie metodom dinamicheskogo programmirovaniya [Stochastic differential system output control by the quadratic criterion. I. Dynamic programming optimal solution]. In- formatika i ee Primeneniya - Inform. Appl. 12(3):99-106. doi: 10.14357/19922264180314.
- Fleming, W H., and R. W. Rishel. 1975. Deterministic and stochastic optimal control. New York, NY: Springer-Verlag. 222 p.
- Bohacek, S., andB. Rozovskii. 2004. A diffusion model of roundtrip time. Comput. Stat. Data An. 45(1):25-50. doi: 10.1016/S0167-9473(03)00114-2.
- Cox, J. C., J. E. Ingersoll, and S. A. Ross. 1985. A theory of the term structure of interest rates. Econometrica 53:385-407. doi: 10.2307/1911242.
- Bohacek, S. 2003. A stochastic model of TCP and fair video transmission. Proc. IEEE INFOCOM. 1134-1144. doi: 10.1109/INFC0M.2003.1208950.
- Sauliev, V. K. 1960. Integrirovanie uravneniy parabolicheskogo tipa metodom setok [Integration of parabolic equa-tions by the grid method]. Moscow: Fizmatlit. 324 p.
- Borisov, A. V., and G. B. Miller. 2005. Analysis and filtra-tion of special discrete-time Markovprocesses. II. Optimal filtration. Autom. Rem. Contr. 66(7):1125-1136.
- Borisov, A. V., G. B. Miller, and K. V. Semenikhin. 2015. Filtering of the Markov jump process given the observa-tions of multivariate point process. Autom. Rem. Contr. 76(2):219-240.
- Borisov, A., A. Bosov, and G. Miller. 2015. Modeling and monitoring of RTP link on the receiver side. Internet of things, smart spaces, and next generation networks and systems. Eds. S. I. Balandin, S. D. Andreev, and Y Koucheryavy. Lecture notes in computer science ser. Springer, 2015. 9247:229-241. doi: 10.1007/978-3-319- 23126-6_21.
- Borisov, A. V. 2016. Application of optimal filtering methods for on-line of queueing network states. Autom. Rem. Contr. 77(2):277-296.
[+] About this article
Title
STOCHASTIC DIFFERENTIAL SYSTEM OUTPUT CONTROL BY THE QUADRATIC CRITERION. II. DYNAMIC PROGRAMMING EQUATIONS NUMERICAL SOLUTION
Journal
Informatics and Applications
2019, Volume 13, Issue 1, pp 9-15
Cover Date
2019-04-30
DOI
10.14357/19922264190102
Print ISSN
1992-2264
Publisher
Institute of Informatics Problems, Russian Academy of Sciences
Additional Links
Key words
stochastic differential equation; optimal control; dynamic programming; Bellman function; Riccati equation; linear differential equations of parabolic type
Authors
A. V. Bosov  and A. I. Stefanovich
Author Affiliations
Institute of Informatics Problems, Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences, 44-2 Vavilov Str., Moscow 119333, Russian Federation
|