Quantifying capability gaps via information relaxation and deep reinforcement learning in infinite-horizon Markov decision processes: A military air battle management application.
Saved in:
| Title: | Quantifying capability gaps via information relaxation and deep reinforcement learning in infinite-horizon Markov decision processes: A military air battle management application. |
|---|---|
| Authors: | Liles IV, Joseph M.1 (AUTHOR) joseph.liles@us.af.mil, Robbins, Matthew J.1 (AUTHOR), Lunday, Brian J.1 (AUTHOR) |
| Source: | Journal of the Operational Research Society. May2026, Vol. 77 Issue 5, p1322-1337. 16p. |
| Subjects: | Markov processes, Air warfare, Stochastic control theory, Reinforcement learning, Mathematical optimization |
| Abstract: | This paper presents a novel application of information relaxation techniques to quantify upper bounds on solution quality in a complex, stochastic, and dynamic assignment problem in military air battle management. Information relaxation refers to relaxing the non-anticipativity constraints in a sequential decision-making problem that require a decision-maker to act only on currently available information. We introduce a temporal event horizon—–an adjustable window into future stochastic outcomes—–to explore the marginal value of information in shaping decision policies. Whereas previous work has investigated information relaxation with regard to problems that can be solved more easily under a deterministic relaxation, we demonstrate a methodology for applying the approach to a continuous-time, continuous-space problem that remains computationally challenging even after relaxation. We formulate the problem as a discounted, infinite-horizon Markov decision process and solve it by employing a deep neural network-based approximate policy iteration algorithm in concert with several designed computational experiments. We demonstrate how a multidimensional sensitivity analysis of the event horizon and other problem features helps quantify potential improvements to decision policy effectiveness resulting from either a change to tactics or a modification to capabilities. Our findings provide a methodology for objective, data-driven insights that can augment traditionally subjective capability gap analysis to guide decision-making and establish more effective requirements for acquisition programs. [ABSTRACT FROM AUTHOR] |
| Copyright of Journal of the Operational Research Society is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Database: | Engineering Source |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: egs DbLabel: Engineering Source An: 193084159 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Quantifying capability gaps via information relaxation and deep reinforcement learning in infinite-horizon Markov decision processes: A military air battle management application. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Liles+IV%2C+Joseph+M%2E%22">Liles IV, Joseph M.</searchLink><relatesTo>1</relatesTo> (AUTHOR)<i> joseph.liles@us.af.mil</i><br /><searchLink fieldCode="AR" term="%22Robbins%2C+Matthew+J%2E%22">Robbins, Matthew J.</searchLink><relatesTo>1</relatesTo> (AUTHOR)<br /><searchLink fieldCode="AR" term="%22Lunday%2C+Brian+J%2E%22">Lunday, Brian J.</searchLink><relatesTo>1</relatesTo> (AUTHOR) – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%22Journal+of+the+Operational+Research+Society%22">Journal of the Operational Research Society</searchLink>. May2026, Vol. 77 Issue 5, p1322-1337. 16p. – Name: Subject Label: Subjects Group: Su Data: <searchLink fieldCode="DE" term="%22Markov+processes%22">Markov processes</searchLink><br /><searchLink fieldCode="DE" term="%22Air+warfare%22">Air warfare</searchLink><br /><searchLink fieldCode="DE" term="%22Stochastic+control+theory%22">Stochastic control theory</searchLink><br /><searchLink fieldCode="DE" term="%22Reinforcement+learning%22">Reinforcement learning</searchLink><br /><searchLink fieldCode="DE" term="%22Mathematical+optimization%22">Mathematical optimization</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: This paper presents a novel application of information relaxation techniques to quantify upper bounds on solution quality in a complex, stochastic, and dynamic assignment problem in military air battle management. Information relaxation refers to relaxing the non-anticipativity constraints in a sequential decision-making problem that require a decision-maker to act only on currently available information. We introduce a temporal event horizon—–an adjustable window into future stochastic outcomes—–to explore the marginal value of information in shaping decision policies. Whereas previous work has investigated information relaxation with regard to problems that can be solved more easily under a deterministic relaxation, we demonstrate a methodology for applying the approach to a continuous-time, continuous-space problem that remains computationally challenging even after relaxation. We formulate the problem as a discounted, infinite-horizon Markov decision process and solve it by employing a deep neural network-based approximate policy iteration algorithm in concert with several designed computational experiments. We demonstrate how a multidimensional sensitivity analysis of the event horizon and other problem features helps quantify potential improvements to decision policy effectiveness resulting from either a change to tactics or a modification to capabilities. Our findings provide a methodology for objective, data-driven insights that can augment traditionally subjective capability gap analysis to guide decision-making and establish more effective requirements for acquisition programs. [ABSTRACT FROM AUTHOR] – Name: AbstractSuppliedCopyright Label: Group: Ab Data: <i>Copyright of Journal of the Operational Research Society is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=egs&AN=193084159 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1080/01605682.2025.2528915 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 16 StartPage: 1322 Subjects: – SubjectFull: Markov processes Type: general – SubjectFull: Air warfare Type: general – SubjectFull: Stochastic control theory Type: general – SubjectFull: Reinforcement learning Type: general – SubjectFull: Mathematical optimization Type: general Titles: – TitleFull: Quantifying capability gaps via information relaxation and deep reinforcement learning in infinite-horizon Markov decision processes: A military air battle management application. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Liles IV, Joseph M. – PersonEntity: Name: NameFull: Robbins, Matthew J. – PersonEntity: Name: NameFull: Lunday, Brian J. IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 05 Text: May2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 01605682 Numbering: – Type: volume Value: 77 – Type: issue Value: 5 Titles: – TitleFull: Journal of the Operational Research Society Type: main |
| ResultId | 1 |
