Research on Action Strategies and Simulations of DRL and MCTS-based Intelligent Round Game

Journal article


Sun, Yuxiang, Yuan, Bo, Zhang, Yongliang, Zheng, Wanwen, Xia, Qingfeng, Tang, Bojian and Zhou, Xianzhong 2021. Research on Action Strategies and Simulations of DRL and MCTS-based Intelligent Round Game. International Journal of Control, Automation and Systems. https://doi.org/10.1007/s12555-020-0277-0
AuthorsSun, Yuxiang, Yuan, Bo, Zhang, Yongliang, Zheng, Wanwen, Xia, Qingfeng, Tang, Bojian and Zhou, Xianzhong
Abstract

The reinforcement learning problem of complex action control in multiplayer online battlefield games has brought considerable interest in the deep learning field. This problem involves more complex states and action spaces than traditional confrontation games, making it difficult to search for any strategy with human-level performance. This paper presents a deep reinforcement learning model to solve this problem from the perspective of game simulations and algorithm implementation. A reverse reinforcement-learning model based on high-level player training data is established to support downstream algorithms. With less training data, the proposed model is converged quicker, and more consistent with the action strategies of high-level players’ decision-making. Then an intelligent deduction algorithm based on DDQN is developed to achieve a better generalization ability under the guidance of a given reward function. At the game simulation level, this paper constructs Monte Carlo Tree Search Intelligent Decision Model for turn-based antagonistic deduction games to generate next-step actions. Furthermore, a prototype game simulator that combines offline with online functions is implemented to verify the performance of proposed model and algorithm. The experiments show that our proposed approach not only has a better reference value to the antagonistic environment using incomplete information, but also accurate and effective in predicting the return value. Moreover, our work provides a theoretical validation platform and testbed for related research on game AI for deductive games.

KeywordsControl and Systems Engineering; Computer Science Applications
Year2021
JournalInternational Journal of Control, Automation and Systems
PublisherSpringer Science and Business Media LLC
ISSN1598-6446
2005-4092
Digital Object Identifier (DOI)https://doi.org/10.1007/s12555-020-0277-0
Web address (URL)http://hdl.handle.net/10545/625866
https://www.springer.com/tdm
http://creativecommons.org/licenses/by-sa/4.0/
hdl:10545/625866
Publication dates16 Jun 2021
Publication process dates
Deposited09 Jul 2021, 15:47
Accepted28 Nov 2020
Rights

Attribution-ShareAlike 4.0 International

ContributorsNanjing University, China, University of Derby and Army Engineering University, Nanjing, China
File
File Access Level
Open
File
File Access Level
Open
Permalink -

https://repository.derby.ac.uk/item/95018/research-on-action-strategies-and-simulations-of-drl-and-mcts-based-intelligent-round-game

Download files

  • 50
    total views
  • 0
    total downloads
  • 3
    views this month
  • 0
    downloads this month

Export as

Related outputs

Large-scale Data Integration Using Graph Probabilistic Dependencies (GPDs)
Zada, Muhammad Sadiq Hassan, Yuan, Bo, Anjum, Ashiq, Azad, Muhammad Ajmal, Khan, Wajahat Ali and Reiff-Marganiec, Stephan 2020. Large-scale Data Integration Using Graph Probabilistic Dependencies (GPDs). IEEE. https://doi.org/10.1109/bdcat50828.2020.00028
Explaining probabilistic Artificial Intelligence (AI) models by discretizing Deep Neural Networks
Saleem, Rabia, Yuan, Bo, Kurugollu, Fatih and Anjum, Ashiq 2020. Explaining probabilistic Artificial Intelligence (AI) models by discretizing Deep Neural Networks. IEEE. https://doi.org/10.1109/ucc48980.2020.00070
Research and implementation of intelligent decision based on a priori knowledge and DQN algorithms in wargame environment
Sun, Yuxiang, Yuan, Bo, Zhang, Tao, Tang, Bojian, Zheng, Wanwen and Zhou, Xianzhong 2020. Research and implementation of intelligent decision based on a priori knowledge and DQN algorithms in wargame environment. Electronics. 9 (10), p. 1668. https://doi.org/10.3390/electronics9101668
An experimental online judge system based on docker container for learning and teaching assistance
Yibo, Han, Zhang, Zheng, Yuan, Bo, Bi, Haixia, Shahzad, Mohammad Nasir and Liu, Lu 2020. An experimental online judge system based on docker container for learning and teaching assistance. IEEE. https://doi.org/10.1109/smartworld-uic-atc-scalcom-iop-sci.2019.00264
A privacy-preserved probabilistic routing index model for decentralised online social networks
Yuan, Bo, Gu, Jiayan and Liu, Lu 2020. A privacy-preserved probabilistic routing index model for decentralised online social networks. IEEE. https://doi.org/10.1109/SmartWorld-UIC-ATC-SCALCOM-IOP-SCI.2019.00305
A survey of interpretability of machine learning in accelerator-based high energy physics
Turvill, Danielle, Barnby, Lee, Yuan, Bo and Zahir, Ali 2020. A survey of interpretability of machine learning in accelerator-based high energy physics. IEEE. https://doi.org/10.1109/bdcat50828.2020.00025
Exploring network embedding for efficient message routing in opportunistic mobile social networks
Yuan, Bo, Anjum, Ashiq, Panneerselvam, J. and Liu, Lu 2020. Exploring network embedding for efficient message routing in opportunistic mobile social networks. IEEE. https://doi.org/10.1109/ICDMW.2019.00077
An efficient evolutionary user interest community discovery model in dynamic social networks for internet of people
Jiang, Liang, Shi, Leilei, Lu, Liu, Yao, Jingjing, Yuan, Bo and Zheng, Yongjun 2019. An efficient evolutionary user interest community discovery model in dynamic social networks for internet of people. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2019.2893625
A GRU-based prediction framework for intelligent resource management at cloud data centres in the age of 5G
Lu, Yao, Liu, Lu, Panneerselvam, J., Yuan, Bo, Gu, Jiayan and Antonopoulos, Nick 2019. A GRU-based prediction framework for intelligent resource management at cloud data centres in the age of 5G. IEEE Transactions on Cognitive Communications and Networking. 6 (2), pp. 486-498. https://doi.org/10.1109/tccn.2019.2954388
An inductive content-augmented network embedding model for edge artificial intelligence
Yuan, Bo, Panneerselvam, J., Liu, Lu, Antonopoulos, Nick and Lu, Yao 2019. An inductive content-augmented network embedding model for edge artificial intelligence. IEEE Transactions on Industrial Informatics. 15 (7), pp. 4295-4305. https://doi.org/10.1109/TII.2019.2902877
Efficient service discovery in decentralized online social networks.
Yuan, Bo, Liu, Lu and Antonopoulos, Nikolaos 2017. Efficient service discovery in decentralized online social networks. Future Generation Computer Systems. https://doi.org/10.1016/j.future.2017.04.022
A novel service discovery model for decentralised online social networks.
Yuan, Bo 2018. A novel service discovery model for decentralised online social networks. PhD Thesis https://doi.org/10.48773/93w19
Mobilouds: An energy efficient MCC collaborative framework with extended mobile participation for next generation networks
Panneerselvam, J., Hardy, J., Liu, Lu, Yuan, Bo and Antonopoulos, Nikolaos 2017. Mobilouds: An energy efficient MCC collaborative framework with extended mobile participation for next generation networks. IEEE Access. https://doi.org/10.1109/ACCESS.2016.2602321
An efficient algorithm for partially matched services in internet of services
Ahmed, Mariwan, Liu, Lu, Hardy, J., Yuan, Bo and Antonopoulos, Nikolaos 2016. An efficient algorithm for partially matched services in internet of services. Personal and Ubiquitous Computing. https://doi.org/10.1007/s00779-016-0917-9