Mobile Robot Path Planning using Q-Learning with Guided Distance

Authors

  • Ee Soong Low
  • Pauline Ong
  • Cheng Yee Low

DOI:

https://doi.org/10.14419/ijet.v7i4.27.22480

Published:

2018-11-30

Keywords:

Guided distance, Mobile robot, Path planning, Q-learning, Reinforcement learning.

Abstract

In path planning for mobile robot, classical Q-learning algorithm requires high iteration counts and longer time taken to achieve convergence. This is due to the beginning stage of classical Q-learning for path planning consists of mostly exploration, involving random direction decision making. This paper proposed the addition of distance aspect into direction decision making in Q-learning. This feature is used to reduce the time taken for the Q-learning to fully converge. In the meanwhile, random direction decision making is added and activated when mobile robot gets trapped in local optima. This strategy enables the mobile robot to escape from local optimal trap. The results show that the time taken for the improved Q-learning with distance guiding to converge is longer than the classical Q-learning. However, the total number of steps used is lower than the classical Q-learning.

 

References

style='mso-bidi-font-size:8.0pt'>

style='mso-spacerun:yes'> ADDIN EN.REFLIST

field-separator'>[1] Hofner C, & Schmidt G, "Path planning and guidance techniques for an autonomous mobile cleaning robot." pp.610-617.

[2] Aasen T, "Mobile cleaning robot for floors," Google Patents, 2005.

[3] Sabattini L, Digani V, & Secchi C (2013), "Technological roadmap to boost the introduction of agvs in industrial applications." pp.203-208.

[4] Song-hua Y (2008), “Trajectory tracking and control of logistics AGV [J],†Modular Machine Tool & Automatic Manufacturing Technique, vol. 6, pp. 022.

[5] Bojarski M, Del Testa D & Dworakowski D (2010), “End to end learning for self-driving cars,†arXiv preprint arXiv:1604.07316, 2016.

[6] S. Thrun, “Toward robotic cars,†Communications of the ACM, vol. 53, no. 4, pp. 99-106.

[7] Häne C, Sattler T, & Pollefeys M, "Obstacle detection for self-driving cars using only monocular cameras and wheel odometry." pp. 5101-5108.

[8] Liu X & Gong D (1992), "A comparative study of a-star algorithms for search and rescue in perfect maze." pp. 24-27.

[9] Dean T, Basye K, & Shewchuk J (1992), “Reinforcement learning for planning and control,†Machine learning methods for planning, pp. 67-92.

[10] Watkins CJ & Dayan P (1992), “Q-learning,†Machine learning, vol. 8, no. 3-4, pp. 279-292.

[11] Xiao J, Michalewicz Z & Zhang L (1997), “Adaptive evolutionary planner/navigator for mobile robots,†IEEE transactions on evolutionary computation, vol. 1, no. 1, pp. 18-28.

[12] Konar A, Chakraborty IG & Singh SJ (2013), “A deterministic improved Q-learning for path planning of a mobile robot,†IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 43, no. 5, pp. 1141-1153.

[13] Konar A, Goswami I & Singh SJ (2013), “A deterministic improved q-learning for path planning of a mobile robot,†IEEE Trans. Systems, Man, and Cybernetics: Systems, vol. 43, no. 5, pp. 1141-1153.

[14] Rakshit P, Konar K, Bhowmik P (2013), “Realization of an adaptive memetic algorithm using differential evolution and q-learning: a case study in multirobot path planning,†IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 43, no. 4, pp. 814-831.

[15] Kim DH, Kim YJ & Kim KC (2000), “Vector field based path planning and petri-net based role selection mechanism with q-learning for the soccer robot system,†Intelligent Automation & Soft Computing, vol. 6, no. 1, pp. 75-87.

[16] Chen C, Li HX, & Dong D (2008), “Hybrid control for robot navigation-a hierarchical q-learning algorithm,†IEEE Robotics & Automation Magazine, vol. 15, no. 2.

[17] Xiao H, Liao L, & Zhou F, "Mobile robot path planning based on q-ann." pp. 2650-2654.

[18] Guo M, Liu Y, and Malec J (2004), “A new Q-Learning algorithm based on the metropolis criterion,†IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 34, no. 5, pp. 2140-2143.

[19] Das P, Behera H, & Panigrahi B (2016), “Intelligent-based multi-robot path planning inspired by improved classical q-learning and improved particle swarm optimization with perturbed Velocity,†Engineering Science and Technology, an International Journal, vol. 19, no. 1, pp. 651-669.

[20] Juang CF, & Lu CM (2009), “Ant colony optimization incorporated with fuzzy q-learning for reinforcement fuzzy control,†IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 39, no. 3, pp. 597-608.

[21] Muñoz P, Barco R, & de la Bandera I (2013), “Optimization of load balancing using fuzzy q-learning for next generation wireless networks,†Expert systems with applications, vol. 40, no. 4, pp. 984-994.

[22] Khajenejad M, Afshinmanesh F, Marandi A, "Intelligent particle swarm optimization using Q-learning." pp. 7-12.

[23] Rakshit P, Konar A & S. Das S, "ABC-TDQL: An adaptive memetic algorithm." pp. 35-42.

[24] Li C, Zhang J, & Li Y, "Application of artificial neural network based on q-learning for mobile robot path planning." pp. 978-982.

[25] Duguleana M, & Mogan G (2016), “Neural networks based reinforcement learning for mobile robots obstacle avoidance,†Expert Systems with Applications, vol. 62, pp. 104-115.

[26] Huang BQ, Cao GY, & Guo M, "Reinforcement learning neural network to the problem of autonomous mobile robot obstacle avoidance." pp. 85-89.

[27] Hwang KS, Tan SW, & Chen CC (2004), “Cooperative strategy based on adaptive q-learning for robot soccer systems,†IEEE Transactions on Fuzzy Systems, vol. 12, no. 4, pp. 569-576.

[28] Park KH, Kim YJ & Kim JH (2001), “Modular Q-learning based multi-agent cooperation for robot soccer,†Robotics and Autonomous Systems, vol. 35, no. 2, pp. 109-122.

[29] Yijing Z, Zheng Z & Xiaoyi Z, "Q learning algorithm based uav path learning and obstacle avoidence approach." pp. 3397-3402.

mso-fareast-font-family:Batang;mso-ansi-language:EN-US;mso-fareast-language:

KO;mso-bidi-language:AR-SA'>

View Full Article:

How to Cite

Soong Low, E., Ong, P., & Yee Low, C. (2018). Mobile Robot Path Planning using Q-Learning with Guided Distance. International Journal of Engineering & Technology, 7(4.27), 57–62. https://doi.org/10.14419/ijet.v7i4.27.22480

License

Authors who publish with this journal agree to the following terms:

    1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution Licensethat allows others to share the work with an acknowledgement of the work''s authorship and initial publication in this journal.

    1. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal''s published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.

  1. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
Received 2018-11-30
Accepted 2018-11-30
Published 2018-11-30