Mobile Robot Path Planning using Q-Learning with Guided Distance
Keywords:Guided distance, Mobile robot, Path planning, Q-learning, Reinforcement learning.
In path planning for mobile robot, classical Q-learning algorithm requires high iteration counts and longer time taken to achieve convergence. This is due to the beginning stage of classical Q-learning for path planning consists of mostly exploration, involving random direction decision making. This paper proposed the addition of distance aspect into direction decision making in Q-learning. This feature is used to reduce the time taken for the Q-learning to fully converge. In the meanwhile, random direction decision making is added and activated when mobile robot gets trapped in local optima. This strategy enables the mobile robot to escape from local optimal trap. The results show that the time taken for the improved Q-learning with distance guiding to converge is longer than the classical Q-learning. However, the total number of steps used is lower than the classical Q-learning.
style='mso-spacerun:yes'> ADDIN EN.REFLIST
field-separator'> Hofner C, & Schmidt G, "Path planning and guidance techniques for an autonomous mobile cleaning robot." pp.610-617.
 Aasen T, "Mobile cleaning robot for floors," Google Patents, 2005.
 Sabattini L, Digani V, & Secchi C (2013), "Technological roadmap to boost the introduction of agvs in industrial applications." pp.203-208.
 Song-hua Y (2008), â€œTrajectory tracking and control of logistics AGV [J],â€ Modular Machine Tool & Automatic Manufacturing Technique, vol. 6, pp. 022.
 Bojarski M, Del Testa D & Dworakowski D (2010), â€œEnd to end learning for self-driving cars,â€ arXiv preprint arXiv:1604.07316, 2016.
 S. Thrun, â€œToward robotic cars,â€ Communications of the ACM, vol. 53, no. 4, pp. 99-106.
 HÃ¤ne C, Sattler T, & Pollefeys M, "Obstacle detection for self-driving cars using only monocular cameras and wheel odometry." pp. 5101-5108.
 Liu X & Gong D (1992), "A comparative study of a-star algorithms for search and rescue in perfect maze." pp. 24-27.
 Dean T, Basye K, & Shewchuk J (1992), â€œReinforcement learning for planning and control,â€ Machine learning methods for planning, pp. 67-92.
 Watkins CJ & Dayan P (1992), â€œQ-learning,â€ Machine learning, vol. 8, no. 3-4, pp. 279-292.
 Xiao J, Michalewicz Z & Zhang L (1997), â€œAdaptive evolutionary planner/navigator for mobile robots,â€ IEEE transactions on evolutionary computation, vol. 1, no. 1, pp. 18-28.
 Konar A, Chakraborty IG & Singh SJ (2013), â€œA deterministic improved Q-learning for path planning of a mobile robot,â€ IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 43, no. 5, pp. 1141-1153.
 Konar A, Goswami I & Singh SJ (2013), â€œA deterministic improved q-learning for path planning of a mobile robot,â€ IEEE Trans. Systems, Man, and Cybernetics: Systems, vol. 43, no. 5, pp. 1141-1153.
 Rakshit P, Konar K, Bhowmik P (2013), â€œRealization of an adaptive memetic algorithm using differential evolution and q-learning: a case study in multirobot path planning,â€ IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 43, no. 4, pp. 814-831.
 Kim DH, Kim YJ & Kim KC (2000), â€œVector field based path planning and petri-net based role selection mechanism with q-learning for the soccer robot system,â€ Intelligent Automation & Soft Computing, vol. 6, no. 1, pp. 75-87.
 Chen C, Li HX, & Dong D (2008), â€œHybrid control for robot navigation-a hierarchical q-learning algorithm,â€ IEEE Robotics & Automation Magazine, vol. 15, no. 2.
 Xiao H, Liao L, & Zhou F, "Mobile robot path planning based on q-ann." pp. 2650-2654.
 Guo M, Liu Y, and Malec J (2004), â€œA new Q-Learning algorithm based on the metropolis criterion,â€ IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 34, no. 5, pp. 2140-2143.
 Das P, Behera H, & Panigrahi B (2016), â€œIntelligent-based multi-robot path planning inspired by improved classical q-learning and improved particle swarm optimization with perturbed Velocity,â€ Engineering Science and Technology, an International Journal, vol. 19, no. 1, pp. 651-669.
 Juang CF, & Lu CM (2009), â€œAnt colony optimization incorporated with fuzzy q-learning for reinforcement fuzzy control,â€ IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, vol. 39, no. 3, pp. 597-608.
 MuÃ±oz P, Barco R, & de la Bandera I (2013), â€œOptimization of load balancing using fuzzy q-learning for next generation wireless networks,â€ Expert systems with applications, vol. 40, no. 4, pp. 984-994.
 Khajenejad M, Afshinmanesh F, Marandi A, "Intelligent particle swarm optimization using Q-learning." pp. 7-12.
 Rakshit P, Konar A & S. Das S, "ABC-TDQL: An adaptive memetic algorithm." pp. 35-42.
 Li C, Zhang J, & Li Y, "Application of artificial neural network based on q-learning for mobile robot path planning." pp. 978-982.
 Duguleana M, & Mogan G (2016), â€œNeural networks based reinforcement learning for mobile robots obstacle avoidance,â€ Expert Systems with Applications, vol. 62, pp. 104-115.
 Huang BQ, Cao GY, & Guo M, "Reinforcement learning neural network to the problem of autonomous mobile robot obstacle avoidance." pp. 85-89.
 Hwang KS, Tan SW, & Chen CC (2004), â€œCooperative strategy based on adaptive q-learning for robot soccer systems,â€ IEEE Transactions on Fuzzy Systems, vol. 12, no. 4, pp. 569-576.
 Park KH, Kim YJ & Kim JH (2001), â€œModular Q-learning based multi-agent cooperation for robot soccer,â€ Robotics and Autonomous Systems, vol. 35, no. 2, pp. 109-122.
 Yijing Z, Zheng Z & Xiaoyi Z, "Q learning algorithm based uav path learning and obstacle avoidence approach." pp. 3397-3402.
View Full Article:
How to Cite
LicenseAuthors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under aÂ Creative Commons Attribution Licensethat allows others to share the work with an acknowledgement of the work''s authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal''s published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (SeeÂ The Effect of Open Access).