Real-time Big Data Processing System to Improve Semiconductor Production Efficiency in Smart Factory


  • Hyeopgeon Lee
  • Young-Woon Kim
  • Ki-Young Kim





Real-time Big data, Big data Processing System, Smart Factory, Spark, Memory DB


Semiconductor production efficiency is closely related to the defect rate in the production process. The temperature and humidity control in the production line are very important because these affect the defect rate. So many smart factory of semiconductor production uses sensor. It is installed in the semiconductor process, which send huge amounts of data per second to a central server to carry out temperature and humidity control in each production line. However, big data processing systems that analyze and process large-scale data are subject to frequent delays in processing, and transmitted data are lost owing to bottlenecks and insufficient memory caused by traffic concentrated in the central server. In this paper, we propose a real-time big data processing system to improve semiconductor production efficiency. The proposed system consists of a production line collection system, task processing system and data storage system, and improves the productivity of the semiconductor manufacturing process by reducing data processing delays as well as data loss and discarded data.




[1] Hyeop-Geon Lee, Young-Woon Kim, Ki-Young Kim and Jong-Seok Choi, â€Design of GlusterFs Based Big Data Distributed Processing System in Smart Factoryâ€, Journal of Korea Institute of information, Electronics, and Communication Technology, Vol.11, No.1, (2018), pp:70-75

[2] Hyeopgeon Lee, Young-Woon Kim and Ki-Young Kim, â€Implementation of an Efficient Big Data Collection Platform for Smart Manufacturingâ€, Journal of Korea Institute of information, Electronics, and Communication Technology, Vol.12, No.22, (2017), pp:6304-6307

[3] In-Hak Joo, â€Spatial Big Data Query Processing System Supporting SQL-based Query Language in Hadoopâ€, Journal of Engineering and Applied Sciences, Vol.10, No.1, (2017), pp:1-8

[4] Young-Woon Kim and Hyeopgeon Lee, †Implementation of Big Data Analysis System to Prevent Illegal Sales in the Cable TV Industryâ€, Journal of Korea Institute of information, Electronics, and Communication Technology, Vol.12, No.23, (2017), pp:6542-6545

[5] Jianguo Chen, Kenli Li, Zhuo Tang, Kashif Bilal, Shui Yu, Chuliang Weng and Keqin Li, â€A Parallel Random Forest Algorithm for Big Data in a Spark Cloud Computing Environmentâ€, IEEE Transactions on Parallel and Distributed Systems EEE Transactions on Smart Grid, Vol.28, No.4, (2016), pp:919-933

[6] Neha Bharill, Aruna Tiwari and Aayushi Malviya, †Fuzzy Based Scalable Clustering Algorithms for Handling Big Data Using Apache Sparkâ€, IEEE Transactions on Big Data, Vol.2, No.4, (2016), pp:339-352

[7] Xing He, Lei Chu, Robert Caiming Qiu, Qian Ai and Zenan Ling, â€A Novel Data-Driven Situation Awareness Approach for Future Grids—Using Large Random Matrices for Big Data Modelingâ€, IEEE Access, Vol.6, No.1, (2018), pp:13855-13865

[8] Ling Hu, Qiang Ni and Feng Yuan, â€Big data oriented novel background subtraction algorithm for urban surveillance systemsâ€, Big Data Mining and Analytics, Vol.1, No.2, (2018), pp:137-145

View Full Article: