Redundancy Prevention and Secure Audit of Encrypted BigData in HDFS Cloud using CloudGaurd+ System


  • Vinit Atul Shevade
  • D A. Kulkarni





Redundancy Prevention, Cloud Storage, File Access Control, BigData, Data-Tag.


The recent emerging Cloud Storageand Computing Technology provides a better approach for administrations and service provisioning over the Internet byoffering various computing and storage resources. The key significant and favored cloud service’s is information storage. Therefore, to maintain and conserve the privacy of data owners, data is often encrypted and stored onto the cloud as ciphered data. Nevertheless, a ciphered data introduces some issues for cloud data redundancy, which has been proved to be critical for Bigdata storage and computation in cloudenvironment. Conventional redundancy prevention methods do not work efficiently over ciphered data. Existing systems forciphered data redundancy prevention raises security issues and eventually effects the cloud processing. These existing systems are not capable of efficiently handling data access control and revocations. Therefore aiming to overcome these issues and achieving both redundancy prevention and integrity of cloud data, in this paper we propose a system, namely CloudGaurd+, to prevent redundancy of the stored ciphered data in HDFS Cloud based on authentication provocation and cryptography techniques. CloudGaurd+ incorporates cloud information redundancy prevention with access control. We assess its efficiency and performance based on substantial analysis and computer programming and simulations. The investigation and reenactment comes about demonstrates the unrivaled productivity and adequacy of the CloudGaurd+ system for  prospective deployment practically , particularly for Bigdata redundancy prevention in HDFS distributed cloud storage can be achieved.




[1] Z.O. Wilcox, “Convergent Encryption Reconsidered,†2011, /msg08949.html.

[2] J. Pettitt, “Hash of Plaintext as Key?† 1996/02 /msg02013.html.

[3] The Freenet Project. Freenet.

[4] M. Bellare, S. Keelveedhi, and T. Ristenpart, “Message-Locked Encryption and Secure Deduplication,†Proceedings of Cryptology– EUROCRYPT 2013, 2013, pp. 296–312, doi:10.1007/978-3- 642-38348-9_18.

[5] D. Perttula, B. Warner, and Z. Wilcox-O'Hearn, “Attacks on Convergent Encryption,†

[6] D.T. Meyer and W.J Bolosky, “A Study of Practical Deduplication,†ACM Transactions on Storage, 7(4), pp. 1-20, 2012, doi:10.1145/2078861.2078864.

[7] Opendedup.

[8] C. Liu, C. Yang, X.Y. Zhang, and J.J. Chen, “External Integrity Verification for Outsourced Big Data in Cloud and IoT: A Big Picture,†Future Generation Computer Systems, 2015, vol. 49, pp. 58-67.

[9] C.W. Tsai, C.F. Lai, H.C. Chao, and A.V. Vasilakos, “Big data analytics: a survey,†Journal of Big Data, vol. 2, no. 1, pp. 1-32, 2015, doi:10.1186/s40537-015-0030-3.

[10] T.T. Wu, W.C. Dou, C.H. Hu, and J.J. Chen. “Service Mining for Trusted Service Composition in Cross-Cloud Environment,†IEEE Systems Journal, 2014, vol. PP, no. 99, pp. 1-12, doi:10.1109/JSYST.2014.2361841.

[11] L.F. Wei, H.J. Zhu, Z.F. Cao, X.L. Dong, W.W. Jia, Y.L. Chen, and A.V. Vasilakos, “Security and privacy for storage and computation in cloud computing,†Information Sciences, 2014, vol. 258, pp. 371-386, doi:10.1016/j.ins.2013.04.028.

[12] M. Ali, S.U. Khan, and A.V. Vasilakos, “Security in cloud computing: Opportunities and challenges,†Information Sciences, 2015, vol. 305, pp. 357-383, doi:10.1016/j.ins.2015.01.025.

[13] M. Ali, R. Dhamotharan, E. Khan, S.U. Khan, A.V. Vasilakos, K.Q. Li, and A.Y. Zomaya, “SeDaSC: secure data sharing in clouds,†IEEE Systems Journal, vol. PP, no. 99, pp. 1-10, 2015, doi: 10.1109/JSYST.2014.2379646.

[14] N.X. Xiong, A.V. Vasilakos, L.T. Yang, L.Y. Song, Y. Pan, R. Kannan, and Y.S. Li, “Comparative Analysis of Quality of Service and Memory Usage for Adaptive Failure Detectors in Healthcare Systems,†IEEE Journal on Selected Areas in Communications, 2009, vol. 27, no. 4, pp. 495-509, doi:10.1109/JSAC.2009.090512.

[15] Y.Z. Zhou, Y.X. Zhang, H. Liu, N.X. Xiong, and A.V. Vasilakos, “A Bare-Metal and Asymmetric Partitioning Approach to Client Virtualization,†IEEE Transactions on Services Computing, 2014, vol. 7, no. 1, pp.40-53, doi:10.1109/TSC.2012.32. 2332-7790 (c) 2016 IEEE.

[16] P. Meye, P. Raipin, F. Tronel, and E. Anceaume, “A Secure Two-Phase Data Deduplication Scheme,†Proceedings of HPCC/CSS/ICESS, 2014, pp. 802-809, doi:10.1109/HPCC.2014.134.

[17] M. Kaczmarczyk, M. Barczynski, W. Kilian, and C. Dubnicki, “Reducing Impact of Data Fragmentation Caused by In-Line Deduplication,†Proceedings of the 5th Annual International Systems and Storage Conference, 2012, pp. 15:1–15:12, doi:10.1145/2367589.2367600.

[18] Dropbox, “A File-Storage and Sharing Service,†

[19] Google Drive,

[20] Mozy, “Mozy: A File-storage and Sharing Service,†

[21] Z. Sun, J. Shen, and J.M. Yong, “DeDu: Building a Deduplication Storage System over Cloud Computing,†Proceedings of IEEE International Conference on Computer Supported Cooperative Work in Design, 2011, pp. 348-355, doi:10.1109/ CSCWD.2011.5960097.

[22] H. Shacham and B. Waters, “Compact proofs of retrievability,†in Proceedings of the 14th International Conference on the Theory and Application of Cryptology and Information Security: Advances in Cryp-tology, ser. ASIACRYPT ’08. Springer Berlin Heidelberg, 2008, pp. 90–107.

[23] E. Stefanov, M. van Dijk, A. Juels, and A. Oprea, “Iris: A scalable cloud file system with efficient integrity checks,†inProceedings of the 28th Annual Computer Security Applications Conference, ser. ACSAC ’12. New York, NY, USA: ACM, 2012, pp. 229–238.

[24] M. Azraoui, K. Elkhiyaoui, R. Molva, and M. ¨ Onen, “Stealthguard: Proofs of retrievability with hidden watchdogs,†inComputer Security -ESORICS 2014, ser. Lecture Notes in Computer Science, M. Kutyłowski and J. Vaidya, Eds., vol. 8712. Springer International Publishing, 2014, pp. 239–256.

[25] J. Li, X. Tan, X. Chen, and D. Wong, “An efficient proof of retrievability with public auditing in cloud computing,†in5th International Con-ference on Intelligent Networking and Collaborative Systems (INCoS), 2013, pp. 93–98.

[26] G. Ateniese, R. Burns, R. Curtmola, J. Herring, L. Kissner, Z. Peterson, and D. Song, “Provable data possession at untrusted stores,†inProceed-ings of the 14th ACM Conference on Computer and Communications Security, ser. CCS ’07. New York, NY, USA: ACM, 2007, pp. 598– 609.

[27] G. Ateniese, R. Burns, R. Curtmola, J. Herring, O. Khan, L. Kissner, Z. Peterson, and D. Song, “Remote data checking using provable data possession,â€ACMTrans. Inf. Syst. Secur., vol. 14, no. 1, pp. 12:1–12:34, 2011.

[28] G. Ateniese, R. Di Pietro, L. V. Mancini, and G. Tsudik, “Scalable and efficient provable data possession,†in Proceedings of the 4th International Conference on Security and Privacy in Communication Netowrks, ser. SecureComm ’08. New York, NY, USA: ACM, 2008, pp. 9:1–9:10.

[29] C. Erway, A. K¨upc¸¨u, C. Papamanthou, and R. Tamassia, “Dynamic provable data possession,†inProceedings of the 16th ACM Conference on Computer and Communications Security, ser. CCS ’09. New York, NY, USA: ACM, 2009, pp. 213–222.

[30] F. Seb´e, J. Domingo-Ferrer, A. Martinez-Balleste, Y. Deswarte, and J.-J. Quisquater, “Efficient remote data possession checking in critical information infrastructures,†IEEE Trans. on Knowl. and Data Eng., vol. 20, no. 8, pp. 1034–1038, 2008.

[31] Y. Zhu, H. Hu, G.-J. Ahn, and M. Yu, “Cooperative provable data possession for integrity verification in multicloudstorage,â€IEEE Trans-actions on Parallel and Distributed Systems, vol. 23, no. 12, pp. 2231– 2244, 2012.

[32] H. Wang, “Proxy provable data possession in public clouds,†IEEE Transactions on Services Computing, vol. 6, no. 4, pp. 551–559, 2013.

[33] S. Halevi, D. Harnik, B. Pinkas, and A. Shulman-Peleg, “Proofs of ownership in remote storage systems,†inProceedings of the 18th ACM Conference on Computer and Communications Security. ACM, 2011, pp. 491–500.

[34] W. K. Ng, Y. Wen, and H. Zhu, “Private data deduplication protocols in cloud storage,†in Proceedings of the 27th Annual ACM Symposium on Applied Computing, ser. SAC ’12. New York, NY, USA: ACM, 2012, pp. 441–446.

[35] J. Douceur, A. Adya, W. Bolosky, P. Simon, and M. Theimer, “Reclaim-ing space from duplicate files in a serverless distributed file system,†in 22nd International Conference on Distributed Computing Systems, 2002, pp. 617–624.

[36] M. Bellare, S. Keelveedhi, and T. Ristenpart, “Message-locked encryp-tion and secure deduplication,†in Advances in Cryptology – EURO-CRYPT 2013, ser. Lecture Notes in Computer Science, T. Johansson and P. Nguyen, Eds. Springer Berlin Heidelberg, 2013, vol. 7881, pp. 296–312.

[37] M. Abadi, D. Boneh, I. Mironov, A. Raghunathan, and G. Segev, “Message-locked encryption for lock-dependent messages,†inAdvances in Cryptology – CRYPTO 2013, ser. Lecture Notes in Computer Science, R. Canetti and J. Garay, Eds. Springer Berlin Heidelberg, 2013, vol. 8042, pp. 374–391.

[38] S. Keelveedhi, M. Bellare, and T. Ristenpart, “Dupless: Server-aided encryption for deduplicated storage,†in Proceedings of the 22Nd USENIX Conference on Security, ser. SEC’13. Washington, D.C.: USENIX Association, 2013, pp. 179–194. [Online]. Avail-able:

[39] J. Li, X. Chen, M. Li, J. Li, P. Lee, and W. Lou, “Secure deduplication with efficient and reliable convergent key management,†IEEE Transactions on Parallel and Distributed Systems, vol. 25, no. 6, pp. 1615–1625, June 2014.

[40] J. Yuan and S. Yu, “Secure and constant cost public cloud storage auditing with deduplication,†in IEEE Conference on Communications and Network Security (CNS), 2013, pp. 145–153.

[41] Jingwei Li, Jin Li, DongqingXie, and Zhang Cai “Secure Auditing and Deduplicating Data in Cloud†IEEE TRANSACTIONS ON COMPUTERS, VOL. 65, NO. 8, AUGUST 2016.

[42] Zheng Yan, Senior Member, IEEE, Wenxiu Ding, Xixun Yu, Haiqi Zhu, and Robert H. Deng,Fellow, IEEE “Deduplication on Encrypted Big Data in Cloud†IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID.

[43] William Stallings “Cryptography and Network Security: Principles and Practice, Sixth Edition†ISBN 10: 0-13-335469-5, ISBN 13: 978-0-13-335469-0.

View Full Article:

How to Cite

Atul Shevade, V., & A. Kulkarni, D. (2018). Redundancy Prevention and Secure Audit of Encrypted BigData in HDFS Cloud using CloudGaurd+ System. International Journal of Engineering & Technology, 7(3.12), 933–937.
Received 2018-07-30
Accepted 2018-07-30
Published 2018-07-20