Abstract– In today’s high-tech world, the amount of data generated or saved in any given system is staggeringly massive, which means that a substantial quantity of storage capacity is required. This is as a result of the fact that an increasing number of things are transitioning to a digital format. Not only are the points that are given, but also the speed at which the data can be retrieved is slowed down as a result of the massive amount of storage capacity that is being used up. Before any data can be saved to physical storage, it must first go through a process known as “deduplication,” which eliminates redundant copies of the data. This process identifies and removes any duplicate data that may be present within the enormous amounts of data that are being processed. Using this method, the amount of storage space that is required can be reduced to a more manageable level. The concept of decompression is applicable to a wide variety of settings, and over the course of the past few years, it has evolved into one of the most complex focuses of academic inquiry. In addition, the idea of decompression is applicable to a wide variety of settings. Using a solution that is based in the cloud, which is suggested in this article, it is proposed that the vast amounts of data that are already available will be deduplicated. The model incorporates both the forward and the backward compression of data in its overall structure. In addition to this, the essay investigates the numerous challenges that come up during the process of duplicating, as well as the data formats and methods that can be applied in order to successfully accomplish deduplication. Ultimately, the goal of the essay is to eliminate duplicates. The components that make up the cloud-based architecture that has been suggested have been taken apart and reassembled.
Wu, Y., Jiang, Z. L., Wang, X., Yiu, S. M., & Zhang, P. (2017, July). Dynamic data operations with deduplication in privacy-preserving public auditing for secure cloud storage. In 2017 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC) (Vol. 1, pp. 562-567). IEEE.
Saraswathi, S. Sabeetha, and N. Malarvizhi. “Distributed deduplication with fingerprint index management model for big data storage in the cloud.” Evolutionary Intelligence 14, no. 2 (2019): 683-690.
Sun, Z., Shen, J., & Yong, J. (2013). A novel approach to data deduplication over the engineering-oriented cloud systems. Integrated Computer-Aided Engineering, 20(1), 45-57.
Yan, Z., Ding, W., Yu, X., Zhu, H., & Deng, R. H. (2016). Deduplication on encrypted big data in cloud. IEEE transactions on big data, 2(2), 138-150.
Prajapati, Priteshkumar, and Parth Shah. “A review on secure data deduplication: Cloud storage security issue.” Journal of King Saud University-Computer and Information Sciences (2019).
Suresh, L., & Bharathi, M. A. (2019). Analysis of Block-Level Data Deduplication on Cloud Storage. In Ambient Communications and Computer Systems (pp. 401-409). Springer, Singapore.
Leesakul, W., Townend, P., & Xu, J. (2014, April). Dynamic data deduplication in cloud storage. In 2014 IEEE 8th International Symposium on Service Oriented System Engineering (pp. 320-325). IEEE.
Suttisirikul, Kiatchumpol, and Putchong Uthayopas. “Accelerating the cloud backup using gpu based data deduplication.” In 2012 IEEE 18th International Conference on Parallel and Distributed Systems, pp. 766-769. IEEE, 2012.
Kirubakaran, R., Prathibhan, C. M., & Karthika, C. (2015, March). A cloud based model for deduplication of large data. In 2015 IEEE International Conference on Engineering and Technology (ICETECH) (pp. 1-4). IEEE.
Dhanabal, D Yamini, Lakshmi P, Jeevanekasha, D Gokul, K Kabilan
Department of Information Technology,
Rathinam Technical Campus, Coimbatore, India