Title: An IoT-oriented real-time storage mechanism for massive small files based on Swift

Authors: Dongjie Zhu; Haiwen Du; Yuhua Wang; Xuan Peng

Addresses: School of Computer Science and Technology, Harbin Institute of Technology, Weihai, China ' School of Computer Science and Technology, Harbin Institute of Technology, Weihai, China ' School of Computer Science and Technology, Harbin Institute of Technology, Weihai, China ' School of Computer Science and Technology, Harbin Institute of Technology, Weihai, China

Abstract: In the internet of things (IoT), large amounts of small files are generated from various structure sensors in cloud storage platforms. Real-time storage of massive small files will put great pressures to the traditional file system. By studying the real-time storage strategy of massive small files based on the object storage architecture Swift, we proved a unique aggregation storage strategy, called sequential data aggregation strategy (SDAS), for storage of small files. We designed a two-level index structure to improve writing rate by transferring randomly write to sequentially write. To improve overall data access efficiency and solve the performance bottleneck of proxy node, we utilise a file's potential relevance of timing to prefetch related files that are merged in other blocks. Simulation results show that relative to the original system, SDAS has shorter response time of writing operation, lower cost of index maintenance cost, more balanced node load and 30% reduction in disk overhead.

Keywords: cloud storage; internet of things; IoT; massive small files; object storage; Swift.

DOI: 10.1504/IJES.2020.105280

International Journal of Embedded Systems, 2020 Vol.12 No.1, pp.72 - 80

Received: 17 Dec 2017
Accepted: 06 Feb 2018
Published online: 24 Feb 2020 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article