Applied Mathematics and Nonlinear Sciences
Journal license

Journal

Applied Mathematics and Nonlinear Sciences


Volume
& Issue

Volume 10, Issue 1


Published
on

February 3, 2025


Pages


DOI

Article

Research on Efficient Algorithms for Intelligent Computing in Big Data Analytics

Check for updates


Authors

Xiguo Zhou Affiliation:
College of Computer and Information Engineering, Tianjin Normal University, Tianjin, 300387, China.
, Ziping Zhao Affiliation:
College of Computer and Information Engineering, Tianjin Normal University, Tianjin, 300387, China.
and Wentao Gao Affiliation:
College of Computer and Information Engineering, Tianjin Normal University, Tianjin, 300387, China.


Abstract

In this paper, we first collect massive data based on the Hadoop HDFS system and perform distributed data storage for it. Then, the massive data in the system is queried quickly by designing a secondary index and a parallel region query method based on Hilbert coding. Experiments are carried out to measure the efficiency of the Hadoop HDFS system in efficient data storage and querying, and to investigate its scalability. After completing the efficient storage and querying of data, the Spark DBSCAN algorithm is proposed to mine the data efficiently. The performance of the Spark DBSCAN algorithm is examined by comparing and analysing the clustering performance and running speed of the Spark DBSCAN algorithm with the traditional DBSCAN algorithm. The query performance of Hadoop HDFS is superior to that of Jena-Hbase and SHARD systems. The fast query time of Hadoop HDFS stays the same, and the scalability is nearly linear with the data size in the case of a slow query. The clustering accuracy of the Spark DBSCAN algorithm is slightly better than that of the traditional parallel DBSCAN algorithm, but the overall difference is small. The Spark DBSCAN algorithm runs significantly faster than the traditional parallel DBSCAN algorithm on different datasets.


Keywords

Hadoop HDFS, Spark DBSCAN, Efficient storage query, Data mining, 68T05


Citation

Zhou, X., Zhao, Z., & Gao, W. (2025). Research on efficient algorithms for intelligent computing in big data analytics. Applied Mathematics and Nonlinear Sciences, 10(1). https://doi.org/10.2478/amns-2025-0020

Published by: Engineering Journals

Engineering Journals Logo