Information standardization series · Big Data Engine (BDE)
SQL query interface, stream data processing, machine learning
The big data engine (BDE) built based on Spark is equipped with built-in SQL query interface, flow data processing and machine learning. It provides a large-scale parallel processing framework based on distributed memory, which greatly advances the performance of big data analysis.
It provides reliable storage of HDFS and MapReduce programming paradigms through Hadoop for the large-scale parallel processing of data.
Through Hbase, large-scale distributed NoSQL database is realized to provide random access to large amounts of unstructured and semi-structured mass data.
Structured, semi-structured and unstructured data processing capability.
Sound data quality control capability by the concerted work with data quality management platform (DQMP). Noise data is eliminated to ensure the correctness and accuracy of the analysis on the quality of mass data.
