Map Reduce with Hybrid Optimization for Advanced Scalable Sparql Query Processing and Loading Speed in Big Data

Abstract

The expansion in web content had presented unique issues in the detailed term of adequately querying Resource Description Framework (RDF) graphs. Large amounts of information are kept on a massive range of connected servers that share storage space. Calculation methods were developed to directly conduct computation activities upon those machines, which were formerly utilized mostly for storage. In this research, we propose Smart Knowledge Graph (smart-KG) a technique that combines Transaction Processing Facility (TPF) with shipping reduced KG segments to divide the traffic load across the clients and lowering information transfers in volume to convert a specified Query language into a Hive application. We suggest a model-driven technique to accomplish this, develop a meta-model to specify a mapping between the Simple Protocol and RDF Query Language  (SPARQL) metamodel and Big Data query languages. The Atlas Transformation Language (ATL) was used to carry out the transformation. According to our tests, smart-KG surpasses cutting-edge customer solutions and boosts server-side availability, enabling more affordable and equitable administration of public and decentralized KGs. To evaluate our methodology, we tried an experiment on three different datasets that each contained a sizable amount of dispersed RDF data on a potent server cluster.