Skip navigation
Please use this identifier to cite or link to this item: http://uvadoc.uva.es/handle/10324/27615
Title: Compresión de datasets RDF en HDT usando Spark
Authors: Barrales Ruiz, Carlos Vladimir
Editors: Universidad de Valladolid. Escuela Técnica Superior de Ingenieros de Telecomunicación
Tutor: Fuente Redondo, Pablo de la
Martínez Prieto, Miguel Ángel
Issue Date: 2017
Degree : Máster en Investigación en Tecnologías de la Información y las Comunicaciones
Abstract: Apache Spark is a general purpose big data processing framework using the mapreduce paradigm, quickly becoming very popular. Although the information provided by Spark authors indicates a substantial improvement in performance against Hadoop, there is very little evidence in the literature of specific tests that reliably proves such claims. In this Master Work study the benefits of Spark and the most important factors on which they depend, considering as a reference the transformation of RDF datasets into HDT format. The main objective of this work is to perform one exploratory study to leverage Spark solving the HDT serialization problem, finding ways to remove limitations of the current implementations, like the memory need which use to increase with the dataset size. To do that, first we’ve setup a open environment to ensure reproducibility and contributed with 3 different approaches implementing the most heavy task in the HDT serialization. The test performed with different dataset sizes showed the benefits obtained with the proposed solution compared to legacy Hadoop MapReduce implementation, as well as some highlights to improve even more the serialization algorithm.
Classification: Semantic web
RDF
Compression
HDT
Departament : Departamento de Informática (Arquitectura y Tecnología de Computadores, Ciencias de la Computación e Inteligencia Artificial, Lenguajes y Sistemas Informáticos)
Language: eng
URI: http://uvadoc.uva.es/handle/10324/27615
Rights: info:eu-repo/semantics/openAccess
Appears in Collections:Trabajos Fin de Máster UVa

Files in This Item:
File Description SizeFormat 
TFM-G806.zip3,24 MBZIPView/Open

This item is licensed under a Creative Commons License Creative Commons

Suggestions
University of Valladolid
Powered by MIT's. DSpace software, Version 5.5
UVa-STIC