A network authentication tool for hadoop

Computer Engineering

Due to the advent of new technologies, devices, and communication means like social networking sites, the amount of data production is growing rapidly every year and as a adoption of this, Apache Hadoop brings out the authentication and authorization issue of the data resource security. We discuss an innovation solution to implement the token authentication based on the Kerberos pre-authentication framework. We put forth a pre authentication mechanism for Kerberos that allows to have a centralized access control for the platform. Users authenticate to Key Distribution Center (KDC) using a standardized token, and develop a plug-in for MIT Kerberos that can be implemented separately to employ the new mechanism. Depending on this, we propose a solution of token authentication for whole Hadoop data base which will make user authentication easier and more secure. Cloud service providers are offering the popular Hadoop analytics platform following an “–as-a-service” model, i.e. clusters of machines in their cloud infrastructures pre-configured with Hadoop software. Such offerings lower the cost and complexity of deploying a comparable system on-premises, however security considerations and in particular data confidentiality hamper wider adoption of such services by enterprises that handle data of sensitive nature. In this paper, we describe our efforts in providing security for data-at-rest (i.e. data that is stored) when Hadoop is offered as a cloud service. We analyze the requirements and architecture for such service and further describe a new distributed file system that we developed for Hadoop called SDFS, towards supporting this premise. We analyze parameter tuning for SDFS and through experiments on a real test-bed we evaluate its performance. We further present simulation results that explore the parameter space and can guide tuning.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.