Hadoop Security Guide

Hadoop Security Guide

PHD 3.0

Hadoop Security Features

For organizations that store sensitive data in the Hadoop ecosystem, such as proprietary or personal data that is subject to regulatory compliance (HIPPA, PCI, DSS, FISAM, etc), security is essential. Many orgranizations also have to adhere to strict internal security polices. See the Reference Guide for more information.

Pivotal HD provides a comprehensive approach to security in the following key areas:

  • Perimeter security: PHD enables isolatation of the Hadoop cluster using a gateway and properly configured firewall rules. PHD supports the following perimeter security:

  • Authentication: PHD provides single authentication point for services and users that integrates with existing enterprise identity and access management systems. PHD Supports the following authentication services:

    • Kerberos

    • LDAP

    • Local Unix System

    • SSO (at the perimeter through Apache Knox Gateway)

  • Authorization (Access Control): PHD provides features that allow system administrators to control access to Hadoop data using role-based authorization. PHD supports the following authorization models:

    • Fine-grained access control for data stored in HDFS

    • Resource-level access control for YARN

    • Coarser-grained service level access control for MapReduce Operations

    • Table and column family level access control for HBase data

    • Table level access control for Apache Hive data sets

  • Accounting (Security auditing and monitoring): PHD allows you to track Hadoop activity using Native Auditing (audit logs), perimeter security auditing logs on the Knox Gateway, and from a central location, the PHD Security Administration console, including:

    • Access requests

    • Data processing operations

    • Data changes

  • Data Protection: PHD provides the mechanisms for encrypting data in flight, and requires the use of partner solutions for encrypting data at rest, data discovery, and data masking. PHD supports the following wire encryption methods:

    • SSL for PHD Components

    • RPC encryption

    • Data Transfer Protocol