Ranger User Guide

Ranger User Guide

PHD 3.0

Ranger Console Operations

Introduction

Note: For information about installing Ranger, see Installing Ranger in Installing PHD Manually.

Centralized security administration in a Hadoop environment has four aspects:

  • Authentication

    Effected by Kerberos in native Apache Hadoop, and secured by the Apache Knox Gateway via the HTTP/REST API. (For further information, see the Apache Knox Gateway Manager Guide.)

  • Authorization

    Fine-grained access control provides flexibility in defining policies...

    • on the folder and file level, via HDFS

    • on the database, table and column level, via Hive

    • on the table, column family and column level, via HBase

  • Audit

    Controls access into the system via extensive user access auditing in HDFS, Hive and HBase at...

    • IP address

    • Resource/resource type

    • Timestamp

    • Access granted or denied

  • Data Protection

    Provided by wire encryption, volume encryption and (via HDFS TDE and Pivotal partners) file/column encryption

Central security administration is provided through the the Apache Ranger console, which delivers a ‘single pane of glass’ for the security administrator. The console ensures consistent security policy coverage across the entire Hadoop stack.

Apache Ranger Console Operations

Opening and Closing the Console

To invoke the Ranger Console, log in to the Ranger portal.

To log in, use your username and password.

Once you log in, your username is also displayed on the Ranger Console home page.

To log out of the Ranger Console, click your username, in the top right-hand corner of the screen. At the drop-down menu, click Logout.

Console Operations Summary

Ranger Administration Console

The Ranger console controls five types of functions:

Repository Manager

The Ranger Repository Manager is open by default in the Ranger Console. To return to the Repository Manager from any tab in the Ranger Console, go to the top left corner of the console and click Ranger.

  • To add a new repository to the Policy Manager, click the + button in the appropriate box on the Ranger Policy Manager, then complete the repository screen. When you are finished filling in the screen, click the green Add button.

  • To edit a repository in the Policy Manager, click the Edit icon to the right of the entry for that repository. The Policy Manager displays an expanded view of that repository, including a list of the policies it contains, their status, and the groups designated to administer those policies. (For a closer look at this screen, see Working With Policies.)

  • To delete a repository from the Policy Manager, click the Delete icon to the right of the entry for that repository.

Ranger Policy Manager Console

This section describes how to configure repositories in:

HDFS Repository Configuration

To add a repository to HDFS, complete the HDFS Create Repository screen as follows:

Field name

Description

Repository Name

The name of the repository; required when configuring agents.

Description

A description of the repository.

Active Status

Enabled or Disabled

Repository Type

HDFS (cannot be modified)

User Name

The end system username that can be used for connection.

Password

The password for the username entered above.

fs.default.name

The location of the Hadoop HDFS service, as noted in the hadoop configuration file core-site.xml OR (if this is a HA environment) the path for the primary NameNode.

hadoop.security.authorization

The type of authorization in use, as noted in the hadoop configuration file core-site.xml; either simple or Kerberos. (Required only if authorization is enabled.)

hadoop.security.auth_to_local

Maps the login credential to a username with Hadoop; use the value noted in the hadoop configuration file, core-site.xml.

dfs.datanode.kerberos.principal

The principal associated with the datanode where the repository resides, as noted in the hadoop configuration file hdfs-site.xml. (Required only if Kerberos authenticaiton is enabled.)

dfs.namenode.kerberos.principal

The principal associated with the NameNode where the repository resides, as noted in the hadoop configuration file hdfs-site.xml. (Required only if Kerberos authentication is enabled.)

dfs.secondary.namenode.kerberos.principal

The principal associated with the secondary NameNode where the repository resides, as noted in the hadoop configuration file hdfs-site.xml. (Required only if Kerberos authentication is enabled.)

Common Name For Certificate

The name of the certificate.

Hive Repository Configuration

To add a repository to Hive, complete the Hive Create Repository screen as follows:

Field name

Description

Repository Name

The name of the repository; required when configuring agents.

Description

A description of the repository.

Active Status

Enabled or Disabled

Repository Type

Hive (cannot be modified)

User Name

The end system username that can be used for connection.

Password

The password for the username entered above.

jdbc.driver ClassName

The full classname of the driver used for Hive connections. Default: org.apache.hive.jdbc.HiveDriver

jdbc.url

The complete connection URL, including port and database name. (Default port: 10000.) For example, on the sandbox, jdbc:hive2://sandbox:10000/.

Common Name For Certificate

The name of the certificate.

HBase Repository Configuration

To add a repository to HBase, complete the HBase Create Repository screen as follows:

Field name

Description

Repository Name

The name of the repository; required when configuring agents.

Description

A description of the repository.

Active Status

Enabled or Disabled

Repository Type

HBase (cannot be modified)

User Name

The end system username that can be used for connection.

Password

The password for the username entered above.

hadoop.security.authorization

The complete connection URL, including port and database name. (Default port: 10000.) For example, on the sandbox, jdbc:hive2://sandbox:10000/.

hbase.master.kerberos.principal

The Kerberos principal for the HBase Master. (Required only if Kerberos authenticaiton is enabled.)

hbase.security.authentication

As noted in the hadoop configuration file hbase-site.xml.

hbase.zookeeper.property.cientPort

As noted in the hadoop configuration file hbase-site.xml.

hbase.zookeeper.quorum

As noted in the hadoop configuration file hbase-site.xml.

zookeeper.znode.parent

As noted in the hadoop configuration file hbase-site.xml.

Knox Repository Configuration

To add a repository to Knox, complete the Knox Create Repository screen as follows:

Field name

Description

Repository Name

The name of the repository; required when configuring agents.

Description

A description of the repository.

Active Status

Enabled or Disabled

Repository Type

Knox (cannot be modified)

User Name

The end system username that can be used for connection.

Password

The password for the username entered above.

knox.url

The Gateway URL for Knox.

Common Name For Certificate

The name of the certificate.

Policy Manager

To examine the policies associated with each repository, go to the service where the repository resides and click the edit button. The Ranger Policy Manager view opens and an expanded view of that repository displays, with the policies listed beneath. The policy view includes a search window.

  • To add a new policy to the repository, click the Add New Policy button. The form looks slightly different, depending on the type of repository to which you are adding the policy.

  • To edit a policy, click the Edit icon to the right of the entry for that repository. The Policy Manager displays an expanded view of that policy.

  • To delete a policy, click the Delete icon to the right of the entry for that repository.

Open Repository With Policy List

This section describes the requirements for policy creation in

HDFS Policy Creation

Through configuration, Apache Ranger enables both Ranger policies and HDFS permissions to be checked for a user request. When the NameNode receives a user request, the Ranger plugin checks for policies set through the Ranger Policy Manager. If there are no policies, the Ranger plugin checks for permissions set in HDFS.

We recommend that permissions be created at the Ranger Policy Manager, and to have restrictive permissions at the HDFS level.

To add a policy to an HDFS repsitory, use the HDFS Add Policy form.

HDFS Add Policy Form

Complete the HDFS Add Policy Form as follows:

Field

Description

Enter Policy Name

Enter a unique name for this policy. The name cannot be duplicated anywhere in the system.

Resource Path

Define the resource path for the policy folder/file. To avoid the need to supply the full path OR to enable the policy for all subfolders or files, you can either complete this path using wild cards (for example, /home*) or specify that the policy should be Recursive. (See below.)

Description

(Optional:) Describe the purpose of the policy.

Recursive

Select if all files or subfolders within the existing folder will be included in this policy. (Use this option if you have specified a specific Resource Path to the top level folder, but want all subfolders or file to be included.)

Audit Logging

Specify whether this policy is audited. (De-select to disable auditing.)

Group Permissions

Use the pick list to assign group permissions appropriate to this policy. If desired, assign the group Administration privileges for the chosen resource. To add users or groups to the list, click the + button. (For further information, see Users

User Permissions

Use the pick list to assign individual user permissions appropriate to this policy. If desired, designate one or more users as Administrators for the chosen resource.

Enable/Disable

Policies are enabled by default. To restrict user/group access for a policy, disable the policy.

Hive Policy Creation

You can create policies in Hive at the Database Name, Table Name, and Column Name level.

To add a policy to a Hive repository, use the Hive Add Policy form.

Hive Add Policy Form

Complete the Hive Add Policy Form as follows:

Select Column Name

For the selected database and table﴾s﴿, select columns for the which the policy will be applicable.

Enter UDF Name

When UDF is selected, this field displays in place of Select Table Name and Select Column Name. Enter the name of the User Defined Function that should be the subject of the new policy.

Audit Logging

Specify whether this policy is audited. (De-select to disable auditing.)

Group Permissions

Specify the group to which this policy applies. To designate the group as an Administrator for the chosen research, specify Admin permission. (Administrators can create child policies based on existing policies.)

User Permissions

Specify a particular user to which this policy applies (outside of an already-specified group) OR designate a particular user as Admin for this policy. (Administrators can create child policies based on existing policies.)

Include/Exclude

Flags particular fields (table names or column names) as being included or excluded from consideration in the policy.

Enable/Disable

Policies are enabled by default. To restrict user or group access for the policy, select Disable.

Field

Description

Enter Policy Name

Enter an appropriate policy name.

This name is cannot be duplicated across the system.This field is mandatory.

Select DataBase Name

Select the appropriate database. Multiple databases can be selected for a particular policy.

particular policy. This field is mandatory.

Table/UDF Drop-down

To continue adding a Table-based policy, keep Table selected. To add a User Defined Function (UDF) select UDF. Select Table Name and Select Column Name are replaced with

Select Table Name

For the selected database, select table(s) for which the policy will be applicable.

table﴾s﴿ for the which the policy will be applicable.

Wild cards can be included in the resource path, in the database name, the table name, or column name:

  • * indicates zero or more occurences of characters

  • ? indicates a single character

Providing User Access to Hive Database Tables from the Command Line

Hive provides the means to manage user access to Hive database tables directly from the command line. The most commonly-used commands are:

  • GRANT

    Syntax: grant <permissions> on table <table> to user <user or group>;

    For example, to create a policy that grants user1 SELECT permission on the table default-hivesmoke22074, the command is grant select on table default.hivesmoke22074 to user user1;

    The syntax is the same for granting UPDATE, CREATE, DROP, ALTER, INDEX, LOCK, ALL and ADMIN rights.

  • REVOKE

    Syntax: revoke <permissions> on table <table> from user <user or group>;

    For example, to revoke the SELECT rights of user1 to the table default.hivesmoke22074, the command is revoke select on table default.hivesmoke22074 from user user1;

    The syntax is the same for revoking UPDATE, CREATE, DROP, ALTER, INDEX, LOCK, ALL and ADMIN rights.

HBase Policy Creation

To add a policy to an HBase repository, use the HBase Create Policy form.

Complete the HBase Create Repository screen as follows:

Select Column Families

For the selected table, specify the column families to which the policy applies.

Select Column Name

For the selected table and column families, specify the columns to which the policy applies.

Audit Logging

Specify whether this policy is audited. (De-select to disable auditing.)

Group Permissions

Specify the group to which this policy applies. To designate the group as an Administrator for the chosen resource, specify Admin permissions. (Administrators can create child policies based on existing policies.)

User Permissions

Specify a particular user to which this policy applies (outside of an already-specified group) OR designate a particular user as Admin for this policy. (Administrators can create child policies based on existing policies.)

Enable/Disable

Policies are enabled by default. To restrict user or group access to the policy, select Disable.

Label

Description

Enter Policy Name

Enter an appropriate policy name.

This name is cannot be duplicated across the system.This field is mandatory.

Select Table Name

Select the appropriate database. Multiple tables can be selected for a particular policy. This field is mandatory

Wild cards can be included in the resource path, in the database name, the table name, or column name:

  • * indicates zero or more occurences of characters

  • ? indicates a single character

Providing User Access to HBase Database Tables from the Command Line

HBase provides the means to manage user access to Hive database tables directly from the command line. The most commonly-used commands are:

  • GRANT

    Syntax: grant '<user-or-group>','<permissions>','<table>

    For example, to create a policy that grants user1 read/write permission on the table usertable, the command is grant 'user1','RW','usertable'

    The syntax is the same for granting CREATE and ADMIN rights.

  • REVOKE

    Syntax: revoke '<user-or-group>','<usertable>'

    For example, to revoke the read/write access of user1 to the table usertable, the command is revoke 'user1','usertable'

Knox Policy Creation

To add a policy to a Knox repository, use the Knox Add Policy Form.

Complete the Knox Create Policy screen as follows:

Select Service Name

Service Name:Binds a Hadoop service with an internal URL that the gateway uses to proxy requests from external clients to the internal cluster services. Enter an appropriate Service Name.

Audit Logging

Specify whether this policy is audited. (De-select to disable auditing.)

Group Permissions

Specify the group to which this policy applies. To designate the group as an Administrator for the chosen resource, specify Admin permissions. (Administrators can create chid policies based on existing policies.)

User Permissions

Specify a particular user to which this policies applies (outside of an already-specified group) OR designate a particular user as Admin for this policy. (Administrators can create child policies based on existing policies.)

Enable/Disable

Policies are enabled by default. To restrict user or group access to the policy, select Disable.

Label

Description

Enter Policy Name

Enter an appropriate policy name.

This name is cannot be duplicated across the system.

Select Topology Name

A topology is a graph of computation. Each node in a topology contains processing logic, and links between nodes indicate how data should be passed around between nodes.Enter an appropriate Topology Name.

Wild cards can be included in the resource path, in the database name, the table name, or column name:

  • * indicates zero or more occurences of characters

  • ? indicates a single character

Since Knox does not provide a command line methodology for assigning privileges or roles to users, the User and Group Permissions portion of the Knox Create Policy form is especially important.

Admin

Gives the user Admin privileges.

Permission

Description

IP Address

The IP address from which the user logs in

Allow

Permits user to access topology that is specified in topology name

Users/Groups Administration

To examine the list of users and groups who can access the Ranger portal or its repositories, click the User/Groups tab at the top of the Ranger Console. The User/sGroup view displays:

  • internal users who can log in to the Ranger portal; created by the Ranger console Policy Manager

  • external users who can access repositories controlled by the Ranger portal; created at other systems like Active Directory, LDAP or UNIX, and synched with those systems

  • Admins, who are the only users with permission to create users and create repositories, run reports, and perform other administrative tasks. Admins can also create child policies based on the original policy (base policy).

Users/Group List with Users tab active

Add User

To add a new user to the user list, click the Users sub-tab, then click Add New User. Add the appropriate user details, then click Save. The user is immediately added to the list.

Ranger User Detail

Edit User

(Admins only:) To edit a user, click the Users sub-tab, then click the user name and edit the appropriate details.

(All users:) To edit your own user details, click your name in the upper-right hand corner of the Ranger Console, then click Profile.

Edit User

Add Group

To add a group, click the Groups sub-tab, then click Add a Group. Enter a unique name for the group, and an optional description, then click Save.

Add Group with Group Detail

Edit Group

(Admins only:) To edit a group, click the Groups sub-tab, then click the group name and edit the appropriate details. Click Save.

Edit Group

Analytics Administration

To perform analytics on one or more policies, click the Analytics tab at the top of the Ranger Console. A list of all HDFS, Hive, HBase and Knox policies displays, and a search window.

You can search:

  • Policy Name: The policy name assigned to the policy while creating it.

  • Resource Path:The resource path used while creating the policy.

  • Group’ / ‘User Name:The group and the users to which the policy is assigned

Edit Group

Auditing

To explore options for auditing policies in Ranger, click the Audit tab.

The Audit view contains four sub-tabs:

Access Sub-tab

The Access sub-tab provides Repository activity data for all policies that have Audit set to On. The default repository Policy is configured to log all user activity within the Repository. This default policy does not contain user and group access rules.

You can filter the data based on the following criteria:

  • Actions -- Operations performed on resources, such as CREATE, UPDATE, DELETE, and password change

  • Audit Type -- Resource (for operations performed on resources), Assets (for operations performed on Policy) or Users

  • Session ID -- The session count increments each time you try to log into the system

  • User -- The mode through which the user tries to log in (username through which the operation was performed)

  • Start Date, End Date -- Filters results for a particular date range.

Ranger Console Audit tab, Access sub-tab

Different view when we click on an operation (Update operation in this case)

  • Login Sessions:‐

This module logs the information related to the sessions for each login. You can filter the data based on

Start Date,End Date

Login time and date is stored for each

Search Criteria

Description

Login ID

The user name through which you login to the system

Session­id

The session count increments each time you try to login to the system

Login time

Result indicates whether the Login was successful or not.It can be one of the following﴾‘Success’ , ‘Wrong Password’ , ’Account Disabled’ , ‘Locked’ , ‘Password Expired’ , ’User Not Found’ ﴿.

session.A date range is used to filter the results for that particular date range

Login Type

The mode through which the user tries to login.﴾By entering username and password﴿

IP

The IP of the system through which we log in

User Agent

Login time and date is stored for each session

Show Actions displays each action performed in that session

Agents:‐

This module shows the upload history of the Security Agents.This module displays all the repositories Exported from the system.You can filter the data based on the following

Http Response Code:

The http code which you get when you try

Search Criteria

Description

Agent IP

Ip of the agent which tries to export the repository

Agent ID

Name of the agent which tries to export the repository.

Repository Name

The repository name we are trying to export.

to export the Repositories

Start Date,End Date

Export time and date is stored for each agent. A date range is used to filter the results for that particular date range.

Admin Sub-tab

The Admin sub-tab module contains all events for the PHD Security Administration Web UI, including Repository, Policy Manager, Log in, etc. (actions like create,update,delete,password change).

You can filter the data based on the following criteria:

  • Actions -- Operations performed on resources, such as CREATE, UPDATE, DELETE, and password change

  • Audit Type -- Resource (for operations performed on resources), Assets (for operations performed on Policy) or Users

  • Session ID -- The session count increments each time you try to log into the system

  • User -- The mode through which the user tries to log in (username through which the operation was performed)

  • Start Date, End Date -- Filters results for a particular date range.

Ranger Console Audit tab, Access sub-tab

Login Sessions Sub-tab

The Login Sessions sub-tab logs the information related to the sessions for each login. You can filter the data based on the following criteria:

  • Login ID -- The username through which someone logs in to the system

  • Session-id -- The session count increments each time the user tries to log into the system

  • Start Date, End Date -- Specifies that results should be filtered based on a particular start date and end date

  • Login Type -- The mode through which the user tries to login (by entering username and password)

  • IP -- The IP address of the system through which the user logged in

  • User Agent -- The login time and date for each session

  • Login time -- Logs whether or not the login was successful. Possible results can be Success, Wrong Password, Account Disabled, Locked, Password Expired or User Not Found.

Ranger Console Audit tab, Login Sessions sub-tab

Agents Sub-tab

The Agents sub-tab shows the upload history of the Security Agents. This module lists all the repositories exported from the system.

You can filter the data based on the following criteria:

  • Agent IP -- the IP address of the agent that tried to export the repository

  • Agent ID -- the name of the agent that tried to export the repository

  • HTTP Response Code -- the HTTP code obtained when the export was attempted

  • Start Date, End Date -- filters results for a particular start date and end date

  • Repository Name -- the name of the exported repository

Ranger Console Audit tab, Agents sub-tab

Appendices

Special Requirements for High Availability Environments

In a HA environment, primary and secondary NameNodes must be configured as described in the PHD System Administration Guides. (See NameNode High Availability for Hadoop.)

To enable Ranger in the HDFS HA environment, an HDFS plugin must be set up in each NameNode, and then pointed to the same HDFS repository set up in the Security Manager. Any policies created within that HDFS repository are automatically synchronized to the primary and secondary NameNodes through the installed Apache Ranger plugin. That way, if the primary NameNode fails, the secondary namenode takes over and the Ranger plugin at that NameNode begins to enforce the same policies for access control.

When creating the repository, you must include the fs.default.name for the primary NameNode. If the primary NameNode fails during policy creation, you can then temporarily use the fs.default.name of the secondary NameNode in the repository details to enable directory lookup for policy creation.