Package org.apache.hadoop.hdfs.security.token.block

Class BlockTokenSecretManager

java.lang.Object

org.apache.hadoop.security.token.SecretManager<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>

org.apache.hadoop.hdfs.security.token.block.BlockTokenSecretManager

@Private
public class BlockTokenSecretManager
extends org.apache.hadoop.security.token.SecretManager<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>

BlockTokenSecretManager can be instantiated in 2 modes, master mode and worker mode. Master can generate new block keys and export block keys to workers, while workers can only import and use block keys received from master. Both master and worker can generate and verify block tokens. Typically, master mode is used by NN and worker mode is used by DN.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.security.token.SecretManager

org.apache.hadoop.security.token.SecretManager.InvalidToken

Field Summary

Fields

Modifier and Type	Field	Description
static org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>	DUMMY_TOKEN
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor	Description
BlockTokenSecretManager(long keyUpdateInterval, long tokenLifetime, int nnIndex, int numNNs, java.lang.String blockPoolId, java.lang.String encryptionAlgorithm, boolean useProto)
BlockTokenSecretManager(long keyUpdateInterval, long tokenLifetime, int nnIndex, int numNNs, java.lang.String blockPoolId, java.lang.String encryptionAlgorithm, boolean useProto, boolean shouldWrapQOP)
BlockTokenSecretManager(long keyUpdateInterval, long tokenLifetime, java.lang.String blockPoolId, java.lang.String encryptionAlgorithm, boolean useProto)	Constructor for workers.

Method Summary

Modifier and Type	Method	Description
void	addKeys(ExportedBlockKeys exportedKeys)	Set block keys, only to be used in worker mode
void	checkAccess(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier id, java.lang.String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode)
void	checkAccess(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier id, java.lang.String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode, org.apache.hadoop.fs.StorageType[] storageTypes)	Check if access should be allowed.
void	checkAccess(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier id, java.lang.String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode, org.apache.hadoop.fs.StorageType[] storageTypes, java.lang.String[] storageIds)	Check if access should be allowed.
void	checkAccess(org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> token, java.lang.String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode)	Check if access should be allowed.
void	checkAccess(org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> token, java.lang.String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode, org.apache.hadoop.fs.StorageType[] storageTypes, java.lang.String[] storageIds)	Check if access should be allowed.
static <T> void	checkAccess(T[] candidates, T[] requested, java.lang.String msg)	Check if the requested values can be satisfied with the values in the BlockToken.
void	clearAllKeysForTesting()
org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier	createIdentifier()	Create an empty block token identifier
protected byte[]	createPassword(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier identifier)	Create a new password/secret for the given block token identifier.
ExportedBlockKeys	exportKeys()	Export block keys, only to be used in master mode
org.apache.hadoop.hdfs.security.token.block.DataEncryptionKey	generateDataEncryptionKey()	Generate a data encryption key for this block pool, using the current BlockKey.
org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>	generateToken(java.lang.String userId, org.apache.hadoop.hdfs.protocol.ExtendedBlock block, java.util.EnumSet<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode> modes, org.apache.hadoop.fs.StorageType[] storageTypes, java.lang.String[] storageIds)	Generate a block token for a specified user
org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>	generateToken(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, java.util.EnumSet<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode> modes, org.apache.hadoop.fs.StorageType[] storageTypes, java.lang.String[] storageIds)	Generate an block token for current user
BlockKey	getCurrentKey()
int	getSerialNoForTesting()
boolean	hasKey(int keyId)
byte[]	retrieveDataEncryptionKey(int keyId, byte[] nonce)	Recreate an encryption key based on the given key id and nonce.
byte[]	retrievePassword(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier identifier)	Look up the token password/secret for the given block token identifier.
void	setBlockPoolId(java.lang.String blockPoolId)
void	setKeyUpdateIntervalForTesting(long millis)
void	setSerialNo(int nextNo)
void	setTokenLifetime(long tokenLifetime)	set token lifetime.
boolean	updateKeys(long updateTime)	Update block keys if update time > update interval.

Methods inherited from class org.apache.hadoop.security.token.SecretManager

checkAvailableForRead, createPassword, createSecretKey, generateSecret, retriableRetrievePassword

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

DUMMY_TOKEN

public static final org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> DUMMY_TOKEN

Constructor Detail

BlockTokenSecretManager

public BlockTokenSecretManager(long keyUpdateInterval,
                               long tokenLifetime,
                               java.lang.String blockPoolId,
                               java.lang.String encryptionAlgorithm,
                               boolean useProto)

Constructor for workers.

Parameters:

keyUpdateInterval - how often a new key will be generated

tokenLifetime - how long an individual token is valid

useProto - should we use new protobuf style tokens

BlockTokenSecretManager

public BlockTokenSecretManager(long keyUpdateInterval,
                               long tokenLifetime,
                               int nnIndex,
                               int numNNs,
                               java.lang.String blockPoolId,
                               java.lang.String encryptionAlgorithm,
                               boolean useProto)

BlockTokenSecretManager

public BlockTokenSecretManager(long keyUpdateInterval,
                               long tokenLifetime,
                               int nnIndex,
                               int numNNs,
                               java.lang.String blockPoolId,
                               java.lang.String encryptionAlgorithm,
                               boolean useProto,
                               boolean shouldWrapQOP)

Method Detail

setSerialNo

@VisibleForTesting
public void setSerialNo(int nextNo)

setBlockPoolId

public void setBlockPoolId(java.lang.String blockPoolId)

exportKeys

public ExportedBlockKeys exportKeys()

Export block keys, only to be used in master mode

addKeys

public void addKeys(ExportedBlockKeys exportedKeys)
             throws java.io.IOException

Set block keys, only to be used in worker mode

Throws:

java.io.IOException

updateKeys

public boolean updateKeys(long updateTime)
                   throws java.io.IOException

Update block keys if update time > update interval.

Returns:

true if the keys are updated.

Throws:

java.io.IOException

generateToken

public org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> generateToken(org.apache.hadoop.hdfs.protocol.ExtendedBlock block,
                                                                                                                              java.util.EnumSet<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode> modes,
                                                                                                                              org.apache.hadoop.fs.StorageType[] storageTypes,
                                                                                                                              java.lang.String[] storageIds)
                                                                                                                       throws java.io.IOException

Generate an block token for current user

Throws:

java.io.IOException

generateToken

public org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> generateToken(java.lang.String userId,
                                                                                                                              org.apache.hadoop.hdfs.protocol.ExtendedBlock block,
                                                                                                                              java.util.EnumSet<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode> modes,
                                                                                                                              org.apache.hadoop.fs.StorageType[] storageTypes,
                                                                                                                              java.lang.String[] storageIds)

Generate a block token for a specified user

checkAccess

public void checkAccess(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier id,
                        java.lang.String userId,
                        org.apache.hadoop.hdfs.protocol.ExtendedBlock block,
                        org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode,
                        org.apache.hadoop.fs.StorageType[] storageTypes,
                        java.lang.String[] storageIds)
                 throws org.apache.hadoop.security.token.SecretManager.InvalidToken

Check if access should be allowed. userID is not checked if null. This method doesn't check if token password is correct. It should be used only when token password has already been verified (e.g., in the RPC layer). Some places need to check the access using StorageTypes and for other places the StorageTypes is not relevant.

Throws:

org.apache.hadoop.security.token.SecretManager.InvalidToken

checkAccess

public void checkAccess(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier id,
                        java.lang.String userId,
                        org.apache.hadoop.hdfs.protocol.ExtendedBlock block,
                        org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode,
                        org.apache.hadoop.fs.StorageType[] storageTypes)
                 throws org.apache.hadoop.security.token.SecretManager.InvalidToken

Check if access should be allowed. userID is not checked if null. This method doesn't check if token password is correct. It should be used only when token password has already been verified (e.g., in the RPC layer). Some places need to check the access using StorageTypes and for other places the StorageTypes is not relevant.

Throws:

org.apache.hadoop.security.token.SecretManager.InvalidToken

checkAccess

public void checkAccess(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier id,
                        java.lang.String userId,
                        org.apache.hadoop.hdfs.protocol.ExtendedBlock block,
                        org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode)
                 throws org.apache.hadoop.security.token.SecretManager.InvalidToken

Throws:

org.apache.hadoop.security.token.SecretManager.InvalidToken

checkAccess

public static <T> void checkAccess(T[] candidates,
                                   T[] requested,
                                   java.lang.String msg)
                            throws org.apache.hadoop.security.token.SecretManager.InvalidToken

Check if the requested values can be satisfied with the values in the BlockToken. This is intended for use with StorageTypes and StorageIDs. The current node can only verify that one of the storage [Type|ID] is available. The rest will be on different nodes.

Throws:

org.apache.hadoop.security.token.SecretManager.InvalidToken

checkAccess

public void checkAccess(org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> token,
                        java.lang.String userId,
                        org.apache.hadoop.hdfs.protocol.ExtendedBlock block,
                        org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode,
                        org.apache.hadoop.fs.StorageType[] storageTypes,
                        java.lang.String[] storageIds)
                 throws org.apache.hadoop.security.token.SecretManager.InvalidToken

Check if access should be allowed. userID is not checked if null

Throws:

org.apache.hadoop.security.token.SecretManager.InvalidToken

checkAccess

public void checkAccess(org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> token,
                        java.lang.String userId,
                        org.apache.hadoop.hdfs.protocol.ExtendedBlock block,
                        org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode mode)
                 throws org.apache.hadoop.security.token.SecretManager.InvalidToken

Check if access should be allowed. userID is not checked if null

Throws:

org.apache.hadoop.security.token.SecretManager.InvalidToken

setTokenLifetime

public void setTokenLifetime(long tokenLifetime)

set token lifetime.

createIdentifier

public org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier createIdentifier()

Create an empty block token identifier

Specified by:

createIdentifier in class org.apache.hadoop.security.token.SecretManager<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>

Returns:

a newly created empty block token identifier

createPassword

protected byte[] createPassword(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier identifier)

Create a new password/secret for the given block token identifier.

Specified by:

createPassword in class org.apache.hadoop.security.token.SecretManager<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>

Parameters:

identifier - the block token identifier

Returns:

token password/secret

retrievePassword

public byte[] retrievePassword(org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier identifier)
                        throws org.apache.hadoop.security.token.SecretManager.InvalidToken

Look up the token password/secret for the given block token identifier.

Specified by:

retrievePassword in class org.apache.hadoop.security.token.SecretManager<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>

Parameters:

identifier - the block token identifier to look up

Returns:

token password/secret as byte[]

Throws:

org.apache.hadoop.security.token.SecretManager.InvalidToken

generateDataEncryptionKey

public org.apache.hadoop.hdfs.security.token.block.DataEncryptionKey generateDataEncryptionKey()

Generate a data encryption key for this block pool, using the current BlockKey.

Returns:

a data encryption key which may be used to encrypt traffic over the DataTransferProtocol

retrieveDataEncryptionKey

public byte[] retrieveDataEncryptionKey(int keyId,
                                        byte[] nonce)
                                 throws org.apache.hadoop.hdfs.protocol.datatransfer.InvalidEncryptionKeyException

Recreate an encryption key based on the given key id and nonce.

Parameters:

keyId - identifier of the secret key used to generate the encryption key.

nonce - random value used to create the encryption key

Returns:

the encryption key which corresponds to this (keyId, blockPoolId, nonce)

Throws:

org.apache.hadoop.hdfs.protocol.datatransfer.InvalidEncryptionKeyException

getCurrentKey

public BlockKey getCurrentKey()

setKeyUpdateIntervalForTesting

@VisibleForTesting
public void setKeyUpdateIntervalForTesting(long millis)

clearAllKeysForTesting

@VisibleForTesting
public void clearAllKeysForTesting()

hasKey

@VisibleForTesting
public boolean hasKey(int keyId)

getSerialNoForTesting

@VisibleForTesting
public int getSerialNoForTesting()