Package org.apache.hadoop.hdfs.server.federation.store.driver.impl

Class StateStoreFileBaseImpl

java.lang.Object

org.apache.hadoop.hdfs.server.federation.store.driver.StateStoreDriver

org.apache.hadoop.hdfs.server.federation.store.driver.impl.StateStoreBaseImpl

org.apache.hadoop.hdfs.server.federation.store.driver.impl.StateStoreSerializableImpl

org.apache.hadoop.hdfs.server.federation.store.driver.impl.StateStoreFileBaseImpl

All Implemented Interfaces:

StateStoreRecordOperations

Direct Known Subclasses:

StateStoreFileImpl, StateStoreFileSystemImpl

public abstract class StateStoreFileBaseImpl
extends StateStoreSerializableImpl

StateStoreDriver implementation based on files. In this approach, we use temporary files for the writes and renaming "atomically" to the final value. Instead of writing to the final location, it will go to a temporary one and then rename to the final destination.

Field Summary

Fields inherited from class org.apache.hadoop.hdfs.server.federation.store.driver.impl.StateStoreSerializableImpl

COLON_MARK, SLASH_MARK

Constructor Summary

Constructors

Constructor	Description
StateStoreFileBaseImpl()

Method Summary

Modifier and Type	Method	Description
void	close()	Close the State Store driver connection.
protected abstract boolean	exists(java.lang.String path)	Check if a path exists.
<T extends BaseRecord> QueryResult<T>	get(java.lang.Class<T> clazz)	Get all records of the requested record class from the data store.
protected abstract java.util.List<java.lang.String>	getChildren(java.lang.String path)	Get the children for a path.
protected abstract int	getConcurrentFilesAccessNumThreads()
protected abstract <T extends BaseRecord> java.io.BufferedReader	getReader(java.lang.String path)	Get the reader of a record for the file system.
protected abstract java.lang.String	getRootDir()	Get root directory.
abstract <T extends BaseRecord> java.io.BufferedWriter	getWriter(java.lang.String path)	Get the writer of a record for the file system.
boolean	initDriver()	Prepare the driver to access data storage.
<T extends BaseRecord> boolean	initRecordStorage(java.lang.String className, java.lang.Class<T> recordClass)	Initialize storage for a single record class.
boolean	isDriverReady()	Check if the driver is currently running and the data store connection is valid.
static boolean	isOldTempRecord(java.lang.String pathRecord)	Check if a record is temporary and old.
protected abstract boolean	mkdir(java.lang.String path)	Make a directory.
<T extends BaseRecord> StateStoreOperationResult	putAll(java.util.List<T> records, boolean allowUpdate, boolean errorIfExists)	Creates multiple records.
<T extends BaseRecord> int	remove(java.lang.Class<T> clazz, Query<T> query)	Remove multiple records of a specific class that match a query.
protected abstract boolean	remove(java.lang.String path)	Remove a file.
<T extends BaseRecord> boolean	removeAll(java.lang.Class<T> clazz)	Remove all records of this class from the store.
protected abstract boolean	rename(java.lang.String src, java.lang.String dst)	Rename a file.
void	setInitialized(boolean ini)	Set the driver as initialized.

Methods inherited from class org.apache.hadoop.hdfs.server.federation.store.driver.impl.StateStoreSerializableImpl

getOriginalPrimaryKey, getPrimaryKey, init, newRecord, serialize, serializeString

Methods inherited from class org.apache.hadoop.hdfs.server.federation.store.driver.impl.StateStoreBaseImpl

get, getMultiple, put, remove

Methods inherited from class org.apache.hadoop.hdfs.server.federation.store.driver.StateStoreDriver

getConf, getIdentifier, getMetrics, getTime, verifyDriverReady

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StateStoreFileBaseImpl

public StateStoreFileBaseImpl()

Method Detail

getReader

protected abstract <T extends BaseRecord> java.io.BufferedReader getReader(java.lang.String path)

Get the reader of a record for the file system.

Type Parameters:

T - Type of the state store record.

Parameters:

path - Path of the record to read.

Returns:

Reader for the record.

getWriter

@VisibleForTesting
public abstract <T extends BaseRecord> java.io.BufferedWriter getWriter(java.lang.String path)

Get the writer of a record for the file system.

Type Parameters:

T - Type of the state store record.

Parameters:

path - Path of the record to write.

Returns:

Writer for the record.

exists

protected abstract boolean exists(java.lang.String path)

Check if a path exists.

Parameters:

path - Path to check.

Returns:

If the path exists.

mkdir

protected abstract boolean mkdir(java.lang.String path)

Make a directory.

Parameters:

path - Path of the directory to create.

Returns:

If the directory was created.

rename

protected abstract boolean rename(java.lang.String src,
                                  java.lang.String dst)

Rename a file. This should be atomic.

Parameters:

src - Source name.

dst - Destination name.

Returns:

If the rename was successful.

remove

protected abstract boolean remove(java.lang.String path)

Remove a file.

Parameters:

path - Path for the file to remove

Returns:

If the file was removed.

getChildren

protected abstract java.util.List<java.lang.String> getChildren(java.lang.String path)

Get the children for a path.

Parameters:

path - Path to check.

Returns:

List of children.

getRootDir

protected abstract java.lang.String getRootDir()

Get root directory.

Returns:

Root directory.

getConcurrentFilesAccessNumThreads

protected abstract int getConcurrentFilesAccessNumThreads()

setInitialized

public void setInitialized(boolean ini)

Set the driver as initialized.

Parameters:

ini - If the driver is initialized.

initDriver

public boolean initDriver()

Description copied from class: StateStoreDriver

Prepare the driver to access data storage.

Specified by:

initDriver in class StateStoreDriver

Returns:

True if the driver was successfully initialized. If false is returned, the state store will periodically attempt to re-initialize the driver and the router will remain in safe mode until the driver is initialized.

close

public void close()
           throws java.lang.Exception

Description copied from class: StateStoreDriver

Close the State Store driver connection.

Specified by:

close in class StateStoreDriver

Throws:

java.lang.Exception - if something goes wrong while closing the state store driver connection.

initRecordStorage

public <T extends BaseRecord> boolean initRecordStorage(java.lang.String className,
                                                        java.lang.Class<T> recordClass)

Description copied from class: StateStoreDriver

Initialize storage for a single record class.

Specified by:

initRecordStorage in class StateStoreDriver

Type Parameters:

T - Type of the state store record.

Parameters:

className - String reference of the record class to initialize, used to construct paths and file names for the record. Determined by configuration settings for the specific driver.

recordClass - Record type corresponding to the provided name.

Returns:

True if successful, false otherwise.

get

public <T extends BaseRecord> QueryResult<T> get(java.lang.Class<T> clazz)
                                          throws java.io.IOException

Description copied from interface: StateStoreRecordOperations

Get all records of the requested record class from the data store. To use the default implementations in this class, getAll must return new instances of the records on each call. It is recommended to override the default implementations for better performance.

Type Parameters:

T - Record class of the records.

Parameters:

clazz - Class of record to fetch.

Returns:

List of all records that match the class.

Throws:

java.io.IOException - Throws exception if unable to query the data store.

isOldTempRecord

@VisibleForTesting
public static boolean isOldTempRecord(java.lang.String pathRecord)

Check if a record is temporary and old.

Parameters:

pathRecord - Path for the record to check.

Returns:

If the record is temporary and old.

isDriverReady

public boolean isDriverReady()

Description copied from class: StateStoreDriver

Check if the driver is currently running and the data store connection is valid.

Specified by:

isDriverReady in class StateStoreDriver

Returns:

True if the driver is initialized and the data store is ready.

putAll

public <T extends BaseRecord> StateStoreOperationResult putAll(java.util.List<T> records,
                                                               boolean allowUpdate,
                                                               boolean errorIfExists)
                                                        throws StateStoreUnavailableException

Description copied from interface: StateStoreRecordOperations

Creates multiple records. Optionally updates existing records that have the same primary key.

Type Parameters:

T - Record class of the records.

Parameters:

records - List of data records to update or create. All records must be of class clazz.

allowUpdate - True if update of exiting record is allowed.

errorIfExists - True if an error should be returned when inserting an existing record. Only used if allowUpdate = false.

Returns:

The result of the putAll operation.

Throws:

StateStoreUnavailableException

remove

public <T extends BaseRecord> int remove(java.lang.Class<T> clazz,
                                         Query<T> query)
                                  throws StateStoreUnavailableException

Description copied from interface: StateStoreRecordOperations

Remove multiple records of a specific class that match a query. Requires the getAll implementation to fetch fresh records on each call.

Type Parameters:

T - Record class of the records.

Parameters:

clazz - The class to match the records with.

query - Query to filter what to remove.

Returns:

The number of records removed.

Throws:

StateStoreUnavailableException

removeAll

public <T extends BaseRecord> boolean removeAll(java.lang.Class<T> clazz)
                                         throws StateStoreUnavailableException

Description copied from interface: StateStoreRecordOperations

Remove all records of this class from the store.

Type Parameters:

T - Record class of the records.

Parameters:

clazz - Class of records to remove.

Returns:

True if successful.

Throws:

StateStoreUnavailableException