Package org.apache.hadoop.hdfs.server.namenode

Interface JournalManager

All Superinterfaces:

java.lang.AutoCloseable, java.io.Closeable, Storage.FormatConfirmable

All Known Implementing Classes:

FileJournalManager, JournalSet, QuorumJournalManager

@Private
@Evolving
public interface JournalManager
extends java.io.Closeable, Storage.FormatConfirmable

A JournalManager is responsible for managing a single place of storing edit logs. It may correspond to multiple files, a backup node, etc. Even when the actual underlying storage is rolled, or failed and restored, each conceptual place of storage corresponds to exactly one instance of this class, which is created when the EditLog is first opened.

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static class	JournalManager.CorruptionException	Indicate that a journal is cannot be used to load a certain range of edits.

Method Summary

Modifier and Type	Method	Description
boolean	canRollBack(StorageInfo storage, StorageInfo prevStorage, int targetLayoutVersion)	Return true if this JM can roll back to the previous storage state, false otherwise.
void	close()	Close the journal manager, freeing any resources it may hold.
void	discardSegments(long startTxId)	Discard the segments whose first txid is >= the given txid.
void	doFinalize()	Finalize the upgrade.
void	doPreUpgrade()	Perform any steps that must succeed across all JournalManagers involved in an upgrade before proceeding onto the actual upgrade stage.
void	doRollback()	Perform the rollback to the previous FS state.
void	doUpgrade(Storage storage)	Perform the actual upgrade of the JM.
void	finalizeLogSegment(long firstTxId, long lastTxId)	Mark the log segment that spans from firstTxId to lastTxId as finalized and complete.
void	format(NamespaceInfo ns, boolean force)	Format the underlying storage, removing any previously stored data.
long	getJournalCTime()
void	purgeLogsOlderThan(long minTxIdToKeep)	Remove all edit logs with transaction IDs lower than the given transaction ID.
void	recoverUnfinalizedSegments()	Recover segments which have not been finalized.
void	selectInputStreams(java.util.Collection<EditLogInputStream> streams, long fromTxId, boolean inProgressOk, boolean onlyDurableTxns)	Get a list of edit log input streams.
void	setOutputBufferCapacity(int size)	Set the amount of memory that this stream should use to buffer edits
EditLogOutputStream	startLogSegment(long txId, int layoutVersion)	Begin writing to a new segment of the log stream, which starts at the given transaction ID.

Methods inherited from interface org.apache.hadoop.hdfs.server.common.Storage.FormatConfirmable

hasSomeData, toString

Method Detail

format

void format(NamespaceInfo ns,
            boolean force)
     throws java.io.IOException

Format the underlying storage, removing any previously stored data.

Throws:

java.io.IOException

startLogSegment

EditLogOutputStream startLogSegment(long txId,
                                    int layoutVersion)
                             throws java.io.IOException

Begin writing to a new segment of the log stream, which starts at the given transaction ID.

Throws:

java.io.IOException

finalizeLogSegment

void finalizeLogSegment(long firstTxId,
                        long lastTxId)
                 throws java.io.IOException

Mark the log segment that spans from firstTxId to lastTxId as finalized and complete.

Throws:

java.io.IOException

setOutputBufferCapacity

void setOutputBufferCapacity(int size)

Set the amount of memory that this stream should use to buffer edits

recoverUnfinalizedSegments

void recoverUnfinalizedSegments()
                         throws java.io.IOException

Recover segments which have not been finalized.

Throws:

java.io.IOException

doPreUpgrade

void doPreUpgrade()
           throws java.io.IOException

Perform any steps that must succeed across all JournalManagers involved in an upgrade before proceeding onto the actual upgrade stage. If a call to any JM's doPreUpgrade method fails, then doUpgrade will not be called for any JM.

Throws:

java.io.IOException

doUpgrade

void doUpgrade(Storage storage)
        throws java.io.IOException

Perform the actual upgrade of the JM. After this is completed, the NN can begin to use the new upgraded metadata. This metadata may later be either finalized or rolled back to the previous state.

Parameters:

storage - info about the new upgraded versions.

Throws:

java.io.IOException

doFinalize

void doFinalize()
         throws java.io.IOException

Finalize the upgrade. JMs should purge any state that they had been keeping around during the upgrade process. After this is completed, rollback is no longer allowed.

Throws:

java.io.IOException

canRollBack

boolean canRollBack(StorageInfo storage,
                    StorageInfo prevStorage,
                    int targetLayoutVersion)
             throws java.io.IOException

Return true if this JM can roll back to the previous storage state, false otherwise. The NN will refuse to run the rollback operation unless at least one JM or fsimage storage directory can roll back.

Parameters:

storage - the storage info for the current state

prevStorage - the storage info for the previous (unupgraded) state

targetLayoutVersion - the layout version we intend to roll back to

Returns:

true if this JM can roll back, false otherwise.

Throws:

java.io.IOException

doRollback

void doRollback()
         throws java.io.IOException

Perform the rollback to the previous FS state. JMs which do not need to roll back their state should just return without error.

Throws:

java.io.IOException

discardSegments

void discardSegments(long startTxId)
              throws java.io.IOException

Discard the segments whose first txid is >= the given txid.

Parameters:

startTxId - The given txid should be right at the segment boundary, i.e., it should be the first txid of some segment, if segment corresponding to the txid exists.

Throws:

java.io.IOException

getJournalCTime

long getJournalCTime()
              throws java.io.IOException

Returns:

the CTime of the journal manager.

Throws:

java.io.IOException

close

void close()
    throws java.io.IOException

Close the journal manager, freeing any resources it may hold.

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Throws:

java.io.IOException

purgeLogsOlderThan

void purgeLogsOlderThan(long minTxIdToKeep)
                 throws java.io.IOException

Remove all edit logs with transaction IDs lower than the given transaction ID.

Parameters:

minTxIdToKeep - the lowest transaction ID that should be retained

Throws:

java.io.IOException - in the event of error

selectInputStreams

void selectInputStreams(java.util.Collection<EditLogInputStream> streams,
                        long fromTxId,
                        boolean inProgressOk,
                        boolean onlyDurableTxns)
                 throws java.io.IOException

Get a list of edit log input streams. The list will start with the stream that contains fromTxnId, and continue until the end of the journal being managed.

Parameters:

fromTxId - the first transaction id we want to read

inProgressOk - whether or not in-progress streams should be returned

onlyDurableTxns - whether or not streams should be bounded by durable TxId. A durable TxId is the committed txid in QJM or the largest txid written into file in FJM

Throws:

java.io.IOException - if the underlying storage has an error or is otherwise inaccessible