Package org.apache.hadoop.hdfs.server.namenode

Class EditLogFileOutputStream

java.lang.Object

org.apache.hadoop.hdfs.server.namenode.EditLogOutputStream

org.apache.hadoop.hdfs.server.namenode.EditLogFileOutputStream

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

@Private
public class EditLogFileOutputStream
extends EditLogOutputStream

An implementation of the abstract class EditLogOutputStream, which stores edits in a local file.

Field Summary

Fields

Modifier and Type	Field	Description
static int	MIN_PREALLOCATION_LENGTH

Constructor Summary

Constructors

Constructor	Description
EditLogFileOutputStream(org.apache.hadoop.conf.Configuration conf, java.io.File name, int size)	Creates output buffers and file object.

Method Summary

Modifier and Type	Method	Description
void	abort()	Close the stream without necessarily flushing any pending data.
void	close()	Close the journal.
void	create(int layoutVersion)	Create empty edits logs file.
void	flushAndSync(boolean durable)	Flush ready buffer to persistent store.
java.nio.channels.FileChannel	getFileChannelForTesting()
boolean	isOpen()
void	setFileChannelForTesting(java.nio.channels.FileChannel fc)
void	setReadyToFlush()	All data that has been written to the stream so far will be flushed.
static void	setShouldSkipFsyncForTesting(boolean skip)	For the purposes of unit tests, we don't need to actually write durably to disk.
boolean	shouldForceSync()	Implement the policy when to automatically sync the buffered edits log The buffered edits can be flushed when the buffer becomes full or a certain period of time is elapsed.
java.lang.String	toString()
void	write(FSEditLogOp op)	Write edits log operation to the stream.
static void	writeHeader(int layoutVersion, java.io.DataOutputStream out)	Write header information for this EditLogFileOutputStream to the provided DataOutputSream.
void	writeRaw(byte[] bytes, int offset, int length)	Write a transaction to the stream.

Methods inherited from class org.apache.hadoop.hdfs.server.namenode.EditLogOutputStream

flush, flush, generateReport, getCurrentLogVersion, getLastJournalledTxId, getNumSync, setCurrentLogVersion

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

MIN_PREALLOCATION_LENGTH

public static final int MIN_PREALLOCATION_LENGTH

See Also:

Constant Field Values

Constructor Detail

EditLogFileOutputStream

public EditLogFileOutputStream(org.apache.hadoop.conf.Configuration conf,
                               java.io.File name,
                               int size)
                        throws java.io.IOException

Creates output buffers and file object.

Parameters:

conf - Configuration object

name - File name to store edit log

size - Size of flush buffer

Throws:

java.io.IOException

Method Detail

write

public void write(FSEditLogOp op)
           throws java.io.IOException

Description copied from class: EditLogOutputStream

Write edits log operation to the stream.

Specified by:

write in class EditLogOutputStream

Parameters:

op - operation

Throws:

java.io.IOException

writeRaw

public void writeRaw(byte[] bytes,
                     int offset,
                     int length)
              throws java.io.IOException

Write a transaction to the stream. The serialization format is:

• the opcode (byte)
• the transaction id (long)
• the actual Writables for the transaction

Specified by:

writeRaw in class EditLogOutputStream

Parameters:

bytes - the bytes to write.

offset - offset in the bytes to write from

length - number of bytes to write

Throws:

java.io.IOException

create

public void create(int layoutVersion)
            throws java.io.IOException

Create empty edits logs file.

Specified by:

create in class EditLogOutputStream

Parameters:

layoutVersion - The LayoutVersion of the journal

Throws:

java.io.IOException

writeHeader

@VisibleForTesting
public static void writeHeader(int layoutVersion,
                               java.io.DataOutputStream out)
                        throws java.io.IOException

Write header information for this EditLogFileOutputStream to the provided DataOutputSream.

Parameters:

layoutVersion - the LayoutVersion of the EditLog

out - the output stream to write the header to.

Throws:

java.io.IOException - in the event of error writing to the stream.

close

public void close()
           throws java.io.IOException

Description copied from class: EditLogOutputStream

Close the journal.

Specified by:

close in interface java.lang.AutoCloseable

Specified by:

close in interface java.io.Closeable

Specified by:

close in class EditLogOutputStream

Throws:

java.io.IOException - if the journal can't be closed, or if there are unflushed edits

abort

public void abort()
           throws java.io.IOException

Description copied from class: EditLogOutputStream

Close the stream without necessarily flushing any pending data. This may be called after a previous write or close threw an exception.

Specified by:

abort in class EditLogOutputStream

Throws:

java.io.IOException

setReadyToFlush

public void setReadyToFlush()
                     throws java.io.IOException

All data that has been written to the stream so far will be flushed. New data can be still written to the stream while flushing is performed.

Specified by:

setReadyToFlush in class EditLogOutputStream

Throws:

java.io.IOException

flushAndSync

public void flushAndSync(boolean durable)
                  throws java.io.IOException

Flush ready buffer to persistent store. currentBuffer is not flushed as it accumulates new log records while readyBuffer will be flushed and synced.

Specified by:

flushAndSync in class EditLogOutputStream

Parameters:

durable - if true, the edits should be made truly durable before returning

Throws:

java.io.IOException

shouldForceSync

public boolean shouldForceSync()

Description copied from class: EditLogOutputStream

Implement the policy when to automatically sync the buffered edits log The buffered edits can be flushed when the buffer becomes full or a certain period of time is elapsed.

Overrides:

shouldForceSync in class EditLogOutputStream

Returns:

true if the number of buffered data exceeds the intial buffer size

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object

isOpen

public boolean isOpen()

Returns:

true if this stream is currently open.

setFileChannelForTesting

@VisibleForTesting
public void setFileChannelForTesting(java.nio.channels.FileChannel fc)

getFileChannelForTesting

@VisibleForTesting
public java.nio.channels.FileChannel getFileChannelForTesting()

setShouldSkipFsyncForTesting

@VisibleForTesting
public static void setShouldSkipFsyncForTesting(boolean skip)

For the purposes of unit tests, we don't need to actually write durably to disk. So, we can skip the fsync() calls for a speed improvement.

Parameters:

skip - true if fsync should not be called