Package org.apache.hadoop.tools.fedbalance

Class DistCpProcedure

java.lang.Object

org.apache.hadoop.tools.fedbalance.procedure.BalanceProcedure

org.apache.hadoop.tools.fedbalance.DistCpProcedure

All Implemented Interfaces:

org.apache.hadoop.io.Writable

public class DistCpProcedure
extends BalanceProcedure

Copy data through distcp. Super user privilege needed. PRE_CHECK :pre-check of src and dst. INIT_DISTCP :the first round of distcp. DIFF_DISTCP :copy snapshot diff round by round until there is no diff. DISABLE_WRITE :disable write operations. FINAL_DISTCP :close all open files and do the final round distcp. FINISH :procedure finish.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	DistCpProcedure.Stage

Nested classes/interfaces inherited from class org.apache.hadoop.tools.fedbalance.procedure.BalanceProcedure

BalanceProcedure.RetryException

Field Summary

Fields

Modifier and Type	Field	Description
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor	Description
DistCpProcedure()
DistCpProcedure(java.lang.String name, java.lang.String nextProcedure, long delayDuration, FedBalanceContext context)	The constructor of DistCpProcedure.

Method Summary

Modifier and Type	Method	Description
static void	disableForTest()
protected void	disableWrite(FedBalanceContext fbcontext)	Disable write by cancelling the execute permission of the source path.
static void	enableForTest()
protected void	enableWrite()	Enable write.
boolean	execute()	The main process.
void	readFields(java.io.DataInput in)
protected void	updateStage(DistCpProcedure.Stage value)
void	write(java.io.DataOutput out)

Methods inherited from class org.apache.hadoop.tools.fedbalance.procedure.BalanceProcedure

delayMillisBeforeRetry, equals, hashCode, isSchedulerShutdown, name, nextProcedure, setNextProcedure, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

DistCpProcedure

public DistCpProcedure()

DistCpProcedure

public DistCpProcedure(java.lang.String name,
                       java.lang.String nextProcedure,
                       long delayDuration,
                       FedBalanceContext context)
                throws java.io.IOException

The constructor of DistCpProcedure.

Parameters:

name - the name of the procedure.

nextProcedure - the name of the next procedure.

delayDuration - the delay duration when this procedure is delayed.

context - the federation balance context.

Throws:

java.io.IOException

Method Detail

enableForTest

public static void enableForTest()

disableForTest

public static void disableForTest()

execute

public boolean execute()
                throws BalanceProcedure.RetryException,
                       java.io.IOException

Description copied from class: BalanceProcedure

The main process. This is called by the ProcedureScheduler. Make sure the process quits fast when it's interrupted and the scheduler is shut down. One procedure may have many phases and all the phases share the same member variables. Each time this method returns, the journal is saved. User can serialize the current phase in write(DataOutput) so the job can continue with the last unfinished phase after it is recovered. The return value indicates whether the job should go to the next procedure. Return true after all the phases finish. Example: class ProcedureWithManyPhase extends BalanceProcedure { enum PHASE { P1, P2, P3 } PHASE phase; public boolean execute(T lastProcedure) throws RetryException, IOException { switch (phase) { case P1: // do something. return false; case P2: // do something. return false; case P3: // do something. return true; default: throw new IOException("Unexpected phase " + phase); } } public void write(DataOutput out) { out.writeInt(phase.ordinal()); } public void readFields(DataInput in) throws IOException { stage = Stage.values()[in.readInt()]; } }

Specified by:

execute in class BalanceProcedure

Returns:

true if the procedure has done and the job will go to the next procedure, otherwise false.

Throws:

BalanceProcedure.RetryException - if this procedure needs delay a while then retry.

java.io.IOException

disableWrite

protected void disableWrite(FedBalanceContext fbcontext)
                     throws java.io.IOException

Disable write by cancelling the execute permission of the source path. TODO: Disable the super user from writing.

Parameters:

fbcontext - the context.

Throws:

java.io.IOException - if can't disable write.

enableWrite

protected void enableWrite()
                    throws java.io.IOException

Enable write.

Throws:

java.io.IOException - if can't enable write.

updateStage

@VisibleForTesting
protected void updateStage(DistCpProcedure.Stage value)

write

public void write(java.io.DataOutput out)
           throws java.io.IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Overrides:

write in class BalanceProcedure

Throws:

java.io.IOException

readFields

public void readFields(java.io.DataInput in)
                throws java.io.IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Overrides:

readFields in class BalanceProcedure

Throws:

java.io.IOException