DistCpOptionSwitch (Apache Hadoop Distributed Copy 3.4.1.100-eep-941 API)

java.lang.Object
- java.lang.Enum<DistCpOptionSwitch>
- - org.apache.hadoop.tools.DistCpOptionSwitch

All Implemented Interfaces:

java.io.Serializable, java.lang.Comparable<DistCpOptionSwitch>
```
public enum DistCpOptionSwitch
extends java.lang.Enum<DistCpOptionSwitch>
```
Enumeration mapping configuration keys to distcp command line options.

Enum Constant Summary

Enum Constants
Enum Constant	Description
`APPEND`
`ATOMIC_COMMIT`	Copy all the source files and commit them atomically to the target This is typically useful in cases where there is a process polling for availability of a file/dir.
`BANDWIDTH`	Specify bandwidth per map in MB, accepts bandwidth as a fraction
`BLOCKING`	Should DisctpExecution be blocking
`BLOCKS_PER_CHUNK`
`COPY_BUFFER_SIZE`	Configurable copy buffer size.
`COPY_STRATEGY`	Copy strategy is use.
`DELETE_MISSING`	Deletes missing files in target that are missing from source.
`DIFF`
`DIRECT_WRITE`	Write directly to the final location, avoiding the creation and rename of temporary files.
`FILE_LIMIT`
`FILTERS`	Path containing a list of strings, which when found in the path of a file to be copied excludes that file from the copy job.
`IGNORE_FAILURES`	Ignores any failures during copy, and continues with rest.
`KEEP_LINKS`
`LISTING_CHECK`	Abort job if copy listing has errors
`LOG_PATH`	Log path where distcp output logs are written to
`MAX_MAPS`	Max number of maps to use during copy.
`NUM_LISTSTATUS_THREADS`	Number of threads for building source file listing (before map-reduce phase, max one listStatus per thread at a time).
`OVERWRITE`	Overwrite target-files unconditionally.
`PRESERVE_STATUS`	Preserves status of file/path in the target.
`RDIFF`
`SIZE_LIMIT`
`SKIP_CRC`	Skip CRC checks between source and target, when determining what files need to be copied.
`SOURCE_FILE_LISTING`	Source file listing can be provided to DistCp in a file.
`SYNC_FOLDERS`	Update target location by copying only files that are missing in the target.
`TRACK_MISSING`	Track missing files in target that are missing from source This allows for other applications to complete the synchronization, possibly with object-store-specific delete algorithms.
`UPDATE_ROOT`
`USE_ITERATOR`
`VERBOSE_LOG`	Log additional info (path, size) in the SKIP/COPY log.
`WORK_PATH`	Work path to be used only in conjunction in Atomic commit

Field Summary

Fields
Modifier and Type Field Description

static java.lang.String PRESERVE_STATUS_DEFAULT

Fields
Modifier and Type	Field	Description
`static java.lang.String`	`PRESERVE_STATUS_DEFAULT`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`static void`	`addToConf(org.apache.hadoop.conf.Configuration conf, DistCpOptionSwitch option)`	Helper function to set an option to hadoop configuration object
`static void`	`addToConf(org.apache.hadoop.conf.Configuration conf, DistCpOptionSwitch option, java.lang.String value)`	Helper function to add an option to hadoop configuration object
`java.lang.String`	`getConfigLabel()`	Get Configuration label for the option
`org.apache.commons.cli.Option`	`getOption()`	Get CLI Option corresponding to the distcp option
`java.lang.String`	`getSwitch()`	Get Switch symbol
`java.lang.String`	`toString()`
`static DistCpOptionSwitch`	`valueOf(java.lang.String name)`	Returns the enum constant of this type with the specified name.
`static DistCpOptionSwitch[]`	`values()`	Returns an array containing the constants of this enum type, in the order they are declared.

Methods inherited from class java.lang.Enum
clone, compareTo, equals, finalize, getDeclaringClass, hashCode, name, ordinal, valueOf

Methods inherited from class java.lang.Object
getClass, notify, notifyAll, wait, wait, wait

- Enum Constant Detail
  - IGNORE_FAILURES
```
public static final DistCpOptionSwitch IGNORE_FAILURES
```
    Ignores any failures during copy, and continues with rest. Logs failures in a file
  - LISTING_CHECK
```
public static final DistCpOptionSwitch LISTING_CHECK
```
    Abort job if copy listing has errors
  - PRESERVE_STATUS
```
public static final DistCpOptionSwitch PRESERVE_STATUS
```
    Preserves status of file/path in the target. Default behavior with -p, is to preserve replication, block size, user, group, permission, checksum type and timestamps on the target file. Note that when preserving checksum type, block size is also preserved. If any of the optional switches are present among rbugpcaxt, then only the corresponding file attribute is preserved.
  - SYNC_FOLDERS
```
public static final DistCpOptionSwitch SYNC_FOLDERS
```
    Update target location by copying only files that are missing in the target. This can be used to periodically sync two folders across source and target. Typically used with DELETE_MISSING Incompatible with ATOMIC_COMMIT
  - DELETE_MISSING
```
public static final DistCpOptionSwitch DELETE_MISSING
```
    Deletes missing files in target that are missing from source. This allows the target to be in sync with the source contents Typically used in conjunction with SYNC_FOLDERS Incompatible with ATOMIC_COMMIT
  - TRACK_MISSING
```
@Unstable
public static final DistCpOptionSwitch TRACK_MISSING
```
    Track missing files in target that are missing from source This allows for other applications to complete the synchronization, possibly with object-store-specific delete algorithms. Typically used in conjunction with SYNC_FOLDERS Incompatible with ATOMIC_COMMIT
  - NUM_LISTSTATUS_THREADS
```
public static final DistCpOptionSwitch NUM_LISTSTATUS_THREADS
```
    Number of threads for building source file listing (before map-reduce phase, max one listStatus per thread at a time).
  - MAX_MAPS
```
public static final DistCpOptionSwitch MAX_MAPS
```
    Max number of maps to use during copy. DistCp will split work as equally as possible among these maps
  - SOURCE_FILE_LISTING
```
public static final DistCpOptionSwitch SOURCE_FILE_LISTING
```
    Source file listing can be provided to DistCp in a file. This allows DistCp to copy random list of files from source and copy them to target
  - ATOMIC_COMMIT
```
public static final DistCpOptionSwitch ATOMIC_COMMIT
```
    Copy all the source files and commit them atomically to the target This is typically useful in cases where there is a process polling for availability of a file/dir. This option is incompatible with SYNC_FOLDERS and DELETE_MISSING
  - WORK_PATH
```
public static final DistCpOptionSwitch WORK_PATH
```
    Work path to be used only in conjunction in Atomic commit
  - LOG_PATH
```
public static final DistCpOptionSwitch LOG_PATH
```
    Log path where distcp output logs are written to
  - VERBOSE_LOG
```
public static final DistCpOptionSwitch VERBOSE_LOG
```
    Log additional info (path, size) in the SKIP/COPY log.
  - COPY_STRATEGY
```
public static final DistCpOptionSwitch COPY_STRATEGY
```
    Copy strategy is use. This could be dynamic or uniform size etc. DistCp would use an appropriate input format based on this.
  - SKIP_CRC
```
public static final DistCpOptionSwitch SKIP_CRC
```
    Skip CRC checks between source and target, when determining what files need to be copied.
  - OVERWRITE
```
public static final DistCpOptionSwitch OVERWRITE
```
    Overwrite target-files unconditionally.
  - APPEND
```
public static final DistCpOptionSwitch APPEND
```
  - DIFF
```
public static final DistCpOptionSwitch DIFF
```
  - RDIFF
```
public static final DistCpOptionSwitch RDIFF
```
  - BLOCKING
```
public static final DistCpOptionSwitch BLOCKING
```
    Should DisctpExecution be blocking
  - FILE_LIMIT
```
public static final DistCpOptionSwitch FILE_LIMIT
```
  - SIZE_LIMIT
```
public static final DistCpOptionSwitch SIZE_LIMIT
```
  - BLOCKS_PER_CHUNK
```
public static final DistCpOptionSwitch BLOCKS_PER_CHUNK
```
  - KEEP_LINKS
```
public static final DistCpOptionSwitch KEEP_LINKS
```
  - COPY_BUFFER_SIZE
```
public static final DistCpOptionSwitch COPY_BUFFER_SIZE
```
    Configurable copy buffer size.
  - BANDWIDTH
```
public static final DistCpOptionSwitch BANDWIDTH
```
    Specify bandwidth per map in MB, accepts bandwidth as a fraction
  - FILTERS
```
public static final DistCpOptionSwitch FILTERS
```
    Path containing a list of strings, which when found in the path of a file to be copied excludes that file from the copy job.
  - DIRECT_WRITE
```
public static final DistCpOptionSwitch DIRECT_WRITE
```
    Write directly to the final location, avoiding the creation and rename of temporary files. This is typically useful in cases where the target filesystem implementation does not support atomic rename operations, such as with the S3AFileSystem which translates file renames to potentially very expensive copy-then-delete operations.
  - USE_ITERATOR
```
public static final DistCpOptionSwitch USE_ITERATOR
```
  - UPDATE_ROOT
```
public static final DistCpOptionSwitch UPDATE_ROOT
```
- Field Detail
  - PRESERVE_STATUS_DEFAULT
```
public static final java.lang.String PRESERVE_STATUS_DEFAULT
```
    See Also:
    
    Constant Field Values
- Method Detail
  - values
```
public static DistCpOptionSwitch[] values()
```
    Returns an array containing the constants of this enum type, in the order they are declared. This method may be used to iterate over the constants as follows:
```
for (DistCpOptionSwitch c : DistCpOptionSwitch.values())
    System.out.println(c);
```
    Returns:
    
    an array containing the constants of this enum type, in the order they are declared
  - valueOf
```
public static DistCpOptionSwitch valueOf(java.lang.String name)
```
    Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)
    
    Parameters:
    
    name - the name of the enum constant to be returned.
    
    Returns:
    
    the enum constant with the specified name
    
    Throws:
    
    java.lang.IllegalArgumentException - if this enum type has no constant with the specified name
    
    java.lang.NullPointerException - if the argument is null
  - getConfigLabel
```
public java.lang.String getConfigLabel()
```
    Get Configuration label for the option
    
    Returns:
    
    configuration label name
  - getOption
```
public org.apache.commons.cli.Option getOption()
```
    Get CLI Option corresponding to the distcp option
    
    Returns:
    
    option
  - getSwitch
```
public java.lang.String getSwitch()
```
    Get Switch symbol
    
    Returns:
    
    switch symbol char
  - toString
```
public java.lang.String toString()
```
    Overrides:
    
    toString in class java.lang.Enum<DistCpOptionSwitch>
  - addToConf
```
public static void addToConf(org.apache.hadoop.conf.Configuration conf,
                             DistCpOptionSwitch option,
                             java.lang.String value)
```
    Helper function to add an option to hadoop configuration object
    
    Parameters:
    
    conf - - Configuration object to include the option
    
    option - - Option to add
    
    value - - Value
  - addToConf
```
public static void addToConf(org.apache.hadoop.conf.Configuration conf,
                             DistCpOptionSwitch option)
```
    Helper function to set an option to hadoop configuration object
    
    Parameters:
    
    conf - - Configuration object to include the option
    
    option - - Option to add

Enum DistCpOptionSwitch

Enum Constant Summary

Field Summary

Method Summary

Methods inherited from class java.lang.Enum

Methods inherited from class java.lang.Object

Enum Constant Detail

IGNORE_FAILURES

LISTING_CHECK

PRESERVE_STATUS

SYNC_FOLDERS

DELETE_MISSING

TRACK_MISSING

NUM_LISTSTATUS_THREADS

MAX_MAPS

SOURCE_FILE_LISTING

ATOMIC_COMMIT

WORK_PATH

LOG_PATH

VERBOSE_LOG

COPY_STRATEGY

SKIP_CRC

OVERWRITE

APPEND

DIFF

RDIFF

BLOCKING

FILE_LIMIT

SIZE_LIMIT

BLOCKS_PER_CHUNK

KEEP_LINKS

COPY_BUFFER_SIZE

BANDWIDTH

FILTERS

DIRECT_WRITE

USE_ITERATOR

UPDATE_ROOT

Field Detail

PRESERVE_STATUS_DEFAULT

Method Detail

values

valueOf

getConfigLabel

getOption

getSwitch

toString

addToConf

addToConf