Index

B C D E F G H I L O P R S T W 
All Classes and Interfaces|All Packages|Constant Field Values

B

BlobUtils - Class in org.apache.flink.fs.gs.utils
Utility functions related to blobs.
BlobUtils() - Constructor for class org.apache.flink.fs.gs.utils.BlobUtils
 
bucketName - Variable in class org.apache.flink.fs.gs.storage.GSBlobIdentifier
The bucket name.

C

ChecksumUtils - Class in org.apache.flink.fs.gs.utils
Utility class related to checksums.
ChecksumUtils() - Constructor for class org.apache.flink.fs.gs.utils.ChecksumUtils
 
cleanupRecoverableState(RecoverableWriter.ResumeRecoverable) - Method in class org.apache.flink.fs.gs.writer.GSRecoverableWriter
 
close() - Method in interface org.apache.flink.fs.gs.storage.GSBlobStorage.WriteChannel
Closes the channel.
compose(List<GSBlobIdentifier>, GSBlobIdentifier) - Method in interface org.apache.flink.fs.gs.storage.GSBlobStorage
Composes multiple blobs into one.
compose(List<GSBlobIdentifier>, GSBlobIdentifier) - Method in class org.apache.flink.fs.gs.storage.GSBlobStorageImpl
 
COMPOSE_MAX_BLOBS - Static variable in class org.apache.flink.fs.gs.utils.BlobUtils
The maximum number of blobs that can be composed in a single operation.
configure(Configuration) - Method in class org.apache.flink.fs.gs.GSFileSystemFactory
 
ConfigUtils - Class in org.apache.flink.fs.gs.utils
Utilities class for configuration of Hadoop and Google Storage.
ConfigUtils() - Constructor for class org.apache.flink.fs.gs.utils.ConfigUtils
 
ConfigUtils.ConfigContext - Interface in org.apache.flink.fs.gs.utils
Interface that provides context-specific config helper functions, factored out to support unit testing
convertChecksumToString(int) - Static method in class org.apache.flink.fs.gs.utils.ChecksumUtils
Converts an int crc32 checksum to the string format used by Google storage, which is the base64 string for the int in big-endian format.
copy(GSBlobIdentifier, GSBlobIdentifier) - Method in interface org.apache.flink.fs.gs.storage.GSBlobStorage
Copies from a source blob id to a target blob id.
copy(GSBlobIdentifier, GSBlobIdentifier) - Method in class org.apache.flink.fs.gs.storage.GSBlobStorageImpl
 
CRC_HASH_FUNCTION - Static variable in class org.apache.flink.fs.gs.utils.ChecksumUtils
THe crc hash function used by Google storage.
create(URI) - Method in class org.apache.flink.fs.gs.GSFileSystemFactory
 
createBlob(GSBlobIdentifier) - Method in interface org.apache.flink.fs.gs.storage.GSBlobStorage
Create an empty blob.
createBlob(GSBlobIdentifier) - Method in class org.apache.flink.fs.gs.storage.GSBlobStorageImpl
 

D

delete(Iterable<GSBlobIdentifier>) - Method in interface org.apache.flink.fs.gs.storage.GSBlobStorage
Deletes blobs.
delete(Iterable<GSBlobIdentifier>) - Method in class org.apache.flink.fs.gs.storage.GSBlobStorageImpl
 

E

ENABLE_FILESINK_ENTROPY - Static variable in class org.apache.flink.fs.gs.GSFileSystemOptions
 
equals(Object) - Method in class org.apache.flink.fs.gs.storage.GSBlobIdentifier
 

F

fromBlobId(BlobId) - Static method in class org.apache.flink.fs.gs.storage.GSBlobIdentifier
Construct an abstract blob identifier from a Google BlobId.

G

GCS_HTTP_CONNECT_TIMEOUT - Static variable in class org.apache.flink.fs.gs.GSFileSystemOptions
Flink config option to set the http connection timeout.
GCS_HTTP_READ_TIMEOUT - Static variable in class org.apache.flink.fs.gs.GSFileSystemOptions
Flink config option to set the http read timeout.
GCS_RETRY_INIT_RPC_TIMEOUT - Static variable in class org.apache.flink.fs.gs.GSFileSystemOptions
 
GCS_RETRY_MAX_ATTEMPT - Static variable in class org.apache.flink.fs.gs.GSFileSystemOptions
Flink config option to set the http read timeout.
GCS_RETRY_MAX_RPC_TIMEOUT - Static variable in class org.apache.flink.fs.gs.GSFileSystemOptions
 
GCS_RETRY_RPC_TIMEOUT_MULTIPLIER - Static variable in class org.apache.flink.fs.gs.GSFileSystemOptions
 
GCS_RETRY_TOTAL_TIMEOUT - Static variable in class org.apache.flink.fs.gs.GSFileSystemOptions
 
getBlobId() - Method in class org.apache.flink.fs.gs.storage.GSBlobIdentifier
Get a Google blob id for this identifier, with generation=null.
getChecksum() - Method in interface org.apache.flink.fs.gs.storage.GSBlobStorage.BlobMetadata
The crc32 checksum for the blob.
getCommitRecoverableSerializer() - Method in class org.apache.flink.fs.gs.writer.GSRecoverableWriter
 
getenv(String) - Method in interface org.apache.flink.fs.gs.utils.ConfigUtils.ConfigContext
Returns a named environment variable.
getGcsRootUrl(Configuration) - Static method in class org.apache.flink.fs.gs.utils.ConfigUtils
 
getHadoopConfiguration(Configuration, ConfigUtils.ConfigContext) - Static method in class org.apache.flink.fs.gs.utils.ConfigUtils
Loads the Hadoop configuration, by loading from a Hadoop conf dir (if one exists) and then overlaying properties derived from the Flink config.
getHTTPConnectionTimeout() - Method in class org.apache.flink.fs.gs.GSFileSystemOptions
Timeout in millisecond to establish the connection.
getHTTPReadTimeout() - Method in class org.apache.flink.fs.gs.GSFileSystemOptions
Timeout in millisecond to read content from connection.
getInitialRpcTimeout() - Method in class org.apache.flink.fs.gs.GSFileSystemOptions
 
getMaxAttempts() - Method in class org.apache.flink.fs.gs.GSFileSystemOptions
 
getMaxRpcTimeout() - Method in class org.apache.flink.fs.gs.GSFileSystemOptions
 
getMetadata(GSBlobIdentifier) - Method in interface org.apache.flink.fs.gs.storage.GSBlobStorage
Gets blob metadata.
getMetadata(GSBlobIdentifier) - Method in class org.apache.flink.fs.gs.storage.GSBlobStorageImpl
 
getResumeRecoverableSerializer() - Method in class org.apache.flink.fs.gs.writer.GSRecoverableWriter
 
getRpcTimeoutMultiplier() - Method in class org.apache.flink.fs.gs.GSFileSystemOptions
 
getScheme() - Method in class org.apache.flink.fs.gs.GSFileSystemFactory
 
getStorageCredentials(Configuration, ConfigUtils.ConfigContext) - Static method in class org.apache.flink.fs.gs.utils.ConfigUtils
Creates an (optional) GoogleCredentials instance for the given Hadoop config and environment.
getTemporaryBlobIdentifier(GSBlobIdentifier, UUID, GSFileSystemOptions) - Static method in class org.apache.flink.fs.gs.utils.BlobUtils
Resolves a temporary blob identifier for a provided temporary object id and the provided options.
getTemporaryBucketName(GSBlobIdentifier, GSFileSystemOptions) - Static method in class org.apache.flink.fs.gs.utils.BlobUtils
Returns the temporary bucket name.
getTemporaryObjectName(GSBlobIdentifier, UUID) - Static method in class org.apache.flink.fs.gs.utils.BlobUtils
Returns a temporary object name, formed by appending the temporary object id to the temporary object partial name, i.e. .inprogress/foo/bar/abc for the final blob with object name "foo/bar" and temporary object id "abc".
getTemporaryObjectNameWithEntropy(GSBlobIdentifier, UUID) - Static method in class org.apache.flink.fs.gs.utils.BlobUtils
Returns a temporary object name with entropy, formed by adding the temporary object id to the temporary object partial name in both start and end of path, i.e. abc.inprogress/foo/bar/abc for the final blob with object name "foo/bar" and temporary object id "abc".
getTemporaryObjectPartialName(GSBlobIdentifier) - Static method in class org.apache.flink.fs.gs.utils.BlobUtils
Returns a temporary object partial name, i.e. .inprogress/foo/bar/ for the final blob with object name "foo/bar".
getTotalTimeout() - Method in class org.apache.flink.fs.gs.GSFileSystemOptions
 
getWriterChunkSize() - Method in class org.apache.flink.fs.gs.GSFileSystemOptions
The chunk size to use for writes on the underlying Google WriteChannel.
getWriterTemporaryBucketName() - Method in class org.apache.flink.fs.gs.GSFileSystemOptions
The temporary bucket name to use for recoverable writes, if different from the final bucket name.
GSBlobIdentifier - Class in org.apache.flink.fs.gs.storage
An abstraction for the Google BlobId type.
GSBlobIdentifier(String, String) - Constructor for class org.apache.flink.fs.gs.storage.GSBlobIdentifier
Construct an abstract blob identifier.
GSBlobStorage - Interface in org.apache.flink.fs.gs.storage
Abstract blob storage, used to simplify interface to Google storage and make it mockable.
GSBlobStorage.BlobMetadata - Interface in org.apache.flink.fs.gs.storage
Abstract blob metadata.
GSBlobStorage.WriteChannel - Interface in org.apache.flink.fs.gs.storage
Abstract blob write channel.
GSBlobStorageImpl - Class in org.apache.flink.fs.gs.storage
BlobStorage implementation for Google storage.
GSBlobStorageImpl(Storage) - Constructor for class org.apache.flink.fs.gs.storage.GSBlobStorageImpl
Constructs a GSBlobStorage instance.
GSFileSystemFactory - Class in org.apache.flink.fs.gs
Implementation of the Flink FileSystemFactory interface for Google Storage.
GSFileSystemFactory() - Constructor for class org.apache.flink.fs.gs.GSFileSystemFactory
Constructs the Google Storage file system factory.
GSFileSystemOptions - Class in org.apache.flink.fs.gs
The GS file system options.
GSFileSystemOptions(Configuration) - Constructor for class org.apache.flink.fs.gs.GSFileSystemOptions
Constructs an options instance.
GSRecoverableWriter - Class in org.apache.flink.fs.gs.writer
The recoverable writer implementation for Google storage.
GSRecoverableWriter(GSBlobStorage, GSFileSystemOptions) - Constructor for class org.apache.flink.fs.gs.writer.GSRecoverableWriter
Construct a GS recoverable writer.

H

hashCode() - Method in class org.apache.flink.fs.gs.storage.GSBlobIdentifier
 

I

isFileSinkEntropyEnabled() - Method in class org.apache.flink.fs.gs.GSFileSystemOptions
Whether entropy insertion is enabled in filesink path.

L

list(String, String) - Method in interface org.apache.flink.fs.gs.storage.GSBlobStorage
Lists all the blobs in a bucket matching a given prefix.
list(String, String) - Method in class org.apache.flink.fs.gs.storage.GSBlobStorageImpl
 
loadHadoopConfigFromDir(String) - Method in interface org.apache.flink.fs.gs.utils.ConfigUtils.ConfigContext
Loads the Hadoop configuration from a directory.
loadStorageCredentialsFromFile(String) - Method in interface org.apache.flink.fs.gs.utils.ConfigUtils.ConfigContext
Loads the Google credentials from a file.

O

objectName - Variable in class org.apache.flink.fs.gs.storage.GSBlobIdentifier
The object name, within the bucket.
open(Path) - Method in class org.apache.flink.fs.gs.writer.GSRecoverableWriter
 
org.apache.flink.fs.gs - package org.apache.flink.fs.gs
 
org.apache.flink.fs.gs.storage - package org.apache.flink.fs.gs.storage
 
org.apache.flink.fs.gs.utils - package org.apache.flink.fs.gs.utils
 
org.apache.flink.fs.gs.writer - package org.apache.flink.fs.gs.writer
 

P

parseUri(URI) - Static method in class org.apache.flink.fs.gs.utils.BlobUtils
Parses a blob id from a Google storage uri, i.e. gs://bucket/foo/bar yields a blob with bucket name "bucket" and object name "foo/bar".

R

recover(RecoverableWriter.ResumeRecoverable) - Method in class org.apache.flink.fs.gs.writer.GSRecoverableWriter
 
recoverForCommit(RecoverableWriter.CommitRecoverable) - Method in class org.apache.flink.fs.gs.writer.GSRecoverableWriter
 
requiresCleanupOfRecoverableState() - Method in class org.apache.flink.fs.gs.writer.GSRecoverableWriter
 

S

SCHEME - Static variable in class org.apache.flink.fs.gs.GSFileSystemFactory
The scheme for the Google Storage file system.
stringifyHadoopConfig(Configuration) - Static method in class org.apache.flink.fs.gs.utils.ConfigUtils
Helper to serialize a Hadoop config to a string, for logging.
supportsResume() - Method in class org.apache.flink.fs.gs.writer.GSRecoverableWriter
 

T

toString() - Method in class org.apache.flink.fs.gs.GSFileSystemOptions
 
toString() - Method in class org.apache.flink.fs.gs.storage.GSBlobIdentifier
 

W

write(byte[], int, int) - Method in interface org.apache.flink.fs.gs.storage.GSBlobStorage.WriteChannel
Writes data to the channel.
writeBlob(GSBlobIdentifier) - Method in interface org.apache.flink.fs.gs.storage.GSBlobStorage
Creates a write channel with the default chunk size.
writeBlob(GSBlobIdentifier) - Method in class org.apache.flink.fs.gs.storage.GSBlobStorageImpl
 
writeBlob(GSBlobIdentifier, MemorySize) - Method in interface org.apache.flink.fs.gs.storage.GSBlobStorage
Creates a write channel with the specified chunk size.
writeBlob(GSBlobIdentifier, MemorySize) - Method in class org.apache.flink.fs.gs.storage.GSBlobStorageImpl
 
WRITER_CHUNK_SIZE - Static variable in class org.apache.flink.fs.gs.GSFileSystemOptions
 
WRITER_TEMPORARY_BUCKET_NAME - Static variable in class org.apache.flink.fs.gs.GSFileSystemOptions
 
B C D E F G H I L O P R S T W 
All Classes and Interfaces|All Packages|Constant Field Values