Package org.apache.flink.fs.gs.storage

Interface GSBlobStorage

All Known Implementing Classes:
GSBlobStorageImpl
public interface GSBlobStorage
Abstract blob storage, used to simplify interface to Google storage and make it mockable.

  • Method Details

    • writeBlob

      GSBlobStorage.WriteChannel writeBlob(GSBlobIdentifier blobIdentifier)
      Creates a write channel with the default chunk size.
      Parameters:
      blobIdentifier - The blob identifier to which to write
      Returns:
      The WriteChannel helper

    • writeBlob

      GSBlobStorage.WriteChannel writeBlob(GSBlobIdentifier blobIdentifier, org.apache.flink.configuration.MemorySize chunkSize)
      Creates a write channel with the specified chunk size.
      Parameters:
      blobIdentifier - The blob identifier to which to write
      chunkSize - The chunk size, must be > 0
      Returns:
      The WriteChannel helper

    • createBlob

      void createBlob(GSBlobIdentifier blobIdentifier)
      Create an empty blob.
      Parameters:
      blobIdentifier - The blob to create

    • getMetadata

      Optional<GSBlobStorage.BlobMetadata> getMetadata(GSBlobIdentifier blobIdentifier)
      Gets blob metadata.
      Parameters:
      blobIdentifier - The blob identifier
      Returns:
      The blob metadata, if the blob exists. Empty if the blob doesn't exist.

    • list

      List<GSBlobIdentifier> list(String bucketName, String prefix)
      Lists all the blobs in a bucket matching a given prefix.
      Parameters:
      bucketName - The bucket name
      prefix - The object prefix
      Returns:
      The found blobs ids

    • copy

      void copy(GSBlobIdentifier sourceBlobIdentifier, GSBlobIdentifier targetBlobIdentifier)
      Copies from a source blob id to a target blob id. Does not delete the source blob.
      Parameters:
      sourceBlobIdentifier - The source blob identifier
      targetBlobIdentifier - The target glob identifier

    • compose

      void compose(List<GSBlobIdentifier> sourceBlobIdentifiers, GSBlobIdentifier targetBlobIdentifier)
      Composes multiple blobs into one. Does not delete any of the source blobs.
      Parameters:
      sourceBlobIdentifiers - The source blob identifiers to combine, max of 32
      targetBlobIdentifier - The target blob identifier

    • delete

      List<Boolean> delete(Iterable<GSBlobIdentifier> blobIdentifiers)
      Deletes blobs. Note that this does not fail if blobs don't exist.
      Parameters:
      blobIdentifiers - The blob identifiers to delete
      Returns:
      The results of each delete operation.