Package org.apache.flink.runtime.blob

Interface BlobWriter

All Known Implementing Classes:
BlobServer, VoidBlobWriter
public interface BlobWriter
BlobWriter is used to upload data to the BLOB store.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final org.slf4j.Logger
    LOG
     

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    deletePermanent(org.apache.flink.api.common.JobID jobId, PermanentBlobKey permanentBlobKey)
    Delete the uploaded data with the given JobID and PermanentBlobKey.
    int
    getMinOffloadingSize()
    Returns the min size before data will be offloaded to the BLOB store.
    static <T> org.apache.flink.types.Either<org.apache.flink.util.SerializedValue<T>,PermanentBlobKey>
    offloadWithException(org.apache.flink.util.SerializedValue<T> serializedValue, org.apache.flink.api.common.JobID jobId, BlobWriter blobWriter)
     
    PermanentBlobKey
    putPermanent(org.apache.flink.api.common.JobID jobId, byte[] value)
    Uploads the data of the given byte array for the given job to the BLOB server and makes it a permanent BLOB.
    PermanentBlobKey
    putPermanent(org.apache.flink.api.common.JobID jobId, InputStream inputStream)
    Uploads the data from the given input stream for the given job to the BLOB server and makes it a permanent BLOB.
    static <T> org.apache.flink.types.Either<org.apache.flink.util.SerializedValue<T>,PermanentBlobKey>
    serializeAndTryOffload(T value, org.apache.flink.api.common.JobID jobId, BlobWriter blobWriter)
    Serializes the given value and offloads it to the BlobServer if its size exceeds the minimum offloading size of the BlobServer.
    static <T> org.apache.flink.types.Either<org.apache.flink.util.SerializedValue<T>,PermanentBlobKey>
    tryOffload(org.apache.flink.util.SerializedValue<T> serializedValue, org.apache.flink.api.common.JobID jobId, BlobWriter blobWriter)
     

  • Field Details

    • LOG

      static final org.slf4j.Logger LOG

  • Method Details

    • putPermanent

      PermanentBlobKey putPermanent(org.apache.flink.api.common.JobID jobId, byte[] value) throws IOException
      Uploads the data of the given byte array for the given job to the BLOB server and makes it a permanent BLOB.
      Parameters:
      jobId - the ID of the job the BLOB belongs to
      value - the buffer to upload
      Returns:
      the computed BLOB key identifying the BLOB on the server
      Throws:
      IOException - thrown if an I/O error occurs while writing it to a local file, or uploading it to the HA store

    • putPermanent

      PermanentBlobKey putPermanent(org.apache.flink.api.common.JobID jobId, InputStream inputStream) throws IOException
      Uploads the data from the given input stream for the given job to the BLOB server and makes it a permanent BLOB.
      Parameters:
      jobId - ID of the job this blob belongs to
      inputStream - the input stream to read the data from
      Returns:
      the computed BLOB key identifying the BLOB on the server
      Throws:
      IOException - thrown if an I/O error occurs while reading the data from the input stream, writing it to a local file, or uploading it to the HA store

    • deletePermanent

      boolean deletePermanent(org.apache.flink.api.common.JobID jobId, PermanentBlobKey permanentBlobKey)
      Delete the uploaded data with the given JobID and PermanentBlobKey.
      Parameters:
      jobId - ID of the job this blob belongs to
      permanentBlobKey - the key of this blob

    • getMinOffloadingSize

      int getMinOffloadingSize()
      Returns the min size before data will be offloaded to the BLOB store.
      Returns:
      minimum offloading size

    • serializeAndTryOffload

      static <T> org.apache.flink.types.Either<org.apache.flink.util.SerializedValue<T>,PermanentBlobKey> serializeAndTryOffload(T value, org.apache.flink.api.common.JobID jobId, BlobWriter blobWriter) throws IOException
      Serializes the given value and offloads it to the BlobServer if its size exceeds the minimum offloading size of the BlobServer.
      Type Parameters:
      T - type of the value to serialize
      Parameters:
      value - to serialize
      jobId - to which the value belongs.
      blobWriter - to use to offload the serialized value
      Returns:
      Either the serialized value or the stored blob key
      Throws:
      IOException - if the data cannot be serialized

    • tryOffload

      static <T> org.apache.flink.types.Either<org.apache.flink.util.SerializedValue<T>,PermanentBlobKey> tryOffload(org.apache.flink.util.SerializedValue<T> serializedValue, org.apache.flink.api.common.JobID jobId, BlobWriter blobWriter)

    • offloadWithException

      static <T> org.apache.flink.types.Either<org.apache.flink.util.SerializedValue<T>,PermanentBlobKey> offloadWithException(org.apache.flink.util.SerializedValue<T> serializedValue, org.apache.flink.api.common.JobID jobId, BlobWriter blobWriter)