Package org.apache.flink.orc.writer

Class PhysicalWriterImpl

java.lang.Object
org.apache.flink.orc.writer.PhysicalWriterImpl
All Implemented Interfaces:
org.apache.orc.PhysicalWriter
@Internal public class PhysicalWriterImpl extends Object implements org.apache.orc.PhysicalWriter
A slightly customised clone of PhysicalFsWriter.

Whereas PhysicalFsWriter implementation works on the basis of a Path, this implementation leverages Flink's FSDataOutputStream to write the compressed data.

NOTE: If the ORC dependency version is updated, this file may have to be updated as well to be in sync with the new version's PhysicalFsWriter.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface org.apache.orc.PhysicalWriter

    org.apache.orc.PhysicalWriter.OutputReceiver

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected final org.apache.orc.impl.OutStream
    writer
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    PhysicalWriterImpl(org.apache.flink.core.fs.FSDataOutputStream out, org.apache.orc.OrcFile.WriterOptions opts)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    appendRawStripe(ByteBuffer buffer, org.apache.orc.OrcProto.StripeInformation.Builder dirEntry)
     
    void
    close()
     
    org.apache.orc.PhysicalWriter.OutputReceiver
    createDataStream(org.apache.orc.impl.StreamName name)
     
    void
    finalizeStripe(org.apache.orc.OrcProto.StripeFooter.Builder footerBuilder, org.apache.orc.OrcProto.StripeInformation.Builder dirEntry)
     
    void
    flush()
     
    org.apache.orc.CompressionCodec
    getCompressionCodec()
     
    long
    getFileBytes(int column)
     
    void
    writeBloomFilter(org.apache.orc.impl.StreamName name, org.apache.orc.OrcProto.BloomFilterIndex.Builder bloom, org.apache.orc.CompressionCodec codec)
     
    protected void
    writeFileFooter(org.apache.orc.OrcProto.Footer footer)
     
    void
    writeFileFooter(org.apache.orc.OrcProto.Footer.Builder builder)
     
    void
    writeFileMetadata(org.apache.orc.OrcProto.Metadata.Builder builder)
     
    void
    writeHeader()
     
    void
    writeIndex(org.apache.orc.impl.StreamName name, org.apache.orc.OrcProto.RowIndex.Builder index, org.apache.orc.CompressionCodec codec)
     
    protected void
    writeMetadata(org.apache.orc.OrcProto.Metadata metadata)
     
    long
    writePostScript(org.apache.orc.OrcProto.PostScript.Builder builder)
     
    protected void
    writeStripeFooter(org.apache.orc.OrcProto.StripeFooter footer)
     

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • writer

      protected final org.apache.orc.impl.OutStream writer

  • Constructor Details

    • PhysicalWriterImpl

      public PhysicalWriterImpl(org.apache.flink.core.fs.FSDataOutputStream out, org.apache.orc.OrcFile.WriterOptions opts) throws IOException
      Throws:
      IOException

  • Method Details

    • writeHeader

      public void writeHeader() throws IOException
      Specified by:
      writeHeader in interface org.apache.orc.PhysicalWriter
      Throws:
      IOException

    • createDataStream

      public org.apache.orc.PhysicalWriter.OutputReceiver createDataStream(org.apache.orc.impl.StreamName name) throws IOException
      Specified by:
      createDataStream in interface org.apache.orc.PhysicalWriter
      Throws:
      IOException

    • writeIndex

      public void writeIndex(org.apache.orc.impl.StreamName name, org.apache.orc.OrcProto.RowIndex.Builder index, org.apache.orc.CompressionCodec codec) throws IOException
      Specified by:
      writeIndex in interface org.apache.orc.PhysicalWriter
      Throws:
      IOException

    • writeBloomFilter

      public void writeBloomFilter(org.apache.orc.impl.StreamName name, org.apache.orc.OrcProto.BloomFilterIndex.Builder bloom, org.apache.orc.CompressionCodec codec) throws IOException
      Specified by:
      writeBloomFilter in interface org.apache.orc.PhysicalWriter
      Throws:
      IOException

    • finalizeStripe

      public void finalizeStripe(org.apache.orc.OrcProto.StripeFooter.Builder footerBuilder, org.apache.orc.OrcProto.StripeInformation.Builder dirEntry) throws IOException
      Specified by:
      finalizeStripe in interface org.apache.orc.PhysicalWriter
      Throws:
      IOException

    • writeFileMetadata

      public void writeFileMetadata(org.apache.orc.OrcProto.Metadata.Builder builder) throws IOException
      Specified by:
      writeFileMetadata in interface org.apache.orc.PhysicalWriter
      Throws:
      IOException

    • writeFileFooter

      public void writeFileFooter(org.apache.orc.OrcProto.Footer.Builder builder) throws IOException
      Specified by:
      writeFileFooter in interface org.apache.orc.PhysicalWriter
      Throws:
      IOException

    • writePostScript

      public long writePostScript(org.apache.orc.OrcProto.PostScript.Builder builder) throws IOException
      Specified by:
      writePostScript in interface org.apache.orc.PhysicalWriter
      Throws:
      IOException

    • close

      public void close()
      Specified by:
      close in interface org.apache.orc.PhysicalWriter

    • flush

      public void flush() throws IOException
      Specified by:
      flush in interface org.apache.orc.PhysicalWriter
      Throws:
      IOException

    • appendRawStripe

      public void appendRawStripe(ByteBuffer buffer, org.apache.orc.OrcProto.StripeInformation.Builder dirEntry) throws IOException
      Specified by:
      appendRawStripe in interface org.apache.orc.PhysicalWriter
      Throws:
      IOException

    • getCompressionCodec

      public org.apache.orc.CompressionCodec getCompressionCodec()
      Specified by:
      getCompressionCodec in interface org.apache.orc.PhysicalWriter

    • getFileBytes

      public long getFileBytes(int column)
      Specified by:
      getFileBytes in interface org.apache.orc.PhysicalWriter

    • writeMetadata

      protected void writeMetadata(org.apache.orc.OrcProto.Metadata metadata) throws IOException
      Throws:
      IOException

    • writeFileFooter

      protected void writeFileFooter(org.apache.orc.OrcProto.Footer footer) throws IOException
      Throws:
      IOException

    • writeStripeFooter

      protected void writeStripeFooter(org.apache.orc.OrcProto.StripeFooter footer) throws IOException
      Throws:
      IOException