Package org.apache.flink.orc

Class AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT>

java.lang.Object
org.apache.flink.orc.AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT>
Enclosing class:
AbstractOrcFileInputFormat<T,BatchT,SplitT extends org.apache.flink.connector.file.src.FileSourceSplit>
protected abstract static class AbstractOrcFileInputFormat.OrcReaderBatch<T,BatchT> extends Object
The OrcReaderBatch class holds the data structures containing the batch data (column vectors, row arrays, ...) and performs the batch conversion from the ORC representation to the result format.

This base class only holds the ORC Column Vectors, subclasses hold additionally the result structures and implement the conversion in convertAndGetIterator(OrcVectorizedBatchWrapper, long).

  • Constructor Details

  • Method Details

    • recycle

      public void recycle()
      Puts this batch back into the pool. This should be called after all records from the batch have been returned, typically in the BulkFormat.RecordIterator.releaseBatch() method.

    • orcVectorizedRowBatch

      public OrcVectorizedBatchWrapper<BatchT> orcVectorizedRowBatch()
      Gets the ORC VectorizedRowBatch structure from this batch.

    • convertAndGetIterator

      public abstract org.apache.flink.connector.file.src.reader.BulkFormat.RecordIterator<T> convertAndGetIterator(OrcVectorizedBatchWrapper<BatchT> orcVectorizedRowBatch, long startingOffset) throws IOException
      Converts the ORC VectorizedRowBatch into the result structure and returns an iterator over the entries.

      This method may, for example, return a single element iterator that returns the entire batch as one, or (as another example) return an iterator over the rows projected from this column batch.

      The position information in the result needs to be constructed as follows: The value of startingOffset is the offset value (RecordAndPosition.getOffset()) for all rows in the batch. Each row then increments the records-to-skip value (RecordAndPosition.getRecordSkipCount()).

      Throws:
      IOException