parquet.avro

Class AvroParquetOutputFormat

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<K,V>

org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<Void,T>

parquet.hadoop.ParquetOutputFormat<org.apache.avro.generic.IndexedRecord>

parquet.avro.AvroParquetOutputFormat

public class AvroParquetOutputFormat
extends ParquetOutputFormat<org.apache.avro.generic.IndexedRecord>

A Hadoop OutputFormat for Parquet files.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.Counter

Field Summary

Fields inherited from class parquet.hadoop.ParquetOutputFormat

BLOCK_SIZE, COMPRESSION, DICTIONARY_PAGE_SIZE, ENABLE_DICTIONARY, ENABLE_JOB_SUMMARY, MEMORY_POOL_RATIO, MIN_MEMORY_ALLOCATION, PAGE_SIZE, VALIDATION, WRITE_SUPPORT_CLASS, WRITER_VERSION

Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

BASE_OUTPUT_NAME, PART

Constructor Summary

Constructors

Constructor and Description
AvroParquetOutputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
static void	setSchema(org.apache.hadoop.mapreduce.Job job, org.apache.avro.Schema schema) Set the Avro schema to use for writing.

Methods inherited from class parquet.hadoop.ParquetOutputFormat

getBlockSize, getBlockSize, getCompression, getCompression, getDictionaryPageSize, getDictionaryPageSize, getEnableDictionary, getEnableDictionary, getLongBlockSize, getOutputCommitter, getPageSize, getPageSize, getRecordWriter, getRecordWriter, getRecordWriter, getValidation, getValidation, getWriterVersion, getWriteSupport, getWriteSupportClass, isCompressionSet, isCompressionSet, setBlockSize, setCompression, setDictionaryPageSize, setEnableDictionary, setPageSize, setValidation, setValidation, setWriteSupportClass, setWriteSupportClass

Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat

checkOutputSpecs, getCompressOutput, getDefaultWorkFile, getOutputCompressorClass, getOutputName, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputName, setOutputPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AvroParquetOutputFormat

public AvroParquetOutputFormat()

Method Detail

setSchema

public static void setSchema(org.apache.hadoop.mapreduce.Job job,
             org.apache.avro.Schema schema)

Set the Avro schema to use for writing. The schema is translated into a Parquet schema so that the records can be written in Parquet format. It is also stored in the Parquet metadata so that records can be reconstructed as Avro objects at read time without specifying a read schema.

Parameters:

job -

schema -

See Also:

AvroParquetInputFormat.setAvroReadSchema(org.apache.hadoop.mapreduce.Job, org.apache.avro.Schema)