Package org.apache.flink.streaming.api.functions.sink.filesystem.legacy

Class StreamingFileSink.BulkFormatBuilder<IN,BucketID,T extends StreamingFileSink.BulkFormatBuilder<IN,BucketID,T>>

java.lang.Object

org.apache.flink.streaming.api.functions.sink.filesystem.legacy.StreamingFileSink.BucketsBuilder<IN,BucketID,T>

org.apache.flink.streaming.api.functions.sink.filesystem.legacy.StreamingFileSink.BulkFormatBuilder<IN,BucketID,T>

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

StreamingFileSink.DefaultBulkFormatBuilder

Enclosing class:

StreamingFileSink<IN>

@Internal
public static class StreamingFileSink.BulkFormatBuilder<IN,BucketID,T extends StreamingFileSink.BulkFormatBuilder<IN,BucketID,T>>
extends StreamingFileSink.BucketsBuilder<IN,BucketID,T>

A builder for configuring the sink for bulk-encoding formats, e.g. Parquet/ORC.

See Also:

• Serialized Form

Field Summary

Fields inherited from class org.apache.flink.streaming.api.functions.sink.filesystem.legacy.StreamingFileSink.BucketsBuilder

DEFAULT_BUCKET_CHECK_INTERVAL

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	BulkFormatBuilder(org.apache.flink.core.fs.Path basePath, org.apache.flink.api.common.serialization.BulkWriter.Factory<IN> writerFactory, org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner<IN,BucketID> assigner)
protected	BulkFormatBuilder(org.apache.flink.core.fs.Path basePath, org.apache.flink.api.common.serialization.BulkWriter.Factory<IN> writerFactory, org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner<IN,BucketID> assigner, org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.CheckpointRollingPolicy<IN,BucketID> policy, long bucketCheckInterval, BucketFactory<IN,BucketID> bucketFactory, org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig outputFileConfig)

Method Summary

Modifier and Type	Method	Description
StreamingFileSink<IN>	build()	Creates the actual sink.
Buckets<IN,BucketID>	createBuckets(int subtaskIndex)
org.apache.flink.streaming.api.functions.sink.filesystem.BucketWriter<IN,BucketID>	createBucketWriter()
long	getBucketCheckInterval()
T	withBucketAssigner(org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner<IN,BucketID> assigner)
T	withBucketCheckInterval(long interval)
T	withBucketFactory(BucketFactory<IN,BucketID> factory)
<ID> StreamingFileSink.BulkFormatBuilder<IN,ID,? extends StreamingFileSink.BulkFormatBuilder<IN,ID,?>>	withNewBucketAssigner(org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner<IN,ID> assigner)
T	withOutputFileConfig(org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig outputFileConfig)
T	withRollingPolicy(org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.CheckpointRollingPolicy<IN,BucketID> rollingPolicy)

Methods inherited from class org.apache.flink.streaming.api.functions.sink.filesystem.legacy.StreamingFileSink.BucketsBuilder

self

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

BulkFormatBuilder

protected BulkFormatBuilder(org.apache.flink.core.fs.Path basePath,
 org.apache.flink.api.common.serialization.BulkWriter.Factory<IN> writerFactory,
 org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner<IN,BucketID> assigner)

BulkFormatBuilder

protected BulkFormatBuilder(org.apache.flink.core.fs.Path basePath,
 org.apache.flink.api.common.serialization.BulkWriter.Factory<IN> writerFactory,
 org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner<IN,BucketID> assigner,
 org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.CheckpointRollingPolicy<IN,BucketID> policy,
 long bucketCheckInterval,
 BucketFactory<IN,BucketID> bucketFactory,
 org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig outputFileConfig)

Method Details

getBucketCheckInterval

public long getBucketCheckInterval()

withBucketCheckInterval

public T withBucketCheckInterval(long interval)

withBucketAssigner

public T withBucketAssigner(org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner<IN,BucketID> assigner)

withRollingPolicy

public T withRollingPolicy(org.apache.flink.streaming.api.functions.sink.filesystem.rollingpolicies.CheckpointRollingPolicy<IN,BucketID> rollingPolicy)

withBucketFactory

@VisibleForTesting
public T withBucketFactory(BucketFactory<IN,BucketID> factory)

withOutputFileConfig

public T withOutputFileConfig(org.apache.flink.streaming.api.functions.sink.filesystem.OutputFileConfig outputFileConfig)

withNewBucketAssigner

public <ID>
StreamingFileSink.BulkFormatBuilder<IN,ID,? extends StreamingFileSink.BulkFormatBuilder<IN,ID,?>> withNewBucketAssigner(org.apache.flink.streaming.api.functions.sink.filesystem.BucketAssigner<IN,ID> assigner)

build

public StreamingFileSink<IN> build()

Creates the actual sink.

createBucketWriter

@Internal
public org.apache.flink.streaming.api.functions.sink.filesystem.BucketWriter<IN,BucketID> createBucketWriter()
                                                                                                      throws IOException

Specified by:

createBucketWriter in class StreamingFileSink.BucketsBuilder<IN,BucketID,T extends StreamingFileSink.BulkFormatBuilder<IN,BucketID,T>>

Throws:

IOException

createBuckets

@Internal
public Buckets<IN,BucketID> createBuckets(int subtaskIndex)
                                   throws IOException

Specified by:

createBuckets in class StreamingFileSink.BucketsBuilder<IN,BucketID,T extends StreamingFileSink.BulkFormatBuilder<IN,BucketID,T>>

Throws:

IOException