Package org.apache.flink.runtime.io.network.partition
package org.apache.flink.runtime.io.network.partition
-
ClassDescriptionBase partition tracker implementation, providing underlying data-structures for storing partitions, their associated keys and meta-information.A output data result of an individual task (one partition of an intermediate result), produced and communicated in a batch manner: The result must be produced completely before it can be consumed.The reader (read view) of a BoundedBlockingSubpartition based on
FileRegion.The type of the BoundedBlockingSubpartition.Listener interface implemented by consumers ofResultSubpartitionViewthat want to be notified of availability of further buffers.Putting and getting of a sequence of buffers to/from a FileChannel or a ByteBuffer.Buffer and the corresponding subpartition index.AResultPartitionwhich writes buffers directly toResultSubpartitions.Implemented by entities that hold any kind of channel state and need a reference to theChannelStateWriter.Interface for partitions that are checkpointed, meaning they store data as part of unaligned checkpoints.Interface for components that manage cluster partitions.Data of different subpartitions can be appended to aDataBufferand after theDataBufferis full or finished, the appended data can be copied from it in subpartition index order.Container for meta-data of a data set.A basic queue with duplicated elements ignored duringDeduplicatedQueue.add(Object).* ADataBufferimplementation which sorts all appended records only by subpartition index.Utility for tracking partitions and issuing release calls to task executors and shuffle masters.Utility for tracking partitions and issuing release calls to task executors and shuffle masters.A dummy implementation of theResultSubpartitionView.Utility for accessing the system page size.PartitionedFileis the persistent file type of sort-merge based blocking shuffle.File writer which can write buffers and generatePartitionedFile.Exception for covering all the scenarios of consuming partition failure which causes the consumer task failed, and the job master would decide whether to restart the producer based on this exception.Exception for failed partition requests due to non-existing partitions.Request execution state of partition producer, the response accepts state check callbacks.Result of state query, accepts state check callbacks.When the netty server receives a downstream task's partition request event and finds its upstream task doesn't register its partition yet, the netty server will construct aPartitionRequestListenerand notify the listener when the task deploys itself and registers its partition toResultPartitionManager.Manages partition request listener with input channel id.PartitionTracker<K,M> Utility for tracking partitions.Encapsulates all information that aPartitionTrackerkeeps for a partition.Factory forJobMasterPartitionTracker.Lookup function forTaskExecutorGateway.A pipelined in-memory only subpartition, which allows to reconnect after failure.View over a pipelined in-memory only subpartition allowing reconnecting.A result output of a task, pipelined (streamed) to the receivers.A pipelined in-memory only subpartition, which can be consumed once.View over a pipelined in-memory only subpartition.A deque-like data structure that supports prioritization of elements, such they will be polled before any non-priority elements.Network-stack level Exception to notify remote receiver about a failed partition producer.Utility for tracking and releasing partitions on the ResourceManager.Factory forResourceManagerPartitionTracker.DefaultResourceManagerPartitionTrackerimplementation.A result partition for data produced by a single task.Factory forResultPartitionto use inNettyShuffleEnvironment.Runtime identifier of a producedIntermediateResultPartition.The result partition manager keeps track of all currently produced/consumed partitions of a task manager.Interface for creating result partitions.Type of a result partition.A single subpartition of aResultPartitioninstance.A combination of aBufferand the backlog length indicating how many non-event buffers are available in the subpartition.A collection of subpartition indexes.A view to consume aResultSubpartitioninstance.Availability of theResultSubpartitionViewand the backlog in the correspondingResultSubpartition.ASubpartitionSelectorthat selects all subpartitions in round-robin order.When getting buffers, TheSortBasedDataBuffershould recycle the read target buffer with the givenBufferRecycler.ADataBufferimplementation which sorts all appended records only by subpartition index.SortMergeResultPartitionappends records and events toDataBufferand after theDataBufferis full, all data in theDataBufferwill be copied and spilled to aPartitionedFilein subpartition index order sequentially.SubpartitionSelectorhelps to choose from multiple available subpartitions when their output buffers should union into one stream.Interface for releasing cluster partitions on a task executor.Encapsulates meta-information the TaskExecutor requires to be kept for each partition.Utility for tracking partitions.Utility for tracking partitions and issuing release calls to task executors and shuffle masters.A wrapper to union the output from multipleResultSubpartitionViews.