All Classes and Interfaces
Class
Description
AbstractAccessExecutionGraphHandler handles requests that require accessing the job's
AccessExecutionGraph.Based
MessageHeaders class for aggregating metrics.Base
MessageParameters class for aggregating metrics.Abstract request handler for querying aggregated metrics.
The default implementation of
AggregatingState, which delegates all async requests to
StateRequestHandler.HTTP handlers for asynchronous operations.
This operator is an abstract class that give the
AbstractStreamOperator the ability to
perform AsyncStateProcessing.This operator is an abstract class that give the
AbstractStreamOperatorV2 the ability to
perform AsyncStateProcessing.This is used as the base class for operators that have a user-defined function.
Abstract base class for permanent and transient BLOB files.
Base class for Broadcast State transformations.
Abstract gauge implementation for calculating the buffer usage percent.
A
DataInputView that is backed by a FileIOChannel,
making it effectively a data input stream.Abstract channel state handle.
Describes the underlying content.
A
DataOutputView that is backed by a FileIOChannel,
making it effectively a data output stream.Base class for checkpoint related REST handler.
The base class of all checkpoint messages.
Base class for checkpoint statistics.
Abstract class for checkpoint handlers that will cache the
CheckpointStatsSnapshot
object.A buffer which encapsulates the logic of dealing with the response from the
CollectSinkFunction.The abstract class of
CompletedCheckpointStore, which holds the SharedStateRegistry and provides the registration of shared state.An implementation of
Buffer which contains multiple partial buffers for network data
communication.A base
DispatcherLeaderProcess.An accessor of the
DispatcherGateway.The state of the
DispatcherLeaderProcess.This type of event can be used to exchange notification messages between different
TaskExecutor objects at runtime using the communication channels.Base class for all
ExecutionGraphInfo based REST handlers.Deprecated.
An implementation of durable checkpoint storage to file systems.
Super class for netty-based handlers that work with
RequestBody.Abstract high availability services based on distributed system(e.g.
Abstract base class for heap (object array) based implementations of priority queues, with
support for fast deletes via
HeapPriorityQueueElement.Abstract base class for
HeapPriorityQueueElement.Base class for partitioned
State implementations that are backed by a regular heap hash
map.Abstract superclass for all
IncrementalKeyedStateHandle.The
AbstractInternalWatermarkDeclaration class implements the
WatermarkDeclaration interface and provides some internal-oriented methods, such as
getting/setting the align flag and creating the WatermarkCombiner.A base implementation of
TaskInvokable, CheckpointableTask, and CoordinatedTask with most methods throwing UnsupportedOperationException or doing
nothing.The abstract base class for all tasks able to participate in an iteration.
Base class for serving files from the JobManager.
Base class for request handlers whose response depends on a specific job vertex (defined via the
"
JobVertexIdPathParameter.KEY" parameter) in a specific job, defined via (defined via the
"JobIDPathParameter.KEY" parameter).The
AbstractKeyedState is the root of the abstract state implementation hierarchy,
similar to the State being the root of the public API state hierarchy.Base implementation of KeyedStateBackend.
An abstract base implementation of the
StateBackendBuilder interface.A default implementation of
ListState which delegates all async requests to StateRequestHandler.Abstract base class for state backends that use managed memory.
A default implementation of
MapState which delegates all async requests to StateRequestHandler.An implementation of the
JoinTaskIterator that
realizes the joining through a sort-merge join strategy.An implementation of the
JoinTaskIterator that
realizes the outer join through a sort-merge join strategy.Abstract
MetricGroup that contains key functionality for adding metrics and groups.Enum for indicating which child group should be created.
Request handler that returns for a given task a list of all available metrics or the values for a
set of metrics.
Base class for transformations representing the application of a
MultipleInputStreamOperator to input
Transformations.Abstract base class for non high-availability services.
The abstract outer join driver implements the logic of an outer join operator at runtime.
The base class for all input views that are backed by multiple memory pages.
The base class for all output views that are backed by multiple memory pages.
Base partition tracker implementation, providing underlying data-structures for storing
partitions, their associated keys and meta-information.
A basic reader implementation, which wraps an input gate and handles events.
A default implementation of
ReducingState which delegates all async requests to StateRequestHandler.Abstract common base class for implementations of
ResourceManagerDriver.AbstractResourceManagerHandler<T extends RestfulGateway,R extends RequestBody,P extends ResponseBody,M extends MessageParameters>
Base class for REST handlers which need access to the
ResourceManager.AbstractRestHandler<T extends RestfulGateway,R extends RequestBody,P extends ResponseBody,M extends MessageParameters>
Super class for netty-based handlers that work with
RequestBodys and ResponseBodys.Base class for service connection managers, taking care of the connection handling.
Abstract SlotPoolServiceFactory.
An abstract base implementation of the
StateBackend interface.A
StateIterator implementation to facilitate async data load of iterator.Base class for all stream operators.
Base class for all stream operator factories.
New base class for all stream operators, intended to eventually replace
AbstractStreamOperator.AbstractStreamTaskNetworkInput<T,R extends RecordDeserializer<DeserializationDelegate<StreamElement>>>
Base class for network-based StreamTaskInput where each channel has a designated
RecordDeserializer for spanning records.Base class for request handlers whose response depends on a specific attempt (defined via the
"
SubtaskAttemptPathParameter.KEY" of a specific subtask (defined via the "SubtaskIndexPathParameter.KEY" in a specific job vertex, (defined via the "JobVertexIdPathParameter.KEY" parameter) in a specific job, defined via (defined via the "JobIDPathParameter.KEY" parameter).Base class for request handlers whose response depends on a specific subtask (defined via the
"
SubtaskIndexPathParameter.KEY" in a specific job vertex, (defined via the "JobVertexIdPathParameter.KEY" parameter) in a specific job, defined via (defined via the "JobIDPathParameter.KEY" parameter).Base class for serving files from the
TaskExecutor.An abstract class for threadsafe implementations of the
JobResultStore.Base class for TTL logic wrappers.
Abstract class for executing a
TwoInputStreamOperator.AbstractTwoInputTransformationTranslator<IN1,IN2,OUT,OP extends org.apache.flink.api.dag.Transformation<OUT>>
A base class with functionality used during translating
transformations
with two inputs.This is used as the base class for operators that have a user-defined function.
Abstract class for the JobGraphRetriever which supports getting user classpaths.
A default implementation of
ValueState which delegates all async requests to AsyncExecutionController.Common interface for the runtime
Execution and ArchivedExecution.Common interface for the runtime
DefaultExecutionGraph and ArchivedExecutionGraph.Common interface for the runtime
ExecutionJobVertex and ArchivedExecutionJobVertex.Common interface for the runtime
ExecutionVertex and ArchivedExecutionVertex.Main accumulator registry which encapsulates user-defined accumulators.
A report about the current values of all accumulators of the TaskExecutor for a given job.
Query parameter for job's accumulator handler
JobAccumulatorsHandler.This class encapsulates a map of accumulators for a single task.
A generic acknowledgement message.
This message is sent from the
TaskExecutor to the
JobMaster to signal that the checkpoint of an
individual task is completed.An
OperatorEvent sent from a subtask to its OperatorCoordinator to signal that
the checkpoint of an individual task is completed.A
FileMergingSnapshotManager that merging files across checkpoints.An active implementation of
ResourceManager.Factory class for creating
ActiveResourceManager with various implementations of ResourceManagerDriver.This scheduler decides the parallelism of JobVertex according to the data volume it consumes.
Factory for
AdaptiveBatchScheduler.The
AdaptiveExecutionHandler interface defines the operations for handling the adaptive
execution of batch jobs.A factory class for creating instances of
AdaptiveExecutionHandler.The
AdaptiveExecutionPlanSchedulingContext class implements the ExecutionPlanSchedulingContext interface to provide a dynamic scheduling context that adapts
execution plans based on runtime conditions.Defines the mechanism for dynamically adapting the graph topology of a Flink job at runtime.
Default implementation for
AdaptiveGraphGenerator.A
SchedulerNG implementation that uses the declarative resource management and
automatically adapts the parallelism in case not enough resource could be acquired to run at the
configured parallelism, as described in FLIP-160.Consolidated settings for the adaptive scheduler.
Factory for the adaptive scheduler.
A source event that adds splits to a source reader.
Helper class that aggregates input information with the same typeNumber so that they can be
processed as a single unit.
Headers for aggregating job metrics.
Parameters for aggregating job metrics.
Response type for aggregated metrics.
Response type for a collection of aggregated metrics.
JSON deserializer for
AggregatedMetricsResponseBody.JSON serializer for
AggregatedMetricsResponseBody.Headers for aggregating subtask metrics.
Parameters for aggregating subtask metrics.
Aggregated info of a set of tasks.
Headers for aggregating task manager metrics.
Parameters for aggregating task manager metrics.
Request handler that returns, aggregated across jobs, a list of all available metrics or the
values for a set of metrics.
An adaptor that transforms
InternalAggregatingState into InternalAggregatingState.Request handler that returns, aggregated across subtasks, a list of all available metrics or the
values for a set of metrics.
Request handler that returns, aggregated across task managers, a list of all available metrics or
the values for a set of metrics.
Holder class for aggregation types that can be used on a windowed stream or keyed stream.
Aggregation types that can be used on a windowed stream or keyed stream.
Flink compressor that wraps
Compressor.Flink decompressor that wraps
Decompressor.An interface used to represent the special
WatermarkDeclarations can create aligned
Watermarks, which need to be aligned when the operator receives them from input channels.An alignable
BoolWatermarkDeclaration.An alignable
LongWatermarkDeclaration.A
WatermarkCombiner is design to align Watermarks.AllFinishedInputConsumableDecider is a special InputConsumableDecider.Factory for
AllFinishedInputConsumableDecider.Non-chained driver for the partial group reduce operator that acts like a combiner with a custom
output type OUT.
GroupReduceDriver task which is executed by a Task Manager.
Interface for components which have to perform actions on allocated slots.
Information about an allocated slot which is owned by a JobMaster.
The slot pool is responsible for maintaining a set of
AllocatedSlots.A collection of
AllocatedSlots and their reservation status.Information about a free slot.
The report of currently allocated slots from a given TaskExecutor by a JobMaster.
Unique identifier for a physical slot allocated by a JobManager via the ResourceManager from a
TaskManager.
Reduce task which is executed by a Task Manager.
This is a collection of all
TieredShuffleMasterSnapshots from every tier in one snapshot
round.Information of All-To-All result.
Helper class that computes VertexInputInfo for all to all like inputs.
A
AllWindowedStream represents a data stream where the stream of elements is split into
windows based on a WindowAssigner.Base interface for functions that are evaluated over non-keyed windows.
Event sent by the
IterationSynchronizationSinkTask to each IterationHead
signaling to start a new superstep.The status of an application.
Implementation of
WorkerResourceSpecFactory that creates arbitrary WorkerResourceSpec.ArchivedExecution is a readonly representation of Execution.An archived execution graph represents a serializable form of an
ExecutionGraph.ArchivedExecutionVertex is a readonly representation of ExecutionVertex.A simple container for a handler's JSON response and the REST URLs for which the response
would've been returned.
Snapshot class for the
ArrayListSerializer.This mapPartition function require a DataSet with RangeBoundaries as broadcast input, it generate
Tuple2 which includes range index and record itself as output.
A
AsyncTrigger that continuously fires based on a given time interval.A
AsyncTrigger that fires once the count of elements in a pane reaches the given count.A
AsyncTrigger that fires once the watermark passes the end of the window to which a pane
belongs.A
AsyncWindowOperator that also allows an Evictor to be used.An interface marking a task as capable of handling exceptions thrown by different threads, other
than the one executing the task itself.
The Async Execution Controller (AEC) receives processing requests from operators, and put them
into execution according to some strategies.
A listener listens the key context switch.
A reader that reads data in blocks from a file channel.
An asynchronous implementation of the
BlockChannelWriterWithCallback that queues I/O
requests and calls a callback once they have been handled.An exception for wrapping exceptions that are thrown by an operator in threads other than the
main compute thread of that operator.
A base class for readers and writers that accept read or write requests for whole blocks.
A pair of
JobID and TriggerId used as a key to a hash based collection.Interface for REST resources that are created asynchronously.
Basic information object for asynchronous operations.
Result of an asynchronous operation.
Message headers for the status polling of an asynchronous operation.
Message headers for the triggering of an asynchronous operation.
An
operator to execute time-bounded stream inner joins.A
StreamOperator for executing KeyedProcessFunction with async state processing.An async keyed state backend provides methods supporting to access keyed state asynchronously and
in batch.
A adaptor that transforms
KeyedStateBackend into AsyncKeyedStateBackend.A
Trigger that fires once the current system time passes the end of the window to which a
pane belongs.A trigger that can turn any
AsyncTrigger into a purging Trigger.Base class that outlines the strategy for asynchronous snapshots.
An exception for wrapping exceptions that are thrown by
StateFuture callback framework.This class defines the basic interfaces to process a state in operator/input layer.
A more detailed interface based on
AsyncStateProcessing, which gives the essential
methods for an operator to perform async state processing.A
AbstractAsyncStateStreamOperator for executing FlatMapFunctions.A
Trigger determines when a pane of a window should be evaluated to emit the results for
that part of the window.Extension of
AsyncTrigger.TriggerContext that is given to AsyncTrigger.onMerge(Window, OnMergeContext).A context object that is given to
AsyncTrigger methods to allow them to register
timer callbacks and deal with state.An operator that implements the logic for windowing based on a
WindowAssigner and AsyncTrigger.AvailabilityNotifier is used to notify that the data from the specific partition and
subpartition in tiered storage is available.Interface defining couple of essential methods for listening on data availability using
CompletableFuture.A availability implementation for providing the helpful functions of resetting the
available/unavailable states.
Exceptions which indicate that a state backend building has failed.
BackendRestorerProcedure<T extends Closeable & org.apache.flink.util.Disposable,S extends StateObject>
This class implements the logic that creates (and potentially restores) a state backend.
An interface with methods related to the interplay between the
Broadcast
State and the OperatorStateBackend.Utility for barrier alignment.
A handle to a delayed action which can be cancelled.
It can register a task to be executed some time later.
The base class containing the functionality available to all broadcast process function.
BaseTopology<VID extends VertexID,RID extends ResultID,V extends Vertex<VID,RID,V,R>,R extends Result<VID,RID,V,R>>
Base topology for all logical and execution topologies.
Utility class for using java utilities in bash scripts.
Commands that BashJavaUtils supports.
A
TwoInputStreamOperator for executing KeyedBroadcastProcessFunctions in RuntimeExecutionMode.BATCH
execution mode.A
TwoInputStreamOperator for executing BroadcastProcessFunctions in RuntimeExecutionMode.BATCH
execution mode.A simple
CheckpointStorage which is used in a BATCH style execution.An implementation of a
InternalTimerService that manages timers with a single active key
at a time.An implementation of a
InternalTimeServiceManager that manages timers with a single
active key at a time.An implementation of a
InternalTimerService that manages timers with a single active key
at a time.A
CheckpointableKeyedStateBackend which keeps values for a single key at a time.Internal configuration options for the batch job execution.
A simple
StateBackend which is used in a BATCH style execution.A utility class for applying sorting inputs.
A
StreamOperator for executing a ReduceFunction on a KeyedStream in a RuntimeExecutionMode.BATCH
mode.Context for batch job recovery.
Interface for handling batch job recovery.
A fixed-size
MemorySegment pool used by batch shuffle for shuffle data read (currently
only used by sort-merge blocking shuffle).The base class for all batch tasks.
Utility class for bisection search.
The BLOB cache provides access to BLOB services for permanent and transient BLOBs.
BlobCacheSizeTracker uses
LinkedHashMap to maintain the LRU order for the files in the
cache.The BLOB client can communicate with the BLOB server and either upload (PUT), download (GET), or
delete (DELETE) BLOBs.
A BLOB key uniquely identifies a BLOB.
Provides facilities to download a set of libraries (typically JAR files) for a job from a
PermanentBlobService and create a class loader with references to them.This class implements the BLOB server.
A simple store and retrieve binary large objects (BLOBs).
A blob store.
Service interface for the BlobStore which allows to close and clean up its data.
Utility class to work with blob data.
View on blobs stored in a
BlobStore.BlobWriter is used to upload data to the BLOB store.
A reader that reads data in blocks from a file channel.
A writer that writes data in blocks to a file channel.
Each compression codec has a implementation of
BlockCompressionFactory to create
compressors and decompressors.A compressor which compresses a whole byte array each time.
A decompressor which decompresses a block each time.
This class represents a blocked node record.
This class helps to retrieve the blocked nodes.
This checker helps to query whether a given task manager is blocked.
A concurrent datastructure that establishes a backchannel buffer between an iteration head and an
iteration tail.
Singleton class for the threadsafe handover of
BlockingBackChannels from iteration heads
to iteration tails.Helper class that provides read-only information of input for
VertexParallelismAndInputInfosDecider.A Blocking
PhysicalFilePool which may block when polling physical files.Broker for the BlockingQueue used in StreamIterationHead and StreamIterationTail for implementing feedback streams.The blocking result info, which will be used to calculate the vertex parallelism and input infos.
This class is responsible for blocking and unblocking resources.
A
DeclarativeSlotPool implementation that supports blocklist.Factory for
BlocklistDeclarativeSlotPool.This class is responsible for managing all
BlockedNodes and performing them on resources.Factory to instantiate
BlocklistHandler.A listener that want to be notified when blocklist changes.
Operations to perform on the blocklist.
A tracker for blocklist.
Utility class for blocklist.
Implementation of an iterator that fetches a block of data into main memory and offers resettable
access to the data in that block.
BloomFilter is a probabilistic data structure for set membership check.
Minimal implementation of an array-backed list of booleans
A
WatermarkCombiner for unaligned BoolWatermarks.Tools for starting JobManager and TaskManager processes.
A output data result of an individual task (one partition of an intermediate result), produced
and communicated in a batch manner: The result must be produced completely before it can be
consumed.
The reader (read view) of a BoundedBlockingSubpartition based on
FileRegion.The type of the BoundedBlockingSubpartition.
BoundedFIFOQueue collects elements up to given amount.Wrapper around a FSDataInputStream to limit the maximum read offset.
Interface for multi-input operators that need to be notified about the logical/semantical end of
input.
Interface for one-input operators that need to be notified about the logical/semantical end of
input.
A BroadcastConnectedStream represents the result of connecting a keyed or non-keyed stream, with
a
BroadcastStream with broadcast
state(s).Partitioner that selects all the output channels.
A function to be applied to a
BroadcastConnectedStream that
connects BroadcastStream, i.e.BroadcastRecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>
A special record-oriented runtime result writer only for broadcast mode.
A non-keyed
AbstractBroadcastStateTransformation.A
BroadcastStream is a stream with broadcast state(s).An identifier for a
BroadcastVariableMaterialization based on the task's JobVertexID, broadcast variable name and iteration superstep.The BroadcastVariableManager is used to manage the materialization of broadcast variables.
This class represents a single materialization of a broadcast variable and maintains a reference
count for it.
A concurrent data structure that allows the hand-over of an object between a pair of threads.
Wrapper for pooled
MemorySegment instances with reference counting.Used to identify the type of data contained in the
Buffer so that we can get the
information without deserializing the serialized data.Accumulates received records into buffers.
Listener interface implemented by consumers of
ResultSubpartitionView that want to be
notified of availability of further buffers.Not thread safe class for filling in the content of the
MemorySegment.A
BufferCompressionException is thrown when the target data cannot be compressed, such as
insufficient target buffer space for compression, etc.Compressor for
Buffer.Not thread safe class for producing
Buffer.BufferConsumer with partial record length if a record is spanning over buffers
Configuration for
BufferDebloater.Class for automatic calculation of the buffer size based on the current throughput and
configuration.
A
BufferDecompressionException is thrown when the target data cannot be decompressed,
such as data corruption, insufficient target buffer space for decompression, etc.Decompressor for compressed
Buffer.Helper class to read
Buffers from files into objects.Header fields of a
Buffer.Interface of the availability of buffers.
The general buffer manager used by
InputChannel to request/recycle exclusive or floating
buffers.Either type for
Buffer or AbstractEvent instances tagged with the channel index,
from which they were received.A dynamically sized buffer pool.
A factory for buffer pools.
A buffer provider to request buffers from in a synchronous or asynchronous fashion.
Putting and getting of a sequence of buffers to/from a FileChannel or a ByteBuffer.
Interface for recycling
MemorySegments.The buffer recycler does nothing for recycled segment.
Implementation of 'Exponential moving average' algorithm.
Buffer and the corresponding subpartition index.
A
ResultPartition which writes buffers directly to ResultSubpartitions.Utility routines to process
ByteBuf.A state handle that contains stream state in a byte array.
CachedDataStream represents a DataStream whose intermediate result will be cached
at the first time when it is computed.ShuffleDescriptors cache for a ConsumedPartitionGroup.When in batch mode, the
CacheTransformation represents the intermediate result of the
upper stream should be cached when it is computed at the first time.Translator for
CacheTransformationTranslator.The
CacheTransformationTranslator.IdentityStreamOperator acts as a dummy source to consume cached intermediate
dataset.The
CacheTransformationTranslator.NoOpStreamOperator acts as a dummy sink so that the upstream can produce the
intermediate dataset to be cached.A
StateFutureImpl.CallbackRunner that gives info of CallbackRunnerWrapper.isHasMail() to the AEC and notifies new mail if needed.The CancelCheckpointMarker travels through the data streams, similar to the
CheckpointBarrier, but signals that a certain checkpoint should be canceled.Thrown to trigger a canceling of the executing task.
The interface to be implemented by drivers that do not run in an own task context, but are
chained to other tasks.
Helper class to help maintain the hash info of an operator chain.
Chained version of ReduceCombineDriver.
Helper class to help maintain the chained source info of an operator chain.
Handle to state handles for the operators in an operator chain.
Defines the chaining scheme for the operator.
A handle to ChangelogStateBackend state.
State handle for local copies of
ChangelogStateHandleStreamImpl.A handle to saved
state changes.ChangelogStateHandle implementation based on StreamStateHandle.Changelog's implementation of a
TaskLocalStateStore.A
DataInputView that is backed by a BlockChannelReader, making it effectively a data input stream.A simple iterator over the input read though an I/O channel.
ChannelSelector<T extends org.apache.flink.core.io.IOReadableWritable>
The
ChannelSelector determines to which logical channels a record should be written to.ChannelSelectorRecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>
A regular record-oriented runtime result writer.
Implemented by entities that hold any kind of channel state and need a reference to the
ChannelStateWriter.The pending result of channel state for a specific checkpoint-subtask.
Helper class for persisting channel state via
ChannelStateWriter.Writes channel state during checkpoint/savepoint.
Channel state write result.
No-op implementation of
ChannelStateWriter.The factory of
ChannelStateWriteRequestExecutor.ChannelStateWriter implemented using CheckpointStateOutputStreams.A
DataOutputView that is backed by a BlockChannelWriter, making it effectively a data output stream.A checkpoint, pending or completed.
Extra interface for discarding the checkpoint.
Input, with just basic methods for blocking and resuming consumption.
Interface that combines both, the
KeyedStateBackend interface, which encapsulates methods
responsible for keyed state management and the Snapshotable which tells the system how to
snapshot the underlying state.A task that participates in checkpointing.
Checkpoint barriers are used to align checkpoints throughout the streaming topology.
The
CheckpointBarrierHandler reacts to checkpoint barrier arriving from the input
channels.The
CheckpointBarrierTracker keeps track of what checkpoint barriers have been received
from which input channels.KeyedStateHandle that is bound to a specific checkpoint.A
CheckpointCommittableManager collects committables for one checkpoint across
potentially multiple upstream subtasks.Configuration that captures all checkpointing related settings.
Handler which serves the checkpoint configuration.
Message headers for the
CheckpointConfigHandler.Response class of the
CheckpointConfigHandler.Contains information about the externalized checkpoint configuration.
Processing mode.
Processing mode deserializer.
JSON deserializer for
CheckpointConfigInfo.ProcessingMode.The checkpoint coordinator coordinates the distributed snapshots of operators and state.
Configuration settings for the
CheckpointCoordinator.CheckpointCoordinatorConfiguration builder.This actor listens to changes in the JobStatus and activates or deactivates the periodic
checkpoint scheduler.
RPC Gateway interface for messages to the CheckpointCoordinator.
A buffer which encapsulates the logic of dealing with the response from the
CollectSinkFunction.This is the core interface for stateful transformation functions, meaning functions that
maintain state across individual stream records.
The
CheckpointedInputGate uses CheckpointBarrierHandler to handle incoming CheckpointBarrier from the InputGate.Interface for partitions that are checkpointed, meaning they store data as part of unaligned
checkpoints.
The scope for a chunk of checkpointed state.
Base class for checkpoint related exceptions.
The checkpoint failure manager which centralized manage checkpoint failure processing logic.
A callback interface about how to fail a job.
Various reasons why a checkpoint was failure.
HTTP handlers for asynchronous triggering of checkpoints.
HTTP handler to query for the status of the checkpoint.
Handler for the checkpoint trigger operation.
A checkpoint ID counter.
Path parameter for the checkpoint id of type
Long.Represents information about a triggered checkpoint.
Deprecated.
This class has been moved to
CheckpointingMode.Response of the
CheckpointingStatisticsHandler.Checkpoint counts.
Statistics about the latest checkpoints.
Statistics for a restored checkpoint.
Checkpoint summary.
Handler which serves the checkpoint statistics.
Message headers for the
CheckpointingStatisticsHandler.Message parameters for checkpoint related messages.
The metadata of a snapshot (checkpoint or savepoint).
Encapsulates all the meta data for a checkpoint.
An output stream for checkpoint metadata.
A collection of simple metrics, around the triggering of a checkpoint.
A builder for
CheckpointMetrics.Options for performing the checkpoint.
How a checkpoint should be aligned.
The plan of one checkpoint, indicating which tasks to trigger, waiting for acknowledge or commit
for one specific checkpoint.
Calculates the plan of the next checkpoint, including the tasks to trigger, wait or commit for
each checkpoint.
Provides the context for
DefaultCheckpointPlanCalculator to compute the plan of
checkpoints.The configuration of a checkpoint.
A factory for per Job checkpoint recovery components.
CheckpointResourcesCleanupRunner implements JobManagerRunner in a way, that only
the checkpoint-related resources are instantiated.CheckpointResourcesCleanupRunnerFactory implements CleanupRunnerFactory providing
a factory method for creating CheckpointResourcesCleanupRunner instances.Responder for checkpoint acknowledge and decline messages in the
Task.Policy for whether checkpoints are retained after a job terminates.
A utility class with the methods to write/load/dispose the checkpoint and savepoint metadata.
CheckpointScheduling provides methods for starting and stopping the periodic scheduling
of checkpoints.Delegate class responsible for checkpoints cleaning and counting the number of checkpoints yet to
clean.
A dedicated output stream that produces a
StreamStateHandle when closed.A toolset of operations that can be performed on a location embedded within the class.
REST handler which returns the details for a checkpoint.
Headers for the
CheckpointStatisticDetailsHandler.Statistics for a checkpoint.
Statistics for a completed checkpoint.
Statistics for a failed checkpoint.
Statistics for a pending checkpoint.
A size-based cache of accessed checkpoints for completed and failed checkpoints.
Counts of checkpoints.
An array based history of checkpoint stats.
An interface that allows listening on the checkpoint lifecycle.
A snapshot of the checkpoint stats.
Status of the tracked checkpoint.
Tracker for checkpoint statistics.
These headers define the protocol for querying the status of a checkpoint operation.
The parameters for triggering a checkpoint.
CheckpointStorage defines how
StateBackend's store their state for fault tolerance in
streaming applications.Implementations of this interface should implement methods acting as an administration role for
checkpoint storage, which defined in
CheckpointStorageCoordinatorView.This interface creates a
CheckpointStorageLocation to which an individual checkpoint or
savepoint is stored.A factory to create a specific
CheckpointStorage.This class contains utility methods to load checkpoint storage from configurations.
A storage location for one particular checkpoint, offering data persistent, metadata persistence,
and lifecycle/cleanup methods.
A reference to a storage location.
This interface implements the durable storage of checkpoint data and metadata streams.
CompletedCheckpointStore utility interfaces.A factory for checkpoint output streams, which are used to persist data for checkpoints.
Interface that provides access to a CheckpointStateOutputStream and a method to provide the
SnapshotResult.Factory method for a
KeyedStateHandle to be used in CheckpointStreamWithResultProvider.toKeyedStateHandleSnapshotResult(SnapshotResult, KeyGroupRangeOffsets, KeyedStateHandleFactory).Implementation of
CheckpointStreamWithResultProvider that creates both, the
primary/remote/jm-owned state and the secondary/local/tm-owned state.Implementation of
CheckpointStreamWithResultProvider that only creates the
primary/remote/jm-owned state.These headers define the protocol for triggering a checkpoint.
The parameters for triggering a checkpoint.
RequestBody to trigger checkpoints.The type of checkpoint to perform.
CleanupRetryStrategyFactory is used to create the RetryStrategy for the DispatcherResourceCleanerFactory.CleanupRunnerFactory provides a factory method for creating CheckpointResourcesCleanupRunner instances.A pair of an
Iterator to receive results from a streaming application and a JobClient to interact with the program.Handler that receives the coordination requests from the client and returns the response from the
coordinator.
Message headers for the
ClientCoordinationHandler.Request that carries a serialized
CoordinationRequest to a specified coordinator.Response that carries a serialized
CoordinationResponse to the client.ClientHighAvailabilityServices provides services those are required on client-side.Factory interface for
ClientHighAvailabilityServices.Contains utility methods for clients.
Utility interface for a provider of an input that can be closed.
Handler which serves the cluster's configuration.
Configuration class which contains the parsed command line arguments for the
ClusterEntrypoint.Message headers for the
ClusterConfigHandler.Parser factory which generates a
ClusterConfiguration from the given list of command line
arguments.Indicates some task fail to consume cluster dataset.
Handler for
ClusterDataSetDeleteTriggerHeaders.A
Void alternative that implements Serializable.Specification for retrieving the status for the delete operation of a cluster data set.
Specification for triggering the deletion of a cluster data set.
The entry for a single cluster data set.
Path parameter identifying cluster data sets.
Handler for
ClusterDataSetListHeaders.Specification for retrieving an overview over all available cluster partitions.
Base class for the Flink cluster entry points.
Execution mode of the dispatcher.
Shutdown behaviour of a
ClusterEntrypoint.Exceptions thrown by the
ClusterEntrypoint.Exception utils to handle and enrich exceptions occurring in the ClusterEntrypoint.
Utility class for
ClusterEntrypoint.Information about the cluster which is shared with the cluster components.
Response to the
RequestStatusOverview message, carrying a description of the Flink
cluster status.Handler which returns the cluster overview information with version.
Message headers for the
ClusterOverviewHandler.Cluster overview message including the current Flink version and commit id.
Interface for components that manage cluster partitions.
A report about the current status of all cluster partitions of the TaskExecutor, describing which
partitions are available.
An entry describing all partitions belonging to one dataset.
Provides statistics of cluster resources.
Utility for handling any uncaught exceptions
A
TwoInputStreamOperator for executing KeyedBroadcastProcessFunctions.A
TwoInputStreamOperator for executing BroadcastProcessFunctions.A CoFlatMapFunction implements a flat-map transformation over two connected streams.
CoGroup task which is executed by a Task Manager.
CoGroupedStreams represents two DataStreams that have been co-grouped.TypeSerializer for TaggedUnion.The
TypeSerializerSnapshot for the CoGroupedStreams.UnionSerializer.A co-group operation that has
KeySelectors defined for both inputs as
well as a WindowAssigner.Interface describing the methods that have to be implemented by local strategies for the CoGroup
Pact.
A
CoordinationRequest from the client indicating that it wants a new batch of query
results.A
CoordinationResponse from the coordinator containing the required batch or new results
and other necessary information in serialized form.A fetcher which fetches query results from sink and provides exactly-once semantics.
An iterator which iterates through the results of a query job.
An
OperatorEvent that passes the socket server address in the sink to the coordinator.A sink function that collects query results and sends them back to the client.
A
StreamSink that collects query results and sends them back to the client.Provider for
CollectSinkOperatorCoordinator.The Factory class for
CollectSinkOperator.A
DataStreamSink which is used to collect results of a data stream.A
CoLocationConstraint stores the ID of CoLocationGroup and an ID referring to
the actual subtask (i.e.CoLocationGroup refers to a list of JobVertex instances, where the i-th
subtask of one vertex has to be executed on the same TaskManager as the i-th
subtask of all other JobVertex instances in the same group.A
CoLocationGroup implementation.A CoFlatMapFunction implements a map() transformation over two connected streams.
Container class for command line options.
Command line parser which produces a result from the given command line arguments.
Internal implementation to commit a specific committable and handle the response.
Internal state of a committable.
This class is responsible to book-keep the committing progress across checkpoints and upstream
subtasks.
The serializer for the
CommittableCollector.The message send from
SinkWriter to Committer.The serializer to serialize
CommittableMessages in custom operators.The type information for the
CommittableMessage to make it consumable from a DataStream.This class tracks the information about committables belonging to one checkpoint coming from one
subtask.
Provides metadata.
A
StreamOperatorFactory for CommitterOperator.Common memory components of Flink processes (e.g.
A hash table that uses Flink's managed memory and supports replacement of records or updates to
records.
An
AggregationFunction that computes values based on comparisons of Comparables.Internal comparator implementation, for use with
ComparableAggregator.A CompletedCheckpoint describes a checkpoint after all required tasks acknowledged it (with their
state) and that is considered successful.
Statistics for a successfully completed checkpoint.
Summary over all completed checkpoints.
Immutable snapshot of
CompletedCheckpointStatsSummary.The CompletedCheckpointStorageLocation describes the storage aspect of a completed checkpoint.
A bounded LIFO-queue of
CompletedCheckpoint instances.Cache to manage ongoing operations.
A
StateIterator that has all elements.A util class to help with a clean component shutdown.
Abstract
MetricGroup for system components (e.g., TaskManager,
Job, Task, Operator).An implementation of
Buffer which contains multiple partial buffers for network data
communication.Utils for RocksDB state serialization and deserialization.
Base of all snapshots that are taken by
StateBackends and some other components in tasks.FSDataInputStream that delegates all reading operations to a wrapping StreamCompressionDecorator.FSDataOutputStream that delegates all writing operations to a wrapping StreamCompressionDecorator.Utils for
BlockCompressor.An interface for checkpoint storage types that pick up additional parameters from a
configuration.
An interface for state backends that pick up additional parameters from a configuration.
Interface for
StreamPartitioner which have to be configured with the maximum parallelism
of the stream transformation.Represents the set of command-line options related to update and get configuration.
Response of the
ClusterConfigHandler, represented as a list of key-value pairs of the
cluster Configuration.A single key-value pair entry in the
ConfigurationInfo response.If
DeploymentOptions.PROGRAM_CONFIG_ENABLED is disabled, this error denotes the not
allowed configuration.Utility class to extract related parameters from
Configuration and to sanity check them.ConnectedStreams represent two connected streams of (possibly) different data types.
Exception which is thrown if the
RestClient detects that a connection was closed.Message for indicating connection error.
Base class for all connection related exception thrown by the
RestClient.A
ConnectionID identifies a connection to a remote task manager by the socket address and
a connection index.Exception which is thrown by the
RestClient if a connection becomes idle.The connection manager manages physical connections for the (logical) remote input channels at
runtime.
Utilities to determine the network interface and address that should be used to bind the
TaskManager communication to.
A
LeaderRetrievalListener that allows retrieving an InetAddress for the
current leader.Group of consumed
IntermediateResultPartitionIDs.This view maintains the finished progress of consumer
SchedulingPipelinedRegions for each
ConsumedPartitionGroup.Group of consumer
ExecutionVertexIDs.This class describes the basic parameters for launching a TaskManager process.
Details about the operator containing task (such as
StreamTask).A state future that holds the
RecordContext and maintains the reference count of it.A value that will have different values across different contexts.
Deprecated.
This class is based on the
SourceFunction API, which is due to be removed.The operator that reads the
splits received from the preceding
ContinuousFileMonitoringFunction.ContinuousFileReaderOperator factory.Exception which is thrown if an input cannot be converted into the requested type.
A factory class for the
StreamOperators implementing OperatorEventHandler.An task that is coordinated, i.e. contains operators coordinated by
OperatorCoordinator.Root interface for all requests from the client to a
OperatorCoordinator which requests
for a CoordinationResponse.Client interface which sends out a
CoordinationRequest and expects for a CoordinationResponse from a OperatorCoordinator.Coordinator interface which can handle
CoordinationRequests and response with CoordinationResponses to the client or operator.Root interface for all responses from a
OperatorCoordinator to the client which is the
response for a CoordinationRequest.CoordinatorStore can be used for sharing some information among OperatorCoordinator instances.Basic implementation of
CoordinatorStore.A function that processes elements of two streams and produces a single output one.
A
StreamOperator for executing CoProcessFunctions.Implementation of Flink's in-memory state maps with copy-on-write support.
One entry in the
CopyOnWriteStateMap.This class represents the snapshot of a
CopyOnWriteStateMap.This implementation of
StateTable uses CopyOnWriteStateMap.This class represents the snapshot of a
CopyOnWriteStateTable and has a role in operator
state checkpointing.Exception indicating that the parsed configuration was corrupt.
StreamOperator for processing CoFlatMapFunctions.StreamOperator for processing CoMapFunctions.An
Evictor that keeps up to a certain amount of elements.Wrapping
Output that updates metrics on the number of emitted elements.A
Trigger that fires once the count of elements in a pane reaches the given count.State which waits for the creation of the
ExecutionGraph.Gauge metric measuring the input buffers usage for
SingleInputGates under credit based
mode.Cross task which is executed by a Task Manager.
A wrapper for curatorFramework and unHandledErrorListener which should be unregister from
curatorFramework before closing it.
Decorator class for
MessageHeaders that adds the ability to include custom HTTP headers.Partitioner that selects the channel with a user defined partitioner function on a key.
This class is responsible to hold operator Uid hashes from the common operators of the sink.
Builder to construct
CustomSinkOperatorUidHashes.Enumeration for the different dam behaviors of an algorithm or a driver strategy.
Handler which returns the dashboard configuration.
Response of the
DashboardConfigHandler containing general configuration values such as
the time zone and the refresh interval.Collection of features that are enabled/disabled.
Message headers for the
DashboardConfigHandler.Data of different subpartitions can be appended to a
DataBuffer and after the DataBuffer is full or finished, the appended data can be copied from it in subpartition index
order.It is an internal equivalent of
InputStatus that provides
additional non public statuses.Container for meta-data of a data set.
DataSinkTask which is executed by a task manager.
DataSourceTask which is executed by a task manager.
A DataStream represents a stream of elements of the same type.
This class acts as an accessor to elements collected via
DataStream.collectAsync(Collector).A Stream Sink.
The DataStreamSource represents the starting point of a DataStream.
This class represents a
CompletedCheckpointStore if checkpointing has been disabled.This class represents a
CheckpointIDCounter if checkpointing is deactivated.A declaration chain allows to declare multiple async operations in a single chain.
A context to declare parts of process in user-defined function/operator.
Exception thrown when something wrong with declaration happens.
The manager holds all the declaration information and manage the building procedure.
Slot pool interface which uses Flink's declarative resource management protocol to acquire
resources.
Listener interface for newly available slots.
No-op
DeclarativeSlotPool.NewSlotsListener implementation.SlotPool implementation which uses the DeclarativeSlotPool to allocate slots.Factory for
DeclarativeSlotPoolBridge.Factory for a
DeclarativeSlotPool.SlotPoolService implementation for the DeclarativeSlotPool.Factory for the
DeclarativeSlotPoolService.A variable declared in async state processing.
ServiceConnectionManager for declaring resource requirements.Service that accepts resource requirements.
A keyed function that processes elements of a stream.
This message is sent from the
TaskExecutor to the
JobMaster to tell the checkpoint coordinator that a
checkpoint request could not be heeded.A basic queue with duplicated elements ignored during
DeduplicatedQueue.add(Object).The
DefaultAdaptiveExecutionHandler implements the AdaptiveExecutionHandler
interface to provide an incrementally generated job graph.Default
AllocatedSlotPool implementation.Default implementation of
BatchJobRecoveryHandler and JobEventReplayHandler.Default implementation of
BlocklistHandler.The factory to instantiate
DefaultBlocklistHandler.Default implementation of
BlocklistTracker.The default
BroadcastVariableInitializer implementation that initializes the broadcast
variable into a list.The default implementation of he
CheckpointPlan.Default implementation for
CheckpointPlanCalculator.Default factory for creating client high availability services.
Default implementation of
CompletedCheckpointStore.Helper methods related to
DefaultCompletedCheckpointStore.Default
DeclarativeSlotPool implementation.Factory for
DefaultDeclarativeSlotPool.Manager for delegation tokens in a Flink cluster.
A factory for
DefaultDelegationTokenManager.A facade over the
Dispatcher that exposes specific functionality.Abstract class which implements the creation of the
DispatcherResourceManagerComponent
components.Runner for the
Dispatcher which is responsible for
the leader election.DispatcherRunnerFactory implementation which creates DefaultDispatcherRunner
instances.Default implementation of
ExecutionDeployer.Factory to instantiate the
DefaultExecutionDeployer.Default
ExecutionDeploymentReconciler implementation.Default
ExecutionDeploymentTracker implementation.Default implementation of the
ExecutionGraph.Utility class to encapsulate the logic of building an
DefaultExecutionGraph from a JobGraph.Default implementation of
ExecutionGraphCache.Default
ExecutionGraphFactory implementation.Default implementation of
ExecutionOperations.Default implementation for
ExecutionPlanStore.Adapter of
ExecutionGraph to SchedulingTopology.The default implementation of
FailureEnricher.Context class.Default implements of
FreeSlotTracker.Default implement of
GroupCache.The Factory of
DefaultGroupCache.The default implementation of
HeartbeatMonitor.Default factory for
InMemorySorter.Default implementation of
InputConsumableDecider.Factory for
DefaultInputConsumableDecider.Service which retrieves for a registered job the current job leader id (the leader id of the job
manager responsible for the job).
Default implementation of
JobLeaderService.Default implementation of
JobManagerJobMetricGroupFactory which creates for a given
ExecutionPlan a JobManagerJobMetricGroup.DefaultJobManagerRunnerRegistry is the default implementation of the JobManagerRunnerRegistry interface.Default
JobMasterServiceProcess which is responsible for creating and running a JobMasterService.Default implementation of the
JobTable.Default implementation of KeyedStateStore that currently forwards state registration to a
RuntimeContext.A simple ConnectionState listener that remembers the last state.
Default implementation for leader election service.
The counterpart to the
DefaultLeaderElectionService.Default implementation for
LineageDataset.Implementation of LineageEdge.
Default implementation for
LineageGraph.Build the default lineage graph from
LineageEdge.Default implementation of
LogicalEdge.Set of
LogicalVertex that are connected through pipelined LogicalResult.Default implementation of
LogicalResult.Default implementation of
LogicalTopology.Default implementation of
LogicalVertex.Default
MetricFilter implementation that filters metrics based on MetricOptions.REPORTER_INCLUDES/MetricOptions.REPORTER_EXCLUDES.Default handler for the
OperatorCoordinators.Default implementation of OperatorStateStore that provides the ability to make snapshots.
Builder class for
DefaultOperatorStateBackend which handles all necessary initializations
and clean ups.Default
PartitionWithMetrics implementation.Default implementation of
PreferredLocationsRetriever.Default implementation of
RequirementMatcher.The default implementation of
ResourceAllocationStrategy.DefaultResourceCleaner is the default implementation of ResourceCleaner.Builder for creating DefaultResourceCleaner instances.Default
ResourceTracker implementation.The future default scheduler.
Components to create a
DefaultScheduler.Factory for
DefaultScheduler.Default implementation of
SchedulingPipelinedRegion.Default
ShuffleMasterSnapshotContext implementation.Default
ShuffleMetrics implementation.Simple
SlotAssigner that treats all slots and slot sharing groups equally.Default
SlotPoolServiceSchedulerFactory implementation.Default implementation of
SlotStatusSyncer for fine-grained slot management.The default implementation of
SpeculativeExecutionHandler.DefaultStateTransitionManager is a state machine which manages the AdaptiveScheduler's state transitions based on the previous transition time and the available
resources.Default implementation for
StreamGraphContext.Simple container for subtask attempt counts backed by a list.
Service to register timeouts for a given key.
DEP Flink specific helper class to apply default TLS configuration.
Maintains the attempt number per subtask.
Default implementation of
VertexParallelismAndInputInfosDecider.A
VertexParallelismInformation implementation that provides common validation.Maintains the configured parallelisms for vertices, which should be defined by a scheduler.
An interface to delegate state backend.
Container for delegation tokens.
Manager for delegation tokens in a Flink cluster.
Listener for events in the
DelegationTokenManager.Repository for delegation token receivers.
Metrics that capture how long a job was deploying tasks.
The
DescriptiveStatisticsHistogram use a DescriptiveStatistics DescriptiveStatistics as a Flink Histogram.DescriptiveStatistics histogram statistics implementation returned by
DescriptiveStatisticsHistogram.The deserialization delegate is used during deserialization to read an arbitrary element as if it
implements
IOReadableWritable, with the help of a type serializer.Envelope that carries whether the wrapped value is deterministic or not.
This state handle represents a directory.
This state handle represents a directory, usually used to be registered to
SharedStateRegistry to track the life cycle of the directory.The
DiskIOScheduler is a scheduler that controls the reading of data from shuffle files.The data client is used to fetch data from disk tier.
The implementation of
TierFactory for disk tier.The disk tier implementation of
TierProducerAgent.Base class for the Dispatcher component.
Enum to distinguish between initial job submission and re-submission for recovery.
An interface containing the logic of bootstrapping the
Dispatcher of a cluster.A factory to create a
DispatcherBootstrap.A handler for async operations triggered by the
Dispatcher whose keys and results are
cached.Base class for
Dispatcher related exceptions.Dispatcher factory interface.Gateway for the Dispatcher component.
Fencing token of the
Dispatcher.Factory for the
DispatcherLeaderProcess.Factory for
DispatcherLeaderProcessFactory.Encapsulates caches for results of asynchronous operations triggered by the
Dispatcher.DispatcherResourceCleanerFactory instantiates ResourceCleaner instances that
clean cleanable resources from the Dispatcher.Factory for the
DispatcherResourceManagerComponent.REST endpoint for the
Dispatcher component.The
DispatcherRunner encapsulates how a Dispatcher is being executed.Factory interface for the
DispatcherRunner.Dispatcher services container.Thread factory that creates threads with a given name, associates them with a given thread group,
and set them to daemon mode.
A standalone implementation of the
RuntimeContext, created by runtime UDF operators.A distribution pattern determines, which sub tasks of a producing task are connected to which
consuming sub tasks.
A
Future that is always done and will just yield the object that was given at creation
time.A gauge that returns (in milliseconds) how long a job has not been not running any more, in case
it is in a failing/recovering situation.
The interface to be implemented by all drivers that run alone (or as the primary driver) in a
task.
Enumeration of all available operator strategies.
Map which stores values under two different indices.
A dummy implementation of the
BatchJobRecoveryHandler.The dummy implementation of
SpeculativeExecutionHandler.Exception which is returned upon job submission if the submitted job is currently being executed.
A CheckpointStateOutputStream that wraps a primary and a secondary CheckpointStateOutputStream
and duplicates all writes into both streams.
A dynamic JAAS configuration.
DynamicParametersConfigurationParserFactory can be used to extract the dynamic parameters
from command line.Class that manages all the connections between tasks.
Utilities for building
EdgeManager.This enum defines the element order of being processed.
An embedded in-memory checkpoint store, which supports shutdown and suspend.
An implementation of the
HighAvailabilityServices for the non-high-availability case
where all participants (ResourceManager, JobManagers, TaskManagers) run in the same process.EmbeddedHaServices extension to expose leadership granting and revoking.A thread-safe in-memory implementation of the
JobResultStore.A simple leader election service, which selects a leader among contenders and notifies listeners.
An empty
FileMergingOperatorStreamStateHandle that is only used as a placeholder to
prevent file merging directory from being deleted.An empty iterator that never returns anything.
MessageParameters implementation which has no parameters.
An empty mutable object iterator that never returns anything.
Request which do not have a request payload.
Empty
ResponseBody implementation.An empty
SegmentFileStateHandle that is only used as a placeholder.A singleton implementation of
ShuffleMasterSnapshot that represents an empty snapshot of
shuffle master.A singleton implementation of
TieredShuffleMasterSnapshot that represents an empty
snapshot of tiered shuffle master.Marks the end of recovered state of
RecoveredInputChannel of this subtask or ResultSubpartition on the
upstream.This event indicates there will be no more data records in a subpartition.
This event marks a subpartition as fully consumed.
EndOfSegmentEvent is used to notify the downstream switch tiers in tiered storage shuffle
mode.Marks the end of a superstep of one particular iteration superstep.
Basic
ClusterConfiguration for entry points.Parser factory for
EntrypointClusterConfiguration.The Environment gives the code executed in a task access to the task's properties (such as name,
parallelism), the configurations, the data stream readers and writers, as well as the various
components that are provided by the TaskManager, such as memory manager, I/O manager, ...
The response of environment info.
Utility class that gives access to the execution environment of the JVM, like the executing user,
startup options, or the JVM version.
Revision information encapsulates information about the source code revision of the Flink
code.
Epoch manager segments inputs into distinct epochs, marked by the arrival of non-records(e.g.
All inputs are segment into distinct epochs, marked by the arrival of non-record inputs.
This enum defines whether parallel execution between epochs is allowed.
Simple container to hold an exception and the corresponding timestamp.
Generic response body for communicating errors on the server.
EventAnnouncement is announcing presence or receiving of an AbstractEvent.Utility class to serialize and deserialize task events.
A
Trigger that fires once the watermark passes the end of the window to which a pane
belongs.A
WatermarkCombiner used to combine EventTimeExtension related watermarks in
input channels.This class is used to handle
EventTimeExtension related watermarks in operator, such as
EventTimeExtension.EVENT_TIME_WATERMARK_DECLARATION and EventTimeExtension.IDLE_STATUS_WATERMARK_DECLARATION.This class represents event-time updated status.
A
WindowOperator that also allows an Evictor to be used.An
Evictor can remove elements from a pane before/after the evaluation of WindowFunction
and after the window evaluation gets triggered by a TriggerA context object that is given to
Evictor methods.Interface to be implemented by classes that handle exceptions.
ExceptionHistoryEntry collects information about a single failure that triggered the
scheduler's failure handling.ArchivedTaskManagerLocation represents a archived (static) version of a TaskManagerLocation.A special exception that signifies that the cause exception came from a chained operator.
A special exception to indicate that an exception occurred in the nested call of a chained stub.
Gauge metric measuring the exclusive buffers usage gauge for
SingleInputGates.A single execution of a vertex.
Unique identifier for the attempt to execute a tasks.
This deployer is responsible for deploying executions.
Factory to instantiate the
ExecutionDeployer.A listener that is called when the deployment of an execution has been started/completed.
Component for reconciling the deployment state of executions.
Factory for
ExecutionDeploymentReconciler.Interface for triggering actions in case of state mismatches.
A report about the currently deployed executions of a TaskExecutor.
Possible states for the deployment of an execution.
A tracker for deployed executions.
An adapter for using an
ExecutionDeploymentTracker as an ExecutionDeploymentListener.This handler deals with task failures to return a
FailureHandlingResult which contains
tasks to restart to recover from failures.The execution graph is the central data structure that coordinates the distributed execution of a
data flow.
Cache for
ExecutionGraphInfo which are obtained from the Flink cluster.A
CheckpointPlanCalculatorContext implementation based on the information from an ExecutionGraph.Base class for exceptions occurring in the
ExecutionGraph.Exceptions thrown by operations on the
ExecutionGraph by the JobMaster.Factory for creating an
ExecutionGraph.Handler for the
ExecutionGraph which offers some common operations.A class for statistically unique execution graph IDs.
ExecutionGraphInfo serves as a composite class that provides different ExecutionGraph-related information.Interface for a
ExecutionGraphInfo store.A
ResultPartitionAvailabilityChecker which decides the intermediate result partition
availability based on whether the corresponding result partition in the execution graph is
tracked.An implementation of
InputsLocationsRetriever based on the ExecutionGraph.Utility methods related to
ExecutionGraph.This class hosts the historical executions of an
ExecutionVertex in a LinkedHashMap with a size bound.Factory to create
ExecutionJobVertex.This class is used to record the completion info of
ExecutionJobVertex.This class is used to record the initialization info of
ExecutionJobVertex.Operations on the
Execution.An interface representing a general execution plan, which can be implemented by different types
of graphs such as JobGraph and StreamGraph.
Interface for retrieving stream graph context details for adaptive batch jobs.
ExecutionPlan instances for recovery.A listener for
ExecutionPlan instances.ExecutionPlanStore utility interfaces.
A watcher on
ExecutionPlanStore.Utilities for generating
ExecutionPlan.Allows to store and remove execution plans.
Component responsible for assigning slots to a collection of
Execution.Interface for
ExecutionSlotAllocator factories.Represents execution vertices that will run the same shared slot.
An enumeration of all states that a task can be in during its execution.
A listener that is called when an execution switched to a new state.
The slow task detector which detects slow tasks based on their execution time.
The ExecutionVertex is a parallel subtask of the execution.
This class is used to record the state of
ExecutionVertex.Id identifying
ExecutionVertex.This class describe the inputs(partitions and subpartitions that belong to the same intermediate
result) information of an execution vertex.
This class is used to record the
ExecutionVertex that has been reset.This class helps to record version of an execution vertex.
Records modifications of
ExecutionVertices, and allows for checking whether a vertex was modified.This class is used to coordinate between two components, where one component has an executor
following the mailbox model and the other component notifies it when needed.
Restart strategy which tries to restart indefinitely number of times with an exponential backoff
time in between.
The factory for creating
ExponentialDelayRestartBackoffTimeStrategy.Deprecated.
This interface is based on the
SourceFunction API, which is due to be
removed.Through the
CheckpointTrigger, the source function notifies the Flink source operator
when to trigger the checkpoint.Provide the information of external resources.
Utility class for external resource framework.
The
ExternalSorter is a full fledged sorter.A builder for an
ExternalSorter.Statistics for a failed checkpoint.
New interface for failover strategies.
The factory to instantiate
FailoverStrategy.A utility class to load NG failover strategy factories from the configuration.
Utils class for loading and running pluggable failure enrichers.
Result containing the tasks to restart upon a task failure.
FailureHandlingResultSnapshot creates a snapshot of a FailureHandlingResult
providing the actual Executions.Specifies a collections of failure labels, filtering the exceptions returned for
JobExceptionsHandler.
Represents a failure label consisting of a KV pair of strings.
Restart strategy which can restart when failure rate is not exceeded.
The factory for creating
FailureRateRestartBackoffTimeStrategy.FailureResultUtil contains helper methods for FailureResult.These classes encapsulate the logic of accessing a field specified by the user as either an index
or a field expression string.
Static factories for the
FieldAccessor utilities.CloseableIterator of buffers over file content.A
CheckpointStateOutputStream that writes into a specified file and returns a FileStateHandle upon closing.The FileCache is used to access registered cache files when a task is deployed.
A
DataInputView that is backed by a BlockChannelReader, making it effectively a data input stream.The manager used for creating/getting file IO channels based on config temp dirs.
The manager used for creating/deleting file channels based on config temp dirs.
A
DataOutputView that is backed by a BlockChannelWriter, making it effectively a data output stream.A cache layer of hybrid data index.
FileDataIndexRegionHelper is responsible for writing a FileDataIndexRegionHelper.Region to the file or
reading a FileDataIndexRegionHelper.Region from file.A
FileDataIndexRegionHelper.Region Represents a series of buffers that are:
From the same subpartition
Logically (i.e. buffer index) consecutive
Physically (i.e. offset in the file) consecutive
This class is responsible for spilling region to disk and managing these spilled regions.
Factory of
FileDataIndexSpilledRegionManager.Default implementation of
FileDataIndexSpilledRegionManager.Factory of
FileDataIndexSpilledRegionManager.Store for
ExecutionGraphInfo instances.A Channel represents a collection of files that belong logically to the same resource.
An enumerator for channels that logically belong together.
An ID identifying an underlying file channel.
JobGraphRetriever implementation which retrieves the JobGraph from a file on
disk.A
CheckpointStateOutputStream that writes into a segment of a file and returns a SegmentFileStateHandle upon closing.A proxy of the
FileMergingSnapshotManager that owns this FileMergingCheckpointStateOutputStream, with the interfaces for dealing with physical files.Metrics related to the file merging snapshot manager.
A
OperatorStreamStateHandle that works for file merging checkpoints.FileMergingSnapshotManager provides an interface to manage files and meta information for
checkpoint files with merging checkpoint files enabled.
Space usage statistics of a managed directory.
A key identifies a subtask.
Base implementation of
FileMergingSnapshotManager.This class wrap DirectoryStreamStateHandle with reference count by ongoing checkpoint.
A builder that builds the
FileMergingSnapshotManager.A wrapper that wraps
FileMergingSnapshotManager and a Closeable.How the checkpoint files can be segmented.
Parameters for
JobManagerCustomLogHandler.Deprecated.
Internal class deprecated in favour of
ContinuousFileMonitoringFunction.The watch type of the
FileMonitoringFunction.The mode in which the
ContinuousFileMonitoringFunction operates.Deprecated.
Internal class deprecated in favour of
ContinuousFileMonitoringFunction.This class represents a chunk of data in a file channel.
Utils for read and write
FileDataIndexRegionHelper.Region.File based
SlotAllocationSnapshotPersistenceService that persists the SlotAllocationSnapshot as local files.StreamStateHandle for state that was written to a file stream.Blob store backed by
FileSystem.FileSystemCheckpointStorage checkpoints state as files to a file system.An implementation of the
JobResultStore which persists job result data to an underlying
distributed filesystem.RetrievableStateStorageHelper implementation which stores the state in the given
filesystem path.Different file types to request from the
TaskExecutor.Client uploading a file.
Writes multipart/form-data to disk.
A container for uploaded files.
Implementation of
SlotManager supporting fine-grained resource management.A FineGrainedTaskManagerRegistration represents a TaskManager.
A FineGrainedTaskManagerSlot represents a slot located in a TaskManager.
Implementation of
TaskManagerTracker supporting fine-grained resource management.An empty
PushingAsyncDataInput.DataOutput which is used by StreamOneInputProcessor once an DataInputStatus.END_OF_DATA is received.A fake
Input for finished on restore tasks.A fake main operator output that skips all the following operators for finished on restored
tasks.
The
OperatorChain that is used for restoring tasks that are TaskStateManager.isTaskDeployedAsFinished().A specialized
OperatorSubtaskState representing the subtask is finished.Collects and fulfills the finished state for the subtasks or operators.
Thrown when some subtasks of the operator have been finished but state doesn't support that
(e.g.
Restart strategy which tries to restart a fixed number of times with a fixed backoff time in
between.
The factory for creating
FixedDelayRestartBackoffTimeStrategy.The
FixedLengthByteKeyAndValueComparator is used by KeyedSortPartitionOperator to
compare records according to both the record key and record value.Flame Graph type query parameter.
Flame Graph type.
Map task which is executed by a Task Manager.
Util class for loading configuration from commandline arguments.
Same as
HttpObjectDecoder but
returns HTTP 413 to the client if the payload exceeds MessageAggregator.maxContentLength.Exception indicating that we could not find a Flink job with the given job ID.
Exception indicating that the Flink job with the given job ID has terminated without
cancellation.
Memory components which constitute the Total Flink Memory.
Utility to derive the
FlinkMemory components.Exception which indicates that the parsing of command line arguments failed.
Simple wrapper for ZooKeeper's
QuorumPeer, which reads a ZooKeeper config file and writes
the required 'myid' file before starting the peer.Gauge metric measuring the floating buffers usage gauge for
SingleInputGates.If there are multiple consecutive and the same hash shuffles, SQL planner will change them except
the first one to use forward partitioner, so that these operators can be chained to reduce
unnecessary shuffles.
When the parallelism of both upstream and downstream is
ExecutionConfig.PARALLELISM_DEFAULT and the edge's partitioner is not specified
(partitioner==null), the edge's partitioner will be set to FORWARD by default(See StreamGraph.createActualEdge(java.lang.Integer, java.lang.Integer, int, org.apache.flink.streaming.runtime.partitioner.StreamPartitioner<?>, org.apache.flink.util.OutputTag, org.apache.flink.streaming.api.transformations.StreamExchangeMode, org.apache.flink.runtime.jobgraph.IntermediateDataSetID) method for details).A forward group is a set of job vertices or stream nodes connected via forward edges.
Common utils for computing forward groups.
Input stream, that wraps another input stream and forwards all method calls to the wrapped
stream.
OutputStream, that wraps another OutputStream and forwards all method calls to
the wrapped stream.Partitioner that forwards elements only to the locally running downstream operation.
A simple buffer recycler that frees the memory segments.
A function for freeing slots.
Track all free slots.
Deprecated.
This class is based on the
SourceFunction API, which is due to be removed.Deprecated.
This class is based on the
SourceFunction API, which is due to be removed.Deprecated.
This class is based on the
SourceFunction API, which is due to be removed.Metric group which forwards all registration calls to a variable parent metric group that injects
a variable reporter index into calls to
MetricGroup.getMetricIdentifier(String) or MetricGroup.getMetricIdentifier(String, CharacterFilter).A
CheckpointMetadataOutputStream that writes a specified file and directory, and returns
a FsCompletedCheckpointStorageLocation upon closing.An implementation of
CheckpointStateToolset that does file based duplicating with as
PathsCopyingFileSystem.An implementation of durable checkpoint storage to file systems.
A storage location for checkpoints on a file system.
A
CheckpointStreamFactory that produces streams that write to a FileSystem.A
CheckpointStateOutputStream that writes into a file and returns a StreamStateHandle upon closing.A
CompletedCheckpointStorageLocation that resides on a file system.Implementation of
MetadataOutputStreamWrapper encapsulates the FSDataOutputStream
for FsCheckpointMetadataOutputStream.Utility class for writing an archive file to a
FileSystem and reading it back.An implementation of file merging checkpoint storage to file systems.
An implementation of checkpoint storage location for file merging checkpoints.
This class is a
WrappingProxy for FSDataInputStream that is
used to read from a file segment.The full outer join driver implements the logic of an outer join operator at runtime.
An asynchronous writer that can write a full snapshot/savepoint from a
FullSnapshotResources.A
SnapshotResources to be used with the backend-independent FullSnapshotAsyncWriter.The procedure of restoring state from a savepoint written with the unified binary format.
Utility methods and constants around creating and restoring full snapshots using
FullSnapshotAsyncWriter.An implementation of
Buffer represents a fully filled buffer which contains multiple
partial buffers for network data communication.A special operator state implementation representing the operators whose instances are all
finished.
This interface provides a context in which user functions can initialize by registering to
managed state (i.e. state that is managed by state backends).
This interface provides a context in which user functions that use managed state (i.e. state that
is managed by state backends) can participate in a snapshot.
The buffer specs of the
InputGate include exclusive buffers per channel, required/total
floating buffers and the target of total buffers.GatewayRetriever<T extends org.apache.flink.runtime.rpc.RpcGateway>
Generic retriever interface for
RpcGateway.Implementation of
PeriodTimer based on TimerGauge.Handler for retrieving the log url of a specified TaskManager or JobManager.
Serializer for
JobEvent instances that uses Flink's InstantiationUtil for
serialization and deserialization.A
GenericMetricGroup for representing the key part of a key-value metric group pair.A simple named
MetricGroup that is used to hold subgroups of
metrics.A
GenericMetricGroup for representing the value part of a key-value metric group pair.This interface gives access to transient, named, global aggregates.
Implements the
GlobalCommitter.Transformation for global committer.
A
TransformationTranslator for the GlobalCommitterOperator.An interface for handling global failures.
GloballyCleanableResource is supposed to be implemented by any class that provides
artifacts for a given job that need to be cleaned up after the job reached a global terminal
state.Partitioner that sends all elements to the downstream operator with subtask ID=0.
This mode decides the default
ResultPartitionType of job edges.The default window into which all data is placed (via
GlobalWindows).A
TypeSerializer for GlobalWindow.Serializer configuration snapshot for compatibility and format evolution.
A
WindowAssigner that assigns all elements to the same GlobalWindow.A trigger that fires iff the input stream reaches EndOfStream.
A trigger that never fires, as default Trigger for GlobalWindows.
This
GroupCache can cache group, key and value.Non-chained combine driver which is used for a CombineGroup transformation or a GroupReduce
transformation where the user supplied a RichGroupReduceFunction with a combine method.
GroupReduce task which is executed by a Task Manager.
Delegation token serializer and deserializer functionality.
Hadoop delegation token receiver base class.
Responsible telling if specific Hadoop dependencies are on classpath.
Delegation token provider for Hadoop filesystems.
Delegation token receiver for Hadoop filesystems.
Responsible for installing a Hadoop login user.
Hadoop security context which runs a Callable with the previously initialized UGI and appropriate
security credentials.
Default security context factory that instantiates
SecurityContext based on installed
modules, it would instantiate HadoopSecurityContext if a HadoopModuleFactory is
included.Utility class for working with Hadoop user related classes.
Interface to grant and revoke leadership of HA components.
Utilities to extract a redirect address.
Simple container for the request to a handler, that contains the
RequestBody and
path/query parameters.Base class for all
HandlerRequest related exceptions.Utils for getting query parameters from
HandlerRequest.Utilities for the REST handlers.
Convenience class to extract hardware specifics of the computer executing the running JVM.
A hardware description describes the resources available to a task manager.
JobPersistenceComponentFactory implementation which creates a ExecutionPlanStore
using the provided HighAvailabilityServices.* A
DataBuffer implementation which sorts all appended records only by subpartition
index.The hash implementation of the
BufferAccumulator.Common methods for all Hash Join Iterators.
This state backend holds the working state in the memory (JVM heap) of the TaskManagers and
checkpoints based on the configured
CheckpointStorage.A factory that creates an
HashMapStateBackend from a configuration.Iterator over the build side entries of a HashPartition.HashSubpartitionBufferAccumulator accumulates the records in a subpartition.This interface is used by
HashSubpartitionBufferAccumulator to operate HashBufferAccumulator.Delegation token provider implementation for HBase.
Delegation token receiver implementation for HBase.
A
DataInputView that is backed by a BlockChannelReader, making it effectively a data input stream.A
Broadcast State backed a heap-based Map.A
AbstractKeyedStateBackend that keeps state on the Java Heap and will serialize state to
streams provided by a CheckpointStreamFactory upon checkpointing.Builder class for
HeapKeyedStateBackend which handles all necessary initializations and
clean ups.A
KeyValueStateIterator over Heap backend snapshot
resources.Basic heap-based priority queue for
HeapPriorityQueueElement objects.This class has similar functions with
HeapPriorityQueue.Interface for objects that can be managed by a
HeapPriorityQueue.This interface works similar to
Comparable and is used to prioritize between two
objects.Interface for objects that can be managed by a
HeapPriorityQueue.A heap-based priority queue with set semantics, based on
HeapPriorityQueue.Factory for
HeapPriorityQueueSet.Manages creating heap priority queues along with their counterpart meta info.
This wrapper combines a HeapPriorityQueue with backend meta data.
This class represents the snapshot of an
HeapPriorityQueueSet.Implementation of heap restore operation.
Implementation of heap savepoint restore operation.
Interface for the interaction with the
HeartbeatManager.A heartbeat manager has to be able to start/stop monitoring a
HeartbeatTarget, and report
heartbeat timeouts for this target.Heartbeat monitor which manages the heartbeat state of the associated heartbeat target.
This factory provides an indirection way to create
HeartbeatMonitor.The receiver implementation of
HeartbeatTarget, which mutes the HeartbeatTarget#requestHeartbeat(ResourceID, I).The sender implementation of
HeartbeatTarget, which mutes the HeartbeatTarget#receiveHeartbeat(ResourceID, I).HeartbeatServices gives access to all services needed for heartbeating.
A default
HeartbeatServices implementation.Interface for components which can be sent heartbeats and from which one can request a heartbeat
response.
High availability mode for Flink's cluster execution.
The HighAvailabilityServices give access to all services needed for a highly-available setup.
Factory interface for
HighAvailabilityServices.Utils class to instantiate
HighAvailabilityServices implementations.Writer for an
ExecutionGraphInfo.Utility class for the HistoryServer.
Represents an HTTP header with a name and a value.
This class wraps netty's
HttpMethods into an enum, allowing us to use them in switches.A special
IllegalStateException indicating a mismatch in the expected and actual ExecutionState of an Execution.Helper class that provides read-only StreamEdge.
Helper class that provides read-only StreamGraph.
Helper class that provides read-only StreamNode.
Custom netty inbound handler factory in order to make custom changes on netty inbound data.
This tracker remembers CompletableFutures as long as they are incomplete and allows us to fail
them later.
Common interface to all incremental
KeyedStateHandle.A Holder of StreamStateHandle and the corresponding localPath.
State handle for local copies of
IncrementalRemoteKeyedStateHandle.The handle to states of an incremental snapshot.
An
InputGate with a specific index.This class represents the range of index.
Utils for
IndexRange.Captures ambiguous mappings of old channels to new channels for a particular gate or partition.
Captures ambiguous mappings of old channels to new channels.
Type of mapping which should be used for this in-flight data.
Abstract base interface of all info messages exchanged between the JobManager an for example the
runtime monitor.
Status of the initialization.
Indicates that a broadcast variable was initialized with a
DefaultBroadcastVariableInitializer as a non-List type, and later accessed
using RuntimeContext.getBroadcastVariable(String) which may only return lists.In-memory
ChangelogStateHandle.In-memory partition with overflow buckets for
CompactingHashTableFactory for
InMemorySorter.An in-memory (non-production) implementation of
StateChangelogStorage.An
StateChangelogStorageFactory for creating InMemoryStateChangelogStorage.This hash table supports updating elements.
Input interface used in MultipleInputStreamOperator.Gauge metric measuring the number of queued input buffers for
SingleInputGates.Gauge metric measuring the size in bytes of queued input buffers for
SingleInputGates.An input channel consumes a single
ResultSubpartitionView.A combination of a
Buffer and a flag indicating availability of further buffers, and
the backlog length indicating how many non-event buffers are available in the subpartitions.Identifier for input channels.
Identifies
InputChannel in a given
subtask.Collects metrics for
RemoteInputChannel and LocalInputChannel.Handle to an InputChannel state.InputConsumableDecider is responsible for determining whether the input of an
executionVertex or a consumed partition group is consumable.Factory for
InputConsumableDecider.Input format source operator factory.
Deprecated.
This class is based on the
SourceFunction API, which is due to be removed.An input gate consumes one or more partitions of a single produced intermediate result.
Simple pojo for INPUT, DATA and moreAvailable.
Deployment descriptor for a single input gate instance.
Runtime identifier of a consumed
IntermediateResult.Collects metrics of an input gate.
Utils to manage the specs of the
InputGate, for example, GateBuffersSpec.Utility for dealing with input gates.
This class wraps
InputGate provided by shuffle service and it is mainly used for
increasing general input metrics from TaskIOMetricGroup.A container for
InputFormats and OutputFormats, along
with their Configuration.Container for multiple wrappers containing
InputFormat and OutputFormat code.A task vertex that runs an initialization and a finalization on the master.
Utility for creating
CheckpointedInputGate based on checkpoint mode for StreamOneInputProcessor.Interface for stream operators that can select the input to get
StreamRecord.Describe the input selection that stream operators want to read records.
Utility class for creating
InputSelection.Component to retrieve the inputs locations of an
ExecutionVertex.An input split provider can be successively queried to provide a series of
InputSplit
objects a task is supposed to consume in the course of its execution.Class for statistically unique instance IDs.
Minimal implementation of an array-backed list of ints
ResourceVersion implementation with Integer value.An intermediate data set is the data set produced by an operator - either a source or any
intermediate operation.
Id identifying
IntermediateDataSet.Id identifying
IntermediateResultPartition.Internal window function for wrapping a
ProcessAllWindowFunction that takes an
Iterable and an AggregateFunction.Internal window function for wrapping a
ProcessWindowFunction that takes an
Iterable and an AggregateFunction.Internal window function for wrapping a
ProcessWindowFunction that takes an
Iterable and an AggregateFunction.The peer to the
AggregatingState in the internal state type hierarchy.This class defines the internal interface for aggregating state.
The peer to the
AppendingState in the internal state type hierarchy.This class defines the internal interface for appending state.
Internal interface for functions that are evaluated over keyed (grouped) windows.
A context for
InternalAsyncWindowFunction, similar to ProcessWindowFunction.Context but for
internal use.The
InternalBoolWatermarkDeclaration class implements the AbstractInternalWatermarkDeclaration interface and provides additional functionality specific to
boolean-type watermarks.A
CacheMetricGroup which register all cache related metrics under a subgroup of the
parent metric group.This interface encapsulates all methods needed by ExecutionJobVertex / ExecutionVertices /
Execution from the DefaultExecutionGraph.
This interface enables subscribing to failures that are detected from the JobMaster side (e.g.,
from within the
ExecutionGraph).Internal window function for wrapping an
AllWindowFunction that takes an Iterable
when the window state also is an Iterable.Internal window function for wrapping a
WindowFunction that takes an Iterable
when the window state also is an Iterable.Internal window function for wrapping a
ProcessAllWindowFunction that takes an
Iterable when the window state also is an Iterable.Internal window function for wrapping a
ProcessWindowFunction that takes an
Iterable when the window state also is an Iterable.Internal window function for wrapping a
ProcessWindowFunction that takes an
Iterable when the window state also is an Iterable.Internal window function for wrapping a
WindowFunction that takes an Iterable
when the window state also is an Iterable.This interface is the current context of a keyed state.
The default
InternalKeyContext implementation.This class defines the internal interface for all keyed state.
The
InternalKvState is the root of the internal state type hierarchy, similar to the
State being the root of the public API state hierarchy.The state entry visitor which supports remove and update of the last returned entries.
The peer to the
ListState in the internal state type hierarchy.This class defines the internal interface for list state.
The
InternalLongWatermarkDeclaration class implements the AbstractInternalWatermarkDeclaration interface and provides additional functionality specific to
long-type watermarks.The peer to the
MapState in the internal state type hierarchy.This class defines the internal interface for map state.
The peer to the
MergingState in the internal state type hierarchy.This class defines the internal interface for merging state.
Special
MetricGroup representing an OperatorCoordinator.Metric group that contains shareable pre-defined IO-related metrics.
Special
MetricGroup representing an Operator.A state that is partitioned into different namespaces.
Interface for collection that gives in order access to elements w.r.t their priority.
Internal reusable context wrapper.
Internal reusable context wrapper.
Internal reusable context wrapper.
Internal context wrapper for async window operator.
Internal reusable context wrapper.
The peer to the
ReducingState in the internal state type hierarchy.This class defines the internal interface for reducing state.
Internal window function for wrapping an
AllWindowFunction that takes an Iterable
when the window state is a single value.Internal window function for wrapping a
WindowFunction that takes an Iterable
when the window state is a single value.Internal window function for wrapping a
ProcessAllWindowFunction that takes an
Iterable when the window state is a single value.Internal window function for wrapping a
ProcessWindowFunction that takes an
Iterable when the window state is a single value.Internal window function for wrapping a
ProcessWindowFunction that takes an
Iterable when the window state is a single value.Internal window function for wrapping a
WindowFunction that takes an Iterable
when the window state is a single value.Special
MetricGroup representing an Operator.Special
MetricGroup representing an Operator.Special
MetricGroup representing an Operator.Special
MetricGroup representing an SplitEnumerator.Internal interface for in-flight timers.
Interface for working with time and timers.
An implementation of
InternalTimerService that is used by AbstractAsyncStateStreamOperator.InternalTimerService that stores timers on the Java heap.Serialization proxy for the timer services for a given key-group.
A snapshot of internal timers, containing event and processing timers and the serializers to use
to write / read them.
Readers and writers for different versions of the
InternalTimersSnapshot.A reader for a
InternalTimersSnapshot.A writer for a
InternalTimersSnapshot.A
TypeSerializer used to serialize/deserialize a TimerHeapInternalTimer.An entity keeping all the time-related services.
A provider pattern for creating an instance of a
InternalTimeServiceManager.Signals whether the watermark should continue advancing.
An entity keeping all the time-related services.
The peer to the
ValueState in the internal state type hierarchy.This class defines the internal interface for value state.
The
Watermark that contains extra information to be used internally.Internal interface for functions that are evaluated over keyed (grouped) windows.
A context for
InternalWindowFunction, similar to ProcessWindowFunction.Context but for
internal use.An
operator to execute time-bounded stream inner joins.A container for elements put in the left/write buffer.
A
serializer for the IntervalJoinOperator.BufferEntry.A
TypeSerializerSnapshot for IntervalJoinOperator.BufferEntrySerializer.The facade for the provided I/O manager services.
A version of the
IOManager that uses asynchronous I/O.An instance of this class represents a snapshot of the io-related metrics of a single task.
IO metrics information.
A source event that notify the source operator of the backlog status.
Functional interface for checking whether a slot is available and free.
A
StateSnapshot that can return an iterator over all contained StateEntries.Base class for iteration
TaskEvent transmitting operator aggregators.The head is responsible for coordinating an iteration and can run a
Driver inside.An intermediate iteration task, which runs a
Driver
inside.The task responsible for synchronizing all iteration heads, implemented as an output task.
An iteration tail, which runs a driver inside.
Responsible for installing a process-wide JAAS configuration.
A
SecurityModuleFactory for JaasModule.A
TypeSerializer that uses Java serialization.Serializer configuration snapshot for compatibility and format evolution.
Message indicating a registration rejection from the
JobMaster for the TaskExecutor.Message indicating a successful
JobMaster and TaskExecutor registration.Request handler that returns the aggregated accumulators of a job.
Message headers for the
JobAccumulatorsHandler.Response type of the
JobAccumulatorsHandler.Json serializer for the
JobAccumulatorsInfo.Json serializer for the
JobAccumulatorsInfo.Request parameter for job accumulator's handler
JobAccumulatorsHandler.Information about allocations of Job Vertices.
Information about the allocations of a single Job Vertex.
An exception which is thrown by the JobClient if a job is aborted as a result of a user
cancellation.
JobCancellationFailedException is thrown if the cancellation of a job failed.Request handler for the cancel and stop request.
Message headers for the
JobCancellationHandler.Parameters for job related REST handlers.
The JobCheckpointingSettings are attached to a JobGraph and describe the settings for the
asynchronous checkpoints of the JobGraph, such as interval.
Receive the heartbeat from the client.
Message headers for the
JobClientHeartbeatHandler.The parameter when the client reports heartbeat.
RequestBody to report heartbeat for client.Handler serving the job configuration.
Message headers for the
JobConfigHandler.Response type of the
JobConfigHandler.Json deserializer for the
JobConfigInfo.Nested class to encapsulate the execution configuration.
Json serializer for the
JobConfigInfo.An actor message with a detailed overview of the current status of a job.
The CurrentAttempts holds the attempt number of the current representative execution attempt,
the attempt numbers of all the running attempts, and whether the current execution has
reached terminal state.
Handler returning the details for the specified job.
Message headers for the
JobDetailsHandler.Details about a job.
Detailed information about a job vertex.
This class represent edges (communication channels) in a job graph.
A class that represents an event that happens during the job execution.
The manager for recording and replaying
JobEvent.The interface for process job event.
A class hosts all the type ids of events.
A store for recording the
JobEvent.Indicates that a job has failed.
Handler serving the job exceptions.
Message headers for the
JobExceptionsHandler.JobExceptionsInfoWithHistory providing a history of previously caused failures.Json equivalent of
ExceptionHistoryEntry.JobExceptionHistory collects all previously caught errors.Json equivalent of
RootExceptionHistoryEntry.This exception is the base exception for all exceptions that denote any failure during the
execution of a job.
Returns the
JobExecutionResult for a given JobID.Helper class to simplify job failure reporting through a metric group.
The JobGraph represents a Flink dataflow program, at the low level that the JobManager accepts.
Builder for the
JobGraph.JobInformation created from a JobGraph.Interface which allows to retrieve the
JobGraph.This interface defines operations for components that are interested in being notified when new
job vertices are added to the job graph.
Json deserializer for
JobID.Path parameter identifying jobs.
Json serializer for
JobID.Handler for job IDs.
Message headers for the
JobIdsWithStatusOverview.An overview of how many jobs are in which status.
Container class for job information which is stored in the
ExecutionGraph.Information about the job.
Information about a single vertex.
An exception indicating that the job has failed in the INITIALIZING job status.
Interface for actions called by the
DefaultJobLeaderIdService.Service which retrieves for a registered job the current job leader id (the leader id of the job
manager responsible for the job).
Listener for the
JobLeaderService.This service has the responsibility to monitor the job leaders (the job manager which is leader
for a given job) for all registered jobs.
The
CheckpointStorage checkpoints state directly to the JobManager's memory (hence the
name), but savepoints will be persisted to a file system.Rest handler which serves the custom log file from JobManager.
Headers for the
JobManagerCustomLogHandler.Handler which serves the jobmanager's environment variables.
Message headers for the
JobManagerEnvironmentHandler.Base exception thrown by the
JobMaster.Flink internal memory components of Job Manager.
FlinkMemoryUtils for Job Manager.Handler which serves the jobmanager's configuration of a specific job.
Message headers for the
JobManagerJobConfigurationHandler.Handler which serves the jobmanager's environment variables of a specific job.
Message headers for the
JobManagerJobEnvironmentHandler.Special
MetricGroup representing everything belonging to a
specific job, running on the JobManager.Factory interface for
JobManagerJobMetricGroup.The scope format for the
JobMetricGroup.Rest handler which serves the log files from JobManager.
Headers for the
JobManagerLogFileHandler.Handler which serves detailed JobManager log list information.
Headers for the
JobManagerLogListHandler.Headers for the log url retriever of JobManager.
Special
MetricGroup representing a JobManager.Factory interface for
JobManagerMetricGroup.Handler that returns JobManager metrics.
Special
MetricGroup representing everything belonging to the
components running on the JobManager of an Operator.Handler that returns job manager operator metrics.
The scope format for the
JobManagerOperatorMetricGroup.Describe the specifics of different resource dimensions of the JobManager process.
JobManager utils to calculate
JobManagerProcessSpec and JVM args.Rest handler which serves the profiler result file from JobManager.
Headers for the
JobManagerProfilingFileHandler.Rest handler which serves the profiling info from the JobManager.
Headers for the
JobManagerProfilingHandler.Rest handler which serves the profiling list from the JobManager.
Headers for the
JobManagerProfilingHandler.Container for JobManager related registration information, such as the leader id or the job id.
Interface for a runner which executes a
JobMaster.Factory for a
JobManagerRunner.JobManagerRunner collects running jobs represented by JobManagerRunner.The result of the
JobManagerRunner.The scope format for the
JobManagerMetricGroup.Utility class which holds all auxiliary shared services used by the
JobMaster.Headers for the
JobManagerStdoutFileHandler.This class encapsulates the data from the job manager to restore a task.
Rest handler which serves the thread dump info from the JobManager.
Headers for the
JobManagerThreadDumpHandler.JobMaster implementation.
Configuration for the
JobMaster.Base class for
JobMaster related exceptions.JobMaster rpc gateway interface.The
JobMaster fencing token.Gateway to send an
OperatorEvent or CoordinationRequest from the Task Manager to
to the OperatorCoordinator on the JobManager side.Utility for tracking partitions and issuing release calls to task executors and shuffle masters.
Utility for tracking partitions and issuing release calls to task executors and shuffle masters.
Base class for responses from the ResourceManager to a registration attempt by a JobMaster.
Interface which specifies the JobMaster service.
Factory for the
JobMasterService.Leadership runner for the
JobMasterServiceProcess.Factory which creates a
JobMasterServiceLeadershipRunner.JobMasterServiceProcess is responsible for running a
JobMasterService.Factory for the
JobMasterServiceProcess.Message parameters which require a job path parameter.
Special abstract
MetricGroup representing everything belonging
to a specific job.Request handler that returns for a given job a list of all available metrics or the values for a
set of metrics.
Exception indicating that a Flink job has not been finished.
PermanentBlobService extension that gives access to register and release job artifacts.Factory for components that are responsible for persisting a job for recovery.
Handler serving the job execution plan.
Message headers for the
JobPlanHandler.Response type of the
JobPlanHandler.Simple wrapper around a raw JSON string.
Json deserializer for the
JobPlanInfo.RawJson.Json serializer for the
JobPlanInfo.RawJson.Information about the parallelism of job vertices.
Body for change job requests.
Rest handler for reading current
resource requirements of a given job.Headers for REST request to get details on job's resources.
Rest handler for updating
resource requirements of a given job.Headers for REST request to patch a job.
RestEndpointFactory which creates a MiniDispatcherRestEndpoint.Similar to
JobExecutionResult but with an optional SerializedThrowable when the job failed.Builder for
JobResult.JSON deserializer for
JobResult.JobResultEntry is the entity managed by the JobResultStore.JSON serializer for
JobResult.A storage for the results of globally terminated jobs.
The set of configuration options relating to the Job Result Store.
A plan that describes how to execute
JobGraph.Assignment of a slot to some target (e.g. a slot sharing group).
MessageQueryParameter for selecting jobs when aggregating metrics.Job level shuffle context which can offer some job information like job ID and through it, the
shuffle plugin notify the job to stop tracking the lost result partitions.
The default implementation of
JobShuffleContext.An overview of how many jobs are in which status.
Overview handler for jobs.
Message headers for
JobsOverviewHandler.Handler for requesting job status.
The status of a specific job.
Interface for observers that monitor the status of a job.
A simple message that holds the state of a job execution.
Metrics that capture the time that a job spends in each
JobStatus.Interface for querying the state of a job and the timestamps of state transitions.
Listens for and exposes the current job state and state timestamps.
This exception denotes an error while submitting a job to the JobManager.
This handler can be used to submit jobs to a Flink cluster.
These headers define the protocol for submitting a job to a flink cluster.
Request for submitting a job.
Descriptor for a distributed cache file.
Response to the submission of a job, containing a URL from which the status of the job can be
retrieved from.
A
JobTable's task is to manage the lifecycle of a job on the TaskExecutor.A connection contains services bound to the lifetime of a connection with a JobManager.
A job contains services which are bound to the lifetime of a Flink job.
Services associated with a job.
Message parameters which require a job path parameter and a TaskManager id path parameter.
Enum to distinguish JobGraphs between batch and streaming, currently used by the scheduler.
The base class for job vertexes.
The context exposes some runtime infos for finalization.
Request handler for the job vertex accumulators.
Message headers for the
JobVertexAccumulatorsHandler.Response type of the
JobVertexAccumulatorsHandler.Request handler for the job vertex back pressure.
Message headers for the
JobVertexBackPressureHandler.Response type of the
JobVertexBackPressureHandler.Nested class to encapsulate the sub tasks back pressure.
Level of vertex back-pressure.
Status of vertex back-pressure.
Helper class encapsulates all necessary information and configurations required during the
construction of job vertices.
Request handler for the job vertex details.
Message headers for the
JobVertexDetailsHandler.Response type of the
JobVertexDetailsHandler.Request handler for the job vertex Flame Graph.
Message headers for the
JobVertexFlameGraphHandler.Message parameters for job vertex Flame Graph REST handler.
Job vertex level implement for
ForwardGroup.A class for statistically unique job vertex IDs.
Jackson deserializer for
JobVertexID.Jackson deserializer for
JobVertexID.Jackson serializer for
JobVertexID used as a key serializer.Path parameter specifying a
JobVertexID.Jackson serializer for
JobVertexID.This class describe how a job vertex consume an input(intermediate result).
Message parameters for job vertex REST handlers.
Deprecated.
This class is subsumed by
SubtaskMetricsHandler and is only kept for
backwards-compatibility.Information about the parallelism of job vertices.
A request handler that provides the details of a job vertex, including id, name, and the runtime
and metrics of all its subtasks aggregated by TaskManager.
Message headers for the
JobVertexTaskManagersHandler.Response type of the
JobVertexTaskManagersHandler.Detailed information about task managers.
Handler that returns the watermarks given a
JobID and JobVertexID.MessageHeaders for retrieving watermarks.The join driver implements the logic of a join operator at runtime.
JoinedStreams represents two DataStreams that have been joined.A join operation that has
KeySelectors defined for both inputs as well as
a WindowAssigner.Interface of an iterator that performs the logic of a match task.
Interface for all classes that want to participate in the archiving of job-related json
responses.
A holder for the singleton Jackson JSON factory.
Helper class for generating a JSON representation from a
StreamGraph.JVM metaspace and overhead memory sizes.
Options to calculate JVM Metaspace and Overhead.
A utility that guards against blocking shutdown hooks that block JVM shutdown.
Utilities for
ManagementFactory.Utilities to write.
Provides Kerberos login functionality.
Provides vendor-specific Kerberos
AppConfigurationEntry instances.Key accounting unit holds the current in-flight key and tracks the corresponding ongoing records,
which is used to preserve the ordering of independent chained
StateFuture.Interface for setting and querying the current key of keyed operations.
This interface is used to optimize the calls of
Input.setKeyContextElement(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<IN>), StreamOperator.setKeyContextElement1(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<?>) and StreamOperator.setKeyContextElement2(org.apache.flink.streaming.runtime.streamrecord.StreamRecord<?>).Interface for objects that have a key attribute.
Serialization proxy for all meta data in keyed state backends.
A function to be applied to a
BroadcastConnectedStream that
connects BroadcastStream, i.e.A keyed
AbstractBroadcastStateTransformation.A function that processes elements of two keyed streams and produces a single output one.
A
StreamOperator for executing keyed KeyedCoProcessFunction.AbstractMultipleInputTransformation implementation for keyed streams.KeyedPartitionWindowedStream represents a data stream that collects all records with the
same key separately into a full window.A keyed function that processes elements of a stream.
A
StreamOperator for executing KeyedProcessFunctions.The
KeyedSortPartitionOperator sorts records of a partition on KeyedStream.A keyed state backend provides methods for managing keyed state.
Listener is given a callback when
KeyedStateBackend.setCurrentKey(K) is called (key context changes).Internal POJO implementing
StateBackend.KeyedStateBackendParametersCheckpoint output stream that allows to write raw keyed state in a partitioned way, split into
key-groups.
This factory produces concrete internal state objects.
A function to be applied to all keyed states.
Base for the handles of the checkpointed states in keyed streams.
A
KeyedStream represents a DataStream on which operator state is partitioned by
key using a provided KeySelector.Perform a join over a time interval.
IntervalJoined is a container for two streams that have keys for both sides as well as the
time boundaries over which elements should be joined.
Function to extract a key from a given object.
Part of a savepoint representing data for a single key group.
This interface exists as (temporary) adapter between the new
InternalPriorityQueue and
the old way in which timers are written in a snapshot.Part of a savepoint representing data for a single state entry in a key group.
This implementation of
InternalPriorityQueue is internally partitioned into sub-queues
per key-group and essentially works as a heap-of-heaps.KeyGroupPartitionedPriorityQueue.PartitionQueueSetFactory<T,PQS extends InternalPriorityQueue<T> & HeapPriorityQueueElement>
Factory that produces the sub-queues that represent the partitions of a
KeyGroupPartitionedPriorityQueue.Class that contains the base algorithm for partitioning data into key-groups.
This functional interface defines how one element is read from a
DataInputView.This functional interface defines how one element is written to a
DataOutputView.Functional interface to consume elements from a key group.
This represents the result of key-group partitioning.
This class defines a range of key-group indexes.
This class combines a key-group range with offsets that correspond to the key-groups in the
range.
This interface offers ordered random read access to multiple key group ids.
A
KeyGroupsStateHandle that describes a savepoint in the unified format.A handle to the partitioned stream operator state after it has been checkpointed.
This class provides access to an input stream that contains state data for one key group and the
key group id.
Partitioner selects the target channel based on the key group index.
Utility class that contains helper methods to manipulating
KeySelector for streaming.A key selector for selecting individual array fields as keys and returns them as a Tuple.
A key selector for selecting key fields via a TypeComparator.
Key extractor that extracts a single field via a generic comparator.
Iterator that over all key-value state entries in a
KeyedStateBackend.An interface for the Queryable State Client Proxy running on each Task Manager in the cluster.
An entry holding the
InternalKvState along with its KvStateInfo.Handler for common queryable state logic.
Metadata about a
InternalKvState.Location information for all key groups of a
InternalKvState instance.Oracle for
KvStateLocation in the cluster.Simple registry, which maps
InternalKvState registration notifications to KvStateLocation instances.A registry for
InternalKvState instances per task manager.Gateway to report key-value state registration and deregistrations.
A listener for a
KvStateRegistry.An interface for the Queryable State Server running on each Task Manager in the cluster.
KvState related components of each
TaskExecutor instance.An interface for handling and associating global failures with labels from
FailureEnricher.ConnectionStateListener which records the last state it was notified about.Special record type carrying a timestamp of its creation time at a source operator and the
vertexId and subtask index of the operator.
The
LatencyStats objects are used to track and report on the behavior of latencies across
measurements.Granularity for latency metrics.
Config to create latency tracking state metric.
LatencyTrackingStateFactory<K,N,V,S extends org.apache.flink.api.common.state.State,IS extends InternalKvState<K,N,?>>
Factory to create
AbstractLatencyTrackState.Interface which has to be implemented to take part in the leader election process of the
LeaderElectionService.A leader election driver that allows to write
LeaderInformation for multiple components.Listener interface for state changes of the
LeaderElectionDriver.Factory for
LeaderElectionDriver.This exception is used in leader election-related scenarios.
Interface for a service which allows to elect a leader among a group of contenders.
LeaderElectionUtils collects helper methods to handle LeaderElection-related issues.LeaderGatewayRetriever<T extends org.apache.flink.runtime.rpc.RpcGateway>
Retrieves and stores the leading
RpcGateway.Information about leader including the confirmed leader session id and leader address.
A register containing the
LeaderInformation for multiple contenders based on their
componentId.A
LeaderRetrievalDriver is responsible for retrieves the current leader which has been
elected by the LeaderElectionDriver.Factory for creating
LeaderRetrievalDriver with different implementation.Interface which should be implemented to notify to
LeaderInformation changes in LeaderRetrievalDriver.This exception is thrown by the
LeaderRetrievalUtils when
the method retrieveLeaderGateway fails to retrieve the current leader's gateway.SimpleChannelInboundHandler which encapsulates the leader retrieval logic for the REST
endpoints.Classes which want to be notified about a changing leader by the
LeaderRetrievalService
have to implement this interface.This interface has to be implemented by a service which retrieves the current leader and notifies
a listener about it.
Utility class to work with
LeaderRetrievalService class.Helper class which is used by the retrieveLeaderInformation method to retrieve the leader's
rpc URL and the current leader session ID.
Retrieves and stores the current leader address.
This exception is used in the scenario that the leadership is lost.
The left outer join driver implements the logic of an outer join operator at runtime.
Deprecated.
Replaced by
KeyedCoProcessOperator which takes KeyedCoProcessFunctionDeprecated.
Replaced by
KeyedProcessOperator which takes KeyedProcessFunctionThis Transformation represents a stream Sink.
A
TransformationTranslator for the LegacySinkTransformation.This represents a Source.
The LibraryCacheManager is responsible for creating and managing the user code class loaders.
Handle to retrieve a user code class loader for the associated job.
Lease which allows to signal when the user code class loader is no longer needed.
Lineage dataset represents the source or sink in the job.
Facet interface for dataset.
Lineage edge from source to sink.
Job lineage graph that users can get sources, sinks and relationships from lineage and manage the
relationship between jobs and tables.
Utils for building lineage graph from transformations.
Lineage vertex represents the connectors in lineage graph, including source
SourceLineageVertex and sink.Create lineage vertex for source and sink in DataStream.
Deprecated.
If you need to do non-keyed state snapshots of your operator, use
CheckpointedFunction.Encapsulates a logic of serialization and deserialization of a list with a delimiter.
Simple memory segment source that draws segments from a list.
An adaptor that transforms
InternalListState into
InternalListState.A buffer pool used to manage a number of
Buffer instances from the NetworkBufferPool.This registry is responsible for deleting changlog's local handles which are not in use.
A connection manager implementation to bypass setup overhead for task managers running in local
execution mode.
An input channel, which requests a local subpartition.
LocallyCleanableResource is supposed to be implemented by any class that provides
artifacts for a given job that need to be cleaned up after the job reached a local terminal
state.An input channel reads recovered state from previous unaligned checkpoint snapshots and then
converts into
LocalInputChannel finally.This class encapsulates the completed configuration for local recovery, i.e. the root directories
into which all file-based snapshots can be written and the general mode for the local recover
feature.
Provides directories for local backup or local recovery.
Implementation of
LocalSnapshotDirectoryProvider.Enumeration of all available local processing strategies tasks.
The LocalStreamEnvironment is a StreamExecutionEnvironment that runs the program locally,
multi-threaded, in the JVM where the environment is instantiated.
Defines the location preference constraint.
This class implements a
SlotSelectionStrategy that is based on location preference hints.Parameters used by
TaskManagerCustomLogHandler and JobManagerCustomLogHandler.Represents an edge in
LogicalTopology, i.e.An abstraction of logical files in file-merging checkpoints.
ID for
LogicalFile.Pipelined region on logical level, i.e.,
JobVertex level.Utils for computing
LogicalPipelinedRegions.Represents a data set produced by a
LogicalVertex, i.e.A logical slot represents a resource on a TaskManager into which a single task can be deployed.
Payload for a logical slot.
Represents a logical topology, i.e.
Represents a vertex in
LogicalTopology, i.e.Contains information about one log of TaskManager.
Class containing a collection of
LogInfo.Response of log URLs.
Minimal implementation of an array-backed list of longs
A
WatermarkCombiner for unaligned LongWatermarks.This class implements
HeapPriorityQueue.HeapPriorityQueueElement to help find the maximum/minimum
LongWatermark.Implementation of
BlockCompressionFactory for Lz4 codec.Encode data into LZ4 format (not compatible with the LZ4 Frame format).
Decode data written with
Lz4BlockCompressor.An executable bound to a specific operator in the chain, such that it can be picked for
downstream mailbox.
Interface for the default action that is repeatedly invoked in the mailbox-loop.
This controller is a feedback interface for the default action to interact with the mailbox
execution.
Represents the suspended state of a
MailboxDefaultAction, ready to resume.A factory for creating mailbox executors with a given priority.
Implementation of an executor service build around a mailbox-based execution model.
Mailbox metrics controller class.
This class encapsulates the logic of the mailbox-based execution model.
Implementation of
MailboxDefaultAction.Controller that is connected to a MailboxProcessor instance.A helper class to let operators emit watermarks incrementally from mailbox.
This interface provides a context in which operators can initialize by registering to managed
state (i.e. state that is managed by state backends).
Utils for configuration and calculations related to managed memory and its various use cases.
This interface provides a context in which operators that use managed state (i.e. state that is
managed by state backends) can perform a snapshot.
Map task which is executed by a Task Manager.
MapPartition task which is executed by a Task Manager.
The
MapPartitionIterator is an iterator used in the MapPartitionOperator.The task
main thread will add records to it.The
MapPartitionOperator is used to process all records in each partition on non-keyed
stream.A repartitioner that assigns the same channel state to multiple subtasks according to some
mapping.
An adaptor that transforms
InternalMapState into
InternalMapState.This strategy is used to decide whether partition needs to be marked as finished.
Collection of methods to deal with checkpoint master hooks.
Simple encapsulation of state generated by checkpoint coordinator.
The interface for hooks that can be called by the checkpoint coordinator when triggering or
restoring a checkpoint.
A factory to instantiate a
MasterTriggerRestoreHook.Indicates that the
BroadcastVariableMaterialization has materialized the broadcast
variable at some point but discarded it already.CheckpointStreamFactory that produces streams that write to in-memory byte arrays.A
CheckpointStateOutputStream that writes into a byte array.An exception to be thrown when a memory allocation operation is not successful.
An implementation of a checkpoint storage for the
JobManagerCheckpointStorage.The memory block iterator is an iterator that always buffers a block of elements in memory.
ExecutionGraphInfoStore implementation which stores the ArchivedExecutionGraph in
memory.A thread the periodically logs statistics about:
Heap and non-heap memory usage
Memory pools and pool usage
Garbage collection times and counts
The memory manager governs the memory that Flink uses for sorting, hashing, caching or off-heap
state backends (e.g.
An exception to be thrown when a memory reservation operation is not successful.
The data client is used to fetch data from memory tier.
The implementation of
TierFactory for memory tier.The memory tier implementation of
TierProducerAgent.An iterator that returns a sorted merge of the sequences of elements from a set of iterators,
assuming those sequences are ordered themselves.
An adaptor that transforms
InternalMergingState
into InternalMergingState.A
WindowAssigner that can merge windows.Callback to be used in
MergingWindowAssigner.mergeWindows(Collection, MergeCallback) for specifying which
windows should be merged.Utility for keeping track of merging
Windows when using a MergingWindowAssigner in a WindowOperator.Callback for
MergingWindowSet.addWindow(Window, MergeFunction).This class links
RequestBodys to ResponseBodys types and contains meta-data
required for their http headers.This class represents a single path/query parameter that can be used for a request.
Enum for indicating whether a parameter is mandatory or optional.
This class defines the path/query
MessageParameters that can be used for a request.This class represents path parameters of a request.
This class represents query parameters of a request.
The wrapper manages metadata output stream close and commit.
Deserializer for checkpoint metadata.
Helper to access
MetadataSerializers for specific format versions.Deserializer for checkpoints written in format
1 (Flink 1.2.x format).(De)serializer for checkpoint metadata format version 2.
Base (De)serializer for checkpoint metadata format version 2 and 3.
A context that keeps information needed during serialization.
(De)serializer for checkpoint metadata format version 3.
V4 serializer that adds
CheckpointProperties
serialization.V5 serializer that adds Operator name and uid.
Response type for a Metric and Metric-Value-Pair.
Response type for a collection of metrics.
JSON deserializer for
MetricCollectionResponseBody.JSON serializer for
MetricCollectionResponseBody.A container for a dumped metric that contains the scope, name and value(s) of the metric.
Container for the value of a
Counter.Container for the value of a
Gauge as a string.Container for the values of a
Histogram.Container for the rate of a
Meter.Utility class for the serialization of metrics.
Deserializer for reading a list of
MetricDumps from a MetricDumpSerialization.MetricSerializationResult.Serializes a set of metrics into a
MetricDumpSerialization.MetricSerializationResult.This class encapsulates all serialized metrics and a count for each metric type.
The MetricFetcher can be used to fetch metrics from the JobManager and all registered
TaskManagers.
Implementation of
MetricFetcher which fetches metrics from the MetricQueryServiceGateway.A filter for metrics.
Collection of metric names.
The MetricQueryService creates a key-value representation of all metrics currently registered
with Flink when queried.
MetricQueryService rpc gateway interface.Retriever for
MetricQueryServiceGateway.Interface for a metric registry.
Configuration object for
MetricRegistryImpl.A MetricRegistry keeps track of all registered
Metrics.TODO: add javadoc.
The available aggregations.
MessageQueryParameter for filtering metrics provided by MetricStore.A component that can register metrics.
Nested data-structure to store metrics.
Structure containing metrics of a single component.
Sub-structure containing metrics of a single subtask.
Sub-structure containing metrics of a single TaskManager.
Sub-structure containing metrics of a single Task.
Utility class to register pre-defined metric sets.
Simple utility class that resolves file extensions to MIME types.
MiniCluster to execute Flink jobs locally.
Factory which returns always the common
RpcService.HA Services to use.
Internal factory for
RpcService.Configuration object for the
MiniCluster.Builder for the MiniClusterConfiguration.
A
JobClient for a MiniCluster.Determines the behavior of the
MiniClusterJobClient when the job finishes.REST endpoint for the ApplicationClusterEntryPoint.
A
Gauge for exposing the minimum watermark of chosen WatermarkGauges.Configuration class which contains the parsed command line arguments for the
ClusterEntrypoint.A class can be used to extract the configuration from command line and modify it.
An input that wraps an underlying input and sorts the incoming records.
A wrapper that combines sorting
inputs with a InputSelectable
that should be used to choose which input to consume next from.A
TransformationTranslator for the MultipleInputTransformation and the KeyedMultipleInputTransformation.A utility for
FileUploadHandler to determine whether it
should accept a request.This class is semi-thread safe.
This handler is mainly used for selecting the next available input index in
StreamMultipleInputProcessor.Interface for stream operators with multiple
Inputs.A
StreamTask for executing a MultipleInputStreamOperator and supporting the
MultipleInputStreamOperator to select input for reading.AbstractMultipleInputTransformation implementation for non-keyed streams.An actor messages describing details of various jobs.
MultipleRecordWriters<T extends org.apache.flink.core.io.IOReadableWritable>
The specific delegate implementation for the multiple outputs case.
An implementation of a Hybrid Hash Join.
Iterate all the elements in memory which has not been probed during probe phase.
This class is a mutable version of the
IOMetrics class that allows adding up IO-related
metrics.MutableReader<T extends org.apache.flink.core.io.IOReadableWritable>
A record-oriented reader for mutable record types.
MutableRecordReader<T extends org.apache.flink.core.io.IOReadableWritable>
Implementation of the record-oriented reader for mutable record types.
Mutability extension to the
VertexAttemptNumberStore.Mutability extension to the
VertexParallelismStore.A named version of
BiFunction.A named callback that can be identified and checkpoint.
A named version of
Consumer.A named version of
Function.Extends around Netty's
PooledByteBufAllocator with strict control over the number of
created arenas.NettyConnectionId indicates the unique id of netty connection.NettyConnectionReader is used by TierConsumerAgent to read buffer from netty.The default implementation of
NettyConnectionReader.NettyConnectionWriter is used by TierProducerAgent to write buffers to netty
connection.The default implementation of
NettyConnectionWriter.A simple and generic interface to serialize messages to Netty's buffer space.
Decodes messages from the received netty buffers.
Partition request client for remote partition requests.
Implementation of
PartitionRequestListener for netty partition request.The
NettyPayload represents the payload that will be transferred to netty connection.NettyPayloadManager is used to contain all netty payloads from a storage tier.Defines the server and client channel handlers, i.e. the protocol, used by netty.
NettyServiceProducer is used as the callback to register NettyConnectionWriter
and disconnect netty connection in TierProducerAgent.Default implementation of
ShuffleDescriptor for NettyShuffleMaster.Local partition connection information.
Remote partition connection information with index to query partition.
Information for connection to partition producer for shuffle exchange.
The implementation of
ShuffleEnvironment based on netty network communication, local
memory and disk files.Configuration object for the network stack.
Default
ShuffleMaster for netty and local file based shuffle implementation.Factory for netty shuffle service metrics.
Netty based shuffle service implementation.
Utils to calculate network memory requirement of a vertex from network configuration and details
of input and output.
Utility class for logging actions that happened in the network stack for debugging purposes.
Wrapper for pooled
MemorySegment instances.The NetworkBufferPool is a fixed size pool of
MemorySegment instances for the network
stack.Channel handler to read and write network messages on client side.
Simple wrapper for the partition readerQueue iterator, which increments a sequence number for
each returned buffer and remembers the receiver ID.
Exception for not finding suitable security factories.
A source event sent from the SplitEnumerator to the SourceReader to indicate that no more splits
will be assigned to the source reader anymore.
A
NonAdaptiveExecutionHandler implements the AdaptiveExecutionHandler interface
to provide an immutable static job graph.The
NonAdaptiveExecutionPlanSchedulingContext is a final class that implements a
scheduling context for execution plans that do not require adaptive changes.A Non-Blocking
PhysicalFilePool which will always provide usable physical file without
blocking.Used by operator chain and represents a non-chained output of the corresponding stream operator.
Abstract class to implement custom checkpoint output streams which should not be closable for
user code.
Decorator for input streams that ignores calls to
InputStream.close().Decorator for input streams that ignores calls to
OutputStream.close().NonKeyedPartitionWindowedStream represents a data stream that collects all records of
each subtask separately into a full window.* Gateway for restful endpoints without leader retrieval logic. * *
A checkpoint storage location for the
MemCheckpointStreamFactory in case no durable
persistence for metadata has been configured.NonRecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>
The specific delegate implementation for the non output case like sink task.
Implementation of an iterator that fetches a block of data into main memory and offers resettable
access to the data in that block.
An implementation of the
JoinTaskIterator that
uses a hybrid-hash-join internally to match the records with equal key.An implementation of the
JoinTaskIterator that
uses a hybrid-hash-join internally to match the records with equal key.A
DeserializationDelegate that always creates a new instance upon deserialization.The key grouped iterator returns a key and all values that share the same key.
This class wraps a
MutableObjectIterator into a regular Iterator.ResourceAllocator that not support to allocate/release resources.
No-op implementation of
BlocklistHandler.The factory to instantiate
NoOpBlocklistHandler.A chained driver that just passes on the input as the output
No-op implementation of the
DeclareResourceRequirementServiceConnectionManager.DelegationTokenManager implementation which does nothing.A
DispatcherBootstrap which submits the provided job graphs for
execution upon dispatcher initialization.A driver that does nothing but forward data from its input to its output.
No-op implementation of
ExecutionDeploymentListener.Singleton
ExecutionPlanStoreWatcher empty implementation.HeartbeatManager implementation which does nothing.HeartbeatServices implementation which does nothing.The empty implementation for
TierMasterAgent.Metric registry which does nothing.
A dummy implementation of the
ResultSubpartitionView.A security context that simply runs a Callable without performing a login action.
Default security context factory for
NoOpSecurityContext.SlotAllocationSnapshotPersistenceService that does nothing.A dummy implementation of the
TaskActions which is mainly used for tests.This class implements a
TaskLocalStateStore with no functionality and is used when local
recovery is disabled.The empty implementation for
TierShuffleDescriptor.An implementation of
TimestampsAndWatermarks where all watermarking/event-time operations
are no-ops.Indicates resource allocation failures.
Restart strategy which does not restart tasks when tasks fail.
The factory for creating
NoRestartBackoffTimeStrategy.An empty implementation of
CheckpointStateToolset.A special exception that indicates that an element was not found and that the request should be
answered with a
404 return code.Does not release intermediate result partitions during job execution.
Factory for
NotReleasingPartitionGroupReleaseStrategy.Interface for completion actions once a Flink job has reached a terminal state.
Interface for stream operators with one input.
A factory to create
OneInputStreamOperator.A
StreamTask for executing a OneInputStreamOperator.This Transformation represents the application of a
OneInputStreamOperator to one input Transformation.A
TransformationTranslator for the OneInputTransformation.Interface for all classes that want to participate in the archiving of job-related json responses
but only provide
AccessExecutionGraph-related information.OnMainThreadJobManagerRunnerRegistry implements JobManagerRunnerRegistry guarding
the passed JobManagerRunnerRegistry instance in a way that it only allows modifying
methods to be executed on the component's main thread.An opaque memory resource, meaning a memory resource not understood by Flink or the JVM.
Exception indicating that a requested operation already exists and has failed.
Any operation key for the
AbstractAsynchronousOperationHandlers must extend this class.Encapsulates the result of an asynchronous operation.
Current status of an asynchronous operation.
OperatorAttributes element provides Job Manager with information that can be used to optimize job
performance.
The builder class for
OperatorAttributes.Serialization proxy for all meta data in operator state backends.
The
OperatorChain contains all operators that are executed as one chain within a single
StreamTask.Wrapper class to access the chained sources and their's outputs.
Helper class to help maintain the information of an operator chain.
A coordinator for runtime operators.
The context gives the OperatorCoordinator access to contextual information and provides a
gateway to interact with other components, such as sending operator events.
The provider creates an OperatorCoordinator and takes a
OperatorCoordinator.Context to pass to the
OperatorCoordinator.The
SubtaskGateway is the way to interact with a specific parallel instance of the
Operator (an Operator subtask), like sending events to the operator.This context is the interface through which the
CheckpointCoordinator interacts with an
OperatorCoordinator during checkpointing and checkpoint restoring.Handler for the
OperatorCoordinators.The
OperatorCoordinatorHolder holds the OperatorCoordinator and manages all its
interactions with the remaining components.Root interface for all events sent between
OperatorCoordinator and an OperatorEventHandler.The dispatcher through which Operators receive
OperatorEvents and through which they can
send OperatorEvents back to the OperatorCoordinator.An implementation of the
OperatorEventDispatcher.The gateway through which an Operator can send an
OperatorEvent to the OperatorCoordinator on the JobManager side.Interface for handlers of operator events on the operator side.
A class for statistically unique operator IDs.
Formed of a mandatory operator ID and optionally a user defined operator ID.
An interface to access basic properties of an operator in the context of its coordinator.
Helper class to help maintain the information of an operator.
An ID for physical instance of the operator.
An adaptor that transforms
InternalListState into
InternalListState.The scope format for the
InternalOperatorMetricGroup.This class represents the information about the finished operators.
This class finalizes
OperatorSnapshotFutures.Simple container class which contains the raw/managed operator state and key-group state handles
from all subtasks of an operator and therefore represents the complete state of a logical
operator.
Interface that combines both, the user facing
OperatorStateStore interface and the system
interface Snapshotable.Internal POJO implementing
StateBackend.OperatorStateBackendParametersCheckpoint output stream that allows to write raw operator state in a partitioned way.
Interface of a state handle for operator state.
The modes that determine how an
OperatorStreamStateHandle is assigned to tasks during
restore.Meta information about the operator state handle.
Interface that allows to implement different strategies for repartitioning of operator state as
parallelism changes.
Implementation of operator state restore operation.
State handle for partitionable operator state.
This class generates a string that can be used to identify an operator subtask.
This class encapsulates the state for one parallel instance of an operator.
The builder for a new
OperatorSubtaskState which can be obtained by OperatorSubtaskState.builder().Path parameter identifying operators.
Custom netty outbound handler factory in order to make custom changes on netty outbound data.
A
StreamOperator is supplied with an object of
this interface that can be used to emit elements and other messages, such as barriers and
watermarks, from an operator.Gauge metric measuring the output buffer pool usage gauge for
ResultPartitions.Gauge metric measuring the number of queued output buffers for
ResultPartitions.Gauge metric measuring the size in bytes of queued output buffers for
ResultPartitions.The OutputCollector collects records, and emits them to the
RecordWriters.The output emitter decides to which of the possibly multiple output channels a record is sent.
Interface for operator factories which create the sink operator containing an
OutputFormat.Deprecated.
Please use the
StreamingFileSink for writing to files from a streaming program.This is a wrapper for outputs to check whether the collected record has been emitted to a
downstream subtask or to a chained operator.
This interface represents the administrative interface to
TaskLocalStateStore, that only
the owner of the object should see.Utility for accessing the system page size.
The parallelism and
JobVertexInputInfos of a job vertex .Deprecated.
This interface is based on the
SourceFunction API, which is due to be
removed.Parser result factory used by the
CommandLineParser.Partial
DispatcherServices services container which needs to be completed before being
given to the Dispatcher.DispatcherFactory services container.PartialFinishedInputConsumableDecider is a special InputConsumableDecider.Factory for
PartialFinishedInputConsumableDecider.This class implements a priority-queue, which maintains a partial ordering of its elements such
that the least element can always be found in constant time.
Implementation of operator list state.
The
PartitionAggregateOperator is used to apply the aggregate transformation on all
records of each partition.Exception for failed partition requests due to connection failure with unreachable producer.
Partition descriptor for
ShuffleMaster to obtain ShuffleDescriptor.PartitionedFile is the persistent file type of sort-merge based blocking shuffle.File writer which can write buffers and generate
PartitionedFile.Exception for covering all the scenarios of consuming partition failure which causes the consumer
task failed, and the job master would decide whether to restart the producer based on this
exception.
PartitionFileReader defines the read logic for different types of shuffle files.A wrapper class of the reading buffer result, including the read buffers, the hint of
continue reading, and the read progress, etc.
This
PartitionFileReader.ReadProgress defines the read progress of the PartitionFileReader.The
PartitionFileWriter interface defines the write logic for different types of shuffle
files.The
PartitionFileWriter.SegmentBufferContext contains all the buffers belonging to the segment.The
PartitionFileWriter.SubpartitionBufferContext contains all the buffers belonging to one subpartition.Interface for strategies that decide when to release
ConsumedPartitionGroups.Factory for
PartitionGroupReleaseStrategy.Instantiates a
RegionPartitionGroupReleaseStrategy.Contains information where to find a partition.
Exception for failed partition requests due to non-existing partitions.
Exception returned to a TaskManager on JobMaster requesting partition state, if the producer of a
partition has been disposed.
Intermediate partition state checker to query the JobManager about the state of the producer of a
result partition.
Request execution state of partition producer, the response accepts state check callbacks.
Result of state query, accepts state check callbacks.
The
PartitionReduceOperator is used to apply the reduce transformation on all records of
each partition.Client to send messages or task events via network for
RemoteInputChannel.When the netty server receives a downstream task's partition request event and finds its upstream
task doesn't register its partition yet, the netty server will construct a
PartitionRequestListener and notify the listener when the task deploys itself and registers its
partition to ResultPartitionManager.Manages partition request listener with input channel id.
Thread-safe Utility for tracking partitions.
Utility for tracking partitions.
Encapsulates all information that a
PartitionTracker keeps for a partition.Factory for
JobMasterPartitionTracker.Lookup function for
TaskExecutorGateway.This transformation represents a change of partitioning of the input elements.
A
TransformationTranslator for the PartitionTransformation.Exception indicating a problem with the result partitions on the
TaskExecutor side.PartitionWindowedStream represents a data stream that collects all records of each
partition separately into a full window.Interface representing the description and metrics of a result partition.
A
AllWindowFunction that just emits each input element.A
WindowFunction that just emits each input element.A
RelativeClock whose time progress with respect to the wall clock can be paused and
un-paused.A pending checkpoint is a checkpoint that has been started, but has not been acknowledged by all
tasks that need to acknowledge it.
Result of the
PendingCheckpoint.acknowledgedTasks method.Statistics for a pending checkpoint that is still in progress.
Represents a pending task manager in the
SlotManager.Id of
PendingTaskManager.Interface for the measurement of the period of time.
Simple
CheckpointRecoveryFactory which creates and keeps separate CompletedCheckpointStore and CheckpointIDCounter for each JobID.PerJobCheckpointRecoveryFactory.CheckpointStoreRecoveryHelper<StoreType extends CompletedCheckpointStore>
Restores or creates a
CompletedCheckpointStore, optionally using an existing one.Provides a cache for permanent BLOB files including a per-job ref-counting and a staged cleanup.
BLOB key referencing permanent BLOB files.
A service to retrieve permanent binary large objects (BLOBs).
A checkpoint storage location for the
MemCheckpointStreamFactory when it durably persists
the metadata in a file system.An abstraction of physical files in file-merging checkpoints.
Functional interface to create the physical file.
Functional interface to delete the physical file.
A pool for reusing
PhysicalFile.Types of supported physical file pool.
The context of an
AllocatedSlot.Payload which can be assigned to an
AllocatedSlot.The provider serves physical slot requests.
The provider serves physical slot requests.
Represents a request for a physical slot.
Result of a
PhysicalSlotRequest.Represents a bulk of physical slot requests.
This class tracks a fulfillability timeout of a bulk of physical slot requests.
Default implementation of
PhysicalSlotRequestBulkChecker.Unique ID that allows for physical comparison between state handles.
A
Transformation that creates a physical operation.A pipelined in-memory only subpartition, which allows to reconnect after failure.
View over a pipelined in-memory only subpartition allowing reconnecting.
PipelinedRegion<VID extends VertexID,RID extends ResultID,V extends Vertex<VID,RID,V,R>,R extends Result<VID,RID,V,R>>
A pipelined region is a set of vertices connected via pipelined data exchanges.
Common utils for computing pipelined regions.
SchedulingStrategy instance which schedules tasks in granularity of pipelined regions.The factory for creating
PipelinedRegionSchedulingStrategy.A result output of a task, pipelined (streamed) to the receivers.
A pipelined in-memory only subpartition, which can be consumed once.
View over a pipelined in-memory only subpartition.
This is the last handler in the pipeline.
A placeholder state handle for shared state that will replaced by an original that was created in
a previous checkpoint.
Information of Pointwise result.
Helper class that computes VertexInputInfo for pointwise input.
PossibleInconsistentStateException represents errors that might have lead to an
inconsistent state within the HA resources.RequestSlotMatchingStrategy that takes the preferred allocations into account.Component to retrieve the preferred locations of an execution vertex.
This class implements a
SlotSelectionStrategy that is based on previous allocations and
falls back to using location preference hints if there is no previous allocation.Deprecated.
This interface will be removed in future versions.
A deque-like data structure that supports prioritization of elements, such they will be polled
before any non-priority elements.
This class is a wrapper over multiple alternative
OperatorSubtaskState that are (partial)
substitutes for each other and imposes a priority ordering over all alternatives for the
different states which define an order in which the operator should attempt to restore the state
from them.A builder for PrioritizedOperatorSubtaskState.
Interface for objects that can be compared by priority.
This interface works similar to
Comparable and is used to prioritize between two objects.Factory for
KeyGroupedInternalPriorityQueue instances.Base abstract class for functions that are evaluated over non-keyed windows using a context for
retrieving extra information.
A function that processes elements of a stream.
Defines the current processing time and handles all related actions, such as register timers for
tasks to be executed in the future.
Interface for
StreamOperatorFactory, which
enables stream operators to access ProcessingTimeService.A factory for creating processing time services with a given
MailboxExecutor.Utility for classes that implements the
ProcessingTimeService interface.A
Trigger that fires once the current system time passes the end of the window to which a
pane belongs.A function that processes two joined elements and produces a single output one.
Common Flink's options to describe its JVM process memory configuration for JM or TM.
Common interface for Flink JVM process memory components.
Common utils to parse JVM process memory configuration for JM or TM.
AbstractImitatingJobManagerMetricGroup implementation for process related metrics.A
StreamOperator for executing ProcessFunctions.Base abstract class for functions that are evaluated over keyed (grouped) windows using a context
for retrieving extra information.
Partition producer descriptor for
ShuffleMaster to obtain ShuffleDescriptor.Network-stack level Exception to notify remote receiver about a failed partition producer.
The partition file in the producer-merge mode.
The
ProducerMergedPartitionFileIndex is used by ProducerMergedPartitionFileWriter
and ProducerMergedPartitionFileReader, to maintain the offset of each buffer in the
physical file.Represents a series of buffers that are:
From the same subpartition
Logically (i.e. buffer index) consecutive
Physically (i.e. offset in the file) consecutive
The implementation of
PartitionFileReader with producer-merge mode.The implementation of
PartitionFileReader.ReadProgress mainly includes current
reading offset, end of read offset, etc.The implementation of
PartitionFileWriter with producer-side merge mode.Parameters for
JobManagerProfilingFileHandler.Parameters used by
JobManagerProfilingFileHandler.Contains information of a Profiling Instance.
Supported profiling mode in async-profiler.
Profiling Status.
Class containing a collection of
ProfilingInfo.RequestBody to trigger profiling instance.Create and keep profiling requests with rolling policy.
An implementation of
TimestampsAndWatermarks that does periodic watermark emission and
keeps track of watermarks on a per-split basis.ProxyMetricGroup<P extends org.apache.flink.metrics.MetricGroup>
Metric group which forwards all registration calls to its parent metric group.
Interface defining couple of essential methods for asynchronous and non blocking data polling.
Status for describing if we have reached the end of data.
A trigger that can turn any
Trigger into a purging Trigger.The variant of
PullingAsyncDataInput that is defined for handling both network input and
source input in a unified way via PushingAsyncDataInput.emitNext(DataOutput) instead of returning
Optional.empty() via PullingAsyncDataInput.pollNext().Basic data output interface used in emitting the next element from data input.
A push-based
Sorter.Internal operator handling queryable AppendingState instances.
Simple configuration object for the parameters for the server-side of queryable state.
Deprecated.
The Queryable State feature is deprecated since Flink 1.18, and will be removed in a
future Flink major version.
Utility class to initialize entities used in queryable state.
Internal operator handling queryable ValueState instances.
Container for scope related information as required by the MetricQueryService.
Container for the job manager operator scope.
Container for the job manager scope.
Container for the job scope.
Container for the operator scope.
Container for the task manager scope.
Container for the task scope.
Response type for temporary queue resources, i.e., resources that are asynchronously created.
Defines queue statuses.
A
RequestDoneCallback that adds the memory segments to a blocking queue.Build RangeBoundaries with input records.
Range and fraction of a memory component, which is a capped fraction of another component.
Json deserializer which deserializes raw json.
A record-oriented reader for immutable record types.
The basic API for every reader.
A
MutableObjectIterator that wraps a reader from an input channel and produces the
reader's records.An
OperatorEvent that registers a SourceReader to the SourceCoordinator.Minimal best-effort read-only sliced
Buffer implementation wrapping a NetworkBuffer's sub-region based on io.netty.buffer.SlicedByteBuf and
io.netty.buffer.ReadOnlyByteBuf.Partitioner that distributes the data equally by cycling through the output channels.
A RecordAttributes describes the attributes of records from the current RecordAttributes until
the next one is received.
The builder class for
RecordAttributes.RecordAttributesValve combine RecordAttributes from different input channels.
A context that preserves the necessary variables required by each operation, all operations for
one record will share the same element context.
RecordDeserializer<T extends org.apache.flink.core.io.IOReadableWritable>
Interface for turning sequences of memory segments into records.
Status of the deserialization result.
Utility class for creating record processor for
Input StreamOperator.RecordReader<T extends org.apache.flink.core.io.IOReadableWritable>
Record oriented reader for immutable types.
RecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>
An abstract record-oriented runtime result writer.
RecordWriterBuilder<T extends org.apache.flink.core.io.IOReadableWritable>
Utility class to encapsulate the logic of building a
RecordWriter instance.RecordWriterDelegate<T extends org.apache.flink.core.io.IOReadableWritable>
The record writer delegate provides the availability function for task processor, and it might
represent a single
RecordWriter or multiple RecordWriter instances in specific
implementations.Implementation of
Output that sends data using a RecordWriter.A
StreamTaskInput used during recovery of in-flight data.Implementation of
MetadataOutputStreamWrapper encapsulates the RecoverableFsDataOutputStream for FsCheckpointMetadataOutputStream.An input channel reads recovered state from previous unaligned checkpoint snapshots.
RecoveryMetadata contains the metadata used during a recovery process.A class that will recreate a new
OperatorCoordinator instance when reset to checkpoint.The provider for a private RecreateOnResetOperatorCoordinator.
SSL handler which automatically redirects Non-SSL requests to SSL address.
Internal
AllWindowFunction that is used for implementing a fold on a window configuration
that only allows AllWindowFunction and cannot directly execute a ReduceFunction.Internal
ProcessAllWindowFunction that is used for implementing a fold on a window
configuration that only allows AllWindowFunction and cannot directly execute a ReduceFunction.Internal
ProcessWindowFunction that is used for implementing a fold on a window
configuration that only allows AllWindowFunction and cannot directly execute a ReduceFunction.Internal
WindowFunction that is used for implementing a fold on a window configuration
that only allows WindowFunction and cannot directly execute a ReduceFunction.Combine operator for Reduce functions, standalone (not chained).
Reduce driver which is executed by a Task Manager.
A
Transformation that describes a reduce operation on a KeyedStream.A
TransformationTranslator for the ReduceTransformation.An adaptor that transforms
InternalReducingState
into InternalReducingState.An object that can be reference counted, the internal resource would be released when the
reference count reaches zero.
Releases blocking intermediate result partitions that are incident to a
SchedulingPipelinedRegion, as soon as the region's execution vertices are finished.Factory for
PartitionGroupReleaseStrategy.Compound meta information for a registered state in a keyed state backend.
Compound meta information for a registered state in a keyed state backend.
Compound meta information for a registered state in an operator state backend.
Meta information about a priority queue state in a backend.
RegisteredRpcConnection<F extends Serializable,G extends org.apache.flink.runtime.rpc.RpcGateway,S extends RegistrationResponse.Success,R extends RegistrationResponse.Rejection>
This utility class implements the basis of RPC connecting from one component to another
component, for example the RPC connection from TaskExecutor to ResourceManager.
Base class for all registered state in state backends.
Wrapper class that can be used to represent the wrapped
RegisteredStateMetaInfoBase
as key in a Map.RegistrationConnectionListener<T extends RegisteredRpcConnection<?,?,S,?>,S extends RegistrationResponse.Success,R extends RegistrationResponse.Rejection>
Classes which want to be notified about the registration result by the
RegisteredRpcConnection have to implement this interface.Base class for responses given to registration attempts from
RetryingRegistration.A registration failure.
A rejected (declined) registration.
Base class for a successful registration.
Exception which indicates that the
TaskExecutor could not register at the master in time.A regular non finished on restore
OperatorChain.A
StreamStateHandle for state that was written to a file stream.Handles the response of
PartitionProducerStateProvider.A set of tools used by batch and streaming remote environments when preparing their
configurations.
An input channel, which requests a remote partition queue.
An input channel reads recovered state from previous unaligned checkpoint snapshots and then
converts into
RemoteInputChannel finally.The
RemoteStorageScanner is introduced to notify asynchronously for file reading on
remote storage.A
StreamExecutionEnvironment for executing on a cluster.The data client is used to fetch data from remote tier.
The implementation of
TierFactory for remote tier.The implementation of
TierMasterAgent for the remote tier.The implementation of
TierProducerAgent for the remote tier.The shuffle descriptor implementation for remote tier.
Reports last emitted
Watermark from a subtask to the SourceCoordinator.Encapsulates all settings that are defined per reporter.
Encapsulates everything needed for the instantiation and configuration of a
MetricReporter.Marker interface for all requests of the REST API.
Callback to be executed on completion of an asynchronous I/O request.
This message requests an overview of the jobs on the JobManager, including running jobs and/or
finished jobs.
This message requests an overview of how many jobs are in which status.
This message requests an overview of how many jobs are in which status.
A
LinkedBlockingQueue that is extended with closing methods.Strategy to match slot requests to slots.
Result class representing matches.
An event to request splits, sent typically from the Source Reader to the Source Enumerator.
This message requests an overview of the status, such as how many TaskManagers are currently
connected, how many slots are available, how many are free, ...
A matcher for resource profiles to requirements.
Contains the fine-grain channel mappings that occur when a connected operator has been rescaled.
Partitioner that distributes the data equally by cycling through the output channels.
Rest handler to trigger and poll the rescaling of a running job.
Parallelism for the rescaling of jobs specified as a
MessageQueryParameter.Message headers for polling the status of an ongoing rescaling operation.
MessageParameters for polling the status of a rescaling operation.A
StreamTaskNetworkInput implementation that demultiplexes virtual channels.Message headers for triggering the rescaling of a job.
MessageParameters for triggering the rescaling of a job.Container for the set of reserved slots for
ExecutionVertexID.A function for reserving slots.
This interface marks a
Driver as resettable, meaning that will reset part of their
internal state but otherwise reuse existing data structures.The resettable iterator is a specialization of the iterator, allowing to reset the iterator and
re-retrieve elements.
The resettable iterator is a specialization of the iterator, allowing to reset the iterator and
re-retrieve elements.
Contains the results of the
ResourceAllocationStrategy.Strategy for allocating slots and task managers to fulfill the unfulfilled requirements.
Resource related actions which the
SlotManager can perform.Budget manager for
ResourceProfile.ResourceCleaner executes instances on the given JobID.ResourceCleanerFactory provides methods to create ResourceCleaner for local and
global cleanup.Counter for
ResourceProfiles.ResourceDeclaration for
ResourceAllocator.Callback interfaces for handling resource events from external resource managers.
Listener for resource events of
SlotManager.Class for Resource Ids identifying Flink's distributed components.
Json deserializer for
ResourceID.An interface to retrieve the ResourceID of an object.
Json serializer for
InstanceID.ResourceManager implementation.
Current address and fencing token of the leading ResourceManager.
A
ResourceManagerDriver is responsible for requesting and releasing resources from/to a
particular external resource manager.Base class for
ResourceManager exceptions.ResourceManager factory.The
ResourceManager's RPC gateway interface.Fencing token for the
ResourceManager.Metric group which is used by the
ResourceManager to register metrics.Utility for tracking and releasing partitions on the ResourceManager.
Factory for
ResourceManagerPartitionTracker.Default
ResourceManagerPartitionTracker implementation.This class contains information and services needed for creating a
ResourceManager, which do not change during the
lifetime of the process and can be reused between multiple resource manager instances in the
process.Container class for the
ResourceManager services.Configuration class for the
ResourceManagerRuntimeServices class.Service that maintains lifecycle of
ResourceManager.Default implementation of
ResourceManagerService.Common utils for ResourceManager.
Class containing information about the available cluster resources.
Describe the immutable resource profile of the slot, either when requiring or offering it.
Builder for the
ResourceProfile.Contains information of a
ResourceProfile.Contains the results of the
ResourceAllocationStrategy.Represents the number of required resources for a specific
ResourceProfile.Represents the total resource requirements for a job, and the information required to connect to
the corresponding job master.
Tracks for each job how many resource are required/acquired.
Resource version for specific state handle on the underlying storage.
Marker interface for all responses of the REST API.
Interface for all versions of the REST API.
A failover strategy that proposes to restart all vertices when a vertex fails.
The factory to instantiate
RestartAllFailoverStrategy.Strategy to decide whether to restart failed tasks and the delay to do the restarting.
The factory to instantiate
RestartBackoffTimeStrategy.A utility class to load
RestartBackoffTimeStrategy.Factory from the configuration.A failover strategy that proposes to restart involved regions when a vertex fails.
The factory to instantiate
RestartPipelinedRegionFailoverStrategy.This client is the counter-part to the
RestServerEndpoint.A configuration object for
RestClients.An exception that is thrown if the failure of a REST operation was detected on the client.
This class contains constants to be used by rest components.
WebMonitorEndpoint factory.Gateway for restful endpoints.
Configuration object containing values for the rest handler configuration.
An exception that is thrown if the failure of a REST operation was detected by a handler.
Enum to control logging behavior of RestHandlerExceptions.
Rest handler interface which all rest handler implementation have to implement.
This class contains utilities for mapping requests and responses to/from JSON.
Statistics for a restored checkpoint.
Interface for restore operation.
An abstract class for netty-based REST server endpoints.
Comparator for Rest URLs.
Comparator for Rest URLs.
A configuration object for
RestServerEndpoints.Rest service interface.
Result<VID extends VertexID,RID extends ResultID,V extends Vertex<VID,RID,V,R>,R extends Result<VID,RID,V,R>>
ID of a
Result.A result partition for data produced by a single task.
This checker helps to query result partition availability.
This class represents a snapshot of the result partition bytes metrics.
This counter will count the data size of a partition.
Deployment descriptor for a result partition.
Factory for
ResultPartition to use in NettyShuffleEnvironment.Runtime identifier of a produced
IntermediateResultPartition.The result partition manager keeps track of all currently produced/consumed partitions of a task
manager.
Collects metrics of a result partition.
Interface for creating result partitions.
State of a
SchedulingResultPartition.Type of a result partition.
A record-oriented runtime result writer API for producing results.
A single subpartition of a
ResultPartition instance.A combination of a
Buffer and the backlog length indicating how many non-event
buffers are available in the subpartition.A collection of subpartition indexes.
Identifies
ResultSubpartition in a given subtask.Handle to a ResultSubpartition state.A view to consume a
ResultSubpartition instance.Availability of the
ResultSubpartitionView and the backlog in the corresponding
ResultSubpartition.Handle to state that can be read back again via
RetrievableStateHandle.retrieveState().State storage helper which is used by
StateHandleStore to persist state before the state
handle is written to the underlying system.Wrapper around a
StreamStateHandle to make the referenced state object retrievable trough
a simple get call.RetryingRegistration<F extends Serializable,G extends org.apache.flink.runtime.rpc.RpcGateway,S extends RegistrationResponse.Success,R extends RegistrationResponse.Rejection>
This utility class implements the basis of registering one component at another component, for
example registering the TaskExecutor at the ResourceManager.
Configuration for the cluster components.
Implementation of an iterator that fetches a block of data into main memory and offers resettable
access to the data in that block.
An implementation of the
JoinTaskIterator that
uses a hybrid-hash-join internally to match the records with equal key.An implementation of the
JoinTaskIterator that
uses a hybrid-hash-join internally to match the records with equal key.A
DeserializationDelegate that reuses an existing instance upon deserialization.The KeyValueIterator returns a key and all values that belong to the key (share the same key).
This class wraps a
MutableObjectIterator into a regular Iterator.Deprecated.
This class is based on the
SourceFunction API, which is due to be removed.Deprecated.
This interface will be removed in future versions.
Deprecated.
This class is based on the
SourceFunction API, which is due to be removed.The right outer join driver implements the logic of an outer join operator at runtime.
RootExceptionHistoryEntry extending ExceptionHistoryEntry by providing a list of
ExceptionHistoryEntry instances to store concurrently caught failures.RoundRobinChannelSelector<T extends org.apache.flink.core.io.IOReadableWritable>
This is the default implementation of the
ChannelSelector interface.Current default implementation of
OperatorStateRepartitioner that redistributes state in
round robin fashion.A
SubpartitionSelector that selects all subpartitions in round-robin order.Class for handling
HttpRequest with associated RouteResult.This is adopted and simplified code from tv.cntt:netty-router library.
This is adopted and simplified code from tv.cntt:netty-router library.
Inbound handler that converts HttpRequest to Routed and passes Routed to the matched handler.
RpcGatewayRetriever<F extends Serializable,T extends org.apache.flink.runtime.rpc.FencedRpcGateway<F>>
LeaderGatewayRetriever implementation using the RpcService.KvStateRegistryListener implementation for the new RPC service.MetricQueryServiceRetriever implementation for rpc based MetricQueryService.Enum which defines whether the mini cluster components use a shared RpcService or whether every
component gets its own dedicated RpcService started.
Implementation of the
TaskManagerGateway for Flink's RPC system.An OperatorEventSender that calls the RPC gateway
JobMasterOperatorEventGateway to send
the messages to the coordinator.Utils related to
Runnable.In implementation of the
Environment.Subclasses of this event are recognized as events exchanged by the core runtime.
This class links
RequestBodys to ResponseBodys types and contains meta-data
required for their http headers in runtime module.An enum for all versions of the REST API.
Message headers for a web handler request that belongs to runtime module.
Handlers to trigger the disposal of a savepoint.
Request body for a savepoint disposal call.
AsynchronousOperationTriggerMessageHeaders implementation for the SavepointDisposalHandlers.SavepointDisposalStatusHandler.HTTP handlers for asynchronous triggering of savepoints.
HTTP handler to query for the status of the savepoint.
Represents information about a finished savepoint.
A
KeyedStateHandle that points to a savepoint taken in the unified format.Savepoint resources for a
KeyedStateBackend.A result from restoring a single
KeyedStateHandle.Savepoint restore settings.
A
SnapshotStrategy that produces unified savepoints.These headers define the protocol for triggering a savepoint.
The parameters for triggering a savepoint.
These headers define the protocol for triggering a savepoint.
The parameters for triggering a savepoint.
RequestBody to trigger savepoints.The type of checkpoint to perform.
What's the intended action after the checkpoint (relevant for stopping with savepoint).
Interface to interact with optional Scala field accessors.
Base class which can be used to implement
SchedulerNG.Interface for scheduling Flink jobs.
Factory for
SchedulerNG.Component which is used by
SchedulingStrategy to commit scheduling decisions.Utils class for Flink's scheduler implementations.
Scheduling representation of
ExecutionVertex.Pipelined region on execution level, i.e.,
ExecutionGraph level.Utils for computing
SchedulingPipelinedRegions.Representation of
IntermediateResultPartition.Component which encapsulates the scheduling logic.
Factory interface for
SchedulingStrategy.Topology of
SchedulingExecutionVertex.This listener will be notified whenever the scheduling topology is updated.
This class represents the format after which the "scope" (or namespace) of the various component
metric groups is built.
A container for component scope formats.
The global security configuration.
A security context with may be required to run a Callable.
A factory for a
SecurityContext.Indicates a problem with installing or uninstalling a security module or context.
The Service provider discovery for searching suitable security factory.
An installable security module.
Indicates a problem with installing or uninstalling a security module.
A factory for a
SecurityModule.Security Environment that holds the security context and modules installed.
A
DataInputView that is backed by a BlockChannelReader, making it effectively a data input stream.FileStateHandle for state that was written to a file segment.The partition file with segment file mode.
The implementation of
PartitionFileReader with segment file mode.The implementation of
PartitionFileWriter with segment file mode.A logical timestamp to draw a boundary between the materialized and non-materialized changes.
Generic
SequenceNumber.Reads channel state saved during checkpoint/savepoint.
SequentialChannelStateReader implementation.The serialization delegate exposes an arbitrary element as a
IOReadableWritable for
serialization, with the help of a type serializer.Serialized checkpoint exception which wraps the checkpoint failure reason and its serialized
throwable.
Responsible for serialization of currentKey, currentGroup and namespace.
A variant of the
JobExecutionResult that holds its
accumulator data in serialized form.JSON deserializer for
SerializedThrowable.JSON serializer for
SerializedThrowable.AbstractPagedOutputView used by the BlockingBackChannel for transmitting
superstep results.JSON deserializer for
SerializedValue.JSON serializer for
SerializedValue.Base interface for managers of services that are explicitly connected to / disconnected from.
Base class for session cluster entry points.
DispatcherFactory which creates a StandaloneDispatcher.Process which encapsulates the job recovery logic and life cycle management of a
Dispatcher.Factory for the
SessionDispatcherLeaderProcess.Factory for the
SessionDispatcherLeaderProcessFactory.RestEndpointFactory which creates a DispatcherRestEndpoint.A map that keeps track of acquired shared resources and handles their allocation disposal.
A resource handle with size.
This registry manages state that is shared across (incremental) checkpoints, and is responsible
for deleting shared state that is no longer used in any valid checkpoint.
Simple factory to produce
SharedStateRegistry objects.SharedStateRegistry implementation.An object with empty discardState for registering.
This class represents a key that uniquely identifies (on a logical level) state handles for
registration in the
SharedStateRegistry.Implementation of
PhysicalSlotRequestBulk for SlotSharingExecutionSlotAllocator.Enumeration defining the different shipping types of the output, such as local forward,
re-partitioning by hash, or re-partitioning by range.
Interface for shuffle deployment descriptor of result partition resource.
The retriever for shuffle descriptor.
Interface for the implementation of shuffle service local environment.
Local context used to create
ShuffleEnvironment.Context of shuffle input/output owner used to create partitions or gates belonging to the owner.
Intermediate result partition registry to use in
JobMaster.Shuffle context used to create
ShuffleMaster.The default implementation of
ShuffleMasterContext.This class represents a snapshot of shuffle master, which can be used to restore the internal
state of the shuffle master.
Snapshot context used to create
ShuffleMasterSnapshot.Utility class for handling cluster-level snapshots for the ShuffleMaster.
Interface provides access to the shuffle metrics which includes the meta information of
partition(partition bytes, etc).
Partitioner that distributes the data equally by selecting one output channel randomly.
ShuffleServiceFactory<SD extends ShuffleDescriptor,P extends ResultPartitionWriter,G extends IndexedInputGate>
Interface for shuffle service factory implementations.
Utility to load the pluggable
ShuffleServiceFactory implementations.Options to configure shuffle service.
Common utility methods for shuffle service.
REST handler which allows to shut down the cluster.
Message headers for
ShutdownHandler.A
SideOutputDataStream represents a DataStream that contains elements that are
emitted from upstream into a side output with some tag.This transformation represents a selection of a side output of an upstream operation with a given
OutputTag.A
TransformationTranslator for the SideOutputTransformation.This signal handler / signal logger is based on Apache Hadoop's
org.apache.hadoop.util.SignalLogger.
A simple iterator provider that returns a supplied iterator and does nothing when closed.
The list with the full segments contains at any point all completely full segments, plus the
segment that is currently filled.
A simple implementation of
ExecutionSlotAllocator.Factory to instantiate a
SimpleExecutionSlotAllocator.A simple pojo based implementation of
JobVertex.InitializeOnMasterContext.Input format source operator factory which just wrap existed
StreamSource.Simple factory which just wrap existed
StreamOperator.A simple operator factory which create an operator containing an
OutputFormat.Simple implementation of the
RequestSlotMatchingStrategy that matches the pending
requests in order as long as the resource profile can be fulfilled.Implementation of
TimerService that uses a InternalTimerService.A base class for all
TransformationTranslators who translate
Transformations that have a single operator in their runtime
implementation.Udf stream operator factory which just wrap existed
AbstractUdfStreamOperator.A
ListState that uses a SimpleVersionedSerializer instead of a TypeSerializer.SingleCheckpointBarrierHandler is used for triggering checkpoint while reading the first
barrier and keeping track of the number of received barriers and consumed barriers.An
Iterator that contains only a single element.An input gate consumes one or more partitions of a single produced intermediate result.
Factory for
SingleInputGate to use in NettyShuffleEnvironment.Statistics of input channels.
Implementation of the
LogicalSlot.SingleOutputStreamOperator represents a user defined transformation applied on a DataStream with one predefined output type.SingleRecordWriter<T extends org.apache.flink.core.io.IOReadableWritable>
The specific delegate implementation for the single output case.
Deprecated.
This interface will be removed in future versions.
Context that
SinkFunctions can use for getting additional data about an
input record.A
Transformation for Sink.A
TransformationTranslator for the SinkTransformation.This class offers the possibility to deserialize committables that have been written with older
Flink releases (i.e. 1.13, 1.14).
A
StreamOperatorFactory for SinkWriterOperator.A
WindowAssigner that windows elements into sliding windows based on the timestamp of the
elements.A
WindowAssigner that windows elements into sliding windows based on the current system
time of the machine the operation is running on.Interface to trigger slot actions from within the
TaskSlotTable.Exception indicating that the slot allocation on the task manager failed.
Model to save local slot allocation info.
Service for persisting
SlotAllocationSnapshot.Component for calculating the slot requirements and mapping of vertices to slots.
Interface for assigning slots to slot sharing groups.
Interface for the context of a
LogicalSlot.Unique identifier for a slot on a TaskManager.
Interface that provides basic information in the context of a slot.
Class containing information for a slot of
TaskManagerSlotInformation.The slot manager is responsible for maintaining a view on all registered task manager slots,
their allocation and all pending slot requests.
The result of task manager registration.
Configuration for the
SlotManager.Base class for exceptions thrown by the
SlotManager.Metric group which is used by the
SlotManager to register metrics.Utilities for
SlotManager implementations.Exception indicating that the given
TaskSlot was not in state active.Exception indicating that a
TaskSlot could not be found.Exception which signals that a slot is already occupied by the given
AllocationID.Describe the slot offering to job manager provided by task manager.
Interface for components that hold slots and to which slots get released / recycled.
The Interface of a slot pool that manages slots.
Service used by the
JobMaster to manage a slot pool.Factory interface for
SlotPoolService.Factory for
SlotPoolService and SchedulerNG.A slot profile describes the profile of a slot into which a task wants to be scheduled.
A report about the current status of all slots of the TaskExecutor, describing which slots are
available and allocated, and what jobs (JobManagers) the allocated slots have been allocated to.
This ID identifies the request for a slot from the Execution to the
SlotPool or PhysicalSlotProvider.Interface for slot selection strategies.
Utility class for selecting
SlotSelectionStrategy.Factory for
SlotSharingExecutionSlotAllocator.A slot sharing units defines which different task (from different job vertices) can be deployed
together within a slot.
Jackson deserializer for
SlotSharingGroupId.Jackson serializer for
SlotSharingGroupId.SlotAllocator implementation that supports slot sharing.This describes the slot current status which located in TaskManager.
Syncer for slot status.
Component responsible for detecting slow tasks.
Component responsible for listening on slow tasks.
This implementation decorates the stream with snappy compression.
Interface for objects that can snapshot its state (state backends currently).
This class represents a directory that is the target for a state snapshot.
Determines how the write-part of a snapshot should be executed.
A set of resources produced in the synchronous part of a snapshot required to finish the
snapshot.
This class contains the combined results from the snapshot of a state backend:
A state object representing the state that will be reported to the Job Manager to
acknowledge the checkpoint.
Interface for different snapshot approaches in state backends.
A supplier for a
SnapshotResult with an access to a CloseableRegistry for io
tasks that need to be closed when cancelling the async part of the checkpoint.A class to execute a
SnapshotStrategy.The type of checkpoint to perform.
Defines what files can be shared across snapshots.
Deprecated.
This interface will be removed in future versions.
Deprecated.
This class is based on the
SourceFunction API, which is due to be removed.Used to hand over the hash-join from the iteration head to the solution-set match.
A
Collector to update the solution set of a workset iteration.A
Collector to update the solution set of a workset iteration.Resettable barrier to synchronize the
IterationHeadTask and the IterationTailTask
in case of iterations that contain a separate solution set tail.A
Collector to update the solution set of a workset iteration.When getting buffers, The
SortBasedDataBuffer should recycle the read target buffer with
the given BufferRecycler.A
DataBuffer implementation which sorts all appended records only by subpartition index.The sort-based implementation of the
BufferAccumulator.The SortMerger interface representing the public interface to all specific Sort-Merge
implementations.
A
StreamTaskInput which sorts in the incoming records from a chained input.SortMergeResultPartition appends records and events to DataBuffer and after the
DataBuffer is full, all data in the DataBuffer will be copied and spilled to a
PartitionedFile in subpartition index order sequentially.The
SortPartitionOperator sorts records of a partition on non-keyed data stream.The default implementation of the
OperatorCoordinator for the Source.The watermark element for
HeapPriorityQueue.SourceCoordinatorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>
A context class for the
OperatorCoordinator.SourceCoordinatorProvider<SplitT extends org.apache.flink.api.connector.source.SourceSplit>
The provider of
SourceCoordinator.A thread factory class that provides some helper methods.
A serialization util class for the
SourceCoordinator.A wrapper operator event that contains a custom defined operator event.
Deprecated.
This interface will be removed in future versions.
Interface that source functions use to emit elements, and possibly watermarks.
Lineage vertex for source which has
Boundedness to indicate whether the data for the
source is bounded.Base source operator only used for integrating the source reader which is proposed by FLIP-27.
The Factory class for
SourceOperator.A subclass of
StreamTask for executing the SourceOperator.Implementation of
PushingAsyncDataInput.DataOutput that wraps a specific Output.Implementation of the SourceOutput.
Deprecated.
This class is based on the
SourceFunction API, which is due to be removed.A
PhysicalTransformation for Source.SourceTransformationTranslator<OUT,SplitT extends org.apache.flink.api.connector.source.SourceSplit,EnumChkT>
A
TransformationTranslator for the SourceTransformation.This Transformation is a phantom transformation which is used to expose a default parallelism to
downstream.
Interface for managing speculative execution of tasks and handling slow task detection.
The ExecutionJobVertex which supports speculative execution.
Factory to create
SpeculativeExecutionJobVertex.The ExecutionVertex which supports speculative execution.
SpillingAdaptiveSpanningRecordDeserializer<T extends org.apache.flink.core.io.IOReadableWritable>
An output view that buffers written data in memory pages and spills them when they are full.
Implementation of a resettable iterator.
Implementation of a resettable iterator.
SplitAssignmentTracker<SplitT extends org.apache.flink.api.connector.source.SourceSplit>
A class that is responsible for tracking the past split assignments made by
SplitEnumerator.Util to analyze inputs & outputs of
ExecutionJobVertex and calculate network memory
requirement for slot sharing group (SSG).Creates and configures
SslHandler instances.Common utilities to manage SSL transport settings.
An interface for different stages of the sorting process.
A marker interface for sending messages to different stages.
A dispatcher for inter-stage communication.
CheckpointIDCounter instances for JobManagers running in HighAvailabilityMode.NONE.CheckpointCoordinator components in HighAvailabilityMode.NONE.Non-HA implementation for
ClientHighAvailabilityServices.CompletedCheckpointStore for JobManagers running in HighAvailabilityMode.NONE.ExecutionPlan instances for JobManagers running in HighAvailabilityMode.NONE.An implementation of the
HighAvailabilityServices for the non-high-availability case.StandaloneLeaderElection implements LeaderElection for non-HA cases.Standalone implementation of the
LeaderRetrievalService.A standalone implementation of the resource manager.
ResourceManagerFactory which creates a StandaloneResourceManager.Entry point for the standalone session cluster.
This utility class provides building blocks for custom topologies.
An base implementation of state adaptor from v1 to v2.
This class encapsulates the operation of assigning restored state when restoring from a
checkpoint.
A State Backend defines how the state of a streaming application is stored locally within
the cluster.
Parameters passed to
StateBackend.createKeyedStateBackend(KeyedStateBackendParameters).Parameters passed to
StateBackend.createOperatorStateBackend(OperatorStateBackendParameters).A mutable builder to build a state backend instance.
A factory to create a specific state backend.
This class contains utility methods to load state backends from configurations.
Change of state of a keyed operator.
Allows to read state changelog referenced by the provided
ChangelogStateHandle.A reader for
ChangelogStateHandleStreamImpl that iterates over its underlying stream handles and offsets.Reads a stream of state changes starting from a specified offset.
A storage for changelog.
A factory for
StateChangelogStorage.A thin wrapper around
PluginManager to load StateChangelogStorage.A storage view for changelog.
Allows to write data to the log.
Utilities for transforming
StateDescriptor to StateDescriptor.Interface of entries in a state backend.
Executor for executing batch
StateRequests.StateFactory<T extends org.apache.flink.runtime.scheduler.adaptive.State>
Factory for creating adaptive scheduler state instances.
Deprecated.
This class is based on the
SourceFunction API, which is due to be removed.An internal factory for
InternalStateFuture that build future with necessary context
switch and wired with mailbox executor.Unique ID that allows for logical comparison between state handles.
Class which stores state via the provided
RetrievableStateStorageHelper and writes the
returned state handle to distributed coordination system(e.g.The key already exists in ConfigMap or the Zookeeper node already exists.
The key does not exist in ConfigMap or the Zookeeper node does not exists.
StateHandleStoreUtils collects utility methods that might be usefule for any StateHandleStore implementation.This interface provides a context in which operators can initialize by registering to managed
state (i.e. state that is managed by state backends) or iterating over streams of state
partitions written as raw state in a previous snapshot.
Default implementation of
StateInitializationContext.A
SlotAssigner that assigns slots based on the number of local key groups.Component to retrieve the state location of an execution vertex.
Base class for state maps.
Base class for snapshots of a
StateMap.Functional interface to read
StateMetaInfoSnapshot.Generalized snapshot for meta information about one state in a state backend (e.g.
Enum that defines the different types of state that live in Flink backends.
Predefined keys for the most common options in the meta info.
Predefined keys for the most common serializer types in the meta info.
Static factory that gives out the write and readers for different versions of
StateMetaInfoSnapshot.Functional interface to write
StateMetaInfoSnapshot.Base of all handles that represent checkpointed state in some form.
Enum for state locations.
Collector for size and location stats from a state object via
StateObject.collectSizeStats(StateObjectSizeStatsCollector).This class represents a generic collection for
StateObjects.This class provides access to input streams that contain data of one state partition of a
partitionable state.
A request encapsulates the necessary data to perform a state request.
A buffer to hold state requests to execute state requests in batch, which can only be manipulated
within task thread.
A container which is used to hold
StateRequests.The handler which can process
StateRequest.The type of processing request for
State from **users' perspective**.A
StateSerializerProvider wraps logic on how to obtain serializers for registered state,
either with the previous schema of state in checkpoints or the current schema of state.Managed Keyed State size estimates used to make scheduling decisions.
General interface for state snapshots that should be written partitioned by key-groups.
Interface for writing a snapshot that is partitioned into key-groups.
This interface provides a context in which operators that use managed (i.e. state that is managed
by state backends) or raw (i.e. the operator can write it's state streams) state can perform a
snapshot.
This class is a default implementation for StateSnapshotContext.
Interface for state de-serialization into
StateTables by key-group.Interface to deal with state snapshot and restore of state.
Transformer of state values which are included or skipped in the snapshot.
Collection state specific transformer which says how to transform entries of the collection.
This factory creates state transformers depending on the form of values to transform.
Collection of common state snapshot transformers and their factories.
General implementation of list state transformer.
General implementation of map state transformer.
Base class for state tables.
This class provides a static factory method to create different implementations of
StateSnapshotKeyGroupReader depending on the provided serialization format version.Utility to define metrics that capture the time that some component spends in a state.
Interface for a binary function that is used for push-down of state transformation into state
backends.
The
StateTransitionManager decides on whether AdaptiveScheduler state transition
should happen or not.The interface that can be used by the
StateTransitionManager to communicate with the
underlying system.An interface covering all possible
State transitions.Interface covering transition to the
Canceling state.Interface covering transition to the
CreatingExecutionGraph state.Interface covering transition to the
Executing state.Interface covering transition to the
Failing state.Interface covering transition to the
Finished state.Interface covering transition to the
Restarting state.Interface covering transition to the
StopWithSavepoint state.Interface covering transition to the
WaitingForResources state.Helpers for
StateObject related code.A partitioned state that wraps a declared namespace and hide the namespace switching from user.
Static implementation of
ExternalResourceInfoProvider which return fixed collection of
ExternalResourceInfo.Simple file server handler that serves requests to web frontend's static files, such as HTML,
CSS, or JS files.
Represents one or more statistics samples.
Aggregated values of some measurement such as min/max/average state size.
Transfer object for
statistics summary.Immutable snapshot of
StatsSummary.Utilities to calculate status duration.
A
StatusWatermarkValve embodies the logic of how Watermark and WatermarkStatus are propagated to downstream outputs, given a set of one or multiple
subpartitions that continuously receive them.An
SubpartitionStatus keeps track of a subpartition's last watermark, stream status,
and whether or not the subpartition's current watermark is aligned with the overall watermark
output from the valve.Tells if the job is stopping because of consuming all data.
DispatcherLeaderProcess implementation which is stopped.RequestBody for stopping a job with a savepoint.Exception thrown when a savepoint has been created successfully when stopping with savepoint, but
the job has not finished.
StopWithSavepointTerminationHandler handles the steps necessary for the
stop-with-savepoint operation to finish.StopWithSavepointTerminationHandlerImpl implements StopWithSavepointTerminationHandler.StopWithSavepointTerminationManager fulfills the contract given by StopWithSavepointTerminationHandler to run the stop-with-savepoint steps in a specific order.These headers define the protocol for triggering a "stop" action.
Implementations of this interface decorate streams with a compression scheme.
Internal configuration for a
StreamOperator.Interface representing chained inputs.
Requirements of the different inputs of an operator.
A representation of a Network
StreamConfig.InputConfig.A serialized representation of an input.
An edge in the streaming topology.
Helper class carries the data required to updates a stream edge.
An element in a data stream.
Configuration snapshot specific to the
StreamElementSerializer.The data exchange mode between operators during
StreamGraph generation.The StreamExecutionEnvironment is the context in which a streaming program is executed.
Factory class for stream execution environments.
A
StreamOperator for executing FilterFunctions.A
StreamOperator for executing FlatMapFunctions.Class representing the streaming topology.
Defines a context for optimizing and working with a read-only view of a StreamGraph.
Interface for observers that monitor the status of a StreamGraph.
A generator that generates a
StreamGraph from a graph of Transformations.Interface for different implementations of generating hashes over a stream graph.
StreamGraphHasher from Flink 1.2.
Defines an optimization strategy for StreamGraph.
The
StreamGraphOptimizer class is responsible for optimizing a StreamGraph based on
runtime information.StreamGraphHasher that works with user provided hashes.
Utility class that contains helper methods to generating
StreamGraph.Utility class that contains helper methods to work with Flink Streaming
Functions.The StreamingJobGraphGenerator converts a
StreamGraph into a JobGraph.Implementation of the
RuntimeContext, for streaming
operators.Interface for processing records by
StreamTask.A special
StreamTask that is used for executing feedback edges.A special
StreamTask that is used for executing feedback edges.A
StreamOperator for executing MapFunctions.Input processor for
MultipleInputStreamOperator.A factory for
StreamMultipleInputProcessor.Class representing the operators in the streaming programs, with all their properties.
Stream node level implement for
ForwardGroup.Helper class carries the data required to updates a stream edge.
Input reader for
OneInputStreamTask.Basic interface for stream operators.
A factory to create
StreamOperator.A utility to instantiate new operators with a given factory.
Helper class to construct
AbstractStreamOperatorV2.This interface represents a context from which a stream operator can initialize everything
connected to state such as e.g. backends, raw state, and timer service manager.
Class encapsulating various state backend handling logic for
StreamOperator
implementations.Custom state handling hooks to be invoked by
StreamOperatorStateHandler.This class handles the finish, endInput and other related logic of a
StreamOperator.A special
ChannelSelector for use in streaming programs.A
StreamOperator for executing projections on streams.The result of
DataStream.project(int...).One value in a data stream.
A
StreamOperator for executing SinkFunctions.Deprecated.
This class is based on the
SourceFunction API, which is due to be removed.Deprecated.
This class is based on the
SourceFunction API, which is due to be removed.A
StateObject that represents state that was written to a stream.Base class for all streaming tasks.
Check whether records can be emitted in batch.
Deprecated.
this class should only be used in
SourceStreamTask which exposes the
checkpoint lock as part of Public API.Context on the
StreamTask for figuring out whether it has been cancelled.An exception that is thrown by the stream vertices when encountering an illegal condition.
A subclass of
StreamTaskSourceInput for ExternallyInducedSourceReader.A special source input implementation that immediately emit END_OF_INPUT.
Basic interface for inputs of stream operators.
Implementation of
StreamTaskInput that wraps an input from network taken from CheckpointedInputGate.Factory for
StreamTaskNetworkInput and RescalingStreamTaskNetworkInput.Implementation of
StreamTaskInput that reads data from the SourceOperator and
returns the DataInputStatus to indicate whether the source state is available,
unavailable or finished.This is the interface through which stream task expose a
StreamOperatorStateContext to
their operators.This class is the main implementation of a
StreamTaskStateInitializer.StreamTaskStateInitializerImpl.KeyedStateBackendCreator<K,R extends org.apache.flink.util.Disposable & Closeable>
Functional interface to create the keyed state backend.
A factory to create
StreamMultipleInputProcessor for two input case.Container class that transports the result of an accumulator as set of strings.
ResourceVersion implementation with String value.Utility for computing strongly connected components.
SubpartitionSelector helps to choose from multiple available subpartitions when their
output buffers should union into one stream.Helper class that describes the statistics of all subpartitions with a specific index within the
given partition range.
The type Subtask attempt message parameters.
Contains the attempt number per subtask.
Path parameter identifying subtask attempt.
Coordinates checkpointing-related work for a subtask (i.e.
Checkpoint statistics for a subtask.
Checkpoint statistics for a completed subtask checkpoint.
Alignment statistics of the checkpoint.
Duration of the checkpoint.
Checkpoint statistics for a pending subtask checkpoint.
An event that is used to (de)multiplex old channels over the same new channel.
Request handler providing details about a single task execution attempt.
Message headers for the
SubtaskCurrentAttemptDetailsHandler.Request handler for the subtask execution attempt accumulators.
Message headers for the
SubtaskExecutionAttemptAccumulatorsHandler.Response type of the
SubtaskExecutionAttemptAccumulatorsHandler.Handler of specific sub task execution attempt.
Message headers for the
SubtaskExecutionAttemptDetailsHandler.The sub task execution attempt response.
Restore operation that restores file-merging information belonging to one subtask for
FileMergingSnapshotManager.Path parameter specifying the index of a subtask.
Query parameter specifying the index of a subtask.
A collection of simple metrics, around the triggering of a checkpoint.
A builder for
SubTaskInitializationMetrics.Message parameters for subtask REST handlers.
Handler that returns subtask metrics.
Request handler for the subtasks all accumulators.
Message headers for the
SubtasksAllAccumulatorsHandler.Response type of the
SubtasksAllAccumulatorsHandler.Detailed information about subtask accumulators.
MessageQueryParameter for selecting subtasks when aggregating metrics.Deprecated.
Internal class for savepoint backwards compatibility.
The
SubtaskStateMapper narrows down the subtasks that need to be read during rescaling to
recover from a particular subtask when in-flight data has been stored in the checkpoint.Statistics for a single subtask that is part of a checkpoint.
Request handler for the subtasks times info.
Message headers for the
SubtasksTimesHandler.Response type of the
SubtasksTimesHandler.Nested class to encapsulate the sub task times info.
An
AggregationFunction that sums up fields.Internal function for summing up contents of fields.
A resettable one-shot latch.
Latch used to wait for the previous superstep to complete.
Allows expert users to implement a custom topology after
Committer.Allows expert users to implement a custom topology after
SinkWriter and before Committer.Allows expert users to implement a custom topology before
SinkWriter.Exception thrown in order to suppress job restarts.
Listener for
WorkerDoneEvent which also aggregates all aggregators from iteration tasks
and signals the end of the superstep.A synchronous
BufferFileReader implementation.The chained variant of the combine driver which is also implemented in GroupReduceCombineDriver.
A base class for synchronous readers and writers.
Component to retrieve the preferred locations of an execution vertex.
Utility class for redirect the
System.out and System.err.A
TimerService which assigns as current processing time the result of calling System.currentTimeMillis() and registers timers using a ScheduledThreadPoolExecutor.Daemon thread probing system resources.
Utility class to initialize system resource metrics.
The Task represents one execution of a parallel subtask on a TaskManager.
Various operation of notify checkpoint.
Actions which can be performed on a
Task.Response to the task back pressure request rpc call.
Message parameters for subtask related checkpoint message.
REST handler which serves checkpoint statistics for subtasks.
Checkpoint statistics for a single task.
Headers for the
TaskCheckpointStatisticDetailsHandler.Task checkpoint statistics which also includes information about the sub task checkpoint
statistics.
Alignment information for a specific checkpoint at a given task.
Duration of a checkpoint split up into its synchronous and asynchronous part.
Summary of the checkpoint statistics for a given task.
Configuration class which stores all relevant parameters required to set up the Pact tasks.
The task context gives a driver (e.g.,
MapDriver, or JoinDriver) access to the
runtime components and configuration that they can use to fulfil their task.A task deployment descriptor contains all the information necessary to deploy a task on a task
manager.
Wrapper class for serialized values which may be offloaded to the
BlobServer or not.A serialized value that is not offloaded to the
BlobServer.Reference to a serialized value that was offloaded to the
BlobServer.Defines whether the partition's location must be known at deployment time or can be unknown
and, therefore, updated later.
This class represents the shuffle descriptor with it index in
ConsumedPartitionGroup.A set of shuffle descriptors that will be serialized together.
Subclasses of this event are recognized as custom events that are not part of the core flink
runtime.
The task event dispatcher dispatches events flowing backwards from a consuming task to the task
producing the consumed result.
The event handler manages
EventListener instances and allows to publish events to them.The task event publisher is used for publishing the event to the registered
EventListener
instances.Exception indicating a task related problem on the
TaskExecutor.This class represents an update about a task's execution state.
Wraps
TaskExecutionState, along with actions to take if it is FAILED state.TaskExecutor implementation.
BlobService that gives access to a JobPermanentBlobService and allows to set the
target blob server address.This class holds the all
ChannelStateWriteRequestExecutorFactory objects for a task
executor (manager).Interface for releasing cluster partitions on a task executor.
This class is responsible for grouping the TaskExecutorGateway and the InstanceID of a registered
task executor.
There is one
FileMergingSnapshotManager for each job per task manager.Flink internal memory components of Task Executor.
FlinkMemoryUtils for Task Executor.TaskExecutor RPC gateway interface.A class that decorates/forwards calls to a
TaskExecutorGateway.Payload for heartbeats sent from the TaskExecutor to the ResourceManager.
This class holds the all
TaskLocalStateStoreImpl objects for a task executor (manager).TaskExecutorConfiguration collects the configuration of a TaskExecutor instance.
The gateway through which the
OperatorCoordinator can send an event to an Operator on the
Task Manager side.Encapsulates meta-information the TaskExecutor requires to be kept for each partition.
Utility for tracking partitions.
Utility for tracking partitions and issuing release calls to task executors and shuffle masters.
Describe the specifics of different resource dimensions of the TaskExecutor process.
Builder for
TaskExecutorProcessSpec.Utility class for TaskExecutor memory configurations.
Information provided by the TaskExecutor when it registers to the ResourceManager.
Rejection response from the
ResourceManager for the TaskExecutor.Base class for responses from the ResourceManager to a registration attempt by a TaskExecutor.
Specification of resources to use in running
TaskExecutor.Utility class for
TaskExecutorResourceSpec of running TaskExecutor.This class holds the all
StateChangelogStorage objects for a task executor (manager).RPC gateway for requesting
ThreadInfoSample.Payload for heartbeats sent from the TaskExecutor to the JobManager.
The connection between a TaskExecutor and the ResourceManager.
Simple adapter for
TaskExecutor to adapt to TaskManagerRunner.TaskExecutorService.Container class for operator/task specific information which are stored at the
ExecutionJobVertex.Describes inputs and outputs information of a task.
An invokable part of the task.
Metric group that contains shareable pre-defined IO-related metrics.
Supplier for sizes.
A helper for KvState registrations of a single task.
Classes that implement this interface serve as a task-manager-level local storage for local
checkpointed state.
Main implementation of a
TaskLocalStateStore.A task mailbox provides read and write access to a mailbox and has a lifecycle of open ->
(quiesced) -> closed.
Exception thrown when
TaskMailbox is closed.This enum represents the states of the mailbox lifecycle.
Implementation of
TaskMailbox in a BlockingQueue fashion and
tailored towards our use case with multiple writers and single reader.Interface for the communication of the
Task with the TaskExecutor.Configuration object for
TaskExecutor.Rest handler which serves the custom file of the
TaskExecutor.Headers for the
TaskManagerCustomLogHandler.Handler which serves detailed TaskManager information.
Headers for the
TaskManagerDetailsHandler which serves the TaskManager details.Message containing base information about a
TaskExecutor and more detailed metrics.Base exception thrown by the
TaskExecutor.Parameters for
TaskManagerCustomLogHandler.Task manager gateway interface to communicate with the task manager.
TaskManager id path parameter used by TaskManager related handlers.
Information of a TaskManager needed in
SlotManager.Base class containing information for a
TaskExecutor.Contains the base information about a
TaskExecutor and its allocated slots.Special
MetricGroup representing everything belonging to a
specific job, running on the TaskManager.The scope format for the
JobMetricGroup.This class encapsulates the connection information of a TaskManager.
This Supplier class could retrieve the FQDN host name of the given InetAddress on demand,
extract the pure host name and cache the results for later use.
This Supplier class returns the IP address of the given InetAddress directly, therefore no
reverse DNS lookup is required.
The DNS resolution mode for TaskManager's IP address.
Rest handler which serves the log files from
TaskExecutor.Headers for the
TaskManagerLogFileHandler.Handler which serves detailed TaskManager log list information.
Headers for the
TaskManagerLogListHandler.Headers for the log url retriever of TaskManager.
Message parameter for a TaskManager specific handler.
Special
MetricGroup representing a TaskManager.Handler that returns TaskManager metrics.
Contains information about the TaskManager metrics.
Information about the garbage collector metrics.
Rest handler which serves the profiling result file of the
TaskExecutor.Headers for the
TaskManagerProfilingFileHandler.Parameters for
TaskManagerProfilingFileHandler.Rest handler which serves the profiling service from a
TaskExecutor.Headers for the
TaskManagerProfilingHandler.Handler which serves detailed TaskManager profiling list information.
Headers for the
TaskManagerProfilingListHandler.DTO for TaskManager registration information.
This class is the executable entry point for the task manager in yarn or standalone mode.
Factory for
TaskExecutor.Interface to access
TaskExecutor information.The scope format for the
TaskManagerMetricGroup.Configuration for the task manager services such as the memory manager, the io manager and the
metric registry.
MessageQueryParameter for selecting task managers when aggregating metrics.Returns an overview over all registered TaskManagers of the cluster.
Message headers for the
TaskManagersHandler.Class containing a collection of
TaskManagerInfo.Basic information about a TaskManager slot.
Rest handler which serves the stdout file of the
TaskExecutor.Headers for the
TaskManagerStdoutFileHandler.Rest handler which serves the thread dump info from a
TaskExecutor.Headers for the
TaskManagerThreadDumpHandler.Special
MetricGroup representing a Flink runtime Task.An exception indicating that a target task is not running.
Gateway to send an
OperatorEvent or CoordinationRequest from a Task to the OperatorCoordinator JobManager side.The scope format for the
TaskMetricGroup.Container for multiple
tasks belonging to the same slot.Payload interface for
TaskSlot.Container for multiple
TaskSlot instances.Default implementation of
TaskSlotTable.Deprecated.
Internal class for savepoint backwards compatibility.
This interface provides methods to report and retrieve state for a task.
This class is the default implementation of
TaskStateManager and collaborates with the
job manager through CheckpointResponder) as well as a task-manager-local state store.This class encapsulates state handles to the snapshots of all operator instances executed within
one task.
Statistics for a single task/operator that gathers all statistics of its subtasks and provides
summary statistics about all subtasks.
Summary of the subtask stats of a single task/operator.
Encapsulates the common functionality for requesting statistics from tasks and combining their
responses.
A pending task stats request, which collects samples from individual tasks and completes the
response future upon gathering all of of them.
Exception indicating a problem with the
Task submission at the TaskException.Response to the request to collect thread details samples.
This class facilitates JVM-local exchange between stages of a batch job.
Models the functionality that the termination of an iterative task can be requested from outside.
Signals that the iteration is completely executed, participating tasks must terminate now.
Termination mode query parameter.
Termination mode.
A keyed state backend interface for internal testing purpose.
Class containing thread dump information.
Class containing information about a thread.
A coordinator for triggering and collecting thread info stats of running job vertex subtasks.
A serializable wrapper container for transferring parts of the
ThreadInfo.A wrapper for parameters of a thread info sampling request.
A timestamp queue based threshold meter.
Exception thrown when a threshold exceeds.
Class for measuring the throughput based on incoming data size and measurement period.
Annotation to classify subclasses of
Throwable into categories w.r.t. recovery.Helper class, given a exception do the classification.
Enum for the classification of
Throwable objects into failure/recovery classes.ExecutionPlanWriter implementation which does not allow to store ExecutionPlan.Modified
Iterator interface that lets the implementation throw an IOException.The
TierConsumerAgent is the consumer agent of each tier in tiered store, which could
read data from responding tier.A wrapper internal shuffle master class for tiered storage.
The internal
ShuffleMasterSnapshot for hybrid shuffle.TieredResultPartition appends records and events to the tiered storage, which supports
the upstream dynamically switches storage tier for writing shuffle data, and the downstream will
read data from the relevant tier.TieredResultPartitionFactory contains the components to set up tiered storage.This class represents a snapshot of tiered shuffle master, which can be used to restore the
internal state of the shuffle master.
The bytes based unique identification for the Tiered Storage.
Configurations for the Tiered Storage.
TieredStorageConsumerClient is used to read buffer from tiered store.Describe the different data sources in
TieredStorageConsumerClient.Identifier interface in the Tiered Storage.
Utils to convert the Ids to Tiered Storage Ids, or vice versa.
Identifier of an InputChannel.
Client of the Tiered Storage used by the master.
The
TieredStorageMemoryManager is to request or recycle buffers from LocalBufferPool for different memory owners, for example, the tiers, the buffer accumulator,
etc.The implementation for
TieredStorageMemoryManager.The memory specs for a memory owner, including the owner itself, the number of guaranteed buffers
of the memory owner, etc.
TieredStorageNettyService is used to create writers and readers to netty.The default implementation of
TieredStorageNettyService.Identifier of a partition.
Client of the Tiered Storage used by the producer.
The metric statistics for the tiered storage producer.
The resource (e.g., local files, remote storage files, etc.) for the Tiered Storage.
A registry that maintains local or remote resources that correspond to a certain set of data in
the Tiered Storage.
The
TieredStorageResultSubpartitionView is the implementation of ResultSubpartitionView of TieredResultPartition.When getting buffers, The
SortBasedDataBuffer need not recycle the read target buffer..Identifier of a subpartition.
Identifier of a topic.
Utils for reading from or writing to tiered storage.
A factory that creates all the components of a tier.
A class responsible for initializing and returning a list of
TierFactory instances based
on the provided TieredStorageConfiguration.The master-side agent of a Tier.
The producer-side agent of a Tier.
The
TierShuffleDescriptor interface is the shuffle descriptor for a single tier.A handler to process the call back result from each tier, the callbacks can be some events, some
errors, some exceptions, etc.
TimeDomain specifies whether a firing timer is based on event time or processing time.Listener for timeout events by the
TimerService.RuntimeException for wrapping exceptions that are thrown in the timer callback of the
timer service in StreamTask.TimerGauge measures how much time is spent in a given state, with entry into that state
being signaled by TimerGauge.markStart().Listens for
TimerGauge.markStart() and TimerGauge.markEnd() events.Implementation of
InternalTimer to use with a HeapPriorityQueueSet.A serializer for
TimerHeapInternalTimer objects that produces a serialization format that
is lexicographically aligned the priority of the timers.Snapshot class for the
TimerSerializer.Service to register timeouts for a given key.
Interface for working with time and timers.
A common timer service interface with life cycle methods.
A
FileInputSplit with TimestampedInputSplit.An extended
InputSplit that also includes information about:
The modification time of the file this split belongs to.Stores the value and the timestamp of the record.
Basic interface for the timestamp extraction and watermark generation logic for the
SourceReader.Lets the owner/creator of the output know about latest emitted watermark.
A simple implementation of a context that is both
TimestampAssignerSupplier.Context and
WatermarkGeneratorSupplier.Context.A stream operator that may do one or both of the following: extract timestamps from events and
generate watermarks.
Implementation of the
WatermarkEmitter, based on the components that are available
inside a stream operator.Gauge metric measuring the maximal time to consume all buffers of all input gates.
The serializer used to write the TimeWindow type.
Serializer configuration snapshot for compatibility and format evolution.
Topology<VID extends VertexID,RID extends ResultID,V extends Vertex<VID,RID,V,R>,R extends Result<VID,RID,V,R>,PR extends PipelinedRegion<VID,RID,V,R>>
Extends the
BaseTopology by pipelined regions.Encapsulates everything needed for the instantiation and configuration of a
TraceReporter.A
TransformationTranslator is responsible for translating a given Transformation
to its runtime implementation depending on the execution mode.A context giving the necessary information for the translation of a given transformation.
A
Transformation that contains lineage information.Provides access to transient BLOB files stored at the
BlobServer.BLOB key referencing transient BLOB files.
A service to retrieve transient binary large objects (BLOBs) which are deleted on the
BlobServer when they are retrieved.A
Trigger determines when a pane of a window should be evaluated to emit the results for
that part of the window.Extension of
Trigger.TriggerContext that is given to Trigger.onMerge(Window, OnMergeContext).A context object that is given to
Trigger methods to allow them to register timer
callbacks and deal with state.Interface for things that can be called by
InternalTimerService.Wrapper interface for functions triggering checkpoints.
Identifies a savepoint trigger request.
JSON deserializer for
TriggerId.JSON serializer for
TriggerId.MessagePathParameter for the trigger id of an asynchronous operation.Response to the triggering of an asynchronous operation.
Result type for trigger methods.
Wrapper interface for functions triggering savepoints.
Describes the context of taking a savepoint: Whether it is a savepoint for a running job or
whether the job is cancelled, suspended or terminated with a savepoint.
This class wraps aggregating function with TTL logic.
This class wraps a
TypeSerializer with ttl awareness.A
TypeSerializerSnapshot for TtlAwareSerializer.Wrap the TypeSerializerSnapshot restored from
StateMetaInfoSnapshot to
TtlAwareSerializerSnapshotThis class wraps reducing function with TTL logic.
This state factory wraps state objects, produced by backends, with TTL logic.
Serializer for user state value with TTL.
Serializer for user state value with TTL.
A
TypeSerializerSnapshot for TtlSerializer.A
TypeSerializerSnapshot for TtlSerializer.Provides time to TTL logic to judge about state expiration.
Common functions related to State TTL.
This class wraps user value of state with TTL.
A
WindowAssigner that windows elements into windows based on the timestamp of the
elements.A
WindowAssigner that windows elements into windows based on the current system time of
the machine the operation is running on.Interface for stream operators with two inputs.
A factory to create
TwoInputStreamOperator.A
StreamTask for executing a TwoInputStreamOperator and supporting the TwoInputStreamOperator to select input for reading.This Transformation represents the application of a
TwoInputStreamOperator to two input
Transformations.A
TransformationTranslator for the TwoInputTransformation.Udf stream operator factory.
Exception indicating that a Dispatcher operation is temporarily unavailable.
A buffer which encapsulates the logic of dealing with the response from the
CollectSinkFunction.This implementation does not decorate the stream with any compression.
Exception denoting that a slot request can not be fulfilled by any slot in the cluster.
Input gate wrapper to union the input from multiple input gates.
A wrapper to union the output from multiple
ResultSubpartitionViews.This transformation represents a union of several input
Transformations.A
TransformationTranslator for the UnionTransformation.Thrown if there is no
KvStateLocation found for the requested registration name.Exception that indicates that there is no ongoing or completed savepoint for a given
JobID and TriggerId pair.Unknown
ShuffleDescriptor for which the producer has not been deployed yet.Exception denoting that a requested
TaskExecutor could not be found.This is a placeholder when the tier shuffle descriptor should be unknown because the netty
shuffle descriptor is unknown.
JobManagerJobMetricGroupFactory which returns an unregistered JobManagerJobMetricGroup.A collection of safe drop-in replacements for existing
ComponentMetricGroups.A safe drop-in replacement for
JobManagerJobMetricGroups.A safe drop-in replacement for
JobManagerMetricGroups.A safe drop-in replacement for
JobManagerOperatorMetricGroups.A safe drop-in replacement for
InternalOperatorMetricGroups.A safe drop-in replacement for
ProcessMetricGroups.A safe drop-in replacement for
ResourceManagerMetricGroups.A safe drop-in replacement for
SlotManagerMetricGroups.A safe drop-in replacement for
TaskManagerJobMetricGroups.A safe drop-in replacement for
TaskManagerMetricGroups.A safe drop-in replacement for
TaskMetricGroups.This class encapsulates the connection information of a TaskManager, without resolving the
hostname.
Executor implementation which fails when UnsupportedOperationExecutor.execute(Runnable) is called.An exception that indicates that a time characteristic was used that is not supported in the
current operation.
Message headers for a web handler request.
Calls
SchedulerNG.updateTaskExecutionState(TaskExecutionStateTransition) on task failure.Specifies the upper limit of exceptions to return for JobExceptionsHandler.
Tracks status of upstream channels while they recover.
A gauge that returns (in milliseconds) how long a job has been running.
User accumulator info.
UserFunctionProvider<F extends org.apache.flink.api.common.functions.Function>
The interface with user function.
An adaptor that transforms
InternalValueState
into InternalValueState.The
VariableLengthByteKeyAndValueComparator is used by KeyedSortPartitionOperator
to compare records according to both the record key and record value.Vertex<VID extends VertexID,RID extends ResultID,V extends Vertex<VID,RID,V,R>,R extends Result<VID,RID,V,R>>
Represents a logical or execution task.
Contains the attempt numbers per vertex.
Records the end of data event of each task, and allows for checking whether all tasks of a
JobGraph have reached the end of data.This class encapsulates the operation that checks if there are illegal modification to the
JobGraph when restoring from a checkpoint with partially or fully finished operator states.
Flame Graph representation for a job vertex or an execution vertex.
Graph node.
Factory class for creating Flame Graph representations.
Common utils for processing vertex groups.
ID of a
Vertex.Util to compute
JobVertexInputInfos for execution job vertex.A store contains all the
JobVertexInputInfos.Core result of
SlotAllocator.determineParallelism(JobInformation, Collection) among with
slotAssignments, describing the parallelism each vertex could be scheduled with.VertexParallelismAndInputInfosDecider is responsible for deciding the parallelism and
JobVertexInputInfos of a job vertex, based on the information of the consumed blocking
results.Utils class for VertexParallelismAndInputInfosDecider.
Manages the parallelism properties for a vertex in the execution graph, as well as how they can
change during runtime.
Contains the max parallelism per vertex, along with metadata about how these maxes were
calculated.
Interface for a tracker of statistics for
AccessExecutionJobVertex.Thread info statistics of single JobVertex or ExecutionVertex.
Tracker of thread infos for
ExecutionJobVertex.Builder for
VertexThreadInfoTracker.SchedulingStrategy instance which schedules tasks in granularity of vertex (which
indicates this strategy only supports batch jobs).The factory for creating
VertexwiseSchedulingStrategy.The ViewUpdater is responsible for updating all metrics that implement the
View
interface.A blob store doing nothing.
BlobWriter which does not support writing BLOBs to a store.
No-op implementation of the
HistoryServerArchivist.No-op implementation of the
MetricFetcher.Singleton placeholder class for state without a namespace.
Serializer for
VoidNamespace.Serializer configuration snapshot for compatibility and format evolution.
TypeInformation for VoidNamespace.A Watermark tells operators that no elements with a timestamp older or equal to the watermark
timestamp should arrive at the operator.
Signals source operators the maximum watermark that emitted records can have.
A class used to combine
Watermarks.This event wraps the
Watermark, it is used in propagate Watermark between shuffle
components, and should not be visible to operators and functions.A
Gauge for exposing the current input/output watermark.An
Output that measures the last emitted watermark with a WatermarkGauge.A Watermark Status element informs stream tasks whether or not they should continue to expect
watermarks from the input stream that sent them.
The
WatermarkStrategyWithPeriodicWatermarks assigns event time timestamps to elements,
and generates low watermarks that signal event time progress within the stream.The
AssignerWithPunctuatedWatermarks assigns event time timestamps to elements, and
generates low watermarks that signal event time progress within the stream.An adapter that exposes a
WatermarkOutput based on a PushingAsyncDataInput.DataOutput.Utils class for
Watermark.Rest handler specification for the web content handler.
Rest endpoint which serves the web frontend REST calls.
Interface for
WebMonitorEndpoint extensions.Empty web monitor extension which can be used as a null object.
Utilities for the web runtime monitor.
Singleton to hold the log file, the stdout file, the log directory.
A
Window is a grouping of elements into finite buckets.A
WindowAssigner assigns zero or more Windows to an element.A context provided to the
WindowAssigner that allows it to query the current
processing time.A
WindowedStream represents a data stream where elements are grouped by key, and for each
key, the stream of elements is split into windows based on a WindowAssigner.Base interface for functions that are evaluated over keyed (grouped) windows.
An operator that implements the logic for windowing based on a
WindowAssigner and Trigger.Internal class for keeping track of in-flight timers.
A builder for creating
WindowOperators.A
WindowStagger staggers offset in runtime for each window assignment.An interface to be implemented by transformations that have explicitly set
Boundedness.A
FileMergingSnapshotManager that merging files within a checkpoint.This interface can be implemented by streaming functions that need to trigger a "global action"
on the master (in the checkpoint coordinator) as part of every checkpoint and restore operation.
Completion event sent from each
IterationHead to the
IterationSynchronizationSinkTask.This class extends the
TaskExecutorConnection, adding the worker information.Resource specification of a worker, mainly used by SlotManager requesting from ResourceManager.
Builder for
WorkerResourceSpec.Factory for creating deployment specific default
WorkerResourceSpec.Class that manages a working directory for a process/instance.
A workset iteration is by definition converged if no records have been updated in the
solutionset.
A
Collector to update the iteration workset (partial solution for bulk iterations).Deprecated.
This should be removed once we can send arbitrary REST calls via the Yarn proxy.
Deprecated.
This should be removed once we can send arbitrary REST calls via the Yarn proxy.
A V1 operator that needs access to the
MailboxExecutor should implement this interface.Deprecated.
CheckpointIDCounter instances for JobManagers running in HighAvailabilityMode.ZOOKEEPER.CheckpointCoordinator components in HighAvailabilityMode.ZOOKEEPER.Singleton
CheckpointStoreUtil implementation for ZooKeeper.ZooKeeper based implementation for
ClientHighAvailabilityServices.Singleton
ExecutionPlanStoreUtil implementation for ZooKeeper.ExecutionPlanStoreWatcher implementation for ZooKeeper.ZooKeeper based
LeaderElectionDriver implementation.Factory for
ZooKeeperLeaderElectionDriver.ZooKeeper HA services that only use a single leader election per process.
The counterpart to the
ZooKeeperLeaderElectionDriver.Policy when to clear the leader information and to notify the listener about it.
LeaderRetrievalDriverFactory implementation for Zookeeper.Responsible for installing a process-wide ZooKeeper security configuration.
Wrapper class for a
SharedCount so that we don't expose a curator dependency in our
internal APIs.Wrapper class for a
SharedValue so that we don't expose a curator dependency in our
internal APIs.Class which stores state via the provided
RetrievableStateStorageHelper and writes the
returned state handle to ZooKeeper.Class containing helper functions to interact with ZooKeeper.
Secure
ACLProvider implementation.ZooKeeper client ACL mode enum.
Wrapper class for a
VersionedValue so that we don't expose a curator dependency in our
internal APIs.
CheckpointStoragefunctionality.