Package org.apache.flink.connector.base.source.reader.fetcher

Class SplitFetcher<E,SplitT extends org.apache.flink.api.connector.source.SourceSplit>

java.lang.Object
org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher<E,SplitT>
All Implemented Interfaces:
Runnable
@PublicEvolving public class SplitFetcher<E,SplitT extends org.apache.flink.api.connector.source.SourceSplit> extends Object implements Runnable
The internal fetcher runnable responsible for polling message from the external system.

  • Method Details

    • run

      public void run()
      Specified by:
      run in interface Runnable

    • addSplits

      public void addSplits(List<SplitT> splitsToAdd)
      Add splits to the split fetcher. This operation is asynchronous.
      Parameters:
      splitsToAdd - the splits to add.

    • removeSplits

      public void removeSplits(List<SplitT> splitsToRemove)
      Notice the split fetcher that some splits finished. This operation is asynchronous.
      Parameters:
      splitsToRemove - the splits need to be removed.

    • pauseOrResumeSplits

      public void pauseOrResumeSplits(Collection<SplitT> splitsToPause, Collection<SplitT> splitsToResume)
      Called when some splits of this source instance progressed too much beyond the global watermark of all subtasks. If the split reader implements SplitReader, it will relay the information asynchronously through the split fetcher thread.
      Parameters:
      splitsToPause - the splits to pause
      splitsToResume - the splits to resume

    • enqueueTask

      public void enqueueTask(SplitFetcherTask task)

    • getSplitReader

      public SplitReader<E,SplitT> getSplitReader()

    • fetcherId

      public int fetcherId()

    • shutdown

      public void shutdown()
      Shutdown the split fetcher.

    • shutdown

      public void shutdown(boolean waitingForRecordsProcessed)
      Shutdown the split fetcher. When waitingForRecordsProcessed is set to true, the split fetcher will block waiting for the previously emitted records to be processed before it closes the encapsulated SplitReader. Otherwise, it will just close the SplitReader.

      This method is package private because it should only be used by the SplitFetcherManager when closing the idle fetchers.

      Parameters:
      waitingForRecordsProcessed - whether wait for the previously emitted records to be processed.

    • pause

      public void pause()

    • resume

      public void resume()