Package org.apache.flink.runtime.jobmaster.event

Class FileSystemJobEventStore

java.lang.Object
org.apache.flink.runtime.jobmaster.event.FileSystemJobEventStore
All Implemented Interfaces:
JobEventStore
public class FileSystemJobEventStore extends Object implements JobEventStore
Implementation of JobEventStore that stores all JobEvent instances in a FileSystem. Events are written and read sequentially, ensuring a consistent order of events.

Write operations to the file system are primarily asynchronous, leveraging a ScheduledExecutorService to periodically flush the buffered output stream, which executes write tasks at an interval determined by the configure option JobEventStoreOptions.FLUSH_INTERVAL.

Read operations are performed synchronously, with the calling thread directly interacting with the file system to fetch event data.

  • Constructor Details

    • FileSystemJobEventStore

      public FileSystemJobEventStore(org.apache.flink.api.common.JobID jobID, org.apache.flink.configuration.Configuration configuration) throws IOException
      Throws:
      IOException

    • FileSystemJobEventStore

      @VisibleForTesting public FileSystemJobEventStore(org.apache.flink.core.fs.Path workingDir, org.apache.flink.configuration.Configuration configuration) throws IOException
      Throws:
      IOException

  • Method Details

    • start

      public void start() throws IOException
      Description copied from interface: JobEventStore
      Start the store. This method should be called before any other operations.
      Specified by:
      start in interface JobEventStore
      Throws:
      IOException

    • createJobEventWriterExecutor

      protected ScheduledExecutorService createJobEventWriterExecutor()

    • stop

      public void stop(boolean clearEventLogs)
      Description copied from interface: JobEventStore
      Stop the store.
      Specified by:
      stop in interface JobEventStore
      Parameters:
      clearEventLogs - Whether to clear the job events that have been recorded in the store.

    • writeEvent

      public void writeEvent(JobEvent event, boolean cutBlock)
      Description copied from interface: JobEventStore
      Write a job event.
      Specified by:
      writeEvent in interface JobEventStore
      Parameters:
      event - The job event that will be recorded.
      cutBlock - If set to true, the current output file will be closed after writing this event, and a new output file will be created for subsequent events. This parameter effectively controls the segmentation of event data into separate files.

    • writeEventRunnable

      @VisibleForTesting protected void writeEventRunnable(JobEvent event, boolean cutBlock)

    • readEvent

      public JobEvent readEvent() throws Exception
      Description copied from interface: JobEventStore
      Read a job event.
      Specified by:
      readEvent in interface JobEventStore
      Returns:
      job event.
      Throws:
      Exception

    • isEmpty

      public boolean isEmpty() throws Exception
      Description copied from interface: JobEventStore
      Returns whether the store is empty.
      Specified by:
      isEmpty in interface JobEventStore
      Returns:
      false if the store contains any job events, true otherwise.
      Throws:
      Exception