Interface CheckpointStatsTracker

All Known Implementing Classes:
DefaultCheckpointStatsTracker, NoOpCheckpointStatsTracker

public interface CheckpointStatsTracker
Tracker for checkpoint statistics.

This is tightly integrated with the CheckpointCoordinator in order to ease the gathering of fine-grained statistics.

The tracked stats include summary counts, a detailed history of recent and in progress checkpoints as well as summaries about the size, duration and more of recent checkpoints.

Data is gathered via callbacks in the CheckpointCoordinator and related classes like PendingCheckpoint and CompletedCheckpoint, which receive the raw stats data in the first place.

The statistics are accessed via createSnapshot() and exposed via both the web frontend and the Metric system.

  • Method Details

    • reportRestoredCheckpoint

      void reportRestoredCheckpoint(long checkpointID, CheckpointProperties properties, String externalPath, long stateSize)
    • reportCompletedCheckpoint

      void reportCompletedCheckpoint(CompletedCheckpointStats completed)
      Callback when a checkpoint completes.
      Parameters:
      completed - The completed checkpoint stats.
    • getPendingCheckpointStats

      @Nullable PendingCheckpointStats getPendingCheckpointStats(long checkpointId)
    • reportIncompleteStats

      void reportIncompleteStats(long checkpointId, ExecutionAttemptID attemptId, CheckpointMetrics metrics)
    • reportInitializationStarted

      void reportInitializationStarted(Set<ExecutionAttemptID> toInitialize, long initializationStartTs)
    • reportInitializationMetrics

      void reportInitializationMetrics(ExecutionAttemptID executionAttemptId, SubTaskInitializationMetrics initializationMetrics)
    • reportPendingCheckpoint

      @Nullable PendingCheckpointStats reportPendingCheckpoint(long checkpointId, long triggerTimestamp, CheckpointProperties props, Map<JobVertexID,Integer> vertexToDop)
      Creates a new pending checkpoint tracker.
      Parameters:
      checkpointId - ID of the checkpoint.
      triggerTimestamp - Trigger timestamp of the checkpoint.
      props - The checkpoint properties.
      vertexToDop - mapping of JobVertexID to DOP
      Returns:
      Tracker for statistics gathering or null if no stats were tracked.
    • reportFailedCheckpoint

      void reportFailedCheckpoint(FailedCheckpointStats failed)
    • reportFailedCheckpointsWithoutInProgress

      void reportFailedCheckpointsWithoutInProgress()
      Callback when a checkpoint failure without in progress checkpoint. For example, it should be callback when triggering checkpoint failure before creating PendingCheckpoint.
    • createSnapshot

      CheckpointStatsSnapshot createSnapshot()
      Creates a new snapshot of the available stats.
      Returns:
      The latest statistics snapshot.