Package org.apache.flink.client.program

Class MiniClusterClient

java.lang.Object
org.apache.flink.client.program.MiniClusterClient
All Implemented Interfaces:
AutoCloseable, ClusterClient<MiniClusterClient.MiniClusterId>
public class MiniClusterClient extends Object implements ClusterClient<MiniClusterClient.MiniClusterId>
Client to interact with a MiniCluster.

  • Constructor Details

    • MiniClusterClient

      public MiniClusterClient(@Nonnull org.apache.flink.configuration.Configuration configuration, @Nonnull org.apache.flink.runtime.minicluster.MiniCluster miniCluster)

  • Method Details

    • getFlinkConfiguration

      public org.apache.flink.configuration.Configuration getFlinkConfiguration()
      Description copied from interface: ClusterClient
      Return the Flink configuration object.
      Specified by:
      getFlinkConfiguration in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Returns:
      The Flink configuration object

    • submitJob

      public CompletableFuture<org.apache.flink.api.common.JobID> submitJob(org.apache.flink.streaming.api.graph.ExecutionPlan executionPlan)
      Description copied from interface: ClusterClient
      Submit the given ExecutionPlan to the cluster.
      Specified by:
      submitJob in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      executionPlan - to submit
      Returns:
      JobID of the submitted job

    • requestJobResult

      public CompletableFuture<org.apache.flink.runtime.jobmaster.JobResult> requestJobResult(@Nonnull org.apache.flink.api.common.JobID jobId)
      Description copied from interface: ClusterClient
      Request the JobResult for the given JobID.
      Specified by:
      requestJobResult in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      jobId - for which to request the JobResult
      Returns:
      Future which is completed with the JobResult

    • cancel

      public CompletableFuture<org.apache.flink.runtime.messages.Acknowledge> cancel(org.apache.flink.api.common.JobID jobId)
      Description copied from interface: ClusterClient
      Cancels a job identified by the job id.
      Specified by:
      cancel in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      jobId - the job id

    • cancelWithSavepoint

      public CompletableFuture<String> cancelWithSavepoint(org.apache.flink.api.common.JobID jobId, @Nullable String savepointDirectory, org.apache.flink.core.execution.SavepointFormatType formatType)
      Description copied from interface: ClusterClient
      Cancels a job identified by the job id and triggers a savepoint.
      Specified by:
      cancelWithSavepoint in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      jobId - the job id
      savepointDirectory - directory the savepoint should be written to
      formatType - a binary format of the savepoint
      Returns:
      future of path where the savepoint is located

    • stopWithSavepoint

      public CompletableFuture<String> stopWithSavepoint(org.apache.flink.api.common.JobID jobId, boolean advanceToEndOfEventTime, @Nullable String savepointDirectory, org.apache.flink.core.execution.SavepointFormatType formatType)
      Description copied from interface: ClusterClient
      Stops a program on Flink cluster whose job-manager is configured in this client's configuration. Stopping works only for streaming programs. Be aware, that the program might continue to run for a while after sending the stop command, because after sources stopped to emit data all operators need to finish processing.
      Specified by:
      stopWithSavepoint in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      jobId - the job ID of the streaming program to stop
      advanceToEndOfEventTime - flag indicating if the source should inject a MAX_WATERMARK in the pipeline
      savepointDirectory - directory the savepoint should be written to
      formatType - a binary format of the savepoint
      Returns:
      a CompletableFuture containing the path where the savepoint is located

    • stopWithDetachedSavepoint

      public CompletableFuture<String> stopWithDetachedSavepoint(org.apache.flink.api.common.JobID jobId, boolean advanceToEndOfEventTime, @Nullable String savepointDirectory, org.apache.flink.core.execution.SavepointFormatType formatType)
      Description copied from interface: ClusterClient
      Stops a program on Flink cluster whose job-manager is configured in this client's configuration. Stopping works only for streaming programs. Be aware, that the program might continue to run for a while after sending the stop command, because after sources stopped to emit data all operators need to finish processing.
      Specified by:
      stopWithDetachedSavepoint in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      jobId - the job ID of the streaming program to stop
      advanceToEndOfEventTime - flag indicating if the source should inject a MAX_WATERMARK in the pipeline
      savepointDirectory - directory the savepoint should be written to
      formatType - a binary format of the savepoint
      Returns:
      the savepoint trigger id

    • triggerSavepoint

      public CompletableFuture<String> triggerSavepoint(org.apache.flink.api.common.JobID jobId, @Nullable String savepointDirectory, org.apache.flink.core.execution.SavepointFormatType formatType)
      Description copied from interface: ClusterClient
      Triggers a savepoint for the job identified by the job id. The savepoint will be written to the given savepoint directory, or CheckpointingOptions.SAVEPOINT_DIRECTORY if it is null.
      Specified by:
      triggerSavepoint in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      jobId - job id
      savepointDirectory - directory the savepoint should be written to
      formatType - a binary format of the savepoint
      Returns:
      path future where the savepoint is located

    • triggerCheckpoint

      public CompletableFuture<Long> triggerCheckpoint(org.apache.flink.api.common.JobID jobId, org.apache.flink.core.execution.CheckpointType checkpointType)
      Description copied from interface: ClusterClient
      Triggers a checkpoint for the job identified by the job id. The checkpoint will be written to the checkpoint directory for the job.
      Specified by:
      triggerCheckpoint in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      jobId - job id
      checkpointType - the checkpoint type (configured / full / incremental)

    • triggerDetachedSavepoint

      public CompletableFuture<String> triggerDetachedSavepoint(org.apache.flink.api.common.JobID jobId, @Nullable String savepointDirectory, org.apache.flink.core.execution.SavepointFormatType formatType)
      Description copied from interface: ClusterClient
      Triggers a detached savepoint for the job identified by the job id. The savepoint will be written to the given savepoint directory, or CheckpointingOptions.SAVEPOINT_DIRECTORY if it is null. Notice that: the detached savepoint will return with a savepoint trigger id instead of the path future, that means the client will return very quickly.
      Specified by:
      triggerDetachedSavepoint in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      jobId - job id
      savepointDirectory - directory the savepoint should be written to
      formatType - a binary format of the savepoint
      Returns:
      the savepoint trigger id

    • disposeSavepoint

      public CompletableFuture<org.apache.flink.runtime.messages.Acknowledge> disposeSavepoint(String savepointPath)
      Description copied from interface: ClusterClient
      Dispose the savepoint under the given path.
      Specified by:
      disposeSavepoint in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      savepointPath - path to the savepoint to be disposed
      Returns:
      acknowledge future of the dispose action

    • listJobs

      public CompletableFuture<Collection<org.apache.flink.runtime.client.JobStatusMessage>> listJobs()
      Description copied from interface: ClusterClient
      Lists the currently running and finished jobs on the cluster.
      Specified by:
      listJobs in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Returns:
      future collection of running and finished jobs

    • getAccumulators

      public CompletableFuture<Map<String,Object>> getAccumulators(org.apache.flink.api.common.JobID jobID, ClassLoader loader)
      Description copied from interface: ClusterClient
      Requests and returns the accumulators for the given job identifier. Accumulators can be requested while a is running or after it has finished.
      Specified by:
      getAccumulators in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      jobID - The job identifier of a job.
      loader - The class loader for deserializing the accumulator results.
      Returns:
      A Map containing the accumulator's name and its value.

    • getJobStatus

      public CompletableFuture<org.apache.flink.api.common.JobStatus> getJobStatus(org.apache.flink.api.common.JobID jobId)
      Description copied from interface: ClusterClient
      Requests the JobStatus of the job with the given JobID.
      Specified by:
      getJobStatus in interface ClusterClient<MiniClusterClient.MiniClusterId>

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface ClusterClient<MiniClusterClient.MiniClusterId>

    • getClusterId

      public MiniClusterClient.MiniClusterId getClusterId()
      Description copied from interface: ClusterClient
      Returns the cluster id identifying the cluster to which the client is connected.
      Specified by:
      getClusterId in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Returns:
      cluster id of the connected cluster

    • shutDownCluster

      public void shutDownCluster()
      Description copied from interface: ClusterClient
      Shut down the cluster that this client communicate with.
      Specified by:
      shutDownCluster in interface ClusterClient<MiniClusterClient.MiniClusterId>

    • getWebInterfaceURL

      public String getWebInterfaceURL()
      Description copied from interface: ClusterClient
      Returns an URL (as a string) to the cluster web interface.
      Specified by:
      getWebInterfaceURL in interface ClusterClient<MiniClusterClient.MiniClusterId>

    • sendCoordinationRequest

      public CompletableFuture<org.apache.flink.runtime.operators.coordination.CoordinationResponse> sendCoordinationRequest(org.apache.flink.api.common.JobID jobId, String operatorUid, org.apache.flink.runtime.operators.coordination.CoordinationRequest request)
      Description copied from interface: ClusterClient
      Sends out a request to a specified coordinator and return the response.

      On the client side, a unique operatorUid must be defined to identify an operator. Otherwise, the query cannot be executed correctly. Note that we use operatorUid instead of operatorID because the latter is an internal runtime concept that cannot be recognized by the client.

      Specified by:
      sendCoordinationRequest in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      jobId - specifies the job which the coordinator belongs to
      operatorUid - specifies which coordinator to receive the request
      request - the request to send
      Returns:
      the response from the coordinator

    • listCompletedClusterDatasetIds

      public CompletableFuture<Set<org.apache.flink.util.AbstractID>> listCompletedClusterDatasetIds()
      Description copied from interface: ClusterClient
      Return a set of ids of the completed cluster datasets.
      Specified by:
      listCompletedClusterDatasetIds in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Returns:
      A set of ids of the completely cached intermediate dataset.

    • invalidateClusterDataset

      public CompletableFuture<Void> invalidateClusterDataset(org.apache.flink.util.AbstractID clusterDatasetId)
      Description copied from interface: ClusterClient
      Invalidate the cached intermediate dataset with the given id.
      Specified by:
      invalidateClusterDataset in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      clusterDatasetId - id of the cluster dataset to be invalidated.
      Returns:
      Future which will be completed when the cached dataset is invalidated.

    • reportHeartbeat

      public CompletableFuture<Void> reportHeartbeat(org.apache.flink.api.common.JobID jobId, long expiredTimestamp)
      Description copied from interface: ClusterClient
      The client reports the heartbeat to the dispatcher for aliveness.
      Specified by:
      reportHeartbeat in interface ClusterClient<MiniClusterClient.MiniClusterId>
      Parameters:
      jobId - The jobId for the client and the job.
      Returns: