Package org.apache.flink.runtime.jobmaster.slotpool

Interface DeclarativeSlotPool

All Known Implementing Classes:
BlocklistDeclarativeSlotPool, DefaultDeclarativeSlotPool
public interface DeclarativeSlotPool
Slot pool interface which uses Flink's declarative resource management protocol to acquire resources.

In order to acquire new resources, users need to increase the required resources. Once they no longer need the resources, users need to decrease the required resources so that superfluous resources can be returned.

  • Method Details

    • increaseResourceRequirementsBy

      void increaseResourceRequirementsBy(ResourceCounter increment)
      Increases the resource requirements by increment.
      Parameters:
      increment - increment by which to increase the resource requirements

    • decreaseResourceRequirementsBy

      void decreaseResourceRequirementsBy(ResourceCounter decrement)
      Decreases the resource requirements by decrement.
      Parameters:
      decrement - decrement by which to decrease the resource requirements

    • setResourceRequirements

      void setResourceRequirements(ResourceCounter resourceRequirements)
      Sets the resource requirements to the given resourceRequirements.
      Parameters:
      resourceRequirements - new resource requirements

    • getResourceRequirements

      Collection<ResourceRequirement> getResourceRequirements()
      Returns the current resource requirements.
      Returns:
      current resource requirements

    • offerSlots

      Collection<SlotOffer> offerSlots(Collection<? extends SlotOffer> offers, TaskManagerLocation taskManagerLocation, TaskManagerGateway taskManagerGateway, long currentTime)
      Offers slots to this slot pool. The slot pool is free to accept as many slots as it needs.
      Parameters:
      offers - offers containing the list of slots offered to this slot pool
      taskManagerLocation - taskManagerLocation is the location of the offering TaskExecutor
      taskManagerGateway - taskManagerGateway is the gateway to talk to the offering TaskExecutor
      currentTime - currentTime is the time the slots are being offered
      Returns:
      collection of accepted slots; the other slot offers are implicitly rejected

    • registerSlots

      Collection<SlotOffer> registerSlots(Collection<? extends SlotOffer> slots, TaskManagerLocation taskManagerLocation, TaskManagerGateway taskManagerGateway, long currentTime)
      Registers the given set of slots at the slot pool. The slot pool will try to accept all slots unless the slot is unavailable (for example, the TaskManger is blocked).

      The difference from offerSlots(java.util.Collection<? extends org.apache.flink.runtime.taskexecutor.slot.SlotOffer>, org.apache.flink.runtime.taskmanager.TaskManagerLocation, org.apache.flink.runtime.jobmanager.slots.TaskManagerGateway, long) is that this method allows accepting slots which exceed the currently required, but the offerSlots(java.util.Collection<? extends org.apache.flink.runtime.taskexecutor.slot.SlotOffer>, org.apache.flink.runtime.taskmanager.TaskManagerLocation, org.apache.flink.runtime.jobmanager.slots.TaskManagerGateway, long) only accepts those slots that are currently required.

      Parameters:
      slots - slots to register
      taskManagerLocation - taskManagerLocation is the location of the offering TaskExecutor
      taskManagerGateway - taskManagerGateway is the gateway to talk to the offering TaskExecutor
      currentTime - currentTime is the time the slots are being offered
      Returns:
      the successfully registered slots; the other slot offers are implicitly rejected

    • getFreeSlotTracker

      FreeSlotTracker getFreeSlotTracker()
      Returns the free slot tracker.
      Returns:
      free slot tracker

    • getAllSlotsInformation

      Collection<? extends SlotInfo> getAllSlotsInformation()
      Returns the slot information for all slots (free and allocated slots).
      Returns:
      collection of slot information

    • containsFreeSlot

      boolean containsFreeSlot(AllocationID allocationId)
      Checks whether the slot pool contains a slot with the given AllocationID and if it is free.
      Parameters:
      allocationId - allocationId specifies the slot to check for
      Returns:
      true if the slot pool contains a free slot registered under the given allocation id; otherwise false

    • reserveFreeSlot

      PhysicalSlot reserveFreeSlot(AllocationID allocationId, ResourceProfile requiredSlotProfile)
      Reserves the free slot identified by the given allocationId and maps it to the given requiredSlotProfile.
      Parameters:
      allocationId - allocationId identifies the free slot to allocate
      requiredSlotProfile - requiredSlotProfile specifying the resource requirement
      Returns:
      a PhysicalSlot representing the allocated slot
      Throws:
      IllegalStateException - if no free slot with the given allocationId exists or if the specified slot cannot fulfill the requiredSlotProfile

    • freeReservedSlot

      ResourceCounter freeReservedSlot(AllocationID allocationId, @Nullable Throwable cause, long currentTime)
      Frees the reserved slot identified by the given allocationId. If no slot with allocationId exists, then the call is ignored.

      Whether the freed slot is returned to the owning TaskExecutor is implementation dependent.

      Parameters:
      allocationId - allocationId identifying the slot to release
      cause - cause for releasing the slot; can be null
      currentTime - currentTime when the slot was released
      Returns:
      the resource requirements that the slot was fulfilling

    • releaseSlots

      ResourceCounter releaseSlots(ResourceID owner, Exception cause)
      Releases all slots belonging to the owning TaskExecutor if it has been registered.
      Parameters:
      owner - owner identifying the owning TaskExecutor
      cause - cause for failing the slots
      Returns:
      the resource requirements that all slots were fulfilling; empty if all slots were currently free

    • releaseSlot

      ResourceCounter releaseSlot(AllocationID allocationId, Exception cause)
      Releases the slot specified by allocationId if one exists.
      Parameters:
      allocationId - allocationId identifying the slot to fail
      cause - cause for failing the slot
      Returns:
      the resource requirements that the slot was fulfilling; empty if the slot was currently free

    • containsSlots

      boolean containsSlots(ResourceID owner)
      Returns whether the slot pool has a slot registered which is owned by the given TaskExecutor.
      Parameters:
      owner - owner identifying the TaskExecutor for which to check whether the slot pool has some slots registered
      Returns:
      true if the given TaskExecutor has a slot registered at the slot pool

    • releaseIdleSlots

      void releaseIdleSlots(long currentTimeMillis)
      Releases slots which have exceeded the idle slot timeout and are no longer needed to fulfill the resource requirements.
      Parameters:
      currentTimeMillis - current time

    • registerNewSlotsListener

      void registerNewSlotsListener(DeclarativeSlotPool.NewSlotsListener listener)
      Registers a listener which is called whenever new slots become available.
      Parameters:
      listener - which is called whenever new slots become available