trait RapidsShuffleTransport extends AutoCloseable
This defines what a "transport" should support. The intention is to allow for various transport implementations to exist, for different communication frameworks.
It is an AutoCloseable and so the caller should close when the transport is no longer
needed.
- Alphabetic
- By Inheritance
- RapidsShuffleTransport
- AutoCloseable
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Abstract Value Members
-
abstract
def
cancelPending(handler: RapidsShuffleFetchHandler): Unit
Cancel requests that are waiting in the queue (not in-flight) for a specific handler
-
abstract
def
close(): Unit
- Definition Classes
- AutoCloseable
- Annotations
- @throws( classOf[java.lang.Exception] )
-
abstract
def
connect(peerBlockManagerId: BlockManagerId): ClientConnection
Connect to peer given a
BlockManagerIdConnect to peer given a
BlockManagerIdThis function is used during transport early start, if enabled.
- peerBlockManagerId
the peer's block manager id
- returns
a client connection for the peer
-
abstract
def
doneBytesInFlight(bytesCompleted: Long): Unit
(throttle) Signals that
bytesCompletedare done, allowing more requests through the throttle.(throttle) Signals that
bytesCompletedare done, allowing more requests through the throttle.- bytesCompleted
amount of bytes handled
-
abstract
def
getDirectByteBuffer(size: Long): RefCountedDirectByteBuffer
Returns a wrapped buffer of size Long.
Returns a wrapped buffer of size Long. The buffer may or may not be pooled.
The caller should call .close() on the returned RefCountedDirectByteBuffer when done.
- size
size of buffer required
- returns
the ref counted buffer
-
abstract
def
makeClient(blockManagerId: BlockManagerId): RapidsShuffleClient
This function will connect (if not connected already) to a peer described by
blockManagerId.This function will connect (if not connected already) to a peer described by
blockManagerId. Connections are cached.- blockManagerId
the peer's block manager id
- returns
RapidsShuffleClient instance that can be used to interact with the peer
-
abstract
def
makeServer(requestHandler: RapidsShuffleRequestHandler): RapidsShuffleServer
This function should only be needed once.
This function should only be needed once. The caller creates *a* server and it is used for the duration of the process.
- requestHandler
used to get metadata info, and acquire tables used in the shuffle.
- returns
the server instance
-
abstract
def
queuePending(reqs: Seq[PendingTransferRequest]): Unit
(throttle) Adds a set of requests to be throttled as limits allowed.
(throttle) Adds a set of requests to be throttled as limits allowed.
- reqs
requests to add to the throttle queue
-
abstract
def
tryGetReceiveBounceBuffers(remaining: Long, totalRequired: Int): Seq[BounceBuffer]
Get receive bounce buffers needed for a receive, limited by the amount of bytes to be received, and a hard limit on the number of buffers set by the caller using
totalRequired.Get receive bounce buffers needed for a receive, limited by the amount of bytes to be received, and a hard limit on the number of buffers set by the caller using
totalRequired.This function is non blocking. If it can't satisfy the bounce buffer request, an empty sequence is returned.
- remaining
amount of bytes remaining in the receive
- totalRequired
maximum no. of buffers that should be returned
- returns
a sequence of bounce buffers, or empty if the request can't be satisfied
-
abstract
def
tryGetSendBounceBuffers(remaining: Long, totalRequired: Int): Seq[SendBounceBuffers]
Get send bounce buffers needed for a receive, limited by the amount of bytes to be sent, and a hard limit on the number of buffers set by the caller using
totalRequired.Get send bounce buffers needed for a receive, limited by the amount of bytes to be sent, and a hard limit on the number of buffers set by the caller using
totalRequired.This function is non blocking. If it can't satisfy the bounce buffer request, an empty sequence is returned.
- remaining
amount of bytes remaining in the receive
- totalRequired
maximum no. of buffers that should be returned
- returns
a sequence of send bounce buffers, or empty if the request can't be satisfied
- Note
the send bounce buffer object most likely includes both a device buffer and a host memory buffer, since sends can come from the device or the host.
Concrete Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native() @HotSpotIntrinsicCandidate()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )