Packages

c

com.nvidia.spark.rapids

GpuColumnarBatchSerializer

class GpuColumnarBatchSerializer extends Serializer with Serializable

Serializer for serializing ColumnarBatchs for use during normal shuffle.

The serialization write path takes the cudf Table that is described by the ColumnarBatch and uses cudf APIs to serialize the data into a sequence of bytes on the host. The data is returned to the Spark shuffle code where it is compressed by the CPU and written to disk.

The serialization read path is notably different. The sequence of serialized bytes IS NOT deserialized into a cudf Table but rather tracked in host memory by a ColumnarBatch that contains a SerializedTableColumn. During query planning, each GPU columnar shuffle exchange is followed by a GpuShuffleCoalesceExec that expects to receive only these custom batches of SerializedTableColumn. GpuShuffleCoalesceExec coalesces the smaller shuffle partitions into larger tables before placing them on the GPU for further processing.

Note

The RAPIDS shuffle does not use this code.

Linear Supertypes
Serializable, Serializer, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. GpuColumnarBatchSerializer
  2. Serializable
  3. Serializer
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new GpuColumnarBatchSerializer(dataSize: GpuMetric)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  6. var defaultClassLoader: Option[ClassLoader]
    Attributes
    protected
    Definition Classes
    Serializer
  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  11. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  12. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  13. def newInstance(): SerializerInstance
    Definition Classes
    GpuColumnarBatchSerializer → Serializer
  14. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  15. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  16. def setDefaultClassLoader(classLoader: ClassLoader): Serializer
    Definition Classes
    Serializer
  17. def supportsRelocationOfSerializedObjects: Boolean
    Definition Classes
    GpuColumnarBatchSerializer → Serializer
  18. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  19. def toString(): String
    Definition Classes
    AnyRef → Any
  20. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  22. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated
    Deprecated

Inherited from Serializable

Inherited from Serializer

Inherited from AnyRef

Inherited from Any

Ungrouped