Packages

•  package root

  Definition Classes

  root

•  package com

  Definition Classes

  root

•  package nvidia

  Definition Classes

  com

•  package spark

  Definition Classes

  nvidia

•  package rapids

  Definition Classes

  spark

•  package shuffle

  Definition Classes

  rapids

•  class WindowedBlockIterator[T <: BlockWithSize] extends Iterator[Seq[BlockRange[T]]]

  Given a set of blocks, this iterator returns BlockRanges of such blocks that fit windowSize.

  Given a set of blocks, this iterator returns BlockRanges of such blocks that fit windowSize. The ranges are just logical chunks of the blocks, so this class performs no memory management or copying.

  If a block is too large for the window, the block will be returned in next() until the full block can be covered.

  For example, given a block that is 4 window-sizes in length: block = [sb1, sb2, sb3, sb4]

  The window will return on next() four "sub-blocks", governed by windowSize: window.next() // sb1 window.next() // sb2 window.next() // sb3 window.next() // sb4

  If blocks are smaller than the windowSize, they will be packed: block1 = [b1] block2 = [b2] window.next() // [b1, b2]

  A mix of both scenarios above is possible: block1 = [sb11, sb12, sb13] // where sb13 is smaller than window length block2 = [b2]

  window.next() // sb11 window.next() // sb12 window.next() // [sb13, b2]

  T

  - the specific type of BlockWithSize

  Definition Classes

  shuffle

  Note

  this class does not own transferBlocks

  ,

  this class is not thread safe

• BlocksForWindow
• GroupedIterator

com.nvidia.spark.rapids.shuffle.WindowedBlockIterator

BlocksForWindow 