Type Members

1.  case class BB(instructionTable: SortedMap[Int, Instruction]) extends Logging with Product with Serializable

   A Basic Block (BB) is a set of instructions defined by instructionTable, where each entry in this table is a mapping of offset to Instruction.

   A Basic Block (BB) is a set of instructions defined by instructionTable, where each entry in this table is a mapping of offset to Instruction.

   The case class also provides some helpers, most importantly the propagateState helper which generates a map of BB to State.

2.  case class CFG(basicBlocks: List[BB], predecessor: Map[BB, List[BB]], successor: Map[BB, List[(Int, BB)]]) extends Product with Serializable

   The Control Flow Graph object.

   The Control Flow Graph object.

   basicBlocks

   : the basic blocks for this CFG

   predecessor

   : given a BB this maps the BBs to its predecessors

   successor

   : given a BB this maps the BBs to its successors. Each element in the succssor list also has an Int value which is used as a case for tableswitch and lookupswitch.

3.  case class CatalystExpressionBuilder(function: AnyRef) extends Logging with Product with Serializable

   CatalystExpressionBuilder

   CatalystExpressionBuilder

   This compiles a scala lambda expression into a catalyst expression.

   Here are the high-level steps:

   1) Use SerializedLambda and javaassist to get a reflection based interface to function this is done in LambdaReflection

   2) Obtain the Control Flow Graph (CFG) using the reflection interface obtained above.

   3) Get catalyst Expressions based on the basic blocks BB obtained in the CFG and simplify before replacing in the Spark Logical Plan.

   function

   the original Scala UDF provided by the user

4.  case class GpuScalaUDFLogical(udf: ScalaUDF) extends Expression with ShimExpression with Logging with Product with Serializable
5.  case class Instruction(opcode: Int, operand: Int, instructionStr: String) extends Logging with Product with Serializable

6.  class LambdaReflection extends AnyRef
7.  case class LogicalPlanRules() extends Rule[LogicalPlan] with Logging with Product with Serializable
8.  class Plugin extends (SparkSessionExtensions) ⇒ Unit with Logging
9.  case class State(locals: IndexedSeq[Expression], stack: List[Expression] = List(), cond: Expression = Literal.TrueLiteral, expr: Option[Expression] = None) extends Product with Serializable

   State is used as the main representation of block state, as we walk the bytecode.

   State is used as the main representation of block state, as we walk the bytecode.

   Given a set of instructions, we will use State variables to track what happens to the stack.

   The final State generated is later used to simplify expressions.

   Example 1: { return 0 }

   This is in java byte code: iconst 0 ireturn

   iconst 0 => pushes 0 to stack ireturn => pops and returns

   Example 2: { return 2 + 2 + 1 }

   1) State(locals, empty stack, no condition, expr?) // expr ==it is no expression

   iconst 2 NOTE: 2 is literal here so is 1 2) State(locals, 2::Nil, no condition, expr is still empty)

   iconst 2 3) State(locals, 2::2::Nil, no condiiton...)

   iadd (pop 2 and 2 + push 4 into stack) 4) State(locals, Add(2, 2)::Nil, no condition, expr is still empoty)

   iconst 1 5) State(locals, 1::Add(2,2)::Nil, ..)

   iadd (pop 1 and 4 + push 5 into sack) 6) Add(1, Add(2,2)) :: Nil

   ireturn (pop 5 from stack) 7) return Add...

   State == Add stack == lhs/rhs