Packages

c

org.apache.spark.sql.hive.execution

HiveOutputWriter

class HiveOutputWriter extends OutputWriter with HiveInspectors

Linear Supertypes
HiveInspectors, OutputWriter, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. HiveOutputWriter
  2. HiveInspectors
  3. OutputWriter
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new HiveOutputWriter(path: String, fileSinkConf: ShimFileSinkDesc, jobConf: JobConf, dataSchema: StructType)

Type Members

  1. implicit class typeInfoConversions extends AnyRef
    Definition Classes
    HiveInspectors

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  6. def close(): Unit
    Definition Classes
    HiveOutputWriter → OutputWriter
  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  11. def inspectorToDataType(inspector: ObjectInspector): DataType
    Definition Classes
    HiveInspectors
  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. def javaTypeToDataType(clz: Type): DataType
    Definition Classes
    HiveInspectors
  14. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  15. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  16. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  17. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  18. def toInspector(expr: Expression): ObjectInspector

    Map the catalyst expression to ObjectInspector, however, if the expression is Literal or foldable, a constant writable object inspector returns; Otherwise, we always get the object inspector according to its data type(in catalyst)

    Map the catalyst expression to ObjectInspector, however, if the expression is Literal or foldable, a constant writable object inspector returns; Otherwise, we always get the object inspector according to its data type(in catalyst)

    expr

    Catalyst expression to be mapped

    returns

    Hive java objectinspector (recursively).

    Definition Classes
    HiveInspectors
  19. def toInspector(dataType: DataType): ObjectInspector

    dataType

    Catalyst data type

    returns

    Hive java object inspector (recursively), not the Writable ObjectInspector We can easily map to the Hive built-in object inspector according to the data type.

    Definition Classes
    HiveInspectors
  20. def toString(): String
    Definition Classes
    AnyRef → Any
  21. def unwrapperFor(field: StructField): (Any, InternalRow, Int) ⇒ Unit

    Builds unwrappers ahead of time according to object inspector types to avoid pattern matching and branching costs per row.

    Builds unwrappers ahead of time according to object inspector types to avoid pattern matching and branching costs per row.

    field

    The HiveStructField to create an unwrapper for.

    returns

    A function that performs in-place updating of a MutableRow. Use the overloaded ObjectInspector version for assignments.

    Definition Classes
    HiveInspectors
  22. def unwrapperFor(objectInspector: ObjectInspector): (Any) ⇒ Any

    Builds unwrappers ahead of time according to object inspector types to avoid pattern matching and branching costs per row.

    Builds unwrappers ahead of time according to object inspector types to avoid pattern matching and branching costs per row.

    Strictly follows the following order in unwrapping (constant OI has the higher priority): Constant Null object inspector => return null Constant object inspector => extract the value from constant object inspector If object inspector prefers writable => extract writable from data and then get the catalyst type from the writable Extract the java object directly from the object inspector

    NOTICE: the complex data type requires recursive unwrapping.

    objectInspector

    the ObjectInspector used to create an unwrapper.

    returns

    A function that unwraps data objects. Use the overloaded HiveStructField version for in-place updating of a MutableRow.

    Definition Classes
    HiveInspectors
  23. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  24. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  25. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. def wrap(row: Seq[Any], wrappers: Array[(Any) ⇒ Any], cache: Array[AnyRef], dataTypes: Array[DataType]): Array[AnyRef]
    Definition Classes
    HiveInspectors
  27. def wrap(row: InternalRow, wrappers: Array[(Any) ⇒ Any], cache: Array[AnyRef], dataTypes: Array[DataType]): Array[AnyRef]
    Definition Classes
    HiveInspectors
  28. def wrap(a: Any, oi: ObjectInspector, dataType: DataType): AnyRef
    Definition Classes
    HiveInspectors
  29. def wrapperFor(oi: ObjectInspector, dataType: DataType): (Any) ⇒ Any

    Wraps with Hive types based on object inspector.

    Wraps with Hive types based on object inspector.

    Attributes
    protected
    Definition Classes
    HiveInspectors
  30. def write(row: InternalRow): Unit
    Definition Classes
    HiveOutputWriter → OutputWriter

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated @deprecated
    Deprecated

    (Since version ) see corresponding Javadoc for more information.

Inherited from HiveInspectors

Inherited from OutputWriter

Inherited from AnyRef

Inherited from Any

Ungrouped