software.amazon.awssdk.services.firehose.model

Class OrcSerDe

java.lang.Object

software.amazon.awssdk.services.firehose.model.OrcSerDe

All Implemented Interfaces:

Serializable, SdkPojo, ToCopyableBuilder<OrcSerDe.Builder,OrcSerDe>

@Generated(value="software.amazon.awssdk:codegen")
public final class OrcSerDe
extends Object
implements SdkPojo, Serializable, ToCopyableBuilder<OrcSerDe.Builder,OrcSerDe>

A serializer to use for converting data to the ORC format before storing it in Amazon S3. For more information, see Apache ORC.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	OrcSerDe.Builder

Method Summary

Modifier and Type	Method and Description
Integer	blockSizeBytes() The Hadoop Distributed File System (HDFS) block size.
List<String>	bloomFilterColumns() The column names for which you want Kinesis Data Firehose to create bloom filters.
Double	bloomFilterFalsePositiveProbability() The Bloom filter false positive probability (FPP).
static OrcSerDe.Builder	builder()
OrcCompression	compression() The compression code to use over data blocks.
String	compressionAsString() The compression code to use over data blocks.
Double	dictionaryKeyThreshold() Represents the fraction of the total number of non-null rows.
Boolean	enablePadding() Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries.
boolean	equals(Object obj)
OrcFormatVersion	formatVersion() The version of the file to write.
String	formatVersionAsString() The version of the file to write.
<T> Optional<T>	getValueForField(String fieldName, Class<T> clazz)
int	hashCode()
Double	paddingTolerance() A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size.
Integer	rowIndexStride() The number of rows between index entries.
List<SdkField<?>>	sdkFields()
static Class<? extends OrcSerDe.Builder>	serializableBuilderClass()
Integer	stripeSizeBytes() The number of bytes in each stripe.
OrcSerDe.Builder	toBuilder() Take this object and create a builder that contains all of the current property values of this object.
String	toString()

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface software.amazon.awssdk.utils.builder.ToCopyableBuilder

copy

Method Detail

stripeSizeBytes

public Integer stripeSizeBytes()

The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.

Returns:

The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.

blockSizeBytes

public Integer blockSizeBytes()

The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

Returns:

The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

rowIndexStride

public Integer rowIndexStride()

The number of rows between index entries. The default is 10,000 and the minimum is 1,000.

Returns:

The number of rows between index entries. The default is 10,000 and the minimum is 1,000.

enablePadding

public Boolean enablePadding()

Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.

Returns:

Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.

paddingTolerance

public Double paddingTolerance()

A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size.

For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task.

Kinesis Data Firehose ignores this parameter when OrcSerDe$EnablePadding is false.

Returns:

A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size.

For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task.

Kinesis Data Firehose ignores this parameter when OrcSerDe$EnablePadding is false.

compression

public OrcCompression compression()

The compression code to use over data blocks. The default is SNAPPY.

If the service returns an enum value that is not available in the current SDK version, compression will return OrcCompression.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available from compressionAsString().

Returns:

The compression code to use over data blocks. The default is SNAPPY.

See Also:

OrcCompression

compressionAsString

public String compressionAsString()

The compression code to use over data blocks. The default is SNAPPY.

If the service returns an enum value that is not available in the current SDK version, compression will return OrcCompression.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available from compressionAsString().

Returns:

The compression code to use over data blocks. The default is SNAPPY.

See Also:

OrcCompression

bloomFilterColumns

public List<String> bloomFilterColumns()

The column names for which you want Kinesis Data Firehose to create bloom filters. The default is null.

Attempts to modify the collection returned by this method will result in an UnsupportedOperationException.

Returns:

The column names for which you want Kinesis Data Firehose to create bloom filters. The default is null.

bloomFilterFalsePositiveProbability

public Double bloomFilterFalsePositiveProbability()

The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.

Returns:

The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.

dictionaryKeyThreshold

public Double dictionaryKeyThreshold()

Represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.

Returns:

Represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.

formatVersion

public OrcFormatVersion formatVersion()

The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.

If the service returns an enum value that is not available in the current SDK version, formatVersion will return OrcFormatVersion.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available from formatVersionAsString().

Returns:

The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.

See Also:

OrcFormatVersion

formatVersionAsString

public String formatVersionAsString()

The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.

If the service returns an enum value that is not available in the current SDK version, formatVersion will return OrcFormatVersion.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available from formatVersionAsString().

Returns:

The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.

See Also:

OrcFormatVersion

toBuilder

public OrcSerDe.Builder toBuilder()

Description copied from interface: ToCopyableBuilder

Take this object and create a builder that contains all of the current property values of this object.

Specified by:

toBuilder in interface ToCopyableBuilder<OrcSerDe.Builder,OrcSerDe>

Returns:

a builder for type T

builder

public static OrcSerDe.Builder builder()

serializableBuilderClass

public static Class<? extends OrcSerDe.Builder> serializableBuilderClass()

hashCode

public int hashCode()

Overrides:

hashCode in class Object

equals

public boolean equals(Object obj)

Overrides:

equals in class Object

toString

public String toString()

Overrides:

toString in class Object

getValueForField

public <T> Optional<T> getValueForField(String fieldName,
                                        Class<T> clazz)

sdkFields

public List<SdkField<?>> sdkFields()

Specified by:

sdkFields in interface SdkPojo

Returns:

List of SdkField in this POJO. May be empty list but should never be null.