AWS SDK for C++  1.8.95
AWS SDK for C++
Public Member Functions | List of all members
Aws::Firehose::Model::OrcSerDe Class Reference

#include <OrcSerDe.h>

Public Member Functions

 OrcSerDe ()
 
 OrcSerDe (Aws::Utils::Json::JsonView jsonValue)
 
OrcSerDeoperator= (Aws::Utils::Json::JsonView jsonValue)
 
Aws::Utils::Json::JsonValue Jsonize () const
 
int GetStripeSizeBytes () const
 
bool StripeSizeBytesHasBeenSet () const
 
void SetStripeSizeBytes (int value)
 
OrcSerDeWithStripeSizeBytes (int value)
 
int GetBlockSizeBytes () const
 
bool BlockSizeBytesHasBeenSet () const
 
void SetBlockSizeBytes (int value)
 
OrcSerDeWithBlockSizeBytes (int value)
 
int GetRowIndexStride () const
 
bool RowIndexStrideHasBeenSet () const
 
void SetRowIndexStride (int value)
 
OrcSerDeWithRowIndexStride (int value)
 
bool GetEnablePadding () const
 
bool EnablePaddingHasBeenSet () const
 
void SetEnablePadding (bool value)
 
OrcSerDeWithEnablePadding (bool value)
 
double GetPaddingTolerance () const
 
bool PaddingToleranceHasBeenSet () const
 
void SetPaddingTolerance (double value)
 
OrcSerDeWithPaddingTolerance (double value)
 
const OrcCompressionGetCompression () const
 
bool CompressionHasBeenSet () const
 
void SetCompression (const OrcCompression &value)
 
void SetCompression (OrcCompression &&value)
 
OrcSerDeWithCompression (const OrcCompression &value)
 
OrcSerDeWithCompression (OrcCompression &&value)
 
const Aws::Vector< Aws::String > & GetBloomFilterColumns () const
 
bool BloomFilterColumnsHasBeenSet () const
 
void SetBloomFilterColumns (const Aws::Vector< Aws::String > &value)
 
void SetBloomFilterColumns (Aws::Vector< Aws::String > &&value)
 
OrcSerDeWithBloomFilterColumns (const Aws::Vector< Aws::String > &value)
 
OrcSerDeWithBloomFilterColumns (Aws::Vector< Aws::String > &&value)
 
OrcSerDeAddBloomFilterColumns (const Aws::String &value)
 
OrcSerDeAddBloomFilterColumns (Aws::String &&value)
 
OrcSerDeAddBloomFilterColumns (const char *value)
 
double GetBloomFilterFalsePositiveProbability () const
 
bool BloomFilterFalsePositiveProbabilityHasBeenSet () const
 
void SetBloomFilterFalsePositiveProbability (double value)
 
OrcSerDeWithBloomFilterFalsePositiveProbability (double value)
 
double GetDictionaryKeyThreshold () const
 
bool DictionaryKeyThresholdHasBeenSet () const
 
void SetDictionaryKeyThreshold (double value)
 
OrcSerDeWithDictionaryKeyThreshold (double value)
 
const OrcFormatVersionGetFormatVersion () const
 
bool FormatVersionHasBeenSet () const
 
void SetFormatVersion (const OrcFormatVersion &value)
 
void SetFormatVersion (OrcFormatVersion &&value)
 
OrcSerDeWithFormatVersion (const OrcFormatVersion &value)
 
OrcSerDeWithFormatVersion (OrcFormatVersion &&value)
 

Detailed Description

A serializer to use for converting data to the ORC format before storing it in Amazon S3. For more information, see Apache ORC.

See Also:

AWS API Reference

Definition at line 37 of file OrcSerDe.h.

Constructor & Destructor Documentation

◆ OrcSerDe() [1/2]

Aws::Firehose::Model::OrcSerDe::OrcSerDe ( )

◆ OrcSerDe() [2/2]

Aws::Firehose::Model::OrcSerDe::OrcSerDe ( Aws::Utils::Json::JsonView  jsonValue)

Member Function Documentation

◆ AddBloomFilterColumns() [1/3]

OrcSerDe& Aws::Firehose::Model::OrcSerDe::AddBloomFilterColumns ( const Aws::String value)
inline

The column names for which you want Kinesis Data Firehose to create bloom filters. The default is null.

Definition at line 292 of file OrcSerDe.h.

◆ AddBloomFilterColumns() [2/3]

OrcSerDe& Aws::Firehose::Model::OrcSerDe::AddBloomFilterColumns ( Aws::String &&  value)
inline

The column names for which you want Kinesis Data Firehose to create bloom filters. The default is null.

Definition at line 298 of file OrcSerDe.h.

◆ AddBloomFilterColumns() [3/3]

OrcSerDe& Aws::Firehose::Model::OrcSerDe::AddBloomFilterColumns ( const char *  value)
inline

The column names for which you want Kinesis Data Firehose to create bloom filters. The default is null.

Definition at line 304 of file OrcSerDe.h.

◆ BlockSizeBytesHasBeenSet()

bool Aws::Firehose::Model::OrcSerDe::BlockSizeBytesHasBeenSet ( ) const
inline

The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

Definition at line 85 of file OrcSerDe.h.

◆ BloomFilterColumnsHasBeenSet()

bool Aws::Firehose::Model::OrcSerDe::BloomFilterColumnsHasBeenSet ( ) const
inline

The column names for which you want Kinesis Data Firehose to create bloom filters. The default is null.

Definition at line 262 of file OrcSerDe.h.

◆ BloomFilterFalsePositiveProbabilityHasBeenSet()

bool Aws::Firehose::Model::OrcSerDe::BloomFilterFalsePositiveProbabilityHasBeenSet ( ) const
inline

The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.

Definition at line 319 of file OrcSerDe.h.

◆ CompressionHasBeenSet()

bool Aws::Firehose::Model::OrcSerDe::CompressionHasBeenSet ( ) const
inline

The compression code to use over data blocks. The default is SNAPPY.

Definition at line 225 of file OrcSerDe.h.

◆ DictionaryKeyThresholdHasBeenSet()

bool Aws::Firehose::Model::OrcSerDe::DictionaryKeyThresholdHasBeenSet ( ) const
inline

Represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.

Definition at line 350 of file OrcSerDe.h.

◆ EnablePaddingHasBeenSet()

bool Aws::Firehose::Model::OrcSerDe::EnablePaddingHasBeenSet ( ) const
inline

Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.

Definition at line 141 of file OrcSerDe.h.

◆ FormatVersionHasBeenSet()

bool Aws::Firehose::Model::OrcSerDe::FormatVersionHasBeenSet ( ) const
inline

The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.

Definition at line 379 of file OrcSerDe.h.

◆ GetBlockSizeBytes()

int Aws::Firehose::Model::OrcSerDe::GetBlockSizeBytes ( ) const
inline

The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

Definition at line 77 of file OrcSerDe.h.

◆ GetBloomFilterColumns()

const Aws::Vector<Aws::String>& Aws::Firehose::Model::OrcSerDe::GetBloomFilterColumns ( ) const
inline

The column names for which you want Kinesis Data Firehose to create bloom filters. The default is null.

Definition at line 256 of file OrcSerDe.h.

◆ GetBloomFilterFalsePositiveProbability()

double Aws::Firehose::Model::OrcSerDe::GetBloomFilterFalsePositiveProbability ( ) const
inline

The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.

Definition at line 312 of file OrcSerDe.h.

◆ GetCompression()

const OrcCompression& Aws::Firehose::Model::OrcSerDe::GetCompression ( ) const
inline

The compression code to use over data blocks. The default is SNAPPY.

Definition at line 219 of file OrcSerDe.h.

◆ GetDictionaryKeyThreshold()

double Aws::Firehose::Model::OrcSerDe::GetDictionaryKeyThreshold ( ) const
inline

Represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.

Definition at line 342 of file OrcSerDe.h.

◆ GetEnablePadding()

bool Aws::Firehose::Model::OrcSerDe::GetEnablePadding ( ) const
inline

Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.

Definition at line 134 of file OrcSerDe.h.

◆ GetFormatVersion()

const OrcFormatVersion& Aws::Firehose::Model::OrcSerDe::GetFormatVersion ( ) const
inline

The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.

Definition at line 373 of file OrcSerDe.h.

◆ GetPaddingTolerance()

double Aws::Firehose::Model::OrcSerDe::GetPaddingTolerance ( ) const
inline

A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size.

For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task.

Kinesis Data Firehose ignores this parameter when OrcSerDe$EnablePadding is false.

Definition at line 170 of file OrcSerDe.h.

◆ GetRowIndexStride()

int Aws::Firehose::Model::OrcSerDe::GetRowIndexStride ( ) const
inline

The number of rows between index entries. The default is 10,000 and the minimum is 1,000.

Definition at line 108 of file OrcSerDe.h.

◆ GetStripeSizeBytes()

int Aws::Firehose::Model::OrcSerDe::GetStripeSizeBytes ( ) const
inline

The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.

Definition at line 50 of file OrcSerDe.h.

◆ Jsonize()

Aws::Utils::Json::JsonValue Aws::Firehose::Model::OrcSerDe::Jsonize ( ) const

◆ operator=()

OrcSerDe& Aws::Firehose::Model::OrcSerDe::operator= ( Aws::Utils::Json::JsonView  jsonValue)

◆ PaddingToleranceHasBeenSet()

bool Aws::Firehose::Model::OrcSerDe::PaddingToleranceHasBeenSet ( ) const
inline

A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size.

For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task.

Kinesis Data Firehose ignores this parameter when OrcSerDe$EnablePadding is false.

Definition at line 184 of file OrcSerDe.h.

◆ RowIndexStrideHasBeenSet()

bool Aws::Firehose::Model::OrcSerDe::RowIndexStrideHasBeenSet ( ) const
inline

The number of rows between index entries. The default is 10,000 and the minimum is 1,000.

Definition at line 114 of file OrcSerDe.h.

◆ SetBlockSizeBytes()

void Aws::Firehose::Model::OrcSerDe::SetBlockSizeBytes ( int  value)
inline

The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

Definition at line 93 of file OrcSerDe.h.

◆ SetBloomFilterColumns() [1/2]

void Aws::Firehose::Model::OrcSerDe::SetBloomFilterColumns ( const Aws::Vector< Aws::String > &  value)
inline

The column names for which you want Kinesis Data Firehose to create bloom filters. The default is null.

Definition at line 268 of file OrcSerDe.h.

◆ SetBloomFilterColumns() [2/2]

void Aws::Firehose::Model::OrcSerDe::SetBloomFilterColumns ( Aws::Vector< Aws::String > &&  value)
inline

The column names for which you want Kinesis Data Firehose to create bloom filters. The default is null.

Definition at line 274 of file OrcSerDe.h.

◆ SetBloomFilterFalsePositiveProbability()

void Aws::Firehose::Model::OrcSerDe::SetBloomFilterFalsePositiveProbability ( double  value)
inline

The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.

Definition at line 326 of file OrcSerDe.h.

◆ SetCompression() [1/2]

void Aws::Firehose::Model::OrcSerDe::SetCompression ( const OrcCompression value)
inline

The compression code to use over data blocks. The default is SNAPPY.

Definition at line 231 of file OrcSerDe.h.

◆ SetCompression() [2/2]

void Aws::Firehose::Model::OrcSerDe::SetCompression ( OrcCompression &&  value)
inline

The compression code to use over data blocks. The default is SNAPPY.

Definition at line 237 of file OrcSerDe.h.

◆ SetDictionaryKeyThreshold()

void Aws::Firehose::Model::OrcSerDe::SetDictionaryKeyThreshold ( double  value)
inline

Represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.

Definition at line 358 of file OrcSerDe.h.

◆ SetEnablePadding()

void Aws::Firehose::Model::OrcSerDe::SetEnablePadding ( bool  value)
inline

Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.

Definition at line 148 of file OrcSerDe.h.

◆ SetFormatVersion() [1/2]

void Aws::Firehose::Model::OrcSerDe::SetFormatVersion ( const OrcFormatVersion value)
inline

The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.

Definition at line 385 of file OrcSerDe.h.

◆ SetFormatVersion() [2/2]

void Aws::Firehose::Model::OrcSerDe::SetFormatVersion ( OrcFormatVersion &&  value)
inline

The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.

Definition at line 391 of file OrcSerDe.h.

◆ SetPaddingTolerance()

void Aws::Firehose::Model::OrcSerDe::SetPaddingTolerance ( double  value)
inline

A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size.

For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task.

Kinesis Data Firehose ignores this parameter when OrcSerDe$EnablePadding is false.

Definition at line 198 of file OrcSerDe.h.

◆ SetRowIndexStride()

void Aws::Firehose::Model::OrcSerDe::SetRowIndexStride ( int  value)
inline

The number of rows between index entries. The default is 10,000 and the minimum is 1,000.

Definition at line 120 of file OrcSerDe.h.

◆ SetStripeSizeBytes()

void Aws::Firehose::Model::OrcSerDe::SetStripeSizeBytes ( int  value)
inline

The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.

Definition at line 62 of file OrcSerDe.h.

◆ StripeSizeBytesHasBeenSet()

bool Aws::Firehose::Model::OrcSerDe::StripeSizeBytesHasBeenSet ( ) const
inline

The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.

Definition at line 56 of file OrcSerDe.h.

◆ WithBlockSizeBytes()

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithBlockSizeBytes ( int  value)
inline

The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Kinesis Data Firehose uses this value for padding calculations.

Definition at line 101 of file OrcSerDe.h.

◆ WithBloomFilterColumns() [1/2]

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithBloomFilterColumns ( const Aws::Vector< Aws::String > &  value)
inline

The column names for which you want Kinesis Data Firehose to create bloom filters. The default is null.

Definition at line 280 of file OrcSerDe.h.

◆ WithBloomFilterColumns() [2/2]

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithBloomFilterColumns ( Aws::Vector< Aws::String > &&  value)
inline

The column names for which you want Kinesis Data Firehose to create bloom filters. The default is null.

Definition at line 286 of file OrcSerDe.h.

◆ WithBloomFilterFalsePositiveProbability()

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithBloomFilterFalsePositiveProbability ( double  value)
inline

The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. The default value is 0.05, the minimum is 0, and the maximum is 1.

Definition at line 333 of file OrcSerDe.h.

◆ WithCompression() [1/2]

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithCompression ( const OrcCompression value)
inline

The compression code to use over data blocks. The default is SNAPPY.

Definition at line 243 of file OrcSerDe.h.

◆ WithCompression() [2/2]

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithCompression ( OrcCompression &&  value)
inline

The compression code to use over data blocks. The default is SNAPPY.

Definition at line 249 of file OrcSerDe.h.

◆ WithDictionaryKeyThreshold()

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithDictionaryKeyThreshold ( double  value)
inline

Represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1.

Definition at line 366 of file OrcSerDe.h.

◆ WithEnablePadding()

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithEnablePadding ( bool  value)
inline

Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is false.

Definition at line 155 of file OrcSerDe.h.

◆ WithFormatVersion() [1/2]

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithFormatVersion ( const OrcFormatVersion value)
inline

The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.

Definition at line 397 of file OrcSerDe.h.

◆ WithFormatVersion() [2/2]

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithFormatVersion ( OrcFormatVersion &&  value)
inline

The version of the file to write. The possible values are V0_11 and V0_12. The default is V0_12.

Definition at line 403 of file OrcSerDe.h.

◆ WithPaddingTolerance()

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithPaddingTolerance ( double  value)
inline

A number between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. The default value is 0.05, which means 5 percent of stripe size.

For the default values of 64 MiB ORC stripes and 256 MiB HDFS blocks, the default block padding tolerance of 5 percent reserves a maximum of 3.2 MiB for padding within the 256 MiB block. In such a case, if the available size within the block is more than 3.2 MiB, a new, smaller stripe is inserted to fit within that space. This ensures that no stripe crosses block boundaries and causes remote reads within a node-local task.

Kinesis Data Firehose ignores this parameter when OrcSerDe$EnablePadding is false.

Definition at line 212 of file OrcSerDe.h.

◆ WithRowIndexStride()

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithRowIndexStride ( int  value)
inline

The number of rows between index entries. The default is 10,000 and the minimum is 1,000.

Definition at line 126 of file OrcSerDe.h.

◆ WithStripeSizeBytes()

OrcSerDe& Aws::Firehose::Model::OrcSerDe::WithStripeSizeBytes ( int  value)
inline

The number of bytes in each stripe. The default is 64 MiB and the minimum is 8 MiB.

Definition at line 68 of file OrcSerDe.h.


The documentation for this class was generated from the following file: