Interface DataIntegrationFlowDatasetOptions.Builder

All Superinterfaces:
Buildable, CopyableBuilder<DataIntegrationFlowDatasetOptions.Builder,DataIntegrationFlowDatasetOptions>, SdkBuilder<DataIntegrationFlowDatasetOptions.Builder,DataIntegrationFlowDatasetOptions>, SdkPojo
Enclosing class:
DataIntegrationFlowDatasetOptions
@Mutable @NotThreadSafe public static interface DataIntegrationFlowDatasetOptions.Builder extends SdkPojo, CopyableBuilder<DataIntegrationFlowDatasetOptions.Builder,DataIntegrationFlowDatasetOptions>

  • Method Details

    • loadType

      DataIntegrationFlowDatasetOptions.Builder loadType(String loadType)

      The target dataset's data load type. This only affects how source S3 files are selected in the S3-to-dataset flow.

      • REPLACE - Target dataset will get replaced with the new file added under the source s3 prefix.

      • INCREMENTAL - Target dataset will get updated with the up-to-date content under S3 prefix incorporating any file additions or removals there.

      Parameters:
      loadType - The target dataset's data load type. This only affects how source S3 files are selected in the S3-to-dataset flow.

      • REPLACE - Target dataset will get replaced with the new file added under the source s3 prefix.

      • INCREMENTAL - Target dataset will get updated with the up-to-date content under S3 prefix incorporating any file additions or removals there.

      Returns:
      Returns a reference to this object so that method calls can be chained together.
      See Also:

    • loadType

      DataIntegrationFlowDatasetOptions.Builder loadType(DataIntegrationFlowLoadType loadType)

      The target dataset's data load type. This only affects how source S3 files are selected in the S3-to-dataset flow.

      • REPLACE - Target dataset will get replaced with the new file added under the source s3 prefix.

      • INCREMENTAL - Target dataset will get updated with the up-to-date content under S3 prefix incorporating any file additions or removals there.

      Parameters:
      loadType - The target dataset's data load type. This only affects how source S3 files are selected in the S3-to-dataset flow.

      • REPLACE - Target dataset will get replaced with the new file added under the source s3 prefix.

      • INCREMENTAL - Target dataset will get updated with the up-to-date content under S3 prefix incorporating any file additions or removals there.

      Returns:
      Returns a reference to this object so that method calls can be chained together.
      See Also:

    • dedupeRecords

      DataIntegrationFlowDatasetOptions.Builder dedupeRecords(Boolean dedupeRecords)

      The option to perform deduplication on data records sharing same primary key values. If disabled, transformed data with duplicate primary key values will ingest into dataset, for datasets within asc namespace, such duplicates will cause ingestion fail. If enabled without dedupeStrategy, deduplication is done by retaining a random data record among those sharing the same primary key values. If enabled with dedupeStragtegy, the deduplication is done following the strategy.

      Note that target dataset may have partition configured, when dedupe is enabled, it only dedupe against primary keys and retain only one record out of those duplicates regardless of its partition status.

      Parameters:
      dedupeRecords - The option to perform deduplication on data records sharing same primary key values. If disabled, transformed data with duplicate primary key values will ingest into dataset, for datasets within asc namespace, such duplicates will cause ingestion fail. If enabled without dedupeStrategy, deduplication is done by retaining a random data record among those sharing the same primary key values. If enabled with dedupeStragtegy, the deduplication is done following the strategy.

      Note that target dataset may have partition configured, when dedupe is enabled, it only dedupe against primary keys and retain only one record out of those duplicates regardless of its partition status.

      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • dedupeStrategy

      DataIntegrationFlowDatasetOptions.Builder dedupeStrategy(DataIntegrationFlowDedupeStrategy dedupeStrategy)

      The deduplication strategy to dedupe the data records sharing same primary key values of the target dataset. This strategy only applies to target dataset with primary keys and with dedupeRecords option enabled. If transformed data still got duplicates after the dedupeStrategy evaluation, a random data record is chosen to be retained.

      Parameters:
      dedupeStrategy - The deduplication strategy to dedupe the data records sharing same primary key values of the target dataset. This strategy only applies to target dataset with primary keys and with dedupeRecords option enabled. If transformed data still got duplicates after the dedupeStrategy evaluation, a random data record is chosen to be retained.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • dedupeStrategy

      default DataIntegrationFlowDatasetOptions.Builder dedupeStrategy(Consumer<DataIntegrationFlowDedupeStrategy.Builder> dedupeStrategy)

      The deduplication strategy to dedupe the data records sharing same primary key values of the target dataset. This strategy only applies to target dataset with primary keys and with dedupeRecords option enabled. If transformed data still got duplicates after the dedupeStrategy evaluation, a random data record is chosen to be retained.

      This is a convenience method that creates an instance of the DataIntegrationFlowDedupeStrategy.Builder avoiding the need to create one manually via DataIntegrationFlowDedupeStrategy.builder().

      When the Consumer completes, SdkBuilder.build() is called immediately and its result is passed to dedupeStrategy(DataIntegrationFlowDedupeStrategy).

      Parameters:
      dedupeStrategy - a consumer that will call methods on DataIntegrationFlowDedupeStrategy.Builder
      Returns:
      Returns a reference to this object so that method calls can be chained together.
      See Also: