AWS SDK for C++  1.9.104
AWS SDK for C++
Public Member Functions | List of all members
Aws::SageMaker::Model::S3DataSource Class Reference

#include <S3DataSource.h>

Public Member Functions

 S3DataSource ()
 
 S3DataSource (Aws::Utils::Json::JsonView jsonValue)
 
S3DataSourceoperator= (Aws::Utils::Json::JsonView jsonValue)
 
Aws::Utils::Json::JsonValue Jsonize () const
 
const S3DataTypeGetS3DataType () const
 
bool S3DataTypeHasBeenSet () const
 
void SetS3DataType (const S3DataType &value)
 
void SetS3DataType (S3DataType &&value)
 
S3DataSourceWithS3DataType (const S3DataType &value)
 
S3DataSourceWithS3DataType (S3DataType &&value)
 
const Aws::StringGetS3Uri () const
 
bool S3UriHasBeenSet () const
 
void SetS3Uri (const Aws::String &value)
 
void SetS3Uri (Aws::String &&value)
 
void SetS3Uri (const char *value)
 
S3DataSourceWithS3Uri (const Aws::String &value)
 
S3DataSourceWithS3Uri (Aws::String &&value)
 
S3DataSourceWithS3Uri (const char *value)
 
const S3DataDistributionGetS3DataDistributionType () const
 
bool S3DataDistributionTypeHasBeenSet () const
 
void SetS3DataDistributionType (const S3DataDistribution &value)
 
void SetS3DataDistributionType (S3DataDistribution &&value)
 
S3DataSourceWithS3DataDistributionType (const S3DataDistribution &value)
 
S3DataSourceWithS3DataDistributionType (S3DataDistribution &&value)
 
const Aws::Vector< Aws::String > & GetAttributeNames () const
 
bool AttributeNamesHasBeenSet () const
 
void SetAttributeNames (const Aws::Vector< Aws::String > &value)
 
void SetAttributeNames (Aws::Vector< Aws::String > &&value)
 
S3DataSourceWithAttributeNames (const Aws::Vector< Aws::String > &value)
 
S3DataSourceWithAttributeNames (Aws::Vector< Aws::String > &&value)
 
S3DataSourceAddAttributeNames (const Aws::String &value)
 
S3DataSourceAddAttributeNames (Aws::String &&value)
 
S3DataSourceAddAttributeNames (const char *value)
 

Detailed Description

Describes the S3 data source.

See Also:

AWS API Reference

Definition at line 34 of file S3DataSource.h.

Constructor & Destructor Documentation

◆ S3DataSource() [1/2]

Aws::SageMaker::Model::S3DataSource::S3DataSource ( )

◆ S3DataSource() [2/2]

Aws::SageMaker::Model::S3DataSource::S3DataSource ( Aws::Utils::Json::JsonView  jsonValue)

Member Function Documentation

◆ AddAttributeNames() [1/3]

S3DataSource& Aws::SageMaker::Model::S3DataSource::AddAttributeNames ( Aws::String &&  value)
inline

A list of one or more attribute names to use that are found in a specified augmented manifest file.

Definition at line 528 of file S3DataSource.h.

◆ AddAttributeNames() [2/3]

S3DataSource& Aws::SageMaker::Model::S3DataSource::AddAttributeNames ( const Aws::String value)
inline

A list of one or more attribute names to use that are found in a specified augmented manifest file.

Definition at line 522 of file S3DataSource.h.

◆ AddAttributeNames() [3/3]

S3DataSource& Aws::SageMaker::Model::S3DataSource::AddAttributeNames ( const char *  value)
inline

A list of one or more attribute names to use that are found in a specified augmented manifest file.

Definition at line 534 of file S3DataSource.h.

◆ AttributeNamesHasBeenSet()

bool Aws::SageMaker::Model::S3DataSource::AttributeNamesHasBeenSet ( ) const
inline

A list of one or more attribute names to use that are found in a specified augmented manifest file.

Definition at line 492 of file S3DataSource.h.

◆ GetAttributeNames()

const Aws::Vector<Aws::String>& Aws::SageMaker::Model::S3DataSource::GetAttributeNames ( ) const
inline

A list of one or more attribute names to use that are found in a specified augmented manifest file.

Definition at line 486 of file S3DataSource.h.

◆ GetS3DataDistributionType()

const S3DataDistribution& Aws::SageMaker::Model::S3DataSource::GetS3DataDistributionType ( ) const
inline

If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated.

If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key. If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.

Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipe modes. Keep this in mind when developing algorithms.

In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key. If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File), this copies 1/n of the number of objects.

Definition at line 379 of file S3DataSource.h.

◆ GetS3DataType()

const S3DataType& Aws::SageMaker::Model::S3DataSource::GetS3DataType ( ) const
inline

If you choose S3Prefix, S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.

If you choose ManifestFile, S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.

If you choose AugmentedManifestFile, S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe.

Definition at line 55 of file S3DataSource.h.

◆ GetS3Uri()

const Aws::String& Aws::SageMaker::Model::S3DataSource::GetS3Uri ( ) const
inline

Depending on the value specified for the S3DataType, identifies either a key name prefix or a manifest. For example:

  • A key name prefix might look like this: s3://bucketname/exampleprefix

  • A manifest might look like this: s3://bucketname/example.manifest

    A manifest is an S3 object which is a JSON file consisting of an array of elements. The first element is a prefix which is followed by one or more suffixes. SageMaker appends the suffix elements to the prefix to get a full set of S3Uri. Note that the prefix must be a valid non-empty S3Uri that precludes users from specifying a manifest whose individual S3Uri is sourced from different S3 buckets.

    The following code example shows a valid manifest format:

    [ {"prefix": "s3://customer_bucket/some/prefix/"},

    "relative/path/to/custdata-1",

    "relative/path/custdata-2",

    ...

    "relative/path/custdata-N"

    ]

    This JSON is equivalent to the following S3Uri list:

    s3://customer_bucket/some/prefix/relative/path/to/custdata-1

    s3://customer_bucket/some/prefix/relative/path/custdata-2

    ...

    s3://customer_bucket/some/prefix/relative/path/custdata-N

    The complete set of S3Uri in this manifest is the input data for the channel for this data source. The object that each S3Uri points to must be readable by the IAM role that Amazon SageMaker uses to perform tasks on your behalf.

Definition at line 155 of file S3DataSource.h.

◆ Jsonize()

Aws::Utils::Json::JsonValue Aws::SageMaker::Model::S3DataSource::Jsonize ( ) const

◆ operator=()

S3DataSource& Aws::SageMaker::Model::S3DataSource::operator= ( Aws::Utils::Json::JsonView  jsonValue)

◆ S3DataDistributionTypeHasBeenSet()

bool Aws::SageMaker::Model::S3DataSource::S3DataDistributionTypeHasBeenSet ( ) const
inline

If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated.

If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key. If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.

Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipe modes. Keep this in mind when developing algorithms.

In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key. If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File), this copies 1/n of the number of objects.

Definition at line 399 of file S3DataSource.h.

◆ S3DataTypeHasBeenSet()

bool Aws::SageMaker::Model::S3DataSource::S3DataTypeHasBeenSet ( ) const
inline

If you choose S3Prefix, S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.

If you choose ManifestFile, S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.

If you choose AugmentedManifestFile, S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe.

Definition at line 69 of file S3DataSource.h.

◆ S3UriHasBeenSet()

bool Aws::SageMaker::Model::S3DataSource::S3UriHasBeenSet ( ) const
inline

Depending on the value specified for the S3DataType, identifies either a key name prefix or a manifest. For example:

  • A key name prefix might look like this: s3://bucketname/exampleprefix

  • A manifest might look like this: s3://bucketname/example.manifest

    A manifest is an S3 object which is a JSON file consisting of an array of elements. The first element is a prefix which is followed by one or more suffixes. SageMaker appends the suffix elements to the prefix to get a full set of S3Uri. Note that the prefix must be a valid non-empty S3Uri that precludes users from specifying a manifest whose individual S3Uri is sourced from different S3 buckets.

    The following code example shows a valid manifest format:

    [ {"prefix": "s3://customer_bucket/some/prefix/"},

    "relative/path/to/custdata-1",

    "relative/path/custdata-2",

    ...

    "relative/path/custdata-N"

    ]

    This JSON is equivalent to the following S3Uri list:

    s3://customer_bucket/some/prefix/relative/path/to/custdata-1

    s3://customer_bucket/some/prefix/relative/path/custdata-2

    ...

    s3://customer_bucket/some/prefix/relative/path/custdata-N

    The complete set of S3Uri in this manifest is the input data for the channel for this data source. The object that each S3Uri points to must be readable by the IAM role that Amazon SageMaker uses to perform tasks on your behalf.

Definition at line 184 of file S3DataSource.h.

◆ SetAttributeNames() [1/2]

void Aws::SageMaker::Model::S3DataSource::SetAttributeNames ( Aws::Vector< Aws::String > &&  value)
inline

A list of one or more attribute names to use that are found in a specified augmented manifest file.

Definition at line 504 of file S3DataSource.h.

◆ SetAttributeNames() [2/2]

void Aws::SageMaker::Model::S3DataSource::SetAttributeNames ( const Aws::Vector< Aws::String > &  value)
inline

A list of one or more attribute names to use that are found in a specified augmented manifest file.

Definition at line 498 of file S3DataSource.h.

◆ SetS3DataDistributionType() [1/2]

void Aws::SageMaker::Model::S3DataSource::SetS3DataDistributionType ( const S3DataDistribution value)
inline

If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated.

If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key. If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.

Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipe modes. Keep this in mind when developing algorithms.

In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key. If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File), this copies 1/n of the number of objects.

Definition at line 419 of file S3DataSource.h.

◆ SetS3DataDistributionType() [2/2]

void Aws::SageMaker::Model::S3DataSource::SetS3DataDistributionType ( S3DataDistribution &&  value)
inline

If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated.

If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key. If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.

Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipe modes. Keep this in mind when developing algorithms.

In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key. If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File), this copies 1/n of the number of objects.

Definition at line 439 of file S3DataSource.h.

◆ SetS3DataType() [1/2]

void Aws::SageMaker::Model::S3DataSource::SetS3DataType ( const S3DataType value)
inline

If you choose S3Prefix, S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.

If you choose ManifestFile, S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.

If you choose AugmentedManifestFile, S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe.

Definition at line 83 of file S3DataSource.h.

◆ SetS3DataType() [2/2]

void Aws::SageMaker::Model::S3DataSource::SetS3DataType ( S3DataType &&  value)
inline

If you choose S3Prefix, S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.

If you choose ManifestFile, S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.

If you choose AugmentedManifestFile, S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe.

Definition at line 97 of file S3DataSource.h.

◆ SetS3Uri() [1/3]

void Aws::SageMaker::Model::S3DataSource::SetS3Uri ( Aws::String &&  value)
inline

Depending on the value specified for the S3DataType, identifies either a key name prefix or a manifest. For example:

  • A key name prefix might look like this: s3://bucketname/exampleprefix

  • A manifest might look like this: s3://bucketname/example.manifest

    A manifest is an S3 object which is a JSON file consisting of an array of elements. The first element is a prefix which is followed by one or more suffixes. SageMaker appends the suffix elements to the prefix to get a full set of S3Uri. Note that the prefix must be a valid non-empty S3Uri that precludes users from specifying a manifest whose individual S3Uri is sourced from different S3 buckets.

    The following code example shows a valid manifest format:

    [ {"prefix": "s3://customer_bucket/some/prefix/"},

    "relative/path/to/custdata-1",

    "relative/path/custdata-2",

    ...

    "relative/path/custdata-N"

    ]

    This JSON is equivalent to the following S3Uri list:

    s3://customer_bucket/some/prefix/relative/path/to/custdata-1

    s3://customer_bucket/some/prefix/relative/path/custdata-2

    ...

    s3://customer_bucket/some/prefix/relative/path/custdata-N

    The complete set of S3Uri in this manifest is the input data for the channel for this data source. The object that each S3Uri points to must be readable by the IAM role that Amazon SageMaker uses to perform tasks on your behalf.

Definition at line 242 of file S3DataSource.h.

◆ SetS3Uri() [2/3]

void Aws::SageMaker::Model::S3DataSource::SetS3Uri ( const Aws::String value)
inline

Depending on the value specified for the S3DataType, identifies either a key name prefix or a manifest. For example:

  • A key name prefix might look like this: s3://bucketname/exampleprefix

  • A manifest might look like this: s3://bucketname/example.manifest

    A manifest is an S3 object which is a JSON file consisting of an array of elements. The first element is a prefix which is followed by one or more suffixes. SageMaker appends the suffix elements to the prefix to get a full set of S3Uri. Note that the prefix must be a valid non-empty S3Uri that precludes users from specifying a manifest whose individual S3Uri is sourced from different S3 buckets.

    The following code example shows a valid manifest format:

    [ {"prefix": "s3://customer_bucket/some/prefix/"},

    "relative/path/to/custdata-1",

    "relative/path/custdata-2",

    ...

    "relative/path/custdata-N"

    ]

    This JSON is equivalent to the following S3Uri list:

    s3://customer_bucket/some/prefix/relative/path/to/custdata-1

    s3://customer_bucket/some/prefix/relative/path/custdata-2

    ...

    s3://customer_bucket/some/prefix/relative/path/custdata-N

    The complete set of S3Uri in this manifest is the input data for the channel for this data source. The object that each S3Uri points to must be readable by the IAM role that Amazon SageMaker uses to perform tasks on your behalf.

Definition at line 213 of file S3DataSource.h.

◆ SetS3Uri() [3/3]

void Aws::SageMaker::Model::S3DataSource::SetS3Uri ( const char *  value)
inline

Depending on the value specified for the S3DataType, identifies either a key name prefix or a manifest. For example:

  • A key name prefix might look like this: s3://bucketname/exampleprefix

  • A manifest might look like this: s3://bucketname/example.manifest

    A manifest is an S3 object which is a JSON file consisting of an array of elements. The first element is a prefix which is followed by one or more suffixes. SageMaker appends the suffix elements to the prefix to get a full set of S3Uri. Note that the prefix must be a valid non-empty S3Uri that precludes users from specifying a manifest whose individual S3Uri is sourced from different S3 buckets.

    The following code example shows a valid manifest format:

    [ {"prefix": "s3://customer_bucket/some/prefix/"},

    "relative/path/to/custdata-1",

    "relative/path/custdata-2",

    ...

    "relative/path/custdata-N"

    ]

    This JSON is equivalent to the following S3Uri list:

    s3://customer_bucket/some/prefix/relative/path/to/custdata-1

    s3://customer_bucket/some/prefix/relative/path/custdata-2

    ...

    s3://customer_bucket/some/prefix/relative/path/custdata-N

    The complete set of S3Uri in this manifest is the input data for the channel for this data source. The object that each S3Uri points to must be readable by the IAM role that Amazon SageMaker uses to perform tasks on your behalf.

Definition at line 271 of file S3DataSource.h.

◆ WithAttributeNames() [1/2]

S3DataSource& Aws::SageMaker::Model::S3DataSource::WithAttributeNames ( Aws::Vector< Aws::String > &&  value)
inline

A list of one or more attribute names to use that are found in a specified augmented manifest file.

Definition at line 516 of file S3DataSource.h.

◆ WithAttributeNames() [2/2]

S3DataSource& Aws::SageMaker::Model::S3DataSource::WithAttributeNames ( const Aws::Vector< Aws::String > &  value)
inline

A list of one or more attribute names to use that are found in a specified augmented manifest file.

Definition at line 510 of file S3DataSource.h.

◆ WithS3DataDistributionType() [1/2]

S3DataSource& Aws::SageMaker::Model::S3DataSource::WithS3DataDistributionType ( const S3DataDistribution value)
inline

If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated.

If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key. If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.

Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipe modes. Keep this in mind when developing algorithms.

In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key. If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File), this copies 1/n of the number of objects.

Definition at line 459 of file S3DataSource.h.

◆ WithS3DataDistributionType() [2/2]

S3DataSource& Aws::SageMaker::Model::S3DataSource::WithS3DataDistributionType ( S3DataDistribution &&  value)
inline

If you want Amazon SageMaker to replicate the entire dataset on each ML compute instance that is launched for model training, specify FullyReplicated.

If you want Amazon SageMaker to replicate a subset of data on each ML compute instance that is launched for model training, specify ShardedByS3Key. If there are n ML compute instances launched for a training job, each instance gets approximately 1/n of the number of S3 objects. In this case, model training on each machine uses only the subset of training data.

Don't choose more ML compute instances for training than available S3 objects. If you do, some nodes won't get any data and you will pay for nodes that aren't getting any training data. This applies in both File and Pipe modes. Keep this in mind when developing algorithms.

In distributed training, where you use multiple ML compute EC2 instances, you might choose ShardedByS3Key. If the algorithm requires copying training data to the ML storage volume (when TrainingInputMode is set to File), this copies 1/n of the number of objects.

Definition at line 479 of file S3DataSource.h.

◆ WithS3DataType() [1/2]

S3DataSource& Aws::SageMaker::Model::S3DataSource::WithS3DataType ( const S3DataType value)
inline

If you choose S3Prefix, S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.

If you choose ManifestFile, S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.

If you choose AugmentedManifestFile, S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe.

Definition at line 111 of file S3DataSource.h.

◆ WithS3DataType() [2/2]

S3DataSource& Aws::SageMaker::Model::S3DataSource::WithS3DataType ( S3DataType &&  value)
inline

If you choose S3Prefix, S3Uri identifies a key name prefix. Amazon SageMaker uses all objects that match the specified key name prefix for model training.

If you choose ManifestFile, S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for model training.

If you choose AugmentedManifestFile, S3Uri identifies an object that is an augmented manifest file in JSON lines format. This file contains the data you want to use for model training. AugmentedManifestFile can only be used if the Channel's input mode is Pipe.

Definition at line 125 of file S3DataSource.h.

◆ WithS3Uri() [1/3]

S3DataSource& Aws::SageMaker::Model::S3DataSource::WithS3Uri ( Aws::String &&  value)
inline

Depending on the value specified for the S3DataType, identifies either a key name prefix or a manifest. For example:

  • A key name prefix might look like this: s3://bucketname/exampleprefix

  • A manifest might look like this: s3://bucketname/example.manifest

    A manifest is an S3 object which is a JSON file consisting of an array of elements. The first element is a prefix which is followed by one or more suffixes. SageMaker appends the suffix elements to the prefix to get a full set of S3Uri. Note that the prefix must be a valid non-empty S3Uri that precludes users from specifying a manifest whose individual S3Uri is sourced from different S3 buckets.

    The following code example shows a valid manifest format:

    [ {"prefix": "s3://customer_bucket/some/prefix/"},

    "relative/path/to/custdata-1",

    "relative/path/custdata-2",

    ...

    "relative/path/custdata-N"

    ]

    This JSON is equivalent to the following S3Uri list:

    s3://customer_bucket/some/prefix/relative/path/to/custdata-1

    s3://customer_bucket/some/prefix/relative/path/custdata-2

    ...

    s3://customer_bucket/some/prefix/relative/path/custdata-N

    The complete set of S3Uri in this manifest is the input data for the channel for this data source. The object that each S3Uri points to must be readable by the IAM role that Amazon SageMaker uses to perform tasks on your behalf.

Definition at line 329 of file S3DataSource.h.

◆ WithS3Uri() [2/3]

S3DataSource& Aws::SageMaker::Model::S3DataSource::WithS3Uri ( const Aws::String value)
inline

Depending on the value specified for the S3DataType, identifies either a key name prefix or a manifest. For example:

  • A key name prefix might look like this: s3://bucketname/exampleprefix

  • A manifest might look like this: s3://bucketname/example.manifest

    A manifest is an S3 object which is a JSON file consisting of an array of elements. The first element is a prefix which is followed by one or more suffixes. SageMaker appends the suffix elements to the prefix to get a full set of S3Uri. Note that the prefix must be a valid non-empty S3Uri that precludes users from specifying a manifest whose individual S3Uri is sourced from different S3 buckets.

    The following code example shows a valid manifest format:

    [ {"prefix": "s3://customer_bucket/some/prefix/"},

    "relative/path/to/custdata-1",

    "relative/path/custdata-2",

    ...

    "relative/path/custdata-N"

    ]

    This JSON is equivalent to the following S3Uri list:

    s3://customer_bucket/some/prefix/relative/path/to/custdata-1

    s3://customer_bucket/some/prefix/relative/path/custdata-2

    ...

    s3://customer_bucket/some/prefix/relative/path/custdata-N

    The complete set of S3Uri in this manifest is the input data for the channel for this data source. The object that each S3Uri points to must be readable by the IAM role that Amazon SageMaker uses to perform tasks on your behalf.

Definition at line 300 of file S3DataSource.h.

◆ WithS3Uri() [3/3]

S3DataSource& Aws::SageMaker::Model::S3DataSource::WithS3Uri ( const char *  value)
inline

Depending on the value specified for the S3DataType, identifies either a key name prefix or a manifest. For example:

  • A key name prefix might look like this: s3://bucketname/exampleprefix

  • A manifest might look like this: s3://bucketname/example.manifest

    A manifest is an S3 object which is a JSON file consisting of an array of elements. The first element is a prefix which is followed by one or more suffixes. SageMaker appends the suffix elements to the prefix to get a full set of S3Uri. Note that the prefix must be a valid non-empty S3Uri that precludes users from specifying a manifest whose individual S3Uri is sourced from different S3 buckets.

    The following code example shows a valid manifest format:

    [ {"prefix": "s3://customer_bucket/some/prefix/"},

    "relative/path/to/custdata-1",

    "relative/path/custdata-2",

    ...

    "relative/path/custdata-N"

    ]

    This JSON is equivalent to the following S3Uri list:

    s3://customer_bucket/some/prefix/relative/path/to/custdata-1

    s3://customer_bucket/some/prefix/relative/path/custdata-2

    ...

    s3://customer_bucket/some/prefix/relative/path/custdata-N

    The complete set of S3Uri in this manifest is the input data for the channel for this data source. The object that each S3Uri points to must be readable by the IAM role that Amazon SageMaker uses to perform tasks on your behalf.

Definition at line 358 of file S3DataSource.h.


The documentation for this class was generated from the following file: