inferenceAmiVersion

Specifies an option from a collection of preconfigured Amazon Machine Image (AMI) images. Each image is configured by Amazon Web Services with a set of software and driver versions. Amazon Web Services optimizes these configurations for different machine learning workloads.

By selecting an AMI version, you can ensure that your inference environment is compatible with specific software requirements, such as CUDA driver versions, Linux kernel versions, or Amazon Web Services Neuron driver versions.

The AMI version names, and their configurations, are the following:

al2-ami-sagemaker-inference-gpu-2

• Accelerator: GPU

• NVIDIA driver version: 535

• CUDA version: 12.2

al2-ami-sagemaker-inference-gpu-2-1

• Accelerator: GPU

• NVIDIA driver version: 535

• CUDA version: 12.2

• NVIDIA Container Toolkit with disabled CUDA-compat mounting

al2-ami-sagemaker-inference-gpu-3-1

• Accelerator: GPU

• NVIDIA driver version: 550

• CUDA version: 12.4

• NVIDIA Container Toolkit with disabled CUDA-compat mounting

al2-ami-sagemaker-inference-neuron-2

• Accelerator: Inferentia2 and Trainium

• Neuron driver version: 2.19