Class DevEndpoint

java.lang.Object
software.amazon.awssdk.services.glue.model.DevEndpoint
All Implemented Interfaces:
Serializable, SdkPojo, ToCopyableBuilder<DevEndpoint.Builder,DevEndpoint>
@Generated("software.amazon.awssdk:codegen") public final class DevEndpoint extends Object implements SdkPojo, Serializable, ToCopyableBuilder<DevEndpoint.Builder,DevEndpoint>

A development endpoint where a developer can remotely debug extract, transform, and load (ETL) scripts.

See Also:

  • Method Details

    • endpointName

      public final String endpointName()

      The name of the DevEndpoint.

      Returns:
      The name of the DevEndpoint.

    • roleArn

      public final String roleArn()

      The Amazon Resource Name (ARN) of the IAM role used in this DevEndpoint.

      Returns:
      The Amazon Resource Name (ARN) of the IAM role used in this DevEndpoint.

    • hasSecurityGroupIds

      public final boolean hasSecurityGroupIds()
      For responses, this returns true if the service returned a value for the SecurityGroupIds property. This DOES NOT check that the value is non-empty (for which, you should check the isEmpty() method on the property). This is useful because the SDK will never return a null collection or map, but you may need to differentiate between the service returning nothing (or null) and the service returning an empty collection or map. For requests, this returns true if a value for the property was specified in the request builder, and false if a value was not specified.

    • securityGroupIds

      public final List<String> securityGroupIds()

      A list of security group identifiers used in this DevEndpoint.

      Attempts to modify the collection returned by this method will result in an UnsupportedOperationException.

      This method will never return null. If you would like to know whether the service returned this field (so that you can differentiate between null and empty), you can use the hasSecurityGroupIds() method.

      Returns:
      A list of security group identifiers used in this DevEndpoint.

    • subnetId

      public final String subnetId()

      The subnet ID for this DevEndpoint.

      Returns:
      The subnet ID for this DevEndpoint.

    • yarnEndpointAddress

      public final String yarnEndpointAddress()

      The YARN endpoint address used by this DevEndpoint.

      Returns:
      The YARN endpoint address used by this DevEndpoint.

    • privateAddress

      public final String privateAddress()

      A private IP address to access the DevEndpoint within a VPC if the DevEndpoint is created within one. The PrivateAddress field is present only when you create the DevEndpoint within your VPC.

      Returns:
      A private IP address to access the DevEndpoint within a VPC if the DevEndpoint is created within one. The PrivateAddress field is present only when you create the DevEndpoint within your VPC.

    • zeppelinRemoteSparkInterpreterPort

      public final Integer zeppelinRemoteSparkInterpreterPort()

      The Apache Zeppelin port for the remote Apache Spark interpreter.

      Returns:
      The Apache Zeppelin port for the remote Apache Spark interpreter.

    • publicAddress

      public final String publicAddress()

      The public IP address used by this DevEndpoint. The PublicAddress field is present only when you create a non-virtual private cloud (VPC) DevEndpoint.

      Returns:
      The public IP address used by this DevEndpoint. The PublicAddress field is present only when you create a non-virtual private cloud (VPC) DevEndpoint.

    • status

      public final String status()

      The current status of this DevEndpoint.

      Returns:
      The current status of this DevEndpoint.

    • workerType

      public final WorkerType workerType()

      The type of predefined worker that is allocated to the development endpoint. Accepts a value of Standard, G.1X, or G.2X.

      • For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.

      • For the G.1X worker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

      • For the G.2X worker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

      Known issue: when a development endpoint is created with the G.2X WorkerType configuration, the Spark drivers for the development endpoint will run on 4 vCPU, 16 GB of memory, and a 64 GB disk.

      If the service returns an enum value that is not available in the current SDK version, workerType will return WorkerType.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available from workerTypeAsString().

      Returns:
      The type of predefined worker that is allocated to the development endpoint. Accepts a value of Standard, G.1X, or G.2X.

      • For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.

      • For the G.1X worker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

      • For the G.2X worker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

      Known issue: when a development endpoint is created with the G.2X WorkerType configuration, the Spark drivers for the development endpoint will run on 4 vCPU, 16 GB of memory, and a 64 GB disk.

      See Also:

    • workerTypeAsString

      public final String workerTypeAsString()

      The type of predefined worker that is allocated to the development endpoint. Accepts a value of Standard, G.1X, or G.2X.

      • For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.

      • For the G.1X worker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

      • For the G.2X worker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

      Known issue: when a development endpoint is created with the G.2X WorkerType configuration, the Spark drivers for the development endpoint will run on 4 vCPU, 16 GB of memory, and a 64 GB disk.

      If the service returns an enum value that is not available in the current SDK version, workerType will return WorkerType.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available from workerTypeAsString().

      Returns:
      The type of predefined worker that is allocated to the development endpoint. Accepts a value of Standard, G.1X, or G.2X.

      • For the Standard worker type, each worker provides 4 vCPU, 16 GB of memory and a 50GB disk, and 2 executors per worker.

      • For the G.1X worker type, each worker maps to 1 DPU (4 vCPU, 16 GB of memory, 64 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

      • For the G.2X worker type, each worker maps to 2 DPU (8 vCPU, 32 GB of memory, 128 GB disk), and provides 1 executor per worker. We recommend this worker type for memory-intensive jobs.

      Known issue: when a development endpoint is created with the G.2X WorkerType configuration, the Spark drivers for the development endpoint will run on 4 vCPU, 16 GB of memory, and a 64 GB disk.

      See Also:

    • glueVersion

      public final String glueVersion()

      Glue version determines the versions of Apache Spark and Python that Glue supports. The Python version indicates the version supported for running your ETL scripts on development endpoints.

      For more information about the available Glue versions and corresponding Spark and Python versions, see Glue version in the developer guide.

      Development endpoints that are created without specifying a Glue version default to Glue 0.9.

      You can specify a version of Python support for development endpoints by using the Arguments parameter in the CreateDevEndpoint or UpdateDevEndpoint APIs. If no arguments are provided, the version defaults to Python 2.

      Returns:
      Glue version determines the versions of Apache Spark and Python that Glue supports. The Python version indicates the version supported for running your ETL scripts on development endpoints.

      For more information about the available Glue versions and corresponding Spark and Python versions, see Glue version in the developer guide.

      Development endpoints that are created without specifying a Glue version default to Glue 0.9.

      You can specify a version of Python support for development endpoints by using the Arguments parameter in the CreateDevEndpoint or UpdateDevEndpoint APIs. If no arguments are provided, the version defaults to Python 2.

    • numberOfWorkers

      public final Integer numberOfWorkers()

      The number of workers of a defined workerType that are allocated to the development endpoint.

      The maximum number of workers you can define are 299 for G.1X, and 149 for G.2X.

      Returns:
      The number of workers of a defined workerType that are allocated to the development endpoint.

      The maximum number of workers you can define are 299 for G.1X, and 149 for G.2X.

    • numberOfNodes

      public final Integer numberOfNodes()

      The number of Glue Data Processing Units (DPUs) allocated to this DevEndpoint.

      Returns:
      The number of Glue Data Processing Units (DPUs) allocated to this DevEndpoint.

    • availabilityZone

      public final String availabilityZone()

      The Amazon Web Services Availability Zone where this DevEndpoint is located.

      Returns:
      The Amazon Web Services Availability Zone where this DevEndpoint is located.

    • vpcId

      public final String vpcId()

      The ID of the virtual private cloud (VPC) used by this DevEndpoint.

      Returns:
      The ID of the virtual private cloud (VPC) used by this DevEndpoint.

    • extraPythonLibsS3Path

      public final String extraPythonLibsS3Path()

      The paths to one or more Python libraries in an Amazon S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.

      You can only use pure Python libraries with a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not currently supported.

      Returns:
      The paths to one or more Python libraries in an Amazon S3 bucket that should be loaded in your DevEndpoint. Multiple values must be complete paths separated by a comma.

      You can only use pure Python libraries with a DevEndpoint. Libraries that rely on C extensions, such as the pandas Python data analysis library, are not currently supported.

    • extraJarsS3Path

      public final String extraJarsS3Path()

      The path to one or more Java .jar files in an S3 bucket that should be loaded in your DevEndpoint.

      You can only use pure Java/Scala libraries with a DevEndpoint.

      Returns:
      The path to one or more Java .jar files in an S3 bucket that should be loaded in your DevEndpoint.

      You can only use pure Java/Scala libraries with a DevEndpoint.

    • failureReason

      public final String failureReason()

      The reason for a current failure in this DevEndpoint.

      Returns:
      The reason for a current failure in this DevEndpoint.

    • lastUpdateStatus

      public final String lastUpdateStatus()

      The status of the last update.

      Returns:
      The status of the last update.

    • createdTimestamp

      public final Instant createdTimestamp()

      The point in time at which this DevEndpoint was created.

      Returns:
      The point in time at which this DevEndpoint was created.

    • lastModifiedTimestamp

      public final Instant lastModifiedTimestamp()

      The point in time at which this DevEndpoint was last modified.

      Returns:
      The point in time at which this DevEndpoint was last modified.

    • publicKey

      public final String publicKey()

      The public key to be used by this DevEndpoint for authentication. This attribute is provided for backward compatibility because the recommended attribute to use is public keys.

      Returns:
      The public key to be used by this DevEndpoint for authentication. This attribute is provided for backward compatibility because the recommended attribute to use is public keys.

    • hasPublicKeys

      public final boolean hasPublicKeys()
      For responses, this returns true if the service returned a value for the PublicKeys property. This DOES NOT check that the value is non-empty (for which, you should check the isEmpty() method on the property). This is useful because the SDK will never return a null collection or map, but you may need to differentiate between the service returning nothing (or null) and the service returning an empty collection or map. For requests, this returns true if a value for the property was specified in the request builder, and false if a value was not specified.

    • publicKeys

      public final List<String> publicKeys()

      A list of public keys to be used by the DevEndpoints for authentication. Using this attribute is preferred over a single public key because the public keys allow you to have a different private key per client.

      If you previously created an endpoint with a public key, you must remove that key to be able to set a list of public keys. Call the UpdateDevEndpoint API operation with the public key content in the deletePublicKeys attribute, and the list of new keys in the addPublicKeys attribute.

      Attempts to modify the collection returned by this method will result in an UnsupportedOperationException.

      This method will never return null. If you would like to know whether the service returned this field (so that you can differentiate between null and empty), you can use the hasPublicKeys() method.

      Returns:
      A list of public keys to be used by the DevEndpoints for authentication. Using this attribute is preferred over a single public key because the public keys allow you to have a different private key per client.

      If you previously created an endpoint with a public key, you must remove that key to be able to set a list of public keys. Call the UpdateDevEndpoint API operation with the public key content in the deletePublicKeys attribute, and the list of new keys in the addPublicKeys attribute.

    • securityConfiguration

      public final String securityConfiguration()

      The name of the SecurityConfiguration structure to be used with this DevEndpoint.

      Returns:
      The name of the SecurityConfiguration structure to be used with this DevEndpoint.

    • hasArguments

      public final boolean hasArguments()
      For responses, this returns true if the service returned a value for the Arguments property. This DOES NOT check that the value is non-empty (for which, you should check the isEmpty() method on the property). This is useful because the SDK will never return a null collection or map, but you may need to differentiate between the service returning nothing (or null) and the service returning an empty collection or map. For requests, this returns true if a value for the property was specified in the request builder, and false if a value was not specified.

    • arguments

      public final Map<String,String> arguments()

      A map of arguments used to configure the DevEndpoint.

      Valid arguments are:

      • "--enable-glue-datacatalog": ""

      You can specify a version of Python support for development endpoints by using the Arguments parameter in the CreateDevEndpoint or UpdateDevEndpoint APIs. If no arguments are provided, the version defaults to Python 2.

      Attempts to modify the collection returned by this method will result in an UnsupportedOperationException.

      This method will never return null. If you would like to know whether the service returned this field (so that you can differentiate between null and empty), you can use the hasArguments() method.

      Returns:
      A map of arguments used to configure the DevEndpoint.

      Valid arguments are:

      • "--enable-glue-datacatalog": ""

      You can specify a version of Python support for development endpoints by using the Arguments parameter in the CreateDevEndpoint or UpdateDevEndpoint APIs. If no arguments are provided, the version defaults to Python 2.

    • toBuilder

      public DevEndpoint.Builder toBuilder()
      Description copied from interface: ToCopyableBuilder
      Take this object and create a builder that contains all of the current property values of this object.
      Specified by:
      toBuilder in interface ToCopyableBuilder<DevEndpoint.Builder,DevEndpoint>
      Returns:
      a builder for type T

    • builder

      public static DevEndpoint.Builder builder()

    • serializableBuilderClass

      public static Class<? extends DevEndpoint.Builder> serializableBuilderClass()

    • hashCode

      public final int hashCode()
      Overrides:
      hashCode in class Object

    • equals

      public final boolean equals(Object obj)
      Overrides:
      equals in class Object

    • equalsBySdkFields

      public final boolean equalsBySdkFields(Object obj)
      Description copied from interface: SdkPojo
      Indicates whether some other object is "equal to" this one by SDK fields. An SDK field is a modeled, non-inherited field in an SdkPojo class, and is generated based on a service model.

      If an SdkPojo class does not have any inherited fields, equalsBySdkFields and equals are essentially the same.

      Specified by:
      equalsBySdkFields in interface SdkPojo
      Parameters:
      obj - the object to be compared with
      Returns:
      true if the other object equals to this object by sdk fields, false otherwise.

    • toString

      public final String toString()
      Returns a string representation of this object. This is useful for testing and debugging. Sensitive data will be redacted from this string using a placeholder value.
      Overrides:
      toString in class Object

    • getValueForField

      public final <T> Optional<T> getValueForField(String fieldName, Class<T> clazz)

    • sdkFields

      public final List<SdkField<?>> sdkFields()
      Specified by:
      sdkFields in interface SdkPojo
      Returns:
      List of SdkField in this POJO. May be empty list but should never be null.