Python script to upload sample data to S3 for training and hpo jobs (#38)

ryansteakley · web-flow · commit 3665703882f4 · 2021-05-28T11:12:20.000-07:00
diff --git a/samples/README.md b/samples/README.md
@@ -6,6 +6,7 @@ This sample demonstrates how to start jobs using your own script, packaged in a
 
 Follow the instructions on our [installation page](/README.md#getting-started) to create a Kubernetes cluster and install sagemaker controller.
 
+### SageMaker execution IAM role
 Run the following commands to create a SageMaker execution IAM role which is used by SageMaker service to access AWS resources. 
 
 ```
@@ -20,7 +21,25 @@ SAGEMAKER_EXECUTION_ROLE_ARN=$(aws iam get-role --role-name ${SAGEMAKER_EXECUTIO
 
 echo $SAGEMAKER_EXECUTION_ROLE_ARN
 ```
-Note down the execution role ARN to use in samples
+Note down the execution role ARN to use in samples.
+
+### S3 Bucket
+Run the following commands to create a S3 bucket which is used by SageMaker service to access and upload data. Use the region you used during installation unless you are trying cross region. SageMaker resources will be created in this region.
+```
+export AWS_DEFAULT_REGION=<REGION>
+export S3_BUCKET_NAME="ack-data-bucket-$AWS_DEFAULT_REGION"
+
+# [Option 1] if your region is us-east-1
+aws s3api create-bucket --bucket $S3_BUCKET_NAME --region $AWS_DEFAULT_REGION
+
+# [Option 2] if your region is NOT us-east-1
+aws s3api create-bucket --bucket $S3_BUCKET_NAME --region $AWS_DEFAULT_REGION \
+--create-bucket-configuration LocationConstraint=$AWS_DEFAULT_REGION
+
+echo $S3_BUCKET_NAME
+```
+Note down your S3 bucket name which will be used in the samples.
+
 ### Creating your first Job
 
 The easiest way to start is taking a look at the sample training jobs and its corresponding [README](/samples/training/README.md)
diff --git a/samples/batch_transform/README.md b/samples/batch_transform/README.md
@@ -4,7 +4,7 @@ This sample demonstrates how to start batch transform jobs using your own batch-
 
 ## Prerequisites
 
-This sample assumes that you have completed the [common prerequisties](/samples/README.md).
+This sample assumes that you have completed the [common prerequisites](/samples/README.md).
 
 You will also need a model in SageMaker for this sample. If you do not have one you must first create a [model](/samples/model/README.md).
 
diff --git a/samples/endpoint/README.md b/samples/endpoint/README.md
@@ -4,7 +4,7 @@ This sample demonstrates how to create Endpoints using your own Endpoint_base/co
 
 ## Prerequisites
 
-This sample assumes that you have completed the [common prerequisties](/samples/README.md).
+This sample assumes that you have completed the [common prerequisites](/samples/README.md).
 
 You will also need a model in SageMaker for this sample. If you do not have one you must first create a [model](/samples/model/README.md)
 
diff --git a/samples/hyperparameter_tuning/README.md b/samples/hyperparameter_tuning/README.md
@@ -4,7 +4,14 @@ This sample demonstrates how to start hyperparameter jobs using your own hyperpa
 
 ## Prerequisites
 
-This sample assumes that you have completed the [common prerequisties](/samples/README.md).
+This sample assumes that you have completed the [common prerequisites](/samples/README.md).
+
+### Upload S3 Data
+
+You will need training data uploaded to an S3 bucket. Make sure you have AWS credentials and and have the bucket in the same region where you plan to create SageMaker resources. Run the following python script to upload sample data to your S3 bucket.
+```
+python3 ../training/s3_sample_data.py $S3_BUCKET_NAME
+```
 
 ### Get an Image
 
@@ -20,6 +27,7 @@ A container image URL and tag looks has the following structure:
 In the `my-hyperparameter-job.yaml` file, modify the placeholder values with those associated with your account and hyperparameter job.
 
 ### Enabling Spot Training
+
 In the `my-hyperparameter-job.yaml` file under `spec.trainingJobDefinition` add `enableManagedSpotTraining` and set the value to true. You will also need to specify a `spec.trainingJobDefinition.stoppingCondition.maxRuntimeInSeconds` and `spec.trainingJobDefinition.stoppingCondition.maxWaittimeInSeconds`
 
 ## Submitting your Hyperparameter Job
diff --git a/samples/hyperparameter_tuning/my-hyperparameter-job.yaml b/samples/hyperparameter_tuning/my-hyperparameter-job.yaml
@@ -63,9 +63,9 @@ spec:
         s3DataSource:
           s3DataType: S3Prefix
           # The source of the training data
-          s3URI: s3://<YOUR BUCKET/PATH>
+          s3URI: s3://<YOUR BUCKET>/sagemaker/xgboost/train
           s3DataDistributionType: FullyReplicated
-      contentType: text/csv
+      contentType: text/libsvm
       compressionType: None
       recordWrapperType: None
       inputMode: File
@@ -74,9 +74,9 @@ spec:
         s3DataSource:
           s3DataType: S3Prefix
           # The source of the validation data
-          s3URI: s3://<YOUR BUCKET/PATH>
+          s3URI: s3://<YOUR BUCKET>/sagemaker/xgboost/validation
           s3DataDistributionType: FullyReplicated
-      contentType: text/csv
+      contentType: text/libsvm
       compressionType: None
       recordWrapperType: None
       inputMode: File
@@ -87,5 +87,7 @@ spec:
       instanceType: ml.m4.xlarge
       instanceCount: 1
       volumeSizeInGB: 25
+    stoppingCondition:
+      maxRuntimeInSeconds: 3600
     enableNetworkIsolation: true
     enableInterContainerTrafficEncryption: false
diff --git a/samples/job_definitions/data_quality/README.md b/samples/job_definitions/data_quality/README.md
@@ -4,7 +4,7 @@ This sample demonstrates how to start data quality job definitions using your ow
 
 ## Prerequisites
 
-This sample assumes that you have completed the [common prerequisties](/samples/README.md).
+This sample assumes that you have completed the [common prerequisites](/samples/README.md).
 
 You will need an [Endpoint](/samples/endpoint/README.md) configured in SageMaker and you will need to run a baselining job to generate baseline statistics and constraints.
 
diff --git a/samples/job_definitions/model_bias/README.md b/samples/job_definitions/model_bias/README.md
@@ -4,7 +4,7 @@ This sample demonstrates how to start model-bias job definitions using your own
 
 ## Prerequisites
 
-This sample assumes that you have completed the [common prerequisties](/samples/README.md).
+This sample assumes that you have completed the [common prerequisites](/samples/README.md).
 
 You will also need an [Endpoint](/samples/endpoint/README.md) configured in SageMaker and you will need to run a baselining job to generate baseline constraints only.
 
diff --git a/samples/job_definitions/model_explainability/README.md b/samples/job_definitions/model_explainability/README.md
@@ -4,7 +4,7 @@ This sample demonstrates how to start model-explainability job definitions using
 
 ## Prerequisites
 
-This sample assumes that you have completed the [common prerequisties](/samples/README.md).
+This sample assumes that you have completed the [common prerequisites](/samples/README.md).
 
 You will also need an [Endpoint](/samples/endpoint/README.md) configured in SageMaker and you will need to run a baselining job to generate baseline constraints only.
 
diff --git a/samples/job_definitions/model_quality/README.md b/samples/job_definitions/model_quality/README.md
@@ -4,7 +4,7 @@ This sample demonstrates how to start model-quality job definitions using your o
 
 ## Prerequisites
 
-This sample assumes that you have completed the [common prerequisties](/samples/README.md).
+This sample assumes that you have completed the [common prerequisites](/samples/README.md).
 
 You will also need an [Endpoint](/samples/endpoint/README.md) configured in SageMaker and you will need to run a baselining job to generate baseline constraints only.
 
diff --git a/samples/model/README.md b/samples/model/README.md
@@ -4,7 +4,7 @@ This sample demonstrates how to start models/create a model using your own model
 
 ## Prerequisites
 
-This sample assumes that you have completed the [common prerequisties](/samples/README.md).
+This sample assumes that you have completed the [common prerequisites](/samples/README.md).
 
 ### Get an Image
 
diff --git a/samples/monitoring_schedule/README.md b/samples/monitoring_schedule/README.md
@@ -4,7 +4,7 @@ This sample demonstrates how to start Monitoring scheduling using your own Monit
 
 ## Prerequisites
 
-This sample assumes that you have completed the [common prerequisties](/samples/README.md). It also assumes you have a job definition of one of these [types](/samples/job_definitions).
+This sample assumes that you have completed the [common prerequisites](/samples/README.md). It also assumes you have a job definition of one of these [types](/samples/job_definitions).
 
 ### Updating the Monitoring Specification
 
diff --git a/samples/processing/README.md b/samples/processing/README.md
@@ -4,7 +4,7 @@ This sample demonstrates how to start processing jobs using your own processing
 
 ## Prerequisites
 
-This sample assumes that you have completed the [common prerequisties](/samples/README.md).
+This sample assumes that you have completed the [common prerequisites](/samples/README.md).
 
 You will need to upload [kmeans_preprocessing.py](/samples/processing/kmeans_preprocessing.py) to an S3 bucket and update the s3Input s3URI path.
 
diff --git a/samples/training/README.md b/samples/training/README.md
@@ -4,7 +4,14 @@ This sample demonstrates how to start training jobs using your own training scri
 
 ## Prerequisites
 
-This sample assumes that you have completed the [common prerequisties](/samples/README.md).
+This sample assumes that you have completed the [common prerequisites](/samples/README.md).
+
+###  Upload S3 Data
+
+You will need training data uploaded to an S3 bucket. Make sure you have AWS credentials and and have the bucket in the same region where you plan to create SageMaker resources. Run the following python script to upload sample data to your S3 bucket.
+```
+python3 s3_sample_data.py $S3_BUCKET_NAME
+```
 
 ### Get an Image
 
diff --git a/samples/training/my-training-job.yaml b/samples/training/my-training-job.yaml
@@ -38,16 +38,16 @@ spec:
         s3DataSource:
           s3DataType: S3Prefix
           # The input path of our train data 
-          s3URI: s3://<YOUR BUCKET/PATH>
+          s3URI: s3://<YOUR BUCKET>/sagemaker/xgboost/train
           s3DataDistributionType: FullyReplicated
-      contentType: text/csv
+      contentType: text/libsvm
       compressionType: None
     - channelName: validation
       dataSource:
         s3DataSource:
           s3DataType: S3Prefix
           # The input path of our validation data 
-          s3URI: s3://<YOUR BUCKET/PATH>
+          s3URI: s3://<YOUR BUCKET>/sagemaker/xgboost/validation
           s3DataDistributionType: FullyReplicated
-      contentType: text/csv
+      contentType: text/libsvm
       compressionType: None
diff --git a/samples/training/s3_sample_data.py b/samples/training/s3_sample_data.py
@@ -0,0 +1,26 @@
+import urllib.request
+from urllib.parse import urlparse
+import sys
+
+# Gets your bucket name from command line
+try:
+    bucket = str(sys.argv[1])
+except Exception as error:
+    print("Please pass your bucket name as a commandline argument")
+    sys.exit(1)
+
+# Download dataset from pinned commit
+url = "https://github.com/aws/amazon-sagemaker-examples/raw/af6667bd0be3c9cdec23fecda7f0be6d0e3fa3ea/sagemaker-debugger/xgboost_realtime_analysis/data_utils.py"
+urllib.request.urlretrieve(url, "data_utils.py")
+
+from data_utils import load_mnist, upload_to_s3
+
+prefix = "sagemaker/xgboost"
+train_file, validation_file = load_mnist()
+upload_to_s3(train_file, bucket, f"{prefix}/train/mnist.train.libsvm")
+upload_to_s3(validation_file, bucket, f"{prefix}/validation/mnist.validation.libsvm")
+
+# Remove downloaded file
+import os
+
+os.remove("data_utils.py")
