-
Notifications
You must be signed in to change notification settings - Fork 4
Setup
Link to Peter Gottesman's work. I strongly suggest utilizing his documentatin in conjunction with mine to help through the setup: https://github.com/open-mpi/aws-pmix-scale-testing
Clone Peter Gottesman's work. The aws-pmix-scale-testing repository from github into an appropriate folder. I will be assuming you clone aws-pmix-scale-testing into your home directory
git clone https://github.com/open-mpi/aws-pmix-scale-testing
Next, you'll need to install CfnCluster and the AWS Command Line Interface. The docs: https://cfncluster.readthedocs.io/en/latest/getting_started.html#configuring-cfncluster https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html
sudo pip install cfncluster //You may have to install pip as well
sudo pip install awscli --upgrade --user
This might need to be tweaked.
Head to AWS and under Services find "VPC". Select "Your VPCs" on the left. "Create VPC" and name it what you'd like. For the "IPv4 CIDR block", give it "10.0.0.0/16" and then create it.
Next, on the left side, click "Subnets". Then "Create subnet". Give it a name and in the VPC drop down, select the VPC you just created. Selected the Availability Zone you are in (Or leave No Preference). For "IPv4 CIDR block", put "10.0.0.0/24".
aws configure
This is where you'll need your AWS Access Key ID and Secret Access Key. Here, at the bottom of the page, are some instructions on getting your Security Credentials. You'll download a csv file and you'll be able to take those values and plug them in. Next is Default region name, select the region you are going to use (Google will tell you the different regions). For example, I've been using "uw-west-2" for everything. Then select the default for the last option.
cd ~/.aws
ls //In this folder you should see a config and credentials file
For reference, my config and credentials files:
[default] //Config
region = uw-west-2
[default] //Credentials
aws_access_key_id = <access_key_id>
aws_secret_access_key = <secret_access_key>
Next, you'll need to configure CfnCluster:
cfncluster configure
ctrl + z
ls -l //In the home directory, there should now be a .cfncluster folder
cd .cfncluster
vim config //Config file for CfnCluster (Used with commands "cfncluster create clusterName"
Your config file should look something like this. You'll need your Security credentials for the aws_access_key_id and aws_secrete_access_key as well as your EC2 Key Pair which will be plugged into the key_name parameter:
[aws]
aws_region_name = us-west-2
aws_access_key_id = <access_key_id>
aws_secret_access_key = <secret_access_key>
[cluster default]
vpc_settings = public
key_name = <AWSKeyPair
[vpc public]
master_subnet_id = <subnet>
vpc_id = <vpc>
[global]
cluster_template = default
sanity_check = true
update_check = true
CfnCluster should now be able to launch a cluster:
cfncluster create test //May have to issue ctrl + c (Still runs process in background)
cfncluster list
cfncluster status test //May have to issue ctrl + c
cfncluster delete test //May have to issue ctrl + c (Deletes in background)
cd aws-pmix-scale-testing
vim config
Edit the config. Include appropriate information:
[aws]
aws_region_name = <region>
aws_access_key_id = <access_key>
aws_secret_access_key = <secrete_access>
[vpc public] //Typically what the VPC looks like
master_subnet_id = <subnet> //In this config file, this VPC is not in use
vpc_id = <subnet>
[global]
cluster_template = default
sanity_check = true
update_check = true
[vpc west2bSubnet] //This VPC is in use because of the line below
vpc_id = <vpc>
master_subnet_id = <subnet>
[cluster default]
vpc_settings = west2bSubnet //References which VPC you'd like to use
key_name = MyAWSKeyPair
scheduler = slurm
ec2_iam_role = cfnClusterRole //Reference roles created in the Cluster Policies, Users, and Roles section
pre_install=https://<test_s3_bucket_name>.s3.amazonaws.com/V2/preinstall.sh
post_install=https://<test_s3_bucket_name>.s3.amazonaws.com/V2/setup.sh
In S3, in Services, find the S3 console. Once in the console, "Create bucket". Give it a name and make sure it will be in the region the rest of your resources are in. Then click next and accept defaults for everything. Create three buckets. One for builds, logs, and tests.
Next, in the aws-pmix-scale-testing folder, create a new file from buckets.sh.template called buckets.sh, then copy the names of the S3 buckets into the file.
cp buckets.sh.template buckets.sh
vim buckets.sh
after editing, buckets.sh will look something like this:
backend="ompitestsnmc" //Your bucket names will be different as ALL bucket names
builds="ompibuildsnmc" // are unique
logs="ompilogsnmc"
Next, you'll need to simlink buckets.sh into aws-pmix-scale-testing/backend/shared so buckets.sh can be access from the backend/shared folder
ln -s /full/path/to/home/aws-pmix-scale-testing/buckets.sh /full/path/to/home/aws-pmix-scale-testing/backend/shared/buckets.sh
cd backend/shared
ls -Al //Check to make sure there is a correctly setup simlink
The EC2 Key Pair allows connection between your computer and the EC2 instance. The EC2 Key Pair should be in your aws-pmix-scale-testing folder. If you are launching things from your personal computer, just copy it into the correct folder. If you are using ssh to enter into another machine and launch from there, you'll need to somehow transfer the file. FileZilla works quite well.
To make sure the S3 buckets reflect all changes made through out this process, you'll have to use the sync database scripts
./sync.sh
Use the less command to figure out what it does
The main uploads to s3 take place in these scripts:
aws-pmix-scale-testing/backend/setup.sh
aws-pmix-scale-testing/backend/shared/shutdown.sh
The s3 upload in setup.sh uploads the Open MPI build to the openmpibuild bucket on s3 The s3 upload in shutdown.sh uploads the OpenM MPI test results to the ompilogs bucket on s3
This is a similar process to Buckets. Create the database, cp mysqlauth.sh.template to mysqlauth.sh and file in info. Make sure to symlink as well. I didn't do this so I'm not aware of the exact process. Just the general process.