Very Important: Please terminate the cluster as soon as you are done. Otherwise, you will continue to be charged!

Prerequisites: You need to have Amazon Educate account and a working ssh setup

Obtain an Amazon AWS account if you have not done it already. Please visit:
AWS Educate

SSH keypair (one time)

(you can run spark jobs with spark-submit and pyspark)

Login to Amazon AWS consol
https://aws.amazon.com/emr/
Click on Services at the top left and choose EMR from the panel of services
Click on "Create Cluster" and choose "Advanced Options"
"Software Configuration": Add Spark to the list and choose "Next" at the bottom
"Hardware": You can leave the default hardware (m3.xlarge) and choose the number of
core instances. (the default are 1 and 2). You will be charged for this. If you want to know how much
you will be charged, visit:
https://aws.amazon.com/emr/pricing/
General Options:
1. choose a name
2. uncheck termination protection
3. choose Bootstrap Action:
  1. Add Bootstrap Action --> choose "Custom Action"
    --> click "Configure and Add"
    --> Choose any name
    --> Copy and paste the following in "Script Location"
    s3://emr-scripts-kindires/install-jupyter-emr-12212016.sh
Security ---> if you have ssh keys created (see below), select that for the EC2 keypair.
You need this for sshing to the cluster.
Create cluster

Terminating the cluster

You can terminate the cluster by pressing the terminate button on the cluster "dashboard".

Working with Spark on the EMR cluster

Working on the EMR cluster (CS 205)