AWS-Big Data Demo(Word Count)

                                       

AWS-Big Data Demo(Word Count)
In The AWS Management Console,click Services then click-> S3,

Click Create Bucket
In the Bucket Name field,type "emr-agoutam"
Select Region
Click Create

Creating and Launching a MapReduce Cluster

In the AWS Management Console, -> Click Services and Click ->EMR

Create cluster ->Go to Advanced Options

On Clciking  Advanced Options ,Advanced Cluster Configuration page, Click Configure sample application

From the Select Sample application drop-down list,Select Wordcount

In the Output location field,enter :s3://emr-agoutam/output/

Check box logging enabled should be selected

 In the Logging section, enter s3://emr-agoutam/logs/ 

Leave Debugging enabled.

In the Tags section Type Owner in the key column.

In the Value column ,type Development

In the Software Configuration section ,Amazon is selected as a default .

Choose EMR 4.0.0.(Hadoop 2.6)

Configure Master ,core and Task Node as Mentioned Below

Master m3.xlarge count=1

Core   m3.xlarge count =2

Task  m3.xlarge count =0

EC2 key Pair ,select Proceed without an EC2 Key pair.

Under Security and Access /IAM Roles,select the default radio button .

Towards the bottom of the page,select Yes option for Auto-terminate

Auto terminate :

 Determines what the cluster does after its last step,

Yes means the cluster auto-terminates after the last step completes.

No means the cluster runs untill you manually terminate it.

Review the Configuration and ,click Create cluster

Monitor the Cluster 

Click Cluster List. This shows a list of clusters and their status .

Click the arrow to the left of cluster name ("word count")

Click Monitoring Tab Below  to see Cluster running Process

Once the EMR job is completed ,A status of Terminated .All Steps completed

Status changes to terminated All steps completed

In The AWS Management Console,click Services then click-> S3,

Click In The AWS Management Console,click Services then click-> S3,

Click in front of Bucket "emr-agoutam" and click on folder name output  to see results

Click in front of Bucket "emr-agoutam" and click on folder name logs  to see LOGS

AWS Elastic Beanstalk

                                             AWS Elastic Beanstalk

Create and deploying a New Application in the AWS Management Console.

AWS Elastic Beanstalk deploys and manages applications in the AWS cloud.

Developer can upload application, and Elastic beanstalk, automatically handles the deployment details

Of capacity provisioning, load Balancing, Auto Scaling, and Application health Monitoring

Deploying the Application

In the AWS Management Console, on the Services menu, click Elastic Beanstalk.

Step 1Application info

                Application name Box Give agoutamsignup-app

                Description :ElasticBeanstalk

Step 2 : Environment type

Click on Create web server,

                                Environment tier Webserver

                                Predefined Configuration Python

                                Environment Type Single Instance

 Step 3 : Application Version

                                Source Select Radio Button Upload your own

                                Upload Sample Application eb-py-flask-signup-v1.1.0

Step 4 : Environment Information

                                Environment Name agoutamsignup-app

Step 5 :Additional Resources

 Don't Select Additional Resources

 Step 6 Configuration Details

                Instance type t1.micro

                emailabhishekvitmca@yahoo.com

Step 7 :  Environment Tags

Step 8 :Review Information

If Status is green means ready to use

AWS Console -Click on IAM

Click Policy- Blue Button create Policy -Select blue button create your own Policy

Policy name agoutamsignupapppolicy

Copy below Policy and paste

{

  Version 2012-10-17,

  Statement [

    {

      Effect Allow,

      Action   [ dynamodbPutItem ],

      Resource [  ]

    },

    {

      Effect Allow,

      Action   [ snsPublish ],

      Resource [  ]

    }

  ]

}

Click -Create Policy Button

 Click Roles in left, Click on role name aws-elasticbeanstalk-ec2-role

Under permission section. Click Attach Policy

Click the check box next to your SignupAppPolicy and click Attach Policy

In Resultant Attach Policy window, click Policy Type and choose Customer Managed polices

Click the checkbox next to your signupapppolicy and click attach policy

Trying the Application.

On Aws service menu .click Elastic Beanstalk, click name of application in the box

On the green ,signupapp-env

Open the application by clicking the URL at the top of elastic Beanstalk dashboard

.

AWS-CloudFront Distribution Webpage

                   CloudFront Distribution Webpage

Create Bucket in Amazon S3 console :goutam and make it Public.

Add Files and Folder and give Permission to avail,

Make public read priviledges

click set details,on set details dialog box,click set Permission

On set Permission dialouge box,Make Everything public

And Click Start Upload

Click on file name and then on properties.

Files Uploaded  so can be avail over Internet

Go to Aws Management console ->click services ->Click cloudFront

Click Create Distribution

On Select a delivery method for your content page,in the Web Section click Get Started to select

select web as delivery method.

 On the next page,click origin Domain Name  box and select bucket containing Public accessible image

Currently while AWS create distribution ,Status of distribution would be in progress and wait unless and untill status changed to Deployed

Once Status changed to Deployed and State is Active ,Note down the Domain name which act as URl

and can Be accessed over internet

Save file as TestCloudFront .html in Desktop

2. My Cloud Front Test-EdurekaProject
4. My Text content goes here
5. alt -"my test image "

In Line 5 Modify Img src with Cloudfront domain name  and  Open TestCloudFront.html file in Browser able to see content over Internet

--------------------To Delete a Distribution---------------------------

Right pane of Amazon CloudFront Console ,check the check box for distribution and click diable

and then window prompt for confirmation click yes Disbale.wait till Status changes from disable ->Deployed

verify and wait till Status changed to deployed

Check the chek box for distribution and click delete and click yes,Delete to confirm

-------------------To Delete an Amazon S3 Bucket----------------------

Go to Amazon S3 Console and ,select Bucket Click Actions and  click Delete ,click Ok in message box