Tuesday, June 7, 2016

04_AWS - Networking

Amazon Web Services


2006: Amazon launched Amazon Web Service (AWS) on a utility computing basis although the initial released dated back to July 2002.

Amazon Web Services (AWS) is a collection of remote computing services (also called web services) that together make up a cloud computing platform, offered over the Internet by

The most central and well-known of these services are Amazon EC2 (Elastic Compute Cloud )and Amazon S3 (Simple Storage Service).



Amazon Web Services is based on SOA standards, including HTTP, REST, and SOAP transfer protocols, open source and commercial operating systems, application servers, and browser-based access.




1.       Amazon VPC

2.       AWS Direct Connect

3.       Elastic Load Balancing

4.       Amazon Route 53



1). Amazon VPC (Virtual Private Cloud)


·         Amazon VPC enables you to launch Amazon Web Services (AWS) resources into a virtual network that you've defined.

·         This virtual network closely resembles a traditional network that you'd operate in your own data center, with the benefits of using the scalable infrastructure of AWS.


VPCs and Subnets

·         VPC is a virtual network dedicated to your AWS account. It is logically isolated from other virtual networks in the AWS cloud.

·         You can launch your AWS resources, such as Amazon EC2 instances, into your VPC.

·         You can configure your VPC; you can select its IP address range, create subnets, and configure route tables, network gateways, and security settings.


·         A Subnet is a range of IP addresses in your VPC.You can launch AWS resources into a subnet that you select.

·         Use a public subnet for resources that must be connected to the Internet, and a private subnet for resources that won't be connected to the Internet



2). AWS Direct Connect 


·         AWS Direct Connect links your internal network to an AWS Direct Connect location over a standard 1 gigabit or 10 gigabit Ethernet fiber-optic cable.

·         One end of the cable is connected to your router, the other to an AWS Direct Connect router.

·         With this connection in place, you can create virtual interfaces directly to the AWS cloud (for example, to Amazon EC2 and Amazon S3) and to Amazon VPC, bypassing Internet service providers in your network path.

·         An AWS Direct Connect location provides access to Amazon Web Services in the region it is associated with, as well as access to other US regions. For example, you can provision a single connection to any AWS Direct Connect location in the US and use it to access public AWS services in all US Regions and AWS GovCloud (US).


To use AWS Direct Connect, your network must meet one of the following conditions:

·         Your network is collocated with an existing AWS Direct Connect location.

·         You are working with an AWS Direct Connect partner who is a member of the AWS Partner Network (APN).

·         You are working with an independent service provider to connect to AWS Direct Connect.


In addition, your network must meet the following conditions:

·         Connections to AWS Direct Connect require single mode fiber, 1000BASE-LX (1310nm) for 1 gigabit Ethernet, or 10GBASE-LR (1310nm) for 10 gigabit Ethernet.

·         Auto Negotiation for the port must be disabled. You must support 802.1Q VLANs across these connections.

·         Your network must support Border Gateway Protocol (BGP) and BGP MD5 authentication.

·         Optionally, you may configure Bidirectional Forwarding Detection (BFD).



3). Elastic Load Balancing


·         Elastic Load Balancing automatically distributes your incoming application traffic across multiple Amazon EC2 instances.

·         It detects unhealthy instances and reroutes traffic to healthy instances until the unhealthy instances have been restored.

·         Elastic Load Balancing automatically scales its request handling capacity in response to incoming traffic. 


Same as in – Compute Section.


4). Amazon Route 53


·         Amazon Route 53 is a highly available and scalable Domain Name System (DNS) web service. 


Amazon Route 53 performs three main functions:


·         Domain registration – Amazon Route 53 lets you register domain names such as

·         DNS service – Amazon Route 53 translates friendly domains names like into IP addresses like Amazon Route 53 responds to DNS queries using a global network of authoritative DNS servers, which reduces latency.

·         Health checking – Amazon Route 53 sends automated requests over the Internet to your application to verify that it's reachable, available, and functional.


You can use any combination of these functions. For example, you can use Amazon Route 53 as both your registrar and your DNS service, or you can use Amazon Route 53 as the DNS service for a domain that you registered with another domain registrar.




Arun Manglick


























02_AWS - Storage & Content Delivery

Amazon Web Services


2006: Amazon launched Amazon Web Service (AWS) on a utility computing basis although the initial released dated back to July 2002.

Amazon Web Services (AWS) is a collection of remote computing services (also called web services) that together make up a cloud computing platform, offered over the Internet by

The most central and well-known of these services are Amazon EC2 (Elastic Compute Cloud )and Amazon S3 (Simple Storage Service).



Amazon Web Services is based on SOA standards, including HTTP, REST, and SOAP transfer protocols, open source and commercial operating systems, application servers, and browser-based access.




1.       Amazon S3

2.       Amazon CloudFront

3.       Amazon EBS

4.       Amazon EFS (preview)

5.       Amazon Glacier

6.       AWS Import/Export

7.       AWS Storage Gateway



1). Amazon S3


·         Amazon S3 is storage for the Internet.

·         Amazon S3 has a simple web services interface that you can use to store and retrieve any amount of data at any time, from anywhere on the web.

·         You can accomplish these tasks using the simple and intuitive web interface of the AWS Management Console.



S3 Concepts:



·         Amazon S3 stores data as objects within buckets. An object consists of a file and optionally any metadata that describes that file.

·         Every object is contained in a bucket. For example, if the object named photos/puppy.jpg is stored in the johnsmith bucket, then it is addressable using the URL

·         When you upload a file, you can set permissions on the object as well as any metadata.

·         Buckets are the containers for objects. You can have one or more buckets.


You can configure buckets so that they are created in a specific Region.

You can also configure a bucket so that every time an object is added to it, Amazon S3 generates a Unique Version ID and assigns it to the object




·         Objects are the fundamental entities stored in Amazon S3. Objects consist of object data and metadata.

·         The data portion is opaque to Amazon S3. The metadata is a set of name-value pairs that describe the object.

·         These include some default metadata, such as the date last modified, and standard HTTP metadata, such as Content-Type.

·         You can also specify custom metadata at the time the object is stored.

·         An object is uniquely identified within a bucket by a key (Name) and a Version ID.




·         A key is the unique identifier for an object within a bucket. Every object in a bucket has exactly one key.

·         Because the combination of a bucket, key, and version ID uniquely identify each object, Amazon S3 can

·         be thought of as a basic data map between "bucket + key + version" and the object itself.

·         Every object in Amazon S3 can be uniquely addressed through the combination of the web service endpoint, bucket name, key, and optionally, a version




·         You can choose the geographical region where Amazon S3 will store the buckets you create.

·         You might choose a region to optimize latency, minimize costs, or address regulatory requirements

·         Objects stored in a region never leave the region unless you explicitly transfer them to another region




2). Amazon CloudFront


·         Amazon CloudFront is a web service that speeds up distribution of your static and dynamic web content, for example, .html, .css, .php, image, and media files, to end users.

·         CloudFront delivers your content through a worldwide network of Edge Locations.

·         When an end user requests content that you're serving with CloudFront, the user is routed to the edge location that provides the lowest latency, so content is delivered with the best possible performance.

·         If the content is already in that edge location, CloudFront delivers it immediately. If the content is not currently in that edge location, CloudFront retrieves it from an Amazon S3 bucket or an HTTP server (for example, a web server) that you have identified as the source for the definitive version of your content.



3). Amazon Elastic Block Store (Amazon EBS)


·         Amazon EBS provides Block Level Storage Volumes for use with EC2 instances.

·         EBS volumes are highly available and reliable storage volumes that can be attached to any running instance that is in the same Availability Zone.

·         EBS volumes that are attached to an EC2 instance are exposed as storage volumes that persist independently from the life of the instance.

·         With Amazon EBS, you pay only for what you use.


·         Amazon EBS is recommended when data changes frequently and requires long-term persistence.

·         EBS volumes are particularly well-suited for use as the primary storage for file systems, databases, or for any applications that require fine granular updates and access to raw, unformatted, block-level storage. Amazon EBS is particularly helpful for database-style applications that frequently encounter many random reads and writes across the data set.

·         For simplified data encryption, you can launch your EBS volumes as encrypted volumes. Amazon EBS encryption offers you a simple encryption solution for your EBS volumes without the need for you to build, manage, and secure your own key management infrastructure. When you create an encrypted EBS volume and attach it to a supported instance type, data stored at rest on the volume, disk I/O, and snapshots created from the volume are all encrypted. The encryption occurs on the servers that hosts EC2 instances, providing encryption of data-in-transit from EC2 instances to EBS storage.


·         Amazon EBS encryption uses AWS Key Management Service (AWS KMS) master keys when creating encrypted volumes and any snapshots created from your encrypted volumes. The first time you create an encrypted EBS volume in a region, a default master key is created for you automatically. This key is used for Amazon EBS encryption unless you select a Customer Master Key (CMK) that you created separately using the AWS Key Management Service. Creating your own CMK gives you more flexibility, including the ability to create, rotate, disable, define access controls, and audit the encryption keys used to protect your data.


·         You can attach multiple volumes to the same instance within the limits specified by your AWS account. Your account has a limit on the number of EBS volumes that you can use, and the total storage available to you.



4). Amazon Elastic File System (Amazon EFS)


·         Amazon EFS provides File Storage for your EC2 instances.

·         With Amazon EFS, you can create a file system, mount the file system on your EC2 instances, and then read and write data from your EC2 instances to and from your file system.

·         With Amazon EFS, storage capacity is elastic, growing and shrinking automatically as you add and remove files, so your applications have the storage they need, when they need it.

·         Multiple Amazon EC2 instances can access an Amazon EFS file system at the same time, providing a common data source for workloads and applications running on more than one instance.

·         The service is designed to be highly scalable. Amazon EFS file systems can grow to petabyte scale, drive high levels of throughput, and support thousands of concurrent NFS connections

·         Amazon EFS stores data and metadata across multiple Availability Zones in a region, providing high availability and durability.

·         Amazon EFS provides read-after-write consistency.

·         Amazon EFS is SSD-based and is designed to deliver low latencies for file operations. In addition, the service is designed to provide high-throughput read and write operations, and can support highly parallel workloads, efficiently handling parallel operations on the same file system from many different instances.


Note: Amazon EFS supports the NFSv4.0 protocol. The native Microsoft Windows Server 2012 and Microsoft Windows Server 2008 NFS client supports NFSv2.0 and NFSv3.0





5). Amazon Glacier


·         Amazon Glacier is a storage service optimized for infrequently used data, or "Cold Data."  (If your application requires fast or frequent access to your data, consider using Amazon S3)

·         The service provides durable and extremely low-cost storage with security features for data archiving and backup.

·         With Amazon Glacier, you can store your data cost effectively for months, years, or even decades.

·         Amazon Glacier enables you to offload the administrative burdens of operating and scaling storage to AWS, so you don't have to worry about capacity planning, hardware provisioning, data replication, hardware failure detection and recovery, or time-consuming hardware migrations.


·         The Amazon Glacier data model core concepts include Vaults and Archives.

·         Amazon Glacier is a REST-based web service. In terms of REST, vaults and archives are the resources.

·         In addition, the Amazon Glacier data model includes Job and Notification-Configuration resources.  These resources complement the core resources.



5). AWS Import/Export


·         AWS Import/Export is a service that accelerates transferring large amounts of data into and out of AWS using physical storage appliances, bypassing the Internet.

·         AWS Import/Export consists of

o    AWS Import/Export Snowball (Snowball), which uses on demand, Amazon-provided secure storage appliances to physically transport terabytes to many petabytes of data, and

o    AWS Import/Export Disk, which utilizes customer-provided portable devices to transfer smaller datasets.


·         AWS transfers data directly onto and off of your storage devices using Amazon’s high-speed internal network.

·         Your data load typically begins the next business day after your storage device arrives at AWS. After the data export or import completes, we return your storage device.

·         For large data sets, AWS Import/Export can be significantly faster than Internet transfer and more cost effective than upgrading your connectivity.


·         AWS Import/Export supports:

o    Import/Export to/from Amazon S3

o    Import to Amazon EBS

o    Import to Amazon Glacier



6). AWS Storage Gateway


·         AWS Storage Gateway connects an on-premises software appliance with cloud-based storage to provide seamless integration (with data security features) between your on-premises IT environment and the Amazon Web Services (AWS) storage infrastructure.

·         You can use the service to store data in the AWS cloud for scalable and cost-effective storage that helps maintain data security.

·         AWS Storage Gateway offers both Volume-Based and Tape-Based storage solutions



·         You may choose to run AWS Storage Gateway either

o    On-premises as a virtual machine (VM) appliance, or

o    In AWS, as an EC2 instance.


·         You deploy your gateway on an EC2 instance to provision iSCSI storage volumes in AWS.

·         Gateways hosted on EC2 instances can be used for disaster recovery, data mirroring, and providing storage for applications hosted on Amazon EC2.





Arun Manglick












The contents of this email and any attachments may include confidential information. Therefore, they may not be disclosed to, used by, or copied in any way by anyone other than the intended recipient and any such disclosure, use or copy can be treated as illegal. In the case this email is sent to you in error, please inform the sender and delete this email.

06_AWS - Management Tools

Amazon Web Services


2006: Amazon launched Amazon Web Service (AWS) on a utility computing basis although the initial released dated back to July 2002.

Amazon Web Services (AWS) is a collection of remote computing services (also called web services) that together make up a cloud computing platform, offered over the Internet by

The most central and well-known of these services are Amazon EC2 (Elastic Compute Cloud )and Amazon S3 (Simple Storage Service).



Amazon Web Services is based on SOA standards, including HTTP, REST, and SOAP transfer protocols, open source and commercial operating systems, application servers, and browser-based access.




1.       Amazon CloudWatch

2.       AWS CloudFormation

3.       AWS CloudTrail

4.       AWS Command Line Interface

5.       AWS Config

6.       AWS Management Console

7.       AWS OpsWorks

8.       AWS Service Catalog

9.       Trusted Advisor

10.   AWS Tools for Windows PowerShell



1). Amazon CloudWatch


·         Amazon CloudWatch monitors your AWS resources and applications you run on AWS in real-time.

·         Amazon CloudWatch is a web service that enables you to collect, view, and analyze Metrics. 

·         Amazon CloudWatch is basically a Metrics Repository. An AWS product—such as Amazon EC2—puts metrics into the repository, and you retrieve statistics based on those metrics.


·         You can use metrics to collect and track metrics to:

o    To calculate statistics and then present the data graphically in the CloudWatch console.

o    Create alarms that initiate Auto Scaling and Amazon SNS actions on your behalf.

o    Gain system-wide visibility into Resource Utilization, Application Performance, and Operational Health.

o    Configure alarm actions to stop, start, or terminate an Amazon EC2 instance when certain criteria are met. For example, you can monitor the CPU usage and Disk Reads and writes of your Amazon EC2 instances and then use this data to determine whether you should launch additional instances to handle increased load. You can also use this data to stop under-used instances to save money.





Amazon CloudWatch Events

·         Use them to deliver a timely stream of system events that describe changes in AWS resources to AWS Lambda functions, streams in Amazon Kinesis Streams, Amazon SNS topics, or built-in targets.

·         Using simple rules that you can set up quickly, you can match events and route them to one or more target functions or streams.

·         CloudWatch Events becomes aware of operational changes as they happen and takes action, sending messages to respond to the environment and activating functions, making changes, capturing state information, and taking corrective action


Amazon CloudWatch Logs

·         You can use this to monitor, store, and access your log files from Amazon EC2 instances, AWS CloudTrail, or other sources.

·         You can then retrieve the associated log data from CloudWatch Logs using the Amazon CloudWatch console, the CloudWatch Logs commands in the AWS CLI, the CloudWatch Logs API, or the CloudWatch Logs SDK.



2). AWS CloudFormation


·         AWS CloudFormation enables you to create and provision AWS infrastructure deployments predictably and repeatedly.

·         It helps you leverage AWS products such as Amazon EC2, Amazon Elastic Block Store, Amazon SNS, Elastic Load Balancing, and Auto Scaling to build highly reliable, highly scalable, cost-effective applications in the cloud without worrying about creating and configuring the underlying AWS infrastructure. AWS CloudFormation enables you to use a template file to create and delete a collection of resources together as a single unit (a stack).


·         AWS CloudFormation is a service that helps you Model and Set up your AWS resources so that you can spend less time managing those resources and more time focusing on your applications that run in AWS.

·         You create a template that describes all the AWS resources that you want (like Amazon EC2 instances or Amazon RDS DB instances), and AWS CloudFormation takes care of provisioning and configuring those resources for you. You don't need to individually create and configure AWS resources and figure out what's dependent on what; AWS CloudFormation handles all of that.


AWS CloudFormation Concepts


·         Templates

·         Stacks


When you use AWS CloudFormation, you work with templates and stacks.

You create templates to describe your AWS resources and their properties.

Whenever you create a stack, AWS CloudFormation provisions the resources that are described in your template.



·         An AWS CloudFormation template is a text file whose format complies with the JSON standard. You can save these files with any extension, such as .json, .template, or .txt.

·         AWS CloudFormation uses these templates as blueprints for building your AWS resources. For example, in a template, you can describe an Amazon EC2 instance, such as the instance type, the AMI ID, block device mappings, and its Amazon EC2 key pair name.

·         Whenever you create a stack, you also specify a template that AWS CloudFormation uses to create whatever you described in the template.



·         When you use AWS CloudFormation, you manage related resources as a single unit called a stack. In other words, you create, update, and delete a collection of resources by creating, updating, and deleting stacks.

·         All the resources in a stack are defined by the stack's AWS CloudFormation template.

·         Suppose you created a template that includes an Auto Scaling group, Elastic Load Balancing load balancer, and an Amazon RDS database instance. To create those resources, you create a stack by submitting the template that you created, and AWS CloudFormation provisions all those resources for you. To update resources, you first modify the original stack template and then update your stack by submitting the modified template.






3). Amazon CloudTrail


·         With AWS CloudTrail, you can monitor your AWS deployments in the cloud by getting a history of AWS API calls for your account.

·         This includes API calls made via the AWS Management Console, the AWS SDKs, the command line tools, and higher-level AWS services.

·         You can also identify which users and accounts called AWS APIs for services that support CloudTrail, the source IP address the calls were made from, and when the calls occurred.

·         You can integrate CloudTrail into applications using the API, automate trail creation for your organization, check the status of your trails, and control how administrators turn CloudTrail logging on and off.


How it works:

·         AWS CloudTrail captures AWS API calls and delivers log files to an Amazon S3 bucket that you specify.

·         Optionally, you can configure AWS CloudTrail to deliver events to a log group to be monitored by CloudWatch Logs.

·         You can also choose to receive Amazon SNS notifications each time a log file is delivered to your bucket.

·         You can create two types of trails:

o    A trail that applies to all regions (Default)

o    A trail that applies to one region


By default, your log files are encrypted using Amazon S3 Server-Side Encryption (SSE).You can store your log files in your bucket for as long as you want, but you can also define Amazon S3 lifecycle rules to archive or delete log files automatically. CloudTrail typically delivers log files within 15 minutes of an API call. In addition, the service publishes new log files multiple times an hour, usually about every five minutes. These log files contain API calls from all of the account's services that support CloudTrail.



4). Amazon Command Line Interface


·         AWS CLI is a unified tool that provides a consistent interface for interacting with all parts of AWS.

·         AWS CLI commands for different services are covered in the accompanying user guide, including descriptions, syntax, and usage examples.



5). AWS Config Documentation


·         AWS Config provides a Detailed View of the Configuration of AWS resources in your AWS account.

·         This inlcudes

o    How they are configured,

o    How they are related to one another, and

o    How the configurations and their relationships have changed over time.


·         AWS Config supports the following AWS resources

o    Amazon EBS

o    Amazon EC2

o    Amazon VPC

o    AWS CloudTrail

o    AWS Identity and Access Management


·         With AWS Config, you can do the following:

o    Evaluate your AWS resource configurations for desired settings.

o    Get a snapshot of the current configurations of the supported resources that are associated with your AWS account.

o    Retrieve configurations of one or more resources that exist in your account.

o    Retrieve historical configurations of one or more resources.

o    Receive a notification whenever a resource is created, modified, or deleted.

o    View relationships between resources. For example, you might want to find all resources that use a particular security group.



6). AWS Management Console


·         The AWS Management Console is a Web Application for managing Amazon Web Services.

·         The console provides an intuitive user interface for performing many AWS tasks, such as working with Amazon S3 buckets, launching and connecting to Amazon EC2 instances, setting Amazon CloudWatch alarms, and so on.

·         Each service has its own console, which you can access from the AWS Management Console. The console also provides information about your account and about billing.



7). AWS OpsWorks


·         AWS OpsWorks provides a simple and flexible way to create and manage Stacks and Applications.

·         With AWS OpsWorks, you can provision AWS resources, manage their configuration, deploy applications to those resources, and monitor their health.


Cloud-based computing usually involves groups of AWS resources, such as Amazon EC2 instances and Amazon RDS instances, which must be created and managed collectively. For example, a web application typically requires application servers, database servers, load balancers, and so on. This Group Of Instances is typically called a Stack; a simple application server stack might look something like the following.





In addition to creating the instances and installing the necessary packages, you typically need a way to distribute applications to the application servers, monitor the stack's performance, manage security and permissions, and so on.


AWS OpsWorks provides a simple and flexible way to create and manage stacks and applications. Here's how a basic application server stack might look with AWS OpsWorks. It consists of a group of application servers running behind an Elastic Load Balancing load balancer, with a backend Amazon RDS database server.




8). AWS Service Catalog


·         AWS Service Catalog allows IT administrators to Create, Manage, And Distribute Portfolios Of Approved Products to end users, who can then access the products they need in a personalized portal.

·         Typical products include servers, databases, websites, or applications that are deployed using AWS resources (for example, an Amazon EC2 instance or an Amazon RDS database). You can control which users have access to specific products to enforce compliance with organizational business standards, manage product lifecycles, and help users find and launch products with confidence.



·         AWS Service Catalog allows organizations to Create And Manage Catalogs of IT services that are approved for use on AWS.

·         These IT services can include everything from virtual machine images, servers, software, and databases to complete multi-tier application architectures.

·         AWS Service Catalog allows organizations to Centrally Manage Commonly Deployed IT services, and helps organizations achieve consistent governance and meet compliance requirements, while enabling users to quickly deploy only the approved IT services they need.



9). AWS Support Documentation


·         AWS Support provides support for users of Amazon Web Services. All users have access to account and billing help in the AWS Support Center.

·         In addition, customers with some support plans have access to additional features, including AWS Trusted Advisor and an API for programmatic access to support cases and Trusted Advisor.



·         AWS Support is a one-on-one, Fast-Response Support Channel that is staffed with experienced support engineers.

·         The service helps customers get the most from the products and features provided by Amazon Web Services.

·         There are four levels, or tiers, of AWS Support:

o    Basic  - Free

o    Developer,

o    Business, and

o    Enterprise.


·         The Basic tier is free of charge and offers support for account and billing questions and service limit increases.

·         The other tiers offer an unlimited number of technical support cases with pay-by-the-month pricing and no long-term contracts, providing developers and businesses flexibility to choose the level of support that meets their needs.



10). AWS Tools for Windows PowerShell Documentation


·         The AWS Tools for Windows PowerShell are a set of PowerShell cmdlets that are built on top of the functionality exposed by the AWS SDK for .NET.

·         The Tools enable you to script operations on your AWS resources from the PowerShell command line.

·         Although the cmdlets are implemented using the service clients and methods from the SDK, the cmdlets provide an idiomatic PowerShell experience for specifying parameters and handling results.

·         For example, the cmdlets for the PowerShell Tools support PowerShell pipelining—that is, you can pipeline PowerShell objects both into and out of the cmdlets.




Arun Manglick