Amazon upload

Django S3

Time Limited Signed UR–how-to-generate-url-for-amazon-s3-files.html

AWS SDK for Python (Boto)

sign URLs with an IP


session based authorization

download private file


Amazon EC2

Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud.

It is designed to make web-scale cloud computing easier for developers.

Amazon EBS

Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes

for use with Amazon EC2 instances in the AWS Cloud.

Each Amazon EBS volume is automatically replicated within its Availability Zone to protect you from component failure,

offering high availability and durability. Amazon EBS volumes offer the consistent

and low-latency performance needed to run your workloads.

With Amazon EBS, you can scale your usage up or down within minutes –

all while paying a low price for only what you provision.

How to serve your media files via Amazon’s Simple Storage Service

AWS RDS Postgres DB instance

Deploying a Django app on Amazon EC2 instance—amazon_app

Amazon ECS

Identity and Access Management

Before the Amazon ECS agent can register container instance into a cluster, the agent must know which account credentials to use.

You can create an IAM role that allows the agent to know which account it should register the container instance with.

When you launch an instance with the Amazon ECS-optimized AMI provided by Amazon using this role, the agent automatically registers the container instance into your default cluster.

The Amazon ECS container agent also makes calls to the Amazon EC2 and Elastic Load Balancing APIs on your behalf, so container instances can be registered and deregistered with load balancers. Before you can attach a load balancer to an Amazon ECS service, you must create an IAM role for your services to use before you start them.

This requirement applies to any Amazon ECS service that you plan to use with a load balancer.

Amazon EC2 Role for EC2 Container Service Role to allow EC2 instances in an Amazon ECS cluster to access Amazon ECS.

Distance To Vancouver From Oregon is: 1692 miles / 2723.01 km / 1470.31 nautical miles

Distance To Virginia From Vancouver is: 1725 miles / 2776.12 km / 1498.98 nautical miles

Distance To Vancouver From California is: 2403 miles / 3867.25 km / 2088.15 nautical miles



aws configure aws ecr get-login –region us-east-1

Effective today, Amazon ECR is available in US East (Northern Virginia) with more regions on the way soon!

Your Amazon ECS tasks run on container instances (Amazon EC2 instances that are running the ECS container agent).

A service lets you specify how many copies of your task definition to run. You could also use Elastic Load Balancing to distribute incoming traffic to your tasks. Amazon ECS keeps that number of tasks running and coordinates task scheduling with the load balancer.

Task definitions specify the container information for your application, such as how many containers are part of your task, what resources they will use, how they are linked together, and which host ports they will use

After you create a cluster, you can launch container instances, and then run tasks


If a storage volume on your primary fails in a Multi-AZ deployment, Amazon RDS automatically initiates a failover to the up-to-date standby. Compare this to a Single-AZ deployment: in case of a Single-AZ database failure, a user-initiated point-in-time-restore operation will be required. This operation can take several hours to complete, and any data updates that occurred after the latest restorable time (typically within the last five minutes) will not be available.

EC2 Container Service

$ sudo apt-cache search awscli
awscli - Universal Command Line Environment for AWS

$ sudo apt-get install awscli

$ aws --version
aws-cli/1.10.1 Python/3.5.1+ Linux/4.4.0-1-amd64 botocore/1.3.23

$ aws configure
AWS Access Key ID []: ****************
AWS Secret Access Key []: ****************
Default region name [oregon]: us-west-2
Default output format [json]:

$ aws iam list-users

$ aws ecs create-cluster help
$ aws ecs list-container-instances help

$ aws ecs create-cluster --cluster-name demo-01
    "cluster": {
        "pendingTasksCount": 0,
        "runningTasksCount": 0,
        "clusterName": "demo-01",
        "status": "ACTIVE",
        "clusterArn": "arn:aws:ecs:us-west-2:642913345125:cluster/demo-01",
        "activeServicesCount": 0,
        "registeredContainerInstancesCount": 0

$ aws ecs list-container-instances --cluster demo-01

Within ECS, you create task definitions, which are very similar to a docker-compose.yml file. A task definition is a collection of container definitions, each of which has a name, the Docker image to run, and options to override the image’s entrypoint and command. The container definition is also where you define environment variables, port mappings, volumes to mount, memory and CPU allocation, and whether or not the specific container should be considered essential, which is how ECS knows whether the task is healthy or needs to be restarted.

You can set up multiple container definitions within the task definition for multi-container applications. ECS knows how to pull from the Official Docker Hub by default and can be configured to pull from private registries as well. Private registries, however, require additional configuration for the Docker client installed on the EC2 host instances.

Once you have a task definition, you can create a service from it. A service allows you to define the number of tasks you want running and associate with an Elastic Load Balancer (ELB). When a task maps to particular ports, like 443, only one task instance can be running per EC2 instance in in the ECS cluster. Therefore, you cannot run more tasks than you have EC2 instances. In fact, you’ll want to make sure you run at least one less task than the number of EC2 instances in order to take advantage of blue-green deployments. Task definitions are versioned, and Services are configured to use a specific version of a task definition.

Kubernetes groups containers into units called pods, which run on physical or virtual hosts called nodes. Collections of nodes that work together to support a pod deployment, and its associated applications, are called clusters.

he core components of Kubernetes

Kubernetes master components include:

Kube-apiserver. The front end of the control plane that exposes Kubernetes APIs to cluster nodes and applications. Etcd. The Kubernetes data plane, in the form of a key-value store that manages cluster-specific but not application data. Kube-scheduler. Monitors resource usage on a cluster and assigns workloads, in the form of Kubernetes pods, to one or more worker nodes based on specified policies about hardware usage, node-pod affinity, security and workload priority. Kube-controller-manager. Runs the controller processes responsible for node monitoring, replication, container deployment and security policy enforcement. Cloud-controller-manager. A feature that primarily service providers use to run cloud-specific control processes.

Kubernetes worker node components include:

Kubelet. An agent that runs on each worker node. Kube-proxy. Manages network communication between cluster nodes. Container runtime. The engine that runs containers and maintains workload isolation within the OS.

aws instance types

Max number of pods per node

Background Jobs with AWS

RDS DB Parameter Group Family

$ aws rds describe-db-engine-versions --query "DBEngineVersions[].DBParameterGroupFamily"

Available PostgreSQL database versions

aws rds describe-db-engine-versions --default-only --engine postgres

Kubernetes storage provisioner

The difference lies in the type of Kubernetes storage provisioner used:

  1. provisioner=””: This is the provisioner for the AWS Elastic Block Store (EBS) volumes using the in-tree volume plugin.

In older versions of Kubernetes, this in-tree plugin was the standard way to provision EBS volumes. However, in recent Kubernetes versions (1.19 and later), in-tree plugins are being deprecated, and users are encouraged to use CSI (Container Storage Interface) drivers.

  1. provisioner: This is the provisioner for the AWS EBS volumes using the CSI driver.

CSI is a standardized interface that allows external storage providers to implement storage plugins for Kubernetes. With the CSI driver, storage vendors can develop and maintain their plugins independently, without relying on in-tree plugins, making it more extensible and easier to maintain.

In summary, provisioner=”” is the legacy in-tree provisioner for AWS EBS, while provisioner: is the CSI driver provisioner for AWS EBS, which is the recommended approach for newer Kubernetes versions. If possible, it’s better to use CSI drivers as they offer better flexibility, compatibility, and future-proofing for your Kubernetes storage needs.

CSI driver for Amazon EBS

CSI Driver for Amazon EFS

Amazon Elastic Cache Security Groups

Q: How do I control access to Amazon ElastiCache?

When not using VPC, Amazon ElastiCache allows you to control access to your clusters through Cache Security Groups. A Security Group acts like a firewall, controlling network access to your cluster. By default, network access is turned off to your clusters. If you want your applications to access your cluster, you must explicitly enable access from hosts in specific EC2 security groups. This process is called ingress.

Amazon EKS IAM Policies, Roles, and Permissions

Fluent Bit CloudWatch plugin





Access logs for Application Load Balancer

NODE AMI ID (Images)

Amazon EKS optimized Amazon Linux AMI IDs (Amazon Machine Image)

Ubuntu on Amazon Elastic Kubernetes Service (EKS)

These images are customised specifically for the EKS service, and are not intended as general OS images.

Resource limit

Difference between Cluster Security Group vs Node Security Group

In Amazon EKS, there are two types of Security Groups are typically associated with an EKS cluster:

  • ClusterSecurityGroup: This is attached to the control plane of the EKS cluster. It’s used to manage access to the Kubernetes API Server.

  • NodeSecurityGroup: This is attached to the EKS worker nodes (and managed node groups). It controls network access to worker nodes in the cluster.

These two different types of security groups provide different levels of control over the network access for your EKS cluster and worker nodes.

Why Use Fluent Bit for Sending Logs to Loki?

Send SES emails from a specific sender using AWS IAM Policy

Restricting the “From” Address


Grant Access To Only One S3 Bucket using AWS IAM Policy

    "Version": "2012-10-17",
    "Statement": [
            "Effect": "Allow",
            "Action": [
            "Resource": "arn:aws:s3:::*"
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [


Internet-facing Classic Load Balancers

Why does ELB need one public IP address for each public subnet?

What’s the source IP address of the traffic that Elastic Load Balancing sends to my web servers?


fields @timestamp, @message
| filter @logStream = 'app-web'
| filter @message like ""
| parse @message 'log":"* - - [*] \"* * *\" * * * \"*",' as  ip, time, method, path, http, status, _, __,ua
| DISPLAY ip, time, method, path, http, status, ua
| sort @timestamp desc
| limit 25
fields @timestamp, @message
| parse @message 'log":"* - - [*] \"* * *\" * * * \"*",' as  ip, time, method, path, http, status, _, __,ua
| filter ispresent(ip)
| stats count() as count by ip
| sort count desc
fields @timestamp, @message
| parse @message 'log":"* - - [*] \"* * *\" * * * \"*",' as  ip, time, method, path, http, status, _, __,ua
| stats count() as count by status, method, path
| sort count desc
fields @timestamp, @message
| parse @message 'log":"* - - [*] \"* * *\" * * * \"*",' as  ip, time, method, path, http, status, _, __,ua
| filter ispresent(status)
| filter status == "500"
| DISPLAY @logStream, ip, time, method, path, http, status, ua
| sort status desc