Today Icelab Learned
 about aws

Creating an IAM user for access to an S3 bucket

When you create an S3 bucket for a web app to use, you should create a specific IAM user to grant access to that bucket.

To do this, first create a user:

  1. Visit the IAM section of the AWS console
  2. Go to the “Users” section and click the “Create New Users” button
  3. Enter the user name(s) you want to create (e.g. my-bucket-s3 for a bucket named my-bucket)
  4. When you’re shown the user credentials, save them somewhere safe, like our team 1Password vault
  5. When you return to the IAM users list, go to the newly created user and click on their “Permissions” tab. Click on the “Inline Policies” section and create a new policy
  6. Choose to create a “Custom Policy”
  7. Name the policy “my-bucket-s3-access” (or something similarly descriptive) and paste the following into the “Policy Document” area (replacing my-bucket with your bucket’s name):

    {
      "Statement": [
        {
          "Effect": "Allow",
          "Action": "s3:ListAllMyBuckets",
          "Resource": "arn:aws:s3:::*"
        },
        {
          "Effect": "Allow",
          "Action": "s3:*",
          "Resource": [
            "arn:aws:s3:::my-bucket",
            "arn:aws:s3:::my-bucket/*"
          ]
        }
      ]
    }
    
  8. Click “Apply Policy”

The access policy should now be active and your app should be able to access the bucket with the new IAM user’s credentials.

Using CloudFront as ‘origin pull’ (with bonus gzip)

CloudFront can act as a CDN that ‘pulls’ content from an origin server. It’s really easy to set up, you simply set the ‘Origin Domain Name’ to wherever you’re hosting your app: foobar.herokuapp.com for example and then … nothing. That’s all you need to do. Requests that come through CloudFront will pass through to your origin server and then be cached thereafter.

For bonus points it’ll also gzip your content for you without you having to do much work at all. Set ‘Compress Objects Automatically’ to ‘Yes’ and you’re done. If CloudFront can serve a gzipped version (and the client can receive one) it’ll serve up a compressed version of your file instead.

Authenticating with AWS Elasticsearch

The AWS Elasticseach service offers authentication via an IAM user, or by whitelisting IPs.

Here’s how to use IAM credentials to sign requests to the service when using Faraday and how to hook that into the Ruby elasticsearch gem.

To sign requests using Faraday, you can use a gem called faraday_middleware-aws-signers-v4, which provides a middleware that will sign your requests.

require 'faraday_middleware'
require 'faraday_middleware/aws_signers_v4'

conn = Faraday.new(url: 'address-of-your-AWS-es-service') do |faraday|
  faraday.request :aws_signers_v4, {
    credentials: Aws::Credentials.new(ENV['AWS_ACCESS_KEY_ID'], ENV['AWS_SECRET_ACCESS_KEY']),
    service_name: 'es',
    region: 'ap-southeast-2'
  }

  faraday.adapter :typhoeus
end

To get the client provided by the elasticsearch gem to use your Faraday configuation, you can pass that configuration to it like so:

faraday_config = lambda do |faraday|
  faraday.request :aws_signers_v4, {
    credentials: Aws::Credentials.new(
      ENV["ELASTICSEARCH_AWS_ACCESS_KEY_ID"],
      ENV["ELASTICSEARCH_AWS_SECRET_ACCESS_KEY"]
      ),
      service_name: "es",
      region: ENV["ELASTICSEARCH_AWS_REGION"]
    }
    faraday.adapter :typhoeus
  end

elasticsearch_host_config = {
  host:   ENV["ELASTICSEARCH_HOST"],
  port:   ENV["ELASTICSEARCH_PORT"],
  scheme: ENV["ELASTICSEARCH_SCHEME"]
}

transport = Elasticsearch::Transport::Transport::HTTP::Faraday.new(hosts: [elasticsearch_host_config], &faraday_config)

client = Elasticsearch::Client.new(transport: transport)

You can then use the client object as usual, and you’ll get automatically signed requests.