David Eisinger

Elsewhere > Email Photos to an S3 Bucket with AWS Lambda (with Cropping, in Ruby)

Posted 2021-04-07 on viget.com

In my annual search for holiday gifts, I came across this digital photo frame that lets you load photos via email. Pretty neat, but I ultimately didn’t buy it for a few reason: 1) it’s pretty expensive, 2) I’d be trusting my family’s data to an unknown entity, and 3) if the company ever goes under or just decides to stop supporting the product, it might stop working or at least stop updating. But I got to thinking, could I build something like this myself? I’ll save the full details for a later article, but the first thing I needed to figure out was how to get photos from an email into an S3 bucket that could be synced onto a device.

I try to keep up with the various AWS offerings, and Lambda has been on my radar for a few years, but I haven’t had the opportunity to use it in anger. Services like this really excel at the extremes of web software — at the low end, where you don’t want to incur the costs of an always-on server, and at the high-end, where you don’t want to pay for a whole fleet of them. Most of our work falls in the middle, where developer time is way more costly than hosting infrastructure and so using a more full-featured stack running on a handful of conventional servers is usually the best option. But an email-to-S3 gateway is a perfect use case for on-demand computing.

The Services

To make this work, we need to connect several AWS services:

• Route 53 (for domain registration and DNS configuration)
• SES (for setting up the email address and “rule set” that triggers the Lambda function)
• S3 (for storing the contents of the incoming emails as well as the resulting photos)
• SNS (for notifying the Lambda function of an incoming email)
• Lambda (to process the incoming email, extract the photos, crop them, and store the results)
• CloudWatch (for debugging issues with the code)
• IAM (for setting the appropriate permissions)

It’s a lot, to be sure, but it comes together pretty easily:

1. Create a couple buckets in S3, one to hold emails, the other to hold photos.
2. Register a domain (“hosted zone”) in Route 53.
3. Go to Simple Email Service > Domains and verify a new domain, selecting the domain you just registered in Route 53.
4. Go to the SES “rule sets” interface and click “Create Rule.” Give it a name and an email address you want to send your photos to.
5. For the rule action, pick “S3” and then the email bucket you created in step 1 (we have to use S3 rather than just calling the Lambda function directly because our emails exceed the maximum payload size). Make sure to add an SNS (Simple Notification Service) topic to go along with your S3 action, which is how we’ll trigger our Lambda function.
6. Go to the Lambda interface and create a new function. Give it a name that makes sense for you and pick Ruby 2.7 as the language.
7. With your skeleton function created, click “Add Trigger” and select the SNS topic you created in step 5. You’ll need to add ImageMagick as a layer¹ and bump the memory and timeout (I used 512 MB and 30 seconds, respectively, but you should use whatever makes you feel good in your heart).
8. Create a couple environment variables: BUCKET should be name of the S3 bucket you want to upload photos to, and AUTHORIZED_EMAILS to hold all the valid email addresses separated by semicolons.
9. Give your function permissions to read and write to/from the two buckets.
10. And finally, the code. We’ll manage that locally rather than using the web-based interface since we need to include a couple gems.

The Code

So as I said literally one sentence ago, we manage the code for this Lambda function locally since we need to include a couple gems: mail to parse the emails stored in S3 and mini_magick to do the cropping. If you don’t need cropping, feel free to leave that one out and update the code accordingly. Without further ado:

require 'json'
require 'aws-sdk-s3'
require 'mail'
require 'mini_magick'

BUCKET = ENV["BUCKET"]
AUTHORIZED_EMAILS = ENV["AUTHORIZED_EMAILS"].split(";")

def lambda_handler(event:, context:)
  message = JSON.parse(event["Records"][0]["Sns"]["Message"])
  s3_info = message["receipt"]["action"]
  client = Aws::S3::Client.new(region: "us-east-1")

  # Get the incoming email from S3
  object = client.get_object(
    bucket: s3_info["bucketName"],
    key: s3_info["objectKey"]
  )

  email = Mail.new(object.body.read)
  sender = email.from.first

  # Confirm that the sender is in the list, otherwise abort
  unless AUTHORIZED_EMAILS.include?(sender)
    puts "Unauthorized email: #{sender}"
    exit
  end

  # Get all the images out of the email
  attachments = email.parts.filter { |p| p.content_type =~ /^image/ }

  attachments.each do |attachment|
    # First, just put the original photo in the `photos` subdirectory
    client.put_object(
      body: attachment.body.to_s,
      bucket: BUCKET,
      key: "photos/#{attachment.filename}"
    )

    thumb = MiniMagick::Image.read(attachment.body.to_s)

    # Crop the photo down for displaying on a webpage
    thumb.combine_options do |i|
      i.auto_orient
      i.resize "440x264^"
      i.gravity "center"
      i.extent "440x264"
    end

    client.put_object(
      body: thumb.to_blob,
      bucket: BUCKET,
      key: "thumbs/#{attachment.filename}"
    )

    dithered = MiniMagick::Image.read(attachment.body.to_s)

    # Crop and dither the photo for displaying on an e-ink screen
    dithered.combine_options do |i|
      i.auto_orient
      i.resize "880x528^"
      i.gravity "center"
      i.extent "880x528"
      i.ordered_dither "o8x8"
      i.monochrome
    end

    client.put_object(
      body: dithered.to_blob,
      bucket: BUCKET,
      key: "dithered/#{attachment.filename}"
    )

    puts "Photo '#{attachment.filename}' uploaded"
  end

  {
    statusCode: 200,
    body: JSON.generate("#{attachments.size} photo(s) uploaded.")
  }
end

If you’re unfamiliar with dithering, here’s a great post with more info, but in short, it’s a way to simulate grayscale with only black and white pixels like what you find on an e-ink/e-paper display.

Deploying

To deploy your code, you’ll use the AWS CLI. Here’s a pretty good walkthrough of how to do it but I’ll summarize:

1. Install your gems locally with bundle install --path vendor/bundle.
2. Edit your code (in our case, it lives in lambda_function.rb).
3. Make a simple shell script that zips up your function and gems and sends it up to AWS:

#!/bin/sh

zip -r function.zip lambda_function.rb vendor 
  && aws lambda update-function-code 
  --function-name [lambda-function-name] 
  --zip-file fileb://function.zip

And that’s it! A simple, resilient, cheap way to email photos into an S3 bucket with no servers in sight (at least none you care about or have to manage).

In closing, this project was a great way to get familiar with Lambda and the wider AWS ecosystem. It came together in just a few hours and is still going strong several months later. My typical bill is something on the order of $0.50 per month. If anything goes wrong, I can pop into CloudWatch to view the result of the function, but so far, so smooth.

I’ll be back in a few weeks detailing the rest of the project. Stay tuned!