Deploying ruby gems with native extensions on AWS Lambda using the serverless toolkit

Iswarya Rajagopal
Francium Tech

An earlier article by Vinoth Kumar Natarajan explains how to use ruby gems with native extensions (like nokogiri) on aws lambda using AWS Serverless Application Model. In this article, I will brief how to do the same using the serverless framework.

  1. Install the serverless framework
$npm install -g serverless

2. Create a new ruby service

Let’s call our service “scrapeservice” for we are going to use “nokogiri” gem which is used to parse html/xml documents. Nokogiri is a ruby gem with a native extension and it is not straightforward to use such gems like we will use normal ruby gems.

$serverless create -t aws-ruby -p scrapeservice

3. The service you just created is already working!

Let’s change into the service directory, deploy it and invoke a function(auto-generated with the serverless create command)

But before that we need to have set aws credentials

$cd scrapeservicescrapeservice$export AWS_ACCESS_KEY_ID=<your-key-here>scrapeservice$export AWS_SECRET_ACCESS_KEY=<your-secret-key-here>scrapeservice$sls deploy
$sls invoke -f hello

Now explore the contents of the service. There is one yml file and one handler.rb

4. Let’s add our code

In handler.rb, replace the hello function with scrape function. Also add relevant require statements

create a Gemfile to use nokogiri gem

5. Building gem dependencies

Like it is explained here, building gem dependencies can be quite tricky. Lambda layers can be used for this.

With the following command, we can install and build them using a docker (lambci) which closely replicates the live AWS Lambda

docker run --rm -it -v $PWD:/var/gem_build -w /var/gem_build lambci/lambda:build-ruby2.5 bundle install --path=.

After this, you can notice that the gems are installed by the docker and are placed in our local project folder

Notice the new ruby folder that has been created.

6. Adding the gem as lambda layers

Modify the functions part in the serverless.yml file.

i) to create a layer with the built gems

path — the name of the folder that was created after we ran the docker to build the gems

gemlayer— name of the layer that you are going to create

ii) change handler.hello to handler.scrape (The function you wrote in handler.rb)

iii) have your function use the layer that you created (gemlayer)

how to specify the layer to be used by the function?

“<your-layer-name-with-capitalized-first-letter>LambdaLayer”

A mistake in this naming convention can waste a lot of time. Beware!

iv) set GEM_PATH environment variable for the lambda function. The gems usually sit inside “opt” folder

The functions and layers portion of your serverless.yml now looks like below

7. Deploy and verify if lambda function and layer is created correctly

$ sls deploy

You can see the layer in aws user interface to verify

Also, see a lambda function created and associated with this layer

and the environment variable you had set

You can now execute the function through the UI and see a success message

or invoke through the command line

$ sls invoke -f scrape

There you go.

P.S In the 7th step, I had an issue

Error: The security token included in the request is invalid.

I then had to reconfigure aws credentials using aws configuration command

Francium Tech is a technology company laser focussed on delivering top quality software of scale at extreme speeds. Numbers and Size of the data don’t scare us. If you have any requirements or want a free health check of your systems or architecture, feel free to shoot an email to contact@francium.tech, we will get in touch with you!

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

Published in Francium Tech

Francium's core purpose is to create technology solutions for progressive and forward-thinking organizations to empower their ascendancy and to magnify their impact.

Responses (2)

What are your thoughts?

It worked straight away for me. Thanks a lot!

Hello Iswarya Rajagopal,
I followed all your steps and got error at step “sls deploy”
I have named my lambda layer as certificate
it is giving me following error, can you pls help me.
TypeError: Cannot create property ‘package’ on string…...