HEIC converter on AWS Lambda

HEIC converter on AWS LambdaVitalii ElenhauptBlockedUnblockFollowFollowingFeb 16In this story, you will find how to create a sample AWS Lambda which converts HEIF images to JPEG.

We will build static binaries and properly configure AWS Lambda runtime to use it.

Also, we will use docker to locally run and test our lambda function.

HEIC converter on AWS LambdaHEIF stands for High Efficient Image Format.

It’s a modern image format that is a replacement for JPEG, and often has the .

HEIC file extension.

The HEIF format produces images with a smaller file size and higher image quality than the older JPEG standard.

In other words, HEIF is just better than JPEG.

ImageMagick supports HEIF format, so that’s possible to do a simple conversion via NodeJS (or any other runtimes).

The main challenge here is that the default installation of ImageMagick doesn’t support HEIF, thus it is not available at AWS Lambda runtime by default.

Fortunately, we are able to compile a needed binary and upload it together with the lambda function.

Since we want to convert HEIF images using AWS Lambda, we will need to compile those binaries in the same environment lambda will be running on (and it’s Linux AMI).

Environment setupAt first, we need to create a folder for our lambda function:$ mkdir aws-heic-converter && cd aws-heic-converterOf source, the best way to emulate AWS Lambda runtime is to create a new EC2 instance of Linux AMI and SSH there.

However, in most situation, you don’t need that.

The alternative is docker-lambda — a sandboxed local environment that replicates the live AWS Lambda environment almost identically.

So let's just spin up a new container with latest, for now, NodeJS runtime:$ docker run -it -v $PWD:/var/task lambci/lambda:build-nodejs8.

10 bashThe command above pulls the pre-built docker image and opens a shell.

Yeah, it is just that simple!We can quickly test our environment and ensure we can build our binaries:$ uname -aLinux 477c92edf189 4.

9.

125-linuxkit #1 SMP Fri Sep 7 08:20:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux$ whoamiroot$ make -versionGNU Make 3.

82Built for x86_64-redhat-linux-gnuCopyright (C) 2010 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later <http://gnu.

org/licenses/gpl.

html>This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.

Note: AWS Lamba runtime (and this docker-emulated environment) already hasImage Magick installed as well as some dependencies (libpng, libjpeg etc.

) This simplifies the work we need to do since we just need to add support of libheif.

Build libheif & libde265To be able to work with HEIF images, ImageMagick delegates it’s work to libheif library, which on itself depends on libde265 h.

265 codec implementation.

That means we will need to build both those libraries first before building new ImageMagick binary.

We mounted the lambda source directory to /var/task, which’s where the lambda function deployed via AWS Console.

Let’s create build folder there to download and track downloaded assets:$ cd /var/task && mkdir build && cd buildNow, download the libde265 release archive and following readme instructions, compile and install it:$ curl -L https://github.

com/strukturag/libde265/releases/download/v1.

0.

3/libde265-1.

0.

3.

tar.

gz | tar zx$ cd libde265-1.

0.

3$ .

/autogen.

sh && .

/configure && make && make install$ cd .

/The next one is libheif.

The procedure is almost identical, however, we need to provide build flags to let the linker find libde265 library we just built:$ curl -L https://github.

com/strukturag/libheif/releases/download/v1.

3.

2/libheif-1.

3.

2.

tar.

gz | tar zx$ cd libheif-1.

3.

2$ export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig$ export LDFLAGS=-L/usr/local/lib$ export CPPLAGS=-I/usr/local/include/libde265$ .

/autogen.

sh && .

/configure && make && make install$ cd .

/Both libraries are built and placed to /usr/local/lib.

ImageMagick will need only two shared libraries (technically speaking, only one — libheif.

so , but it depends on libde265, so it will require another shared library: libde265.

so.

0).

Let’s copy needed ones to our mounted directory into /var/task/lib:$ cd /var/task && mkdir lib$ cp /usr/local/lib/libde265.

so.

0 lib/$ cp /usr/local/lib/libheif.

so.

1 lib/Build ImageMagick with HEIF supportWe have built required dependencies so let’s finally build the image magick static binary.

To do that we are going to follow the same approach as before, but with three small differences:Provide build flags to let the linker find the libheif library.

Use –enable-static=yes flag to build a static binary.

Pass –with-heic flag, which is needed to enable HEIF support.

$ cd /var/task/build$ curl -L https://imagemagick.

org/download/ImageMagick-7.

0.

8-27.

tar.

gz | tar zx$ cd ImageMagick-7.

0.

8-27$ export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig$ export LDFLAGS=-L/usr/local/lib$ export CPPLAGS=-I/usr/local/include/libheif$ .

/configure –prefix=/etc/imagemagick –enable-shared=no –enable-static=yes –with-heic$ make && make installInstallation is done to/etc/imagemagick directory.

Let’s copy binaries to our lambda directory and verify whether it works and supports HEIF (a list of supported features at the bottom):$ cp -r /etc/imagemagick/bin /var/task/$ /var/task/bin/magick -versionVersion: ImageMagick 7.

0.

8-27 Q16 x86_64 2019-02-15 https://imagemagick.

orgCopyright: © 1999-2019 ImageMagick Studio LLCLicense: https://imagemagick.

org/script/license.

phpFeatures: Cipher DPC HDRI OpenMPDelegates (built-in): bzlib fontconfig freetype heic jng jpeg lcms lzma png tiff x xml zlibPerfect, it works.

However, to be sure it is able to convert HEIF images we need to try converting one.

So we could copy some HEIF image to our lambda source directory, re-run the container (with mounted source folder) and run magick convert command:$ /var/task/bin/magick convert assets/example.

heic example.

jpegNow we can safely remove build assets from our source folder.

Only the binary and library once should left:$ cd /var/task$ rm -rf build$ lsbin libWriting LambdaWhat should the lambda function do?.To simplify the things, let it just convert HEIF image example which is existed on the filesystem to JPEG.

To do that we are going to use gm node package, which is simply a wrapper over ImageMagick.

The code to convert images is quite simple: read and write an image with a different extension.

You can follow a huge list of examples in the readme if you want to do more than that.

But, if you remember, we need to use a pre-built ImageMagick binary, which is available in bin folder and depends on libraries in lib folder.

Fortunately, AWS Lambda allows changing runtime environment variables, so we can append paths to our binary and libs to the environment.

Let’s see how the hole lambda looks:Let’s use docker-lambda again to test what we built.

docker run –rm -v $PWD:/var/task lambci/lambda:nodejs8.

10 index.

handler '{"img": "assets/example.

heic"}'A couple of things to note here:We passed the path to the image as an argument to the function (in json format): '{"img": "assets/example.

heic"}'2.

Lambda took 400 ms of time and used 33 MB of memory.

3.

Our image is successfully converted and example.

jpeg is created.

WrapupIn this article we did:Built libde265 shared library, which is h.

265 codec implementation and is a libheif’s dependency.

Built libheif shared library, which is HEIF file format decoder and is ImageMagick’s dependency.

Built ImageMagick static binary with HEIF support.

Wrote AWS Lambda function, which uses pre-built ImageMagick static binary and converts HEIF images to JPEG.

Run and tested built AWS Lambda function locally in the emulated docker environment.

This was just an example of how powerful infrastructure over Lambdas is and what customizations could be done the let it quickly work.

The sample repo can be found on GitHub.

Hope you found it helpful.

Cheers!.

. More details

Leave a Reply