How to build your own secure image processing service with Imaginary and KubernetesRoffeBlockedUnblockFollowFollowingJan 13Some time ago I was asked to look at the Resizing Images with Amazon CloudFront & Lambda@Edge article and see if it was something that could be useful.
I had some thoughts about it as it involved lambda functions that would be executed on every image request & reading and writing files to a s3 bucketLambda counts a request each time it starts executing in response to an event notification or invoke call, including test invokes from the console.
You are charged for the total number of requests across all your functionsDuration is calculated from the time your code begins executing until it returns or otherwise terminates, rounded up to the nearest 100ms.
The price depends on the amount of memory you allocate to your functionLambda@Edge functions are metered at a granularity of 50msWe would also be stuffing the S3 drive with more and more images for each resolution requested as they would be written back to storage by the Lambda function after image processing & serving itWith some spare CPU standing and burning money, wouldn’t it be cool to process our own images with it?I found Imaginary, It had all the docker stuff already done so putting it in Kubernetes was a breeze and so this PoC was bornImaginaryFast HTTP microservice written in Go for high-level image processing backed by bimg and libvips.
imaginary can be used as private or public HTTP service for massive image processing with first-class support for Docker & Heroku.
It's almost dependency-free and only uses net/http native package without additional abstractions for better performanceSupports multiple image operations exposed as a simple HTTP API, with additional optional features such as API token authorization, URL signature protection, HTTP traffic throttle strategy and CORS support for web clientsimaginary can read images from HTTP POST payloads, server local path or remote HTTP servers, supporting JPEG, PNG, WEBP, and optionally TIFF, PDF, GIF and SVG formats if libvips@8.
3+ is compiled with proper library bindingsimaginary is able to output images as JPEG, PNG and WEBP formats, including transparent conversion across themimaginary also optionally supports image placeholder fallback mechanism in case of image processing error or server error of any nature, therefore an image will be always returned by the server in terms of HTTP response body and content MIME type, even in case of error, matching the expected image size and format transparentlyIt uses internally libvips, a powerful and efficient library written in C for image processing which requires a low memory footprint and it's typically 4x faster than using the quickest ImageMagick and GraphicsMagick settings or Go native imagepackage, and in some cases it's even 8x faster processing JPEG imagesTo get started, take a look the installation steps, usage cases and API docsDeploying it to KubernetesMy deployment for Imaginary looks like thisThere is a lot more options if you consult the Imaginary docsAnd we need a service to be able to connect a ingress to itAnd top it off with a ingress ( Don’t forget to update the domain )Signing image url’sThe documentation for Imaginary contains a good example in Go how to sign your URL’s (don’t forget the order of the request fields)Here is my version with a little twist to give me a better url for copy/pastingOnce applied to your cluster you should be able to test your image service by loading uphttp://<your service url>/resize?nocrop=true&type=jpeg&url=https%3A%2F%2Fwww.
google.
com%2Flogos%2Fdoodles%2F2015%2Fgoogles-new-logo-5078286822539264.
3-hp2x.
gif&width=200&sign=9Rawdy5gEmqGTRgxUOO7fFMYegivSQd1jNSuJljY2PMNote that the example URL above assumes you use the signKey in my example.
If you used a different key you need to generate a new URL signatureCache it all!If you are on AWS one can put a CloudFront distribution in front of the whole setup and leverage image caching at the edge to take a lot of load away from the backend.
Image operations would only happen when the images falls out of the cache due to LRU so the more popular image, the less it would have to be requested from the backendI’d imagine Google has some service similar to CloudFront and CloudFlare would probably be able to front the service as wellClosing wordsI casually reference the ingress manifest in this article.
In reality it can be a quiet complex setup to make ingress in Kubernetes to work depending on your environment and I suggest to spend the time to properly research which solution will be best in your usage caseI personally recommend Skipper from Zalando.
It’s serving me well in our production environment at work.
Which you btw can read about here.. More details