Who moved my binaries?

In particular, we have seen how wanting to exploit Tensorflow and Keras libraries, I was forced to include parts of code in a given programming language (eg. Python) in what previuosly was an application principally consisting in modules written for a different technology stack (in my case, Java).What is missing so far is what to do with the different pieces of binary artefacts that we get as a consequence of this write-and-build process..This phase too is affected by the polyglot nature of our application (that is, the fact that different components are written in different programming language), since what results from it consists in a number of binary packages in different, incompatible formats: .jar files for java modules, .whl (“wheels”) for python, and so on and so forth.We’ll need to take care of this..I just like to use different terminology to distinguish the former case, where the functionality is developed with future reuse in mind, from any software module developed to run as a very specific part of an application, maybe conforming to some given component model (think an EJB providing some specific business functionality in a e-commerce application, a Flask python REST service, etc.).Ready-to-deploy executables..Roughly speaking, we can group the above binaries into two groups, according to their intended usage.A first group is composed by those binaries that we need in our code development process as dependencies — pieces of software we bundle in our application code to build other binaries from..What we need in this case is a binary repository manager — a dedicated server application designed to store, version and provide binary components..Artifactory is, too, mainly Maven-oriented, but it is advertised as “the only Universal Repository Manager supporting all major packaging formats, build tools and CI servers” — that includes standard Python tools, such as pip and distutils (though I guess that if you’re working exclusively with Python you could get away with just installing your private PyPI server on your CI env).As for the second group , once youìve built your application or service, you need to keep the finished “executable” bundles ready to be deployed and configured in some target runtime environment..Let’s wrap up what we have so far: We have set up a number of software pieces of “machinery” in our software factory, a source code versioning server (gitlab), a binary repository (nexus, artifactury, …), some model server and a repository for deployable docker containers.What’s still missing is some “glue” that keeps everything together..With this I mean something that allows to go from writing code to building to testing to deploying to … whatever, without having to manually interact with each of the software tools we’ve talked about: namely, we need some kind of Continuous Integration engine (like Jenkins), that helps to automate the whole process.. More details

Leave a Reply