The model now supports privacy.
Trusted Execution EnvironmentTrusted Execution Environments (TEE) provide a hardware solution to computing algorithms securely.
They are an isolated part of CPU that other hardware cannot access and the integrity of the computation can be attested to.
There are a few different frameworks that support TEE.
Intel has a framework called Secure Guard Extensions (SGX).
AMD provides a framework called TrustZone.
To get the House Credit Default model running inside a TEE we use a few different frameworks:AsyloIntel SGXTensorflowTensorflow LiteAsylo is an abstraction that allows us to easily compile programs to run on Intel SGX and other TEEs.
Asylo has an easy to use simulator that allows programs to be tested in the simulator before deploying to Intel SGX machines (which can be hard to find).
They also provide a docker container so that we don’t need to install anything when we’re first running inside of the simulator.
Tensorflow is used to delegate the operations to the TEE.
The motivation here is that you could be processing a larger computation graph where some of the operations will happen in the TEE and some will happen on the CPU.
For our use case, we’re running a full computation graph inside the TEE.
A Tensorflow custom operation is used to connect to the Asylo stack using gRPC which then makes calls to the Tensorflow Lite library in C++.
Tensorflow Lite is able to load the model and inputs, then evaluate the model and return the output.
We had to patch Tensorflow Lite slightly so that the Asylo toolchain was able compile it and execute it inside the Intel SGX simulator and Intel SGX device.
All of this is wrapped up in framework we’ve developed called tf-trusted.
It provides an easy way to use all of these components to run most Tensorflow-based models privately.
To get started running this model on the Intel SGX simulator you should follow the instructions in the tf-encrypted repository examples directory located here.
Benchmark ComparisonWe have benchmarked Tensorflow and Tensorflow Encrypted on Google Cloud Platform.
Both platforms ran on a single CPU with 30 GB of memory.
Due to availability of Intel SGX machines we ran tf-trusted on Microsoft Azure.
Running in plaintext is our baseline, but we can see that both tf-trusted and TFE scale similarly with about one order of magnitude slowdown.
Interestingly, as the batch size increases, TFE becomes more efficient than SGX.
We attribute this switchover due to the significant amounts of work put into Tensorflow by their dev team (TFE runs as plain tensorflow graphs), while SGX doesn’t yet scale as well.
Though we have not included it here, we can improve TFE’s computational performance by using more hardware when either the batch size or model complexity increases.
ConclusionAs we have learned, deploying a model in a secure, privacy-preserving manner does not incur a prohibitive amount of overhead.
There are several known optimizations for both TEEs and TFE that are left for future work.
Frameworks like Slalom can increase model performance in a TEE by outsourcing expensive math operations to GPU while maintaining privacy.
In TFE, there are other protocols to be leveraged and experimented with.
As we saw in our previous post, secure computation has come a long way in 2018 and we expect improvements to keep coming in 2019 and beyond.
Private computation has a reputation of being too slow to be practical.
As we have seen here, there are many use cases today that can both benefit from privacy and be privately deployed with an acceptable overhead.
Many of the applications we use today have stages that take well beyond a few milliseconds to perform their duty.
We are excited about the future where technologies like MPC, HE, TEEs will be fast enough to have feature parity with their plaintext counterparts, but for a broad spectrum of applications, that future is already here.
About Dropout LabsWe are a team of machine learning engineers, software engineers, and cryptographers spread across the United States, France, and Canada.
We’re working on secure computation to enable training, validation, and prediction over encrypted data.
We see a near future where individuals and organizations will maintain control over their data, while still benefiting from cloud-based machine intelligence.
Follow Dropout Labs on Twitter and tf-encrypted on GitHub.
If you’re passionate about data privacy and AI, we’d love to hear from you.
.. More details