Running a TensorFlow model on iOS and Android

Shrink the model size and reduce the computational resources needed to do the inference calculations

So you are interested in running a machine learning model on your phone, here is a quick guide on how you could do so and some of the challenges you would face along the way.

Google’s Inception model is quite huge (by mobile standards), it is about 90 MB. One reason the model is that big, is because it stores weights as 32-bit floating point. That is quite essential during the training phase to allow applying many tiny changes to the weights. But after the training is done switching to 8-bit fixed point will not have a huge impact on accuracy.

That is where quantization comes in, we quantize our model to shrink its size and reduce the computational resources needed to do the inference calculations. Moving calculations over to 8-bit will help you run your models faster, and use less power.