We are looking for a Machine Learning SW expert to improve our deployment from Python to mobile runtime stream. This work focuses on exporting the developed ML models from Python development environment to our mobile runtime environment (C++). There are two main areas of focus, model export to mobile format and then mobile ML model execution on device. This requires compilation and integration into our software product’s runtime environment.

The work includes evaluation of ARM NN SDK (C++) features regarding Onnx and TFLite support and limitations as well as performance evaluation on converted models. A study on the workload for runtime conversion of PyTorch in Android AOSP environment is needed. Also, the work effort required for maintenance (version updates) and dependency identification is part of this work. An important part of the work consists of integrating ARM NN SDK into our existing C++ implemented solution. This includes changes required to make our SW product’s SDK API and its compilation and testing with our ML models suitable. The goal is to ensure modest workload for version updates with automation of testing and compilation, especially in Android Open Source Project (AOSP) project.

The most important part of the work is PyTorch -> mobile conversion and runtime environment design and implementation. The current implementation flow from TensorFlow to TFLite model in TFLite runtime is to be changed to a flow from PyTorch to mobile optimized model. This work includes selection of suitable format at Python level (e.g. Onnx, TFLite, PyTorch mobile optimized) as well as evaluation and creation of conversion path from PyTorch to target format. There are several options, each with different constraints related to e.g. quality or predictable execution speed. Also, model optimization, such as quantization, is needed as part of the conversion flow. Compilation and integration against our SW product’s SDK API and testing with our ML models is to be done to verify the conversion.

The recommended locations are Tampere or Espoo. The assumption is that most of the work can be done remotely, but network access requires a laptop managed by the client. We will provide the devices required for the work.

We estimate that a skilled expert could complete this work by the end of 2023, if the start would be in August.

After successful completion of this project, there may be new needs for a possible follow-up project.

Skills and qualifications required Strong practical experience in Maching Learning development workflow Deep understanding of Onnx, TFLite and PyTorch ML development environments Good understanding of common DNN models model architecture and their parameters in each target ML development environment Excellent C++ SW development skills Understanding of Android AOSP Build engineering skills (CMake, Soong/Bazel for AOSP) Experience with ARM NN SDK Experience in implementing and testing ML-based solutions on ARM with Android OS and HW device Very good general software development skills with experience in using GitLab/GitHub and CI/CD