Multi-Task Neural Network for Medical Diagnosis Targeted for FPGA Deployment

In modern medical practice, automated processing of medical images plays a critical role; however, deploying artificial intelligence systems on portable equipment faces significant computational resource constraints. This paper addresses the challenge of deploying deep learning algorithms on embedded systems, Field-Programmable Gate Arrays (FPGAs), by applying the Multi-Task Learning (MTL).Instead of utilizing separate models for each task, a unified neural network architecture with hard-shared parameters is proposed. The MobileNetV2 network was selected as the backbone, serving as a feature extractor. For each specific task – pneumonia detection on X-ray images and brain tumor detection on MRI scans – a separate head is allocated. Furthermore, to dynamically adapt features extraction depending on the input image type a Task Embedding layer is added before the heads.To optimize the neural network training process, Automatic Mixed Precision (AMP) technology and data loading pipeline optimization were employed. To enhance model generalization, complex geometric and photometric data augmentation was applied according to Inductive Bias principle. For further adaptation of the model to FPGA, a method of model quantization from FP32 to INT8 format was introduced.Experimental results confirm that the proposed approach ensures high diagnostic accuracy for both pathologies while significantly saving memory resources and power consumption.