Project: Machine learning-based decision support in NeoRoo app

The NeoRoo project is a decision-support app that connects with the NeoWarm sensor and provides real-time updates to nurses and caregivers about kangaroo mother care metrics.

This application integrates data from the ECEB app that is integrated with the DHIS2 backend and contains general clinical information about the mother-baby dyad and permanent and semi-permanent information that needs to be understood with the streaming vital signs data that is captured by the NeoWarm biomedical device.

Recent advances and our preliminary work in fusion models that ensemble streaming data (sensors) with tabular data (EHR) using deep learning approaches such as recurrent neural networks (RNN) or, more specifically, Long short-term memory (LSTM) and its variants needs to be applied in NeoRoo as part of this project. Our recent work on human activity recognition from continuous data streaming from triple-connected sensors at Extensor Digitorum (Posterior Forearm), Gastrocnemius (Calf), and Pectoralis (Chest) with 98% F1 score using BiLSTM and ConvLSTM might be feasible to identify high-priority alerts in the NeoRoo app.

The 4 vital signs (body temperature, heart rate, oxygen saturation, and respiratory rate) from the sensors correlate poorly in preterm babies and are streamed in differing frequencies. Thus, a suggestion is to attempt (Fig. below) an attention-based MLCNN + channel attention + ConvLSTM to extract features and classify the baby into low-risk, moderate risk, and high-risk probabilities.


While this neural network might overwhelm a low-end Android phone when multiple babies are tracked with 6-10 simultaneous streaming NeoWarm devices sending data, we will quantize these models, similar to our previous work, where our model compression reduced model size by 70%, with only 2-5% loss in performance to full-precision models [3] and reduced power consumption by over 69% [4].

The deliverables of the project are as follows:

  • Develop an integrative approach using neural networks that combine streaming data with semi-permanent and permanent clinical information for ECEB and NeoRoo apps.
  • Quantize models to be low-powered and efficiently run on Android mobile apps.
  • Show results simply and intuitively for providers and caregivers, with the provision to look into more detailed and explainable views.
  • Integrate outputs of these models into the SPA for appropriate drug and test orders.

Preliminary tasks

A developer working on this project needs to have skills in HTML, CSS, JavaScript frameworks like React/Angular/Polymer, Flask web services skills, and Python for model development.

Mentors: @sunbiz and @shbucher


[1] Tariq A, Celi LA, Newsome JM, Purkayastha S, Bhatia NK, Trivedi H, Gichoya JW, Banerjee I. Patient-specific COVID-19 resource utilization prediction using fusion AI model. NPJ digital medicine. 2021 Jun 3;4(1):1-9.

[2] Mahajan, Y., Pinnamraju, J., Burns, J.L., Gichoya, J.W., Purkayastha, S. (2022). Using Machine Learning Approaches to Identify Exercise Activities from a Triple-Synchronous Biomedical Sensor. In: Abraham, A., Gandhi, N., Hanne, T., Hong, TP., Nogueira Rios, T., Ding, W. (eds) Intelligent Systems Design and Applications. ISDA 2021. Lecture Notes in Networks and Systems, vol 418. Springer, Cham. Using Machine Learning Approaches to Identify Exercise Activities from a Triple-Synchronous Biomedical Sensor | SpringerLink

[3] Sinha, Priyanshu, Judy W. Gichoya, and Saptarshi Purkayastha. “Leapfrogging Medical AI in Low-Resource Contexts Using Edge Tensor Processing Unit.” 2022 IEEE Healthcare Innovations and Point of Care Technologies (HI-POCT). IEEE, 2022.

[4] Sinha P, Tummala SS, Purkayastha S, Gichoya J. Energy Efficiency of Quantized Neural Networks in Medical Imaging. InMedical Imaging with Deep Learning 2022 Apr 22.

Hello Mr. Saptarshi,

I am interested to contribute this project. Currently I am working on similar projects which help clinicians and researchers using Django web services. I have all skillset you are looking for in a developer. I would like to support this project under you guidance.

More information about me :

If you are interested , kindly contact me for further information.

Regards, Mamaka

@mamaka7, you will have to send a proposal and discuss your approach in the proposal.
We wont be able to contact you, and definitely wont be able to communicate privately, as is against our GSoC requirements. All communication has to be public.

@dbmohit - if you’d like to discuss approaches, please ping us on
For implementation suggestion, we expect that you write it in the proposal

@mamaka7 and @dbmohit and anyone else interested, we have added preliminary tasks, so that we can evaluate your skills in a better way. Please add repos for these in your application proposal.

Dear Saptarshi Purkayastha,

I hope this message finds you well. I am writing to express my interest in working on the “Machine learning-based decision support in NeoRoo app” project under GSoC 2023. I have been following the development of NeoRoo and I find it to be an incredibly innovative and promising app.

As a student with a background in machine learning, I am confident that I have the necessary skills and expertise to contribute to this project. I am particularly excited about the prospect of working on decision support using machine learning, as it is an area I am passionate about.

If you are open to it, I would love to discuss the project further with you and get your input on how I can best contribute. Please let me know if this is something you would be interested in, and if there is a convenient time for us to schedule a call.

Thank you for your time and consideration. I look forward to hearing from you soon.

Best regards,

Mohit Kumar

Hello everyone!

I’m Shripad from India, a Mobile and Web Developer, and a second-year CSE student.

My primary tech stack is Flutter, Firebase, HTML, CSS, and javascript.

GitHub - Skdahale (Shripad Dahale) · GitHub LinkedIn -

Regarding the preliminary task mentioned in form for this project:

  1. Inorder to dynamically quantize the CheXNet model does it imply modifying the model to reduce its memory footprint and improve its performance by dynamically adjusting its precision during training or inference.
  2. And showing values dynamically does it mean that the app should display the sensor readings in real-time, updating the displayed values as the device is rotated or moved.

Would be really helpful if someone could guide me regarding these doubts.

yes… model performance usually drops with quantization. But model size reduction and/or reducing it to fp16 or int8 is what we mean by quantization

yes… showing the change of the values coming from the rotation sensor on the app screen.