I designed a sample VR Inference. In few cases, the label shows undefined it is because of the model used.
1 Like
This week’s blog post
https://blog.shivadityas.com/google-summer-of-code-week-4/
POC for Object detection. The model is able to detect polyps from video stream. The pause at the beginning is for model inference it takes about 20-30 seconds depending on the hardware.
The video used here is Generated with the help of images present in the kvasir dataset
This weeks Blog post https://blog.shivadityas.com/google-summer-of-code-week-4-2/
Performance Issues
Object Detection model on android phones display performance issues in terms of inference time and device crashing. For video clips longer than 10mins the WebGL memory exceeds 512mb sometimes causing the browser to crash.
VR without Object detection is Resource intensive. Older phones cannot handle current generation VR content and cause frame drops.
Data Privacy
Kvasir dataset has no specific data storage policy. Storage of videos is a concern.
https://blog.shivadityas.com/google-summer-of-code-week-7/
Corrected Bounding Box
MR raised with the updated webapp
@pri2si17 as discussed in the previous meeting, I have designed the VR Prototype in Unity using inbuilt XR package. As of now, for emulation I have used MockHMD. MockHMD does not support movements in Unity mode as it requires sensors. The video can be accessed remotely using URL or local asset
Settings like IPD can be changed according to HMD used.
Would you like to use these datasets for GSoC? I think the Kvasir is great for proof-of-concept that we need.
@pri2si17 Updated the Repository with the UnityXR and Video Server files. Both Segmented mask generation and Object detection work with the remote server. I have updated the MR
Object Detection
Segmentation
I have submitted the final evaluation. @pri2si17 and @judywawira, thank you for your guidance through out the project.