❤️ Please remember to ⭐ this repo if you find it useful and cite our work if you end up using it in your work! ❤️
❤️ If you have any questions or concerns, please create an issue 📝! ❤️
Remote photoplethysmography (rPPG) emerges as a promising method for non-invasive, convenient measurement of vital signs, utilizing the widespread presence of cameras. Despite significant advancements, existing datasets fall short in terms of size and diversity, limiting comprehensive evaluation under diverse conditions. This paper presents an in-depth analysis across four datasets with 1000 subjects, mainly focused on VitalVideo dataset, the largest real-world rPPG dataset to date, encompassing 900 subjects and 6 skin tones. Through rigorous experimentation with six unsupervised methods and three advanced supervised models(TSCAN, Physnet, Physformer), we demonstrate that just a few hundred subjects are adequate for effective rPPG training, thus attesting to the dataset's utility. Our findings highlight the importance of diversity and consistency in skin tones for precise performance evaluation across different datasets.
The code supports four datasets, namely VitalVideo, UBFC-rPPG, PURE, and MMPD. Please cite the corresponding papers when using these datasets. To use these datasets in a deep learning model, you should organize the files as follows.
- MMPD
- Jiankai Tang, Kequan Chen, Yuntao Wang, Yuanchun Shi, Shwetak Patel, Daniel McDuff, Xin Liu, "MMPD: Multi-Domain Mobile Video Physiology Dataset", IEEE EMBC, 2023
data/MMPD/ | |-- subject1/ | |-- p1_0.mat | |-- p1_1.mat | |... | |-- p1_19.mat | |-- subject2/ | |-- p2_0.mat | |-- p2_1.mat | |... |... | |-- subjectn/ | |-- pn_0.mat | |-- pn_1.mat | |...
- UBFC-rPPG
- S. Bobbia, R. Macwan, Y. Benezeth, A. Mansouri, J. Dubois, "Unsupervised skin tissue segmentation for remote photoplethysmography", Pattern Recognition Letters, 2017.
data/UBFC-rPPG/ | |-- subject1/ | |-- vid.avi | |-- ground_truth.txt | |-- subject2/ | |-- vid.avi | |-- ground_truth.txt |... | |-- subjectn/ | |-- vid.avi | |-- ground_truth.txt
- PURE
- Stricker, R., Müller, S., Gross, H.-M.Non-contact "Video-based Pulse Rate Measurement on a Mobile Service Robot" in: Proc. 23st IEEE Int. Symposium on Robot and Human Interactive Communication (Ro-Man 2014), Edinburgh, Scotland, UK, pp. 1056 - 1062, IEEE 2014
data/PURE/ | |-- 01-01/ | |-- 01-01/ | |-- 01-01.json | |-- 01-02/ | |-- 01-02/ | |-- 01-02.json |... | |-- ii-jj/ | |-- ii-jj/ | |-- ii-jj.json
- VitalVideo
- Toye P J. Vital Videos: A dataset of videos with PPG and blood pressure ground truths[J]. arXiv preprint arXiv:2306.11891, 2023.
data/VitalVideo/ | |-- vv100/ | |-- 0a687dbdecde4cf1b25e00e5f513a323_1.mp4 | |-- 0a687dbdecde4cf1b25e00e5f513a323_2.mp4 | |-- 0a687dbdecde4cf1b25e00e5f513a323.json | |... | |-- vv250/ | |-- 0a687dbdecde4cf1b25e00e5f513a323_1.mp4 | |-- 0a687dbdecde4cf1b25e00e5f513a323_2.mp4 | |-- 0a687dbdecde4cf1b25e00e5f513a323.json | |... | |--vvAllCompressed/ | |-- 0a687dbdecde4cf1b25e00e5f513a323_1.mp4 | |-- 0a687dbdecde4cf1b25e00e5f513a323_2.mp4 | |-- 0a687dbdecde4cf1b25e00e5f513a323.json | |...
We have done experiments across vv100 and other three datasets - PURE, UBFC-rPPG, MMPD.
The figure shows Mean Absolute Error (MAE) performance on each single skin tone of vv100 and vvAll with six unsupervised methods. Colors of lines get darker along with the variation of the skin tone.
The table shows Mean Absolute Error (MAE) performance training on vv100 and testing on other three datasets:
The figure shows three important results of the experiment. Bland-Altman plots can be found in folder /run/exp/ and test samples are provided from a python notebook for visualizing test-set neural method output predictions and labels can be found in tools/output_signal_viz.
The table shows Mean Absolute Error (MAE) performance training on other three datasets and testing on vv100:
Create a conda environment with the configuration file:
conda env create -f environment.yml
Please use config which has train_and_test in TOOLBOX_MODE.
Training on VitalVideo and Testing on MMPD With TSCAN.
STEP 1: Download the VitalVideo and MMPD raw data.
STEP 2: Create a HDF5 file of ppg data list from VitalVideo with running python set_list.py
STEP 3: Modify ./configs/vv100_train_configs/vv100_vv100_MMPD_TSCAN_BASIC.yaml
STEP 4: Run python main.py --config_file ./configs/vv100_train_configs/vv100_vv100_MMPD_TSCAN_BASIC.yaml
Note 1: Set DO PREPROCESS to True on the yaml file if it is the first time. And turn it off when you train the network after the first time.
Note 2: The example yaml setting will allow 100% of VitalVideo to train and 100% of MMPD to test. After training, it will use the last model to test on MMPD.
Please use config which has only_test in TOOLBOX_MODE.
For example, if you want to run The model trained on VitalVideo and tested on PURE.
STEP 1: Modify ./configs/vv100_test_configs/vv100_PURE_TSCAN_BASIC.yaml
STEP 2: Run python main.py --config_file ./configs/vv100_test_configs/vv100_PURE_TSCAN_BASIC.yaml
This repo is based on rPPG-Toolbox : https://github.com/ubicomplab/rPPG-Toolbox.
@inproceedings{tang2024camera,
title={Camera-Based Remote Physiology Sensing for Hundreds of Subjects Across Skin Tones},
author={Tang, Jiankai and Li, Xinyi and Liu, Jiacheng and Zhang, Xiyuxing and Wang, Zeyu and Wang, Yuntao},
booktitle={CHI'24 Workshop PhysioCHI'24},
year={2024}
}