DC Field | Value | Language |
---|---|---|
dc.contributor.author | SEO, WON JU | - |
dc.contributor.author | Kim, Namho | - |
dc.contributor.author | Park, Cheolsoo | - |
dc.contributor.author | PARK, Sung-Min | - |
dc.date.accessioned | 2022-05-03T02:20:06Z | - |
dc.date.available | 2022-05-03T02:20:06Z | - |
dc.date.created | 2022-04-29 | - |
dc.date.issued | 2022-06 | - |
dc.identifier.issn | 1530-437X | - |
dc.identifier.uri | https://oasis.postech.ac.kr/handle/2014.oak/112547 | - |
dc.description.abstract | Work-related stress causes serious negative physiological and socioeconomic effects on employees. Detecting stress levels in a timely manner is important for appropriate stress management; therefore, this study proposes a deep learning (DL) approach that accurately detects work-related stress by using multimodal signals. We designed a protocol that simulates stressful situations and recruited 24 subjects for the experiments. Then, we collected electrocardiogram (ECG), respiration (RESP), and video data. The datasets were pre-processed and 10-s ECG and RESP signals and a sequence of facial features were fed into our deep neural network. Sixty-eight facial landmarks’ coordinates were extracted, and facial textures were extracted from a pre-trained network based on facial expression recognition. Each signal was processed by each of its network branch, and data were fused at two different levels: 1) feature-level and 2) decision-level. The feature-level fusion that used RESP and facial landmarks’ coordinates showed average accuracy of 73.3%, AUC of 0.822, and F1 score of 0.700 in two-level stress classification, and the feature-level fusion that used ECG, RESP, and the coordinates showed average accuracy of 54.4%, AUC of 0.727, and F1 score of 0.508 in three-level stress classification. When analyzing the weights in the decision-level fusion, we found that the importance of each information item varied according to the stress classification problem. When comparing t-stochastic neighbor embedding results, we observed that overlapped samples of different classes caused performance degradation in both classifications. Our findings suggest that the proposed DL approach fusing multimodal and heterogeneous signals can enhance stress detection. IEEE | - |
dc.language | English | - |
dc.publisher | Institute of Electrical and Electronics Engineers | - |
dc.relation.isPartOf | IEEE Sensors Journal | - |
dc.title | Deep Learning Approach for Detecting Work-Related Stress Using Multimodal Signals | - |
dc.type | Article | - |
dc.identifier.doi | 10.1109/JSEN.2022.3170915 | - |
dc.type.rims | ART | - |
dc.identifier.bibliographicCitation | IEEE Sensors Journal, v.22, no.12, pp.11892 - 11902 | - |
dc.identifier.wosid | 000811704400077 | - |
dc.citation.endPage | 11902 | - |
dc.citation.number | 12 | - |
dc.citation.startPage | 11892 | - |
dc.citation.title | IEEE Sensors Journal | - |
dc.citation.volume | 22 | - |
dc.contributor.affiliatedAuthor | SEO, WON JU | - |
dc.contributor.affiliatedAuthor | Kim, Namho | - |
dc.contributor.affiliatedAuthor | PARK, Sung-Min | - |
dc.identifier.scopusid | 2-s2.0-85129422631 | - |
dc.description.journalClass | 1 | - |
dc.description.journalClass | 1 | - |
dc.description.isOpenAccess | N | - |
dc.type.docType | Article | - |
dc.subject.keywordPlus | EMPLOYEES | - |
dc.subject.keywordPlus | EMOTION | - |
dc.subject.keywordPlus | FUSION | - |
dc.subject.keywordAuthor | Biomedical monitoring | - |
dc.subject.keywordAuthor | Brain modeling | - |
dc.subject.keywordAuthor | data fusion | - |
dc.subject.keywordAuthor | deep learning approach | - |
dc.subject.keywordAuthor | Electrocardiography | - |
dc.subject.keywordAuthor | Feature extraction | - |
dc.subject.keywordAuthor | Human factors | - |
dc.subject.keywordAuthor | mental stress detection | - |
dc.subject.keywordAuthor | Monitoring | - |
dc.subject.keywordAuthor | multimodality | - |
dc.subject.keywordAuthor | Stress | - |
dc.subject.keywordAuthor | Work-related stress | - |
dc.relation.journalWebOfScienceCategory | Engineering, Electrical & Electronic | - |
dc.relation.journalWebOfScienceCategory | Instruments & Instrumentation | - |
dc.relation.journalWebOfScienceCategory | Physics, Applied | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Engineering | - |
dc.relation.journalResearchArea | Instruments & Instrumentation | - |
dc.relation.journalResearchArea | Physics | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.
library@postech.ac.kr Tel: 054-279-2548
Copyrights © by 2017 Pohang University of Science ad Technology All right reserved.