Open Access

ARTICLE

An Intelligent Framework for Recognizing Social Human-Object Interactions

Mohammed Alarfaj¹, Manahil Waheed², Yazeed Yasin Ghadi³, Tamara al Shloul⁴, Suliman A. Alsuhibany⁵, Ahmad Jalal², Jeongmin Park^6,*

1 Department of Electrical Engineering, College of Engineering, King Faisal University, Al-Ahsa, 31982, Saudi Arabia
2 Department of Computer Science, Air University, Islamabad, 44000, Pakistan
3 Department of Computer Science and Software Engineering, Al Ain University, Al Ain, 15551, UAE
4 Department of Humanities and Social Science, Al Ain University, Al Ain, 15551, UAE
5 Department of Computer Science, College of Computer, Qassim University, Buraydah, 51452, Saudi Arabia
6 Department of Computer Engineering, Tech University of Korea, Siheung-si, 15073, Gyeonggi-do, Korea

* Corresponding Author: Jeongmin Park. Email:

Computers, Materials & Continua 2022, 73(1), 1207-1223. https://doi.org/10.32604/cmc.2022.025671

Received 01 December 2021; Accepted 31 March 2022; Issue published 18 May 2022

Abstract

Human object interaction (HOI) recognition plays an important role in the designing of surveillance and monitoring systems for healthcare, sports, education, and public areas. It involves localizing the human and object targets and then identifying the interactions between them. However, it is a challenging task that highly depends on the extraction of robust and distinctive features from the targets and the use of fast and efficient classifiers. Hence, the proposed system offers an automated body-parts-based solution for HOI recognition. This system uses RGB (red, green, blue) images as input and segments the desired parts of the images through a segmentation technique based on the watershed algorithm. Furthermore, a convex hull-based approach for extracting key body parts has also been introduced. After identifying the key body parts, two types of features are extracted. Moreover, the entire feature vector is reduced using a dimensionality reduction technique called t-SNE (t-distributed stochastic neighbor embedding). Finally, a multinomial logistic regression classifier is utilized for identifying class labels. A large publicly available dataset, MPII (Max Planck Institute Informatics) Human Pose, has been used for system evaluation. The results prove the validity of the proposed system as it achieved 87.5% class recognition accuracy.

---

Keywords

---