Abstract:  With uses in sports analytics, security, healthcare, and human-computer interaction, Human Activity Recognition (HAR) is growing in popularity in computer vision. Traditional HAR methods often rely solely on motion analysis or pose estimation, which limits their robustness in complex scenarios. This research proposes a multi-modal deep learning system that integrates pose estimation, motion dynamics, and skin texture analysis to enhance HAR accuracy. The suggested model successfully captures temporal and spatial dependencies in human motions by utilizing transformer-based topologies and convolutional neural networks (CNNs). Additionally, it incorporates skin related data for enhanced subject tracking and segmentation. To merge derived characteristics from several modalities and ensure resilience against occlusions, changing lighting conditions, and a range of skin tones, a unique fusion process is presented. The proposed approach outperforms the state-of-the-art methods on benchmark HAR datasets. The results reveal that incorporating skin aware features significantly improves activity recognition in real world scenarios, highlighting the potential of multi-modal deep learning in creating HAR applications.