Music Recommendation based on Face Emotion Using Artificial Intelligence and Machine Learning

Information Science Engineering

This paper presents an intelligent emotion-based music recommendation system that leverages facial expression recognition to enhance user experience in personalized content delivery. The system utilizes deep learning techniques, specifically a mini-XCEPTION convolutional neural network architecture trained on the FER-2013 dataset, to detect seven distinct emotional states from real-time facial expressions captured through webcam. The detected emotions include Angry, Disgust, Fear, Happy, Sad, Surprise, and Neutral states. Upon emotion detection, the system automatically recommends and plays music content that aligns with the user’s current emotional state through YouTube integration, providing a seamless and intuitive user experience. The application features an intuitive Streamlit-based web user interface that enables seamless real-time emotion detection with visual feedback including bounding boxes, emotion labels, and color-coded emotion badges. The mini-XCEPTION model demonstrates robust performance with 66% test accuracy on the FER-2013 dataset while maintaining computational efficiency with only 58,293 parameters. Experimental results demonstrate the system’s capability to accurately classify emotions in real-time and provide personalized music recommendations within 5 seconds, achieving an effective human-computer interaction experience with 85% user satisfaction. The system processes video frames at 8-10 FPS with total detection latency under 120ms, making it suitable for real-time applications. This work bridges affective computing and content recommendation systems, offering potential applications in mental health support, entertainment, personalized content delivery, therapeutic music delivery, smart home integration, and customer experience enhancement in commercial environments.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.