Natural Language Processing Techniques for Ranking Subjective Responses

Computer Science

Each year, universities and educational boards conduct exams offline, requiring students to participate in subjective tests. The manual evaluation of a vast number of such answer sheets is labor-intensive and time consuming. Moreover, the quality of assessment can sometimes be inconsistent, influenced by the evaluator's mindset or fatigue. In contrast, objective or multiple-choice questions, often used in competitive and entrance exams, are easily assessed through automated systems, simplifying the evaluation process. However, the manual grading of subjective responses remains a challenging task. The integration of artificial intelligence (AI) in evaluating subjective responses faces significant hurdles, primarily due to skepticism about the accuracy and reliability of the results. Although there have been various attempts to leverage computer science for assessing student answers, many of these efforts rely heavily on standardized counts or specific keywords, and often suffer from a lack of comprehensive datasets. This paper introduces a novel approach for the automatic evaluation of descriptive answers by employing a combination of machine learning techniques and natural language processing tools such as WordNet, Word2Vec, word mover's distance (WMD), cosine similarity, Multinomial Naive Bayes (MNB), and term frequency-inverse document frequency (TF-IDF). The proposed method assesses answers by comparing them to solution statements and relevant keywords, and it also develops a machine learning model to predict grades. The findings suggest that WMD provides better performance compared to cosine similarity. Additionally, with appropriate training, the machine learning model can be used independently. Experimental results show that the approach achieves an accuracy of 88% without using the MNB model, while incorporating MNB further reduces the error rate by 1.3%.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.