MULTI TASK LEARNING FOR CAPTIONING IMAGES WITH NOVEL WORDS

Computer Engineering

we present a Multi-task Learning Approach for Image Captioning (MLAIC), which is motivated by the observation that humans can easily complete this task since they are skilled in a variety of disciplines. MLAIC is made up of three essential parts in particular:(i)A multi-object classification model that learns detailed category-aware picture representations by employing a CNN image encoder (ii) An image captioning model that generates text descriptions of images by sharing its CNN encoder and LSTM decoder with the object classification task and the syntax that learns better syntax aware LSTM based decoder.The added object classification and grammar knowledge is especially generation task, respectively. (ii) A syntax generation model that improves syntax aware LSTM based decoder. (iii) A syntax generation model advantageous for the picture captioning model. The experimental outcomes on the MS-COCO dataset show that our model outperforms other formidable rivals in terms of performance.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.