Feature Extraction Using Multi-modal Convolution Neural Network for Optical Speech Recognition

GOKUL RAJES P; Sayantan Chatterjee; N SRICHARAN PHANINDRA; Kaustubh Narkhede; Saksham Alag

Feature Extraction Using Multi-modal Convolution Neural Network for Optical Speech Recognition

March 2019

Vol-5, Issue-2

Paper ID: 9653

ISSN: 2395-4396

Downloads: 0

Abstract & Details

Research Area

Electronics and Communication Engineering

Keywords

CNN-convolutional neural network HMM- Hidden Markov model GMM- Gaussian Markov Model and Architecture

Abstract

This column orates the difficulties of continuous speech recognition system from optical information only, without using audio signal. Our method combines a video camera and an ultrasound imagining system for simultaneously detecting the speaker’s lips and the mobility of our tongue. We considered the usage of convolution neural network (CNN) to elicit the optical features directly from the basic ultrasound and optical images. We want to introduce variant architectures among which is a multimodal CNN processing jointly the two optical modalities together with an HMM-GMM decoder, the CNN type approach suppresses our earlier baseline based on principal component analysis. Notably, the recognition efficiency is 4 % lower than which was obtained decoding audio signal which makes it better for practical optical speech recognition system.

License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Author Information

#	Name	Institute / Affiliation
1	GOKUL RAJES P	SRM Institute of Science and Technology
2	Sayantan Chatterjee	SRM Institute of Science and Technology
3	N SRICHARAN PHANINDRA	SRM Institute of Science and Technology
4	Kaustubh Narkhede	SRM Institute of Science and Technology
5	Saksham Alag	SRM Institute of Science and Technology

How to Cite

Use the following formats to cite this article in your research.

APA Style

P, GOKUL RAJES, Chatterjee, Sayantan, PHANINDRA, N SRICHARAN, Narkhede, Kaustubh, & Alag, Saksham (2019). Feature Extraction Using Multi-modal Convolution Neural Network for Optical Speech Recognition. International Journal of Advance Research and Innovative Ideas In Education, 5(2), 476-480.

MLA Style

P, GOKUL RAJES, et al. "Feature Extraction Using Multi-modal Convolution Neural Network for Optical Speech Recognition." International Journal of Advance Research and Innovative Ideas In Education, vol. 5, no. 2, 2019, pp. 476-480.

IEEE Style

GOKUL RAJES P, Sayantan Chatterjee, N SRICHARAN PHANINDRA, Kaustubh Narkhede, and Saksham Alag, "Feature Extraction Using Multi-modal Convolution Neural Network for Optical Speech Recognition," International Journal of Advance Research and Innovative Ideas In Education, vol. 5, no. 2, pp. 476-480, 2019.

Vancouver Style

P GOKUL RAJES, Chatterjee Sayantan, PHANINDRA N SRICHARAN, Narkhede Kaustubh, Alag Saksham. Feature Extraction Using Multi-modal Convolution Neural Network for Optical Speech Recognition. International Journal of Advance Research and Innovative Ideas In Education. 2019;5(2):476-480.

Harvard Style

P, GOKUL RAJES, Chatterjee, Sayantan, PHANINDRA, N SRICHARAN, Narkhede, Kaustubh, & Alag, Saksham (2019) 'Feature Extraction Using Multi-modal Convolution Neural Network for Optical Speech Recognition', International Journal of Advance Research and Innovative Ideas In Education, 5(2), pp. 476-480.

Chicago Style

P, GOKUL RAJES, et al. "Feature Extraction Using Multi-modal Convolution Neural Network for Optical Speech Recognition." International Journal of Advance Research and Innovative Ideas In Education 5, no. 2 (2019): 476-480.

Turabian Style

P, GOKUL RAJES, et al. "Feature Extraction Using Multi-modal Convolution Neural Network for Optical Speech Recognition." International Journal of Advance Research and Innovative Ideas In Education 5, no. 2 (2019): 476-480.