FEDERATED LEARNING FOR NEXT-WORD PREDICTION: A DISTRIBUTED APPROACH FOR IMPROVED LANGUAGE MODELS

Computer Engineering

The surge in machine learning necessitates novel techniques that prioritize data privacy and decentralized training. Federated learning (FL) emerges as a promising solution in this domain. This research explores the potential of FL, particularly focusing on Long Short-Term Memory (LSTM) networks for next-word prediction in a Homogeneous FL setting. We propose a secure FL architecture involving three client devices and a central server, each contributing to the learning process. To ensure data security, a robust two-factor authentication framework is implemented for all participating clients. This framework merges mobile One-Time Passwords (OTPs) with traditional username/password credentials, bolstering security against unauthorized access and data breaches. Within the FL setup, client devices independently train the LSTM model on their local datasets. Subsequently, the server aggregates these local models into a global model, which is then distributed back to the clients for further training iterations. This cyclical process continues until the desired level of prediction accuracy is achieved. Furthermore, this research delves into the impact of the proposed secure FL setup on the performance of the LSTM-based next-word prediction model. We analyze the influence of secure data transmission and robust authentication on the learning outcomes, comparing model performance across different scenarios. This investigation sheds light on the trade-offs between security and performance in FL for next-word prediction tasks. Additionally, this study explores how the LSTM-based next-word prediction model performs in relation to the suggested secure FL setup. We examine how reliable authentication and secure data transmission affect learning outcomes by contrasting model performance in various contexts. The trade-offs between security and performance in FL for next-word prediction tasks are clarified by this work.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.