Have you heard about futo keyboard? It embeds a small LLM model for this very task. It also has an option to retrain based on user data.