In case it might be interesting info, maybe want to check out WebGPU, WebLLM, Wllama, or Transformers.js

You can write web apps that run an llm on the client’s browser. It requires permission and compatibility and isn’t… it might be tough to really make accessible to the average user.

But it’s an option and kinda cool.