I think the benefit of third party AI services is exactly a way around that limited context window. The service can summarize the previous conversations and key facts about the user, store it and feed that back into the AI prompt.

Instead of wasting most of the context window for pages and pages of conversation, it can just prompt the AI with something like “the user is called Timmy, he works as an accountant, he has a girlfriend called Tammy, yesterday he told you that he thinks about proposing.”.

I think even ChatGPT does something like that, but as it’s a very general tool it might not be the best in filtering out information that is relevant for a “personal” conversation.