According to this list it was used figuratively by Jane Austen, who I believe died more than 200 years ago. That page also claims the earliest known use is 1769, so it’s probably less than 300 years in writing? It’s moot either way, if you’re going for an etymological argument you could go further and say literally should mean anything to do with letters or writing, from the original Latin literalis/litteralis “of or belonging to letters or writing”.