Even in the Western half of the empire a lot of the elites spoke Greek. For example: while he would definitely have been capable of speaking Latin, Marcus Aurelius wrote Meditations in Greek.