One possibility is that it is exceptionally difficult to re-evolve baseline feeding features once you become heavily specialized. It could also be that betting on ants and termites tends to pay off

I’m betting a mix of both. I think myrmecophagy is an evolutionary strategy bound to appear when other niches are unavailable due to competition, and to restrict them further.

I’ll use the order Pilosa for the sake of example. Consider the following two maps:
Image Image
The first one shows the suborder Vermilingua (anteaters), the second one Folivora (sloths). Here are their diets and ranges:

Clade	Diet	Areas
Vermilingua (anteaters)	ant/termite eaters	jungle (Amazon), savanna (Cerrado), swamps (Pantanal)
Folivora / genus Bradypus (three-toed sloths)	picky leaf eaters, koala/panda style	jungle (Amazon), savanna (Cerrado)
Folivora / genus Choloepus (two-toed sloths)	omnivores	jungle (Amazon)

I’m simplifying the ranges, mind you. Regarding Choloepus’ omnivory, TL;DR they eat whatever won’t outrun a sloth (eh) - berries, carrion, a few insects, even a lizard or two.

Note all three can be found in the jungle, but only the specialised eaters can be found in the savanna. I don’t think this is a coincidence: the plant life in Amazon is so abundant that monkeys and birds can’t call dibs on all energy sources there, but the same does not apply to Cerrado. This makes Choloepus’ omnivory viable in the former, but not the later - in Cerrado you won’t outcompete birds and monkeys, so the specialised diets pay off there.

But let’s say some Vermilingua species developed a mutation enabling a wider diet; they can eat berries, although it’s a rather small part of their diet. That mutation would likely make them worse at ant/termite-eating, and put them into direct competition with other species - it’s a gambit that simply doesn’t pay off.

So they’re mostly “stuck” with myrmecophagy. And there’s selective pressure against diversification, at least in environments where food is a primary concern (instead of predation).

I think this reasoning can be extended into other clades that are eating ants and termites, too.