It’s being pushed into situations where having a black box of probability is not what you want. Support bots as an example, you want the same outcome for the same problem. Not a different outcome because someone didn’t put a question mark.
You don’t want the bot being able to hijack an account because it was asked in the specific way. See Facebook support bot.
You don’t want a chat bot to tell airline customers they can do something when the airline has a FAQ section specifically saying no you can’t.
See American express support bot.

It’s used as a workforce reduction mask.
Businesses are using AI rollouts to lower the number of front line employees. Only for the tool to fail.
See Oracle firing 7,000 people to boost ai numbers

It’s oversold:
When Gardner says only 30% of AI deployments work as sold after asking over 700 IT executives why are people still treating it as anything but a broken tool?

For a black box of probability it is terrible at maths. There has been instances of AIs failing high school math tests.