Broadly speaking, anxiety is an elevated mood; the reason stimulants exacerbate anxiety is the same reason stimulants are contraindicated in people with bipolar disorder. For someone trying to achieve concentration while suffering from an anxiety disorder, a prescription medication like guanfacine might be better; it has an off-label usage for anxiety disorders, and its two main usages are in treating ADHD and lowering blood pressure (neurostimulants like caffeine, by contrast, elevate your blood pressure).

The caffeine withdrawal thankfully only lasts about a week at the worst.