Fair use, in the US anyway, is based on four factors (source):

(1) the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes; (2) the nature of the copyrighted work; (3) the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and (4) the effect of the use upon the potential market for or value of the copyrighted work.

Not selling fan art helps to bolster factor #1, though that alone isn’t enough. Fair use itself would need to be determined by a judge, but ensuring the work is transformative enough and doesn’t disincentivize someone from purchasing the original work is probably enough commercial or not, but noncommercial theoretically should help.

(Not a lawyer, but I’ve followed this a bit)