There is nothing in the books that says that people without magic can’t understand it? I think there was a plot point in 4 or 5 book where harry is on trial for using some spell to scare away dementors, and his neighboor testifies that he really did it and people don’t believe her cause she doesn’t have magic. But that’s only seeing magical creatures, what stops anyone from understanding it exactly? They do repeatable things that return repeatable results, pretty understandable.