Tooth enamel is ~5 on mohs scale. Quartz, the most common kinda rock (afaik), is ~7.

Fun fact, this is why we know ancient people used to make bread - the way they separated wheat from the chuff included dropping it on the ground, which led to sand being in the bread and consequently destroying their teeth over a lifetime of eating it.