Does it smell nice when you drive over it? Does it allow rainwater to percolate? Can your vehicle get sufficient grip and support?

I guess maybe you’re worried it will dislodge the rocks and make the surface uneven? Because otherwise it seems like a non-problem. If you make a point of squashing the tallest bits each time you won’t even have to cut it.