Yep, you need gloves...leather should do. If you go down the reflex action is to put your hands out to catch yourself. Guess what happens to them when they slide along dirt, gravel, blacktop, pavement, and cactus.
Even riding my 49cc Zuma scooter I wear gloves, boots and a helmet. I may look funny, but I've dumped a few bicycles and never did like that feeling of sliding along and stuff going thru and under my skin. Have you ever had that happen?