I do the same. Those skull candy ear buds are PERFECT! Cuts at least most, if not all, of the wind noise, but still allows the traffic and bike sounds around you through, and you're not blasting out the music just so you can barely hear it at all. It's like neither one is 'overpowering' the other, and BOTH can still be heard, but at a volume that's tolerable but not tiresome. For me, it makes any trip, short or long, just better. I end up feeling less 'tired' at the end of the ride, and I'm not half deaf from the wind noise, too.
With that being said, I also know that my son, who's ADD and Dyslexic, would NOT be able to handle both music and riding at the same time. It would be too distracting for him. So, for some people, it's fine to listen to music and ride, but for others, no way! It all depends on the person, then.