Are my friends true friends, or should I dump them?

I met a real nice guy 3 months ago. He is sooooo fine, and he comes from a very wealthy family. He is so awesome! He is 19 and I am 18. He is in pre med in college. His father and mother are both doctors, and his sister has just become a pediatric dentist. This family lives in a "mansion", not a regular house, but a "mansion"! I feel so lucky to have met this guy because I am far from wealthy and so is my family. We are just average cl people. He treats me so good, and he does not act like he is wealthy or better than just average people. This is my problem. My closest friends whom I have been friends with for years don't want to be around me any more. They say I'm dating someone out of my league and it won't last. They say I'm a "sell-out" to my own average income family and friends. That makes me feel so bad. I love my friends so much. I don't want to lose any of them but I really like this guy. I am so torn right now. What do I do? Please help.