So it’s the people who you actually care about and respect that are holding you back… Wait a second, no, it’s just the people who’s opinions matter to you.
Why do their opinions matter? Do you owe them something? Do they love and respect you? Do they uplift you as a person? If not, why do you care what they think?
This is the point where you really have to decide if you want it more than you are afraid of it, if you want it more than those people around you might comment and tease, or even be outright nasty.