My first saxophone playing was as a freshman in high school, I was 4' 11" and 95 pounds, starting on a then brand spanking new school horn, a Beaugnier made Vito bari (I grew up very near their home base in Kenosha, Wisconsin in 1963). A big help was probably playing bass clarinet in junior high. I marched with a neckstrap, don't know if they even made harnesses back then, never thought anything about it! I bought a tenor for college, thinking that's what you did for playing R&R, but never really bonded (and, now, I hear tons of bari in old R&R songs). Baritone saxophone is such a magical entity that it is worth switching hands when carrying, struggling with it on the bus, paying more for reeds, developing 100 gallon lung capacity, having it take the whole back seat of your car, because when you get good, the reward to your brain is immeasurable. To feel the sweet vibration of low notes blurring your vision while you grin on the inside is sublime. To feel the ethereal upper register is unreal. You can ---always--- be heard.
I recommend starting today, and let nothing stand in your way. If your physical size or gender worries you, look here: If this skinny high school girl can play like this, just about anyone can! (BTW that's not me, I'm a 69 year old Grampa--but I did play that same arrangement as that 95 pound freshman) And, for all of you bari players, do you make actual love to your horn like this young lady does? I hope she kept on with it after high school!
Like a T-shirt I saw: "Baritone sax----like regular sax---- only better!"