When it comes to improvising and playing by ear, the only hard keys are the ones you don't play in regularly. 7 sharps or 1 sharp doesn't matter unless you're reading the music. In any case, it's best to play in as many keys as possible. I play with lots of guitar players also and the better guitarists will call keys like Bb, F, Ab. But you need to play in E, G, A, B, and D just as often with guitarists. Bottom line, you need to know all the keys.