Isn't it possible that a 6' 6" guy would be at least that far behind much taller or much shorter players in both subsets?
Yes, in fact it's the truth. But would you want your massive 7'5" superstar C with great stats in IS, ID and R but not much else playing SF? Similarly, your SG with just great skills in JS, JR and OD? I'm not saying that the middle height players would make great inside-only or outside-only focused positions, I'm saying that they would strike a decent balance and probably do well as a trainable SF, and I think the SF position can be a very valuable position to look to strengthen if you do it properly.