The thing that always separated him from the other greatest players (them being considered higher; Jordan, Kareem, Russell etc) was his defense. He was best suited as a help defender (hence his high SPG) and would've been helped greatly by guarding players closer to his size/athleticism but he was usually stuck guarding point guards. It's a big reason why Kevin Johnson was able to have such big playoff series against L.A. in the early 1990s. I think Run TMC also gave them a little trouble just because Tim Bug was the worst player for Magic to guard. If Magic played today they'd utilize him as a point forward and he'd be okay among bigger forwards.
Regardless he's a top-5 player (imo but I've seen him generally around 4-6) but his lack of defensive prowess in comparison to others wouldn't make him the very greatest to some people. Still, the feel for the game and his ability to lead an elite offense no matter what style they were (LA became a more half court oriented team after about '88 or so) hasn't really been matched. Also being unstoppable in transition and having a great post game helps too. Nothing offensively by about 1989 that Magic could not do. His extended prime is so underrated.
But to answer your question I'd say defense is his biggest flaw as an overall player, which keeps him from achieving GOAT status for a lot of people.