You can quantify it -
Cavs
2009-2010 - 7th
2010-2011 - 29th
They had Varejao, Hickson, Gibson, Williams starting with Jamison taking Bron's place. That's essentially the same roster that played the second half of the 09/10 season, which was a top 10 defensive team.
I'd say his man to man defense on a nightly basis is a 6, when it's a good opponent he turns it up to 8. But, his team defense is where he's the best in the league on the perimeter, nobody else has his versatility. As a team defender he's in no way overrated....