I get your point, but one example of cultural impact not being taken into account doesn't mean that it's not a big part of winning. In general, it helps if your album had a large cultural impact. Also, you're picking an example that's close. GRODT as an album had a much larger impact on the culture, but Hey Ya is one of the biggest singles of all time.

That shyt landed on every demographic. Children all the way to people in nursing homes. And (even though I know correlation doesn't equal causation) whoever wins song of the year for their category often wins album of the year.
It wouldn't surprise me at all if Hey Ya is what tipped the scales. Also, white people love Outkast. They especially loved that Outkast made a double album where half of it wasn't rapping.
edit: The more I think about it,
Speakerboxxx/Love Below is damn near a tailor made album to try and win a Grammy. GRODT should have won, but Outkast winning over 50 that year in hindsight isn't as surprising as it was at the time.