It's hard to get to the Super Bowl. Big Ben benefitted from great defenses, great running games and veterans around him. Eli and the Giants got hot at the right times and found ways to win on a couple runs but it's not like the Giants are in the playoffs every single year.
The Chargers notoriously underachieved in 2006 when they were 14-2 and got bounced in their home playoff game vs. the Patriots. They made it to the AFC Championship game the next year and Rivers played through that torn ACL against the Patriots. They couldn't get past the Steelers in 2008 in the 2nd round and they then lost another home game as a #2 seed vs. the Jets in 2009.
It's not all Rivers. You had Nate Kaeding blunders, you had defensive messing up... Norv Turner as HC