It depends on how it's framed. Floyd didn't fight Pac at his best. He didn't fight Margarito. The fight didn't happen with Mosley until many years after it should have. He didn't fight Paul Williams.
Now it's all about perspective. In retrospect some don't think much of Williams and Margarito but at the time they were contenders people clamored for the fight. You've also gotta remember Floyd took off for a couple of years and he only fights twice a year anyway so obviously he's not gonna he taking on all comers. He doesn't have the need or motivation to do so.
Don't get me wrong, I personally think Floyd wins all of those fights, I'm just making the point that there were definitely fights left on the table. But you can essentially say that about any fighter in the last few eras - hell, even a Bernard Hopkins who we know doesn't duck anybody missed out on a couple of fights for various reasons. And also I'm not saying its Floyds fault he didn't have these fights, I'm just saying that just like with anybody else there's always going to be questions of his resume if you seek them out.