Their biggest hits were novelty songs, gimmicks aren't built to last
Outside of Eminem, the Beasties, and Mac Miller, a lot of mainstream White rappers are heavily catered towards White casuals and non-rap fans who don't actually value hip hop all that much to begin with. But they're comfortable giving a "safe" White rapper a spin over a Black rapper for a period of time.
That's why so many of them switch to Rock/Country when their shining moment in rap has dried up or why so many disappear from the spotlight despite how popular they once were.
The novelty of being the "safe White rapper" is short lived and on borrowed time because the "I don't like rap but G Eazy is kinda cute" or "I'm not a hip hop guy but Macklemore doesn't rap about bytches and hoes" crowd is very fickle and will revert back to being full blown pop, rock, and country fans at any moment when the safe White rapper is no longer popping on the radio.