As someone who got into 90s rap after the fact even I can see the differences. Especially with how quickly artists would fall off then too. Nowadays guys can make the same shytbover and over but in the 90s it seemed like if your latest album sounded like your last shyt you were DONE. There were so many advancements in technique that it was easy to fall behind. The biggest shift I notice happens in 94/95, and then again in 97.
I think it's largely cuz it was stil new. 90s was the first full decade where hip hop was commercially viable so its like a kid growing up...10 and 15 year old you are WILDLY different, whereas 30 and 35 year old you it's more subtle
A lot of that turnover carried on from the 80s. Like you said, hip hop was still a new genre, so advancements were constantly being made and the sound changed rapidly every few years. You were considered old school after a few albums. Run-D.M.C. was at the top of hip hop in 1986. They crossed over to the mainstream, had a triple platinum album, and an endorsement deal with
Adidas. Two years later, their next album sold half of what Raising Hell sold, and they were already considered dated compared to guys like Rakim and Kane. DMC thought his career was over when he heard "Ain't No Half-Steppin'," and a year after that, Slick Rick said in an interview that Run-D.M.C.'s not even part of the conversation anymore. They became has-beens in three years.
It was close to happening to LL before Mama Said Knock You Out. I read a Source article from '93 and they were interviewing Tribe around the time Midnight Marauders came out. According to the article, the streets were saying that Rakim was wack and Run-D.M.C. was back. "Down with the King" made the group popular again, but that's because they hooked up with Pete Rock and C.L. Smooth who were hot at the time. Kane released an album that year, and he said the reason it didn't work was because of him. He had the hot producers, but he didn't realize that around that time, rappers weren't rhyming on top of the beat anymore. Their flows were slower and they were rhyming behind the beat, so Kane sounded old-school by comparison.
When Public Enemy dropped their first album, it was delayed by a couple months so it was already dated. It was a 1986 album that came out in 1987. When Chuck D and Hank Shocklee heard "I Know You Got Soul," it inspired them to come up with a brand new sound and then they made "Rebel Without a Pause." Run-D.M.C. ended up taking that sound and imitating it on their next album because they didn't want to be left behind, but the songs that really hit ("Run's House," "Beats to the Rhyme") were them just being themselves.
This is a long post, but it's just that hip hop was always moving in a different direction for years until it stagnated. Nowadays, you can still have a career even when your albums flop (Chance the Rapper). Back then, the minute you fell off, you were done being a star.