The hipster cac kids and suburban black kids that were Rocky's backbone of support are in their mid to late 30's now...and they didn't stay attached to his music as they got older.
His fan base grew up and he couldn't make good enough music to keep them around.