Part of becoming a super saiyan is channelling your anger into power. Vegeta didn't know how to do that at the time. Hell, he just barely learned how to sense power levels and raise his own power to hide from scouters.
but he channeled his anger outta jealousy for Goku being a super saiyan. he wanted to be the greatest Saiyan alive, thats why he pushed himself to the limit to become SS