There's no support for athletes there, poor infrastructure, and unheard of levels of corruption.
Cultural differences; indian parents don't value sports as a career, they'd rather their kids study. Plus, they only want to play cricket.
Class/caste systems; the majority of poor, talented athletes will probably never be noticed, and the rich ones are mostly lazy or egotistic. The poor probably have to buy themselves in.
If they do well at the olympics they'll be treated great, but if not you might find them begging on the streets a few years later. If you get injured, then what?
The money the government would spend to develop coaching and facilities for olympic sports is eaten at every level and there ends up being crumbs to actually train athletes.
I've never lived in India but I've been there several times and I'm telling you brehs, the corruption is endless