Rutgers football and basketball have no real history of any sustained success. As mentioned before, traditional power programs have come in and raided Jersey talent and they've been doing it for decades. In football, Notre Dame and Penn State probably consider Jersey kids in-state. Before Schiano came along, Rutgers had only appeared in one bowl game and 3 players picked in the top 3 rounds of the NFL draft. In Basketball, again lack of history. It's been 35 years since Rutgers won an NCAA tournament game and almost 30 years since they've been in the tournament period. Rutgers main recruiting base was also a major base for much more successful Big East & ACC programs. And kids who wanted to stay home had a much better in-state option in Seton Hall.
Rutgers is a really good school but the campus is kinda disjointed and in sports they're a small fish in a pond with big fish. And while the pond has gotten bigger since moving to the Big 10, the fish have gotten a lot bigger as well. Rutgers still won't get a full share of the Big 10 TV revenue until 2020-21. They're getting less than 20% this year, slightly less than 30% next year and slightly less than 40% the year after that. On top of that, comparing apparel deals, the total value of the deal Rutgers signed with Adidas last year is 23 times less than the deal Ohio State signed with Nike 2 years ago. They likely won't even be able to compete until they're financially level with the rest of the Big 10 (TV money wise) 3 years from now.
Bottom line not enough advantages and too many disadvantages compared to competitor schools. And until they can find their next Schiano, a legit program building coach who can recruit their ass off, Rutgers will be non-factors and doormats in the Big 10 for the forseeable future in football and basketball.