I felt the same way about VR before I played it.
@Fatboi1 will tell you I gave Sony hell for PSVR before it launched. I criticized it for what I perceived to be a gimmick.
After playing it extensively I can tell you that unless you actually play it and experience all it has to offer first hand, you really have no idea what it's like.
The technology is incredible and so is the immersion it provides. There is nothing else like it.
You can't play Resident Evil 7 in PSVR and walk away saying that it's a gimmick.
I'm not saying that you have to go out and buy a PSVR. I'm just saying that until you use it, you can't speak on it because there is absolutely nothing else like it. I learned that first hand and had to eat my words.
Your statement: "You should want to play the game even with the VR stripped away as in if they made the same game but built it for a conventional controller the game should still be good." is the EXACT experience you get playing Drive Club VR, Eve Valkyrie, Resident Evil 7, and I'm sure is the case with Gran Turismo. These are all games that on a regular TV are great, but when play in VR offer a level of immersion that completely changes the experience.