I visited Japan multiple times for work and never had a issue with racism. When the locals saw me they just wanted to test their English on me.
Ironically the only time I experienced it all was when my Korean co-worker almost got in a fight with a local in Tokyo while taking a smoke. The Japanese guy only picked a fight with him because he was Korean. I had to drag him away. The other time was when we tried to experience some authentic Japanese cuisine in the not tourist parts of Tokyo and we were not allowed in the restaurants and I know it wasn't because of me. It was because of my Korean and Vietnamese co-workers

In regards to living over there, my friends and co-workers who actually lived in country had US jobs so they were straight. They lived like Kings. All of them had nice apartments, some penthouses and others had nice houses in the country.
I had no issues moving around Tokyo, Nagasaki, Kyoto, Sasabo, Fukuoka, Osaka, etc. All the major signs were in English.
Japan has the best food in the world and I'm not talking just Japanese food. I had the best Italian, Indian and Chinese food in Japan.
Its true that you rarely see any trash on the streets and I've never seen a cop walk the beat or a cop cars flying through traffic.
Its true that the men and women under 30 don't interact with each other normally. Its pretty surreal when you walk around and never see young couples out. Its usually packs of boys or packs of girls. Although Japan does have a problem with harassment on the subway so much they have women only subway cars painted pink.