That's just how it is, broski.
Some people won't even talk to you on the fact that you're Black. Let me explain.
In Japan, in general, every Black person you see will be AFRICAN. Not African British, or African American, FROM THE MOTHERLAND.
Typically, those brehs work the club areas (like Kabukicho and Roppongi in Tokyo. I'm not as familiar with Osaka). These guys are known as "touts", and their main goal is to nab a foreigner and get them in their overpriced club. They almost exclusively speak English or a broken form of English. They'll also occasionally be soliciting johns for prostitution-- this happened to me NEAR CONSTANTLY when I was lost in Shinjuku a little while back.
These nikkas are MAD aggressive, and some will even grab you
So I said all that to say this-- Japanese people can't tell you from an African tout breh. We all "look the same" to them. If you approach one of them speaking English, they can't tell you from a tout trying to get them into a club or promote a club.
If you speak Japanese however, their guard goes down, even if only a little.
So you can try to hook up with chicks using Google translate, but personally, I've only seen people get so far without the language. But hey, it's a first time for everything, right?
Good luck, fam.