Yes, languages get mistaken for each other all the time when one is not familiar with the writing system, and sometimes even when one is. I have struggled to understand posts in Spanish before realising they're actually in Portuguese, which I don't speak. (Also I'm pretty sure Norwegian and Danish are actually the same language).

Can you tell the difference between Telugu and Kannada just by looking at them? How about Arabic and Persian? How about Arabic and Ottoman Turkish? Persian and Kurdish? Yoruba and Xhosa? Ukrainian and Kazakh? Sumerian and Akkadian? Actually, could you tell the difference between Akkadian and Old Persian? They are both written with cuneiform characters, but the characters themselves are apparently as different as hiragana and hangul.

If your sentence is written entirely in Chinese characters, there is no way for somebody unfamiliar with them to determine whether it's Japanese, Mandarin, or Hokkien. And if somebody hasn't seen enough Japanese text to figure out the difference between kana and Chinese characters, they still won't be able to tell the difference.

As to why Google can't tell, Google doesn't actually understand anything. It's based on a massive database of which characters and combinations of characters come next to each other (and there's also some Markov stuff to account for common spelling mistakes). If your search string is made entirely of Chinese characters, it's going to get hits on websites written with Chinese characters, many of which will be in Mandarin. Google.com probably isn't able to detect your UI or browser language settings. To ensure you get results from Japan, try using google.jp instead, as it will prioritise Japanese results.