Christianity has a long history in Japan, going back at least to the period before the Tokugawa shogunate closed the country to most trade and tourism. (There is also some reason to believe that the Tea Ceremony may be in part an adaptation of the Eucharistic meal, perhaps in the form in which it was brought to China by the Nestorians more than a thousand years ago.)

I remember hearing Christmas carols (don't recall if it was specifically Kiyoshi kono yoru) in the classic Japanese TV series Kita no kuni kara ("From the North Country").

Christianity has had an influence in Japan much greater than the percentage of the population that officially "belongs to a church" or "has been baptized" would suggest; consider, for example, the fiction of Shusaku Endo. And Gallup polls of the Japanese population suggest that despite the churches' failure to account for more than 1% of the population (and even that tiny slice of the pie has shrunk in the past half-century), as many as 8 to 10% of the Japanese surveyed self-identify as, in some sense, Christians or followers of Christ. When I was a kid in Tokyo (1967-68) a large percentage, probably a majority, of the kids in the neighborhood attended Sunday School at our local (Tomizaka Kyokai) church, which was run by the German East Asia Mission and affiliated with the Free Church of Leipzig, I believe, despite the fact that they and we all took part, too, in the local Shinto festivals (taiko banging and whatnot) and their parents returned a Communist deputy to the Diet... It's a very "fusion-oriented" culture.

On top of that, the secular Christmas of Santa et al is big business in Japan, along with the even more significant New Year. So there's nothing particularly odd about "Silent Night" being sung in Japanese, imo.