One can say "It's just politics" or "It's just religion" or whatever. "It's just..." is simply a way to minimize something that other people are passionate about.

Also it is a very juvenile way to escape an argument you aren't winning. If it is "just fiction" then why did he bother discussing it at all?