Whether it's a problem in your relationship only you can say. I'd have the talk about how you would raise your children sooner rather than later though.
If there is a god, I want to believe that there is a god. If there is not a god, I want to believe that there is no god.