Oh it's a holiday for the celebration of Jesus' birth alright. That the date and trappings are pagan is beside the point. No religious group gets to decide how or if others celebrate or don't celebrate a holiday even if that holiday is very special to the religious group.
If there is a god, I want to believe that there is a god. If there is not a god, I want to believe that there is no god.