I do not believe things unless they have a convincing body of evidence to support them. And if they turn out to be justified true beliefs then I find myself obliged to hold them since I care about whether what I believe is true. If I open myself up to one false belief then I open myself up to any false belief.
"If an injury must be done to a man, it should be so severe that his vengeance need not be feared" - Niccolo Macchiavelli