Just a thought. If not negative reputation points, perhaps a separately tracked number of "troll points." (Or, maybe track positive and negative rep both, separately.)
(Voicing such concerns is likely entirely more candidly self-disclosing than is my druthers, but whatever.)