Silverman is in a difficult and demanding position as a spokesperson for an organization which represents a hated minority, and devotes much of its time to interposing themselves between those who seek to oppress that minority. That he's out there, and the specific content of his advocacy, are strongly determined by that role. Having served on a board of directors before, I understand how such a role both limits what you can say, how you can say it, and also what you need to say, whether you're into it or not. (i.e. The Show Must Go On) I haven't seen much of him, but he seems to do a very credible job of abetting that role. But just as Bill O'Reilly plays a role for his own purposes, I think you need to separate the role from the man. (Continuing this analogy, Silverman plays that role because he's a tireless defender of a minority. Which, I guess in its own right, is exactly what Bill O'Reilly does; he's a tireless defender of the interests of a minority: himself.)
The only time I've seen Silverman at any great length was at an American Atheists conference, and he seemed fairly relaxed there, but there were still elements of the politician in evidence, even among an auditorium full of like-minded people. I admire anyone who takes on a role such as that, and does it well. Silverman isn't a genius, but he does very well.