Sounds like a great idea, It's probably best if you have a small group of likeminded people with a good chemistry to help you get things going and organize. Maybe via social networks? You can't institute an atheist test though of course, so in the end who ever will come will come. Maybe giving the endeavour an appropriate name should suffice. If you should invite a speaker for a talk in your club, it might make sense to advertise it more widely to attract and get to know new people. Of course then guests of other persuasions might attend.
The fool hath said in his heart, There is a God. They are corrupt, they have done abominable works, there is none that doeth good.
Psalm 14, KJV revised edition