Science was once a child of the church.

[One Above All]

Well if I use your formula, both atheist and religion are declining.

Not possible. Even though some atheists are religious (Buddhists, for example), most religious people are theists, like you. If both were declining, it would mean that the entire human population was dying off.

May have helped if religion went after more men, and atheism had more than just mostly white men.

Atheists come in all shapes and sizes, backgrounds, sexualities, skin colors, and so on.
Theists are... a little more restrictive. You have to be what they tell you to be. You have to follow the same insane rules as they do. You have to pray at certain times of the day for random things, eat only specific things or nothing at all (usually during the course of a day), deny everything science has proven, since it also disproves your religion (except when it might save your life, like antibiotics, or make your life easier, like electricity and cars), et cetera.