Religious and atheist diversity

Does diversity only count against religion, or atheism too?