Is atheism a belief?

[unfogged]

I think the difference is often in the question being asked. If the question is "does god exist" then the simple atheist/agnostic/theist options work best but if the question is "do you believe that a god exists" then that is a yes/no question with degrees of certainty and that's where it gets more interesting IMO. You either believe a proposition or you don't. When people say they are undecided as to whether or not they believe they are actually undecided on the first question, not the second.

I grew up with the atheist="there is no god" and agnostic="I don't know" definitions but generally find that other definitions can be more useful because, as you see in the 4-quadrant chart you posted, it lets you be more nuanced quickly. The definitions I tend to prefer are:
theist/atheist = does/does not accept the claim "a god exists"
gnostic/agnostic = is/is not certain
By those definitions I'm an gnostic atheist with regard to christianity, islam, etc and an agnostic atheist with regard to the question if there is anything that deserves the label.

In the end, what is most important is that you ensure that you and your interlocutor are using the same definitions or at least understand what the terms mean to each other. Words are just tools to convey ideas; they don't have single intrinsic meanings.