I'm not eager and it is not a little thing. You incorrectly used the term. He was not using the insult to refute your claims. He was insulting you, on top of refuting your claim.
"You're wrong because you are an idiot" is an ad hom. "You're wrong because of A, B and C, you idiot." is not an ad hom.
"You're wrong because you are an idiot" is an ad hom. "You're wrong because of A, B and C, you idiot." is not an ad hom.