Mary is not a virgin by the Bible accounts

[Minimalist]

I figured she was supposed to be a virgin until Jesus' birth? Then her and Joseph could go at it all they want.

For most protestants yes, however older branches like Catholics, Coptics and Eastern Orthodox; most of them believe in the "perpetual virginity of Mary". This is mostly because, Catholics moreso than the others, consider marriage to be a sacrament and mostly indissoluble.

But only since the 12th century.  Including marriage as a sacrament seems to have been a reaction to the Albigensian Crusade against the Cathars.