You're confusing work (pushing the two masses) with acceleration. When objects are accelerating due to gravity, the mass of the object doesn't matter. Galileo demonstrated this by dropping a cannon ball and a bullet from a height, and they hit the ground at the same time. So no, gravity doesn't 'push' or 'pull' either upwards or downwards - is an attractive force between two masses.
Boru
Boru
‘I can’t be having with this.’ - Esmeralda Weatherwax