Bury gave away two penalties and had a man sent off but still battled from behind to take a point.
They scored first against a rebuilt Rovers side but all seemed lost when they trailed 2-1 and had Colin Woodthorpe dismissed for a second booking.
Both sides were eager to build on their Saturday wins and Bury's David Nugent fired in an early testing shot while for Rovers Ali Gibb saw a long shot hit the bar.
Nugent then won a corner and Rovers were caught out when Paul Trollope gave Dwayne Mattis too much room and the midfielder scored with a free header.
Trollope was never likely to make that mistake again but Rovers almost stumbled into futher trouble as pressure came from another corner.
Mattis continued to be the pivotal player for Bury and could have scored again when Nugent squared the ball to him - but his straightforward shot didn't have enough power.
It began to look like an expensive miss when Rovers were given a dubious penalty five minutes before the break.
Again Mattis and Trollope were centre-stage with the Bury player giving away a free-kick 15 yards inside his half.
Trollope swung the kick to the right and led the appeal for a penalty when Junior Agogo went down under a push. It sparked an extraordinary ten minutes either side of half-time.
Jamie Forrester sent the keeper the wrong way from the spot. Then early in the second half John Anderson put Rovers ahead from a corner.
Within a minute Forrester had a second penalty chance when he was brought down Colin Woodthorpe. This time he went for the opposite corner but Glyn Garner smothered the shot.
Woodthorpe was then sent off when he brought down Agogo.
Bury deserved their point, courtesy of a goal by Brian Barry-Murphy. Unfortunately his celebrations earned him a booking.