They had looked doomed to a second successive defeat as Richard Walker hit home both spot-kicks.
But the power of their fightback shredded Rovers' confidence and with boss Martin Ling using all three subs Leyton could easily have scored more.
Wayne Gray began their stunning revival just short of the hour as he moved on to a deep cross from sub Jabo Ibehre which wrong-footed keeper Steve Phillips.
With Rovers' defence now quivering Leyton's centre-back Tamika Mkandawire grabbed the equaliser with a close-in stab, before sub Jason Demetriou hammered in a long-distance winner.
Like the penalties this one was hotly disputed. A linesman sided with Rovers, but referee Steve Bratt, after brief consultation, over-ruled him.
Leyton's win was deserved but Rovers, still without a home win, were left wondering how this one slipped away.
Rovers took the lead just before half-time when Lewis Haldane, who had been tormenting the O's defence with his speed, won a penalty for handball.
The visitors protested, claiming the shot had accidentally hit Sean Thornton and before the row subsided their keeper Stuart Nelson was booked.
Walker sent him the wrong way and with his second penalty early in the second half hammered his kick straight down the middle.
Like the first penalty Haldane was at the centre of the second one, being tripped as he went past Adam Chambers.
Chambers had been switched to right-back to mark Haldane who had been bypassing Paul Terry with his pace.
Walker has now scored three times from the spot this season the other one coming against Nottingham Forest, who were also two down before fighting back to draw.