Mark Cooper's men moved ahead shortly before the half-hour mark when Michael Smith kept his cool after being played in by Massimo Luongo, before Nathan Byrne extended the lead shortly after the restart.
Orient, who remain without a manager after Russell Slade's resignation last month, scored a consolation deep into injury time through Darius Henderson's header, but it was not enough to stop a Swindon side who have now gone eight games unbeaten in League One.
Goals from Michael Bostwick and Marcus Maddison had cancelled out Jonathan Forte's opener for Oldham, but the visitors left London Road with a point when Genseric Kusunga headed home Mike Jones' corner.
Leaders Bristol City remain unbeaten despite Romaine Sawyers' well-taken 85th minute goal rescuing a 1-1 draw for Walsall.
Jimmy Ryan put the hosts ahead, before United were hindered by Michael Higdon's dismissal for a hefty challenge on Daniel Jones, while Chesterfield goalkeeper Tommy Lee too saw red before the break.
Crewe Alexandra's miserable campaign continued in a 2-0 loss at Bradford City, while beleaguered Scunthorpe United were beaten 2-1 by Doncaster Rovers.
Fleetwood Town ended an seven-match winless league run with a 1-0 victory over lowly Port Vale, Stephen Dobbie's 80th-minute penalty sinking the visitors who had Jordan Slew sent off.
