Cam Atkinson, Foligno, Scott Hartnell and Mark Letestu all scored third-period goals for the Jackets (23-25-3), who won for the second time in as many nights, but it was McElhinney who stole the show with 12 saves in the third period.
"I thought we played a pretty solid game and a very big win on the road for us," said McElhinney, who played seven games for the Senators during the 2010-11 season.
"We talked about it before the third period that it was going to be an ugly goal to get the lead and their goalie was making some great saves so it was just a matter of who was going to break first."
Erik Condra had the only goal for the Senators (20-22-9), who have now lost three straight. Robin Lehner made 29 saves as he started a career high fifth straight time.
"You play and practice a system for a reason and that's to give your team the best chance to win games," Senators coach Dave Cameron said.
"You look at tonight's game and we generated enough chances but we didn't finish them. You have to continue to do the things that gave you those chances and eventually you hope that they're going to go in."
Atkinson scored the game's first goal at 4:13 of the third period when he converted his own rebound. Atkinson grabbed the loose puck at the side of the net on his forehand, pulled it to his backhand and beat Lehner.
Foligno then gave the Blue Jackets some breathing room as he broke down the left side, fought off two Senators and slipped a backhand through the legs of Lehner at 8:51 for a 2-0 lead.
Foligno was drafted in the first round by the Senators back in 2006 and played four seasons in Ottawa before being traded to Columbus prior to the 2012-13 season.
"It's meaningful for sure," Foligno said of his game winner.
"It's a team you used to play for and you get up for those games and you want to do well against them. I'm just proud of the way we played."
That goal proved huge as Condra scored less than two minutes later to cut the lead in half at 10:32.
Hartnell then scored into an empty net at 18:44 and Letestu beat Lehner at 19:47.
The Blue Jackets came into the game off a 7-1 beat down at home over the St. Louis Blues Friday night, but lost defenceman James Wisniewski and Ryan Murray to injury in the process.
That loss was evident at times throughout the game as the Blue Jackets struggled in their own zone, especially in the second period when the Senators had extended pressure on different occasions.
Despite that pressure, the game was scoreless through the first two periods but both teams did have their opportunities as the Blue Jackets held a 24-22 advantage.
Lehner, who was making his fifth straight start, made some key saves but the best chances came at the other end of the ice where McElhinney made a huge pad save off Clarke MacArthur and then got help from his left goal post on a shot from Mark Stone.
Notes- Chris Phillips, Milan Michalek and Patrick Wiercioch were scratches for the Senators while Jordan Leopold, James Wisniewski and Ryan Murray were scratches for the Blue Jackets…Senators defenceman Cody Ceci played his 100th NHL game…Blue Jackets forward Nick Foligno has already set career highs in goals (21) and points (48) this season.