The Spain striker scored on his Chelsea debut in a 3-1 win over Burnley on Monday and was back in the goals five days later, breaking down Leicester's stubborn resistance at Stamford Bridge in the 63rd minute.
Belgium winger Eden Hazard's solo effort in the 77th ensured Chelsea has maximum points after two games against teams promoted from the second-tier League Championship.
Swansea is the only other side with back-to-back wins after beating Burnley 1-0, while Arsenal has four points after scoring twice in the final seven minutes through Aaron Ramsey and Olivier Giroud to recover for a 2-2 draw at Everton.
West Ham won 3-1 at managerless Crystal Palace while the games between Aston Villa-Newcastle and Southampton-West Bromwich Albion finished 0-0.
Didier Drogba was given a standing ovation from the Chelsea fans when he came on as a second-half substitute for his first home match in his second spell with the London club.
But the Stamford Bridge fans have a new goalscoring star in Costa.
Signed for 32 million pounds ($53 million) from Atletico Madrid in the off-season, Costa is proving exactly the kind of ruthless finisher Chelsea was lacking last season as it came up short in the title race.
"I got the feeling in preseason that his adaptation would not be a big deal, he is an easy guy," Chelsea manager Jose Mourinho said. "His style of play has adapted to our needs, so it has not been a problem."
Costa's goal came after Oscar played in Branislav Ivanovic on the right side of the area, and the right back's cutback was chested down by Costa and drilled past Leicester goalkeeper Kasper Schmeichel from six yards.
Midfielder Cesc Fabregas and goalkeeper Thibaut Courtois are other arrivals at Chelsea this summer and played key roles in a hard-fought win against last season's second-tier champions.
Courtois — back at Chelsea after his loan spell at Atletico — again started ahead of Petr Cech and his save from David Nugent when the striker was through on goal in the 55th minute was crucial.
"I don't forget they had a good chance with the result 0-0," Chelsea manager Jose Mourinho said, "but we had so many chances, so many opportunities to score obviously we deserved the victory."
Fabregas had two assists in a man-of-the-match display against Burnley and he set up Hazard for the second goal against Leicester, with the winger collecting the ball on the left, cutting inside and drilling a low shot past Schmeichel into the corner.
"I thought we played exceptionally well in the first half but it's about taking your chances when you get them," Leicester manager Nigel Pearson said. "There are more positives today than there are negatives."
Giroud, a halftime substitute, headed home with 40 seconds left of normal time to grab a point for Arsenal at Goodison Park, with Ramsey's close-range goal starting the visitors' fightback in the 83rd.
Goals by Seamus Coleman and Steven Naismith put Everton comfortably in front at halftime.
"We had a big test today and at halftime it was even a bigger test," Arsenal manager Arsene Wenger said. "We responded well."
West Ham increased the early-season misery for Palace, which conceded a late goal to lose to Arsenal last weekend and is still without a manager following Tony Pulis' resignation two days before the start of the season
Goals by Argentine striker Mauro Zarate — on his debut — Stewart Downing and Carlton Cole secured West Ham's first points of the season after losing 1-0 to Tottenham first time out.Suggest a correction