The Social Democrats (SPD) and Greens won by a single seat after a close race for power in Lower Saxony, Germany's fourth most populous state.
"A one-vote majority can be very stable," Hanover mayor and SPD state premier candidate Stephan Weil told reporters, after a nail-biting six-hour wait for results that were long too close to call.
The opposition is expected to use its victory to create fresh momentum in a bid to deprive Merkel of a third four-year term.
Her Christian Democratic Union (CDU) and the liberal Free Democratic Party (FDP) ruling alliance has lost a number of state elections in recent months.
The SPD’s candidate for chancellor, Peer Steinbrueck, has been struggling to rally support after a series of public gaffes.
Steinbrueck acknowledged "shared responsibility" for a patchy campaign in Lower Saxony, a major agricultural and industrial centre, but told reporters that the result proved that the SPD was clearly still in the race for September.
"This means a change of government and of power are possible this year," Steinbrueck said.
Still, the 58-year-old Chancellor enjoys high personal popularity for her leadership role in the euro crisis where she defended Germany's economic interests.
The result was disappointing for David McAllister, the half-Scottish state premier who has been spoken about as a potential successor for Merkel. She made seven personal appearances in his campaign.
No pundit can remember such a close election.The cliffhanger contest turned Lower Saxony, which is the size of the Netherlands and stretches from the Dutch border to the former East Germany, into an election battleground watched by the whole country.
The euro crisis did not play much of a role in the vote, with local issues like education, infrastructure and state spending dominating Germany's industrial and farming heartland.
With the change of government in Lower Saxony, the center-left opposition now has the majority in the Bundesrat — Germany's upper house of parliament, which represents the federal states.
The Social Democrats have already announced initiatives for a nationwide minimum wage, tougher prosecution of tax evasion and the implementation of a property tax.