Scholz, 63, who served as vice chancellor and finance minister in coalition with Merkel, got a clear majority of 395 votes from lawmakers in the Bundestag lower house of parliament, Bundestag President Baerbel Bas said.
Scholz, wearing a black face mask, waved as he received a standing ovation from lawmakers and was given bouquets of flowers and a basket of apples from the leaders of the parliamentary groups.
Sticking to the democratic procedures enshrined in Germany's Basic Law, Scholz was formally nominated by President Frank-Walter Steinmeier in the nearby Bellevue Palace before returning to parliament to take the oath of office in front of lawmakers.
In the afternoon, Merkel will officially hand over the chancellery to the new leader of a country facing a brutal fourth wave of coronavirus infections and challenges to its democratic order by authoritarian governments.
With his down-to-earth and no-nonsense manner, Scholz has positioned himself as Merkel's natural successor and a safe pair of hands to steer Germany through challenges ranging from tackling the climate crisis to dealing with a more confrontational Russia and increasingly assertive China.
Scholz will lead an unprecedented three-way ruling coalition on the federal level with the pro-spending, environmentalist Greens and the fiscally more conservative, libertarian Free Democrats (FDP) - unlikely political bedfellows in the past.
Scholz is an experienced negotiator and veteran politician who as SPD party secretary general from 2002 until 2004 defended controversial labour market reforms and social welfare cuts under former Chancellor Gerhard Schroeder.
After being mayor of the northern port city of Hamburg from 2011-2018, Scholz returned to Berlin as Merkel's finance minister - a role in which he got rid of the goal of balanced budgets and enabled record new borrowing to shield companies and employees from the impact of the coronavirus pandemic.
On the European level, he worked closely with France and persuaded Merkel to support a debt-financed European Recovery Fund worth 800 billion euros ($900 billion) to help EU member states hit hardest by Covid-19.