Yoon backed down after lawmakers voted to oppose the unexpected declaration, which caught even South Korea's closest allies around the world off guard.
Earlier the National Assembly was sealed and troops entered the building for a short time, while hundreds of protesters gathered outside and faced off with security forces.
"Just a moment ago, there was a demand from the National Assembly to lift the state of emergency, and we have withdrawn the military that was deployed for martial law operations," Yoon said in a televised address around 4:30 am (1930 GMT Tuesday).
"We will accept the National Assembly's request and lift the martial law through the Cabinet meeting."
Yonhap news agency reported that Yoon's cabinet approved the motion to lift the order.
The U-turn prompted jubilation among protesters outside parliament who had braved freezing temperatures to keep vigil through the night in defiance of Yoon's martial law order.
Demonstrators waving banners and South Korean flags and chanting "Arrest Yoon Sul Yeol" outside the National Assembly erupted in cheers.
Lim Myeong-pan, 55, told AFP that Yoon’s decision to rescind martial law did not absolve him of wrongdoing.
"Yoon’s act of imposing it in the first place without legitimate cause is a serious crime in itself," Lim told AFP.
"He has paved his own path to impeachment with this."
Some 190 lawmakers had managed to get in to the assembly in the early hours of Wednesday, where they unanimously voted in favour of a motion to block the martial law declaration and call for its lifting.
Under the constitution, martial law must be lifted when a majority in parliament demands it.
International concern
Yoon had given a range of reasons to justify martial law -- South Korea's first in more than 40 years.
"To safeguard a liberal South Korea from the threats posed by North Korea's communist forces and to eliminate anti-state elements plundering people's freedom and happiness, I hereby declare emergency martial law," Yoon said in a live televised address late on Tuesday night.
Yoon did not give details of the North's threats, but the South remains technically at war with nuclear-armed Pyongyang.
"Our National Assembly has become a haven for criminals, a den of legislative dictatorship that seeks to paralyse the judicial and administrative systems and overturn our liberal democratic order," Yoon said.
Democratic South Korea is a major ally for the United States in Asia, but Washington said it was not given advance notice of Yoon's plan to impose martial law.
"We are watching the recent developments in the ROK with grave concern," Deputy Secretary of State Kurt Campbell said after martial law was imposed, referring to South Korea by its official name, the Republic of Korea.
China, a key ally of North Korea, urged its nationals in the South to stay calm and exercise caution, while Britain said it was "closely monitoring developments".
The decree by martial law commander Park also banned "actions that deny or seek to overthrow the liberal democratic system, including the spread of fake news, public opinion manipulation, and false propaganda".
The president labelled the opposition, which holds a majority in the 300-member parliament, as "anti-state forces intent on overthrowing the regime".
Yoon described the imposition of martial law as "inevitable to guarantee the continuity of a liberal South Korea," adding that it would not impact the country's foreign policy.
He described the current situation as South Korea "on the verge of collapse, with the National Assembly acting as a monster intent on bringing down liberal democracy".
Budget row
Vladimir Tikhonov, professor of Korea studies at the University of Oslo, said Yoon's move to impose martial law was "an attempt to wind the history back".
"I don't think South Korea's civil society can recognize Yoon as a legitimate president any longer," he told AFP.
Yoon's People Power Party and the main opposition Democratic Party are bitterly at odds over next year's budget.
Opposition MPs last week approved a significantly downsized budget plan through a parliamentary committee.
The opposition has slashed approximately 4.1 trillion won ($2.8 billion) from Yoon's proposed 677 trillion won budget plan, cutting the government's reserve fund and activity budgets for Yoon's office, the prosecution, police and the state audit agency.
The imposition of emergency martial law came after Yoon's approval rating dropped to 19 percent in the latest Gallup poll last week, with many expressing dissatisfaction over his handling of the economy and controversies involving his wife, Kim Keon Hee.