Biden, 80, has frequently expressed his intention to run for a second four-year term, and advisers expect him to make it official in the coming weeks.
With Biden standing at his side in the White House East Room, Klain choked back tears in an emotional departure speech to dozens of staff members, many of whom consider themselves "Klainiacs."
"As I did in 1988, 2008 and 2020, I look forward to being on your side when you run for president in 2024," Klain said.
Biden is to complete a physical examination by February 16, his press secretary, Karine Jean-Pierre, told reporters earlier.
The president appeared to be wiping away tears as he listened to Klain's speech.
"I knew this day would come, to state the obvious, but it doesn’t make it any easier," Biden said.
Klain is to be succeeded as chief of staff by Jeff Zients, who had handled the Biden administration's Covid pandemic response.