Emergency response teams first received word just before 1 pm (0500 GMT) Tuesday that a fire had broken out at the Beijing Changfeng Hospital in the capital city's Fengtai District, the Beijing Daily said.
The fire was extinguished about half an hour later and rescue efforts continued for another two hours, by which time 71 patients had been evacuated to another location.
Social media users posted videos on Tuesday of people sitting on external air conditioning units, while others clinging to ropes jumped from the building.
The death toll stood at 21 as of Wednesday morning, the newspaper said.
No further details were available on the number of injuries suffered in the fire or the condition of the victims.
Top city officials visited the hospital shortly after the fire, with Beijing party secretary Yin Li vowing to "quickly identify the cause of the accident and hold the relevant responsible persons accountable", according to the Beijing Daily.
AFP journalists on Wednesday saw dozens of onlookers outside the entrance to the hospital, where a large number of police officers were stationed, discouraging people from taking photos.
No visible damage to the hospital's buildings was visible from the entrance to the compound.
It was not immediately clear if all the occupants of the hospital had been found and evacuated from the fire, which was in the east building of the private hospital's inpatient department.
The deaths were confirmed after the victims were taken to another unidentified hospital for emergency treatment, the Beijing Daily report said.
Many family members have lost contact with patients, with the missing mostly older people with mobility problems, the China Youth Daily said in a separate report on Wednesday.
Beijing Changfeng Hospital is located in the capital's western urban area, about 25 minutes by car from Tiananmen Square.
Deadly fires are common in China due to weak safety standards and lax enforcement.
Ten people died in an apartment block blaze in China's northwestern Xinjiang in November. And 38 people were killed in a fire at a factory in central China, also in November.