They include four firefighters who died in an explosion, officials said on Monday.
The fire, which began on Friday evening and raged all night, also injured more than 100 people, mostly workers in the factory in Pingtung county, according to the local government.
One person was still listed as missing as further testing was required to verify the identity of body fragments found at the scene, a county official told AFP.
Taiwan's Central News Agency, quoting a Pingtung County fire official, said an explosion caused part of the building to collapse at about 6:10 p.m. on Friday, trapping firefighters and workers under rubble. A second explosion followed 20 minutes later.
The cause of the fire remained unclear.
A fire department official told local media on Saturday that chemical peroxide stored inside the factory could have been the cause of the one large and several smaller explosions.
President Tsai Ing-wen and other top officials visited the scene on Saturday to express condolences to the victims' families and vowed to investigate the cause of the fire.
Launch Technologies, the operator of the plant, is a major global supplier of golf balls.
According to its 2021 annual report, Taiwan is a major manufacturer of golf balls with factories supplying major brands including Callaway, TaylorMade, Bridgestone, Mizuno and Wilson.
Launch Technologies, which is listed on the Taipei Exchange, shipped about 260 million golf balls last year, accounting for one-fifth of the global total, the Central News Agency said. About 80% of its sales are in the United States.
Taiwan's Ministry of Labor said on Saturday it would open an occupational safety investigation into Launch Technologies and its parent company.
The factory, which is in an export processing zone, has been fined in the past for violations found during inspections.