Stores on Nguyen Duy Duong Road in District 5 and nearby roads shut down since at the start of this month and have yet to reopen.
The owner of a store said: "Some fear they cannot furnish the required permits to authorities should they be asked."
An employee in another store said small shops have been selling at losses for a long time and any financial penalty would only increase their challenges.
Since early April the city Directorate of Market Surveillance has been discovering wrongdoing by gold stores, some selling knockoff Chanel, Versace and Louis Vuitton earrings, necklaces and bracelets.
Many also sell jewels and gold without documents to indicate their origin.
A top directorate official said the campaign is meant to increase transparency in the market and reduce fakes.
Similar inspections are being done nationwide.
In the northern province of Quang Ninh, the department said it had found 12 stores committing violations and fined them a total of VND221 million (US$8,700).