Indonesia, Thailand, the Philippines, and Malaysia are likely to achieve growth of 3.5 percent or less.
Last year the four countries reported negative growth, while Vietnam grew at 2.9 percent.
Asia is set to grow by 6.5 percent, driven by India (10 percent against negative 7.3 percent last year) and China (8 percent against 2.3 percent last year).
The disruption of supply chains due to the resurgence of Covid-19 combined with price pressures and rising inflation would pose more challenges to growth, the IMF said.
It forecast growth of 5.9 percent for the global economy this year, down 0.1 percentage point from its July projection.