Hoang Van Phuc, who used to manage Vietnam's national football team, is expected to take Nghiem's place.
The decision to dismiss Nghiem followed Hanoi FC's 0-2 loss to SHB Da Nang on Friday.
Nghiem, 49, used to play for LG ACB and Hoa Phat Hanoi. After retiring, he has been managing Hanoi FC since its founding.
Under his tutelage, Hanoi has won the V-League championship thrice, the National Cup twice and Super Cup thrice. The team also once entered the AFC Cup semi-final. But the club took a blow after losing the 2020 V-League crown to Viettel, suffering an array of failures this season. The Friday loss against Da Nang was believed to be the tipping point.
Phuc may debut as the new manager on April 7 as Hanoi FC takes on Viettel in a round eight derby.
Earlier this week, Saigon FC also parted ways with its Japanese manager Shimoda Masahiro after losing four out of six games into the season.