The former world number one has been battling against knee and back injuries this year and was not at his best as he succumbed to the 22-year-old Thiem at the Foro Italico.
Thiem, ranked 15th after reaching a career-high 13th earlier this year, recovered from an early break and streaked into a 6-2 lead in the tiebreak before taking the set with a backhand winner.
He broke 34-year-old Federer's serve again midway through the second set as he claimed the 32nd victory of a breakthrough season.
The Swiss, a four-times finalist in Rome, underwent arthroscopic knee surgery after this year's Australian Open and was sidelined for 10 weeks before returning at the Monte Carlo Masters in April.
Federer missed last week's Madrid Open with a back problem and, with the French Open looming at the end of the month, the 17-times grand slam champion faces a fitness race ahead of Roland Garros.
Later on Thursday Rafael Nadal continues his build-up to the French Open when he takes on Australian Nick Kyrgios.
World number one Novak Djokovic, bidding for a third consecutive Rome title, is up against Brazil's Thomaz Bellucci while women's top seed Serena Williams takes on fellow American Christina McHale.