Previously, corruption investigations in the People's Liberation Army, the world's largest armed forces, were handled in a more ad hoc fashion. But under reforms started last year the military now has a dedicated graft-fighting division.
Xi has led a sweeping anti-corruption campaign targeting high-ranking officials in industry, government, and the military.
The military is reeling from the crackdown and has seen dozens of officers investigated, including two former vice chairmen of the powerful Central Military Commission, Guo Boxiong and Xu Caihou.
Xu died of cancer last year before he could be bought to trial while Guo was accused of accepting bribes last month.
The official Xinhua news agency said the new graft inspectors completed two days of training on Wednesday and had been organised into 10 teams who would go to various units around the country. It did not say which units would get inspectors.
Xu Qiliang, a vice chairman of the Central Military Commission, which controls the military and which the president heads, told the inspectors they were fulfilling an important role in cleaning up and strengthening the armed forces, Xinhua said.
"Keep firmly in mind the expectations and great trust of Chairman Xi," the news agency paraphrased Xu as saying.
The move coincides with broader efforts to reform the military, including the modernisation of its command structure, as China becomes more assertive in its territorial disputes in the East and South China Seas.
China is also investing heavily in new technology, including aircraft carriers, stealth jets and anti-satellite missiles, though it has not fought a war in decades.
Serving and retired officers have warned that corruption in the military could threaten the ability to wage war.