Slightly over a million foreigners from countries such as China and Vietnam were working in Japan as of last October, labor ministry data showed on Friday.
That was up nearly 20 percent from the previous year and a new record for the fourth straight year.
The figures suggest Japan is increasingly turning to workers from overseas to plug its labor shortages despite its reluctance to accept foreigners.
Japan is facing its worst labor crunch since 1991 amid a shrinking and aging population, which has prompted calls from the International Monetary Fund to accept foreign workers.
Prime Minister Shinzo Abe has said more Japanese women and the elderly should work first before accepting immigrants, but policymakers are exploring ways to bring in more foreign workers without calling it "immigration".
The labor shortage is especially severe in the construction sector, where demand has spiked ahead of the 2020 Tokyo Olympics and for rebuilding following the 2011 earthquake and tsunami.
Over 41,000 foreigners powered the construction industry as of last October, up from around 29,000 the previous year.
In November, there were more than eight times as many jobs to build steel construction frames as workers, separate government data showed.
"We have on-site managers through our company, but the people who actually do the work, that's where we lack skilled labor," said a manager at a major Japanese construction firm.
"That's where we have to find the people, and why we are trying to open gates to immigrants."
A Reuters investigation last year showed how asylum seekers banned from working were in fact working on public works projects amid a shortage of Japanese construction workers.
Workers from China made up over 30 percent of the foreign labor force, rising 6.9 percent from the previous year, according to the labor data.
Vietnamese workers were in second place, accounting for around 16 percent of foreign workers, but up over 50 percent compared to the previous year.
Although Japan accepted a record number of foreign workers, these include trainees and exchange students working part time.
The trainee system, aimed at training foreign workers so they can bring skills back to their home country, is often used by labor-strapped companies to secure workers. The program has been criticized by the United States for exploitation.
Nearly 20 percent of foreign workers were trainees as of last October, rising by over 25 percent from the previous year.
"It's not right that exchange students working part time and trainees who are supposed to be here to gain skills are counted as foreign labor," said Hidenori Sakanaka, former head of the Tokyo Immigration Bureau.
"The government may be trying to inflate the numbers and turn its eyes away from the fact Japan closes its doors to immigration. What we need to do is to work toward accepting immigration."