That will complicate Prime Minister Shinzo Abe's efforts to ease the country's tightest job market in decades.
And the bulk of the firms that may hire these immigrants do not plan to support them in securing housing, learning Japanese language skills or getting information on living in Japan, the Reuters Corporate Survey showed.
The survey results underscore the challenge for Japan to cope with its dwindling and ageing population that has put pressure on the government to relax tight foreign labour controls. Immigration has long been taboo here as many Japanese prize ethnic homogeneity.
The lack of language ability, cultural gap, costs of training, mismatches in skills and the fact that many foreign workers cannot stay permanently in Japan under the new system were among factors behind corporate wariness about hiring foreign workers, the Reuters poll showed.
The law, which took effect in April, creates two new categories of visas for blue-collar workers in 14 sectors such as construction and nursing care, which face a labour crunch. It is meant to attract up to 345,000 blue-collar workers to Japan over five years.
But the survey suggests the government may struggle to get the workers it needs to ease the country's labor shortage where there are now 1.63 jobs available for every job seeker, the most since the beginning of 1974.
"Taking education costs, quality risks and yields into account, costs will go up" by hiring foreign workers, wrote a manager at a rubber-making company, who said the firm has no plans to hire foreign workers.
"We have failed in the past by employing foreign workers who could not blend in with a different culture," a manager of a metal-products maker wrote.
Some 41 percent of firms are not considering hiring foreigners at all, 34 percent are not planning to hire many and 26 percent intend to hire such foreign workers, the survey conducted from May 8-17 showed.
Of those considering hiring foreign workers, a majority said they have no plans to support them in areas such as housing, Japanese language study and information on living in the country, it showed.
The survey, conducted monthly for Reuters by Nikkei Research, polled 477 large- and mid-size firms, with managers responding on condition of anonymity. Around 220 answered the questions on foreign workers.
Under the new law, a category of "specified skilled workers" can stay for up to five years but cannot bring family members. The other category is for more skilled foreigners who can bring relatives and be eligible to stay longer.
While foreign workers are generally viewed as cheap labour in Japan, 77 percent of firms see no change in wage levels at Japan Inc as a whole, when hiring specified skilled workers. Some 16 percent expect wages to decline and just 6 percent see wages rising.
Foreign workers "will help ease the labour crunch, bringing down overall wages," a steelmaker manager wrote in the survey.
Abe, whose conservative base fears a rise in crime and a threat to the country's social fabric, has insisted that the new law does not constitute an "immigration policy."
Japan has about 1.28 million foreign workers - more than double the figure a decade ago but still just 2 percent of the workforce. Some 260,000 of them are trainees from countries such as Vietnam and China who can stay three to five years.