Under rules introduced in 2004, transgender people can only change their gender on legal documents if they appeal to a family court and meet criteria including gender confirmation surgery and having no reproductive capacity -- generally requiring sterilization.
Other rules include being unmarried, not having children and being officially diagnosed with gender dysphoria.
Only a handful of countries around the world allow transgender people to change their status with a simple declaration.
But campaigners say that conditions imposed by Japan, the only Group of 7 wealthy country that does not recognize same-sex marriage or civil unions at a national level, force people into highly invasive, lengthy and potentially risky medical procedures.
"The procedure for changing one's legally recognized gender, which requires sterilization surgery and an outdated psychiatric diagnosis, is anachronistic, harmful, and discriminatory," Human Rights Watch said in a 2019 report.
The advocacy group blasted it as rooted on a "pejorative" notion that transgender identity amounts to a "mental illness".
Pressure though is building, with a regional family court ruling this month that the sterilization criteria was "unconstitutional and invalid".
Recent years have seen traditionally conservative Japan take small steps towards embracing sexual diversity.
Opinion polls have shown growing support for LGBTQ-friendly laws, especially among younger people, but the ruling Liberal Democratic Party has been reluctant to push ahead with reforms.
And in recent months activists have warned of a rising tide of hateful online rhetoric, particularly targeting transgender women, fuelled in part by contentious debate around anti-discrimination legislation that passed this year, albeit in watered-down form.
'Still in pain'
At the heart of Wednesday's case is a transgender Japanese woman seeking to be legally listed as female without surgery.
She has called the obligation to be sterilized a "grave human rights violation and unconstitutional".
Her past petitions to a family court, and then a higher court, were both rejected.
This is the second time the Supreme Court has weighed in on the constitutional question of the sterilization requirement, having ruled it "constitutional as of now" in 2019.
Judges had defended the requirement as meant to preclude "confusion" that can arise from transgender people having children after changing their legal gender.
Wednesday's top court ruling has been hotly anticipated among Japan's transgender community, including 34-year-old transgender man Tomoya Asanuma.
At 23, Asanuma had his womb and ovaries taken out in order to legally change gender and be allowed to marry his female fiance at the time.
"Making surgery a condition for changing legal gender is too risky for trans people in all aspects -- physically, emotionally and financially," Asanuma said.
"I am still in pain. It's really frustrating to think I wouldn't be suffering as much as I am now, if only I had been allowed to change my gender without surgery."