Philippines coach Alen Stajcic said on the eve of the New Zealand match that his side wanted to "crash the party", and they did just that with a gutsy 1-0 victory in front of 32,357 fans in Wellington.
Sarina Bolden scored the only goal with a header in the first half for a famous Philippine win - the country's first at either a men's or women's World Cup. It also blows Group A wide open.
The Philippines, who lost 2-0 to Switzerland on their World Cup debut, face Norway on Sunday in the final round of group games. New Zealand clash with Switzerland on the same day with everything to play for.
Norway face Switzerland later Tuesday.
"I can't believe what we have achieved," said tearful Philippines midfielder Sara Eggesvik.
"It shows that it's possible to do something big."
Bolden's goal was a scrappy one, but they won't care.
After New Zealand conceded a free-kick, Bolden out-jumped the home defence to head the ball at home goalkeeper Victoria Esson, who could only watch in horror as her attempted parry looped over her into the net.
New Zealand pushed hard for an equaliser in the second half but failed to find their way past the resolute Philippine defense.
"It's so heartbreaking for everyone in this team," said New Zealand's Czech coach Jitka Klimkova, whose side beat former champions Norway 1-0 in the opening game of the tournament for a first-ever World Cup win of their own.
"We were fighting until the end, but it wasn't enough."
'Another planet'
Colombia were too good for a blunt South Korea with a 2-0 victory in the first game of the day.
Eighteen-year-old striker Caicedo, who overcame cancer three years ago, was the beneficiary of a glaring goalkeeping error when Yoon Young-geul allowed the striker's shot to squirm through her hands.
Catalina Usme had scored a penalty to put the South Americans in front on the half-hour in bright sunshine in Sydney before Caicedo got the second nine minutes later.
South Korea coach Colin Bell brought on U.S.-born attacker Casey Phair as they chased the game in the second half. At 16 years and 26 days, she made history as the youngest Women's World Cup player ever.
But it was all about Caicedo and one of the best young attackers in women's football has now scored at the Under-17 World Cup, the Under-20 World Cup and on the biggest stage of all.
Assistant Angelo Marsiglia, standing in for Colombia coach Nelson Abadia - who was suspended for the game - called Caicedo "extraordinary."
"She wants the ball, never hides, she's from another planet, entirely unique," he said.
Colombia move onto three points in Group H with Germany, who thrashed Morocco 6-0 on Monday. Germany and Colombia meet next.
'Up for the challenge'
Spain, one of the favorites to topple holders the United States, headline Wednesday's action in Australia and New Zealand.
The Spaniards were easy winners 3-0 over Costa Rica in their opener and will be looking to rack the goals up once more against Zambia.
The African side are the lowest-ranked team in the competition and were thrashed 5-0 by Japan in their opening match, but captain Barbra Banda said they were "up for the challenge" against La Roja.
"We started with a loss but that doesn't mean that we have to look down on ourselves," she said.
Also on Wednesday, the 2011 champions Japan play Costa Rica in the same group.
Canada clash with Ireland in the last game of the day.