The match appeared headed for a scoreless draw when Cristiane sent a high shot that just barely made it into the net as New Zealand defenders scrambled to stop it.
The win leaves Brazil with six points from two matches in Group E. New Zealand has no points after its second consecutive 1-0 defeat. It had lost to host Britain in the opener.
Britain and Cameroon will play the group's other match later in the day in Cardiff.
The top two teams from each of the three groups automatically advance, along with the best two third-place finishers.
Brazil is trying to win its first major trophy in women's football. It lost the last two Olympic finals to the United States and the 2007 World Cup final to Germany. Last year it was eliminated from the World Cup quarterfinals after allowing a last-minute equalizer to the U.S. in extra time, eventually losing in a penalty shootout.
Brazil controlled possession but failed to create many significant scoring opportunities in the match at the Millennium Stadium in Cardiff, Wales. Marta had a few dangerous runs but was not able to get much going. New Zealand relied mostly on set pieces to get near Brazil's goal and threatened only a few times.
"We knew it was going to be a very difficult match from the beginning, New Zealand played well against Britain and lost on a set piece," Brazil coach Jorge Barcellos said. "Today we did just enough to get the three points. We know we need to improve and we will do it match by match."
Cristiane got the winner after a free kick cross into the New Zealand penalty area. The defenders were not able to fully clear the cross and the ball fell in front of the Brazilian forward, who used the side of her foot to lobby it toward the goal.
"I got hit when I took the shot so I didn't even see the ball go in," Cristiane said.
New Zealand defender Abby Erceg crashed into the net trying to stop the ball from going in but didn't get to it in time.
"It was a ball we should've dealt with easily as we had been doing during 85 minutes," New Zealand coach Tony Readings said.
Cristiane became the women's top Olympic scorer with 11 goals when she scored in Brazil's 5-0 routing of Cameroon in the opener on Wednesday.
She had a few chances to score with long range shots in the first half, but Brazil's best opportunity until the goal was a 43rd-minute shot by veteran midfielder Formiga, just missing wide of the net.
Just minutes before Brazil's got on the board, striker Hannah Wilkinson almost gave New Zealand the surprise victory but her shot from the top of the area was saved by Brazilian goalkeeper Andreia in the 79th.
"We are devastated we didn't get something out of it," Readings said. "We are disappointed because the girls really executed well."
Brazil had an early scare when Marta went down just a minute into the match after being stepped on by a defender in a ball dispute. Doctors had to come in to treat her but she eventually recovered and was able to stay in the match.
Follow Tales Azzoni at http://twitter.com/tazzoni