Michael Umana scored the shootout winner and keeper Keylor Navas made a brilliant penalty stop on Theofanis Gekas to claim the win.
Greece tied the match one minute into injury time, when Sokratis Papastathopoulos scored on a scramble in the Costa Rican box, sending the game to extra time.
Bryan Ruiz looked like he had the winner for Costa Rica when he scored in the 52nd minute to give his side a 1-0 lead.
Oscar Duarte got his second yellow card in the 66th minute and was sent off, leaving Costa Rica down a man and opening the door for a Greek comeback.
Costa Rica will play the Netherlands in the quarter-finals on Saturday.