Vienna Mayor Michael Haeupl says first results show that 72 per cent oppose the city's ambition to host the games.
Haeupl says "apparently most people are not convinced that the city would benefit in the long term from investing in the games."
The deadline for postal voting is next Monday, though only marginal changes are expected to the final outcome of the referendum, in which more than 1.1 million residents were eligible to vote.
Vienna was a candidate for the 1964 Summer Olympics, which were awarded to Tokyo. Austria hosted the Winter Olympics that year, and again in 1976, in Innsbruck.