Victoria Police say the man is facing several of charges following the Saturday night incident.
Police say the unidentified man braved frigid waters to swim across the Gorge waterway and break into a compound where the heavy equipment was being stored.
A security guard who spotted the man wading ashore had to leap out of the way as the suspect fired up the loader and trundled it out of the lot, crushing a parked motorcycle during the slow-speed getaway.
Police intercepted the vehicle within a few blocks but it took a little longer to round up the suspect.
A police dog eventually sniffed out the man and he was taken into custody after he leaped from the loader and fled on foot. (CFAX)