Coe says the cauldron "wasn't created as a tourist attraction."
The flame was lit at the opening ceremony Friday night in the middle of the Olympic Stadium. It is being moved to a corner of the stadium for the rest of the games, invisible from the outside.
The cauldron will only be visible to fans attending the track and field competition, which starts Friday.
Some spectators in the Olympic Park have complained they can't see the flame or take pictures of it.
Coe says organizers are replicating the 1948 London Olympics, when the flame was kept inside Wembley Stadium.