We used functional magnetic resonance imaging to explore the brain mechanisms of changing point of view (PoV) in a visuospatial memory task in 3D space. Eye movements were monitored and BOLD signal changes were measured while subjects were presented with 3D images of a virtual environment. Subjects were required to encode the position of a lamp in the environment and, after changing the PoV (angular difference varied from 0 degrees to 180 degrees in 45 degrees steps), to decide whether the lamp position had been changed too or not. Performance data and a scan-path analysis based on eye movement support the use of landmarks in the environment for coding lamp position and increasing spatial updating costs with increasing changes of PoV indicating allocentric coding strategies during all conditions (0 degrees - to 180 degrees -condition). Subtraction analysis using SPM revealed that a parieto-temporo-frontal network including left medial temporal areas was activated during this 3D visuospatial task, independent of angular difference. The activity of the left parahippocampal area and the left lingual gyrus (but not the hippocampus) correlated with increasing changes of the PoV between encoding and retrieval, emphasizing their specific role in spatial scene memory and allocentric coding. The results suggest that these areas are involved in a continuous matching process between internal representations of the environment and the external status quo. In addition, hippocampal activation correlated with performance was found indicating successful recall of spatial information. Finally, in a prefrontal area comprising, the so-called "deep" frontal eye field, activation was correlated with the amount of saccadic eye movements confirming its role in oculomotor processes.