The goal of this project in the third phase of the SFB/TR8 is the construction of three-dimensional environment models from the observation of human motions. We record the corresponding activity data with a data suit worn by a user. Our aim is to identify different types of objects from activities and gestures, e.g., the presence of walls, doors, staircases, tables, and chairs and to successively add the recognized objects to the virtual environment. In this context, we investigate machine learning techniques to recognize activities, as well as efficient data structures allowing for updates and modifications of the virtual environment. To maintain a consistent model of the environment, we need to keep track of the human within the environment and be able to recognize interactions with objects that are already contained in the virtual environment. We furthermore investigate possibilities for user interaction with the virtual environment, concerning for instance modifications or corrections in the placement of virtual objects.