Computer graphics can be used to represent animated virtual spaces in three dimensions. Marie-Paule Cani carries out her research in the Modeling Simulation and Learning division of École polytechnique’s Computer Science Laboratory (LIX). A renowned researcher who has won awards for her work, she explains: “Originally, LIX was mainly a fundamental computing laboratory. In other words, the researchers worked mainly on the mathematical and algorithmic foundations of our discipline. Then the laboratory diversified and now covers a wide range of themes, from bioinformatics to artificial intelligence. Since 2017, several computer graphics teams have developed, including the VISTA team, of which I am a member. On the one hand, we are working on new methods for aiding creation, based on knowledge and/or machine learning, as well as on methods for animating movements and deformations in the 3D virtual worlds thus created.”
The applications of computer graphics are many and varied. It helps to prototype and virtually test objects that are to be manufactured. In the special effects sector, it can be used to create spectacular scenes for cinema and animated films. Video games, another key area, use these technologies to plunge players into immersive worlds that are visually as close to reality as possible. Finally, 3D virtual worlds, which can be explored immersively, are essential for training simulators in high-risk situations, whatever the field (transport, medical, energy, military, etc.).
Computer graphics as a support for scientific thought and methodology
“For some years now, my main research project has been to explore the use of computer graphics as a medium for visual representation and experimentation for scientists in other disciplines.” When we think, we have visions, animated representations of what we imagine. The researcher illustrates her point with the example of a cell biology researcher. “Using their knowledge, they mentally visualise the process of cell division, for example during the growth of a tumour. However, the schematic representations that they can create with paper and pencil will only show this process at a fixed moment in time.” If they want to generate 3D animations, they will have to explain their vision to an artist who has mastered modelling software. “After a lot of to-ing and fro-ing, they will probably end up with an illustration that comes close to their vision. However, they will not be able to change the parameters or interact with the model represented.”
![](https://www.polytechnique-insights.com/wp-content/uploads/2024/06/adobestock_633442964-1024x574.jpeg)
“Our aim is to provide scientists with the means to create 3D and animated illustrations themselves, based on an “expressive” modelling methodology, the foundations of which I helped to lay. It is inspired by the way we humans understand the world around us and create objects in it.” The resulting interactive virtual worlds are very rich, and can serve as a support for scientific thought for researchers from many disciplines.
A three-stage methodology for representing complex models
Nature, with all its rich detail, is difficult to model. A moving head of hair, a waterfall or a wind-blown forest are all complex phenomena that are difficult to animate in real time. To meet the representation needs of scientists, the VISTA team has developed a three-stage methodology:
First, it uses multi-layer models, i.e. it breaks down the problem into different sub-models. “The sky is particularly difficult to animate,” explains the researcher. “So, to represent clouds quickly, we combined sub-models. First, we stacked 2D layers representing different cloud layers. Then we had to integrate the air flows. To do this, we combined the models with atmospheric dynamics – i.e. transfers between layers – modelled using fluid mechanics. Finally, to get as close to reality as possible, we classified the different types of clouds (stratus, sirrus, cumulus, etc.) and added procedural details to enhance visual realism. The result is an animated computer-generated image that represents the movement of clouds according to their nature and temperature. Finally, even very complex objects can be animated approximately in real time.”
The second ingredient in creating visual representations is expressive modelling. 3D modelling software is often difficult to learn. Marie-Paule Cani and her team want to make it possible to create directly through gesture. “The idea is to use these expressive gestures, which resemble drawing and sculpting gestures, to enable the scientist to create an animated 3D environment that corresponds to what they imagine.”
Finally, the methodology makes use of learning from examples, be they ad hoc processes, deep learning or reinforcement learning. These processes make it possible to create content (shapes or movements) that is indistinguishable from the results of simulations or examples provided by the user. Whenever possible, ad hoc processes are used, as they are less costly in terms of time and energy than deep learning – where the machine must ingest a large amount of data. “To create a landscape,” illustrates Marie-Paule Cani, “the user will place certain elements (pebbles, trees, tufts of grass, etc.) in a small area by hand, to illustrate the spatial relationships they want to see between these elements and with other factors, such as the slope of the land. Then the computer will statistically learn the distribution, i.e. the correlations between types of elements”. Just as when painting a picture, the user then has brushes with which to “statistically paint” the virtual world in real time.
![](https://www.polytechnique-insights.com/wp-content/uploads/2024/06/adobestock_496590817-1024x683.jpeg)
It is by combining these three parameters (expressive modelling, multi-layering and learning from examples) that the team can create and animate complex and varied virtual environments in 3D. In collaboration with researchers from other disciplines, they have been able to represent the formation of land by glacial erosion1, a Mediterranean or Alpine ecosystem2 or the formation of mountain ranges3.
The tools needed for expressive modelling
In order to create new forms, we need to be able to interact with the models, so that they respond in real time to the user’s gestures. In addition to the models and algorithms, Marie-Paule Cani and her team sometimes have to build specific tools to capture movements, such as an extended mouse with force sensors4. “We developed a hand navigator, a tool that connects a mouse with six degrees of freedom to small sensors activated by the user’s fingers. This tool was used to animate a virtual hand, capable of sculpting 3D modelling clay.” The user can model live on the computer, but in some cases, tactile interactions and virtual reality devices enhance intuitive interaction with models, facilitating creation and allowing users to manipulate complex environments in a natural way. She continues, “inventing these virtual reality tools has sometimes been necessary to fill a gap and advance research. Unlike generative AI (often developed to create for us), the intelligent systems I am interested in allow gestural interaction, and are intended to make us humans more creative.”
Models to test scientific hypotheses
The VISTA team regularly works with scientists from other disciplines. In collaboration with the latter, the researchers develop creation tools based on their models, which enable them to create examples directly. “From 2017 to 2021, for example, we worked with palaeontologists who want to “see” their palaeoclimatic model and the resulting distribution of fauna and flora in the Tautavel valley.” Located in the Pyrénées-Orientales region, the Tautavel valley is a major prehistoric site. Archaeological digs there have unearthed numerous bones, tools and traces dating back more than 300,000 years. “The aim was to reproduce the ecosystem based on the data and hypotheses of the researchers. We represented the palaeoclimate, and generated the flora and fauna that inhabited the valley at that time according to the parameters (temperature, hydration and sunshine) as well as the probable list of species present given by the palaeontologists.” Thanks to the animated 3D representation constructed in this way, researchers can see the flaws in their models and adjust their hypotheses.
Could computer graphics be the first step towards time travel? Without going that far, there is no doubt that this discipline transports us. Imagine taking a virtual walk through a 3D reconstruction of a valley in the Palaeolithic period – it’s a form of time travel. What is certain is that computer graphics can help scientists to improve their mental vision and their models, by combining knowledge and digital learning.
The constant evolution of computer graphics offers fascinating prospects for the future
But it also works the other way round. To test their model, Marie-Paule Cani’s team is reproducing actual landscapes. Comparing their results with the existing landscape provided by satellite images enables them to adjust their methodology if necessary to obtain the most accurate representation of reality possible, and then to validate their model.
The constant evolution of computer graphics offers fascinating prospects for the future. By enabling experts to visualise their mental representations and test their scientific hypotheses more quickly, it opens up new avenues for research and education.
One future project involves using expressive modelling to raise public awareness of climate change. By allowing users to manipulate hypotheses and visualise in 3D the effects of their choices on our planet, this approach would help to get the general public more involved in current environmental issues. In this way, computer graphics not only models reality, it also becomes a crucial tool for supporting scientific research and enabling everyone, students and the general public alike, to gain a better understanding of our world.