2_metavers
π Science and technology
Science at the service of creativity

Museum metaverses promise to redefine the cultural experience

Titus Zaharia, Professor at Télécom SudParis (IP Paris) and Marius Preda, Associate Professor at Institut MINES-Télécom
On June 4th, 2024 |
4 min reading time
Titus ZAHARIA
Titus Zaharia
Professor at Télécom SudParis (IP Paris)
Marius Preda
Marius Preda
Associate Professor at Institut MINES-Télécom
Key takeaways
  • Digitising works of art helps to preserve cultural heritage, make art collections accessible and reinvent museum experiences.
  • As part of the ‘Métavers du patrimoine’ investment plan, experts are using photogrammetry to create 3D representations of works of art.
  • This project, which is being carried out with and for museums, takes into account the needs of institutions and adapts to their business model.
  • 3D digitised works of art are not enough to create a metaverse; the challenge now is above all to discover the right uses for them.
  • The success of virtual museums will depend in particular on the democratisation of technologies such as VR headsets and the adaptation of platforms to the specific needs of museum institutions.

Pre­serv­ing cul­tur­al her­itage, rais­ing the pro­file of art col­lec­tions and mak­ing immer­sive muse­um expe­ri­ences acces­si­ble to as many peo­ple as pos­si­ble in a ded­i­cat­ed meta­verse… As the win­ner in 2023 of wave 3 of the PIA4 (Plan d’in­vestisse­ment d’avenir) call for projects ‘Digi­ti­sa­tion of her­itage and archi­tec­ture’, the Métavers du pat­ri­moine project is tack­ling these ambi­tious objec­tives. Led by Man­za­l­ab, the con­sor­tium also includes Ecole Poly­tech­nique, Télé­com Sud­Paris, Mus’X and Forum des Images. The idea is to digi­tise works of art and then insert them into col­lab­o­ra­tive and inter­ac­tive vir­tu­al worlds. Ulti­mate­ly, the aim is to offer muse­ums the pos­si­bil­i­ty and the right tools to digi­tise their arte­facts and rede­fine cul­tur­al experiences.

Accessibility, conservation and the reinvention of cultural experiences

Europe, and France in par­tic­u­lar, has a rich and well-pre­served muse­um her­itage. Digi­tis­ing works of art ensures their preser­va­tion and makes them more acces­si­ble. This is what Titus Zaharia and Mar­ius Pre­da, pro­fes­sor and lec­tur­er respec­tive­ly at Télé­com Sud­Paris, are seek­ing to achieve under the umbrel­la of the “Métavers du pat­ri­moine” project. The aim is to cre­ate 3D rep­re­sen­ta­tions of objects so that they can be visu­alised in a vir­tu­al envi­ron­ment called a meta­verse. “The idea is to use digi­tised works of art and vir­tu­al worlds to sup­port an expe­ri­ence,” explains Mar­ius Preda.

The fusion of art and tech­nol­o­gy is open­ing up new prospects for muse­ums. While some fear a dema­te­ri­al­i­sa­tion of the artis­tic expe­ri­ence, accord­ing to the experts, it is above all an oppor­tu­ni­ty to broad­en the audi­ence and gen­er­ate new forms of engage­ment. Vir­tu­al worlds know no spa­tial bound­aries. In the meta­verse, for exam­ple, an exhi­bi­tion that is usu­al­ly restrict­ed by the sur­face area allo­cat­ed to it could tran­scend the phys­i­cal bar­ri­ers of the real world. The cre­ation of inno­v­a­tive, immer­sive expe­ri­ences could also arouse new inter­est among vis­i­tors. As Mar­ius Pre­da points out, “I don’t get the impres­sion that the younger gen­er­a­tion is keen to vis­it muse­ums in the tra­di­tion­al way. I see this as an oppor­tu­ni­ty to reach them, as they are already very used to con­sum­ing dig­i­tal con­tent.” Last­ly, the muse­um meta­verse project pro­motes acces­si­bil­i­ty. With­out the need to trav­el, this new way of expe­ri­enc­ing exhi­bi­tions could become a major edu­ca­tion­al and cul­tur­al lever.

Digitising with photogrammetry

Pho­togram­me­try is the cor­ner­stone of this process. Syn­ony­mous with mea­sure­ment by pho­tog­ra­phy, this tech­nique is based on a very old prin­ci­ple: the cal­cu­la­tion of dis­tances by tri­an­gu­la­tion. This method, which enables large dis­tances to be mea­sured accu­rate­ly, was used to map ter­ri­to­ries in the 17th cen­tu­ry1. By imag­in­ing a tri­an­gle on a piece of land, we can deter­mine the length of the oth­er two sides from the angles. Once the length of the first tri­an­gle is known, anoth­er tri­an­gle with a side in com­mon with the first can be cal­cu­lat­ed. To digi­tise objects, sci­en­tists use a sim­i­lar process: sev­er­al cam­eras take images of the object from dif­fer­ent angles. Using the 2D images of the work, the sci­en­tists are able to cal­cu­late posi­tions in three dimen­sions. They find com­mon points between the images cap­tured and cal­cu­late the 3D posi­tion of each pix­el. They then put these mea­sure­ments togeth­er to recon­struct the three-dimen­sion­al object.

O1 and O2 are the obser­va­tion points (posi­tions and ori­en­ta­tions of the cam­eras), p and p’ are two pix­els in the two images cor­re­spond­ing to the same infor­ma­tion, P is the cal­cu­lat­ed 3D point2

Cap­tur­ing the com­plex­i­ty of works of art while respect­ing their artis­tic essence is a chal­lenge. First­ly, this tech­nique can only rep­re­sent the sur­face of the object, not what it con­tains. What’s more, the diver­si­ty of art forms requires con­stant adap­ta­tion, with each cre­ation requir­ing a spe­cif­ic approach. The mate­r­i­al of the work is one of the main obsta­cles. “If it is capa­ble of reflect­ing light, then it can be digi­tised”, says Mar­ius Pre­da. This being the case, translu­cent or trans­par­ent objects can­not be digi­tised using this tech­nique. Anoth­er fac­tor is the size of the object. The small­er the object, the eas­i­er it is to cap­ture. When the object is too large, sci­en­tists use drones to cap­ture the images. This method is used, for exam­ple, to digi­tise archi­tec­tur­al works.

The question of use remains an issue

The digi­ti­sa­tion of works of art opens up new artis­tic pos­si­bil­i­ties. “Once digi­tised, there are an infi­nite num­ber of ways of rep­re­sent­ing the object. You can play with mul­ti­ple para­me­ters such as trans­paren­cy, colour or even shape,” explains the lec­tur­er. But this vast field of pos­si­bil­i­ties rais­es eth­i­cal ques­tions, par­tic­u­lar­ly when it comes to the mod­i­fi­ca­tion or rep­re­sen­ta­tion of ancient cul­tur­al objects that could offend cer­tain communities.

“You have to bear in mind that hav­ing works digi­tised in 3D is not enough to make a meta­verse, there are oth­er aspects that need to be tak­en into account,” insists Titus Zaharia. In the end, although digi­tis­ing works of art involves a num­ber of high­ly tech­ni­cal process­es, there are no major tech­no­log­i­cal bar­ri­ers to gam­i­fi­ca­tion, cap­tur­ing, mak­ing avail­able and cre­at­ing expe­ri­ences. The cur­rent chal­lenge is above all to dis­cov­er the right uses. Titus Zaharia asks, “At what point does a user think they are vis­it­ing a vir­tu­al real­i­ty museum?” 

Accord­ing to the researchers, these vir­tu­al muse­ums will be col­lec­tive expe­ri­ences. Mar­ius Pre­da adds: “I don’t believe in expe­ri­enc­ing things alone at home. I believe more in online group vis­its, like play­ing video games.” To achieve this, tech­nolo­gies need to be democ­ra­tised and devel­oped to ensure that they are user-friend­ly. For exam­ple, cur­rent vir­tu­al real­i­ty head­sets are heavy, uncom­fort­able and expen­sive. With­out “good” devices for access­ing metavers, the user expe­ri­ence will be compromised.

So, while tech­nol­o­gy opens new doors, inno­va­tion is noth­ing with­out ‘uses’. This French project is being run with and for muse­ums. Many muse­ums and cul­tur­al insti­tu­tions are already rep­re­sent­ed, either as part­ners in the project, such as Mus’X and the Forum des Images, or as mem­bers of the steer­ing com­mit­tee, such as the Château de Ver­sailles, the Musée du Quai Bran­ly, the Musée des armées, the Bib­lio­thèque nationale de France, the Cité des sci­ences et de l’in­dus­trie, the Musée d’archéolo­gie nationale, the Musée de la Marine, the Palais des beaux-arts de Lille and the Château de Cham­bord. Ulti­mate­ly, the aim is to enable muse­um staff to car­ry out 3D record­ing inde­pen­dent­ly, using a plat­form that the researchers are devel­op­ing. This plat­form will have to under­stand the needs of the insti­tu­tions and adapt to their busi­ness mod­els. If these cul­tur­al meta­verse projects are to be democ­ra­tised, indi­vid­ual muse­ums will need to be able to cre­ate their own con­tent, with a high degree of cus­tomi­sa­tion and script­ing of the user experience.

Loraine Odot
1https://​bib​num​.obspm​.fr/​e​x​h​i​b​i​t​s​/​s​h​o​w​/​p​i​c​a​r​d​/​p​i​c​a​r​d​-​g​e​o​c​a​r​t​o​/​p​i​c​a​r​d​-​g​e​o​c​a​r​t​o​-​t​r​i​a​n​g​u​l​ation
2https://​web​.stan​ford​.edu/​c​l​a​s​s​/​c​s​2​3​1​a​/​c​o​u​r​s​e​_​n​o​t​e​s​/​0​4​-​s​t​e​r​e​o​-​s​y​s​t​e​m​s.pdf

Our world explained with science. Every week, in your inbox.

Get the newsletter