Museum metaverses promise to redefine the cultural experience

Pre­serv­ing cul­tur­al her­itage, rais­ing the pro­file of art col­lec­tions and mak­ing immer­sive muse­um expe­ri­ences acces­si­ble to as many peo­ple as pos­si­ble in a ded­i­cat­ed meta­verse… As the win­ner in 2023 of wave 3 of the PIA4 (Plan d’in­vestisse­ment d’avenir) call for projects ‘Digi­ti­sa­tion of her­itage and archi­tec­ture’, the Métavers du pat­ri­moine project is tack­ling these ambi­tious objec­tives. Led by Man­za­l­ab, the con­sor­tium also includes Ecole Poly­tech­nique, Télé­com Sud­Paris, Mus’X and Forum des Images. The idea is to digi­tise works of art and then insert them into col­lab­o­ra­tive and inter­ac­tive vir­tu­al worlds. Ulti­mate­ly, the aim is to offer muse­ums the pos­si­bil­i­ty and the right tools to digi­tise their arte­facts and rede­fine cul­tur­al experiences.

Europe, and France in par­tic­u­lar, has a rich and well-pre­served muse­um her­itage. Digi­tis­ing works of art ensures their preser­va­tion and makes them more acces­si­ble. This is what Titus Zaharia and Mar­ius Pre­da, pro­fes­sor and lec­tur­er respec­tive­ly at Télé­com Sud­Paris, are seek­ing to achieve under the umbrel­la of the “Métavers du pat­ri­moine” project. The aim is to cre­ate 3D rep­re­sen­ta­tions of objects so that they can be visu­alised in a vir­tu­al envi­ron­ment called a meta­verse. “The idea is to use digi­tised works of art and vir­tu­al worlds to sup­port an expe­ri­ence,” explains Mar­ius Preda.

The fusion of art and tech­nol­o­gy is open­ing up new prospects for muse­ums. While some fear a dema­te­ri­al­i­sa­tion of the artis­tic expe­ri­ence, accord­ing to the experts, it is above all an oppor­tu­ni­ty to broad­en the audi­ence and gen­er­ate new forms of engage­ment. Vir­tu­al worlds know no spa­tial bound­aries. In the meta­verse, for exam­ple, an exhi­bi­tion that is usu­al­ly restrict­ed by the sur­face area allo­cat­ed to it could tran­scend the phys­i­cal bar­ri­ers of the real world. The cre­ation of inno­v­a­tive, immer­sive expe­ri­ences could also arouse new inter­est among vis­i­tors. As Mar­ius Pre­da points out, “I don’t get the impres­sion that the younger gen­er­a­tion is keen to vis­it muse­ums in the tra­di­tion­al way. I see this as an oppor­tu­ni­ty to reach them, as they are already very used to con­sum­ing dig­i­tal con­tent.” Last­ly, the muse­um meta­verse project pro­motes acces­si­bil­i­ty. With­out the need to trav­el, this new way of expe­ri­enc­ing exhi­bi­tions could become a major edu­ca­tion­al and cul­tur­al lever.

Pho­togram­me­try is the cor­ner­stone of this process. Syn­ony­mous with mea­sure­ment by pho­tog­ra­phy, this tech­nique is based on a very old prin­ci­ple: the cal­cu­la­tion of dis­tances by tri­an­gu­la­tion. This method, which enables large dis­tances to be mea­sured accu­rate­ly, was used to map ter­ri­to­ries in the 17th cen­tu­ry¹. By imag­in­ing a tri­an­gle on a piece of land, we can deter­mine the length of the oth­er two sides from the angles. Once the length of the first tri­an­gle is known, anoth­er tri­an­gle with a side in com­mon with the first can be cal­cu­lat­ed. To digi­tise objects, sci­en­tists use a sim­i­lar process: sev­er­al cam­eras take images of the object from dif­fer­ent angles. Using the 2D images of the work, the sci­en­tists are able to cal­cu­late posi­tions in three dimen­sions. They find com­mon points between the images cap­tured and cal­cu­late the 3D posi­tion of each pix­el. They then put these mea­sure­ments togeth­er to recon­struct the three-dimen­sion­al object.

Cap­tur­ing the com­plex­i­ty of works of art while respect­ing their artis­tic essence is a chal­lenge. First­ly, this tech­nique can only rep­re­sent the sur­face of the object, not what it con­tains. What’s more, the diver­si­ty of art forms requires con­stant adap­ta­tion, with each cre­ation requir­ing a spe­cif­ic approach. The mate­r­i­al of the work is one of the main obsta­cles. “If it is capa­ble of reflect­ing light, then it can be digi­tised”, says Mar­ius Pre­da. This being the case, translu­cent or trans­par­ent objects can­not be digi­tised using this tech­nique. Anoth­er fac­tor is the size of the object. The small­er the object, the eas­i­er it is to cap­ture. When the object is too large, sci­en­tists use drones to cap­ture the images. This method is used, for exam­ple, to digi­tise archi­tec­tur­al works.

The digi­ti­sa­tion of works of art opens up new artis­tic pos­si­bil­i­ties. “Once digi­tised, there are an infi­nite num­ber of ways of rep­re­sent­ing the object. You can play with mul­ti­ple para­me­ters such as trans­paren­cy, colour or even shape,” explains the lec­tur­er. But this vast field of pos­si­bil­i­ties rais­es eth­i­cal ques­tions, par­tic­u­lar­ly when it comes to the mod­i­fi­ca­tion or rep­re­sen­ta­tion of ancient cul­tur­al objects that could offend cer­tain communities.

“You have to bear in mind that hav­ing works digi­tised in 3D is not enough to make a meta­verse, there are oth­er aspects that need to be tak­en into account,” insists Titus Zaharia. In the end, although digi­tis­ing works of art involves a num­ber of high­ly tech­ni­cal process­es, there are no major tech­no­log­i­cal bar­ri­ers to gam­i­fi­ca­tion, cap­tur­ing, mak­ing avail­able and cre­at­ing expe­ri­ences. The cur­rent chal­lenge is above all to dis­cov­er the right uses. Titus Zaharia asks, “At what point does a user think they are vis­it­ing a vir­tu­al real­i­ty museum?” 

Accord­ing to the researchers, these vir­tu­al muse­ums will be col­lec­tive expe­ri­ences. Mar­ius Pre­da adds: “I don’t believe in expe­ri­enc­ing things alone at home. I believe more in online group vis­its, like play­ing video games.” To achieve this, tech­nolo­gies need to be democ­ra­tised and devel­oped to ensure that they are user-friend­ly. For exam­ple, cur­rent vir­tu­al real­i­ty head­sets are heavy, uncom­fort­able and expen­sive. With­out “good” devices for access­ing metavers, the user expe­ri­ence will be compromised.

So, while tech­nol­o­gy opens new doors, inno­va­tion is noth­ing with­out ‘uses’. This French project is being run with and for muse­ums. Many muse­ums and cul­tur­al insti­tu­tions are already rep­re­sent­ed, either as part­ners in the project, such as Mus’X and the Forum des Images, or as mem­bers of the steer­ing com­mit­tee, such as the Château de Ver­sailles, the Musée du Quai Bran­ly, the Musée des armées, the Bib­lio­thèque nationale de France, the Cité des sci­ences et de l’in­dus­trie, the Musée d’archéolo­gie nationale, the Musée de la Marine, the Palais des beaux-arts de Lille and the Château de Cham­bord. Ulti­mate­ly, the aim is to enable muse­um staff to car­ry out 3D record­ing inde­pen­dent­ly, using a plat­form that the researchers are devel­op­ing. This plat­form will have to under­stand the needs of the insti­tu­tions and adapt to their busi­ness mod­els. If these cul­tur­al meta­verse projects are to be democ­ra­tised, indi­vid­ual muse­ums will need to be able to cre­ate their own con­tent, with a high degree of cus­tomi­sa­tion and script­ing of the user experience.

Loraine Odot