E-MUSE Blog

Blog

  • Tip: use Tree to generate HTML directory listings

    Part of our job in the BCC Arte & Cultura project is to provide the “high fidelity” visualizations for each artwork. For some that means one of our 3D scans, for others it’s simply a zoomable interface. Our clients are making a “standard” website and simply linking to our server for the high fidelity version. We are not done processing all the artworks, but we can provide the URL in advance, they are all ordered by a bank/author-title schema. I wrote a script for creating the folder structure, but then I wanted to provide a single page listing all the URLs to our clients.

    With a bit of research I found out a oneliner using the tree command.

    You can try the result on https://e-muse.it/bccaec

    “opere” is just the relative path between the location where I’m running the command and the directory structure I wanted to transverse.

    tree opere -d -H "https://e-muse.it/bccaec" > index.html

  • Expedition report: BCC Arte & Cultura

    The first half of this year was taken up by two large projects for BCC ICCREA. We’re doing the immersive components of the BCC Innovation Festival (a startup festival, we were among the winners of the first edition) and of BCC Arte & Cultura (an effort to catalogue the art and cultural heritage artifacts belonging to the banks of the group). For the latter, we went around Italy digitizing the most complex and 3D objects.

    All the points we visited

    Logistics

    At this point it’s not really clear what we’re allowed to publish, as legal departments are surprisingly slow in responding to our queries, so we are going to err on the side of caution. You’ll have to wait until the official announcement before we post the actual works that we’ve digitised, but there shouldn’t be any harm in discussing how we set up the logistics.

    Sadly, these locations were largely connected too poorly to attempt to do the whole trip using public transport. We really wanted to be ecological, but we didn’t find a way to do it in the given time and cost constraints without heavily relying on cars.

    Once we could confirm the availability of the works on a given day we found cheap accommodations on Booking, loaded all the equipment and all the team members in a car, and we generally tried to avoid traveling during working hours. Northern Italy was covered through a series of short trips, while central and southern Italy were covered sequentially, going roughly clockwise. Some extra trips had to be broken out of the main loop, we managed to make that convenient by relying on some really good friends, who very graciously hosted us.

    The whole thing cost us a bit under €2000, which is not too shabby for a 3-person team. This figure includes highways, fuel, accommodation and food, but it does not include the wear and tear on my car. Or the fact that I forgot some lenses at a friend’s place.

    What we digitized

    Again, we’re not supposed to share images of the works, as many of them are copyright encumbered, so stay tuned for the official launch of the project. I don’t think there’s any arm in sharing a list of what we did, though:

    • Alla fine del Giorno, Alberto Sughi, oil on canvas
    • Natività, Ilario Fioravanti, painted ceramics
    • Presepe, Capuano Brothers, multiple materials
    • Ducal Palace and several artifacts, in Mafalda
    • Ut Unum Sint, Arnaldo Pomodoro, outdoor metal stele
    • Bank building, Ugo Pagliara, building and architectural drawing
    • Carrù Castle, building
    • Chandelier made of Murano glass, 3 paintings, near Venice
    • Acquaviva Picena, whole building (we ended up scanning the whole town)
    • Incontri al Maneggio, Silvano Spessot, iron and colored glass sculpture
    • Untitled, Nane Zavagno, steel sculpture
    • Pinocchio, Venturino Venturini, bronze sculpture
    • Sant’Antonio Abate, Luca della Robbia, painted ceramics
    • Bank builiding, part of the Gradara castle-town
    • Bank building in Alba (no drones for this one, we had to climb literal towers!)
    • Assalto all’Olimpo, Bruno Liberatore, bronze sculpture (a much larger version is in Rome)
    • Untitled, Cesare Berlingeri, bent and stacked colored paper
    • An archeological site in Rome, featuring Imperial-age fresco-ed walls and a very well preserved Roman road

  • Link post: Digital Dark Age

    Just sharing this MIT Technology Review article on digital archives, a cause that’s close to our hearts: https://www.technologyreview.com/2024/08/19/1096284/data-archives-archeologists-tiktok-future-wayback-machine/?src=longreads

    It is always a good occasion to mention: donate to the Internet Archive, it is such an important resource, and it’s constantly under attack. Donating to them was the first thing I did when I got my first wage (rapidly followed by Wikipedia).

  • The Open Access saga goes on

    Read the latest developments on the rules regarding Open Access and cultural heritage in the Italian legislative system: https://www.wikimedia.it/news/il-cammino-verso-lopen-access-in-italia/

    Our own position is that eventually openness will win out, simply because the state is trying to charge for something that people are not willing to pay for, and the state has very little enforcement power over the open web (they should have even less).

  • XR News for 2024-04-23

    People expected big changes after the release of the Apple Vision Pro, and some are visibly disappointed. Apple tends to have a big effect on the mainstream, lots of people seem to think that they invented the smartphone, when the iPhone came out I was using my third smartphone.

    XR technology has been commercially available for 30 years, it tends to progress in the same way that people go bankrupt: gradually, then suddenly.

    We’re seeing some pretty interesting gradual changes, particularly in terms of software platforms. The Apple partnership seems to really have pushed Unity into high gear, they’re releasing a lot of resources, both on the editor side and on the design side. Sadly some of the best stuff is reserved for paying customers, and it’s pretty expensive.

    This seems to also have changed the thinking on the Sony (with the opening up of PSVR2 to PC users) and Meta. In just a few days Meta announced a radical opening up of their efforts.

    First of all they announced a new educational project, which will probably increase the demand for the kind of work we do quite a bit. The fundamental problem is that they keep using the “metaverse” word, and quite a few people have a very negative reaction to it. The concept is really not that clear, but the educational uses of VR are real, hopefully if there is an educational sales push it will also benefit us.

    The most important announcement is the opening up of their operating system. It will be possible to buy compatible headsets from other manufacturers, which should keep prices affordable without selling hardware at a loss, and even more importantly they’re opening up the app store. Multiple stores will be available, which could several important advantages. We could have a cross-platform cultural heritage app store which encourages reuse. Or more prosaically, maybe I could use some of my SteamVR purchases.

    Zuckerberg is explicitly aiming at providing the Windows of the XR market. At the same time, these changes should make them entirely Digital Markets Act compliant.

  • Corrections to the cultural heritage images licensing regime

    Last year a mandatory fee regime was introduced for images of publicly-owned cultural heritage artifacts. This is morally wrong, but it is also impractical.

    A new regulation came out in March modifying the rules and introducing exemptions. Wikimedia Italia wrote about it on their Arkivia blog.

    Nuove regole sulle riproduzioni dei beni culturali: saremo più libere e liberi di usarle?

    Academic works and news are now exempted from the fees, and so are catalogues printed in less than 4000 copies.

    The bureaucratic abomination under-girding the system, however, remains in place.

  • AI News: Music generation, Energy consumption, learn the basics

    Music generation

    There are several use-cases for algorithmically generating music, and indeed there is a long history of both hardware and software in this field. Back in the 80s there was even a significant professional backlash against MIDI and drum machines.

    For example, it would be quite handy to generate anechoic sounds for acoustics listening tests, so that they can be convolved with the Impulse Responses of simulated environments.

    The current mania is randomly generated media, which is an offshoot of a longer-standing mania for using media as filler instead of as a signifier. A lot of what we see and read is just Lorem Ipsum: posts need an image to do well, so people just slap a random image in them. It’s algorithmic garbage causing other algorithmic garbage. One of our deepest worries regarding our own work is that we could be hired to do it just because immersive technology sounds cool, and not because it genuinely adds value to the project at hand.

    Anyway, of course there are going to be attempts at using the newest technology for generating music, and here are three of them:

    1. Suno AI: this one definitely got the most attention
    2. Udio
    3. Stable Audio 2.0

    There is also an OpenAI product, but it’s not generally available.

    The increased repetitiveness in commercial music is definitely not an hindrance for these efforts.

    Fundamentally, all of the do what they say they will do, but we do wonder why they focus on directly generating sounds instead of generating MIDI.

    Energy consumption

    Check out this Ars Technica syndacation of a Financial Times article: https://arstechnica.com/ai/2024/04/power-hungry-ai-is-putting-the-hurt-on-global-electricity-supply/

    This is fundamentally an effect of computing getting increasingly centralised in massive data centers (just think of government over-reliance on Microsoft), which has noticeable economies of scale.

    With AI there is also a significant divergence between the cost of executing the software, and the cost of training the software. The current AI boom is largely focused on just throwing resources at relatively “dumb” approaches, which is working better than it could be reasonably be expected for now, but that will inevitably run into resource limitations. For now, that bottleneck is energy, which is very bad news in environmental terms.

    The silver lining is that if we managed to make companies pay for their environmental impact we would make smarter, more correct, approaches even more competitive.

    Learn the basics

    The always excellent 3Blue1Brown has released two new videos in his Deep Learning series, explaining Transformers. Here is a link to the whole series of videos: https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi&index=1

  • BCC Innovation Festival news coverage

    BCC Innovation Festival news coverage

    We’re working on the immersive component of the BCC Innovation Festival: a startup event that we won back in 2022.

    There are still around two weeks to participate, and we’re aggregating a little bit of media coverage. Mostly it’s because we like seeing our own work (the 3D model of the low-poly Rodin statue) featured so publicly.

    The Gazzetta di Napoli newspaper wrote about it on the 6th of March: https://www.gazzettadinapoli.it/economia-2/industria-4-0/al-via-la-terza-ediz

    ione-del-bcc-innovation-festival/

    Emilia-Romagna Startup, the regional government’s startup promotion agency, wrote about the festival today: https://www.emiliaromagnastartup.it/it/innovative/articoli/2024/04/bcc-innovation-festival-2024-percorso-progetti-e-idee-innovative

    There is time until April 30th 2024 to sign up. The whole process was very useful for anyone starting a company, they provide plenty of educational resources and mentorship.

  • An unflattering 3D reconstruction comparison

    An unflattering 3D reconstruction comparison

    We’re experimenting with macro tubes for photogrammetry, and we tried a comparison between two different cameras and two different reconstruction techniques. The subject was this rather hideous figurine of Bib Fortuna, Jabba the Hutt’s adviser, that a local supermarket gave out a few years ago.

    An ugly plastic figurine, depicting the bust and head of Bib Fortuna, with a ridged grey sphere instead of his legs.

    The difficulties stem from its diminutive size and from its texture. It is quite smooth, with lots of reflections and some subsurface effects. Also the lighting in the room was very direct, we had to move carefully to avoid throwing our shadows on the object, which negatively affected the spatial sampling.

    Our objective was clearly observing how these technologies fail, so it suited us.

    Macro Tube + Photogrammetry

    https://poly.cam/capture/82c44695-e858-4ed1-8132-dd6d44778175

    Macro Tube + Gaussian Splatting

    https://poly.cam/capture/82c44695-e858-4ed1-8132-dd6d44778175

    iPhone + Photogrammetry

    https://poly.cam/capture/0B379F23-0C60-4CD5-BB29-6D21E2A2B4E4

    iPhone + Gaussian Splatting

    https://poly.cam/capture/B3A580C6-09CE-43FD-8297-06E0F9539557

  • TIL-post: point cloud processing

    We’re working on a small archeo-acoustics project, whose starting point is a LiDAR-obtained point cloud in LAS format. This is the first time we’ve worked with this kind of data, so I’m writing down the data processing steps.

    There is a complete tutorial on handling point clouds in Blender by Florent Poux: https://youtu.be/DCkFhHNeSc0

    In order to visualize the data, my first attempt was using Blender, with this plugin: https://github.com/nittanygeek/LiDAR-Importer

    You only need to pay a bit of attention about the Python path while installing it, especially if you have multiple versions installed, but the provided instructions are perfect. We were able to see points in Blender, and to get a general idea for the shape of the cave, but getting from there to an actual mesh is non-trivial.

    Instead, we found a short tutorial for doing the same thing using CloudCompare.

    As a super short summary: open your point cloud in CloudCompare, select it, convert it to a mesh by going to Plugins>PoissonRecon. You can also choose between the color actually captured for each point, or this density heatmap: red is maximum information density, blue is the minimum.

    In the next steps we’ll need to convert the mesh to quads (using Blender), import the model into Ramsete, then assign materials to each face and run the simulation. In order to properly calibrate the model we’ll need to know the specifics of each material, or ideally even perform actual acoustics measurements, like we did for the Tindari paper.