Friday, 3 November 2017

The Lady and the Medal


For these days I had the great joy of traveling again to Peru, it is the ninth time I went to that wonderful country and on this occasion things happened in style with the illustrious company of my beautiful wife. There I revealed the face of a historical figure, known as the Lady of the Four Tupus (Brooches), and I also received an important honor.



The Lady of the Four Tupus was discovered in 2016 and is considered one of the ten greatest triumphs of archeology that year. The name refers to four pins or brooches (tupus) that were found together to her body and that served as a prop for the clothing.

Stepes of facial reconstruction
The reconstruction project of his face was a partnership of the Archaeological Zone of Caral, the University Inca Garcilaso de la Vega and myself. We started the talks in 2016, signed the documents in 2017 and the presentation took place a few days ago in the premises of the Ministry of Culture of Peru, in Lima.

Taking photos of the skull to the 3D photogrammetry
The revelation of the face was made during a press conference and the work was published in several languages, 20 altogether so far (English, Spanish, Portuguese, German, Arabic, Azerbaijani, Burmese, Bulgarian, Chinese, French, Hungarian, Italian, Lithuanian, Persian, Polish, Kyrgyz, Russian, Turkish, EN Braille and Greek).

Video with the steaps of forensic facial reconstruction

I was quite satisfied with the repercussion, even more for seeing a translation in other languages of my speech, which is always curious and pleasant.

Official image
Two days later arrived the great moment to receive the Inca Medal Garcilaso de la Vega. It is offered to personalities who contribute to the sciences and the arts. In my case, I received by the work related to the facial reconstructions of great historical and religious personalities from Peru, as well my life trajectory, according to the documents that were given to me.

My wife and me (at center) at Ministry of Culture of Peru
I do not know how to translate the feelings that took me during that moment. I remembered my childhood in poverty, the struggle of my mother and my brothers to survive with dignity. I remembered that I had a great fear to travel, and thankfully I did it in 2006 for the first time, almost forced of (lol). I remembered the difficulties inherent in my anxiety and panic attacks in the past, the strength I received from friends, family, and members of the computer community over the years. Of course, there was also room to remember those who have always tried to humiliate me throughout this story, especially the individuals connected to the academy, attentive to my lack of formal education in the fields in which I excelled thanks to hard work.

Talking with reporters during the press conference
Receiving further recognition from a higher education institution has in a sense put an ends in this discussion and brought more peace into my heart to the point of completely forgiving the uneasiness born of this egotistical swoop and now I'm enjoying victory in all its length.

For now, dear reader, that's it. Thanks for reading and see you next post!

Saturday, 23 September 2017

Rhinoplasty planning with RhinOnBlender addon

Rhinoplasty planning steps: Measurements, sculpture, post guide, pre-post guide.
I always thought that plastic surgeons were professionals that only served the moneyed elite and who passed away from the treatments that benefit the users of the Brazilian public health system. When I was asked to teach computer graphics courses to these specialists, I had the opportunity to follow their work more closely, which, unlike what I assumed, involves a routine of scientific production, clinical care and a tremendous effort to offer the general population the possibility of solving a series of problems, ranging from a structure that leverages the self-esteem of this individual, to a procedure that returns a motor skill that was limited, either by accident or pathology.


By teaching classes and adjusting Blender parameters to the needs of plastic surgeons, I realized that activities could be performed much more effectively if the commands were grouped into scripts, and these, in turn, were attached to buttons and pre-structures defined, provided by the graphical interface itself.

The idea of creating RhinOnBlender came out after we made Cork on Blender (boolean addon) available and started working on OrtogOnBlender (orthognathic surgery planning). Although the latter is not available, for the moment to the general public, we are already using it for the planning of orthognathic surgeries, which deal with facial deformations. Similar situation has happened with the rhinoplasty addon. It was born from a process of simplifying commands to beginners and we are already using it for planning, yesterday we did one!

Automatic attenuation of the 3D mesh by photogrammetry - option implemented just now
The development of the tool is the result of a partnership with three specialists: Dr. Pablo Maricevich (Cir. Plastic), Dr. Rodrigo Dornelles (Cir. Plastic) and Dr. Everton da Rosa (Cir. Dentist). Two plastic surgeons and a dentist? That's right, although they look like different areas, the planning tools of one field and another are quite compatible, what we did was the joint development over the cases that we have worked on since the year 2014.

Experimental guide made from pre and post surgical mesh
The goal of this project is to offer an alternative based on freeware and free software. The video featured in this post uses a face scanned in 3D by photos (photogrammetry), there are  licenced, freeware, free software, offline and online options of this tool, such as the popular Photoscan, Recap 360, MVE and OpenMVG. But nothing prevents the specialist from importing a face scanned by laser scanner or structured light or even reconstructed from a CT scan. The point here is that there is a possibility to do a good job without having to pay for the tools, since we have free photogrammetry and Blender and its addons are also free!

Experimental guide from different viewpoints
Speaking of addons, I can not fail to mention the enormous help of two developers: Dalai Felinto who wrote Cork on Blender, a tool that boosts the work with boolean, essential when it comes to modeling for healthcare. Also the specialist in dental tools, Patrick Moore, better known in the cybernetic environment as Patmo141. The addons developed by him are geniuses and two of them are used in ours, Cut-Mesh, which lets you draw on a mesh and isolate the desired part and Object-Align, which as its name says, aligns objects, something that we don't have available in the standard Blender installation.

We are working hard. At each surgical analysis and surgical planning we see the need for repairs and increases in the code. Once the tool is more mature and fluid, we will make it available to anyone who has an interest. We hope this happens as soon as possible.

Until then!

Saturday, 16 September 2017

Digital 3D Orthognathic Surgery with the OrtogOnBlender addon

Timelapse video of orthognatic planning steps

The first time I understood the concept of programming, it was in the year 2001, when I realized that a text with some codes was translated into a three-dimensional scene with some animated elements. I was astounded by it, as when I was three years old I realized that the water that was coming into the refrigerator freezer were those transparent pebbles my grandparents used in the juices they prepared for the family.

Since then I have always consumed some information about programming. Sometimes it would even cheer me up and write some codes, but the lack of practical use made me confine myself to just exercising my brain to understand some abstractions that involved that reality.


Many years and many books later, here I start teaching 3D computer graphics courses for the health area and realize that the organization of Blender graphic interface more confused than it helped the doctors in their didactic absorptions.

Faced with this difficulty on the part of the students, I motivated myself to create a methodology that would facilitate students' understanding in a field that proved to be quite challenging in the didactic context: orthognathic 3d digital surgery planning.

Automatically created deformation area

Unlike the other courses I had taught so far, orthognathic surgery, when taught in the "manual mode," where the student needs to understand detail by detail of commands, is almost impossible to apply to a short course. The problem is that it is precisely short courses that are the most sought after and not offering them translates into not generating income.

Together with Dr. Everton da Rosa, Dentist, orthognathic surgery specialist, I began to develop a series of small scripts that would help students automate one task or another. I did one, it worked, I cheered up... I did another, it worked out and another... I picked up a little and kept staggering and achieving, although not in the most elegant way, at least it was functional.

Jaw rotation

Blender uses Python for its scripts and this language is marvelous for anyone who wants to try to program, since the code is pretty clean. My excitement was growing to the point that I was fissured by the small challenges that were appearing.

Initially I intended to create a series of scripts, but realized that I could bundle everything into an addon, which would be installable in Blender. When programming and working with different objects and modes, I was intuitively understanding how everything happened, to the point of thinking of a solution, writing without referencing and code working!

Automatic creation of vertex groups

It was what I needed to become a hermit cyber... locked in the bedroom, just thinking about how to solve the problems and expand the functionalities. In unlikely five days, though not finalized, I was able to mount an addon sufficiently functional to simplify and automate a series of steps that comprise the planning of orthognathic surgery.

Interestingly, as a good nerd who follows the best practices of self-taught effort, I did not ask anyone for anything, I did not post on forums or social media groups on programming, nor did I ask for help from friends who are masters of language. I found everything I needed in the documentation, videos on Youtube, tutorials on the internet and, pampering ... in templates provided by Blender himself! In short, I did not ask for direct help, but I got it from the work and goodwill of people who wanted to help and made excellent materials, I am very grateful to all!

Jaw after osteotomy (Cork on Blender)

I intend to make the addon available soon (as we did with Cork on Blender), after testing it with my students. It is not yet complete, since it lacks the part of the surgical splint (a sort of guide for fitting the teeth) and some other activities such as automatic parenting with bone armature.

Even so, I am very satisfied, mainly for having implemented the automatic process of creating areas of influence based on osteotomy. With it, the cut bones, when moved make the soft tissue (skin, fat, etc.) deform. You can not imagine the challenge of teaching this in the "manual mode." Luckily it all came down to clicking two buttons, which is not bad!

Deformation radiated by the skull

I'm so happy and motivated that the will it gives is to go around programming and automating everything, but I felt on the skin another situation that is inherent in this kind of knowledge ... a code needs to be elegant and be prepared for exceptions and mine does not pass nor close to it. The trend now is to lapse the code and try to make it decent, worthy of being shared, used and improved by the community and interested in using it.

That's it, now I'm going because I had an insight on how to solve a problem in one part of the code... a big hug! : D

Tuesday, 5 September 2017

"Cork on Blender" the addon that has saved lives is available to everyone!

Retrieval of low grade osteosarcoma with virtual planning of microsurgical reconstruction with fibula flaps. Credit: Bruno Aragão.
"First do, then show"... this has been the mantra of our team. Since August 2014, three years ago, great challenges have arisen when I was contacted by Dr. Everton da Rosa to teach him 3D computer graphics techniques so that he could use them in his virtual orthognathic surgery plans.

Orthognathic surgery is a technique that corrects facial deformities in adults. I myself would need this procedure, since my upper jaw is small and the mandible is projected. This makes me look not very pleasing to the eyes, but fortunately I breathe well and ... I'm married, what would be an accomplishment for an antisocial nerd ... imagine for a short, antisocial and ugly nerd!

Joke aside, many people have serious problems as a result of these deformations, which often puts the lives of these individuals at risk. When planning a surgery digitally, specialists such as Dr. da Rosa have the opportunity to approach the procedure without the risk of causing harm to the patient, and when performing the task effectively, will be very well prepared, since he did the studies several times before surgery.

Orthognathic surgery planning with Blender and Cork.
This whole digital process would work very well, if we did not have a problem on the way, this problem was Blender's native Boolean. It should be recognized that with each version the library of these calculations has become better, but when we speak of boolean for use in health sciences it presents problem at inopportune moments.

To solve this problem, still in 2014, I looked for a classic exit for those (like me) working with Linux and free software: research a project that would allow the robust calculation of Boolean. I ended up finding it in the tool Cork, one of the great works of the developer Gilbert Bernstein.


Examples of using Cork on Blender. Left: Reconstructed hull of Freddy's turtle, in the center: facial prosthesis (Credits: Mais Identidade and UNIP) and left: prosthesis of a beak.

I was delighted with the results and proceeded with the tests, although these depended on exporting the files for the calculations to be done externally. As the activity was not very practical, came into the picture the one who made use of Cork within our beloved Blender, the expert in Python Script Dalai Felinto.

He masterfully created a code that turned a series of boring activities (export, command line, etc.) into a few grouped buttons on the upper left of 3D View. This was the trigger for a number of interesting projects.

From that moment, we started to do the surgery plans almost completely within Blender. We imported the reconstructed (skin and bone) surfaces from tomography and performed the planning, now without suffering from the part of the osteotomy, the one in which we cut the bones.

With Cork inside Blender, the osteotomy, which once drove us crazy, was now just another process, being performed not only efficiently but quickly.

In the course of the planning, other projects have touched our daily life, from the maintenance of cultural heritage, to the making of facial prostheses for humans and in the field of Veterinary Medicine.

Together with the Animal Avengers group and specialists from Italy, I created prostheses for no less than 11 animals! There were four toucans, an aracari, a dog, a goose, a crow, a turtle, a parrot, and a macaw. In addition to this area, I worked with Dr. Rodrigo Salazar with the manufacture of facial prostheses for cancer victims who had one eye removed.

Over time, a lot of people started looking for our teams so we could provide training in all of these areas and we would naturally install the addon on the students' computers. In the face of the great demand and security provided by it, we finally decided to share this tool with everyone and today we have the honor to announce it available openly and for free!

For those who want to download, just access the following link:


And follow the instructions.

You can compile Cork or download one of the versions available for the three operating systems: Windows, MacOSX or Linux.

I would like to thank all those who have helped us to develop this tool, either by learning to use it or by making ourselves available to be part of the research.

A big hug!

Sunday, 3 September 2017

3D documentation of ancient millstones: preliminary tests


The traditional drawing of ancient millstones consists in a plan (eventually with shading to give a sense of three dimensions) and one or more cross sections of the object. This is not always easy because of the dimensions, weight and sometimes shape (mainly the Pre-Roman millstones are irregular and asymmetric) of this type of artefacts. Furthermore, the millstones are generally in museums or in storerooms: in these places, it is often difficult to move the objects or to have enough time for drawing quietly and checking well all the details. In short, drawing a millstone is not like drawing a sherd of pottery!
For these reasons, it could be useful applying a methodology based on the Structure from Motion (SfM) techniques in addition to the traditional drawing methods. In this post I’m going to present the preliminary results of a test aimed at the three-dimensional documentation of a fragment of an Iron Age millstone from Northern Italy (a so-called “Olynthus mill” or “Hopper rubber”).

The first step was the construction of a rectangular wooden frame made by 4 rods of different length (40, 60, 80 cm), so that it’s possible to build frames of different areas according to the dimensions of the millstone to be drawn. On the surface of the rods some cross marks equally spaced are signed: these marks will be use as reference points with known coordinates for the rectification of the3D point cloud, mesh and (eventually) texture (something like GCP, Ground Control Points).


Four bolts adjustable for height hold the frame together and allow to level it perfectly. Once the frame is ready, you need to enter the millstone into it, in such a way as to leave sufficient space between the stone and the rods for taking pictures.
Some recognizable markers should be placed in different points of the millstone: these are for aligning and merging the two point clouds that will be generated (see below). A simple solution is to use small spheres of coloured modelling clay visible in the point clouds.



At this stage, you start with the typical workflow of the SfM. You take an appropriate number of pictures of the upper surface first; then do the same to the lower surface, turning upside down the millstone inside the frame.
The pictures can be processed by the software you want. I used Regard3D / OpenMVG for generating two point clouds (one for the upper surface and the other for the lower) and CloudCompare for editing/cleaning the point clouds and for performing their rectification (thanks to the cross marks on the frame), alignment and merging (thanks to the coloured markers on the stone). CloudCompare and MeshLab have also been used for generating meshes and for computing other parameters, among which the measurements.



The final result is a point cloud and a mesh of the millstone.


Using MeshLab you could also obtain the texture of the object, but for my aims it’s enough a 3D model (point clouds or mesh) from which I can get a plan, some cross sections and all the measures I need. Thanks to these data, I can detail or check my handmade drawing or do it from scratch.


In conclusion, the usage of an homemade wooden frame makes easier and more precise the data acquisition for the SfM and make faster and more complete the documentation of this kind of artefacts. The method described leaves room for improvements and developments; it could become a “standard” documentation technique for the ancient millstones and for other archaeological objects with analogous drawing issues.

Thank’s to Alessandro Bezzi (Arc-Team).

Denis Francisci




Saturday, 26 August 2017

Mapping high alpine lakes for archaeologic explorations

Hi all,
as you see we are writing few post in ATOR in the summer season, due to different field projects which take us away from home. Today I try to start again to dedicate some time to our research blog.
The topic of this post regard a solution we are currently using to help us in the archaeological exploration of high alpine lakes: the documentation of the bathymetry through a low cost sonar.
As you maybe know, since a couple of year we are working on underwater archaeology projects in the alpine lakes of our region (here an example). This kind of exploratory mission are difficult, due to the altitude of the site we have to investigate (almost always over 2000 meters asl), so that our divers have to acclimate themselves for one whole day, before starting the working. Also for this reason we started again to study archeorobotics and develop, together with our friends of the WitLab, an open hardware ROV called ArcheoROV (in order to help divers in exploratory mission).


The ArcheoROV (photo by WitLab)
 
This year we focused or research in find a cheap solution to map the bathymetry of the lakes, while WitLab went on working on the  Wi-Fi buoy which gives our ROV a long-range operability (respect the limitation of a simple control on shore). For this reason we tested a cheap sonar called Deeper, which normally is used as a fishfinder.
We started our test in the Lake Tovel, thanks to the hep of Prof. +Tiziano Camagna , who is leading the exploration project since many years. This lake is almost our playground to develop and test new solutions for underwater archaeology, since it is a difficult environment, but not extreme (like other high mountain lakes). We chose this location also because, on unlike other lakes, its bathimetry was documented by Edgardo Baldi in the 30s. We already digitized this map, processing a 3D model in GRASS GIS, so that we have some data to check our results with our small sonar (as you can see in the image below).

On the left the map drawn by Edgardo Baldi between 1937 and 1938; on the right the 3D derived map developed by Arc-Team in GRASS GIS

Some more details of the 3D map developed with GRASS GIS

To test the Deeper sonar, Porf. +Tiziano Camagna designed a small buoy which can be towed by a kayak. This solution stabilize the sonar (which remain always in the right position) and, at the same time, avoid its submersion (which causes the lost of the GPS signal).

The stabilization buoy developed by Prof. +Tiziano Camagna 

First positive results (image below) encouraged us to use this solution on a real mission, at the Monticello lake (almost 2600 meters asl), at Paradiso Pass (near Tonale Pass, Trentino, Italy).


A comparison between the digitized map of E. Baldi (on the left) and the map (work in progress) obtained with the Deeper sonar (on the right)

The expedition was joined also by our friends of the Team Nauticamare (Massimiliano Canossa and Nicola Boninsegna) and gave us the opportunity to accomplish a first mapping of the Lake Monticello, during the first day of acclimatization. This helped us very much during the archaeological underwater mission of the second day. As a result we have now a good 3D map of the bathymetry of the lake, which we will use also in the next expedition (September 2017). Her below is a short video (done with +QGIS plugin qgis2threejs), which shows the 3D model of the lake.




PS
I recorded some videotutorial related with the processing of these data. I will try to upload them ASAP in our channel.

Have a nice day!


Sunday, 30 July 2017

The Lady of K'anamarka and the relevance of computer graphics for tourism



For the eighth time, I traveled to Peru, as I have always been very well treated and enjoyed good moments throughout the social spectrum. The big difference from this episode in relation to the other seven is perhaps the fact that I realized how graphic computing might be relevant to apparently disconnected areas of their initial claims, such as tourism. In the face of this I will share with you my reflections and history of how things unfolded in that beautiful Andean country.

Stages of facial reconstruction of the Lady of K'anamarka. This woman was a victim of a trepanation surgery on her modified skull (artificially deformed skull, "ET skull").

Something similar had happened in previous episodes, it is enough to mention the facial reconstruction of Gufán (the Brazilian of 2000 years) that took in one day the number of visitors of a month to the Museum Paranaense. The presentation of the bust of St. Valentine in St. Valentine of the South city (Brazil), which crowded a 1200-seat church in a town of just over 2000 inhabitants. In addition to these two facts, I can not fail to mention the reconstruction of the men of Lagoa Santa, a feat propagated in the national and international media, which, of course, drew attention to the archaeological site of the city, as well as considerably increased the number of visitors to the Museum of the Lapinha.
Views of the skull of the Lady of K'anamarka

The Lady of K'anamarka


Returning to Peru, the first project presented there was the face of the Lady of K'anamarka, whose remains were found by the archaeologist Marco del Pezo in 2004. I was invited to participate in this project by physical anthropologists Dr. Oliver Medina and Dr. Elva Torres. The work was tranquil, since Dr. Medina is a specialist in photogrammetry and passed me the richly digitized skull in 3D. From there I built the face and designed its presentation with the help of the Inca Garcilaso de la Vega University, of which I am part of the team and with whom I developed a series of projects in that country.


Some interesting events guided the presentation of the face. Initially the revelation of the face was scheduled to be made in a room with 120 seats. On the first day of the seminar where we were introduced, the audience showed up in weight and many people had to watch the event standing up. This forced the organizers to change rooms, the other day the face presentation happened in a space for 400 seats! The press appeared in weight and the news of the reconstruction became headline in the main newspapers and TVs of Peru.


The other day I was invited to visit the archaeological site of K'anamarka, where they discovered the skull. A four-hour trip from Cusco, where we met. Beyond the distance we would climb a little higher, towards 4000 meters of altitude! I feared for my breath and my stomach, but fortunately for all indications I have excellent resistance to altitude, because I felt nothing different, except for the air that was not as abundant as in the lower parts.


Arriving at the site I was treated almost like a head of state, at least it was what I felt. After a four-hour drive and in that place of clean, wonderful visual air, I could see that a delegation was already waiting for us at the entrance to the old city. As I descended, I was approached by reporters, guides and representatives of the Municipality of Espinar, who gave me some gifts, among them showy local clothes, which softened the cold and warmed the heart with so much human warmth.


Each step we took was accompanied by a series of people, all properly filmed and photographed.


After visiting all points of the site, we were treated to a delicious lunch, with delicacies from the local cuisine. The soul was fed, but the body was not yet, we did not waste time and devoured that delicacy, closing the bill with a local drink that aided in digestion.


The other day we had the great joy of seeing the enormous repercussion of the work in newsstands and national TVs. More than that, TV Latina, one of the main ones of Peru made a special matter from the facial reconstruction. The material they composed (thanks immensely to the reporter Jesus Noriega) addressed our work and gave a tremendous emphasis to tourism of the region, while showing the natural beauties and qualities of its warm people. In total there were 17 minutes of video transmitted to the whole country!


The statue of the Lord of Sipán


The facial reconstruction of the Lord of Sipán was a work begun in 2016, also in partnership with the Inca Garcilaso de la Vega University, the EBRAFOL and the Royal Tombs of Sipán Museum. That year we unveiled the face digitally and the event became news worldwide, being propagated in 27 languages!


The closing of this project took place in great style, with the printing of the bust by the CTI Renato Archer and the painting of the piece by the skilled hands of the artist Mari Bueno, award-winning Brazilian painter, whose works were the theme of exhibitions in Brazil and in other countries.


Unlike the 2016 event that took place in Lima, this was held in Lambayeque, more precisely at the archaeological site where they found the body of the Lord of Sipan. In addition to the presence of Dr. Walter Alva, the archaeologist who discovered it, the Peruvian Minister of Culture, Salvador del Solar, a former actor very dear to the population, whose management has high approval ratings, Cultural development.



I was very happy because my presence and that of Dr. Eduardo Ugaz Burga, dean of Garcilaso University was informed during the ceremony and things did not stop there. Soon after the revelation of the bust, me and the university were awarded an honor offered by the Regional Government of Lambayeque.

Like K'anamarka, the other day the newspapers and TVs stamped the presentation of the statue of the Lord of Sipan.

Conclusion


I'm kind of tired of talking about it, but research work is almost always an activity that happens away from the knowledge of the general public and the spotlight of the press. However, when this work is recognized the feeling of gratitude goes beyond the limits of the body and the smile. I am honored to promote the tourism of a region and, in a way, to make known the places where I present the faces. The joy of the conquest is only tainted, when distance myself from the great friends that I am getting on these trips.


BlogItalia - La directory italiana dei blog Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.