Color segmentation and neural networks for automatic graphic relief of the state of conservation of artworks

Authors

  • Annamaria Amura University of Urbino Carlo Bo. Department of Pure and Applied Sciences, Urbino, Italy
  • Anna Tonazzini National Research Council of Italy, Institute of Information Science and Technologies, Signals and Images Laboratory. Pisa, Italy.
  • Emanuele Salerno National Research Council of Italy, Institute of Information Science and Technologies, Signals and Images Laboratory. Pisa, Italy.
  • Stefano Pagnotta National Research Council of Italy, Institute of Chemistry of Organometallic Compounds, Applied and Laser Spectroscopy Laboratory, Pisa, Italy.
  • Vincenzo Palleschi National Research Council of Italy, Institute of Chemistry of Organometallic Compounds, Applied and Laser Spectroscopy Laboratory, Pisa, Italy

DOI:

https://doi.org/10.23738/CCSJ.120201

Keywords:

Multispectral images, Segmentation algorithms, Image analysis, Shape representation and analysis, Cultural heritage, Raster to vector, Neural networks

Abstract

This paper proposes a semi-automated methodology based on a sequence of analysis processes performed on multispectral images of artworks and aimed at the extraction of vector maps regarding their state of conservation. The graphic relief of the artwork represents the main instrument of communication and synthesis of information and data acquired on cultural heritage during restoration. Despite the widespread use of informatics tools, currently, these operations are still extremely subjective and require high execution times and costs. In some cases, manual execution is particularly complicated and almost impossible to carry out. The methodology proposed here allows supervised, partial automation of these procedures avoids approximations and drastically reduces the work times, as it makes a vector drawing by extracting the areas directly from the raster images. We propose a procedure for color segmentation based on principal/independent component analysis (PCA/ICA) and SOM neural networks and, as a case study, present the results obtained on a set of multispectral reproductions of a painting on canvas.

Author Biographies

  • Annamaria Amura, University of Urbino Carlo Bo. Department of Pure and Applied Sciences, Urbino, Italy

    Annamaria Amura is a Ph.D. Candidate in Computer Science at the University of Urbino. She holds a BS degree in Technology for the Conservation and Restoration of Cultural Heritage, Class 41, and an M.Sc. degree in Graphics of Images, LM12, Documentation, and Photography for Cultural Heritage. Her research interests include digital photography, image analysis, features extraction from diagnostic images, raster to vector automation method, GIS database, virtual restoration, and graphic documentation.

  • Anna Tonazzini, National Research Council of Italy, Institute of Information Science and Technologies, Signals and Images Laboratory. Pisa, Italy.

    Anna Tonazzini is a senior researcher at the Institute of Information Science and Technologies, National Research Council of Italy, in Pisa. She coordinated several Projects in Image Processing and Analysis, Neural Networks and Learning, Computational Biology and Document Analysis, and is co-author of over 100 peer-reviewed papers. In particular, she was the ISTI responsible for the UE Project ISYREADET, and several national projects on historical manuscript virtual restoration and analysis.

  • Emanuele Salerno, National Research Council of Italy, Institute of Information Science and Technologies, Signals and Images Laboratory. Pisa, Italy.

    Emanuele Salerno is a senior researcher at the Institute of Information Science and Technologies of the National Research Council of Italy. His interests range from nondestructive evaluation to image processing and analysis, with several applications. Since 2004 he is involved in studies on ICT technologies for cultural heritage, working on the virtual restoration of ancient documents and paintings. He is a senior member of the IEEE, Signal Processing Society, of the Italian Association of Electricity, Electronics, Automation, Informatics, and Telecommunications (AEIT).

  • Stefano Pagnotta, National Research Council of Italy, Institute of Chemistry of Organometallic Compounds, Applied and Laser Spectroscopy Laboratory, Pisa, Italy.

    Stefano Pagnotta Dr. Pagnotta is an editorial board member of the Anthropology Open Journal and "Cultore della Materia" in Archaeology at the University of Pisa. Ph.D. in Earth Sciences, Md in Archaeology and Bd in Conservation of Cultural Heritage. Actually, he is a research fellow at ICCOM CNR. Visiting scientist at the Institut für Chemie in Potsdam, at the Pure and the Applied Science Department University of Urbino and theVinča Nuclear Institute. Author of more than 50 papers in international peer-reviewed journals.

  • Vincenzo Palleschi, National Research Council of Italy, Institute of Chemistry of Organometallic Compounds, Applied and Laser Spectroscopy Laboratory, Pisa, Italy

    Vincenzo Palleschi is a Physicist, Senior Researcher at the Institute of Chemistry of Organometallic Compounds, and Head of the Applied and Laser Spectroscopy Laboratory at Research Area of CNR in Pisa (Italy). Is a world-renowned expert in LIBS, in X-Ray Fluorescence analysis, micro-Raman spectroscopy, Multispectral Imaging, 3D photogrammetry and Chemometrics. He has published more than 200 papers in ISI journals and the book 'Laser-Induced Breakdown Spectroscopy: Principles and Applications'.

References

Amura, A., et al (2019), 'Proposta di una Metodologia Semi Automatica Basata sull’Analisi dell’Immagine Digitale Finalizzata alla Creazione e la Gestione della Documentazione Grafica nei Restauri', XVII Congresso Nazionale IGIIC, Matera, Italy, 10-12 October 2019.

Cardoso, J.F. (1998), 'Blind Signal Separation: Statistical Principles', Proc. IEEE, Vol. 86, No. 10, pp. 2009-2025.

Cichocki, A., Amari, S.-I., (2002), 'Adaptive Blind Signal and Image Processing', Wiley, New York.

Danese, M., Sileo, M., Lasaponara, R., Masini, N. (2019) 'Proximal Remote Sensing e spatial Analysis per la conservazione delle pitture parietali pompeiane. Il caso del Gymnasium' Archeomatica, Anno X- Numero 2, Giugno 2019, pp. 12-18.

Dyer, J. et al (2013). Multispectral Imaging in Reflectance and Photo- induced Luminescence modes: a User Manual. Web publication/site, Online: European CHARISMA Project.

Grifoni E., Campanella B., Legnaioli S., Lorenzetti G., Marras L., Pagnotta S., Palleschi V., Poggialini F., Salerno E., Tonazzini A. (2019) 'A new infrared true-color approach for visible-infrared multispectral image analysis', ACM J Comp. Cultural Heritage 12: pp. 8:1–8:11.

Grilli, E., Petrucci, G., Remondino, F. (2019). Classification of 3D Digital Heritage. Journal of Remote Sensing. 11. 10.3390/rs11070847.

Hyvärinen, A., Karhunen, J., Oja, E. (2001), 'Independent Component Analysis', New York, Wiley.

Hyvärinen, A., Oja, E. (2000), 'Independent component analysis: algorithms and applications', Neural Networks 13, pp. 411- 430.

Koh, J., Suk, M., Bhandarkar, S. M. (1995), 'A multilayer self-organizing feature map for range image segmentation', Neural Networks 8, pp. 67– 86.

Kohonen, T. (1998.), 'The self-organizing map', Neurocomputing 21, pp. 1–6.

Mabrouk, M., Karrar A., Sharawi A. (2013), 'Support Vector Machine Based Computer Aided Diagnosis System for Large Lung Nodules Classification', Journal of Medical Imaging and Health Informatics 3, pp.214-220.

Riccomini, M. (2012), 'Donato Creti. Le opere su carta. Catalogo ragionato', collana: Archivi di Arte antica. Umberto Allemandi & C., ISBN: 88-422-1980-0 - EAN: 9788842219804

Rumelhart, D. E., Zipser, D. (1985), 'Feature discovery by competitive learning', Cogn. Sci. 9, pp.75–112.

Sacco, F. (2002), 'Sistematica della documentazione e progetto di restauro', Bollettino ICR, N.S. 4(1), pp. 28-54.

Tonazzini, A., Salerno, E., Abdel-Salam, Z. A., Harith, M. A., Marras, L., Botto, A., Campanella, B., Legnaioli, S., Pagnotta, S., Poggialini, F., Palleschi, V. (2019b), 'Analytical and mathematical methods for revealing hidden details in ancient manuscripts and paintings: A review', J. Adv. Res. 17, pp. 31–42.

Tonazzini, A., Salerno, E., Bedini, L. (2007), 'Fast correction of bleed- through distortion in grayscale documents by a blind source separation technique', Int. J. Document Analysis and Recognition 10, pp. 17-25.

Tonazzini, A., Savino, P., Salerno, E., Hanif, M., Debole, F. (2019a) 'Virtual restoration and content analysis of ancient degraded manuscripts' Int. J. Inf. Sci. & Technol. 3, no. 5.

Uriarte, E. A., Martín, F. D. (2005), 'Topology preservation in SOM ' J. Appl. Math. Comput. Sci. 1, pp.19–22.

Vallet, J.M., De Luca, L., Feillou, M. (2012) 'Une nouvelle approche spatio-temporelle et analytique pour la conservation des peintures murales sur le long terme', In Situ [En ligne], 19 | URL: http://insitu.revues.org/9829.

Yeo, N. C., Lee, K. H., Venkatesh, Y. V, Ong, S. H. (2005), 'Colour image segmentation using the self-organizing map and adaptive resonance theory', Image Vis. Comput. 23, 1060–1079.

Zhao Y., (2008), 'Image segmentation and pigment mapping of cultural heritage based on spectral imaging'. Thesis. Rochester Institute of Technology. Available from https://scholarworks.rit.edu/theses/3029 [accessed Jan 20 2020].

Downloads

Published

2020-07-01

Issue

Vol. 12 No. 02 (2020)

Section

Papers

License

The "Cultura e Scienza del Colore - Color Culture and Science" journal is registered at the Court of Milan at n.233 of 24.06.2014.

The journal is an open access journal, free for readers and authors and has joined ROAD, the Directory of Open Access scholarly Resources, since 2014. Articles published in the “Cultura e Scienza del Colore - Color Culture and Science" journal are open access articles, distributed under the terms and conditions of the Creative Commons Attribution License (CC BY). The copyright is retained by the author(s).

How to Cite

“Color segmentation and neural networks for automatic graphic relief of the state of conservation of artworks” (2020) Cultura e Scienza del Colore - Color Culture and Science, 12(02), pp. 07–15. doi:10.23738/CCSJ.120201.