Semantic image annotation using convolutional neural network and wordnet ontology

 
 
 
  • Abstract
  • Keywords
  • References
  • PDF
  • Abstract


    Images are a major source of content on the web. The increase in mobile phones and digital cameras have led to huge amount of non-textual data being generated which is mostly images. Accurate annotation is critical for efficient image search and retrieval. Semantic image annotation refers to adding meaningful meta-data to an image which can be used to infer additional knowledge from an image. It enables users to perform complex queries and retrieve accurate image results. This paper proposes an image annotation technique that uses deep learning and semantic labeling. A convolutional neural network is used to classify images and the predicted class labels are mapped to semantic concepts. The results shows that combining semantic class labeling with image classification can help in polishing the results and finding common concepts and themes.


  • Keywords


    Convolutional Neural Networks; Deep Learning; Image Annotation; Semantic Labeling; WordNet Ontology.

  • References


      [1] Millers, George A“WordNet: a lexical database for English.” Communications of the ACM 38.11 (1995): 39-41. https://doi.org/10.1145/219717.219748.

      [2] Stamou, Giorgos, et al., “Multimedia annotations on the semantic web.” IEEE Multimedia 13.1 (2006): 86-90 https://doi.org/10.1109/MMUL.2006.15.

      [3] Little, Suzanne, Ovidio Salvetti, and Petra Perner. “Semi-automatic semantic annotation of images.” Seventh IEEE International Conference on Data Mining Workshops (ICDMW 2007). IEEE, 2007. https://doi.org/10.1109/ICDMW.2007.22.

      [4] Verma, Yashaswi, and C. V. Jawahar. “Image Annotation by Propagating Labels from Semantic Neighborhoods.” International Journal of Computer Vision (2016): 1-23.

      [5] Dureja, Aman, & Payal Pahwa. “Image retrieval techniques: a survey.” International Journal of Engineering & Technology [Online], 7.1.2 (2018): 215-219. Web. 7 May. 2018.

      [6] Berners-Lee, Tim, James Hendler, and Ora Lassila. “The semantic web.” Scientific american 284.5 (2001): 34-43. https://doi.org/10.1038/scientificamerican0501-34.

      [7] Reena Pagare and Anita Shinde, “A Study on Image Annotation Techniques”, Harlow, England: International Journal of Computer applications, Volume 37- No.6, January 2012.

      [8] Luque, Edson F., Daniel L. Rubin, and Dilvan A. Moreira. “Automatic Classification of Cancer Tumors using Image Annotations and Ontologies.” 2015 IEEE 28th International Symposium on Computer-Based Medical Systems. IEEE, 2015.

      [9] Seifert, Sascha, et al., “Semantic annotation of medical images.” SPIE medical imaging. International Society for Optics and Photonics, 2010.

      [10] Rubin, D. L., Rodriguez, C., Shah, P., and Beaulieu, C. “Semantic Annotation and Markup of Radiological Images.” AMIA Annual Symposium Proceedings (2008), Volume 2008, p. 626.

      [11] O. Marques, N. Barman, “Semi-Automatic Semantic Annotation of Images Using Machine Learning Techniques”, Proc. of ISWC, pp. 550565, 2003.

      [12] Carneiro, Gustavo, and Nuno Vasconcelos. “Formulating semantic image annotation as a supervised learning problem.” Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on. Vol. 2. IEEE, 2005.

      [13] Feng, Linan, and Bir Bhanu. “Semantic Concept Co-Occurrence Patterns for Image Annotation and Retrieval.” IEEE transactions on pattern analysis and machine intelligence 38.4 (2016): 785-799. https://doi.org/10.1109/TPAMI.2015.2469281.

      [14] LeCun, Yann, et al., “Gradient-based learning applied to document recognition.” Proceedings of the IEEE 86.11 (1998): 2278-2324. https://doi.org/10.1109/5.726791.

      [15] LeCun, Yann, Koray Kavukcuoglu, and Cl´ement Farabet. “Convolutional networks and applications in vision.” Circuits and Systems (ISCAS), Proceedings of 2010 IEEE International Symposium on. IEEE, 2010. https://doi.org/10.1109/ISCAS.2010.5537907.

      [16] Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. “Imagenet classification with deep convolutional neural networks.” Advances in neural information processing systems. 2012.

      [17] Maxime Oquab, L´eon Bottou, Ivan Laptev, Josef Sivic. “Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks.” IEEE Conference on Computer Vision and Pattern Recognition, Jun 2014, Columbus, OH, United States. https://doi.org/10.1109/CVPR.2014.222.

      [18] Boutell, Matthew R., et al., “Learning multi-label scene classification.” Pattern recognition 37.9 (2004): 1757-1771. https://doi.org/10.1016/j.patcog.2004.03.009.

      [19] Szegedy, Christian, et al., “Rethinking the inception architecture for computer vision.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.

      [20] Pan, Sinno Jialin, and Qiang Yang. “A survey on transfer learning.” IEEE Transactions on knowledge and data engineering 22.10 (2010): 1345-1359. https://doi.org/10.1109/TKDE.2009.191.

      [21] Sai V, Bhavya, Narasimha Rao G, Ramya M, Sujana Sree Y, & Anuradha T. “Classification of skin cancer images using TensorFlow and inception v3.” International Journal of Engineering & Technology [Online], 7.2.7 (2018): 717-721. Web. 9 May. 2018.


 

View

Download

Article ID: 9886
 
DOI: 10.14419/ijet.v7i2.27.9886




Copyright © 2012-2015 Science Publishing Corporation Inc. All rights reserved.