(Publisher of Peer Reviewed Open Access Journals)

International Journal of Advanced Computer Research (IJACR)

ISSN (Print):2249-7277    ISSN (Online):2277-7970
Volume-11 Issue-53 March-2021
Full-Text PDF
Paper Title : Food recognition improvement by using hyper-spectral imagery
Author Name : Shirin Nasr Esfahani, Venkatesan Muthukumar, Emma E. Regentova, Kazem Taghva and Mohamed Trabia
Abstract :

In the past decade, dietary assessment has been one of the most popular topics of research in the food industry, which has resulted in developing several automatic or semi-automatic dietary assessment systems using visible spectrum images for food recognition. However, the main shortcoming of visible spectrum image-based systems is its inability to differentiate foods of similar color. Researchers have added additional features such as shape, size and texture to the color model to improve the overall accuracy. However, the shape and size features are rendered inefficient when recognizing food in the mixed or cooked form. The aim of this research is to show the capability of hyperspectral bands for accurate food recognition based on individual spectral bands. In this work we use a hyperspectral imaging system of 240 spectral bands with the wavelength range between 400 nm to 900 nm. The ReliefF and PCA methods select/extract less, but the most informative features which are important to learn Logistic Regression and Support Vector Machine (SVM) as binary and multiple classifiers, respectively. A total of 20 different food samples in various forms (uncut and cut), shapes, and sizes were used in this study. The prediction results indicate that the hyperspectral images have the advantages of being able to recognize different food items from a mixed form with similar color and similar food types with different colors. In our experiments the highest classification accuracy of 0.6874 with 20 different food samples is produced by SVM multi-classification of ReliefF data with the top 110 hyperspectral features. We are able to obtain approximately 0.90 accuracy, using binary classification on a specific subset of food samples.

Keywords : Food recognition, Hyperspectral, Logistic regression, PCA, ReliefF, RGB, SVM.
Cite this article : Esfahani SN, Muthukumar V, Regentova EE, Taghva K, Trabia M. Food recognition improvement by using hyper-spectral imagery . International Journal of Advanced Computer Research. 2021; 11(53):23-50. DOI:10.19101/IJACR.2021.1152006.
References :
[1]Esfahani SN, Muthukumar V, Regentova EE, Taghva K, Trabia M. Complex food recognition using hyper-spectral imagery. In annual computing and communication workshop and conference 2020 (pp. 662-7). IEEE.
[Crossref] [Google Scholar]
[2]Khanna N, Boushey CJ, Kerr D, Okos M, Ebert DS, Delp EJ. An overview of the technology assisted dietary assessment project at Purdue University. In international symposium on multimedia 2010 (pp. 290-5). IEEE.
[Crossref] [Google Scholar]
[3]Zhu F, Bosch M, Boushey CJ, Delp EJ. An image analysis system for dietary assessment and evaluation. In international conference on image processing 2010 (pp. 1853-6). IEEE.
[Crossref] [Google Scholar]
[4]Zhu F, Bosch M, Woo I, Kim S, Boushey CJ, Ebert DS, Delp EJ. The use of mobile devices in aiding dietary assessment and evaluation. IEEE Journal of Selected Topics in Signal Processing. 2010; 4(4):756-66.
[Crossref] [Google Scholar]
[5]Kong F, Tan J. Dietcam: regular shape food recognition with a camera phone. In international conference on body sensor networks 2011 (pp. 127-32). IEEE.
[Crossref] [Google Scholar]
[6]Bosch M, Zhu F, Khanna N, Boushey CJ, Delp EJ. Combining global and local features for food identification in dietary assessment. In 18th international conference on image processing 2011 (pp. 1789-92). IEEE.
[Crossref] [Google Scholar]
[7]Xu C, Khanna N, Boushey CJ, Delp EJ. Low complexity image quality measures for dietary assessment using mobile devices. In international symposium on multimedia 2011 (pp. 351-6). IEEE.
[Crossref] [Google Scholar]
[8]Rahman MH, Li Q, Pickering M, Frater M, Kerr D, Bouchey C, et al. Food volume estimation in a mobile phone based dietary assessment system. In international conference on signal image technology and internet based systems 2012 (pp. 988-95). IEEE.
[Crossref] [Google Scholar]
[9]Zhu F, Bosch M, Khanna N, Boushey CJ, Delp EJ. Multiple hypotheses image segmentation and classification with application to dietary assessment. IEEE Journal of Biomedical and Health Informatics. 2014; 19(1):377-88.
[Crossref] [Google Scholar]
[10]Liu C, Cao Y, Luo Y, Chen G, Vokkarane V, Yunsheng M, et al. A new deep learning-based food recognition system for dietary assessment on an edge computing service infrastructure. IEEE Transactions on Services Computing. 2017; 11(2):249-61.
[Crossref] [Google Scholar]
[11]Jiang L, Qiu B, Liu X, Huang C, Lin K. DeepFood: food image analysis and dietary assessment via deep model. IEEE Access. 2020; 8:47477-89.
[Crossref] [Google Scholar]
[12]Tan RZ, Chew X, Khaw KW. Quantized deep residual convolutional neural network for image-based dietary assessment. IEEE Access. 2020; 8:111875-88.
[Crossref] [Google Scholar]
[13]Chen M, Dhingra K, Wu W, Yang L, Sukthankar R, Yang J. PFID: pittsburgh fast-food image dataset. In international conference on image processing 2009 (pp. 289-92). IEEE.
[Crossref] [Google Scholar]
[14]Chen HC, Jia W, Li Z, Sun YN, Sun M. 3D/2D model-to-image registration for quantitative dietary assessment. In annual northeast bioengineering conference 2012 (pp. 95-6). IEEE.
[Crossref] [Google Scholar]
[15]He J, Shao Z, Wright J, Kerr D, Boushey C, Zhu F. Multi-task image-based dietary assessment for food recognition and portion size estimation. In conference on multimedia information processing and retrieval 2020 (pp. 49-54). IEEE.
[Crossref] [Google Scholar]
[16]Hafiz R, Islam S, Khanom R, Uddin MS. Image based drinks identification for dietary assessment. In international workshop on computational intelligence 2016 (pp. 192-7). IEEE.
[Crossref] [Google Scholar]
[17]Lo FP, Sun Y, Qiu J, Lo B. A novel vision-based approach for dietary assessment using deep learning view synthesis. In 16th international conference on wearable and implantable body sensor networks 2019 (pp. 1-4). IEEE.
[Crossref] [Google Scholar]
[18]He Y, Xu C, Khanna N, Boushey CJ, Delp EJ. Analysis of food images: Features and classification. In international conference on image processing 2014 (pp. 2744-8). IEEE.
[Crossref] [Google Scholar]
[19]Lo FP, Sun Y, Qiu J, Lo BP. Point2volume: a vision-based dietary assessment approach using view synthesis. IEEE Transactions on Industrial Informatics. 2019; 16(1):577-86.
[Crossref] [Google Scholar]
[20]Subhi MA, Ali SH, Mohammed MA. Vision-based approaches for automatic food recognition and dietary assessment: a survey. IEEE Access. 2019; 7:35370-81.
[Crossref] [Google Scholar]
[21]Park B, Lu R, editors. Hyperspectral imaging technology in food and agriculture. New York: Springer; 2015.
[Crossref] [Google Scholar]
[22]Sun DW. Hyperspectral imaging for food quality analysis and control. Elsevier; 2010.
[Google Scholar]
[23]Bhunia AK, Kim MS, Taitt CR, editors. High throughput screening for food safety assessment: Biosensor technologies, hyperspectral imaging and practical applications. Elsevier; 2014.
[Google Scholar]
[24]Haboudane D, Miller JR, Pattey E, Zarco-Tejada PJ, Strachan IB. Hyperspectral vegetation indices and novel algorithms for predicting green LAI of crop canopies: modeling and validation in the context of precision agriculture. Remote Sensing of Environment. 2004; 90(3):337-52.
[Google Scholar]
[25]Haboudane D, Miller JR, Tremblay N, Zarco-Tejada PJ, Dextraze L. Integrated narrow-band vegetation indices for prediction of crop chlorophyll content for application to precision agriculture. Remote Sensing of Environment. 2002; 81(2-3):416-26.
[Google Scholar]
[26]Rodarmel C, Shan J. Principal component analysis for hyperspectral image classification. Surveying and Land Information Science. 2002; 62(2):115-22.
[Google Scholar]
[27]Chang CI, Du Q, Sun TL, Althouse ML. A joint band prioritization and band-decorrelation approach to band selection for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing. 1999; 37(6):2631-41.
[Crossref] [Google Scholar]
[28]Guo B, Gunn SR, Damper RI, Nelson JD. Band selection for hyperspectral image classification using mutual information. IEEE Geoscience and Remote Sensing Letters. 2006; 3(4):522-6.
[Crossref] [Google Scholar]
[29]MartÍnez-UsÓMartinez-Uso A, Pla F, Sotoca JM, García-Sevilla P. Clustering-based hyperspectral band selection using information measures. IEEE Transactions on Geoscience and Remote Sensing. 2007; 45(12):4158-71.
[Crossref] [Google Scholar]
[30]Du Q, Yang H. Similarity-based unsupervised band selection for hyperspectral image analysis. IEEE Geoscience and Remote Sensing Letters. 2008; 5(4):564-8.
[Crossref] [Google Scholar]
[31]Melgani F, Bruzzone L. Classification of hyperspectral remote sensing images with support vector machines. IEEE Transactions on Geoscience and Remote Sensing. 2004; 42(8):1778-90.
[Crossref] [Google Scholar]
[32]Gualtieri JA, Chettri S. Support vector machines for classification of hyperspectral data. IEEE 2000 international geoscience and remote sensing symposium. Taking the pulse of the planet: the role of remote sensing in managing the environment. Proceedings 2000 (pp. 813-5). IEEE.
[Crossref] [Google Scholar]
[33]Puri M, Zhu Z, Yu Q, Divakaran A, Sawhney H. Recognition and volume estimation of food intake using a mobile device. In workshop on applications of computer vision 2009 (pp. 1-8). IEEE.
[Crossref] [Google Scholar]
[34]Rahmana MH, Pickering MR, Kerr D, Boushey CJ, Delp EJ. A new texture feature for improved food recognition accuracy in a mobile phone based dietary assessment system. In international conference on multimedia and expo workshops 2012 (pp. 418-23). IEEE.
[Crossref] [Google Scholar]
[35]Pouladzadeh P, Villalobos G, Almaghrabi R, Shirmohammadi S. A novel SVM based food recognition method for calorie measurement applications. In international conference on multimedia and expo workshops 2012 (pp. 495-8). IEEE.
[Crossref] [Google Scholar]
[36]Duan P, Wang W, Zhang W, Gong F, Zhang P, Rao Y. Food image recognition using pervasive cloud computing. In international conference on green computing and communications and IEEE internet of things and IEEE cyber, physical and social computing 2013 (pp. 1631-7). IEEE.
[Crossref] [Google Scholar]
[37]Pham C, Thanh TN. Fresh food recognition using feature fusion. In international conference on advanced technologies for communications 2014 (pp. 298-302). IEEE.
[Crossref] [Google Scholar]
[38]Zhang W, Zhao D, Gong W, Li Z, Lu Q, Yang S. Food image recognition with convolutional neural networks. In international conference on Ubiquitous intelligence and computing and international conference on autonomic and trusted computing and 2015 IEEE 15th international conference on Scalable Computing and communications and its associated workshops (UIC-ATC-ScalCom) 2015 (pp. 690-3). IEEE.
[Crossref] [Google Scholar]
[39]Pouladzadeh P, Kuhad P, Peddi SV, Yassine A, Shirmohammadi S. Food calorie measurement using deep learning neural network. In international instrumentation and measurement technology conference proceedings 2016 (pp. 1-6). IEEE.
[Crossref] [Google Scholar]
[40]He H, Kong F, Tan J. DietCam: multiview food recognition using a multikernel SVM. IEEE Journal of Biomedical and Health Informatics. 2015; 20(3):848-55.
[Crossref] [Google Scholar]
[41]Chen J, Ngo CW. Deep-based ingredient recognition for cooking recipe retrieval. In proceedings of the ACM international conference on Multimedia 2016 (pp. 32-41). ACM.
[Crossref] [Google Scholar]
[42]Lu Y. Food image recognition by using convolutional neural networks (cnns). arXiv preprint arXiv:1612.00983. 2016.
[Google Scholar]
[43]Ciocca G, Napoletano P, Schettini R. Food recognition: a new dataset, experiments, and results. IEEE Journal of Biomedical and Health Informatics. 2016; 21(3):588-98.
[Crossref] [Google Scholar]
[44]Chu WT, Lin JH. Food image description based on deep-based joint food category, ingredient, and cooking method recognition. In international conference on multimedia & expo workshops 2017 (pp. 109-14). IEEE.
[Crossref] [Google Scholar]
[45]Deng L, Chen J, Sun Q, He X, Tang S, Ming Z, et al. Mixed-dish recognition with contextual relation networks. In proceedings of the ACM international conference on multimedia 2019 (pp. 112-20).
[Crossref] [Google Scholar]
[46]Zhang L, Zhao J, Li S, Shi B, Duan LY. From market to dish: multi-ingredient image recognition for personalized recipe recommendation. In international conference on multimedia and expo 2019 (pp. 1252-7). IEEE.
[Crossref] [Google Scholar]
[47]Liu C, Liang Y, Xue Y, Qian X, Fu J. Food and ingredient joint learning for fine-grained recognition. IEEE Transactions on Circuits and Systems for Video Technology. 2020.
[Crossref] [Google Scholar]
[48]Liang H, Wen G, Hu Y, Luo M, Yang P, Xu Y. MVANet: multi-tasks guided multi-view attention network for Chinese food recognition. IEEE Transactions on Multimedia. 2020:1-12.
[Crossref] [Google Scholar]
[49]Freitas CN, Cordeiro FR, Macario V. MyFood: a food segmentation and classification system to Aid nutritional monitoring. In SIBGRAPI conference on graphics, patterns and images 2020 (pp. 234-9). IEEE.
[Crossref] [Google Scholar]
[50]Chen J, Zhu B, Ngo CW, Chua TS, Jiang YG. A study of multi-task and region-wise deep learning for food ingredient recognition. IEEE Transactions on Image Processing. 2020; 30:1514-26.
[Crossref] [Google Scholar]
[51]Barbin D, Elmasry G, Sun DW, Allen P. Near-infrared hyperspectral imaging for grading and classification of pork. Meat Science. 2012; 90(1):259-68.
[Google Scholar]
[52]Cluff K, Naganathan GK, Subbiah J, Samal A, Calkins CR. Optical scattering with hyperspectral imaging to classify longissimus dorsi muscle based on beef tenderness using multivariate modeling. Meat Science. 2013; 95(1):42-50.
[Google Scholar]
[53]Marshall S, Kelman T, Qiao T, Murray P, Zabalza J. Hyperspectral imaging for food applications. In European signal processing conference 2015 (pp. 2854-8). IEEE.
[Crossref] [Google Scholar]
[54]Wang L, Liu D, Pu H, Sun DW, Gao W, Xiong Z. Use of hyperspectral imaging to discriminate the variety and quality of rice. Food Analytical Methods. 2015; 8(2):515-23.
[Crossref] [Google Scholar]
[55]Al-Sarayreh M, Reis MM, Yan WQ, Klette R. Deep spectral-spatial features of snapshot hyperspectral images for red-meat classification. In international conference on image and vision computing 2018 (pp. 1-6). IEEE.
[Crossref] [Google Scholar]
[56]Qiu Z, Chen J, Zhao Y, Zhu S, He Y, Zhang C. Variety identification of single rice seed using hyperspectral imaging combined with convolutional neural network. Applied Sciences. 2018; 8(2):1-12.
[Crossref] [Google Scholar]
[57]Liu Y, Wu T, Yang J, Tan K, Wang S. Hyperspectral band selection for soybean classification based on information measure in FRS theory. Biosystems Engineering. 2019; 178:219-32.
[Google Scholar]
[58]Steinbrener J, Posch K, Leitner R. Hyperspectral fruit and vegetable classification using convolutional neural networks. Computers and Electronics in Agriculture. 2019; 162:364-72.
[Google Scholar]
[59]Xia C, Yang S, Huang M, Zhu Q, Guo Y, Qin J. Maize seed classification using hyperspectral image coupled with multi-linear discriminant analysis. Infrared Physics & Technology. 2019; 103:103077.
[Google Scholar]
[60]Xiao H, Guo P, Dong X, Xing S, Sun M. Research on the method of hyperspectral and image deep features for bacon classification. In Chinese control and decision conference 2019 (pp. 4682-6). IEEE.
[Crossref] [Google Scholar]
[61]Weng S, Tang P, Yuan H, Guo B, Yu S, Huang L, Xu C. Hyperspectral imaging for accurate determination of rice variety using a deep learning network with multi-feature fusion. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy. 2020; 234:118237.
[Google Scholar]
[62]Fabiyi SD, Vu H, Tachtatzis C, Murray P, Harle D, Dao TK, et al. Varietal classification of rice seeds using RGB and hyperspectral images. IEEE Access. 2020; 8:22493-505.
[Crossref] [Google Scholar]
[63]Zhang C, Zhao Y, Yan T, Bai X, Xiao Q, Gao P, et al. Application of near-infrared hyperspectral imaging for variety identification of coated maize kernels with deep learning. Infrared Physics & Technology. 2020; 111:103550.
[Google Scholar]
[64]Furlanetto RH, Moriwaki T, Falcioni R, Pattaro M, Vollmann A, Junior AC, et al. Hyperspectral reflectance imaging to classify lettuce varieties by optimum selected wavelengths and linear discriminant analysis. Remote Sensing Applications: Society and Environment. 2020; 20:100400.
[Google Scholar]
[65]Kononenko I, Robnik-Sikonja M, Pompe U. ReliefF for estimation and discretization of attributes in classification, regression, and ILP problems. Artificial Intelligence: Methodology, Systems, Applications. 1996:31-40.
[Google Scholar]
[66]Kira K, Rendell LA. A practical approach to feature selection. In machine learning proceedings 1992 (pp. 249-56). Morgan Kaufmann.
[Google Scholar]
[67]Qian Y, Ye M, Zhou J. Hyperspectral image classification based on structured sparse logistic regression and three-dimensional wavelet texture features. IEEE Transactions on Geoscience and Remote Sensing. 2012; 51(4):2276-91.
[Crossref] [Google Scholar]
[68]Yang Y, Liu X. A re-examination of text categorization methods. In proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval 1999 (pp. 42-9).
[Crossref] [Google Scholar]
[69]Zahavi A, Palshin A, Liyanage DC, Tamre M. Influence of illumination sources on hyperspectral imaging. In international conference on research and education in mechatronics 2019 (pp. 1-5). IEEE.
[Crossref] [Google Scholar]
[70]Ben-Hur A, Weston J. A user s guide to support vector machines. In data mining techniques for the life sciences 2010 (pp. 223-39). Humana Press.
[Crossref] [Google Scholar]