In view of the challenges faced by organizations and departments concerned with agricultural capacity observations, we collected In-Situ data consisting of diverse crops (More than 11 consumable vegetation types) in our pilot region of Harichand Charsadda, Khyber Pakhtunkhwa (KP), Pakistan. Our proposed Long Short-Term Memory based Deep Neural network model was trained for land cover land use statistics generation using the acquired ground truth data, for a synergy between Planet-Scope Dove and European Space Agency's Sentinel-2. Total of 4 bands from both sentinel-2 and planet scope including Red, Green, Near-Infrared (NIR) and Normalised Difference Vegetation Index (NDVI) were used for classification purpose. Using short temporal frame of Sentinel-2 comprising 5 date images, we propose an realistic and implementable procedure for generating accurate crop statistics using remote sensing. Our self collected data-set consists of a total number of 107,899 pixels which was further split into 70% and 30% for training and testing purpose of the model respectively. The collected data is in the shape of field parcels, which has been further split for training, validation and test sets, to avoid spatial auto-correlation. To ensure the quality and accuracy 15% of the training data was left out for validation purpose, and 15% for testing. Prediction was also performed on our trained model and visual analysis of the area from the image showed significant results. Further more a comparison between Sentinel-2 time series is performed separately from the fused Planet-Scope and Sentinel-2 time-series data sets. The results achieved shows a weighted average of 93% for Sentinel-2 time series and 97% for fused Planet-Scope and Sentinel-2 time series.

Department of Agricultural Engineering Peshawar University of Engineering and Technology Peshawar Khyber Pakhtoonkhwa Pakistan

Department of Computer Systems Engineering University of Engineering and Technology Peshawar Khyber Pakhtoonkhwa Pakistan

Department of Quantitative Methods and Economic Informatics Faculty of Operation and Economics of Transport and Communication University of Zilina Zilina Slovakia

Department of Telecommunications Faculty of Electrical Engineering and Computer Science VŠB Technical University of Ostrava Ostrava Czechia

National Center for Big Data and Cloud Computing Pakistan

Zobrazit více v PubMed

Chandio AA, Magsi H, Rehman A, Sahito JGM. Types, sources and importance of agricultural credits in Pakistan. Journal of Applied Environmental and Biological Sciences. 2017;7(3):144–149.

Liaqat MU, Cheema MJM, Huang W, Mahmood T, Zaman M, Khan MM. Evaluation of MODIS and Landsat multiband vegetation indices used for wheat yield estimation in irrigated Indus Basin. Computers and Electronics in Agriculture. 2017;138:39–47. doi: 10.1016/j.compag.2017.04.006 DOI

See L, Fritz S, You L, Ramankutty N, Herrero M, Justice C, et al.. Improved global cropland data as an essential ingredient for food security. Global Food Security. 2015;4:37–45. doi: 10.1016/j.gfs.2014.10.004 DOI

Wójtowicz M, Wójtowicz A, Piekarczyk J, et al.. Application of remote sensing methods in agriculture. Communications in Biometry and Crop Science. 2016;11(1):31–50.

Atzberger C. Advances in remote sensing of agriculture: Context description, existing operational monitoring systems and major information needs. Remote sensing. 2013;5(2):949–981. doi: 10.3390/rs5020949 DOI

You N, Dong J. Examining earliest identifiable timing of crops using all available Sentinel 1/2 imagery and Google Earth Engine. ISPRS Journal of Photogrammetry and Remote Sensing. 2020;161:109–123. doi: 10.1016/j.isprsjprs.2020.01.001 DOI

Duro DC, Franklin SE, Dubé MG. A comparison of pixel-based and object-based image analysis with selected machine learning algorithms for the classification of agricultural landscapes using SPOT-5 HRG imagery. Remote sensing of environment. 2012;118:259–272. doi: 10.1016/j.rse.2011.11.020 DOI

Vuolo F, Neuwirth M, Immitzer M, Atzberger C, Ng WT. How much does multi-temporal Sentinel-2 data improve crop type classification? International journal of applied earth observation and geoinformation. 2018;72:122–130. doi: 10.1016/j.jag.2018.06.007 DOI

Mazzia V, Khaliq A, Chiaberge M. Improvement in land cover and crop classification based on temporal features learning from Sentinel-2 data using recurrent-convolutional neural network (R-CNN). Applied Sciences. 2020;10(1):238. doi: 10.3390/app10010238 DOI

Yi Z, Jia L, Chen Q. Crop classification using multi-temporal Sentinel-2 data in the Shiyang River Basin of China. Remote Sensing. 2020;12(24):4052. doi: 10.3390/rs12244052 DOI

Camps-Valls G. Machine learning in remote sensing data processing. In: 2009 IEEE international workshop on machine learning for signal processing. IEEE; 2009. p. 1–6.

Lary DJ, Alavi AH, Gandomi AH, Walker AL. Machine learning in geosciences and remote sensing. Geoscience Frontiers. 2016;7(1):3–10. doi: 10.1016/j.gsf.2015.07.003 DOI

Minallah N, Khan W. Comparison of neural networks and support vector machines for the mass balance ablation observation of glaciers in Baltoro region. Journal of Information Communication Technologies and Robotic Applications. 2019; p. 37–45.

Zhu XX, Tuia D, Mou L, Xia GS, Zhang L, Xu F, et al.. Deep learning in remote sensing: A comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine. 2017;5(4):8–36. doi: 10.1109/MGRS.2017.2762307 DOI

Minallah N, Tariq M, Aziz N, Khan W, Rehman Au, Belhaouari SB. On the performance of fusion based planet-scope and Sentinel-2 data for crop classification using inception inspired deep convolutional neural network. Plos one. 2020;15(9):e0239746. doi: 10.1371/journal.pone.0239746 PubMed DOI PMC

Kussul N, Lavreniuk M, Skakun S, Shelestov A. Deep learning classification of land cover and crop types using remote sensing data. IEEE Geoscience and Remote Sensing Letters. 2017;14(5):778–782. doi: 10.1109/LGRS.2017.2681128 DOI

Thai LH, Hai TS, Thuy NT. Image classification using support vector machine and artificial neural network. International Journal of Information Technology and Computer Science. 2012;4(5):32–38. doi: 10.5815/ijitcs.2012.05.05 DOI

Hochreiter S, Schmidhuber J. Long short-term memory. Neural computation. 1997;9(8):1735–1780. doi: 10.1162/neco.1997.9.8.1735 PubMed DOI

Graves A. Long short-term memory. In: Supervised sequence labelling with recurrent neural networks. Springer; 2012. p. 37–45.

Schmidhuber J, Hochreiter S, et al.. Long short-term memory. Neural Comput. 1997;9(8):1735–1780. doi: 10.1162/neco.1997.9.8.1735 PubMed DOI

Medsker LR, Jain L. Recurrent neural networks. Design and Applications. 2001;5:64–67.

Boden M. A guide to recurrent neural networks and backpropagation. the Dallas project. 2002.

Breuel TM, Ul-Hasan A, Al-Azawi MA, Shafait F. High-performance OCR for printed English and Fraktur using LSTM networks. In: 2013 12th international conference on document analysis and recognition. IEEE; 2013. p. 683–687.

Ding Z, Xia R, Yu J, Li X, Yang J. Densely connected bidirectional lstm with applications to sentence classification. In: CCF International Conference on Natural Language Processing and Chinese Computing. Springer; 2018. p. 278–287.

Chen Y, Zhong K, Zhang J, Sun Q, Zhao X, et al. LSTM networks for mobile human activity recognition. In: Proceedings of the 2016 International Conference on Artificial Intelligence: Technologies and Applications, Bangkok, Thailand; 2016. p. 24–25.

Aslam M. Agricultural productivity current scenario, constraints and future prospects in Pakistan. Sarhad Journal of Agriculture. 2016;32(4):289–303. doi: 10.17582/journal.sja/2016.32.4.289.303 DOI

Yan WY, Shaker A, El-Ashmawy N. Urban land cover classification using airborne LiDAR data: A review. Remote Sensing of Environment. 2015;158:295–310. doi: 10.1016/j.rse.2014.11.001 DOI

Khan W, Minallah N, Khan IU, Wadud Z, Zeeshan M, Yousaf S, et al.. On the Performance of Temporal Stacking and Vegetation Indices for Detection and Estimation of Tobacco Crop. IEEE Access. 2020;8:103020–103033. doi: 10.1109/ACCESS.2020.2998079 DOI

Aziz N, Minallah N, Junaid A, Gul K. Performance analysis of artificial neural network based land cover classification. International Journal of Marine and Environmental Sciences. 2017;11(5):422–426.

Drusch M, Del Bello U, Carlier S, Colin O, Fernandez V, Gascon F, et al.. Sentinel-2: ESA’s optical high-resolution mission for GMES operational services. Remote sensing of Environment. 2012;120:25–36. doi: 10.1016/j.rse.2011.11.026 DOI

Cheng MC, Zhang C. Formosat-2 for international societal benefits. Remote Sens. 2016;2016:1–7.

Ji S, Zhang C, Xu A, Shi Y, Duan Y. 3D convolutional neural networks for crop classification with multi-temporal remote sensing images. Remote Sensing. 2018;10(1):75. doi: 10.3390/rs10010075 DOI

Buscombe D, Ritchie AC. Landscape classification with deep neural networks. Geosciences. 2018;8(7):244. doi: 10.3390/geosciences8070244 DOI

Pal M. Random forest classifier for remote sensing classification. International journal of remote sensing. 2005;26(1):217–222. doi: 10.1080/01431160412331269698 DOI

Anderson NT, Marchisio GB. WorldView-2 and the evolution of the DigitalGlobe remote sensing satellite constellation: introductory paper for the special session on WorldView-2. In: Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XVIII. vol. 8390. International Society for Optics and Photonics; 2012. p. 83900L.

Palchowdhuri Y, Valcarce-Diñeiro R, King P, Sanabria-Soto M, et al.. Classification of multi-temporal spectral indices for crop type mapping: a case study in Coalville, UK. J Agric Sci. 2018;156(1):24–36. doi: 10.1017/S0021859617000879 DOI

Pelletier C, Webb GI, Petitjean F. Temporal convolutional neural network for the classification of satellite image time series. Remote Sensing. 2019;11(5):523. doi: 10.3390/rs11050523 DOI

Weiss M, Jacob F, Duveiller G. Remote sensing for agricultural applications: A meta-review. Remote Sensing of Environment. 2020;236:111402. doi: 10.1016/j.rse.2019.111402 DOI

Poli D, Remondino F, Angiuli E, Agugiaro G. Radiometric and geometric evaluation of GeoEye-1, WorldView-2 and Pléiades-1A stereo images for 3D information extraction. ISPRS Journal of Photogrammetry and Remote Sensing. 2015;100:35–47. doi: 10.1016/j.isprsjprs.2014.04.007 DOI

Team P. Planet application program interface: In space for life on Earth. San Francisco, CA. 2017;2017:40.

Sonobe R, Yamaya Y, Tani H, Wang X, Kobayashi N, Mochizuki Ki. Crop classification from Sentinel-2-derived vegetation indices using ensemble learning. Journal of Applied Remote Sensing. 2018;12(2):026019. doi: 10.1117/1.JRS.12.026019 DOI

Sonobe R, Yamaya Y, Tani H, Wang X, Kobayashi N, Mochizuki Ki. Assessing the suitability of data from Sentinel-1A and 2A for crop classification. GIScience & Remote Sensing. 2017;54(6):918–938. doi: 10.1080/15481603.2017.1351149 DOI

Yin H, Pflugmacher D, Li A, Li Z, Hostert P. Land use and land cover change in Inner Mongolia-understanding the effects of China’s re-vegetation programs. Remote Sensing of Environment. 2018;204:918–930. doi: 10.1016/j.rse.2017.08.030 DOI

Chan JCW, Paelinckx D. Evaluation of Random Forest and Adaboost tree-based ensemble classification and spectral band selection for ecotope mapping using airborne hyperspectral imagery. Remote Sensing of Environment. 2008;112(6):2999–3011. doi: 10.1016/j.rse.2008.02.011 DOI

Chan JCW, Beckers P, Spanhove T, Borre JV. An evaluation of ensemble classifiers for mapping Natura 2000 heathland in Belgium using spaceborne angular hyperspectral (CHRIS/Proba) imagery. International Journal of Applied Earth Observation and Geoinformation. 2012;18:13–22. doi: 10.1016/j.jag.2012.01.002 DOI

Pawel H, omiej BB, Wezyk P, Szostak M. Estimating defoliation of Scots pine stands using machine learning methods and vegetation indices of Sentinel-2. European Journal of Remote Sensing. 2018;51(1):194–204. doi: 10.1080/22797254.2017.1417745 DOI

Saini R, Ghosh S. Crop classification on single date sentinel-2 imagery using random forest and suppor vector machine. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences. 2018.

LeCun Y, Bengio Y, Hinton G. Deep learning. nature. 2015;521(7553):436–444. doi: 10.1038/nature14539 PubMed DOI

Pan SJ, Yang Q. A survey on transfer learning. IEEE Transactions on knowledge and data engineering. 2009;22(10):1345–1359. doi: 10.1109/TKDE.2009.191 DOI

Dahl GE, Yu D, Deng L, Acero A. Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Transactions on audio, speech, and language processing. 2011;20(1):30–42. doi: 10.1109/TASL.2011.2134090 DOI

Hugo Larochelle and Yoshua Bengio and Louradour Jéréme and Lamblin Pascalú. Deep learning in agriculture: A survey. Journal of machine learning research. 2009;1:10.

Salakhutdinov R, Hinton G. Deep boltzmann machines. In: Artificial intelligence and statistics. PMLR; 2009. p. 448–455.

Moroney L. The firebase realtime database. In: The Definitive Guide to Firebase. Springer; 2017. p. 51–71.

Addabbo P, Focareta M, Marcuccio S, Votto C, Ullo SL. Contribution of Sentinel-2 data for applications in vegetation monitoring. Acta Imeko. 2016;5:44–54. doi: 10.21014/acta_imeko.v5i2.352 DOI

Gandhi GM, Parthiban B, Thummalu N, Christy A. Ndvi: Vegetation change detection using remote sensing and gis–A case study of Vellore District. Procedia computer science. 2015;57:1199–1210. doi: 10.1016/j.procs.2015.07.415 DOI

Ba JL, Kiros JR, Hinton GE. Layer normalization. arXiv preprint arXiv:160706450. 2016.

Li Y, Yuan Y. Convergence analysis of two-layer neural networks with relu activation. arXiv preprint arXiv:170509886. 2017.

Prechelt L. Early stopping-but when? In: Neural Networks: Tricks of the trade. Springer; 1998. p. 55–69.

Smith LN. Cyclical learning rates for training neural networks. In: 2017 IEEE winter conference on applications of computer vision (WACV). IEEE; 2017. p. 464–472.