An efficient geosciences workflow on multi-core processors and GPUs: a case study for aerosol optical depth retrieval from MODIS satellite data
Journal article
Liu, Jia, Feld, Dustin, Xue, Yong, Garcke, Jochen, Soddemann, Thomas, Pan, Peiyuan and Fraunhofer Institute of Algorhithms and Scientific Computing 2016. An efficient geosciences workflow on multi-core processors and GPUs: a case study for aerosol optical depth retrieval from MODIS satellite data. International Journal of Digital Earth. https://doi.org/10.1080/17538947.2015.1130087
| Authors | Liu, Jia, Feld, Dustin, Xue, Yong, Garcke, Jochen, Soddemann, Thomas, Pan, Peiyuan and Fraunhofer Institute of Algorhithms and Scientific Computing |
|---|---|
| Abstract | Quantitative remote sensing retrieval algorithms help understanding the dynamic aspects of Digital Earth. However, the Big Data and complex models in Digital Earth pose grand challenges for computation infrastructures. In this article, taking the aerosol optical depth (AOD) retrieval as a study case, we exploit parallel computing methods for high efficient geophysical parameter retrieval. We present an efficient geocomputation workflow for the AOD calculation from the Moderate Resolution Imaging Spectroradiometer (MODIS) satellite data. According to their individual potential for parallelization, several procedures were adapted and implemented for a successful parallel execution on multi-core processors and Graphics Processing Units (GPUs). The benchmarks in this paper validate the high parallel performance of the retrieval workflow with speedups of up to 5.x on a multi-core processor with 8 threads and 43.x on a GPU. To specifically address the time-consuming model retrieval part, hybrid parallel patterns which combine the multi-core processor’s and the GPU’s compute power were implemented with static and dynamic workload distributions and evaluated on two systems with different CPU–GPU configurations. It is shown that only the dynamic hybrid implementation leads to a greatly enhanced overall exploitation of the heterogeneous hardware environment in varying circumstances. |
| Keywords | Digital earth; High performance computing; Graphics processing units (GPU); Multi-core; Hybrid parallel pattern; Aerosol optical depth; Retrieval workflow |
| Year | 2016 |
| Journal | International Journal of Digital Earth |
| Publisher | Taylor & Francis |
| ISSN | 17538947 |
| 17538955 | |
| Digital Object Identifier (DOI) | https://doi.org/10.1080/17538947.2015.1130087 |
| Web address (URL) | http://hdl.handle.net/10545/621605 |
| hdl:10545/621605 | |
| Publication dates | 25 Feb 2016 |
| Publication process dates | |
| Deposited | 10 May 2017, 15:06 |
| Rights | Archived with thanks to International Journal of Digital Earth |
| Contributors | Chinese Academy of Sciences, University of Cologne, London Metropolitan University and University of Bonn |
| File | File Access Level Open |
Permalink -
https://repository.derby.ac.uk/item/9400q/an-efficient-geosciences-workflow-on-multi-core-processors-and-gpus-a-case-study-for-aerosol-optical-depth-retrieval-from-modis-satellite-data
Download files
33
total views0
total downloads1
views this month0
downloads this month
Export as
Related outputs
An Improved Geographically and Temporally Weighted Regression for Surface Ozone Estimation from Satellite-Based Precursor Data
Wang, X., Xue, Y., Sun, Y., Jin, C. and Wu, S. 2023. An Improved Geographically and Temporally Weighted Regression for Surface Ozone Estimation from Satellite-Based Precursor Data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. pp. 1-14. https://doi.org/10.1109/JSTARS.2023.3327881Studying the Regional Transmission of Air Pollution Based on Spatiotemporal Multivariable Data
Lu, X., Xue, Y., He, B., Jiang, X., Wu, S. and Wang, X. 2023. Studying the Regional Transmission of Air Pollution Based on Spatiotemporal Multivariable Data. Atmosphere. 14 (9), pp. 1-17. https://doi.org/10.3390/atmos14091438
Improved Bi-Angle Aerosol Optical Depth Retrieval Algorithm from AHI Data Based on Particle Swarm Optimization
Jin, Chunlin, Jiang, Xingxing, Sun, Yuxin, Wu, Shuhui and Xue, Yong 2021. Improved Bi-Angle Aerosol Optical Depth Retrieval Algorithm from AHI Data Based on Particle Swarm Optimization. Remote Sensing. 13 (22), p. 4689. https://doi.org/10.3390/rs13224689Estimation of the PM2.5 and PM10 Mass Concentration over Land from FY-4A Aerosol Optical Depth Data
Xue, Yong 2021. Estimation of the PM2.5 and PM10 Mass Concentration over Land from FY-4A Aerosol Optical Depth Data. Remote Sensing. 13 (21), p. 4276. https://doi.org/10.3390/rs13214276COVID-19 pandemic decision support system for a population defense strategy and vaccination effectiveness
Varotsos, Costas A, Krapivin, Vladimir F, Xue, Yong, Soldatov, Vladimir and Voronova, Tatiana 2021. COVID-19 pandemic decision support system for a population defense strategy and vaccination effectiveness. Safety Science. 142, p. 105370. https://doi.org/10.1016/j.ssci.2021.105370Nowcasting of air pollution episodes in megacities: A case study for Athens, Greece
Varotsos, Costas A., Mazei, Yuri, Saldaev, Damir, Efstathiou, Maria, Voronova, Tatiana and Xue, Yong 2021. Nowcasting of air pollution episodes in megacities: A case study for Athens, Greece. Atmospheric Pollution Research. 12 (7), p. 101099. https://doi.org/10.1016/j.apr.2021.101099Remote sensing evaluation of total suspended solids dynamic with markov model: a case study of inland reservoir across administrative boundary in south China
Zhao, Jing, Zhang, Fujie, Chen, Shuisen, Wang, Chongyang, Chen, Jinyue, Zhou, Hui and Xue, Yong 2020. Remote sensing evaluation of total suspended solids dynamic with markov model: a case study of inland reservoir across administrative boundary in south China. Sensors. 20 (23), p. 6911. https://doi.org/10.3390/s20236911Diagnostic model for the society safety under COVID-19 pandemic conditions
Varotsos, Costas A., Krapivin, Vladimir F. and Xue, Yong 2021. Diagnostic model for the society safety under COVID-19 pandemic conditions. Safety Science. 136, p. 105164. https://doi.org/10.1016/j.ssci.2021.105164Polarimetric SAR image semantic segmentation with 3D discrete wavelet transform and Markov random field
Bi, Haixia, Xu, Lin, Cao, Xiangyong, Xue, Yong and Xu, Zongben 2020. Polarimetric SAR image semantic segmentation with 3D discrete wavelet transform and Markov random field. IEEE Transactions on Image Processing. https://doi.org/10.1109/TIP.2020.2992177Deriving a global and hourly data set of aerosol optical depth over land using data from four geostationary satellites: goes-16, msg-1, msg-4, and himawari-8
Xie, Yanqing, Xue, Yong, Guang, Jie, Mei, Linlu, She, Lu, Li, Ying, Che, Yahui and Fan, Cheng 2019. Deriving a global and hourly data set of aerosol optical depth over land using data from four geostationary satellites: goes-16, msg-1, msg-4, and himawari-8. IEEE Transactions on Geoscience and Remote Sensing. https://doi.org/10.1109/TGRS.2019.2944949An active deep learning approach for minimally supervised polsar image classification
Xue, Yong 2019. An active deep learning approach for minimally supervised polsar image classification. IEEE Transactions on Geoscience and Remote Sensing. 57 (11), pp. 9378-9395. https://doi.org/10.1109/TGRS.2019.2926434A Heterogeneous and Interactive Big Earth Data Framework
Bi, H., Xue, Y., Merritt, P., Windmill, C. and Davis, B. 2019. A Heterogeneous and Interactive Big Earth Data Framework. 2019 International Conference on Big Data Engineering. IEEE. https://doi.org/10.1145/3341620.3341628Big earth data: a comprehensive analysis of visualization analytics issues
Merritt, Patrick, Bi, Haixia, Davis, Bradley, Windmill, Christopher and Xue, Yong 2019. Big earth data: a comprehensive analysis of visualization analytics issues. Big Earth Data. 2 (4), pp. 321-350. https://doi.org/10.1080/20964471.2019.1576260High Performance Time Series Quantitative Retrieval from Satellite Images on a GPU Cluster
Liu, J., Xue, Y., Ren, K., Song, J., Windmill, C. and Merritt, P. 2019. High Performance Time Series Quantitative Retrieval from Satellite Images on a GPU Cluster. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 12 (8). https://doi.org/10.1109/JSTARS.2019.2920077Joint retrieval of aerosol optical depth and surface reflectance over land using geostationary satellite data.
She, Lu, Xue, Yong, Yang, Xihua, Leys, John, Guang, Jie, Che, Yahui, Fan, Cheng, Xie, Yanqing and Li, Ying 2018. Joint retrieval of aerosol optical depth and surface reflectance over land using geostationary satellite data. IEEE Transactions on Geoscience and Remote Sensing. https://doi.org/10.1109/TGRS.2018.2867000Evaluation of the AVHRR DeepBlue aerosol optical depth dataset over mainland China.
Che, Yahui, Xue, Yong, Guang, Jie, She, Lu and Guo, Jianping 2018. Evaluation of the AVHRR DeepBlue aerosol optical depth dataset over mainland China. ISPRS Journal of Photogrammetry and Remote Sensing. https://doi.org/10.1016/j.isprsjprs.2018.09.004A physically based PM 2.5 estimation method using AERONET data in Beijing Area
Chen, Guili, Guang, Jie, Xue, Yong, Li, Ying, Che, Yahui and Gong, Shaoqi 2018. A physically based PM 2.5 estimation method using AERONET data in Beijing Area. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. https://doi.org/10.1109/JSTARS.2018.2817243Dust detection and intensity estimation using Himawari-8/AHI observation.
She, Lu, Xue, Yong, Yang, Xihua, Guang, Jie, Li, Ying, Che, Yahui, Fan, Cheng and Xie, Yanqing 2018. Dust detection and intensity estimation using Himawari-8/AHI observation. Remote Sensing. https://doi.org/10.3390/rs10040490SAHARA: A Simplified AtmospHeric Correction AlgoRithm for Chinese gAofen Data: 1. Aerosol Algorithm.
She, Lu, Mei, Linlu, Xue, Yong, Che, Yahui and Guang, Jie 2017. SAHARA: A Simplified AtmospHeric Correction AlgoRithm for Chinese gAofen Data: 1. Aerosol Algorithm. Remote Sensing. https://doi.org/10.3390/rs9030253Validation of aerosol products from AATSR and MERIS/AATSR synergy algorithms—Part 1: Global Evaluation.
Che, Yahui, Mei, Linlu, Xue, Yong, Guang, Jie, She, Lu and Li, Ying 2018. Validation of aerosol products from AATSR and MERIS/AATSR synergy algorithms—Part 1: Global Evaluation. Remote Sensing. https://doi.org/10.3390/rs10091414Improved aerosol optical depth and ångstrom exponent retrieval over land From MODIS based on the non-lambertian forward model
Leiku, Yang, Xue, Yong, Guang, Jie, Hassan, Kazemian, Zhang, Jiahua and Li, Chi 2014. Improved aerosol optical depth and ångstrom exponent retrieval over land From MODIS based on the non-lambertian forward model. IEEE Geoscience and Remote Sensing Letters. https://doi.org/10.1109/LGRS.2014.2303317Using SeaWiFS measurements to evaluate radiometric stability of pseudo-invariant calibration sites at top of atmosphere
Li, Chi, Xue, Yong, Liu, Quanhua, Ouazzane, Karim and Zhang, Jiahua 2014. Using SeaWiFS measurements to evaluate radiometric stability of pseudo-invariant calibration sites at top of atmosphere. IEEE Geoscience and Remote Sensing Letters. https://doi.org/10.1109/LGRS.2014.2329138Ensemble of ESA/AATSR aerosol optical depth products based on the likelihood estimate method with uncertainties
Xie, Yanqing, Xue, Yong, Che, Yahui, Guang, Jie, Mei, Linlu, Voorhis, Dave, Fan, Cheng, She, Lu and Xu, Hui 2017. Ensemble of ESA/AATSR aerosol optical depth products based on the likelihood estimate method with uncertainties. IEEE Transactions on Geoscience and Remote Sensing. https://doi.org/10.1109/TGRS.2017.2757910Long-time series aerosol optical depth retrieval from AVHRR data over land in North China and Central Europe
Xue, Yong, He, Xingwei, de Leeuw, Gerrit, Mei, Linlu, Che, Yahui, Rippin, Wayne, Guang, Jie and Hu, Yincui 2017. Long-time series aerosol optical depth retrieval from AVHRR data over land in North China and Central Europe. Remote Sensing of Environment. https://doi.org/10.1016/j.rse.2017.06.036Multicore processors and graphics processing unit accelerators for parallel retrieval of aerosol optical depth from satellite data: Implementation, performance, and energy efficiency
Liu, Jia, Feld, Dustin, Xue, Yong, Garcke, Jochen and Soddemann, Thomas 2015. Multicore processors and graphics processing unit accelerators for parallel retrieval of aerosol optical depth from satellite data: Implementation, performance, and energy efficiency. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. https://doi.org/10.1109/JSTARS.2015.2438893