收稿日期: 2020-01-03
修回日期: 2020-11-09
网络出版日期: 2021-08-02
基金资助
国家重点研发专项分级质控和多源资料快速融合分析技术(2018YFC1507502);自动土壤水分观测数据质量控制算法研究(AMF201603)
Anomaly data detection method for in situ automatic soil moisture observation
Received date: 2020-01-03
Revised date: 2020-11-09
Online published: 2021-08-02
针对全国自动土壤水分实时小时观测资料,结合仪器标定方法、土壤水文物理常数等因素,研究土壤水分固有变化特征,分析异常数据误差来源和阈值判定,利用频率检测、界限值检测、无降水突变检测、异常峰值检测、恒值检测五类方法对土壤水分观测资料进行质量控制实验和检验。结果表明:(1) 自动土壤水分异常数据主要分为:粗大值、突变、异常峰值和僵值,主要由仪器失灵、土壤水文物理常数测定不准、传感器标校不合理等原因引起。(2) 频率检测可有效检出因仪器故障引起的错误数据。目前该方法已应用于综合气象观测数据质量控制业务系统,用于开展全国实时自动土壤水分小时数据的质量控制和质量评估。
李翠娜,刘天琦,吴东丽 . 自动土壤水分观测资料异常数据检测方法[J]. 干旱区地理, 2021 , 44(4) : 1093 -1103 . DOI: 10.12118/j.issn.1000–6060.2021.04.21
Soil moisture plays a crucial role in the study of agricultural drought monitoring, yield prediction, and soil erosion, which are of great significance for agriculture, drought, and climate studies. Automatic soil moisture observation instruments have become an important component of the automatic soil moisture observation stations run by the meteorological department in China given their high precision, high spatial and temporal resolution, and nondestructive capabilities. The accuracy of data obtained through the process of observing soil moisture is seriously affected by the calibration methods used, the stability of the equipment, and the texture of the soil. Thus, it is especially important to establish a quality control method for automatic soil moisture observation data that are free from these error sources that affect the observation quality. In attempting to solve the outstanding quality problems in automatic soil moisture observation data, this paper studies the inherent variation characteristics of soil moisture on the basis of the data obtained by the automatic soil moisture observation stations in China between 2013 and 2015. Combined with the instrument observation principle and the data characteristics and error sources of abnormal data, this study classifies and statistically analyzes the form of the abnormal data, and given the threshold, it puts forward a preliminary practical set of quality control methods for the hourly soil moisture observation data, which includes frequency detection (FD), threshold detection for volume moisture content and relative humidity, break drop detection, sudden change detection, and constant detection. The application effect of this quality control method has been verified using data obtained in China in 2019. The results show that (1) the four main categories of abnormal data are gross errors, mutations, abnormally high values, and stiffness; these are mainly caused by instrument failure, abnormal soil hydrological constants, and unreasonable calibration of sensors. (2) Based on the FD method, which analyzes the change characteristics through the frequency values measured by the sensors in the air and water, this quality control method can effectively detect the abnormal data caused by instrument failure, with the accuracy rate reaching 100%. (3) The evaluation results from the hourly data of the national automatic soil moisture stations from June to September 2019 show that the accuracy rate of data is 93.8%, the abnormal rate is 1.7%, and the missing rate is 4.5%. The abnormal data are mainly distributed in the Qinghai, Sichuan, Shandong, Hebei, and Guangdong Provinces. The soil surface layer (0-20 cm) has a higher proportion of anomalous data than that of other layers, and only a few stations have an anomaly within the whole observation layer. Although the abnormal phenomena can be observed at most sites across the country, the primary problem is that a small number of sites have long-term abnormalities, which have been caused either by the calibration of the sensor or the inaccurate determination of the soil’s hydrological and physical constants. (4) This quality control method can effectively detect abnormal data in China. Presently, it has been applied to the Integrated Meteorology Observation Data Quality Control System.
[1] | 韩姣姣, 段旭, 赵洋毅. 金沙江干热河谷不同植被坡面土壤水分时空分布特征[J]. 干旱区地理, 2019, 42(1):121-129. |
[1] | [ Han Jiaojiao, Duan Xu, Zhao Yangyi. Spatial and temporal variability of soil moisture on slope land of different vegetation of dry-hot valley in Jinsha River[J]. Arid Land Geography, 2019, 42(1):121-129. ] |
[2] | 蔡亮红, 丁建丽. 基于改进植被指数土壤水分遥感反演[J]. 干旱区地理, 2017, 40(6):1248-1255. |
[2] | [ Cai Lianghong, Ding Jianli. Remote sensing inversion of soil moisture based on modified vegetation index[J]. Arid Land Geography, 2017, 40(6):1248-1255. ] |
[3] | 丁新原, 周智彬, 雷加强, 等. 塔里木沙漠公路防护林土壤水分特征曲线模型分析与比较[J]. 干旱区地理, 2015, 38(5):985-993. |
[3] | [ Ding Xinyuan, Zhou Zhibin, Lei Jiaqiang, et al. Analysis and comparison of models for soil water characteristic curves of Tarim Desert Highway Shelterbelt[J]. Arid Land Geography, 2015, 38(5):985-993. ] |
[4] | 陈丙寅, 杨辽, 陈曦, 等. 基于改进型TVDI在干旱区旱情监测中的应用研究[J]. 干旱区地理, 2019, 42(4):902-913. |
[4] | [ Chen Bingyin, Yang Liao, Chen Xi, et al. Application of modified TVDI in drought monitoring in arid areas[J]. Arid Land Geography, 2019, 42(4):902-913. ] |
[5] | 王新友, 马全林, 靳虎甲, 等. 石羊河下游人工梭梭林土壤呼吸变化特征及其与水热因子的关系[J]. 干旱区地理, 2019, 42(3):570-580. |
[5] | [ Wang Xinyou, Ma Quanlin, Jin Hujia, et al. Soil respiration variation characteristics and its relationship with hydrothermic factor of artificial Haloxylon ammodendron forest in lower reaches of Shiyang River[J]. Arid Land Geography, 2019, 42(3):570-580. ] |
[6] | 王钧, 李广, 聂志刚, 等. 陇中黄土丘陵沟壑区人工草地土壤水蚀预测模型[J]. 干旱区地理, 2020, 43(2):398-405. |
[6] | [ Wang Jun, Li Guang, Nie Zhigang, et al. Testing a soil water erosion predictive model in an artificial grassland in the hill gully Loess Plateau[J]. Arid Land Geography, 2020, 43(2):398-405. ] |
[7] | 张志富. 自动站土壤水分资料质量控制方案的研制[J]. 干旱区地理, 2013, 36(1):101-108. |
[7] | [ Zhang Zhifu. Quality control schemes for hourly soil moisture from automatic weather stations in China[J]. Arid Land Geography, 2013, 36(1):101-108. ] |
[8] | 王晓东, 杨太明, 吴必文, 等. 安徽省自动观测土壤水分质量控制方法[J]. 气象科技, 2015, 43(3):399-404. |
[8] | [ Wang Xiaodong, Yang Taiming, Wu Biwen, et al. Error analysis and quality control method of automatic soil moisture measurements in Anhui[J]. Meteorological Science and Technology, 2015, 43(3):399-404. ] |
[9] | 王良宇, 何延波, 张艳红, 等. 一种自动土壤水分数据质量控制预警方法[J]. 气象科技, 2016, 44(4):528-534. |
[9] | [ Wang Liangyu, He Yanbo, Zhang Yanhong, et al. Data quality early warning of automatic soil water observations based on soil hydrological constants[J]. Meteorological Science and Technology, 2016, 44(4):528-534. ] |
[10] | 吴东丽, 曹婷婷, 薛红喜. 自动土壤水分观测数据质量控制方法及其应用[J]. 土壤科学, 2016, 4(1):1-10. |
[10] | [ Wu Dongli, Cao Tingting, Xue Hongxi. The study of quality control for observing data of automatic soil moisture[J]. Hans Journal of Soil Science, 2016, 4(1):1-10. ] |
[11] | 王佳强, 赵煜飞, 任芝花, 等. 中国自动土壤水分观测资料质量控制方法设计与效果检验[J]. 气象, 2018, 44(2):244-257. |
[11] | [ Wang Jiaqiang, Zhao Yufei, Ren Zhihua, et al. Design and verification of quality control methods for automatic soil moisture observation data in China[J]. Meteorological Monthly, 2018, 44(2):244-257. ] |
[12] | Dorigo W A, Xaver A, Vreugdenhil M, et al. Global automated quality control of in situ soil moisture data from the International Soil Moisture Network[J]. Vadose Zone Journal, 2013, 12(3):918-924. |
[13] | 王良宇, 何延波. 自动土壤水分观测数据异常值值域研究[J]. 气象, 2015, 41(8):1017-1022. |
[13] | [ Wang Liangyu, He Yanbo. Research on outlier threshold of automatic soil moisture observation data[J]. Meteorological Monthly, 2015, 41(8):1017-1022. ] |
[14] | 中国气象局. 自动土壤水分观测规范(试行)[R]. 北京: 中国气象局, 2010. |
[14] | [China Meteorological Administration. Specification for automatic soil moisture observation(trial)[R]. Beijing: China Meteorological Administration, 2010. ] |
[15] | 吴芹芹, 莫淑红, 程圣东, 等. 黄土区冻融期不同土地利用土壤水分与温度的关系[J]. 干旱区研究, 2020, 37(3):627-635. |
[15] | [ Wu Qinqin, Mo Shuhong, Cheng Shengdong, et al. Study on the correlation between soil moisture and temperature of different land uses in the loess area during a freezing-thawing period[J]. Arid Zone Research, 2020, 37(3):627-635. ] |
[16] | 裴步祥, 毛飞. 我国土壤水分的变化规律和土壤湿度测站合理布局的研究[J]. 北京农业大学学报, 1990, 16(增刊1):1-12. |
[16] | [ Pei Buxiang, Mao Fei. Study on soil moisture variation patterns and the rational distribution of soil moisture observation stations of China[J]. Acta Agriculture Universities Pekinensis, 1990, 16(Suppl. 1):1-12. ] |
[17] | 吕明侠, 王一博, 刘国华. 黑河上游高山区浅层土壤水分变化及其影响因素[J]. 干旱区研究, 2020, 37(4):899-907. |
[17] | [ Lyu Mingxia, Wang Yibo, Liu Guohua. Changes in shallow soil moisture and its influencing factors in the alpine region of the upper Heihe River[J]. Arid Zone Research, 2020, 37(4):899-907. ] |
[18] | Zhao Chunlei, Shao Ming’an, Jia Xiaoxu, et al. Spatial distribution of water-active soil layer along the south-north transect in the Loess Plateau of China[J]. Journal of Arid Land, 2019, 11(2):228-240. |
[19] | Gong Yidan, Xing Xuguang, Wang Weihua. Factors determining soil water heterogeneity on the Chinese Loess Plateau as based on an empirical mode decomposition method[J]. Journal of Arid Land, 2020, 12(3):462-472. |
[20] | Basara J B, Crawford T M. Improved installation procedures for deep-layer soil moisture measurements[J]. Journal of Atmospheric Ocean Technology, 2000, 17(6):879-884. |
[21] | You J S, Hubbard K G, Mahmood R, et al. Quality control of soil water data in applied climate information system: Case study in Nebraska[J]. Journal Hydrologic Engineering, 2010(15):200-209. |
[22] | Hubbard K G, Goddard S, Sorensen W D, et al. Performance of quality assurance procedures for an applied climate information system[J]. Journal of Atmospheric and Oceanic Technology, 2005(22):105-112. |
[23] | Gonzálezrouco J F, Jiménez J L, Quesada V, et al. Quality control and homogentity of precipitation data in the douthwest of Europe[J]. Journal of Climate, 2001, 14(14):964-978. |
[24] | Journée M, Bertrand C. Quality control of solar radiation data within the RMIB solar measurements network[J]. Solar Energy, 2011, 85:72-86. |
[25] | 中国气象局. QXT118-2010. 地面气象资料质量控制标准[S]. 北京: 气象出版社, 2010. |
[25] | [China Meteorological Administration. QXT118-2010. Quality control of surface meteorological observational data[S]. Beijing: China Meteorological Press, 2010. ] |
/
〈 |
|
〉 |