Evaluation of Ocean Forecasting in the East China Sea

Abstract

The accuracy of the initial condition of a global ocean forecasting system and its prediction skill was evaluated against in situ temperature, salinity and satellite salinity observations during the winter of 2015 and the summer of 2016 for the East China Sea. The ocean forecasting system demonstrates better skill for the Yangtze River estuary and the East China Sea during winter time than during summer time. During winter time, the root-mean-square error (RMSE) of the initial fields of the system for salinity is 1.90 psu, and the correlation is 0.56. The model has a salty bias of 0.29 psu. The salinity RMSE reduces with increasing distance from the coast. In contrast, the RMSE for temperature is 0.76°C, and the correlation is as high as 0.95. There is no bias between model temperature and observation. During summer time, the accuracy and forecast skill of the global ocean forecasting system are very poor. The RMSE for salinity is 3.14 psu, and the correlation is 0.28. The model has a salty bias of 0.95 psu. The RMSE for temperature is 7.22°C, and the model has a warm bias as high as 5.52°C.

1. Introduction

Recent two decades have seen a rapid development in ocean forecasting systems for various regions of the world ocean due to the development in ocean observing systems, ocean models, data assimilation algorithms and computing hardware and software. Currently, numerous institutions provide ocean forecasting operationally (e.g., [1, 2, 3, 4, 5, 6]). The skill of these forecasting systems may differ for various regions of the ocean because of the model bathymetry, atmospheric forcing fields, and availability of data that can be used in data assimilation process. Given the potential application of these forecasting systems, the evaluation of the forecasting skill of these forecasting systems is a necessary step.

The East China Sea, especially the region of Yangtze River estuary and the coastal region of Jiangsu and Zhejiang provinces, is a region of great economic interest since the region has the largest harbor of the world besides many other economic activities. This region is also influenced heavily by freshwater runoff from the Yangtze River, which has an annual averaged discharge around 30,000–40,000 m³/s and is the fourth largest river of the world. Given the economic and scientific interest of the region, we evaluated the accuracy of the initial fields of a global ocean forecasting system in the region against in situ temperature, salinity measurements, and satellite sea surface salinity measurements.

The global ocean forecasting system is the one used operationally by U.S. Navy. The system is configured from the HYbrid Coordinate Ocean Model (HYCOM, e.g., [5, 7, 8, 9]). Besides many other applications, HYCOM was also used in the Chinese coastal oceans, for example, the South China Sea (e.g., [10, 11, 12]), the Yellow Sea [13], the Yellow Sea and the Kuroshio region [14], and the East China Sea [15, 16]. Based on the comparison between monthly SST with output from a circulation model configured from the HYCOM, Lu et al. [10] identified the spatial difference in the quality of simulation. In general, the model performs well in the deep sea region and tends to have large errors in the shallow regions because of the error in heat flux. The conclusion from Lu et al. [10] is further verified in [11], which indicates that the root mean square error in SST simulation in the South China Sea is generally less than 1°C and less than 0.5°C in the middle of the ocean basin of the South China Sea. However, the root mean square error in the coastal region is larger than 1°C. Comparison of ocean models based on HYCOM, POM, and ROMS in the South China Sea region does indicate better performance of HYCOM in the thermocline and in the regions of steep bathymetry. There is a large difference in the simulation of temperature and salinity field in the Yellow Sea [13]. For temperature field, the accuracy of HYCOM reanalysis is greatly improved after using observations in data assimilation. For salinity field, there is not much improvement between the reanalysis and analysis field.

The global ocean forecasting system used in the study has a horizontal resolution of 0.08° and 32 hybrid vertical coordinate surfaces. The data assimilation algorithm can assimilate various types of data, sea surface height, sea surface temperature from satellites, in situ data from CTD, XBT, Argo floats and gliders, and many other types of data [17]. The atmospheric forcing fields are from the NAVy Global Environmental Model (NAVGEM), with 3-h temporal interval. The river discharge includes climate discharge of 986 rivers in various regions of the world [5]. The forecasting fields are available to the public with a delay less than 24 h and can be used for societal application, such as oil platform operation, ship routing planning, and so on. However, there is not much work to evaluate the performance of the global ocean forecasting system in the Chinese coastal ocean. The present research evaluates the performance of the global ocean forecasting system in the East China Sea region using in situ observation and satellite sea surface salinity observation. The in situ CTD measurements used to evaluate the global forecasting system are from two field campaigns: one in the winter of 2015 and one in the summer of 2016. Thus, our comparison also focuses on these two periods.

This chapter is organized as follows. After Section 1, the details of the global forecasting system are presented in Section 2, so are the in situ campaigns, data and processing details. Section 3 evaluates the initial fields and forecast skill of the global ocean forecasting system for the winter of 2015 and the summer of 2016. Our findings are summarized in Section 4.

2. HYCOM global ocean forecasting system and observations

2.1. HYCOM global ocean forecasting system

The HYbrid Coordinate Ocean Model is based on a density coordinate ocean model developed at University of Miami [7]. The vertical coordinate of HYCOM can be switched between Z coordinate, density coordinate and $\mathrm{\sigma }$ coordinate. The Z-coordinate can be used in the upper ocean mixed layer where stratification is weak or even unstable. The $\mathrm{\sigma }$-coordinate can be used for shallow coastal regions and regions where bathymetry varies significantly. The density coordinate can be used in the open ocean where stratification is strong. Compared to the ocean modeling systems that only use single vertical coordinate, HYCOM provides the flexibility to handle different situations [8, 9, 18]. For instance, it was also demonstrated that HYCOM performs better to simulate variability of thermocline in the South China Sea compared with POM and ROMS [12].

Based on HYCOM, the U.S. Navy has developed a global ocean forecasting system, with horizontal resolution of 0.08°. The system is nearly global, with a domain of 78.64 ${}^{°}$ S-66 ${}^{°}$ N, 180 ${}^{°}$ W-180 ${}^{°}$ E, and provides ocean forecasting for 1 week. The atmospheric forcing fields are from NAVy Global Environmental Model (NAVGEM), with 3-h temporal interval. The river discharge is included through climate river discharge of 986 rivers [5].

Using a 3D multivariate variational data assimilation scheme, the system can assimilate various ocean observations, such as sea surface height, sea surface temperature, sea surface salinity from satellite missions, in situ temperature and salinity from CTD, XBT, Argo floats and gliders, and vector current observations [17].

In our research, we compared the initial fields of the system at 0 GMT and its forecasting fields for 7 days also at 0 GMT. The output of the global forecasting system is already interpolated to Z coordinate. We evaluated the initial fields and forecasting at the following 19 levels when measurements and model fields are available, 0, 2, 4, 8, 10, 12, 15, 20, 25, 30, 35, 40, 45, 50, 60, 70, 80, 90 m.

2.2. Observations

The temperature and salinity measurements are from ship “Rongjiang No1”. The field campaigns include a winter one from December 20 to 30, 2015, and a summer one from August 3 to 13, 2016. The sea-bird SBE plus CTD was used for all the station measurements. For each station, the CTD instrument was put in the water for 3–5 min, and descends with speed of 1 m/s. The procedure was repeated 1–2 times to get measurements. In our analysis, we found there are some spurious measurements and our quality control procedure includes two steps. The first step is to remove all the data that have pressure less than 0. The second step is to remove all data that are outside the 3 standard deviation of the mean. The CTD data were interpolated to 19 model levels for comparison. The initial and forecasting fields were interpolated to the location of CTD stations.

The sea surface salinity observations from NASA Soil Moisture Active and Passive Mission (SMAP, [19]) are also used to evaluate the global ocean forecasting system in the East China Sea. A level-3 gridded product with resolution of 0.25° is used in our evaluation. These data are downloaded from PODAAC of Jet Propulsion Laboratory of NASA (podaac-ftp.jpl.nasa.gov).

3. Evaluating ocean forecasting fields

3.1. Initial condition

3.1.1. Winter time

For the estuary region, salinity plays a major role in determining the density. Figure 1a shows the root-mean-square error (RMSE) for all the CTD stations in the 2015 winter campaign. The RMSE is computed for all the available observations for entire water column at a particular station. There are large salinity RMSE regions at the Yangtze River estuary and coastal regions. The maximum salinity RMSE can reach 3.94 psu. The salinity RMSE reduces with increasing distance from the coast. For the sea surface salinity difference (Figure 1b), observations minus model output, most of the stations are negative, indicating model tends to overestimate sea surface salinity. However, right at the Yangtze River estuary, the salinity difference is positive. Thus, the error of the global ocean forecasting system has complex spatial distributions.

Figure 1.

Comparison of temperature and salinity from a global ocean forecasting system configured from HYCOM and CTD observations conducted from December 20 to 30 in the winter of 2015, (a) root-mean-square error of salinity, (b) salinity difference (observation—model) at the surface, (c) root-mean-square error of temperature, and (d) temperature difference (observation—model) at the surface. The unit for salinity is psu and the unit for temperature is ${}^{°}\mathrm{C}.$

The RMSE for temperatures is generally less than 1 $°\mathrm{C}$ except a few stations with RMSE greater than 1 $°\mathrm{C}$. The maximum temperature RMSE is greater than 1.7 $°\mathrm{C}$ (Figure 1c). The sea surface temperature difference, observation minus model, is generally positive. The temperature difference at the Yangtze River estuary is negative, different from the situations in other regions.

The gridded SMAP data are also used to evaluate the initial condition of the global ocean forecasting system. Figure 2 presents sea surface salinity from SMAP (Figure 2a), CTD stations (Figure 2b), and global ocean forecasting system (Figure 2c). During winter time, freshwater from the Yangtze River estuary tends to spread southward, hugging the coastal line (Figure 2b). The SMAP gridded product does not have spatial resolution to resolve this feature. The global ocean forecasting system seems to have issues to spread freshwater southward, which may explain the CTD and sea surface salinity difference is positive at Yangtze River estuary and negative in other places, because the freshwater from the Yangtze River is not spread efficiently.

Figure 2.

Comparison of sea surface salinity from (a) SMAP, (b) CTD observation, and (c) a global ocean forecasting system. The unit for salinity is psu and the unit for temperature is ${}^{°}\mathrm{C}$.

Figure 3 compares the initial condition of global forecasting system more qualitatively in terms of scatting plot and T-S plot. Overall, during the winter of 2015, the global ocean forecasting system agrees better with CTD observation in terms of temperature field (Figure 3a). The RMSE of temperature is 0.76 $°\mathrm{C}$, with a bias close to 0°C (−0.04 $°\mathrm{C}$) . The correlation coefficient of temperature is 0.95. However, the RMSE of salinity is 1.9 psu, with a bias of −0.29 psu. The global forecasting ocean system tends to overestimate salinity at this estuary region. The correlation coefficient between salinity from the global forecasting system and CTD station is only 0.56.

Figure 3.

The scatting plot for (a) temperature, (b) salinity; distribution of root-mean-square error with depth for (c) temperature and (d) salinity, (e) the T-S relationship for observation (red dot) and model (blue dot). All the comparisons are for the observation made from December 20 to 30 in the winter of 2015. The unit for salinity is psu and the unit for temperature is ${}^{°}\mathrm{C}$.

The RMSE of both temperature and salinity decreases with depth (Figure 3c, d). For water depth shallower than 35 m, the RMSE of temperature is generally around 0.7–0.8 $°\mathrm{C}$, and reduces to less than 0.6 $°\mathrm{C}$ when the water depth is deeper than 35 m. The RMSE of salinity decreases greatly from 2.5 psu at the surface to around 0.2 psu, when the depth is deeper than 80 m (Figure 3d). The water mass T-S property differs greatly between the global forecasting system and CTD observations (Figure 3e). In reality, the surface temperature cools significantly during winter. This cool water mass is associated with salinity around 25–28 psu with temperature around 12–14°C. In the global ocean forecasting system, this water mass is under-represented.

From the average temperature, salinity, and density change with depth, the above feature is more clearly shown (Figure 4). The surface temperature is cooler than temperature at depth during wintertime. The fresh and cool water mass stays on top of the warm and salty water. The global forecasting system cannot represent this feature and maintain a uniform temperature and salinity structure for the entire water column.

Figure 4.

Averaged distribution of (a) temperature, (b) salinity and (c) density for the observation and global ocean forecasting system output from December 20 to 30 in the winter of 2015. The unit for salinity is psu and the unit for temperature is ${}^{°}\mathrm{C}$.

3.1.2. Summer time

Based on historical observations, the path of discharged freshwater from the Yangtze River has distinct seasonal variability. During summer time, the freshwater spreads toward northeast direction. The field campaign in the summer of 2016 made observations along this direction. Compared to the situation of the winter of 2015, the salinity RMSE has much larger values. The maximum RMSE reaches 7.8 psu. The sea surface salinity difference also has a large value, mostly negative, indicating that the global ocean forecasting system overestimates the sea surface salinity by 3–4 psu (Figure 5a, b).

Figure 5.

Comparison of temperature and salinity from a global ocean forecasting system configured from HYCOM and CTD observations conducted from August 3 to 13 in the summer of 2016, (a) root-mean-square error of salinity, (b) salinity difference (observation—model) at the surface, (c) root-mean-square error of temperature, and (d) temperature difference (observation—model) at the surface. The unit for salinity is psu and the unit for temperature is ${}^{°}\mathrm{C}$.

The RMSE of temperature also has large values, with higher RMSE with increasing distance from the coastal region. The sea surface temperature difference is generally between −1^o and 1^o indicating large temperature difference toward deeper depth (Figure 5c, d).

From the two-dimensional sea surface salinity distribution of SMAP (Figure 6a) and CTD stations (Figure 6b), it is clear that the freshwater from the Yangtze River tends to form a low salinity region extending northeastward from the Yangtze River estuary. However, the initial field from the global ocean forecasting system misses the features (Figure 6c).

Figure 6.

Comparison of sea surface salinity from (a) SMAP, (b) CTD observation, and (c) a global ocean forecasting system from August 3 to 13, 2016. The unit for salinity is psu and the unit for temperature is ${}^{°}\mathrm{C}$.

The root-mean-square difference of sea surface salinity between SMAP and CTD is 3.02 psu. The RMSE of sea surface salinity between CTD and the global ocean forecasting system is 6.76 psu, indicating that the SMAP data could provide valuable information around this coastal region.

More quantitative comparisons between temperature from the global ocean forecasting system and CTD observations indicate the difficulty to represent stratification structure during summer time. The RMSE of temperature is 7.22 $°\mathrm{C}$, with the global ocean forecasting system having a warm bias of 5.52 $°\mathrm{C}$. The correlation coefficient between the temperature from the global forecasting system and CTD observation is 0.28 (Figure 7a). For salinity, the RMSE is 3.4 psu; the global forecasting system has a salty bias of 0.95 psu and correlation coefficient for salinity is 0.28 (Figure 7b).

Figure 7.

The scatting plot for (a) temperature, (b) salinity; distribution of root-mean-square error with depth for (c) temperature and (d) salinity, (e) the T-S relationship for observation (red dot) and model (blue dot). All the comparisons are for the observations made from August 3 to 13 in the summer of 2015.

Contrary to the situation in wintertime, the temperature RMSE actually increases with depth, and can reach 14 $°\mathrm{C}$ at 60 m depth, indicating the difficulty to represent summer stratification of the region. The RMSE of salinity decreases from 6 to 7 psu at the surface to less than 1 psu at the depth of 60 m, with large RMSE in top 10 m. The observed T-S relationship indicates a wide spread of warm-fresh and cool-salty characteristic. The global ocean forecasting system, however, has a narrow T-S relationship centered at 24–30 $°\mathrm{C}$ and 26–34 psu.

The stratification structure from the global ocean forecasting system is also very different from that shown in CTD observations. During summer time, the water column here is stratified with a shallow warm and fresh layer at the top and cool and salty water at the bottom. However, the water column in the global forecasting system is mixed well with uniform temperature and salinity from the surface to the bottom of the ocean (Figure 8).

Figure 8.

Averaged distribution of (a) temperature, (b) salinity and (c) density for the observation and global ocean forecasting system output from August 3 to 13 in the summer of 2015.

3.2. Evaluation of forecasting skill

3.2.1. Winter time

The forecasting skill of the global ocean forecasting system is also evaluated using CTD observations. The CTD observations are matched with temperature and salinity prediction with leading time from 1 to 7 days. The forecasting skill is measured by correlation coefficient and root-mean-square error of temperature and salinity (Figure 9). For temperature prediction during winter time, the correlation coefficient decreases from around 0.95 to around 0.85, and the RMSE increases from less than 0.8–1.3 $°\mathrm{C}$ when the leading time increases from 1 to 7 days.

Figure 9.

The root-mean-square error and correlation coefficient for (a) temperature and (b) salinity between CTD observation and the global ocean forecasting system configured from HYCOM with leading time increasing from 1 to 7 days. The comparison is for December 20 to 30 in the winter of 2015.

Overall, the situation reflects the general tendency that when the leading time increases, the prediction skill decreases. For the prediction of salinity, neither RMSE nor correlation coefficient changes significantly with the increase of leading time (Figure 9b) indicating that the forecasting skill for salinity in this region is limited.

3.2.2. Summer time

During the summer of 2016, the RMSE of temperature prediction varies from 7.2 to 8 $°\mathrm{C}$ and the correlation coefficient varies from 0.26 to 0.29 when the leading time increases from 1 to 7 days. The RMSE of salinity prediction varies from 3.1 to 3.2 psu and the correlation coefficient varies around 0.3, when the leading time increases from 1 to 7 days (Figure 10). The comparison indicates relatively pool prediction skill for the region during summer time for both temperature and salinity.

Figure 10.

The root-mean-square error and correlation coefficient for (a) temperature and (b) salinity between CTD observation and the global ocean forecasting system configured from HYCOM with leading time increasing from 1 to 7 days. The comparison is from August 3 to 10 in the summer of 2016.

4. Summary

Using CTD and SMAP observations, our research evaluates the initial condition and forecasting skill of a global ocean forecasting system configured from HYCOM during the winter of 2015 and the summer of 2016 for the East China Sea. During winter time, the temperature of the global ocean forecasting system agrees with observation reasonably well. During summer time, the temperature field from the global ocean forecasting system has large errors and the error tends to increase with depth. For salinity field, the salinity field agrees better with observation during winter time than that during summer time. The global forecasting system seems to have issues to represent the northeastward spreading of the freshwater discharged from the Yangtze River.

In terms of forecasting skill, the global ocean forecasting system shows some skill in predicting temperature during winter time with a lead time of 7 days, with RMSE increasing from 0.8 to 1.3 $°\mathrm{C}$. For salinity prediction, the RMSE is around 1.9 psu during winter time. The RMSE of temperature prediction increases from 7 to 8 $°\mathrm{C}$, and the RMSE for salinity increases from 1.9 psu to 3–3.5 psu during summer. Overall, there is a large space for improvement in terms of temperature and salinity prediction in the East China Sea around the Yangtze River estuary. Evaluation of the ocean forecasting system will be useful for its applications in the East China Sea region and provides guidelines for the development of future ocean forecasting systems.

Acknowledgments

The work is supported by 2015 Program for Innovation Research and Entrepreneurship Team in Jiangsu Province, the Ocean Public Welfare Scientific Research Project, State Oceanic Administration of the People’s Republic of China (201505003-05), the National Key Research and Development Program of China (2016YFC1401600) and the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD).