ranked prob score a5
type: Argument
is argument of: ranked_prob_score
arg order: 5
arg desc: fcst Start time grid.
arg name: Sgrid
argument type: grid
in out: in
optional: false

ranked_prob_score
type: Ingrid Procedure
has argument: ranked prob score a4
syntax ex: ranked_prob_score computes the ranked probability score for probabilistic forecast fcst relatively to observed categories obs. Other common grids (suche as X and Y) must be the same. obs Tgrid units are converted to forecat time fcst_time units. The score is calculated from all available pairs of fcst and obs data once obs data are sampled along forecast time fcst_time grids from their original time grid Tgrid.. There is no check whether the sum of forecast probabilities over all categories sum to 1. The ranked probability score is the cumulative squared forecast probability error across the forecast probability categories being predicted. Unlike the continuous ranked probability score, the squared error is accumulated over the category probabilities rather than over intervals of the variable being predicted (in physical units). The cumulative probability of the forecast is the total of the forecast probability of the given category plus the forecast probability of the lower categories. The cumulative probability of teh observation is 0 until the category of the obsercation is reached, at which time it steps up to 1. The higher the ranked probability score, the larger the squared forecast probability error. Example SOURCESIRIFDSeasonal_ForecastTemperature a:prob :a:target_date :a hometonybobs3-catTemperatureTemp /T /F /C ranked_prob_score
has argument: ranked prob score a3
has argument: ranked prob score a2
has argument: ranked prob score a1
ingrid code: 6 /ranked_prob_score publicproc: {fcst fcst_time obs Tgrid Sgrid Cgrid}inputs fcst units cvn /percent eq { (unitless) unitconvert}if obs name cvntos exch classify exch interp Cgrid renameGRID dup Tgrid get fcst_timeunits cvntos streamgridunitconvert fcst_time 1 index Tgrid get a:first :a:last :a masknotrange 1 index Tgrid get sample-along dup Cgrid getnpts 3 -2 roll grid: /name /Cint def 1 1 7 -1 roll :grid 2 {3 -1 roll dup Cgrid get 2 index replaceGRID Cint integral Cint second last RANGEEDGES Cint 3 index Cgrid get replaceGRID exch}repeat pop sub dup mul Sgrid cvntos interp Cgrid cvntos interp 2 array astore average DATA 0 1 RANGE /long_name (ranked probability score) def /fullname[ /ranked_prob_score fcstfullname obsfullname]def 1output :publicproc
has argument: ranked prob score a7
is described by: verification
func name: ranked_prob_score
has argument: ranked prob score a6
func desc: calculates ranked probability score for fcst relative to obs
has argument: ranked prob score a5

ranked prob score a4
type: Argument
arg order: 4
is argument of: ranked_prob_score
argument type: grid
arg desc: obs Time grid.
optional: false
in out: in
arg name: Tgrid

ranked prob score a3
type: Argument
is argument of: ranked_prob_score
in out: in
arg name: obs
arg desc: observed categories
arg order: 3
argument type: dataset or variable
optional: false

ranked prob score a2
type: Argument
argument type: variable
is argument of: ranked_prob_score
in out: in
arg name: fcst_time
optional: false
arg desc: Forecast time of forecast fcst
arg order: 2

ranked prob score a1
type: Argument
arg order: 1
arg desc: forecast probabilities. Must be in % (values between 0 and 100) or unitless (values between 0 and 1).
argument type: dataset or variable
is argument of: ranked_prob_score
arg name: fcst
in out: in
optional: false

ranked prob score a7
type: Argument
arg order: 7
arg desc: Ranked probability score for fcst relatively to obs.
argument type: dataset or variable
is argument of: ranked_prob_score
in out: out
arg name: ranked_prob_score

verification
type: Keyword
describes: ignorance
describes: ranked_prob_score
describes: hbrier
describes: prob_score
describes: bias_mean
describes: heidke

ignorance
type: Ingrid Procedure
syntax ex: ignorance computes the ignorance score for probabilistic forecast fcst relatively to observed categories obs. Other common grids (suche as X and Y) must be the same. obs Tgrid units are converted to forecast time fcst_time units. The score is calculated from all available pairs of fcst and obs data once obs data are sampled along forecast time fcst_time grids from their original time grid Tgrid. The ignorance score is based on the base-2 logarithm of the probability given to the later onbserved category. The bas-2 logarithm is computed by first finding the natural logarithm and then dividing it by the natural logarithm of 2, which is 0.69315. When the probability given to the observed category is 1 (100%), the ignorance score is at its maximum of 0; as the forecast probability decreases, the ignorance score becomes increasingly negative. The ignorance score is -1.585 for a forecast probability of 1/3 (33.3%), and becomes minus infinity for a forecast probanility of 0. Example SOURCESIRIFDSeasonal_ForecastTemperature a:prob :a:target_date :a hometonybobs3-catTemperatureTemp /T /C ignorance
has argument: ignorance a4
is described by: verification
has argument: ignorance a2
has argument: ignorance a3
func name: ignorance
has argument: ignorance a6
func desc: calculates ignorance score for fcst relative to obs
has argument: ignorance a1
ingrid code: 5 /ignorance publicproc: {fcst fcst_time obs Tgrid Cgrid}inputs fcst units cvn /percent eq { (unitless) unitconvert}if obs name cvntos exch classify exch interp Cgrid renameGRID dup Tgrid get fcst_timeunits cvntos streamgridunitconvert fcst_time 1 index Tgrid get a:first :a:last :a masknotrange 1 index Tgrid get sample-along mul Cgrid cvntos interp 1 array astore sum ln 2. ln div DATA AUTO 0 RANGE /long_name (ignorance score) def /fullname[ /ignorance fcstfullname obsfullname]def 1output :publicproc
has argument: ignorance a5

ignorance a4
type: Argument
arg order: 4
arg desc: Time grid of observations obs
is argument of: ignorance
argument type: grid
optional: false
in out: in
arg name: Tgrid

ignorance a2
type: Argument
argument type: variable
arg desc: Forecast time of the forecast fcst
is argument of: ignorance
arg order: 2
arg name: fcst_time
optional: false
in out: in

ignorance a3
type: Argument
optional: false
in out: in
arg name: obs
is argument of: ignorance
arg order: 3
argument type: dataset or variable
arg desc: observed categories

ignorance a6
type: Argument
argument type: dataset or variable
arg order: 6
arg desc: ignorance score of forecast fcst relative to observations obs
in out: out
arg name: ignorance
is argument of: ignorance

ignorance a1
type: Argument
optional: false
in out: in
arg name: fcst
is argument of: ignorance
argument type: dataset or variable
arg desc: forecast probabilities. Must be in % (values between 0 and 100) or unitless (values between 0 and 1).
arg order: 1

ignorance a5
type: Argument
arg order: 5
argument type: grid
arg desc: Categories grid of forecast fcst
is argument of: ignorance
arg name: Cgrid
in out: in
optional: false

hbrier
type: Ingrid Procedure
has argument: hbrier a3
has argument: hbrier a2
syntax ex: hbrier computes the half-Brier score for each category in Cgrid. Other common grids (suche as X and Y) must be the same. obs Tgrid units are converted to forecast time fcst_time units. The score is calculated from all available pairs of fcst and obs data once obs data are sampled along forecast time fcst_time grids from their original time grid Tgrid. The half Brier score, which equals to Brier score, is the squared probability error for one specific categorical outcome, such as "above normal temperature". The forecast probability, fcst, for the given category is compared with the observational outcome; if the category was not observed, the half Brier score is fcst2 and if the category was observed the half Brier score is (1-fcst)2. The highjer the half Brier score, the larger the squared forecast probability error for the given category. Example SOURCESIRIFDSeasonal_ForecastTemperature a:prob :a:target_date :a hometonybobs3-catTemperatureTemp /T /F /C hbrier
func name: hbrier
has argument: hbrier a1
has argument: hbrier a7
has argument: hbrier a6
is described by: verification
has argument: hbrier a5
ingrid code: 6 /hbrier publicproc: {fcst fcst_time obs Tgrid Sgrid Cgrid}inputs fcst units cvn /percent eq { (unitless) unitconvert}if obs name cvntos exch classify exch interp Cgrid renameGRID dup Tgrid get fcst_timeunits cvntos streamgridunitconvert fcst_time 1 index Tgrid get a:first :a:last :a masknotrange 1 index Tgrid get sample-along sub dup mul Sgrid cvntos interp 1 array astore average DATA 0 1 RANGE /long_name (half-brier score) def /fullname[ /hbrier fcstfullname obsfullname]def 1output :publicproc
func desc: calculates the half-Brier score between observed categories obs and forecast probabilities fcst for each category.
has argument: hbrier a4

hbrier a3
type: Argument
optional: false
in out: in
is argument of: hbrier
arg name: obs
arg order: 3
argument type: dataset or variable
arg desc: Observed categories obs. Must be in a categorical form corresponding to fcst categories grid gridC. Other common grids with fcst must match (such as X Y).

hbrier a2
type: Argument
argument type: variable
arg desc: Forecast time of forecast fcst
arg order: 2
arg name: fcst_time
is argument of: hbrier
in out: in
optional: false

hbrier a1
type: Argument
optional: false
in out: in
arg name: fcst
is argument of: hbrier
argument type: dataset or variable
arg desc: forecast probabilities by categories gridC. Must be in % (values between 0 and 100) or unitless (values between 0 and 1).
arg order: 1

hbrier a7
type: Argument
arg order: 7
argument type: dataset or variable
arg desc: Half-Brier score.
is argument of: hbrier
arg name: hbrier
in out: out

hbrier a6
type: Argument
argument type: grid
arg order: 6
arg desc: fcst grid describind the forecast categories.
is argument of: hbrier
arg name: Cgrid
in out: in
optional: false

hbrier a5
type: Argument
argument type: grid
arg desc: fcst Start or Forecast time grid.
arg order: 5
is argument of: hbrier
optional: false
arg name: Sgrid
in out: in

hbrier a4
type: Argument
is argument of: hbrier
optional: false
arg name: Tgrid
in out: in
arg order: 4
argument type: grid
arg desc: obs Time grid.

prob_score
type: Ingrid Procedure
is described by: verification
has argument: prob score a2
ingrid code: 6 /prob_score publicproc: {fcst fcst_time obs Tgrid Sgrid Cgrid} inputs fcst fcst_time obs Tgrid Sgrid Cgrid hbrier Cgrid cvntos interp 1 array astore average DATA 0 1 RANGE /long_name (probability score) def /fullname[/prob_score fcstfullname obsfullname]def 1output :publicproc
syntax ex: prob_score computes the probability score for probabilistic forecast fcst relatively to observed categroies obs. Other common grids (suche as X and Y) must be the same. obs Tgrid units are converted to forecast time fcst_time units. The score is calculated from all available pairs of fcst and obs data once obs data are sampled along forecast time fcst_time grids from their original time grid Tgrid. Actually, prob_score is nothing less than hbrier [Cgrid] average, that is the average of half-Brier scores over all the categories of Cgrid. The probability score or quadratic socre calculates the squared probabilty error over all of the forecast categories. If the probability forecast for a given category is fcst, then for categories in which the observation did not occur, the contribution to the squared probability error is fcst2, and for categories in which the observation did occur the contribution is (1-fcst)2. The sum of these contributions is then divided by the number of categories. The higher the quadratic score, the larger the squared forecast probability error. Unlike the half-Brier score, the quadratic score is affected not only by the probability given to the category that was later observed, but also by the probabilities given to the other categories. Example SOURCESIRIFDSeasonal_ForecastTemperature a:prob :a:target_date :a hometonybobs3-catTemperatureTemp /T /F /C prob_score
has argument: prob score a3
has argument: prob score a4
has argument: prob score a5
has argument: prob score a6
func desc: calculates probability score for fcst relative to obs
has argument: prob score a7
func name: prob_score
has argument: prob score a1

prob score a2
type: Argument
arg name: fcst_time
in out: in
optional: false
argument type: variable
is argument of: prob_score
arg order: 2
arg desc: Forecast time for forecast fcst

prob score a3
type: Argument
is argument of: prob_score
in out: in
arg name: obs
optional: false
arg desc: observed categories
arg order: 3
argument type: dataset or variable

prob score a4
type: Argument
optional: false
arg desc: obs Time grid.
arg order: 4
argument type: grid
is argument of: prob_score
in out: in
arg name: Tgrid

prob score a5
type: Argument
arg order: 5
is argument of: prob_score
arg desc: fcst Start time grid.
argument type: grid
in out: in
optional: false
arg name: Sgrid

prob score a6
type: Argument
optional: false
in out: in
arg name: Cgrid
is argument of: prob_score
arg order: 6
arg desc: fcst Categories grid.
argument type: grid

prob score a7
type: Argument
arg order: 7
arg desc: Probability score for fcst relatively to obs.
is argument of: prob_score
argument type: dataset or variable
in out: out
arg name: prob_score

prob score a1
type: Argument
argument type: dataset or variable
arg desc: forecast probabilities. Must be in % (values between 0 and 100) or unitless (values between 0 and 1).
arg order: 1
in out: in
arg name: fcst
is argument of: prob_score
optional: false

bias_mean
type: Ingrid Procedure
func desc: calculates the mean bias for deterministic forecasts fcst from observations obs.
has argument: bias mean a1
ingrid code: 5 /bias_mean publicproc: {fcst fcst_time obs Tgrid Sgrid}inputs fcst obs dup Tgrid get fcst_timeunits cvntos streamgridunitconvert fcst_time 1 index Tgrid get a:first :a:last :a masknotrange 1 index Tgrid get sample-along sub Sgrid cvntos interp 1 array astore average correlationcolorscale DATA AUTO AUTO RANGE /fullname[ /bias_mean fcstfullname obsfullname]def 1output :publicproc
syntax ex: Sgrid for fcst and Tgrid for obs need be of the same nature. fcst and obs other common grids must be the same (use of regridLinear in example). obs Tgrid units are converted to forecast time fcst_time units. The mean bias is computed from all pairs of available fcst and obs data once obs data are sampled along forecast time fcst_time grids from their original time grid Tgrid. The mean bias is the difference between the average of a set of deterministic forecasts and the average of the corresponding verifying observations. A positive mean bias indicates the forecasts averaged higher then the observations, and vice versa for a negative mean bias. The lower the magnitude of the bias, the more bias-free the forecasts are. Example SOURCESIRIFDNCEPForecastasstensemble monthly a:t2m (Celsius_scale) unitconvert SOURCESUEACRUTS2p1monthlymeantemp [X Y]regridLinear :a:T :a exch /T /S bias_mean
has argument: bias mean a5
has argument: bias mean a2
has argument: bias mean a6
has argument: bias mean a3
is described by: verification
func name: bias_mean
has argument: bias mean a4

bias mean a1
type: Argument
arg order: 1
arg desc: deterministic forecast
argument type: dataset or variable
is argument of: bias_mean
arg name: fcst
in out: in
optional: false

bias mean a5
type: Argument
is argument of: bias_mean
in out: in
arg name: Sgrid
optional: false
arg order: 5
arg desc: Forecast fcst Start time grid
argument type: grid

bias mean a2
type: Argument
argument type: variable
arg order: 2
arg desc: Forecast time of forecast fcst
is argument of: bias_mean
optional: false
in out: in
arg name: fcst_time

bias mean a6
type: Argument
in out: out
arg order: 6
argument type: dataset or variable
arg desc: mean bias of deterministic forecasts fcst from observations obs.
is argument of: bias_mean
arg name: bias

bias mean a3
type: Argument
optional: false
in out: in
arg name: obs
is argument of: bias_mean
argument type: dataset or variable
arg desc: observations
arg order: 3

bias mean a4
type: Argument
argument type: grid
is argument of: bias_mean
arg desc: Observations obs Time grid.
arg order: 4
arg name: Tgrid
in out: in
optional: false

heidke
type: Ingrid Procedure
func desc: calculates the Heidke score between observed categories obs and forecast categories fcst.
has argument: heidke a4
has argument: heidke a2
has argument: heidke a1
func name: heidke
has argument: heidke a3
syntax ex: heidke computes the Heidke score relative to observed categories obs and forecat categories fcst along forecast time fcst_time Start grid Sgrid that must be of the same nature as Tgrid. Other common grids (such as X and Y) must be the same. obs Tgrid units are converted to forecast time fcst_time units. The score is computed from all available pairs of fcst and obs data once obs data are sampled along forecast time fcst_time grids from their original time grid Tgrid. The Heidke skill score is based on the frequency with which the category having highest forecast probability is actually observed (a "hit"), where the probability value itself does not matter. This frequency is compared with the hit frequency expected by chance, and positive scores occur when the chance rate is exceeded, ranging 100% when every forecast is a hit. Negative scores indicate a hit rate lower than that expected by chance. The Heidke skill score can also be computed for the category having the second highest probability or even the lowest probability, and it would be hoped that these would have lower scores than the Heidke skill score for the category having the highest probability. Example SOURCESIRIFDSeasonal_ForecastTemperature a:prob [C]dominant_class :a:target_date :a hometonybobs3-catTemperatureTemp /T /F [0.67 0.33] heidke
is described by: verification
has argument: heidke a5
ingrid code: 6 /heidke publicproc: {fcst fcst_time obs Tgrid Sgrid catthr}inputs 0 1 catthr {dup 3 -1 roll sub dup mul 3 -1 roll add exch}forall dup mul add fcst obs dup Tgrid get fcst_timeunits cvntos streamgridunitconvert fcst_time 1 index Tgrid get a:first :a:last :a masknotrange 1 index Tgrid get sample-along sub 0 0 flagrange dup Sgrid cvntos interp 1 array astore sum 3 -2 roll 0 flagge Sgrid cvntos interp 1 array astore sum exch 1 index mul 2 index 1 index sub 0 flaglt dup -1 mul 1 add 4 index 3 index sub 5 -1 roll 4 index sub div mul 4 -1 roll 3 index sub 4 -1 roll div 3 -1 roll mul add /CLIST undef std_anomaly_colors DATA -1 1 RANGE /long_name (heidke score) def /fullname[ /heidke fcstfullname obsfullname]def 1output :publicproc
has argument: heidke a7
has argument: heidke a6

heidke a4
type: Argument
arg desc: obs Time grid
argument type: grid
arg order: 4
is argument of: heidke
arg name: Tgrid
optional: false
in out: in

heidke a2
type: Argument
optional: false
in out: in
argument type: variable
arg desc: Forecast time of forecast fcst
arg order: 2
arg name: fcst_time
is argument of: heidke

heidke a1
type: Argument
in out: in
arg name: fcst
is argument of: heidke
optional: false
argument type: dataset or variable
arg order: 1
arg desc: forecast categories

heidke a3
type: Argument
arg order: 3
argument type: dataset or variable
arg desc: observed categories
in out: in
optional: false
is argument of: heidke
arg name: obs

heidke a5
type: Argument
arg order: 5
arg desc: fcst Start or Forecast time grid.
in out: in
arg name: Sgrid
optional: false
argument type: grid
is argument of: heidke

heidke a7
type: Argument
arg desc: Heidke score
arg order: 7
in out: out
arg name: heidke
is argument of: heidke
argument type: dataset or variable

heidke a6
type: Argument
in out: in
optional: false
arg name: catthr
is argument of: heidke
argument type: numbers
arg desc: array of categories thresholds as percentiles in decreasing order. Must be between 1 and 0. 1 and 0 are not to be considered in the array.
arg order: 6

ranked prob score a6
type: Argument
optional: false
arg name: Cgrid
in out: in
is argument of: ranked_prob_score
argument type: grid
arg desc: fcst Categories grid.
arg order: 6