KSA National Agricultural Land Evaluation · App 2 · Module 7

Confidence Scoring

How every class assignment is scored for reliability, what each of the 64 elements can achieve given the source chosen for it, and what would have to change to lift the ceiling.

Components4
Elements scored59 of 64
Ceiling range0.73 – 0.92
Reaching high39

What the score is. Every class assignment carries a number between 0 and 1 stating how much the assignment can be relied on. A cell classed C2 at 0.88 and a cell classed C2 at 0.54 are the same class but not the same finding.

What it is built from. Four components, each answering one question. They are combined into a single score, and the lowest component constrains the result.

1
How was the value obtained?
A soil pit dug and sampled is not the same evidence as a value predicted by a global model.
2
How detailed is the source?
A 10 m satellite pixel resolves a 1.5 ha cell. A 9 km climate grid assigns the same value to thousands of cells.
3
Is the reading near a class edge?
A value 1 cm inside a class boundary could sit either side of it once measurement error is allowed for.
4
Are the thresholds settled?
A class boundary approved by an expert panel carries more weight than one extracted from a published table and not yet reviewed.

Component 1 — Acquisition method

How the value reached the database. This is the component that separates field evidence from remote sensing from inference.

MethodScoreDefinitionExample
field_measured1.00 Physically sampled at or near the cell and analysed. The value is an observation, not an estimate. Soil pit to 1.5 m, ECe and boron by lab analysis of horizon samples.
ministry_record0.95 Taken from an authoritative register. Not measured, but definitive — the register is the fact. Cadastral entry from Watheeq; protected-area boundary from the NCW register.
satellite_direct0.85 Computed directly from imagery or elevation with no intermediate model. The sensor measures the quantity. Slope from a 30 m DEM; NDVI from Sentinel-2 red and near-infrared bands.
satellite_modelled0.70 Predicted by a model trained on samples elsewhere. The sensor measures something correlated, not the quantity itself. SoilGrids depth-to-bedrock — a machine-learning prediction from terrain, climate and spectral covariates.
inferred0.55 Assigned from a related attribute by rule or association. No measurement of the quantity exists at any resolution. Boron assigned from mapped lithology and aquifer type, because no national boron surface exists.
Why this matters for the field programme. conf.element_confidence.field_sample_n records how many field observations exist for each element. Where the count is zero and the method is inferred, the element is a candidate for field sampling — and the confidence model identifies which elements those are without a separate exercise.

Component 2 — Source resolution

How the source pixel compares with the cell, 107.46 m across flats. Where the pixel is larger than the cell, neighbouring cells receive identical values and real variation is lost.

SourcePixelScoreWhat this means at cell level
Sentinel-210 m1.00 150 pixels per cell. Within-cell variation is resolved.
Copernicus DEM30 m1.00 17 pixels per cell. Sufficient for slope and terrain.
SoilGrids250 m0.68 One pixel covers 4.2 cells. Four adjacent cells share one soil value.
MODIS LST1 km0.41 One pixel covers 66 cells.
ERA5-Land9 km0.35 One pixel covers 8,100 cells. Unfloored the formula returns 0.25; the floor is binding. Climate values are regional, not cell-specific.
score = 1.00 where pixel ≤ cell width; otherwise 1 / (1 + 0.5 · log₂(pixel / cell width)), floored at 0.35. Cell width is 107.46 m, the across-flats dimension of an H3 resolution-10 hexagon. Every score in the table above is that expression evaluated at that width — none is assigned by hand. The logarithm is used because the penalty from 10 m to 100 m is more consequential than from 5 km to 9 km — both are already regional.

Component 3 — Boundary stability

Whether the value is far enough from a class boundary that measurement error would not change the class. This is the only component that varies cell by cell within the same source.

Worked example. Rooting depth, class C4 = 25–50 cm, source error ±6 cm. All six values below are classed C4.

ObservedTo edged/σClassStability
26.0 cm1.0 cm0.17C40.57
30.0 cm5.0 cm0.83C40.80
35.0 cm10.0 cm1.67C40.95
42.0 cm8.0 cm1.33C40.91
47.0 cm3.0 cm0.50C40.69
49.5 cm0.5 cm0.08C40.53
Reading the last row. 49.5 cm is 0.5 cm below the C3 boundary at 50 cm. With ±6 cm error there is a 47% chance the true depth exceeds 50 cm, in which case the cell is C3, not C4. Rows with d/σ < 1.0 are flagged: fragile_class = TRUE, and alt_class_id records the class it would otherwise take.

Component 4 — Threshold maturity

How settled the class boundaries themselves are. A correct measurement compared against an unapproved threshold still produces an uncertain class.

StatusScoreMeaning
operational1.00 Approved for national use by the review committee.
expert_reviewed0.92 Reviewed by named specialists against Saudi conditions; formal approval outstanding.
source_validated0.80 Traced to a cited publication and checked against it, but not assessed for Saudi conditions. All current thresholds are here.
extracted0.65 Loaded from a dataset without original-source verification.

How the components combine

-- per element, per cell
source_conf   = SQRT( method_score × res_score )
boundary_conf = Φ( d / σ )
confidence    = CBRT( source_conf × boundary_conf × threshold_conf )

-- per module, per cell
confidence = SQRT( governing_conf × completeness )

Why a geometric mean rather than multiplying. Four multipliers averaging 0.75 multiply to 0.32, which is uninformative. The geometric mean keeps the result on the same scale as its inputs while preserving the property that a low component holds the result down.

Which element's confidence is published. The class is set by one element — the worst one. The confidence published for that class is the confidence of that same element, or the lower value where two elements tie. Averaging across all elements would report a class as reliable when the element that actually decided it was the least reliable measurement on the cell.

The exponents are not equal. Nesting SQRT inside CBRT gives acquisition method and source resolution an exponent of ⅓×½ = ⅟ each, while boundary stability and threshold maturity carry ⅓ each. Threshold maturity therefore moves the score twice as far as whether the value was field-measured or inferred. This is deliberate: method and resolution describe the same thing — how well the source sees the cell — and are paired before entering the outer mean, so that a single source characteristic is not counted twice.

Bands. high ≥ 0.80  ·  medium ≥ 0.65  ·  low ≥ 0.50  ·  very_low < 0.50.

Worked example — one cell

ElementMethodResBoundThrConfidence
LQ-T-01 slopesatellite_direct1.001.000.80 0.90
LQ-SP-03 rootingsatellite_modelled0.680.930.80 0.80
LQ-C-03 radiationsatellite_modelled0.351.000.80 0.73
LQ-SC-01 ECesatellite_modelled0.680.760.80 0.75
LQ-SC-04 boroninferred0.680.710.80 0.70
The class is set by ECe and boron jointly, so the published confidence is 0.70 — the lower of the two. The mean of the five elements shown is 0.78. The gap is one step here, but the direction is the point: an average can only ever be at least as high as its lowest member, so averaging systematically overstates. The two elements that determined the class are also the two weakest on it — one modelled, one inferred from lithology with no direct measurement anywhere in the country. Slope, the best-resolved element on the cell at 0.90, had no influence on the class at all.
Two ceilings currently apply to every output. No threshold version has been approved for Saudi conditions, so component 4 evaluates to 0.80 for every element in the study. And no element is currently sourced from field measurement, so component 1 cannot reach 1.00 anywhere. Improving imagery resolution cannot compensate for either — threshold approval and a field sampling programme are the two changes that raise the ceiling on every figure the study produces.

Table definitions

conf
Confidence
Component configuration, per-pairing scores, per-cell scores and aggregates. cell_element and result_confidence are LIST-partitioned on province_id.
5 tables
53 columns
conf.componentMultiplier configuration for the four confidence components. Reference data, not observations.8 col~30▾
ColumnTypeKeyDefinition
component_idsmallintPKNOT NULL
factortextNOT NULLmethod | resolution | threshold_status.
keytextFKNOT NULLLKP-13→ ref.domain_value. Enumerated key within the factor, e.g. field_measured, satellite_modelled, source_validated. The vocabulary is defined once in ref.domain_value; this table supplies only its multiplier.
scorenumeric(3,2)NOT NULL0.00–1.000.00–1.00 multiplier applied when the key matches.
definitiontextNOT NULLWhat the key means, for UI display.
exampletextA representative element and source for this key.
rationaletextNOT NULLBasis for the assigned score. Required for audit.
is_activebooleanNOT NULLFALSE retains superseded scores without deleting the row.
conf.element_confidenceConfidence attributable to the element-source pairing. Invariant across cells; computed once per pairing.10 col~90▾
ColumnTypeKeyDefinition
element_codetextPKFKNOT NULLLKP-06→ ref.element
source_idsmallintPKFKNOT NULL→ src.data_source
methodtextNOT NULLLKP-13How the value is obtained: field_measured | ministry_record | satellite_direct | satellite_modelled | inferred.
method_scorenumeric(3,2)NOT NULL0.00–1.00Multiplier from conf.component for the method. field_measured 1.00 → inferred 0.55.
res_scorenumeric(3,2)NOT NULL0.35–1.001.00 where source GSD ≤ cell width (107.46 m); otherwise 1/(1 + 0.5·log₂(GSD/w)), floored at 0.35.
source_confnumeric(3,2)NOT NULL0.00–1.00SQRT(method_score × res_score). Upper bound on confidence for this element-source pairing.
expected_errornumeric≥ 0Source RMSE in the element's native unit. REQUIRED — boundary_conf cannot be computed without it.
error_basistextProvenance of expected_error: publisher_rmse | cross_validation | field_comparison | expert_estimate.
field_sample_ninteger≥ 0Count of field observations available for this element nationally. Zero for most; drives the field-validation programme.
last_revieweddateDate the pairing was last assessed.
conf.cell_elementPer-cell, per-element confidence. Varies across cells sharing a source because boundary_conf is value-dependent.12 col~9.4 B▾
ColumnTypeKeyDefinition
h3bigintPKFKNOT NULL
element_codetextPKFKNOT NULLLKP-06
version_idsmallintPKFKNOT NULL→ thr.version. boundary_conf is threshold-dependent.
province_idsmallintPARTNOT NULLLKP-33
source_confnumeric(3,2)NOT NULL0.00–1.00FK-resolved from conf.element_confidence.
boundary_confnumeric(3,2)NOT NULL0.50–1.00P(true value ∈ assigned class interval) = Φ(d/σ). Approaches 0.50 as d → 0.
dist_to_boundarynumeric≥ 0d — distance from observed value to nearest class boundary, native unit.
sigma_to_boundarynumeric(4,2)> 0d/σ. Values < 1.0 indicate an unstable assignment.
threshold_confnumeric(3,2)NOT NULL0.00–1.00Multiplier by thr.version.status. Maximum attainable is currently 0.80.
confidencenumeric(3,2)NOT NULL0.00–1.00CBRT(source_conf × boundary_conf × threshold_conf).
bandtextNOT NULLLKP-10high ≥ 0.80 | medium ≥ 0.65 | low ≥ 0.50 | very_low < 0.50.
is_imputedbooleanNOT NULLTRUE where interpolated or gap-filled. Confidence capped at 0.50 when set.
conf.result_confidenceConfidence in the module-level class. Derived from the governing element, consistent with the maximum-limitation assignment rule.12 col~215 M▾
ColumnTypeKeyDefinition
h3bigintPKFKNOT NULL
module_idsmallintPKFKNOT NULLLKP-01→ ref.module. One row per cell per module.
version_idsmallintPKFKNOT NULL
province_idsmallintPARTNOT NULLLKP-33
governing_confnumeric(3,2)NOT NULL0.00–1.00Confidence of the element that determined the class; MIN() where multiple elements tie at the governing class. Not an aggregate across all elements — the class derives from one element, so its confidence derives from the same element.
completenessnumeric(3,2)NOT NULL0.00–1.00COUNT(non-null elements) / COUNT(module elements). A class computed from a partial element set is qualified by this ratio.
mean_confnumeric(3,2)NOT NULL0.00–1.00AVG(confidence) across all module elements. Stored for diagnostic comparison; not the published value.
confidencenumeric(3,2)NOT NULL0.00–1.00SQRT(governing_conf × completeness). The published value.
bandtextNOT NULLLKP-10high · medium · low · very_low
fragile_classbooleanNOT NULLTRUE where sigma_to_boundary < 1.0 for the governing element. Indicates the class assignment is not stable under source error.
alt_class_idsmallintFKLKP-03→ ref.class. The class the cell would take if the governing value moved one σ. Null where robust.
limiting_factortextNOT NULLLKP-06Component with the lowest score: resolution | derivation | validation | boundary | threshold | completeness. Aggregated, identifies the dominant constraint on output quality.
conf.zone_confidenceZone-level aggregation of cell confidence, including internal class agreement.11 col~pending▾
ColumnTypeKeyDefinition
zone_idtextPKFKNOT NULL→ geo.zone
module_idsmallintPKFKNOT NULLLKP-01
version_idsmallintPKFKNOT NULL
mean_confnumeric(3,2)NOT NULL0.00–1.00Mean cell confidence across members.
min_confnumeric(3,2)NOT NULL0.00–1.00Worst cell in the zone.
p10_confnumeric(3,2)NOT NULL0.00–1.00P10 of member cell confidence. Less sensitive to a single outlier than MIN().
cell_agreementnumeric(3,2)NOT NULL0.00–1.00Share of member cells carrying the zone's class. Below ~0.90 suggests the zone should have been split.
fragile_sharenumeric(3,2)NOT NULL0.00–1.00COUNT(cells with fragile_class = TRUE) / cell_count.
confidencenumeric(3,2)NOT NULL0.00–1.00SQRT(p10_conf × cell_agreement). The published value.
bandtextNOT NULLLKP-10
recommendationtextRemediation required to raise confidence: field_sampling | finer_imagery | threshold_approval | ministry_data.
What each element can ever publish

Two of the four components are fixed the moment a source is chosen. Acquisition method comes from how the source produces the value; source resolution comes from its native pixel against the 107.46 m cell. Neither depends on the cell, so both can be computed now, before anything is processed.

The figures below hold boundary stability at its best case — a cell far from any class edge — and threshold maturity at its current source_validated value of 0.80. They are therefore ceilings: the highest confidence each element could publish on its most favourable cell. Real cells sit below them.

Bandhighmediumlowvery_lowThe dark mark on each bar is where the ceiling would sit if thresholds reached operational.
ElementNameMethodResCeiling now · and with operational thresholdsValue
LQ-SP-02Soil workability——Cannot be scored — the index has no defined construction, so there is no acquisition method to namen/a
LQ-SP-04Surface sealing and crusting——Cannot be scored — the index has no defined construction, so there is no acquisition method to namen/a
LQ-SC-03Nutrient availability——Cannot be scored — the index has no defined construction, so there is no acquisition method to namen/a
SQ-A-02Soil workability——Cannot be scored — the index has no defined construction, so there is no acquisition method to namen/a
SQ-A-04Surface sealing and crusting——Cannot be scored — the index has no defined construction, so there is no acquisition method to namen/a
LQ-T-02Water erosionsatellite_modelled0.350.73
LQ-T-03Wind erosionsatellite_modelled0.350.73
LQ-C-02Thermal regimesatellite_modelled0.350.73
LQ-C-03Radiationsatellite_modelled0.350.73
SQ-D-02Water erosion hazardsatellite_modelled0.350.73
SQ-D-03Wind erosion hazardsatellite_modelled0.350.73
SQ-E-02Thermal suitabilitysatellite_modelled0.350.73
SQ-E-03Radiation and solar energysatellite_modelled0.350.73
SQ-E-04Length of growing periodsatellite_modelled0.350.73
SQ-E-05Frost risksatellite_modelled0.350.73
SQ-F-01Irrigation demand (ETc)satellite_modelled0.350.73
LQ-SC-01Salinity (ECe)satellite_modelled0.380.75
LQ-SC-02Sodicity (ESP)satellite_modelled0.380.75
SQ-B-01Salinity (ECe)satellite_modelled0.380.75
SQ-B-02Sodicity (ESP / SAR)satellite_modelled0.380.75
SQ-E-01Moisture deficit (aridity)satellite_modelled0.380.75
SQ-B-05Calcium carbonatesatellite_modelled0.410.75
SQ-B-06Gypsum contentsatellite_modelled0.410.75
LQ-W-01Drainage conditionministry_record0.410.79
SQ-C-01Drainage conditionministry_record0.410.79
LQ-SP-01Available water capacitysatellite_modelled0.620.81
LQ-SP-03Rooting conditionssatellite_modelled0.620.81
SQ-A-01Available water capacitysatellite_modelled0.620.81
SQ-A-03Rooting conditionssatellite_modelled0.620.81
SQ-A-05Soil texturesatellite_modelled0.620.81
SQ-A-06Coarse fragmentssatellite_modelled0.620.81
SQ-B-07Soil pHsatellite_modelled0.620.81
LQ-SC-04Toxicity (boron)inferred1.000.84
SQ-B-04Toxicity risk (boron)inferred1.000.84
LQ-W-02Flood hazardsatellite_modelled1.000.87
LQ-W-04Waterlogging risksatellite_modelled1.000.87
LQ-T-04Sand encroachmentsatellite_modelled1.000.87
SQ-C-02Flood hazardsatellite_modelled1.000.87
SQ-C-03Waterlogging risksatellite_modelled1.000.87
SQ-D-04Sand encroachment hazardsatellite_modelled1.000.87
LQ-T-01Terrain (slope)satellite_direct1.000.90
SQ-D-01Terrain workabilitysatellite_direct1.000.90
WP-A-01Source typeministry_record1.000.92
WP-A-02Available volumeministry_record1.000.92
WP-A-03Distance to sourceministry_record1.000.92
WP-B-01Water-level trendministry_record1.000.92
WP-B-02Aquifer typeministry_record1.000.92
WP-B-03Remaining supply horizonministry_record1.000.92
WP-C-01Irrigation water salinityministry_record1.000.92
WP-C-02Sodium adsorption ratioministry_record1.000.92
WP-C-03Chlorideministry_record1.000.92
WP-C-04Boron in irrigation waterministry_record1.000.92
WP-C-05Treated wastewater tierministry_record1.000.92
WP-D-01Pumping liftministry_record1.000.92
WP-D-02Conveyance distanceministry_record1.000.92
WP-E-01Basin sustainable yieldministry_record1.000.92
WP-E-02Allocation statusministry_record1.000.92
EPProtected areaministry_record1.000.92
EVRangeland, forest and afforestationministry_record1.000.92
ETTenureministry_record1.000.92
EZZoning designationministry_record1.000.92
EXConflicting useministry_record1.000.92
EHHazard designationministry_record1.000.92
SQ-F-02Crop nutrient requirementministry_record1.000.92
Seven elements cannot be scored at all. They are the index-valued columns with no defined construction — and the reason is worth stating precisely: confidence component 1 asks how a value was acquired, and an undefined index has no acquisition path to name. The gap is not that their confidence is low. It is that the question cannot be asked.
What would actually lift the ceiling

Three levers exist, and they are not equally worth pulling. Each figure below is the same arithmetic applied to all 59 scorable elements.

Baseline today
0.84
mean element ceiling
39 of 59 elements reach the high band at best case; 20 stop at medium.
Lever 1 · calibrate thresholds
+0.06
to 0.90 mean
Taking threshold maturity from source_validated 0.80 to operational 1.00 moves 9 more elements into the high band. It requires expert calibration, not data, and it lifts every element at once because the component is shared.
Lever 2 · field measurement
+0.03
to 0.87 mean
Raising acquisition method to field_measured everywhere — a national sampling programme, and the most expensive option by a wide margin. It buys less than calibration does, because method carries an exponent of ⅟ while threshold maturity carries ⅓.
The order is the finding. Calibrating thresholds is cheaper than a sampling programme and lifts the mean ceiling further, because the formula weights threshold maturity twice as heavily as acquisition method. A proposal that funds field sampling before threshold calibration would spend more to gain less, and this table is the argument against it.

Lever 3 · finer sources, element by element

Resolution is the only lever that acts per element rather than across the board. It is worth pulling exactly where a coarse source meets an element that varies over short distances.

The clearest case is frost. SQ-E-05 sits at 0.73 because ERA5-Land is 9 km. Frost forms by cold-air drainage over hundreds of metres, in the Asir and Al Bahah highlands where terrain is most broken — so the resolution is not merely coarse, it is coarser than the process being measured. NCM station records would move both the method and the resolution term at once.

The weakest case is radiation. LQ-C-03 sits at the same 0.73 from the same source, but radiation varies little over short distances. The low score is arithmetically correct and practically unimportant, and no finer source is worth acquiring for it.

Two elements, the same score, opposite conclusions. A ranked list of ceilings is not on its own a work programme.