A method that produces a plausible result has not automatically demonstrated that it works for its intended users. Validation requires a defined question, suitable reference evidence, and an evaluation design that reflects how the method will be used. This perspective examines which observations should be compared, how they should be sampled, and which errors matter in practice. It also considers the difference between evidence used to develop a method and evidence reserved for evaluating it. The objective is a test that can reveal weaknesses as well as confirm useful performance.
This working guide considers designing ground validation for satellite products in soybean production fields. The immediate decision is to direct scouting to unusual field zones. Useful reference evidence includes planting dates, canopy closure and crop-stage observations. The case is an illustrative planning context, with assumptions to check against the actual sensor, landscape and project requirements.
Start with the intended claim
Validation tests how well a product represents a defined quantity under specified conditions. It differs from calibration, which adjusts an instrument or model using reference information. Begin with the claim the product needs to support: class membership, surface reflectance, canopy height, water quality, or an area estimate. State the required spatial scale, time interval, and acceptable error. These choices determine the reference method and sample design. Collecting convenient field points first and deciding later what they prove usually leaves important conditions untested.
Reference measurements also contain uncertainty. Positioning error can place a sample in the wrong satellite pixel, a point instrument may not represent a heterogeneous footprint, and measurements made on different dates can describe real surface change. Record instrument calibration, sampling protocols, timestamps, coordinates, observation conditions, and uncertainty where available. For interpreted labels, retain the evidence used, class rules, and ambiguous cases. Agreement among interpreters is useful to examine, but agreement alone does not guarantee that the shared interpretation is correct.
Keep the assessment independent
Separate validation data from observations used to fit thresholds, train models, select features, or tune processing. If assessment sites repeatedly influence revisions, they have become development data and a fresh independent assessment is needed. Spatially clustered samples can make performance look stronger when neighboring training and test locations share conditions. Use a design that reflects the target population and report its limitations, including inaccessible areas or environmental conditions missing from the sample.
Choose metrics that match the product. For categorical maps, examine a confusion matrix and errors for individual classes, not just overall agreement. For continuous measurements, report bias and a suitable error measure alongside sample count and the range of reference values. Plot residuals against relevant conditions to expose failures hidden by an average. For area estimates, carry the probability sampling design through estimation and uncertainty calculations. Preserve rejected matchups and their reasons so that quality screening remains auditable. A useful validation report explains where the product works, where evidence is sparse, and which errors matter for the intended decision. Publishing reference protocols and assessment code makes later comparisons more credible and helps others reproduce the evaluation.
The setting: Soybean production fields
Soybean imagery can show substantial variation as fields progress from exposed soil to closed canopy and eventually to maturity. Planting date, stand arrangement, variety and management can shift that progression. Use these factors to define comparison groups before labelling a field unusual. The boundary dataset should distinguish the planted area from tracks, drainage features and other surfaces that can contribute a large share of the measurement in small or irregular fields.
A practical analytical sequence combines quality-screened imagery with the field calendar and relevant management notes. Review whether an apparent difference persists across usable observations and whether it follows a plausible spatial pattern within the field. An index response can be affected by background soil, weeds, residue or illumination as well as the crop. These possibilities should guide the investigation rather than being collapsed into an automatic claim about plant nutrition or disease.
A field team can test whether the mapped zones correspond to different crop stages or other observable conditions. Collect samples across the range of mapped responses, and include typical areas to avoid evaluating only conspicuous anomalies. Where production estimates are proposed, match the spatial and temporal scale of the reference data to the prediction. A field-average yield record cannot directly validate every small pixel-level variation shown on an attractive map.
Keep an auditable review record
A review record should explain how a result moved from an analytical output to an accepted interpretation or operational decision. Preserve the product identifier, the evidence considered, the reviewer role, and the outcome of the review. A short, consistent record is often more useful than an extensive free-text note that cannot be compared between cases. The objective is to make important decisions traceable without turning routine review into unnecessary administrative work.
Separate observations from conclusions within the record. Describe the evidence visible to the reviewer, the alternative explanations considered where relevant, and the reason for accepting, qualifying, or rejecting the result. If a correction is required, identify what should change and how the revised product relates to the earlier version. Keep unresolved questions visible so that they can be addressed by another observation or by the person responsible for the next action.
Use the accumulated record to improve the workflow. Recurring review reasons can reveal unclear requirements, missing contextual information, unstable processing steps, or an evaluation gap. Examine whether similar cases receive similar decisions across reviewers and over time. Preserve the relationship between feedback and subsequent revisions so that improvements can be assessed against the evidence that motivated them. An auditable record supports learning, accountability, and consistent delivery while allowing the technical process to evolve.
Define a representative reference sample
A reference sample should represent the situations in which a method will be used. Begin by defining the target population: locations, time periods, management conditions, surface types, or instrument states that belong within the intended application. A collection of convenient observations may be valuable for exploration, but it does not automatically support conclusions about that wider population. Make the scope of the sample visible before calculating a summary of performance.
Consider a stratified design when important conditions are unevenly represented. Relevant strata might include different landscape types, developmental stages, observation quality levels, or ranges of the variable being estimated. Choose sample locations within those groups using a documented rule, and retain the information needed to understand their selection. Where access restrictions affect the sample, record the excluded conditions so that the limits of the resulting evaluation remain clear.
Keep development and evaluation evidence separate where the purpose of the study requires an independent test. Nearby observations or repeated measurements can share information, so a random split of individual records may overstate independence. A split organised by location, period, or operational unit can provide a more demanding comparison. The appropriate design follows the intended deployment, the structure of the observations, and the errors that matter to the user.
Bring the evidence back to the decision
The validation record should explain the population represented by the test, the reference observations used, and the limitations of the comparison. Report meaningful errors alongside useful successes, and retain examples that challenge the method. Treat wider deployment as a new opportunity to evaluate transfer across relevant conditions. A well-designed validation programme gives users a defensible basis for deciding where the method is useful, where additional evidence is needed, and where another approach may be appropriate.
Sources & further reading
Explore the bhumisat spacecraft concept and our vision towards 2030, or return to the journal.



