A workflow that succeeds on a small study area may encounter different constraints when the number of observations, locations, or users increases. Scaling requires attention to processing capacity, data organisation, reference coverage, and the effort needed to review results. This perspective looks beyond whether a calculation can run on a larger dataset. It asks whether the complete service can preserve its meaning, quality, and delivery commitments as its scope expands. The objective is a design that identifies bottlenecks early and increases throughput without silently weakening the evidence available to users.
This working guide considers designing reproducible cloud processing workflows in cotton production fields. The immediate decision is to prioritise cotton field scouting. Useful reference evidence includes planting dates, growth stage and canopy observations. The case is an illustrative planning context, with assumptions to check against the actual sensor, landscape and project requirements.
Move the computation with a clear plan
Cloud processing can place analysis near a large satellite archive and reduce repeated transfers to a local computer. Its value depends on the workload. A small study may be easier to complete with a few downloaded scenes, while a continental time series benefits from selective reads and distributed processing. Begin with the geographic extent, dates, variables and output resolution that the analysis actually needs.
Inspect the storage format before choosing an execution pattern. Tiled imagery can support reading only intersecting blocks, and chunked multidimensional arrays can support selected time and spatial slices. These benefits depend on how data are organized and how the reader accesses them. Repeatedly opening complete remote files for tiny requests can waste time and bandwidth. Match processing chunks to source layout where practical, and avoid creating more small tasks than the scheduler can manage efficiently.
Make each run explainable and recoverable
Keep source identifiers, processing versions, input checksums where available, parameters and software dependencies with the workflow. Explicitly record projections, resampling methods, scale factors, masks and time aggregation rules. An output filename alone cannot capture these decisions. A small manifest linking inputs to outputs is useful when an archive is reprocessed or a result must be reproduced months later.
Develop on a representative subset that includes awkward cases such as missing bands, empty intersections and scenes with extensive cloud. Check numerical results and output metadata before expanding the workload. Design individual tasks so they can safely restart without duplicating completed results. Write intermediate output to a temporary location, verify it, and only then mark the task complete. Retain useful logs without recording access credentials.
Estimate storage, compute and data transfer costs together. Compression, unnecessary copies and repeated intermediate materialization can change the budget substantially. Monitor resource use during a pilot run and establish sensible limits before launching a large batch. Permissions should give each process the access it needs, and credentials should come from an appropriate secure mechanism. Finally, export an accessible result with provenance and quality information. A fast calculation becomes useful science only when another person can understand and inspect what was produced.
The setting: Cotton production fields
Cotton fields can differ in planting time, variety, stand density and management, producing different image trajectories through the season. A monitoring programme should organise those differences before deciding which fields are unusual. Early observations often combine crop and soil, while later management and crop-stage changes can alter the canopy in ways that require contextual interpretation. A reliable field calendar is therefore as important as the choice of a vegetation measure.
Compare observations that meet consistent quality requirements and inspect whether a mapped pattern persists across suitable dates. Field boundaries, tracks, wet areas and neighbouring crops can affect irregular management units. Record the valid area contributing to a summary, especially when clouds or shadows leave only part of a field visible. If several image products are combined, evaluate their alignment and measurement consistency before treating a difference as evidence of crop change.
A scouting visit should collect observations that address the intended decision, such as uneven development or a need for agronomic review. Record crop stage, recent operations and the sampling area consistently. An optical anomaly is not a direct measurement of fibre characteristics or a confirmed diagnosis of a particular problem. Models for commercial outcomes require corresponding reference data and independent evaluation, while inspection products can be assessed through their usefulness in directing representative and timely field visits.
Use observation labels with consistent meanings
A reference label should describe an observation that another person could recognise or reproduce. Broad terms such as good, damaged, or unusual can conceal substantial differences between collection teams. Begin by defining the feature or condition being recorded, the observation unit, and the evidence needed to assign a label. Where possible, distinguish a direct measurement from an interpretation made by the observer.
Provide a short collection guide with examples, boundary cases, and a way to record uncertainty. Train observers using shared cases before relying on independently collected labels. If two observers disagree, preserve the disagreement long enough to understand its cause. It may reveal an unclear definition, a genuinely ambiguous condition, a difference in observation scale, or a problem in matching the reference record to the remote measurement.
Retain the context that makes the label interpretable. Location, time, collection method, observer role, and relevant surface conditions can be essential for later review. Changes to the label scheme should be documented and related to earlier versions where a defensible correspondence exists. Consistent labels do not eliminate uncertainty, but they make it possible to investigate disagreement, evaluate analytical methods, and identify where better definitions or additional measurements would improve the evidence.
Design field observations around the information need
Field observations are most useful when they answer a defined question within the monitoring workflow. Begin by identifying the variable, condition, or management event that needs to be recorded and the geographic unit to which it applies. Distinguish direct measurements from an observer's interpretation. A consistent record of crop stage, management timing, and visible conditions can provide valuable context even when the field team is not collecting a complex set of measurements.
Prepare a collection protocol that reflects the variability of the site. Define where observations are made within a unit, how many locations are represented, and how the position and time are recorded. Avoid selecting only the most dramatic or convenient examples. Include apparently normal conditions and records that might challenge the remote interpretation. Where a measurement method has known limitations, preserve the information needed to assess whether those limitations affect the intended comparison.
Link field records to the corresponding remote observations without losing their original detail. Geographic matching, timing, and the scale represented by each record should remain visible. Review disagreements with the collecting team before changing a label or excluding an observation. A disagreement may reveal a timing mismatch, a mixed surface, or an incomplete interpretation rather than a simple error. The field programme should improve the understanding of the monitoring process as well as provide evidence for evaluating its outputs.
Bring the evidence back to the decision
Expansion should be guided by measured workload, visible quality conditions, and an understanding of which tasks require expert attention. Test the workflow under representative data volumes before treating a successful demonstration as evidence of operational capacity. Preserve a route for investigating individual observations even when most processing is automated. A scalable service keeps its results traceable and useful as it grows, with clear choices about where to automate, where to review, and where to invest in additional evidence.
Sources & further reading
Explore the bhumisat spacecraft concept and our vision towards 2030, or return to the journal.


