Skip to content

Data Sources — Overview

All data sources inherit from DataSource and implement a single method:

def _fetch(self, grid: GridSpec | None = None) -> RasterData:
    ...

RasterData is a named tuple (array: np.ndarray, crs: str | None, transform: Affine | None). No rasterio objects are ever exposed outside a source module.

Callers use fetch(), which the base class provides. It reads and writes the disk cache, calls _fetch() and applies valid_range. A source becomes cacheable by also implementing _cache_key(grid), which returns a JSON-serialisable dict identifying the request, or None when the source is not cached.

Source catalogue

Source Use case
ArraySource In-memory numpy array
ConstantSource Single scalar broadcast over the grid
RasterSource Local GeoTIFF file
URLSource Remote GeoTIFF via HTTP
WCSSource Generic OGC WCS endpoint
PointGridSource Sample any callable over an N×N lat/lon grid
MosaicSource Combine several sources, best source first
DerivedSource Compute a layer from other sources

Grid-aware vs self-contained sources

Self-contained sources (ArraySource, ConstantSource, RasterSource, URLSource) ignore the grid argument. They carry their own spatial metadata.

Grid-aware sources need the reference grid to determine what geographic area to query. They call grid.extent_wgs84() to obtain (lon_min, lat_min, lon_max, lat_max) before making API or WCS requests. The box is computed from points along all four grid edges, so it covers edges that curve in lon/lat, and it reaches the pole for polar grids. A grid crossing the antimeridian gets a box spanning nearly all longitudes.

Grid-aware sources are: WCSSource, PointGridSource.

MosaicSource follows the sources it wraps. It needs the grid to align them before it can compare them pixel by pixel, but as long as one of them is self-contained the mosaic reports that source when asked without a grid, so an automatic grid can still be derived from it.

DerivedSource works the same way: it aligns its inputs to the grid before calling its function, and reports its first self-contained input when asked without a grid.

Masking sentinel values with valid_range

Every source takes valid_range=(lo, hi). Values below lo or above hi are replaced with NaN, so they are excluded from inference instead of being discretised as real values. Both bounds are inclusive, and either may be None to leave that side unbounded:

geobn.RasterSource("dem.tif", valid_range=(-500.0, 9000.0))  # mask both tails
geobn.RasterSource("dem.tif", valid_range=(0.0, None))       # mask negatives only

The three GeoTIFF-backed sources (RasterSource, URLSource, WCSSource) convert a declared nodata value and any mask band to NaN on their own. valid_range covers the data that declares nothing and encodes missing values as an extreme number, such as −9999 or 32767.

For the cached sources the range is applied to the array on its way out, so the cache holds the data as it arrived and changing the range does not trigger a new request.

Caching and freshness

URLSource, WCSSource and PointGridSource take cache_dir to store what they fetched on disk, and cache_ttl — a timedelta or a number of seconds — for how long an entry stays usable. Without a TTL an entry is used forever, which is what terrain and bathymetry want; with one, an older entry is fetched again.

Each entry is an .npy array beside a .json sidecar holding its CRS, transform and the time it was fetched. For URLSource the sidecar also holds the ETag and Last-Modified headers the server sent, if any, so an expired entry is checked with a conditional request and downloaded again only if the file has changed. cache_ttl is not part of the cache key, so changing it re-reads the same entry rather than orphaning it.

If an entry has expired and the fetch then fails, the expired entry is returned with a UserWarning naming its age, so an offline run still produces a map.

PointGridSource additionally needs name, since the callable it wraps cannot identify itself.

DataSource ABC

DataSource

Bases: ABC

Base class for all geographic data sources.

Every source returns a plain RasterData tuple (array, crs, transform) — no rasterio objects are ever exposed outside the source module.

Subclasses implement :meth:_fetch; :meth:fetch wraps it, reads and writes the disk cache, and applies valid_range. A source becomes cacheable by overriding :meth:_cache_key.

Parameters

valid_range: Optional (lo, hi) tuple. Pixels below lo or above hi are replaced with NaN, so they are excluded from inference instead of being discretised as real values. Both bounds are inclusive, and either may be None to leave that side unbounded. Use this for data that encodes missing values as extreme numbers without declaring them as nodata, e.g. (-500, 9000) for elevation in metres. cache_dir: Optional directory for caching fetched data on disk. Accepted only by sources that can be cached. cache_ttl: Maximum age of a cache entry, as a :class:~datetime.timedelta or a number of seconds. An older entry is refetched. The default None never expires, which suits data that does not change, such as terrain. If the refetch fails and an expired entry exists, that entry is returned with a :class:UserWarning giving its age. resampling: How the source is resampled onto the reference grid when its CRS, resolution or extent differ from the grid's:

- ``"bilinear"`` (default) interpolates between the four source
  pixels around each grid pixel centre.  It suits continuous data
  at a similar or coarser resolution than the grid.
- ``"nearest"`` takes the source pixel under the grid pixel centre.
- ``"mode"`` takes the most frequent value among the source pixels
  within each grid pixel, ties going to the smallest value.  Use it,
  or ``"nearest"``, for class rasters such as land cover or seabed
  type, where interpolating would produce codes that are not
  classes.
- ``"average"``, ``"min"`` and ``"max"`` take the mean, minimum or
  maximum of the source pixels within each grid pixel.  Use them
  when the source is finer than the grid: ``"average"`` for
  continuous data, and ``"max"`` or ``"min"`` for hazard layers,
  where a small steep slope or a shallow shoal must not be averaged
  away.

``"mode"``, ``"average"``, ``"min"`` and ``"max"`` ignore NaN
pixels, so a grid pixel is NaN only when none of its source pixels
has data.  Where the source is coarser than the grid, each grid
pixel lies within one source pixel and these methods give the same
result as ``"nearest"``.

fetch

fetch(grid=None)

Fetch data aligned to grid if provided.

Serves the disk cache when one is configured and the entry is within cache_ttl. Values outside valid_range are returned as NaN.

Parameters

grid: Reference grid context. Required by sources that need to know the spatial domain before querying (e.g. OpenMeteoSource). Ignored by sources that are self-contained (Array, Raster, URL, Constant).