Crate eorst

Expand description

§EORST — Earth Observation and Remote Sensing Toolkit

A high-performance Rust library for processing satellite imagery. Designed for researchers and engineers who need to process large geospatial raster datasets efficiently — from querying STAC catalogs to computing spectral indices across thousands of scenes.

§Quick Start

RasterDataset is the main entry point. Combine it with rss_core to query cloud-optimised satellite imagery and process it in parallel blocks:

use std::path::PathBuf;
use chrono::NaiveDate;
use anyhow::Result;
use ndarray::Array4;

use rss_core::{DEA, query::ImageQueryBuilder, qvf::Collection, utils::{Cmp, Intersects}};
use eorst::{types::BlockSize, RasterDatasetBuilder, DataSourceBuilder, RasterDataBlock};

// ── 1. Query DEA Sentinel-2 ARD via STAC ──────────────────────────────────
let source = DEA.clone();
let query = ImageQueryBuilder::new(
    source,
    Collection::Sentinel2,
    Intersects::Scene(vec!["56jmr"]),
)
.canonical_bands(["red", "nir"])
.start_date(NaiveDate::parse_from_str("2022-01-01", "%Y-%m-%d")?)
.end_date(NaiveDate::parse_from_str("2022-01-15", "%Y-%m-%d")?)
.cloudcover((Cmp::Less, 10))
.build();

let output_dir = PathBuf::from("/tmp/DEA_S2");
let _ = query.get(&output_dir, None, None)?;  // download scenes

// ── 2. Build a RasterDataset from the downloaded scenes ───────────────────────
let scene_files: Vec<_> = std::fs::read_dir(&output_dir)?
    .filter_map(|e| e.ok())
    .filter(|e| e.path().extension().map_or(false, |ext| ext == "tif"))
    .map(|e| e.path())
    .collect();

let rds = RasterDatasetBuilder::<u16>::from_sources(&scene_files)
    .block_size(BlockSize { cols: 2048, rows: 2048 })
    .build();

// ── 3. Apply a parallel worker across all blocks ──────────────────────────────
fn ndvi_worker(block: &RasterDataBlock<u16>) -> Result<Array4<i16>> {
    let red = block.select_layers(&["red"])?;
    let nir = block.select_layers(&["nir"])?;
    let red_f: ndarray::Array4<f32> = red.data.mapv(|v| v as f32);
    let nir_f: ndarray::Array4<f32> = nir.data.mapv(|v| v as f32);
    let ndvi = ((&nir_f - &red_f) / (&nir_f + &red_f + 1e-10)) * 10000.0;
    Ok(ndvi.mapv(|v| v as i16))
}

rds.apply::<i16>(ndvi_worker, 8, &PathBuf::from("ndvi_output.tif"))?;

The same pattern works for any worker function — mosaicking, band math, machine-learning classification, zonal statistics, and more.

§Core Types

RasterDataset — The main data structure. Holds metadata and blocks for an entire raster. Use RasterDatasetBuilder to construct one.
RasterDataBlock — A block of raster data with metadata. Passed to your worker function by apply and apply_reduction.
DataSourceBuilder — Build data sources from files or STAC queries.
Select — Select layers and time slices by name from a RasterDataBlock.

§Processing Methods

RasterDataset provides parallel block-processing methods:

apply — Apply a worker to each block, writing results directly to a GeoTIFF. The most common entry point.
apply_with_mask — Apply a worker using two datasets, where the second acts as a mask.
apply_reduction — Reduce a dimension (e.g. mean over time) and write the result.
apply_reduction_with_mask — Reduce with a mask dataset.
apply_mosaic — Mosaic the dataset to a single file.

§Highlights

Block-based parallel processing — handles datasets larger than memory
Works on laptop, HPC, or cloud — same code, same architecture
On-the-fly reprojection and resolution changes
Point raster sampling extraction
Time series analysis and band math
OpenCV integration via use_opencv feature
LightGBM and XGBoost integration via use_lgbm feature
Nix-managed reproducible environments

§Crate Status

Still iterating and evolving. Breaking changes are expected between versions.
The API is stabilizing. Contributions and feedback are welcome.

§Installation

Add to Cargo.toml:

[dependencies]
eorst = "1.0"

Optional features:

eorst = { version = "1.0", features = ["use_opencv"] }  # OpenCV computer vision
eorst = { version = "1.0", features = ["use_lgbm"] }   # LightGBM ML classification

§CLI

The eors CLI provides command-line access to common workflows. See the install docs for setup instructions.

§Summary

This crate offers a library aiming to simplify the writing of raster processing pipelines in rust.

Re-exports§

pub use core_types::RasterData;
pub use core_types::RasterType;
pub use data_sources::DataSourceBuilder;
pub use data_sources::DateType;
pub use metadata::Extent;
pub use metadata::RasterDataBlock;
pub use metadata::RasterMetadata;
pub use blocks::RasterBlock;
pub use crate::selection::Select;
pub use crate::selection::SelectError;
pub use rasterdataset::RasterDataset;
pub use rasterdataset::RasterDatasetBuilder;
pub use gdal;

Modules§

array_ops: Array manipulation operations for raster processing. Array manipulation operations for raster processing.
async_io: Async I/O using async-tiff for direct S3 access. Module to handle async I/O for raster datasets using async-tiff.
blocks: Block types for raster dataset processing. Block types for raster dataset processing.
core_types: Core type definitions for raster data handling. Core type definitions for raster data handling.
data_sources: Data source definitions for raster datasets. Data source definitions for raster datasets.
filters: Image processing filters and OpenCV integration. Image processing filters and OpenCV integration.
gdal_utils: GDAL utility functions for raster processing. GDAL utility functions for raster processing.
metadata: Metadata types for raster datasets. Metadata types for raster datasets.
parallel_writer: Parallel GeoTIFF writer for direct windowed writes (replaces mosaic subprocess chain). Parallel GeoTIFF writer inspired by Dask/rioxarray’s RasterioWriter.
rasterdataset: RasterDataset and RasterDatasetBuilder for creating and manipulating raster data. RasterDataset module for handling raster data.
selection: Selection and aggregation traits for raster data. Selection and aggregation traits for raster data.
stac_helpers: STAC helpers for querying and processing STAC ItemCollections.
standalone_docs: Standalone documentation pages.
tests 🔒
types: Geospatial and array shape types. Geospatial and array shape types.

Structs§

ItemCollection: A GeoJSON FeatureCollection of items.

Functions§

init_logger: Initializes the logging system with default settings.

Type Aliases§

RasterBlockSlice2: Type alias for 2D raster block slices backed by ndarray.
RasterBlockSlice3: Type alias for 3D raster block slices backed by ndarray.

Crate eorstCopy item path