Struct RasterDataset

pub struct RasterDataset<T>
where
    T: RasterType,
{
    pub metadata: RasterMetadata<T>,
    pub blocks: Vec<RasterBlock<T>>,
    pub(crate) tmp_layers: Vec<PathBuf>,
}

Main data structure of eorst. Stores the raster dataset metadata and blocks.

The easiest way to create a RasterDataset is by using RasterDatasetBuilder:

use std::path::PathBuf;
use eorst::{types::BlockSize, RasterDatasetBuilder, DataSourceBuilder, RasterDataset};

let data_source = DataSourceBuilder::from_file(&PathBuf::from("scene.tif")).build();
let rds: RasterDataset<u16> = RasterDatasetBuilder::from_source(&data_source)
    .block_size(BlockSize { cols: 2048, rows: 2048 })
    .build();

Processing with apply

apply is the main entry point. Pass a worker function that receives a [RasterDataBlock] — the block data, metadata, and no-data value. The worker returns an Array4 which is written directly to the output GeoTIFF:

use eorst::{RasterDataBlock, RasterDataset};
use anyhow::Result;
use ndarray::Array4;

fn worker(block: &RasterDataBlock<u16>) -> Result<Array4<i16>> {
    let red = block.select_layers(&["red"])?;
    let nir = block.select_layers(&["nir"])?;
    let ndvi = ((&nir.data - &red.data) / (&nir.data + &red.data + 1e-10)) * 10000i16;
    Ok(ndvi.cast::<i16>()?)
}

rds.apply::<i16>(worker, 8, &PathBuf::from("output.tif"))?;

The Select trait on RasterDataBlock lets you select layers and time slices by name without tracking indices manually.

Alternative: Manual Block Iteration

You can also iterate over blocks manually:

for iter in rds.iter() {
    let block_id = iter.iter_index;
    let block_data = rds.read_block::<i32>(block_id);
    // ... process block_data directly
}

Fields

metadata: RasterMetadata<T>

Metadata of a RasterDataset

blocks: Vec<RasterBlock<T>>

Blocks of a RasterDataset

tmp_layers: Vec<PathBuf>

Temporaty layers, removed when the RasterDataset is Dropped.

Implementations

impl<R> RasterDataset<R>

where R: RasterType,

pub fn apply<U>( &self, worker: fn(&RasterDataBlock<R>) -> Result<Array4<U>>, n_cpus: usize, out_file: &Path, ) -> Result<()>

where U: RasterType,

Applies a worker function to each block, receiving a RasterDataBlock with metadata.

The worker receives &RasterDataBlock<R> which includes the block data array, layer_indices, date_indices, and other metadata. This enables semantic layer and time selection via the Select trait.

Blocks are processed in parallel using a Rayon thread pool. Each block’s result is written directly to the output GeoTIFF via a ParallelGeoTiffWriter, eliminating intermediate files and the subprocess-based mosaic/translate phase.

Returns Err if any worker invocation fails.

Example

// See apply_reduction below for a fully-runnable doctest.
// apply() follows the same pattern but returns Array4 and writes directly to GeoTIFF:
//
// fn worker(block: &RasterDataBlock<i16>) -> anyhow::Result<Array4<i16>> {
//     Ok(block.data.mapv(|v| v as i16))  // pass-through example
// }
//
// rds.apply::<i16>(worker, 1, &out_path)?;

pub fn apply_cog<U>( &self, worker: fn(&RasterDataBlock<R>) -> Result<Array4<U>>, n_cpus: usize, out_file: &Path, config: &OutputConfig, ) -> Result<()>

where U: RasterType,

Applies a worker function to each block and outputs a Cloud Optimized GeoTIFF.

This is the COG variant of apply. It follows the same block-processing pattern but produces a proper COG with overviews and IFD reordering.

The output process:

1. Write blocks to an intermediate GeoTIFF (same as apply)
2. Build overviews in-place via GDAL’s build_overviews
3. Translate to final COG via gdal_translate -of COG for proper IFD ordering
4. Clean up intermediate file

Arguments

• worker - Worker function receiving &RasterDataBlock<R>, returning Array4<U>
• n_cpus - Number of CPU threads for parallel processing
• out_file - Output COG file path
• config - Output configuration controlling format, compression, overviews

pub fn apply_with_mask<U, V>( &self, other: &RasterDataset<U>, worker: fn(&RasterDataBlock<R>, &RasterDataBlock<U>) -> Result<Array4<V>>, n_cpus: usize, out_file: &Path, ) -> Result<()>

where U: RasterType, V: RasterType,

Applies a worker function to pairs of blocks from two datasets.

The worker receives &RasterDataBlock<R> and &RasterDataBlock<U> which include layer_indices, date_indices, and other metadata. This enables semantic selection via the Select trait on both datasets.

Useful for masking, zonal statistics, or combining two rasters (e.g. applying a cloud mask, or computing NDVI with a land cover classification).

Blocks are processed in parallel using a Rayon thread pool. Each block pair’s result is written directly to the output GeoTIFF.

Returns Err if any worker invocation fails.

// See apply() for the primary usage pattern. apply_with_mask() takes
// a second dataset whose blocks are paired with the primary dataset's
// blocks, and a worker that receives both blocks simultaneously.
// Example worker signature:
//
// fn masked_ndvi(
//     raster: &RasterDataBlock<i16>,
//     mask: &RasterDataBlock<i16>,
// ) -> anyhow::Result<Array4<i16>> {
//     // mask.data is used to nullify invalid pixels
//     ...
// }
//
// rds.apply_with_mask::<i16, i16>(&mask_rds, masked_ndvi, 8, &out_path)?;

pub fn apply_with_mask_cog<U, V>( &self, other: &RasterDataset<U>, worker: fn(&RasterDataBlock<R>, &RasterDataBlock<U>) -> Result<Array4<V>>, n_cpus: usize, out_file: &Path, config: &OutputConfig, ) -> Result<()>

where U: RasterType, V: RasterType,

Applies a worker function to pairs of blocks and outputs a Cloud Optimized GeoTIFF.

COG variant of apply_with_mask.

pub fn apply_mosaic<T>(&self, n_cpus: usize)

where T: RasterType,

Creates a mosaic from the raster blocks.

pub fn mosaic<T>(&self, n_cpus: usize)

where T: RasterType,

👎Deprecated since 0.3.2: Use apply_mosaic() instead. mosaic() will be removed in a future release.

Deprecated: use apply_mosaic instead.

pub fn apply_reduction<U>( &self, worker: fn(&RasterDataBlock<R>, UtilsDimension) -> Array3<U>, dimension: UtilsDimension, n_cpus: usize, out_file: &Path, na_value: U, )

where U: RasterType,

Applies a worker function that reduces the data dimensionality over each block.

The worker receives &RasterDataBlock<R> which includes layer_indices, date_indices, and other metadata. This enables semantic selection via the Select trait.

Common uses: mean over time (timesteps → 1), mean over layers (layers → 1), or any other reduction operation. The result is written directly to a GeoTIFF via a parallel writer — no intermediate files.

Example

extern crate eorst;
use std::path::PathBuf;
use eorst::{
    types::{BlockSize, Dimension}, DataSourceBuilder, RasterDatasetBuilder, RasterDataBlock,
    RasterDataset, Select,
};
use ndarray::{Array3, Axis};

fn mean_over_time(block: &RasterDataBlock<i16>, dim: Dimension) -> Array3<i16> {
    // Reduce along a dimension, returning Array3 (e.g. mean over time)
    let as_f32: ndarray::Array4<f32> = block.data.mapv(|v| v as f32);
    let mean = as_f32.mean_axis(Axis(0)).unwrap();
    mean.mapv(|v| v as i16)
}

let manifest_dir = env!("CARGO_MANIFEST_DIR");
let data_source = DataSourceBuilder::from_file(
    &PathBuf::from(manifest_dir).join("data/cemsre_t55jfm_20200614_sub_abam5.tif"),
)
.bands(vec![3, 4])
.set_names(vec!["red", "nir"])
.build();

let rds: RasterDataset<i16> = RasterDatasetBuilder::from_source(&data_source)
    .block_size(BlockSize { cols: 256, rows: 256 })
    .build();

let out_path = std::env::temp_dir().join("apply_reduction_mean.tif");
rds.apply_reduction::<i16>(mean_over_time, Dimension::Layer, 1, &out_path, i16::MIN);

pub fn reduce<U>( &self, worker: fn(&Array4<R>, UtilsDimension) -> Array3<U>, dimension: UtilsDimension, n_cpus: usize, out_file: &Path, na_value: U, )

where U: RasterType,

👎Deprecated since 0.3.2: Use apply_reduction() instead. reduce() will be removed in a future release.

Deprecated: use apply_reduction instead.

pub fn apply_reduction_with_mask<U, V>( &self, other: &RasterDataset<U>, worker: fn(&RasterDataBlock<R>, &RasterDataBlock<U>, UtilsDimension) -> Array3<V>, dimension: UtilsDimension, n_cpus: usize, out_file: &Path, na_value: V, )

where V: RasterType, U: RasterType,

Reduces the raster dataset with another dataset as a mask.

The worker receives &RasterDataBlock<R> and &RasterDataBlock<U> which include layer_indices, date_indices, and other metadata.

This version uses a parallel writer that writes blocks directly to the output GeoTIFF, eliminating intermediate files and the subprocess-based mosaic/translate phase.

pub fn apply_reduction_with_mask_cog<U, V>( &self, other: &RasterDataset<U>, worker: fn(&RasterDataBlock<R>, &RasterDataBlock<U>, UtilsDimension) -> Array3<V>, dimension: UtilsDimension, n_cpus: usize, out_file: &Path, na_value: V, config: &OutputConfig, )

where V: RasterType, U: RasterType,

Reduces the raster dataset with another dataset as a mask, outputting a Cloud Optimized GeoTIFF.

COG variant of apply_reduction_with_mask.

pub fn reduce_with_mask<U, V>( &self, other: &RasterDataset<U>, worker: fn(&Array4<R>, &Array4<U>, UtilsDimension) -> Array3<V>, dimension: UtilsDimension, n_cpus: usize, out_file: &Path, na_value: V, )

where V: RasterType, U: RasterType,

👎Deprecated since 0.3.2: Use apply_reduction_with_mask() instead. reduce_with_mask() will be removed in a future release.

Deprecated: use apply_reduction_with_mask instead.

pub fn apply_reduction_row_pixel<T>( &self, worker: fn(&Array4<T>, UtilsDimension) -> Array2<T>, na_value: T, dimension: UtilsDimension, n_cpus: usize, out_file: &Path, )

where T: RasterType,

Applies a worker function that reduces rows to pixel values.

Note: this method reads data as type T which may differ from the dataset’s stored type R, so the worker receives &Array4<T> rather than &RasterDataBlock<T>.

pub fn reduce_row_pixel<T>( &self, worker: fn(&Array4<T>, UtilsDimension) -> Array2<T>, na_value: T, dimension: UtilsDimension, n_cpus: usize, out_file: &Path, )

where T: RasterType,

👎Deprecated since 0.3.2: Use apply_reduction_row_pixel() instead. reduce_row_pixel() will be removed in a future release.

Deprecated: use apply_reduction_row_pixel instead.

pub fn apply_reduction_row_pixel_with_mask<V, U>( &self, other: &RasterDataset<V>, worker: fn(&Array4<R>, &Array4<V>, UtilsDimension) -> Array2<U>, na_value: U, dimension: UtilsDimension, n_cpus: usize, out_file: &Path, )

where U: RasterType, V: RasterType,

Reduces rows with a mask dataset.

Note: this method reads data as types R and V matching the datasets, so the worker receives raw &Array4 references.

pub fn reduce_row_pixel_with_mask<V, U>( &self, other: &RasterDataset<V>, worker: fn(&Array4<R>, &Array4<V>, UtilsDimension) -> Array2<U>, na_value: U, dimension: UtilsDimension, n_cpus: usize, out_file: &Path, )

where U: RasterType, V: RasterType,

👎Deprecated since 0.3.2: Use apply_reduction_row_pixel_with_mask() instead. reduce_row_pixel_with_mask() will be removed in a future release.

Deprecated: use apply_reduction_row_pixel_with_mask instead.

impl<R> RasterDataset<R>

where R: RasterType,

pub fn rasterize( &self, vector_fn: &Path, column_name: Option<&str>, map: Option<HashMap<String, i32>>, layer_name: Option<&str>, n_cpus: usize, rasterize_options: Option<RasterizeOptions>, out_file: &Path, )

Burns vector values into the raster dataset.

Rasterizes geometries from a vector file by processing blocks in parallel and writing directly to the output GeoTIFF via [ParallelGeoTiffWriter], avoiding intermediate files and a mosaic step.

pub fn rasterize_cog( &self, vector_fn: &Path, column_name: Option<&str>, map: Option<HashMap<String, i32>>, layer_name: Option<&str>, n_cpus: usize, rasterize_options: Option<RasterizeOptions>, out_file: &Path, config: &OutputConfig, )

Burns vector values into the raster dataset, outputting a Cloud Optimized GeoTIFF.

COG variant of rasterize. Follows the same rasterization logic but produces a proper COG with overviews and IFD reordering.

impl<R> RasterDataset<R>

where R: RasterType,

pub fn geoms_to_global_indices( &self, geoms: BTreeMap<i64, Vec<(f64, f64, f64)>>, ) -> BTreeMap<i64, Index2d>

Converts geometries to global array indices.

fn geo_to_global_rc(&self, point: Coordinates) -> Index2d

pub fn block_id_rowcol( &self, pid: i64, index: Index2d, ) -> (usize, (i64, Index2d))

Gets the block ID and local row/col for a global point ID.

fn global_rc_to_block_rc(&self, global_index: Index2d) -> Index2d

fn id_from_indices(&self, index: Index2d) -> usize

fn n_block_cols(&self) -> usize

pub fn extract_blockwise( &self, vector_path: &PathBuf, id_col_name: &str, method: SamplingMethod, buffer_size: Option<usize>, ) -> BTreeMap<i16, Vec<i16>>

Extracts values from the raster dataset for vector features, block-wise.

pub fn extract( &self, geometries: &[Geometry], point_ids: &[i64], method: SamplingMethod, buffer_size: Option<usize>, ) -> Result<(Array2<i16>, Vec<i64>)>

Extracts values from the raster dataset for point geometries.

impl<R> RasterDataset<R>

where R: RasterType,

pub fn stack( &mut self, other: &RasterDataset<R>, dimension_to_stack: UtilsDimension, ) -> &mut RasterDataset<R>

Stack RasterDataset along a dimension.

pub fn extend<V>(&mut self, other: &RasterDataset<V>) -> &mut RasterDataset<R>

where V: RasterType,

Extend a RasterDataset.

pub fn column_names(&self) -> Vec<String>

Create names like time0_layer0, time0_layer1, time1_layer0, etc.

pub fn iter(&self) -> RasterDatasetIter<'_, R>

An iterator over the crate::blocks::RasterBlocks of a RasterDataset.

impl<R> RasterDataset<R>

where R: RasterType,

Reads a block of raster data using async-tiff.

This is the async equivalent of io::read_block(). Reads from S3 directly using async-tiff instead of opening via GDAL.

pub async fn read_block_async<T: RasterType>( &self, block_id: usize, ) -> RasterData<T>

Async variant of read_block() using async-tiff for S3 access.

Returns RasterData<T> (4D array: times × layers × rows × cols).

pub async fn read_block_async_with_urls<T: RasterType>( &self, s3_urls: &[String], block_id: usize, ) -> RasterData<T>

Async read block using pre-computed S3 URLs.

Caches TIFF handles for efficiency across multiple block reads.

pub async fn apply_async<U>( &self, worker: fn(&RasterDataBlock<R>) -> Result<Array4<U>>, n_cpus: usize, out_file: &Path, ) -> Result<()>

where U: RasterType,

Async variant of apply() using async-tiff for S3 access.

Applies a worker function to each block asynchronously. This is the async equivalent of processing::apply(). Reads from S3 directly using async-tiff (COG-aware, parallel tile fetching).

Arguments

• worker - Function that processes a RasterDataBlock<T> and returns Array4<U>
• n_cpus - Number of parallel workers (used for writing phase)
• out_file - Output file path

pub async fn apply_async_with_urls<U>( &self, s3_urls: &[String], worker: fn(&RasterDataBlock<R>) -> Result<Array4<U>>, n_cpus: usize, out_file: &Path, ) -> Result<()>

where U: RasterType,

Async variant that accepts pre-computed S3 URLs.

Use this when you have the S3 URLs directly (e.g., from STAC assets).

Trait Implementations

impl<T> Clone for RasterDataset<T>

where T: RasterType + Clone,

fn clone(&self) -> RasterDataset<T>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T> Debug for RasterDataset<T>

where T: RasterType + Debug,

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T> Display for RasterDataset<T>

where T: RasterType,

fn fmt(&self, fmt: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<T> PartialEq for RasterDataset<T>

where T: RasterType + PartialEq,

fn eq(&self, other: &RasterDataset<T>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<T> StructuralPartialEq for RasterDataset<T>

where T: RasterType,