Multi-Source Observational Pipelines¶

Atmospheric validation campaigns often require matching satellite overpasses with coincident ground-based observations (e.g. AERONET sun photometers, Pandonia spectrometers, and ACTRIS lidar profiles).

gathering allows you to orchestrate multi-instrument, multi-platform campaigns using a single YAML configuration file or Python Pipeline.

Campaign Example Workflow¶

The following campaign example synchronizes ground observations over Granada (Southern Spain) with satellite overpasses from EUMETSAT MTG FCI, Sentinel-3 OLCI, and ESA EarthCARE ATLID lidar.

# ==============================================================================
# Comprehensive Multi-Source Gathering Campaign Pipeline
# ==============================================================================
# Orchestrates Ground-Truth (AERONET, Pandonia, ACTRIS) and Satellite overpasses
# (EUMETSAT MTG FCI, MSG SEVIRI, Sentinel-3, EarthCARE) in a unified workflow.
# ==============================================================================

logging:
  level: INFO
  log_file: "gathering_campaign.log"

tasks:
  # ----------------------------------------------------------------------------
  # 1. Ground Truth: NASA AERONET Photometer (AOD & Inversions)
  # ----------------------------------------------------------------------------
  - source: aeronet
    site: "Granada"
    start_date: "2024-09-01"
    end_date: "2024-09-02"
    product:
      - AOD15
      - ALM15

  # ----------------------------------------------------------------------------
  # 2. Ground Truth: Pandonia Global Network (PGN Trace Gases)
  # ----------------------------------------------------------------------------
  - source: pandonia
    station: "Granada"
    start_date: "2024-09-01"
    end_date: "2024-09-02"
    product:
      - fnvh3
      - fzo3

  # ----------------------------------------------------------------------------
  # 3. Ground Truth: ACTRIS ARES In-situ Aerosol Optical Properties
  # ----------------------------------------------------------------------------
  - source: actris_ares
    station: "gra"
    data_type: "aer_opt"
    matrix: "pm10"
    start_date: "2024-09-01"
    end_date: "2024-09-02"

  # ----------------------------------------------------------------------------
  # 4. Satellite: EUMETSAT MTG FCI (Smart ROI Chunk Selection)
  # ----------------------------------------------------------------------------
  - source: eumdac
    instrument: FCI
    start_time: "2024-09-01T10:00:00Z"
    end_time: "2024-09-01T14:00:00Z"
    download_minutes:
      - 0
    grid:
      center_lat_lon: [37.164, -3.605]   # Granada
      size_yx: [50, 50]
      resolution_m: 1000.0
      projection: "latlon"
    extract_entries: true

  # ----------------------------------------------------------------------------
  # 5. Satellite: EUMETSAT Sentinel-3 OLCI Ocean Colour
  # ----------------------------------------------------------------------------
  - source: eumdac
    instrument: OLCI
    start_time: "2024-09-01T10:00:00Z"
    end_time: "2024-09-01T12:00:00Z"
    bounding_box: [-6.0, 35.0, -1.0, 39.0]
    limit: 1

  # ----------------------------------------------------------------------------
  # 6. Satellite: ESA EarthCARE Atmospheric Lidar (ATLID Level 2A)
  # ----------------------------------------------------------------------------
  - source: earthcare
    collections:
      - "EARTHCARE_ATLID_L2A"
    product_types:
      - "ATL_AER_2A"
      - "ATL_TC__2A"
    start_time: "2024-09-01T00:00:00Z"
    end_time: "2024-09-02T23:59:59Z"
    grid:
      center_lat_lon: [37.164, -3.605]
      size_yx: [50, 50]
      resolution_m: 1000.0
      projection: "latlon"

Executing Multi-Source Pipelines¶

CLIPython

gathering examples/multi_source_campaign_pipeline.yaml

from gathering import run_from_yaml

results = run_from_yaml("examples/multi_source_campaign_pipeline.yaml")
print(f"Total downloaded files: {len(results.files)}")