Authentication & Credentials¶

Credential Resolution Hierarchy¶

gathering resolves credentials using a smart search hierarchy (highest priority first):

Direct parameters — in Python / YAML task definition (auth: {username: ..., password: ...})
Environment variables — (EARTHCARE_USERNAME, PANDONIA_API_KEY, etc.)
Local .env file — in $GATHERER_FOLDER/.env or current working directory
Global secure file — ~/.config/gathering/credentials.yaml (or $GATHERER_FOLDER/.config/...)

Tip

You only need to configure credentials once. The gathering auth login command stores them securely for all future sessions.

Credentials & Registration Matrix¶

Source	Web Portal	Auth Method	Env Variables	Registration / Account Signup
AERONET	aeronet.gsfc.nasa.gov	None (Open)	—	No registration needed
ACTRIS Ares	ares.actris.eu	None (Open)	—	No registration needed
ACTRIS Cloudnet	cloudnet.fmi.fi	None (Open)	—	No registration needed
Pandonia (PGN)	pandonia-global-network.org	API Key	`PANDONIA_API_KEY`	Request API Key via Contact Form
Copernicus CDSE	dataspace.copernicus.eu	Email + Password	`CDSE_USERNAME`, `CDSE_PASSWORD`	CDSE Account Registration
EUMETSAT (EUMDAC)	data.eumetsat.int	Key + Secret	`EUMETSAT_CONSUMER_KEY`, `EUMETSAT_CONSUMER_SECRET`	EO Portal Account Signup · Generate Keys
ESA EarthCARE	earthcare.esa.int	User/Pass / Token	`EARTHCARE_USERNAME`, `EARTHCARE_PASSWORD`, `EARTHCARE_MAAP_TOKEN`	ESA MAAP Registration · EO Gateway Data Access
NASA PACE	pace.gsfc.nasa.gov	NASA Earthdata Login	`EARTHDATA_USERNAME`, `EARTHDATA_PASSWORD`	NASA Earthdata Signup
NASA MODIS	modis.gsfc.nasa.gov	NASA Earthdata Login	`EARTHDATA_USERNAME`, `EARTHDATA_PASSWORD`	NASA Earthdata Signup

Shared NASA Earthdata Credentials

NASA PACE and MODIS share the exact same NASA Earthdata Login account (EARTHDATA_USERNAME and EARTHDATA_PASSWORD).

How to Register for Each Protected Source¶

1. Copernicus Data Space Ecosystem (CDSE)¶

Portal: dataspace.copernicus.eu
Registration: Visit the CDSE Registration Page and sign up with your email.
Verification: Activate your account via the verification email. Your email and password are used directly by gathering auth login cdse or via CDSE_USERNAME and CDSE_PASSWORD.

2. EUMETSAT Data Store (EUMDAC)¶

Portal: data.eumetsat.int
Registration: Create a free account on the EUMETSAT EO Portal.
Key Generation:
1. Log in to the EUMETSAT API Key Portal.
2. Go to API Key Management and generate a new Consumer Key and Consumer Secret.
3. Store them using gathering auth login eumdac -u "<key>" -p "<secret>".

3. NASA Earthdata (PACE & MODIS)¶

Portal: earthdata.nasa.gov
Registration: Register on the NASA Earthdata Login Registration Page.
Application Authorization: Under your Earthdata profile, ensure authorization for DAAC applications (OB.DAAC, LAADS DAAC, and LP DAAC).

4. ESA EarthCARE (MAAP)¶

Portal: earthcare.esa.int & maap.eo.esa.int
Registration: Create an account on the ESA EarthCARE MAAP IAM.
Data Access: Request access via ESA EO Gateway or copy your offline Bearer Token from MAAP Account Management.

5. Pandonia Global Network (PGN)¶

Portal: pandonia-global-network.org
Registration / API Key: Contact the PGN operations team via the PGN Contact Form to request an operational API key.

CLI Commands¶

Save Credentials Interactively¶

# Store credentials for a specific source
gathering auth login cdse -u "your_email@domain.com" -p "your_password"
gathering auth login eumdac -u "your_consumer_key" -p "your_consumer_secret"
gathering auth login earthcare -u "your_username" -p "your_password"
gathering auth login pandonia -p "your_api_key"
gathering auth login earthdata -u "your_username" -p "your_password"

Write Credentials to `.env`¶

# Store in .env file instead of global credentials.yaml
gathering auth login cdse -u "your_email" -p "your_password" --env

Check Credentials Status¶

# View which sources have configured credentials
gathering auth status

Generate Template `.env`¶

# Create a template .env file with all supported variables
gathering auth env

Remove Credentials¶

gathering auth logout earthcare

Global Credentials File¶

The global credentials file (~/.config/gathering/credentials.yaml) is created with 0600 permissions (owner read/write only). You can also create it manually:

# ~/.config/gathering/credentials.yaml

# Copernicus Data Space Ecosystem (CDSE)
cdse:
  username: "your_email@domain.com"
  password: "your_cdse_password"

# EUMETSAT Data Access Client (EUMDAC)
eumdac:
  consumer_key: "your_eumetsat_consumer_key"
  consumer_secret: "your_eumetsat_consumer_secret"

# ESA EarthCARE (MAAP STAC / OADS)
earthcare:
  username: "your_earthcare_username"
  password: "your_earthcare_password"
  maap_token: "your_optional_earthcare_maap_token"

# Pandonia Global Network (PGN)
pandonia:
  api_key: "your_pandonia_api_key"

# NASA Earthdata Login (shared by PACE and MODIS)
earthdata:
  username: "your_earthdata_username"
  password: "your_earthdata_password"

Warning

Never commit credentials.yaml or .env files containing real credentials to version control. Add them to your .gitignore.

Inline Credentials in YAML¶

For quick testing, credentials can be passed directly in YAML task files. This is not recommended for production use:

tasks:
  - source: cdse
    # Inline credentials (use env vars or credentials.yaml instead)
    # username: "your_email@domain.com"
    # password: "your_password"
    instrument: MSI
    product_type: S2MSI2A
    start_time: "2024-07-01T00:00:00Z"
    end_time: "2024-07-05T23:59:59Z"

Inline Credentials in Python¶

task = Task.cdse(
    instrument="MSI",
    product_type="S2MSI2A",
    start_time="2024-07-01T00:00:00Z",
    end_time="2024-07-05T23:59:59Z",
    # Auth resolved automatically from env/credentials.yaml
    # Or pass explicitly:
    # username="your_email@domain.com",
    # password="your_password",
)

Environment Variable Substitution in YAML¶

YAML configuration files support ${VARIABLE} syntax for environment variable expansion:

tasks:
  - source: earthcare
    auth:
      username: ${EARTHCARE_USERNAME}
      password: ${EARTHCARE_PASSWORD}
    fetch:
      collections:
        - EARTHCARE_ATLID_L1B
      product_types:
        - ATL_NOM_1B
      start_time: "2024-09-01T00:00:00Z"
      end_time: "2024-09-02T23:59:59Z"