Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 41 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,22 +7,55 @@ When started, it does the following:
- Creates a STAC item with assets. The assets `href` is based on the given URL.
- Adds the STAC item to a STAC catalog. If an item with the same id already exists, it will be updated.

## Installation

Install the package using pip:

```bash
# Install from source
pip install .

# Or install in editable mode for development
pip install -e .
```

After installation, the `eopf-stac` command will be available in your environment.

## Usage

The EOPF product must be referenced by an URL which must be provided as a command-line argument. Only `http(s)://`, `s3://` and `file://` URLs are supported. For debugging, a combination of the `--dry-run` and `--debug` option can be used to see the created STAC item in the logs without inserting it into a catalog.

```bash
$ python src/eopf_stac/main.py --help
usage: eopf-stac.py [-h] [--dry-run] [--debug] URL
$ eopf-stac --help
usage: eopf-stac.py [-h] [--source-uri SOURCE_URI] [--dry-run] [--output-file OUTPUT_FILE] [--debug] URL

positional arguments:
URL Local file path or HTTP/S3 URL to the EOPF product.
URL Local file path or URL to the EOPF product

options:
-h, --help show this help message and exit
--source-uri SOURCE_URI Reference to the original product which was input for the EOPF conversion
--dry-run Create STAC item only. Do not insert it into the catalog
--debug Enable verbose output
-h, --help show this help message and exit
--source-uri SOURCE_URI
Reference to the original product which was input for the EOPF conversion
--dry-run Create STAC item without trying to insert it into the catalog
--output-file OUTPUT_FILE
Save the STAC item as JSON to the specified file path
--debug Enable verbose output
```

Example usage:

```bash
# Register STAC item to catalog
eopf-stac s3://path/to/eopf.zarr

# Save STAC item to local file instead of registering
eopf-stac --output-file item.json s3://path/to/eopf.zarr

# Dry run with debug output (only logs, doesn't save or register)
eopf-stac --dry-run --debug s3://path/to/eopf.zarr

# With source URI
eopf-stac --source-uri s3://original/product.nc s3://path/to/eopf.zarr
```

## Settings
Expand All @@ -34,7 +67,7 @@ Additional settings need to be provided through the following environment variab
| AWS_ACCESS_KEY_ID | If an s3 URL is provided, the access key for the object storage where the EOPF product is stored. | None |
| AWS_SECRET_ACCESS_KEY | If an s3 URL is provided, the secret for the access key. | None |
| S3_ENDPOINT_URL | If an s3 URL is provided, the endpoint URL of the object storage | None |
| STAC_API_URL | The URL of the STAC catalog to register the created STAC item | None |
| STAC_API_URL | The URL of the STAC catalog to register the created STAC item. Not required if `--output-file` is used. | None |
| STAC_INGEST_USER | The username to access the transaction endpoints of the STAC API with HTTP Basic Auth | None |
| STAC_INGEST_PASS | The password to access the transaction endpoints of the STAC API with HTTP Basic Auth | None |

Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ dependencies = [
"netCDF4 == 1.7.2",
"footprint-facility"
]
[project.scripts]
eopf-stac = "eopf_stac.main:main"

[project.optional-dependencies]
dev = [
"ruff"
Expand Down
16 changes: 12 additions & 4 deletions src/eopf_stac/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import logging
import os
from sys import exit
from typing import Optional

from eopf_stac.io import create_item, read_metadata, register_item

Expand All @@ -25,7 +26,7 @@ def configure_logging(level: int):
)


def validate_env(url: str, dry_run: bool, env: dict):
def validate_env(url: str, dry_run: bool, output_file: Optional[str], env):
if url.startswith("s3://"):
# if s3 url is provided, the credentials are required?
missing_vars = []
Expand All @@ -41,7 +42,7 @@ def validate_env(url: str, dry_run: bool, env: dict):
if len(missing_vars) > 0:
raise ValueError(f"The following enviroment variables are missing: {missing_vars}")

if not dry_run:
if not dry_run and not output_file:
if ENV_STAC_API_URL not in env:
raise ValueError(f"The enviroment variable {ENV_STAC_API_URL} is missing")

Expand All @@ -62,6 +63,7 @@ def main():
parser.add_argument(
"--dry-run", help="Create STAC item without trying to insert it into the catalog", action="store_true"
)
parser.add_argument("--output-file", help="Save the STAC item as JSON to the specified file path", type=str)
parser.add_argument("--debug", help="Enable verbose output", action="store_true")
args = parser.parse_args()

Expand All @@ -71,7 +73,7 @@ def main():
configure_logging(logging.INFO)

try:
validate_env(args.URL, args.dry_run, env=os.environ)
validate_env(args.URL, args.dry_run, args.output_file, os.environ)

logger.debug("Opening metadata file ...")
metadata = read_metadata(args.URL)
Expand All @@ -81,7 +83,13 @@ def main():
logger.debug(json.dumps(item.to_dict(), indent=4))

if not args.dry_run:
item = register_item(item=item, stac_api_url=os.environ[ENV_STAC_API_URL])
if args.output_file:
logger.info(f"Writing STAC item to {args.output_file}")
with open(args.output_file, "w") as f:
json.dump(item.to_dict(), f, indent=4)
else:
logger.info(f"Registering STAC item to {os.environ[ENV_STAC_API_URL]}")
item = register_item(item=item, stac_api_url=os.environ[ENV_STAC_API_URL])

except Exception as e:
logger.error(str(e))
Expand Down