sofarocean/mlwp-neuralgcm repository overview

⁠MLWP-main

Main working repository for Navy ML SBIR Phase I feasibility study

The objective of this repository is to link multiple repositories from both inside and outside of Sofar in order to organize all of the working components that will go into the construction of a hybrid AI/ML-NWP forecast system.

The main objectives of the Navy SBIR are to:

1. Produce an AI/ML-based ensemble forecast, and evaluate ensemble forecast skill
   a) Generate ensemble forecasts using AI/ML models
   b) Produce cycled DA with AI/ML model forecasts
   c) Develop a strategy to generate large ensembles
2. Produce a coupled atmosphere-wave-ocean forecast system that replaces the atmosphere component with an AI/ML surrogate
   a) Force WaveWatchIII (ww3) with AI/ML model winds
   b) Develop a ESMF/NUOPC cap for an AI/ML model (possibly the hardest step here)
   c) Couple the AI/ML model within our ESM infrastructure

Our execution strategy will be to develop tools to incorporate the AI/ML models into our existing ESM workflow, so as to not require any new significant infrastructure. This will also help us benefit from developments in this project towards our general ESM forecast system goals (e.g. adding new diagnostics and metrics to esm-assessment).

⁠Getting started

(optional, set up python virtual environment) With default python and venv:

python3 -m venv ./.venv

Or with penv: https://dev.to/mattcale/pyenv-poetry-bffs-20k6⁠

pyenv exec python -m venv .venv

Activate:

source ./.venv/bin/activate

For forpy, need to add:

export PYTHONPATH=$PWD/.venv/lib/python3.10/site-packages

locally in your virtual environment for this project
(the way to go if you don't want to use poetry globally and only for this project)

⁠Install poetry:

python3 -m pip install poetry

(optional) Enable poetry completions:

poetry completions bash >> ~/.bash_completion

⁠install dependencies

poetry install

⁠Install with dependency groups

poetry install --with dev,cloud,neuralgcm

⁠List where the packages have been installed to:

poetry config --list

⁠List packages with:

poetry show

⁠To run in Jupyter lab:

poetry run jupyter lab &

(optional) execute pre-commit install to install git hooks in your .git/ directory
this will allow linting and formatting on commit

pre-commit install

To build the dockerfile:

docker build -f ./Dockerfile.neuralgcm -t sofarocean/mlwp-neuralgcm:0.0.0 .