Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite #62

Open
11 of 17 tasks
cathaypacific8747 opened this issue Dec 18, 2024 · 3 comments
Open
11 of 17 tasks

Rewrite #62

cathaypacific8747 opened this issue Dec 18, 2024 · 3 comments

Comments

@cathaypacific8747
Copy link

cathaypacific8747 commented Dec 18, 2024

We are rewriting the project to be more modular and typed. Since there will be significant breaking changes, commits will be added in the dev branch and merged into master once stabilised.

todo:

  • Use uv (modern package manager)
  • make web and era5 optional dependencies
  • Use ruff (linting)
  • Use mkdocs
  • Use types whenever possible, check with mypy
  • update testing
  • do not hardcode cache paths #63
  • replace staticmethods in Calculation, Unit, Nav to pure functions*
  • Performance should not store Bada*
  • split god objects*
  • Use fastapi
    • update client: use simple REST calls instead of socketio (to automatically generate an OpenAPI spec and make
      debugging much easier)
    • remove code duplication
  • drop requirement for conda
  • migrate yarn to npm
  • update docs

* will be addressed in the experimental module

non-goals:

some notes:

  • makes heavy use of structure-of-arrays (Traffic, Autopilot)
  • Environment responsible for many tasks: simulation, socket IO, file IO.
  • Aircraft doesn't hold its own data, but acts as a pointer/slice to the arrays inside Traffic
    • dangling pointers when public Traffic.del_aircraft() is called
  • environment spawned on socketio run_simulation signal
@cathaypacific8747 cathaypacific8747 converted this from a draft issue Dec 18, 2024
@cathaypacific8747 cathaypacific8747 changed the title migrate to uv, ruff and mypy Rewrite Dec 18, 2024
This was referenced Dec 18, 2024
cathaypacific8747 added a commit that referenced this issue Dec 18, 2024
- fix: broken numpy docs in python files
- collapse `rst` api references to single index md
- style: fix test E501
- ci: update joss, paper
- remove: forge `environment.yml` (deprecate conda)
- remove: `MANIFEST.in` (replaced with hatchling)
cathaypacific8747 added a commit that referenced this issue Dec 20, 2024
- original code largely intact. only adding types to check for logic errors.
- fix: explicitly use uint16 for enums
- fix: `end_time` -> `duration_s`
- fix: `global_time` should be `seconds_since_start`
- rename: `traffic.ap` -> `traffic.autopilot`
- rename: `ConvertHistoricDemo` time -> timestamps
- fix misc logic errors
- chore: properly run ruff, mypy and tests with uv
@cathaypacific8747
Copy link
Author

cathaypacific8747 commented Dec 29, 2024

ec51a70 introduces a new MIT-licensed experimental module that houses the rewrite.

The goal is to allow functions to accept not only NumPy arrays, but anything that conforms to the Array API, so we can do things like:

import jax
import jax.numpy as jnp

from airtrafficsim.experimental.geospatial import G_0
from airtrafficsim.experimental.performance.bada3 import atmosphere
from airtrafficsim.experimental.thermodynamics import R_SPECIFIC_DRY_AIR

"""
Test that the BADA3 atmosphere model can be differentiated with JAX,
and that it is under hydrostatic equilibrium.
"""
zs = jnp.linspace(0, 20000, 100)

dpdz = jax.vmap(
    jax.grad(lambda z: atmosphere(z, delta_temperature=0.0).pressure)
)(zs)
rho = atmosphere(zs, delta_temperature=0.0).density(R_SPECIFIC_DRY_AIR)

assert jnp.allclose(dpdz, -rho * G_0)

@cathaypacific8747 cathaypacific8747 pinned this issue Dec 31, 2024
@cathaypacific8747
Copy link
Author

f0256ec introduces support for the polars Lazy API:

import polars as pl

import numpy as np
from airtrafficsim.experimental.performance.bada3 import atmosphere


def calc_atmosphere(z: pl.Expr) -> tuple[pl.Expr, pl.Expr, pl.Expr]:
    res = atmosphere(z, delta_temperature=0.0)
    return (
        z,
        res.pressure.alias("pressure"),
        res.temperature.alias("temperature"),
    )


DATA = {
    "z": np.linspace(0, 20000, 50),
    # ... more data
}

lf = pl.DataFrame(DATA).lazy()
query = lf.select(calc_atmosphere(pl.col("z")))
print(query.explain())
print(query.collect())
Output
SELECT [col("z"), when(col("__POLARS_CSER_0xf23969390f922b2d")).then([(101325.0) * ([([(col("__POLARS_CSER_0x9c1f4281e110a6e6")) - (0.0)]) / (288.15)].pow([dyn float: 5.25588]))]).otherwise([(22632.06) * ([([(col("z")) - (11000.0)]) * (-0.000158)].exp())]).alias("pressure"), col("__POLARS_CSER_0x9c1f4281e110a6e6").alias("temperature")] FROM
   WITH_COLUMNS:
   [[(col("z")) <= (11000.0)].alias("__POLARS_CSER_0xf23969390f922b2d"), when([(col("z")) <= (11000.0)]).then([(288.15) + ([(col("z")) * (-0.0065)])]).otherwise(216.65).alias("__POLARS_CSER_0x9c1f4281e110a6e6")] 
    DF ["z"]; PROJECT 1/1 COLUMNS; SELECTION: None
shape: (50, 3)
┌──────────────┬──────────────┬─────────────┐
│ z            ┆ pressure     ┆ temperature │
│ ---          ┆ ---          ┆ ---         │
│ f64          ┆ f64          ┆ f64         │
╞══════════════╪══════════════╪═════════════╡
│ 0.0          ┆ 101325.0     ┆ 288.15      │
│ 408.163265   ┆ 96516.787991 ┆ 285.496939  │
│ 816.326531   ┆ 91895.021614 ┆ 282.843878  │
│ 1224.489796  ┆ 87454.118816 ┆ 280.190816  │
│ 1632.653061  ┆ 83188.614413 ┆ 277.537755  │
│ …            ┆ …            ┆ …           │
│ 18367.346939 ┆ 7082.460919  ┆ 216.65      │
│ 18775.510204 ┆ 6640.974885  ┆ 216.65      │
│ 19183.673469 ┆ 6227.008934  ┆ 216.65      │
│ 19591.836735 ┆ 5838.8476    ┆ 216.65      │
│ 20000.0      ┆ 5474.882348  ┆ 216.65      │
└──────────────┴──────────────┴─────────────┘

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

No branches or pull requests

1 participant