Skip to content

Commit 88b8103

Browse files
authored
Merge pull request #11 from Achronus/v0.2.0
V0.2.0 - NeuroFlow Update
2 parents 7c074fb + 1d64842 commit 88b8103

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

85 files changed

+8335
-4309
lines changed

README.md

Lines changed: 31 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -12,31 +12,26 @@ Found on:
1212

1313
# Velora
1414

15-
**Velora** is a lightweight and extensible framework built on top of powerful libraries like [Gymnasium](https://gymnasium.farama.org/) and [PyTorch](https://pytorch.org/), specializing in a unique approach to Deep Reinforcement Learning (RL) algorithms, a paradigm we call *Liquid RL*.
15+
**Velora** is a lightweight and modular framework built on top of powerful libraries like [Gymnasium](https://gymnasium.farama.org/) and [PyTorch](https://pytorch.org/). It is home to a new type of RL agent called ***NeuroFlow*** (NF) that specializes in Autonomous Cyber Defence through a novel Deep Reinforcement Learning (RL) approach we call ***Liquid RL***.
1616

17-
Instead of Fully-connected Networks, Velora combines [Liquid Neural Networks](https://arxiv.org/abs/2006.04439) (LNNs) with [Neural Circuit Policies](https://arxiv.org/abs/1803.08554) (NCPs), specifically [Ordinary Neural Circuits](https://proceedings.mlr.press/v119/hasani20a.html) (ONCs).
17+
## Benefits
1818

19-
These two components have interesting benefits:
20-
21-
- LNNs are a powerful RNN architecture that learns system dynamics, not just data patterns.
22-
- NCPs focus on sparsely connected neurons with distinct functions, mimicking biological behaviour.
23-
24-
From what we've seen, these networks are powerful, small-scale architectures that excel in model explainability, making them perfect for control tasks.
25-
26-
Velora offers Liquidfied PyTorch-based implementations of RL algorithms, designed to be intuitive, easy to use, and customizable.
27-
28-
In other frameworks, we've seen a trend of heavy abstraction in favour of minimal lines of code. Our approach aims to offer a best of both worlds, abstracting code away but making the details explainable on the backend, while giving you the freedom to customize as needed.
19+
- **Explainability**: NF agents use [Liquid Neural Networks](https://arxiv.org/abs/2006.04439) (LNNs) and [Neural Circuit Policies](https://arxiv.org/abs/1803.08554) (NCPs) to model Cyber system dynamics, not just data patterns. Also, they use sparse NCP connections to mimic biological efficiency, enabling clear, interpretable strategies via a labeled Strategy Library.
20+
- **Adaptability**: NF agents dynamically grow their networks using a fitness score, adding more neurons to a backbone only when new Cyber strategies emerge, keeping agents compact and robust.
21+
- **Planning**: NF agents use a Strategy Library and learned environment model to plan strategic sequences for proactive Cyber defense.
22+
- **Always Learning**: using [EWC](https://arxiv.org/abs/1612.00796), NF agents refine existing strategies and learn new ones post-training, adapting to evolving Cyber threats like new attack patterns.
23+
- **Customizable**: NF agents are PyTorch-based, designed to be intuitive, easy to use, and modular so you can easily build your own!
2924

3025
## Installation
3126

32-
To get started, simply install it through [pip](https://pypi.org/) using one of the options below.
27+
To get started, simply install it through [pip](https://pypi.org/project/velora) using one of the options below.
3328

3429
### GPU Enabled
3530

3631
For [PyTorch](https://pytorch.org/get-started/locally/) with CUDA (recommended):
3732

3833
```bash
39-
pip install torch torchvision velora --extra-index-url https://download.pytorch.org/whl/cu124
34+
pip install torch torchvision velora --extra-index-url https://download.pytorch.org/whl/cu126
4035
```
4136

4237
### CPU Only
@@ -52,53 +47,34 @@ pip install torch torchvision velora
5247
Here's a simple example that should work 'as is':
5348

5449
```python
55-
from functools import partial
56-
57-
from velora.models import LiquidDDPG
58-
from velora.gym import wrap_gym_env
59-
from velora.utils import set_device, set_seed
60-
61-
import gymnasium as gym
62-
from gymnasium.wrappers import NormalizeObservation, NormalizeReward, ClipReward
63-
64-
# Setup reproducibility and PyTorch device
65-
seed = 64
66-
set_seed(seed)
50+
from velora.models import NeuroFlow, NeuroFlowCT
51+
from velora.utils import set_device
6752

53+
# Setup PyTorch device
6854
device = set_device()
6955

70-
# Add extra wrappers to our environment
71-
env = wrap_gym_env("InvertedPendulum-v5", [
72-
partial(NormalizeObservation, epsilon=1e-8),
73-
partial(NormalizeReward, gamma=0.99, epsilon=1e-8),
74-
partial(ClipReward, max_reward=10.0),
75-
# RecordEpisodeStatistics, # Applied automatically!
76-
# partial(NumpyToTorch, device=device), # Applied automatically!
77-
])
78-
79-
# Or, use the standard gym API (recommended for this env)
80-
env = gym.make("InvertedPendulum-v5")
81-
82-
# Set core variables
83-
state_dim = env.observation_space.shape[0] # in features
84-
n_neurons = 20 # decision/hidden nodes
85-
action_dim = env.action_space.shape[0] # out features
86-
87-
buffer_size = 100_000
88-
batch_size = 128
89-
90-
# Train a model
91-
model = LiquidDDPG(
92-
state_dim,
93-
n_neurons,
94-
action_dim,
95-
buffer_size=buffer_size,
56+
# For continuous tasks
57+
model = NeuroFlowCT(
58+
"InvertedPendulum-v5",
59+
20, # actor neurons
60+
128, # critic neurons
9661
device=device,
62+
seed=64, # remove for automatic generation
9763
)
98-
model.train(env, batch_size, n_episodes=300)
64+
65+
# For discrete tasks
66+
model = NeuroFlow(
67+
"CartPole-v1",
68+
20, # actor neurons
69+
128, # critic neurons
70+
device=device,
71+
)
72+
73+
# Train the model using a batch size of 64
74+
model.train(64, n_episodes=50, display_count=10)
9975
```
10076

101-
Currently, the framework only supports [Gymnasium](https://gymnasium.farama.org/) environments and is planned to expand to [PettingZoo](https://pettingzoo.farama.org/index.html) for Multi-agent (MARL) tasks.
77+
Currently, the framework only supports [Gymnasium](https://gymnasium.farama.org/) environments and is planned to expand to [PettingZoo](https://pettingzoo.farama.org/index.html) for Multi-agent (MARL) tasks, with updated adaptations of [CybORG](https://github.com/cage-challenge/CybORG/tree/main) environments.
10278

10379
## API Structure
10480

@@ -122,16 +98,10 @@ from velora.gym import [method]
12298
from velora.utils import [method]
12399
```
124100

125-
## Customization
126-
127-
Customization is at the heart of Velora but requires a deeper understanding of the API.
128-
129-
You can read more about it in the [documentation tutorials](https://velora.achronus.dev/learn/customize).
130-
131101
## Active Development
132102

133103
🚧 View the [Roadmap](https://velora.achronus.dev/starting/roadmap) 🚧
134104

135-
**Velora** is a tool that is continuously being developed. There's still a lot to do to make it a fully functioning framework, such as detailed API documentation, and more RL algorithms.
105+
**Velora** is a tool that is continuously being developed. There's still a lot to do to make it a great framework, such as detailed API documentation, and expanding our NeuroFlow agents.
136106

137107
Our goal is to provide a quality open-source product that works 'out-of-the-box' that everyone can experiment with, and then gradually fix unexpected bugs and introduce more features on the road to a `v1` release.

docs/changelog/v0.2.0.md

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
# v0.2.0 - 2025-04-22
2+
3+
## 🚀 Features
4+
5+
- *(callbacks)* Enhanced callbacks for flexibility.
6+
- *(force)* Added `force` flag to save methods for file overwrites.
7+
- *(ppo)* Added `LiquidPPO` algorithm.
8+
- *(handler)* Added file saving for training completion details.
9+
- *(ncp)* Added multiple weight initialization options.
10+
- *(sac)* Added `LiquidSAC` agent for continuous action spaces.
11+
- *(sac)* Added `LiquidSACDiscrete` agent for discrete action spaces.
12+
- *(neuroflow)* Added main logic for `NeuroFlow`.
13+
- *(agent)* Added `NeuroFlowDiscrete` agent.
14+
15+
## 🐛 Bug Fixes
16+
17+
- *(ddpg)* Fixed noise handling and prediction bugs.
18+
- *(cell)* Fixed `sparsity_mask` assignment bug.
19+
- *(params)* Fixed parameter counts in in DDPG.
20+
- *(ppo)* Fixed PPO callback bugs and metric tracking.
21+
- *(config)* Fixed bug with `train_params` in `RLAgentConfig`.
22+
- *(load)* Fixed model loading bug.
23+
- *(buffer)* Fixed `warm` method bug when `num_envs=1`.
24+
- *(buffer)* Fixed save bug where directories don't exist.
25+
26+
## 💼 Other
27+
28+
- *(box)* Added Gymnasium box2d environments by default.
29+
30+
## 🚜 Refactor
31+
32+
- *(ncp)* Added `update_mask` helper methods.
33+
- *(metrics)* Updated training metrics name for clarity.
34+
- *(metrics)* Simplified metric classes using base class.
35+
- *(train)* Refactored `TrainHandler`, `TrainConfig` to simplify.
36+
- *(buffer)* Added Actor hidden state to buffer.
37+
- *(seed)* Improved random seed generation.
38+
- *(save)* Simplified `save`, `load` method implementations.
39+
- *(sac)* Moved `SAC` agents to separate folder for simplicity.
40+
- *(ncp)* Renamed `NCPModule` -> `LiquidNCPModule` for clarity.
41+
- *(agents)* Refactored framework to centre around `NeuroFlow`.
42+
- *(save)* Moved `completed.json` to save directory.
43+
- *(warm)* Improved buffer warming step implementation.
44+
- *(utils)* Simplified `capture` utility methods.

docs/index.md

Lines changed: 9 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ hide:
1111

1212
<p id="slogan" align="center" markdown>
1313

14-
*Velora, a lightweight and modular <span style="color: #38e2e2;">Liquid Reinforcement Learning (RL)</span> framework.*
14+
*Velora, a <span style="color: #38e2e2;">Liquid RL</span> framework for <span style="color: #38e2e2;">NeuroFlow</span> agents, empowering <span style="color: #38e2e2;">Autonomous Cyber Defence</span>.*
1515

1616
</p>
1717

@@ -30,20 +30,15 @@ hide:
3030

3131
---
3232

33-
**Velora** is a lightweight and extensible framework built on top of powerful libraries like [Gymnasium [:material-arrow-right-bottom:]](https://gymnasium.farama.org/) and [PyTorch [:material-arrow-right-bottom:]](https://pytorch.org/), specializing in a unique approach to Deep Reinforcement Learning (RL) algorithms, a paradigm we call *Liquid RL*.
33+
**Velora** is a lightweight and modular framework built on top of powerful libraries like [Gymnasium [:material-arrow-right-bottom:]](https://gymnasium.farama.org/) and [PyTorch [:material-arrow-right-bottom:]](https://pytorch.org/). It is home to a new type of RL agent called ***NeuroFlow*** (NF) that specializes in Autonomous Cyber Defence through a novel Deep Reinforcement Learning (RL) approach we call ***Liquid RL***.
3434

35-
Instead of Fully-connected Networks, Velora combines [Liquid Neural Networks [:material-arrow-right-bottom:]](https://arxiv.org/abs/2006.04439) (LNNs) with [Neural Circuit Policies [:material-arrow-right-bottom:]](https://arxiv.org/abs/1803.08554) (NCPs), specifically [Ordinary Neural Circuits [:material-arrow-right-bottom:]](https://proceedings.mlr.press/v119/hasani20a.html) (ONCs).
35+
## Benefits
3636

37-
These two components have interesting benefits:
38-
39-
- LNNs are a powerful RNN architecture that learns system dynamics, not just data patterns.
40-
- NCPs focus on sparsely connected neurons with distinct functions, mimicking biological behaviour.
41-
42-
From what we've seen, these networks are powerful, small-scale architectures that excel in model explainability, making them perfect for control tasks.
43-
44-
Velora offers Liquidfied PyTorch-based implementations of RL algorithms, designed to be intuitive, easy to use, and customizable.
45-
46-
In other frameworks, we've seen a trend of heavy abstraction in favour of minimal lines of code. Our approach aims to offer a best of both worlds, abstracting code away but making the details explainable on the backend, while giving you the freedom to customize as needed.
37+
- **Explainability**: NF agents use [Liquid Neural Networks [:material-arrow-right-bottom:]](https://arxiv.org/abs/2006.04439) (LNNs) and [Neural Circuit Policies [:material-arrow-right-bottom:]](https://arxiv.org/abs/1803.08554) (NCPs) to model Cyber system dynamics, not just data patterns. Also, they use sparse NCP connections to mimic biological efficiency, enabling clear, interpretable strategies via a labeled Strategy Library.
38+
- **Adaptability**: NF agents dynamically grow their networks using a fitness score, adding more neurons to a backbone only when new Cyber strategies emerge, keeping agents compact and robust.
39+
- **Planning**: NF agents use a Strategy Library and learned environment model to plan strategic sequences for proactive Cyber defense.
40+
- **Always Learning**: using [EWC [:material-arrow-right-bottom:]](https://arxiv.org/abs/1612.00796), NF agents refine existing strategies and learn new ones post-training, adapting to evolving Cyber threats like new attack patterns.
41+
- **Customizable**: NF agents are [PyTorch-based [:material-arrow-right-bottom:]](https://pytorch.org/), designed to be intuitive, easy to use, and modular so you can easily build your own!
4742

4843
<div class="grid cards" markdown>
4944

@@ -67,7 +62,7 @@ In other frameworks, we've seen a trend of heavy abstraction in favour of minima
6762

6863
## Active Development
6964

70-
**Velora** is a tool that is continuously being developed. There's still a lot to do to make it a fully functioning framework, such as detailed API documentation, and more RL algorithms.
65+
**Velora** is a tool that is continuously being developed. There's still a lot to do to make it a great framework, such as detailed API documentation, and expanding our NeuroFlow agents.
7166

7267
Our goal is to provide a quality open-source product that works 'out-of-the-box' that everyone can experiment with, and then gradually fix unexpected bugs and introduce more features on the road to a [`v1`](#active-development) release.
7368

docs/learn/customize/acs.md

Lines changed: 0 additions & 47 deletions
This file was deleted.

docs/learn/customize/backbone.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Velora has two prebuilt options for this: an `MLP` and a `BasicCNN`.
88

99
???+ api "API Docs"
1010

11-
[`velora.models.backbone.MLP`](../reference/models/backbone.md#velora.models.backbone.MLP)
11+
[`velora.models.backbone.MLP(in_features, n_hidden, out_features)`](../reference/models/backbone.md#velora.models.backbone.MLP)
1212

1313
The `MLP` is a dynamic class for building Multi-layer Perceptron Networks - the traditional fully-connected neuron architecture.
1414

@@ -50,19 +50,19 @@ This code should work 'as is'.
5050

5151
???+ api "API Docs"
5252

53-
[`velora.models.backbone.BasicCNN`](../reference/models/backbone.md#velora.models.backbone.BasicCNN)
53+
[`velora.models.backbone.BasicCNN(in_channels)`](../reference/models/backbone.md#velora.models.backbone.BasicCNN)
5454

5555
The `BasicCNN` uses a static architecture from the DQN Nature paper: [Human-level control through deep reinforcement learning [:material-arrow-right-bottom:]](https://www.nature.com/articles/nature14236).
5656

5757
The paper used it for Atari games, but has been adopted in other libraries such as [Stable-Baselines3 [:material-arrow-right-bottom:]](https://stable-baselines3.readthedocs.io/en/master/index.html) as a go-to CNN architecture, so we thought we'd use the same one! 😊
5858

5959
As an added bonus, it makes things easier for comparing SB3 baselines with our algorithms 😉.
6060

61-
???+ note "Backbones with Velora algorithms"
61+
???+ abstract "Backbones with Velora agents"
6262

63-
Currently, Velora doesn't directly use backbones in it's prebuilt algorithms, they are strictly LNN architectures. So, you need to manually apply them yourself (we'll show you how to do this shortly).
63+
Currently, Velora doesn't directly use backbones in it's agents, they are strictly LNN or NCP architectures. So, you need to manually apply them yourself (we'll show you how to do this shortly).
6464

65-
We plan to change this in the future, but right now we are focusing on building a robust baseline for our algorithms.
65+
Typically, cyber environments don't use images as inputs, so we have no intention of changing this.
6666

6767
To use the `BasicCNN` architecture, we pass in the number of `in_channels` and then can call the `forward()` or `out_size()` methods:
6868

0 commit comments

Comments
 (0)