Adaptive Elastic Learning with Orthogonal Robust Units
Beyond Low-Rank: Dynamic Plasticity through Amplitude-Direction Decoupling
AeloRu is a research framework for investigating semantic structures in PEFT weight matrices and developing next-generation adaptive low-rank update methods. It combines HiRA, DoRA, Hebbian learning, and asynchronous architectures to enable real-time model alignment with dynamic plasticity.
Core Research Question: How can integrating Hebbian Learning principles with LoRA-based parameter-efficient fine-tuning mitigate catastrophic forgetting during long-term sequential task learning, while enhancing adaptation speed in short-term scenarios?
Research Objectives(the latest progress)
| Phase | Objective | Status |
|---|---|---|
| P0 | Hidden state semantic analysis | ✅ Done(Link) |
| P1 | HiRA-DoRA fusion implementation | ✅ Done(Link) |
| P2 | Hebbian-RL hybrid learning | ✅ Done(Link) |
| P3 | Training on real data & writing paper | 🔄 Doing |
| P4 | Asynchronous PEFT architecture | 📋 Planned |
- First HiRA-DoRA fusion with amplitude-direction decoupling on modulation terms
- Hebbian-RL hybrid learning for real-time adaptive plasticity
- Adapter-level asynchrony enabling zero-latency serving during continuous learning
MIT License - see LICENSE for details.