Skip to content

sotired/RWKV-LM

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

842 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

All Copyright belongs to BlinkDL (rwkv.com)

Just a study and personal use of RWKV-v7

  • check folder RWKV_v7_soTier
  • run on laptop
  • Small set of data
  • Windows 11

About

FROM github/BlinkDL RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages

  • Python 86.4%
  • Cuda 9.1%
  • C++ 2.5%
  • Shell 2.0%