Hello, I'm Dara 👋
I am a machine learning engineer, and an aspiring AGI researcher.
My work primarily evolves around foundation multimodal models, training optimization and inference optims. I am concerned about AI safety and interpretability so I occasionally do some mechnaistic intrepretability probing during the weekends and write about my findings here
- Infuse audio: A framework for aligning audio representations with the embedding space of LLMs (multimodality)
- Ablate compliance: Finding jailbreak directions within the activation subspace of a LLMs
- Flash Attention and Diffusion Kernels in Triton: Highly performant, highly optimized flash attention kernels, linear attention and diffusion models kernels
- Upcycle MoE: A framework for upcylcing any dense model to a sparse Mixture of expert arch