Arabic LLM in the Browser

This repository contains the code for our senior research project, titled "Browser-based Locally Hosted Arabic LLM Optimaization".

You will find our implementation of LLM evaluation and model compression methods such as quantization, pruning, and distillation here.

Installation

1. Clone the repository into your local environment, then move inside it.

   git clone https://github.com/S-Y-A-N/ar-llm-browser.git
   cd ar-llm-browser

2. Install the dependencies using uv (recommended).

   uv sync

   alternatively, if you don't want to install or use uv, run:

   pip install -e .

Running Evaluation

We use lm-evaluation-harness to run model evaluations.

Example for running evaluation on gemma-3-1b-it:

lm_eval --config evaluation/config/gemma-3-1b-it.yaml \ # path to YAML config file
        --tasks metabench arabicmmlu \
        --log_samples \
        --output_path results

For the YAML config file structure, refer to the configuration guide.
For a full list of arguments, refer to the CLI reference.

Compression

All compression methods can be used via our command-line utility llmini.

To view all available options, run llmini --help.

Pruning

To apply pruning to a model, run the following command:

llmini google/gemma-3-1b-it # Path to model or HF model id
       --prune sparsegpt # Choose a pruning method: 'sparsegpt', 'wanda'
       --prune-config sparsity=0.7,mask_structure=2:4 # optionally provide pruning arguments. Defaults to unstructured 50% sparsity

Note: Pruning methods implementations are provided via llm-compressor library.