Geitje.yaml

---
# Thank you for contributing!
# In filling out this yaml file, please follow the criteria as described here:
# https://osai-index.eu/contribute

# You're free to build on this work and reuse the data. It is licensed under CC-BY 4.0, with the
# stipulation that attribution should come in the form of a link to https://osai-index.eu/
# and a citation to the peer-reviewed paper in which the dataset & criteria were published:

# Liesenfeld, A. and Dingemanse, M., 2024. Rethinking open source generative AI: open-washing and the EU AI Act. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (pp. 1774-1787).

# Organization tags:
# - National origin: Netherlands
# - Contributor type: Academic (Individual)

system:
    name: Geitje
    link: https://huggingface.co/BramVanroy/GEITje-7B-ultra
    type: text
    performanceclass: limited
    basemodelname: Mistral-7B-v0.1
    endmodelname: GEITje-7B-ultra
    endmodellicense: CC-BY-ND
    releasedate: 2024-01
    notes: Dutch instruction-tuned model based on Mistral 7B

org:
    name: Bram Vanroy
    link: https://bramvanroy.github.io/
    notes: Independent model creator.

# availability:
datasources_basemodel:
    class: partial
    link: ["https://huggingface.co/mistralai/Mistral-7B-v0.1/discussions/8", "https://huggingface.co/Rijgersberg/GEITje-7B#geitje--trained-further-on-dutch-texts"]
    notes: Mistral provides no documentation of any of its pretraining data. Geitje Ultra 7B is based on Geitje 7B, which does disclose that Dutch pretraining data includes Gigacorpus and MADLAD.

datasources_endmodel:
    class: open
    link: https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch
    notes: Ultrafeedback Dutch (synthetic)

weights_basemodel:
    class: open
    link: https://huggingface.co/mistralai/Mistral-7B-v0.1
    notes: Weights available through HuggingFace.

weights_endmodel:
    class: closed
    link: https://huggingface.co/BramVanroy/GEITje-7B-ultra/tree/main
    notes: Instruction-tuned model formerly made available through HuggingFace, now taken down

trainingcode:
    class: closed
    link: https://huggingface.co/BramVanroy/GEITje-7B-ultra#training-procedure
    notes: Mistral has limited source code available. No training code for Geitje found apart from the recipe used with the alignment handbook.

# documentation:
code:
    class: partial
    link: https://github.com/mistralai/mistral-inference/tree/main/src/mistral_inference
    notes: collection of code on github repository, although not very clearly documented and commented.

hardware_architecture:
    class: partial
    link: https://huggingface.co/BramVanroy/GEITje-7B-ultra
    notes: Some information on architecture provided in github repo and HF model card. Training was done using alignment handbook.

preprint:
    class: open
    link: https://arxiv.org/abs/2312.12852v1
    notes: Preprint published on arXiv.

paper:
    class: closed
    link:
    notes: No peer-reviewed paper found

modelcard:
    class: partial
    link: https://huggingface.co/BramVanroy/GEITje-7B-ultra
    notes: Modelcard on HF provides information on fine-tuning but nothing for the Mistral base LLM 

datasheet:
    class: partial
    link: https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch
    notes: Datasheet available for DPO and for the Dutch portions of pretraining data, but not for original Mistral pretraining data, hence partial.

# access:
licenses:
    class: closed
    link: https://huggingface.co/BramVanroy/GEITje-7B-ultra
    notes: Licensed as CC-BY-ND-4.0 on HuggingFace, though no specific license file or statement found