Quantized Llama model for Question Answering

[AfoDubhashi]

I tried using AutoModelforQuestionAnswering.from_pretrained instead of AutoModelforCausalLM. It seems like this doesnt work. Is this this because Llama 3.1 does not have bidirectional attention or am I doing something wrong? Thank you!