generate function with sampling without temperature rescaling

[nicolaleo]

in the "generate" function with temperature sampling and top-k sampling could be the "Sample from the distribution" indipendent from the temperature setting allowing the sampling features without the temperature rescaling ?
actually if the temperature is 0 its is always applied argmax

def generate(model, idx, max_new_tokens, context_size, temperature, top_k=None, do_sample=True):

[rasbt]

Good question. Yes, you can sample via torch.multinomial without temperature scaling (default). And you can also simulate the greedy decoding via argmax if you set the temperature to 0.

Actually, Exercise 5.2 is kind of related to what you are asking :)

What are the different combinations of settings for the generate function to force deterministic behavior, that is, disabling the random sampling such that it always produces the same outputs similar to the generate_simple function?