Optimal Learning Rate

[david-ben-gurion]

While finding the optimal learning rate, we use the following code:
lrs = 1e-4 * (10**(tf.range(20)/20))
plt.semilogx(lrs, hist1.history["loss"])
plt.xlabel("Learning Rate")
plt.ylabel("Loss");
the output of this:

However, when i change the first line to: lrs = 1e-8 instead of 4, or any change it to any number for that matter, the output plot remains the same as shown above with the only difference being 1e-8 plotted on the x-axis currently where 1e-4 sits on the shown output. This consequently indicates that around 1e-8 is also the optimal learning rate, which is wrong. I copied the code from the lecture notes as well.
Know what I might be doing wrong here? Hope this makes sense.

[SaketMunda]

While plotting Learning Rate (lrs) vs. Loss (hist1.history['loss']), make sure that in both places you use the same lrs.
First, while creating a learning rate schedular callback,
lr_scheduler = tf.keras.callbacks.LearningRateScheduler(lambda epoch: 1e-4 * 10**(epoch/20))
Second, while plotting the learning rate vs loss
lrs = 1e-4 * (10**(tf.range(20)/20))

In both places I used 1e-4

Comparing the results (used the same code as did in lecture),
For 1e-8

For 1e-4

I hope this could be useful.