You need to `hi` `CursorLineNr` so that it doesn't persist from other colorschemes and is set appropriately for this colorscheme.