sos clean --validate ?

[pmoravec]

There are various cases when obfuscation of IPs, hostnames or usernames is not pedantically applied on all instances due to insufficient tokenisation, word boundaries or sensitive words as pure substrings. The obfuscated output then can contain various sensitive leftovers.

My idea is to have a --validate option that will take sanitised sosreport directory and check if no key from obfuscation mapping is present there. Something like in bash:

for key in $(jq -r 'to_entries[] | select(.key|endswith("_map")) | .value | keys[]' /path/to/private_map); do
  grep -rn $key /path/to/sosreport-dir
done

but call it natively in Python :) and take relevant keys from ipv6_map that stores sensitive keys at different json level(s).

Does the feature sound good?

[bmr-cymru]

This sounds good from a high-level PoV: I will just add that the more I see of sos.cleaner the more I think it warrants a major overhaul.

Perhaps it would be good to catalogue the current issues somewhere and then figure out prioritisation for getting them actioned?

[pmoravec]

Sounds a great idea. I will surely add more items to the list, but from top of my (tired) head, I would highlight:

• [cleaner] newly detected items not cleaned in already checked files #4339 : we should traverse the directory tree twice; first to identify all sensitive keys to obfuscate, second to "just" apply the obfuscation. This will require nontrivial refactoring
• I am not happy with (and I think neither @TurboTurtle is) with the "class layout" that each obfuscation type has its own prepper+parser+mapping, but the classes are not fully in par.

I think work on both issues shall be coordinated.