Rayhunter spontaneously stopped working - TomlError { message: "expected ., =", original ...

[wdormann]

I had a working Rayhunter on an Orbic device, and after a few power cycles, there was no colored bar at the top of the screen. Attempting to run rayhunter-daemon manually resulted in:

/ $ /data/rayhunter/rayhunter-daemon /data/rayhunter/config.toml
R A Y H U N T E R 🐳
Error: QmdlStoreError(ParseManifestError(Error { inner: Error { inner: TomlError { message: "expected `.`, `=`", original: Some("[[entries]]\nname = \"1741727884\"\nstart_time = \"2025-03-11T17:18:04.171187707-04:00\"\nlast_message_time = \"2025-03-11T17:20:24.091536717-04:00\"\nqmdl_size_bytes = 9288\nanalysis_size_bytes = 526\n\n[[entries]]\nname = \"1741728031\"\nstart_time = \"2025-03-11T17:20:31.700645829-04:00\"\nlast_message_time = \"2025-03-11T17:20:35.788914734-04:00\"\nqmdl_size_bytes = 100\nanalysis_size_bytes = 526\n\n[[entries]]\nname = \"1741728072\"\nstart_time = \"2025-03-11T17:21:12.080918332-04:00\"\nlast_message_time = \"2025-03-11T17:21:13.202049479-04:00\"\nqmdl_size_bytes = 2443\nanalysis_size_bytes = 526\n\n[[entries]]\nname = \"1741728099\"\nstart_time = \"2025-03-11T17:21:39.562296717-04:00\"\nlast_message_time = \"2025-03-11T17:21:47.083885830-04:00\"\nqmdl_size_bytes = 3250\nanalysis_size_bytes = 526\n\n[[entries]]\nname = \"1741728251\"\nstart_time = \"2025-03-11T17:24:11.465425363-04:00\"\nlast_message_time = \"2025-03-11T17:24:51.207112640-04:00\"\nqmdl_size_bytes = 3347\nanalysis_size_bytes = 526\n\n[[entries]]\nname = \"1741728312\"\nstart_time = \"2025-03-11T17:25:12.245837082-04:00\"\nlast_message_time = \"2025-03-11T17:25:30.306727649-04:00\"\nqmdl_size_bytes = 2970\nanalysis_size_bytes = 526\n\n[[entries]]\nname = \"1741728364\"\nstart_time = \"2025-03-11T17:26:04.369952083-04:00\"\nlast_message_time = \"2025-03-11T17:37:10.367058652-04:00\"\nqmdl_size_bytes = 13075\nanalysis_size_bytes = 526\n\n[[entries]]\nname = \"1741729051\"\nstart_time = \"2025-03-11T17:37:31.742871144-04:00\"\nlast_message_time = \"2025-03-11T22:50:43.807485487-04:00\"\nqmdl_size_bytes = 59151\nanalysis_size_bytes = 526\n\n[[entries]]\nname = \"1741747895\"\nstart_time = \"2025-03-11T22:51:35.221896249-04:00\"\nlast_message_time = \"2025-03-11T22:51:42.745101299-04:00\"\nqmdl_size_bytes = 2208\nanalysis_size_bytes = 526\n\n[[entries]]\nname = \"1741747924\"\nstart_time = \"2025-03-11T22:52:04.385523228-04:00\"\nlast_message_time = \"2025-03-12T11:49:50.726496-04:00\"\nqmdl_size_bytes = 132111\nanalysis_size_bytes = 526\n26\n"), keys: [], span: Some(1911..1912) } } }))
/ $ 

An attempt to reinstall Rayhunter failed:

 $ ./install-mac.sh                                                                                                                                                                                  
Using adb at /Users/wd/bin/adb
Force a switch into the debug mode to enable ADB
Device already in command mode. Doing nothing...
adb enabled, waiting for reboot... it's alive!
waiting for atfwd_daemon to startup... done!
./rootshell: 1 file pushed, 0 skipped. 3.5 MB/s (927624 bytes in 0.254s)
uid=0(root) gid=0(root)
we have root!
./config.toml.example: 1 file pushed, 0 skipped. 0.7 MB/s (389 bytes in 0.001s)
./rayhunter-daemon: 1 file pushed, 0 skipped. 1.4 MB/s (11619420 bytes in 8.033s)
./scripts/rayhunter_daemon: 1 file pushed, 0 skipped. 1.2 MB/s (580 bytes in 0.000s)
./scripts/misc-daemon: 1 file pushed, 0 skipped. 1.9 MB/s (2302 bytes in 0.001s)
waiting for reboot... done!
checking for rayhunter server...timeout reached! failed to reach rayhunter url http://localhost:8080, something went wrong :(
$

So presumably whatever triggered this error state is persistent across installs.
Worst case would be that the device occasionally needs a reinstall of rayhunter and that the reinstall fixes the error state.
Best case would be that the error state is never entered.

[wdormann]

FWIW, manually adb shelling into the device and renaming the /data/rayhunter/qmdl directory and recreating it addressed the problem for me.

Attached is the problematic qmdl directory tarball.

qmdl.tgz

[m0veax]

Could you try clearing your qmdl log directory? Move the files inside to another place and try it again please.

[wdormann]

Note that the faulty manifest.toml file has a stray value at the end without an associated =:

[[entries]]
name = "1741747924"
start_time = "2025-03-11T22:52:04.385523228-04:00"
last_message_time = "2025-03-12T11:49:50.726496-04:00"
qmdl_size_bytes = 132111
analysis_size_bytes = 526
26

[wgreenberg]

hmm that's very odd, i wonder if maybe the daemon somehow shutdown mid-write to the manifest? i can't imagine what else might've caused a stray int to be there

[m0veax]

I ran into the same error, after not properly stopping the service (as is removing the battery). Clearing the log directory helped and everything worked fine afterwards.

[wgreenberg]

added a PR which lets rayhunter gracefully recover by simply creating a new (empty) manifest in this case. i think a better option would be to attempt to recover the old manifest based on the state of the directory (i.e. create a new manifest entry for each QMDL file in it), but for now this'll prevent rayhunter from endlessly crashing.