Commit fd8c264
fix: recover zombie processing jobs and mark failed jobs properly
People background task could leave jobs stuck in "processing" status
forever when processJob fails or writeGate deadlocks, because:
1. processJob error path never updated job status (only logged)
2. No runtime mechanism to detect stale processing jobs (only startup)
3. No panic recovery around writeGate.RLock areas
Changes:
- Add RecoverStaleProcessing to reset local processing jobs older than
30min back to queued, called in runBackground when no jobs are found
- Add markJobFailed to set job=failed and reset photo face status on
processJob error, preventing permanent "processing" state
- Add panic recovery around writeGate.RLock/processJob and
processPendingFaces to prevent RLock leaks on panic
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>1 parent 04de8e3 commit fd8c264
2 files changed
Lines changed: 70 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
37 | 41 | | |
38 | 42 | | |
39 | 43 | | |
| |||
348 | 352 | | |
349 | 353 | | |
350 | 354 | | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
972 | 972 | | |
973 | 973 | | |
974 | 974 | | |
975 | | - | |
| 975 | + | |
| 976 | + | |
| 977 | + | |
| 978 | + | |
| 979 | + | |
| 980 | + | |
| 981 | + | |
| 982 | + | |
976 | 983 | | |
977 | 984 | | |
978 | 985 | | |
979 | 986 | | |
| 987 | + | |
980 | 988 | | |
981 | 989 | | |
982 | 990 | | |
| |||
988 | 996 | | |
989 | 997 | | |
990 | 998 | | |
| 999 | + | |
| 1000 | + | |
| 1001 | + | |
| 1002 | + | |
| 1003 | + | |
| 1004 | + | |
| 1005 | + | |
| 1006 | + | |
| 1007 | + | |
991 | 1008 | | |
992 | 1009 | | |
993 | 1010 | | |
| |||
1023 | 1040 | | |
1024 | 1041 | | |
1025 | 1042 | | |
1026 | | - | |
| 1043 | + | |
| 1044 | + | |
| 1045 | + | |
| 1046 | + | |
| 1047 | + | |
| 1048 | + | |
| 1049 | + | |
| 1050 | + | |
1027 | 1051 | | |
1028 | 1052 | | |
1029 | 1053 | | |
| |||
1053 | 1077 | | |
1054 | 1078 | | |
1055 | 1079 | | |
| 1080 | + | |
| 1081 | + | |
| 1082 | + | |
| 1083 | + | |
| 1084 | + | |
| 1085 | + | |
| 1086 | + | |
| 1087 | + | |
| 1088 | + | |
| 1089 | + | |
| 1090 | + | |
| 1091 | + | |
| 1092 | + | |
| 1093 | + | |
| 1094 | + | |
| 1095 | + | |
| 1096 | + | |
| 1097 | + | |
| 1098 | + | |
| 1099 | + | |
| 1100 | + | |
| 1101 | + | |
1056 | 1102 | | |
1057 | 1103 | | |
1058 | 1104 | | |
| |||
0 commit comments