-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathEpiRank_GUI.py
More file actions
3451 lines (2990 loc) · 163 KB
/
Copy pathEpiRank_GUI.py
File metadata and controls
3451 lines (2990 loc) · 163 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
# coding=utf-8
"""
EpiRank GUI — Epidemic Risk Analysis System (PySide6)
Implements the EpiRank algorithm described in:
Chung-Yuan Huang et al., "EpiRank: Modeling Bidirectional Disease Spread
in Asymmetric Commuting Networks", 2019.
The EpiRank model estimates the relative epidemic risk of 353 townships in
Taiwan by combining a bidirectional commuting network with a PageRank-like
iterative algorithm. The core formula (Eq. 4–6 of the paper) is:
ER = (1 − d) · exFac + d · (daytime · W^T · ER + (1 − daytime) · W · ER)
where
ER = epidemic risk vector (N×1), sums to 1.0 at convergence
d = damping factor (default 0.95); higher → network dominates
daytime = forward/backward movement weight (0–1);
0.0 = backward only (evening commute home → residence),
0.5 = bidirectional (equal weight),
1.0 = forward only (morning commute home → workplace)
W = column-normalised OD (origin-destination) commuting matrix
W^T = column-normalised transpose of the raw OD matrix
exFac = external factor vector (default uniform 1/N)
Classification uses the head/tail breaks method (Jiang 2013), recursively
splitting at the mean three times to produce four levels:
NC (non-core) → C-III → C-II → C-I (highest risk).
The GUI reproduces all key figures/tables from the paper:
Tab 0 Results Table – ranked EpiRank scores for all townships
Tab 1 Network Map – commuting network visualisation
Tab 2 Core Classification – Table 1 (head/tail break counts by method)
Tab 3 Correlations – Table 2 (Pearson/Spearman/Recall/Precision)
Tab 4 Commuter Flow – Figure 2 (7 sub-plots: map, scatter, hist)
Tab 5 Frequency Distributions – Figure 3 (disease frequency + log ratio)
Tab 6 Frequency Distribution – Figure 6 (EpiRank freq. by daytime)
Tab 7 EpiRank vs Disease – Figure 9 (stacked % bars)
Tab 8 Index Comparison – Figure 10 (EpiRank vs PageRank vs HITS)
Tab 9 Disease Map – Figure 4 (spatial disease severity)
Tab 10 EpiRank Map – Figure 7 (spatial EpiRank levels)
Tab 11 EpiRank vs Disease Map – Figure 8 (overlay: prediction vs actual)
Tab 12 Log – computation log
Tab 13 Sensitivity Analysis – Figure 11 (daytime × d heatmaps)
Modernised from ERA.py (Python 2.7, 崇源) → Python 3.13 / PySide6 / NumPy.
"""
import sys
import os
import numpy as np
import networkx as nx
from scipy import stats as st
from openpyxl import load_workbook, Workbook
from openpyxl.styles import Font, Alignment
from PySide6.QtWidgets import (
QApplication, QMainWindow, QWidget, QVBoxLayout, QHBoxLayout,
QGroupBox, QLabel, QDoubleSpinBox, QSpinBox, QPushButton,
QFileDialog, QTextEdit, QTabWidget, QProgressBar,
QTableWidget, QTableWidgetItem, QMessageBox,
QFormLayout, QComboBox, QStatusBar
)
from PySide6.QtCore import Qt, QThread, Signal
from PySide6.QtGui import QFont, QColor, QAction
import matplotlib
from matplotlib.backends.backend_qtagg import FigureCanvasQTAgg as FigureCanvas
from matplotlib.backends.backend_qtagg import NavigationToolbar2QT as NavigationToolbar
from matplotlib.figure import Figure
from matplotlib import font_manager
# ---- Configure Matplotlib for CJK (Traditional Chinese) support ----
# Try fonts in order of preference on macOS
_CJK_FONT_CANDIDATES = [
'Heiti TC', # macOS built-in Traditional Chinese
'PingFang HK', # macOS PingFang
'Arial Unicode MS', # Wide Unicode coverage
'Hiragino Sans', # macOS Hiragino
'Noto Sans CJK TC', # Google Noto (if installed)
]
_cjk_font_found = None
_available_font_names = {f.name for f in font_manager.fontManager.ttflist}
for _candidate in _CJK_FONT_CANDIDATES:
if _candidate in _available_font_names:
_cjk_font_found = _candidate
break
if _cjk_font_found:
matplotlib.rcParams['font.sans-serif'] = [_cjk_font_found] + matplotlib.rcParams.get('font.sans-serif', [])
matplotlib.rcParams['font.family'] = 'sans-serif'
matplotlib.rcParams['axes.unicode_minus'] = False # Fix minus sign display
print(f"[Matplotlib] Using CJK font: {_cjk_font_found}")
else:
print("[Matplotlib] WARNING: No CJK font found. Chinese characters may not display correctly.")
# ============================================================
# Constants — dictionary keys for town_data records
# Each township (鄉鎮市) is stored as town_data[db_ID] = {key: value, ...}
# ============================================================
KEY_POST_CODE = 'post_code'
KEY_DB_ID = 'db_ID'
KEY_COUNTY = 'county'
KEY_TOWN = 'town'
KEY_POS_XY = 'pos_xy'
KEY_POPULATION = 'population'
KEY_AREA = 'area'
KEY_DENSITY = 'density'
KEY_NORMALIZED_DENSITY = 'normalized_density'
KEY_AGE_0_14 = 'age_0_14'
KEY_AGE_15_64 = 'age_15_64'
KEY_AGE_65 = 'age_65'
KEY_LOCAL_COMMUTER_TYPE1 = 'local_commuter_type1'
KEY_OUT_COMMUTER_TYPE1 = 'out_commuter_type1'
KEY_IN_COMMUTER_TYPE1 = 'in_commuter_type1'
KEY_COMMUTER_TYPE1 = 'commuter_type1'
KEY_RAILROAD_ZONE = 'railroad_zone'
KEY_FLU_TOTAL_CASES = 'flu_total_cases'
KEY_EV_AVERAGE_CASES = 'EV_average_cases'
KEY_SARS_TOTAL_CASES = 'sars_total_cases'
# Greater Taipei Metropolitan Area (大台北都會區): 48 townships by db_ID.
# Used for regional zoom views in Figures 7–8 and gTaipei SARS correlation
# analysis. The set covers Taipei City (台北市), New Taipei City (新北市),
# and Keelung City (基隆市) administrative divisions.
GTAIPEI_DB_IDS = set(range(0, 29)) | set(range(303, 310)) | set(range(330, 397))
# ============================================================
# Core EpiRank Engine (modernized from ERA.py)
# ============================================================
#
# 演算法總覽 (Algorithm Overview)
# ──────────────────────────────
# EpiRank 的核心洞見:傳染病的擴散並非隨機,而是沿著人類每日通勤的
# 路徑流動。每天早晨,數百萬人從住家(origin)移動到工作地(destination),
# 晚間再返回——這條雙向的人流,就是疾病傳播的高速公路。
#
# The key insight of EpiRank: epidemic spread is not random — it flows
# along the daily commuting paths of millions of people. Every morning,
# commuters move from home (origin) to work (destination); every evening,
# they return. This bidirectional human flow is the highway of disease
# transmission.
#
# EpiRank 借鏡 Google PageRank 的精神:一個網頁的重要性取決於「誰連結
# 到它」;同理,一個鄉鎮的疫情風險取決於「誰通勤到這裡、誰從這裡回家」。
# 但 EpiRank 比 PageRank 更進一步——它同時考慮了「去程」(早晨通勤,
# 人口從住家擴散到工作地)和「回程」(晚間通勤,人口從工作地回流到住家),
# 用 daytime 參數控制兩個方向的權重。
#
# EpiRank borrows from Google's PageRank philosophy: a webpage's
# importance depends on "who links to it"; analogously, a township's
# epidemic risk depends on "who commutes here and who returns home from
# here." But EpiRank goes further — it simultaneously models the
# *forward trip* (morning: population spreads from home to work) and the
# *return trip* (evening: population flows back from work to home), with
# the ``daytime`` parameter controlling the balance between the two.
#
# 三段式建構流程 (Three-Stage Construction)
# ──────────────────────────────────────────
# Stage 1 — 建網 (Network Construction)
# 從人口普查通勤 OD 矩陣建立 353 節點的有向圖。
# → build_commuting_network()
#
# Stage 2 — 正規化 (Matrix Normalisation)
# 將原始通勤矩陣分別做 column-normalisation,得到兩個隨機矩陣:
# W = 原始 OD 矩陣 column-normalised → 模擬「回家」方向
# W^T = OD 轉置矩陣 column-normalised → 模擬「去上班」方向
# → compute_epidemic_risk() 前半段
#
# Stage 3 — 迭代收斂 (Iterative Convergence)
# 反覆套用 EpiRank 公式直到風險向量穩定:
# ER = (1-d)·(1/N) + d·[daytime·W^T·ER + (1-daytime)·W·ER]
# 收斂後 ER 向量加總為 1.0,每個元素代表該鄉鎮的相對疫情風險。
# → compute_epidemic_risk() 後半段
#
# 為什麼這很優美?(Why is this elegant?)
# ────────────────────────────────────────
# 1. 只需一個通勤 OD 矩陣,就能預測三種不同疾病(流感、腸病毒、SARS)
# 的空間分布——說明通勤結構本身就是疫情風險的根本驅動力。
# 2. daytime 參數讓模型能區分「白天型」和「夜間型」的傳播路徑:
# daytime=0.0 → 純回程(疫情跟著人回家散播到住宅區)
# daytime=0.5 → 雙向等權(最符合現實)
# daytime=1.0 → 純去程(疫情在工作地累積)
# 3. 數學上保證收斂(column-stochastic matrix 的 power iteration),
# 且收斂速度由 damping factor d 控制。
# ============================================================
def sorted_map(mapping):
"""Sort a dictionary by value descending, then by key ascending.
Used to rank townships by EpiRank score for the Results Table (Tab 0)
and Excel auto-save output.
"""
return sorted(mapping.items(), key=lambda kv: (-kv[1], kv[0]))
def build_basic_table_of_towns(town_data, path='bs.xlsx', sheet='town_data',
number_of_sub_towns=409, row_base=2):
"""Load basic township metadata (population, area, density, age structure)
from bs.xlsx.
The spreadsheet contains 409 sub-township rows which are aggregated into
353 unique townships. Only the first row per town_name is kept, which
provides the merged township-level statistics.
"""
wb = load_workbook(path, data_only=True)
s = wb[sheet]
old_town = None
for row_idx in range(number_of_sub_towns):
r = row_idx + row_base
# ── 防禦:跳過含空白儲存格的列,避免 int(None)/float(None) 崩潰 ──
cell_val = s.cell(row=r, column=1).value
if cell_val is None:
continue
db_ID = int(cell_val)
county = s.cell(row=r, column=2).value
town = s.cell(row=r, column=3).value
pos_xy = (round(float(s.cell(row=r, column=7).value or 0), 2),
round(float(s.cell(row=r, column=8).value or 0), 2))
raw_population = s.cell(row=r, column=9).value or 0
sub_percentage = float(s.cell(row=r, column=10).value or 0)
area = float(s.cell(row=r, column=12).value or 0)
density = float(s.cell(row=r, column=13).value or 0)
normalized_density = float(s.cell(row=r, column=14).value or 0)
age_0_14 = float(s.cell(row=r, column=15).value or 0)
age_15_64 = float(s.cell(row=r, column=16).value or 0)
age_65 = float(s.cell(row=r, column=17).value or 0)
# 防禦:sub_percentage 為 0 時以 raw_population 代替,避免除以零
population = (float(raw_population / sub_percentage)
if sub_percentage > 0 else float(raw_population))
if town != old_town:
town_data[db_ID] = {
KEY_COUNTY: county, KEY_TOWN: town, KEY_POS_XY: pos_xy,
KEY_POPULATION: population, KEY_AREA: area,
KEY_DENSITY: density, KEY_NORMALIZED_DENSITY: normalized_density,
KEY_AGE_0_14: age_0_14, KEY_AGE_15_64: age_15_64, KEY_AGE_65: age_65,
KEY_LOCAL_COMMUTER_TYPE1: 0, KEY_OUT_COMMUTER_TYPE1: 0,
KEY_IN_COMMUTER_TYPE1: 0, KEY_RAILROAD_ZONE: 0
}
old_town = town
wb.close()
def build_flu_reported_cases(town_data, path='Flu.xlsx', sheet='2009',
number_of_towns=353, row_base=2):
"""Load 2009 influenza case counts from Flu.xlsx into town_data.
Each township is matched by (county, town) name pair.
Source: Taiwan CDC (疾管署) yearly surveillance data.
"""
wb = load_workbook(path, data_only=True)
s = wb[sheet]
check_list = {}
for row_idx in range(number_of_towns):
r = row_idx + row_base
county = s.cell(row=r, column=1).value
town_name = s.cell(row=r, column=2).value
if not check_list:
for db_ID in town_data.keys():
check_list[(town_data[db_ID][KEY_COUNTY], town_data[db_ID][KEY_TOWN])] = db_ID
db_ID = check_list.get((county, town_name), None)
if db_ID is not None:
raw = s.cell(row=r, column=3).value
town_data[db_ID][KEY_FLU_TOTAL_CASES] = int(raw) if raw is not None else 0
wb.close()
def build_ev_reported_cases(town_data, path='ev.xlsx', sheet='2000_2008',
number_of_towns=353, row_base=2):
"""Load 2000–2008 average enterovirus case counts from ev.xlsx into
town_data. EV data is stored as a float (yearly average)."""
wb = load_workbook(path, data_only=True)
s = wb[sheet]
check_list = {}
for row_idx in range(number_of_towns):
r = row_idx + row_base
county = s.cell(row=r, column=1).value
town_name = s.cell(row=r, column=2).value
if not check_list:
for db_ID in town_data.keys():
check_list[(town_data[db_ID][KEY_COUNTY], town_data[db_ID][KEY_TOWN])] = db_ID
db_ID = check_list.get((county, town_name), None)
if db_ID is not None:
raw = s.cell(row=r, column=3).value
town_data[db_ID][KEY_EV_AVERAGE_CASES] = float(raw) if raw is not None else 0.0
wb.close()
def build_sars_reported_cases(town_data, path='SARS.xlsx', sheet='2003',
number_of_towns=353, row_base=2):
"""Load 2003 SARS case counts from SARS.xlsx into town_data.
SARS data is used for Greater Taipei correlation analysis only (not
for the full-Taiwan EpiRank vs disease comparison in the paper).
"""
wb = load_workbook(path, data_only=True)
s = wb[sheet]
check_list = {}
for row_idx in range(number_of_towns):
r = row_idx + row_base
county = s.cell(row=r, column=1).value
town_name = s.cell(row=r, column=2).value
if not check_list:
for db_ID in town_data.keys():
check_list[(town_data[db_ID][KEY_COUNTY], town_data[db_ID][KEY_TOWN])] = db_ID
db_ID = check_list.get((county, town_name), None)
if db_ID is not None:
raw = s.cell(row=r, column=3).value
town_data[db_ID][KEY_SARS_TOTAL_CASES] = int(raw) if raw is not None else 0
wb.close()
def build_commuting_network(g, town_data, path='cn.xlsx', sheet='353C',
number_of_towns=353, row_base=6, col_base=6):
"""Build the 353×353 directed commuting network from cn.xlsx.
This is Stage 1 of the EpiRank pipeline — constructing the commuting
graph that serves as the "skeleton" for disease transmission modelling.
The OD (Origin-Destination) matrix in the spreadsheet encodes the
daily commuting patterns from the 2000 Taiwan population census:
OD[i][j] = number of commuters living in township i
who work in township j
Key properties of this network:
- Directed: commuting from A→B does not imply B→A
- Weighted: edge weight = commuter count (not binary)
- Self-loops: OD[i][i] = local commuters who live and work in the
same township (typically ~84% of all commuters, per the paper)
- Asymmetric: a bedroom suburb may send 50,000 commuters to the city
centre but receive only 2,000 in return
The resulting DiGraph has ~353 nodes and ~15,000+ directed edges.
Each node carries geographic coordinates (TWD97 TM2, in metres)
for spatial plotting.
Paper Section II.A: "We obtain the number of commuting trips between
each pair of townships from the 2000 population census."
Note: ``read_only=False`` is used intentionally. The ``read_only=True``
mode uses a streaming XML parser that makes random-access .cell(row, col)
calls O(n²)-slow, causing this function to take >100 s instead of <1 s.
"""
wb = load_workbook(path, read_only=False, data_only=True)
s = wb[sheet]
cache = {}
for row_idx in range(number_of_towns):
r = row_idx + row_base
raw_seq = s.cell(row=r, column=1).value
if raw_seq is None:
continue # 防禦:跳過空白列
row_seq_no = int(raw_seq)
if row_seq_no in cache:
row_code, row_db_ID = cache[row_seq_no]
else:
row_code = str(s.cell(row=r, column=2).value or '')
raw_db = s.cell(row=r, column=3).value
if raw_db is None:
continue
row_db_ID = int(raw_db)
cache[row_seq_no] = (row_code, row_db_ID)
for col_idx in range(number_of_towns):
c = col_idx + col_base
raw_col_seq = s.cell(row=1, column=c).value
if raw_col_seq is None:
continue
col_seq_no = int(raw_col_seq)
if col_seq_no in cache:
col_code, col_db_ID = cache[col_seq_no]
else:
col_code = str(s.cell(row=2, column=c).value or '')
raw_col_db = s.cell(row=3, column=c).value
if raw_col_db is None:
continue
col_db_ID = int(raw_col_db)
cache[col_seq_no] = (col_code, col_db_ID)
raw_commuters = s.cell(row=r, column=c).value
commuters = int(raw_commuters) if raw_commuters is not None else 0
if commuters > 0:
# 防禦:確保 db_ID 存在於 town_data 中
if row_db_ID not in town_data or col_db_ID not in town_data:
continue
if not g.has_node(row_seq_no):
g.add_node(row_seq_no, post_code=row_code, db_ID=row_db_ID,
posx=town_data[row_db_ID][KEY_POS_XY][0],
posy=town_data[row_db_ID][KEY_POS_XY][1])
if not g.has_node(col_seq_no):
g.add_node(col_seq_no, post_code=col_code, db_ID=col_db_ID,
posx=town_data[col_db_ID][KEY_POS_XY][0],
posy=town_data[col_db_ID][KEY_POS_XY][1])
g.add_edge(row_seq_no, col_seq_no, weight=float(commuters),
commuter_type1=float(commuters))
town_data[row_db_ID][KEY_OUT_COMMUTER_TYPE1] += commuters
town_data[col_db_ID][KEY_IN_COMMUTER_TYPE1] += commuters
wb.close()
def get_pearson_cor(dic1, dic2):
"""Pearson correlation between two dicts sharing the same keys.
Used in Table 2 and Figure 11 sensitivity analysis to measure the
linear association between network indices and disease case counts.
前提:dic1 與 dic2 的 key 集合必須完全相同(皆來自同一個 nodes 清單)。
Precondition: dic1 and dic2 must share identical key sets (both are
built from the same ``nodes`` list in ``ComputeWorker.run``).
"""
assert set(dic1.keys()) == set(dic2.keys()), \
f"Key mismatch: {len(dic1)} vs {len(dic2)} keys"
keys = list(dic1.keys())
if len(keys) < 3:
return (float('nan'), float('nan'))
n1 = [dic1[k] for k in keys]
n2 = [dic2[k] for k in keys]
r, p = st.pearsonr(n1, n2)
return round(r, 6), round(p, 6)
def get_spearman_cor(dic1, dic2):
"""Spearman rank correlation between two dicts sharing the same keys.
Spearman is rank-based, so it captures monotonic (not necessarily
linear) relationships — more robust to outliers than Pearson.
前提:dic1 與 dic2 的 key 集合必須完全相同。
Precondition: dic1 and dic2 must share identical key sets.
"""
assert set(dic1.keys()) == set(dic2.keys()), \
f"Key mismatch: {len(dic1)} vs {len(dic2)} keys"
keys = list(dic1.keys())
if len(keys) < 3:
return (float('nan'), float('nan'))
n1 = [dic1[k] for k in keys]
n2 = [dic2[k] for k in keys]
r, p = st.spearmanr(n1, n2)
return round(r, 6), round(p, 6)
def get_kendalltau_cor(dic1, dic2):
"""Kendall's tau rank correlation between two dicts sharing the same keys.
Used in the Excel auto-save for SARS vs network index correlations.
前提:dic1 與 dic2 的 key 集合必須完全相同。
Precondition: dic1 and dic2 must share identical key sets.
"""
assert set(dic1.keys()) == set(dic2.keys()), \
f"Key mismatch: {len(dic1)} vs {len(dic2)} keys"
keys = list(dic1.keys())
if len(keys) < 3:
return (float('nan'), float('nan'))
n1 = [dic1[k] for k in keys]
n2 = [dic2[k] for k in keys]
r, p = st.kendalltau(n1, n2)
return round(r, 6), round(p, 6)
# ============================================================
# Head/Tail Breaks Classification
# ============================================================
# EpiRank 計算完成後,我們需要一個方法將連續的風險分數轉化為離散的
# 疫情嚴重度等級。本文選用 head/tail breaks(Jiang 2013)——一個
# 專為重尾分佈(heavy-tailed distribution)設計的自然分類方法。
#
# After EpiRank produces a continuous risk score for each township, we
# need a method to convert these scores into discrete severity levels.
# The paper uses head/tail breaks (Jiang 2013) — a classification
# method specifically designed for heavy-tailed distributions.
#
# 為什麼不用等距分類或分位數?因為 EpiRank 分數(以及疾病案例數)
# 呈現典型的重尾分布:大多數鄉鎮風險很低,少數鄉鎮風險極高。
# 等距分類會把幾乎所有鄉鎮歸為同一級;分位數則會強制各級人數相等,
# 忽略資料本身的自然斷點。Head/tail breaks 讓資料「自己說話」:
#
# Why not equal-interval or quantile classification? Because EpiRank
# scores (and disease case counts) follow a heavy-tailed distribution:
# most townships have very low risk, a few have extremely high risk.
# Equal-interval would lump nearly all townships into one class;
# quantiles would force equal counts per class, ignoring natural breaks
# in the data. Head/tail breaks lets the data "speak for itself":
#
# 遞迴分裂過程 (Recursive splitting):
#
# Round 1: 全部 353 townships
# ├─ tail (≤ mean₁): ~239 townships → NC (non-core)
# └─ head (> mean₁): ~114 townships ← 再分
# Round 2:
# ├─ tail (≤ mean₂): ~67 townships → C-III
# └─ head (> mean₂): ~47 townships ← 再分
# Round 3:
# ├─ tail (≤ mean₃): ~31 townships → C-II
# └─ head (> mean₃): ~16 townships → C-I (highest risk)
#
# 三次分裂 → 四個等級:NC → C-III → C-II → C-I
# 每一次分裂都在「少數高值」與「多數低值」之間找到自然斷點。
#
# Paper Section III.A: "we need to identify the core and non-core
# townships based on a given index. Head/tail breaks recursively
# partition a dataset by its mean."
# ============================================================
LEVEL_COLORS = {
'NC': '#3a8f3e', # muted green — non-core
'C-III': '#c8b840', # olive-yellow — core level III
'C-II': '#e07830', # orange — core level II
'C-I': '#cc2020', # dark red — core level I (highest)
}
LEVEL_ORDER = ['NC', 'C-III', 'C-II', 'C-I'] # drawing order: background → foreground
def head_tail_breaks(values, n_breaks=3):
"""Recursively split at the mean (head/tail breaks, Jiang 2013).
Parameters
----------
values : array-like
Numeric values to classify (e.g. EpiRank scores, disease counts).
n_breaks : int
Number of recursive splits. Default 3 → 4 output levels.
Returns
-------
list[float]
Sorted list of break points (ascending). With n_breaks=3 the
result has exactly 3 values: [mean_all, mean_head1, mean_head2].
"""
breaks = []
head = values.copy()
for _ in range(n_breaks):
if len(head) < 2:
break
m = head.mean()
breaks.append(m)
head = head[head > m]
return sorted(breaks)
def classify_by_breaks(values, breaks):
"""Classify values into NC / C-III / C-II / C-I based on head/tail breaks.
The boundary rule uses ``<=`` for the lower levels, so a value exactly
equal to a break point falls into the lower level. This is consistent
with the paper's "tail" definition (at or below the mean).
若 *breaks* 不足 3 個斷點(資料量過少或變異太小導致 head_tail_breaks
提早終止),則以最後一個斷點重複填充至 3 個,確保不會因 IndexError 崩潰。
If *breaks* has fewer than 3 elements (too few data points or low
variance), pad with the last break value to prevent IndexError.
"""
# ── 防禦:確保至少 3 個斷點 ──
if len(breaks) == 0:
# 無斷點 → 全部歸為 NC
return ['NC'] * len(values)
while len(breaks) < 3:
breaks = list(breaks) + [breaks[-1]]
labels = []
for v in values:
if v <= breaks[0]:
labels.append('NC')
elif v <= breaks[1]:
labels.append('C-III')
elif v <= breaks[2]:
labels.append('C-II')
else:
labels.append('C-I')
return labels
def compute_epidemic_risk(g, town_data, d, daytime, number_of_loops=5000,
progress_callback=None):
"""Core EpiRank algorithm — Stages 2 & 3 of the pipeline.
This function takes the raw commuting network (Stage 1 output) and
computes the stationary epidemic risk vector through two phases:
Phase A — Matrix normalisation (Stage 2)
─────────────────────────────────────────
The raw OD matrix is column-normalised into two stochastic matrices,
each capturing a different direction of disease transmission:
W = col-normalise(OD) → 「回家」backward / push direction
W^T = col-normalise(OD^T) → 「上班」forward / pull direction
Why column-normalisation? Imagine township j has 3 commuting sources:
A sends 600, B sends 300, C sends 100 commuters. Column-normalising
converts these to transition probabilities: A→j = 0.6, B→j = 0.3,
C→j = 0.1. This means 60% of j's infection risk from incoming
commuters comes from A. The absolute commuter count is factored out,
leaving the *relative connectivity structure* — exactly what matters
for epidemic spread patterns.
Phase B — Iterative convergence (Stage 3)
──────────────────────────────────────────
Starting from a uniform distribution (every township equally risky),
the algorithm repeatedly applies:
ER(t+1) = (1-d) · (1/N) + d · [daytime · W^T · ER(t)
+ (1-daytime) · W · ER(t)]
Intuition for each term:
┌──────────────────────┬─────────────────────────────────────────┐
│ (1-d) · (1/N) │ Teleportation / external factor: │
│ │ with probability (1-d), a pathogen │
│ │ arrives from an external source (e.g. │
│ │ international travel, random contact) │
│ │ regardless of the commuting network. │
│ │ This prevents isolated islands from │
│ │ having zero risk. │
├──────────────────────┼─────────────────────────────────────────┤
│ d · daytime · W^T·ER │ Forward (morning) commute contribution: │
│ │ commuters ARRIVE at their workplaces, │
│ │ carrying risk FROM their home townships.│
│ │ W^T propagates risk in the direction │
│ │ home → work. High-risk townships that │
│ │ SEND many workers raise the risk of │
│ │ the destination (pull effect). │
├──────────────────────┼─────────────────────────────────────────┤
│ d·(1-daytime)· W·ER │ Backward (evening) commute contribution:│
│ │ commuters RETURN to their residences, │
│ │ carrying risk FROM their workplaces. │
│ │ W propagates risk in the direction │
│ │ work → home. High-risk workplaces push │
│ │ disease back to the bedroom suburbs │
│ │ (push effect). │
└──────────────────────┴─────────────────────────────────────────┘
The elegance: by adjusting a single parameter ``daytime``, the model
smoothly interpolates between three epidemiologically distinct regimes:
- daytime=0.0: purely backward (evening return; disease spreads to
residential areas — like a flu brought home to family)
- daytime=0.5: bidirectional (realistic; both directions contribute
equally — the paper's recommended default)
- daytime=1.0: purely forward (morning arrival; disease accumulates
at workplaces — like a nosocomial outbreak)
╔══════════════════════════════════════════════════════════════════╗
║ 為什麼 EpiRank 保證收斂? — 數學證明 ║
║ Why is EpiRank guaranteed to converge? — Mathematical proof ║
╚══════════════════════════════════════════════════════════════════╝
令 P = α·W' + (1−α)·W,迭代公式可改寫為:
Let P = α·W' + (1−α)·W, then the iteration becomes:
ER(t) = M · ER(t−1), where M = (1−d)·E + d·P
E = (1/N)·1·1^T (uniform rank-1 matrix)
收斂性由以下四個環環相扣的性質保證:
Convergence is guaranteed by four interlocking properties:
┌─────────────────────────────────────────────────────────────────┐
│ 性質 1 (Property 1): P 是 column-stochastic 矩陣 │
│ P is column-stochastic │
│ │
│ W 和 W' 各自 column-stochastic(每行總和 = 1)。 │
│ P 是兩者的凸組合(α 和 1−α 非負、和為 1), │
│ 因此 P 本身也是 column-stochastic。 │
│ │
│ W and W' are each column-stochastic (each column sums to 1). │
│ P is their convex combination (α + (1-α) = 1), so P is also │
│ column-stochastic. │
├─────────────────────────────────────────────────────────────────┤
│ 性質 2 (Property 2): Google Matrix M 是嚴格正矩陣 │
│ The Google Matrix M is strictly positive │
│ │
│ E 的每個元素都是 (1−d)/N > 0(因為 0 < d < 1)。 │
│ 即使 P 中有零元素,加上 (1−d)·E 後,M 的每一個元素 │
│ 都嚴格大於零。M 同時也是 column-stochastic │
│ (兩個 column-stochastic 矩陣的凸組合)。 │
│ │
│ Every entry of E equals (1-d)/N > 0 (since 0 < d < 1). │
│ Even if P contains zeros, adding (1-d)·E makes every entry │
│ of M strictly positive. M is also column-stochastic (convex │
│ combination of two column-stochastic matrices). │
├─────────────────────────────────────────────────────────────────┤
│ 性質 3 (Property 3): Perron-Frobenius 定理直接適用 │
│ The Perron-Frobenius theorem applies directly │
│ │
│ M 是正的 column-stochastic 矩陣 → 不可約且非週期 │
│ • 唯一最大特徵值 λ₁ = 1 │
│ • 所有其他特徵值 |λᵢ| < 1(嚴格小於 1) │
│ • 對應 λ₁ = 1 的特徵向量即為唯一穩態分布 ER* │
│ │
│ M is a positive column-stochastic matrix → irreducible and │
│ aperiodic. │
│ • Unique dominant eigenvalue: lambda_1 = 1 │
│ • All other eigenvalues: |lambda_i| < 1 (strictly) │
│ • The eigenvector for lambda_1 = 1 is the unique │
│ stationary distribution ER* │
│ │
│ Power iteration from ANY initial vector converges to ER*. │
├─────────────────────────────────────────────────────────────────┤
│ 性質 4 (Property 4): 收斂速率是幾何級數,由 d 控制 │
│ Convergence rate is geometric, controlled by d │
│ │
│ 第二大特徵值滿足 |λ₂| <= d,因此: │
│ The second-largest eigenvalue satisfies |lambda_2| <= d: │
│ │
│ ||ER(t) - ER*|| <= d^t · ||ER(0) - ER*|| │
│ │
│ d = 0.85 → 50 次迭代後誤差衰減至 ~3e-4 │
│ d = 0.95 → 50 次迭代後誤差衰減至 ~0.077 │
│ 100 次迭代後 ~0.006 │
│ d = 0.95 (50 iters): error ~ 0.077 │
│ d = 0.95 (100 iters): error ~ 0.006 │
│ │
│ d 越大 → 網路結構影響力越大,但收斂較慢 │
│ d 越小 → 收斂越快,但結果退化為均勻分布 │
│ Larger d → more network influence, slower convergence │
│ Smaller d → faster convergence, but result → uniform │
└─────────────────────────────────────────────────────────────────┘
直覺總結 / Intuitive summary:
(1−d)·e 這一項是整個收斂的關鍵。它扮演 PageRank 中「隨機跳躍」
(teleportation) 的角色——保證每個節點在每一步都有非零機率被「造訪」,
從而消除了死胡同 (dangling nodes) 和週期性 (periodicity) 兩個阻礙
收斂的因素。只要 d < 1,矩陣 M 就是嚴格正矩陣,Perron-Frobenius
定理便給出無條件的收斂保證。
The (1-d)·e term is the key to convergence. It plays the role of
PageRank's "teleportation" — ensuring every node has a non-zero
probability of being "visited" at each step, thereby eliminating
both dangling nodes and periodicity — the two obstacles to
convergence. As long as d < 1, M is a strictly positive matrix
and the Perron-Frobenius theorem provides an unconditional
convergence guarantee.
Typically converges within 50–200 iterations for d=0.95.
Parameters
----------
g : nx.DiGraph
The commuting network built by ``build_commuting_network()``.
town_data : dict
Township metadata (not modified; used only for key lookups).
d : float
Damping factor (0, 1). Higher values (e.g. 0.95) give more
weight to the network structure; lower values (e.g. 0.50) make
the result more uniform. Analogous to PageRank's alpha.
daytime : float
Forward/backward balance (0, 1). See table above.
number_of_loops : int
Maximum iterations. Convergence typically occurs well before
this limit.
progress_callback : callable or None
Optional ``callback(current_iter, max_iter)`` for progress bars.
Returns
-------
epidemic_risk : np.ndarray, shape (N, 1)
The converged EpiRank vector. Sums to 1.0. Each element ER[i]
represents the *relative* epidemic risk of the i-th node (ordered
by ``list(g.nodes())``).
iterations : int
Actual number of iterations performed before convergence.
CN_C : np.ndarray, shape (N, N)
The raw (un-normalised) OD commuting count matrix. Returned for
use in downstream analyses (e.g. commuter flow statistics).
"""
Ncount = g.order()
# ══════════════════════════════════════════════════════════════
# Phase A: Matrix Construction & Normalisation (Stage 2)
# ══════════════════════════════════════════════════════════════
# ── Step A1: Extract the raw OD matrix from the graph (Eq. 1) ──
#
# CN_C[i,j] = number of commuters from node i to node j.
#
# The row/column ordering follows list(g.nodes()) — the insertion
# order of nodes in the DiGraph, preserved by nx.to_numpy_array().
# This ordering is CRITICAL: it must be used consistently everywhere
# that maps between matrix indices and node identifiers.
#
# Example (3 townships):
# To: A B C
# From: A [[ 800 200 50 ] ← 800 locals, 200 commute A→B
# B [ 150 600 100 ]
# C [ 30 80 500 ]]
#
CN_C = nx.to_numpy_array(g, weight=KEY_COMMUTER_TYPE1)
CN_T = CN_C.copy()
# ── Step A2: Column-normalise OD → W (backward matrix, Eq. 2) ──
#
# W[i,j] = OD[i,j] / Σ_k OD[k,j]
#
# Each column of W sums to 1.0, forming a column-stochastic matrix.
# Interpretation: W[i,j] is the probability that a commuter arriving
# at township j came from township i.
#
# W models the *backward* (evening) commuting pattern. When we
# multiply W · ER, township j's risk "flows back" to all the home
# townships i in proportion to their commuter share. This captures
# the scenario: a worker gets infected at workplace j, then carries
# the pathogen home to residence i.
#
# Continuing the example:
# Column B sums to 200 + 600 + 80 = 880
# W[:,B] = [200/880, 600/880, 80/880] = [0.227, 0.682, 0.091]
# → 68.2% of B's "backward risk" stays local, 22.7% flows to A
#
csum = CN_T.sum(axis=0)
CN = np.zeros((Ncount, Ncount))
for i in range(CN_T.shape[0]):
for j in range(CN_T.shape[1]):
s = float(csum[j])
if s > 0:
CN[i, j] = float(CN_T[i, j]) / s
# ── Step A3: Column-normalise OD^T → W^T (forward matrix, Eq. 3) ──
#
# First transpose the raw OD matrix, then column-normalise.
# CNt = col-normalise(OD^T)
#
# Interpretation: CNt[j,i] is the probability that a commuter
# leaving home township i goes to workplace township j.
#
# CNt models the *forward* (morning) commuting pattern. When we
# multiply CNt · ER, township i's risk "flows forward" to all the
# workplaces j that its residents commute to. This captures the
# scenario: infected residents of township i carry the pathogen
# to their various workplaces.
#
# Key asymmetry: W and CNt generally produce DIFFERENT risk rankings.
# A large bedroom suburb (many outgoing commuters) will have high
# risk under the backward model (W) because workers bring disease
# home. A city-centre business district (many incoming commuters)
# will have high risk under the forward model (CNt) because it
# attracts infected commuters from many sources.
#
ODt = CN_T.T
osum = ODt.sum(axis=0)
CNt = np.zeros((Ncount, Ncount))
for i in range(ODt.shape[0]):
for j in range(ODt.shape[1]):
s = float(osum[j])
if s > 0:
CNt[i, j] = float(ODt[i, j]) / s
# ══════════════════════════════════════════════════════════════
# Phase B: Iterative Power Iteration (Stage 3)
# ══════════════════════════════════════════════════════════════
# ── Step B1: Initialise ER vector and external factor (Eq. 5) ──
#
# Start with uniform distribution: every township has equal risk
# (1/N). The external factor vector is also uniform — representing
# "background noise" of infection from sources outside the commuting
# network (e.g. international travellers, random community contact).
#
# The initial distribution doesn't affect the final result (the
# stationary vector is unique), but uniform is a natural choice and
# converges faster than a random starting point.
#
other_factors = np.ones((Ncount, 1)) / float(Ncount)
epidemic_risk = np.ones((Ncount, 1)) / float(Ncount)
# ── Step B2: Iterate until convergence (Eq. 4) ──
#
# ER(t+1) = (1-d) · exFac + d · [ daytime · CNt · ER(t) ← forward
# + (1-daytime) · CN · ER(t) ] ← backward
#
# This is a power iteration on a modified stochastic matrix. At
# each step, the new risk of township i is a weighted blend of:
# 1. A uniform "teleportation" baseline [(1-d)/N]
# 2. Risk propagated forward through morning commute [d·daytime·CNt·ER]
# 3. Risk propagated backward through evening return [d·(1-daytime)·CN·ER]
#
# 收斂保證 / Convergence guarantee:
# CN (W) 和 CNt (W') 皆為 column-stochastic,其凸組合 P 亦然。
# 加入 (1-d)·e 後,迭代矩陣 M = (1-d)·E + d·P 為嚴格正矩陣。
# 由 Perron-Frobenius 定理,M 有唯一最大特徵值 λ₁=1,
# 所有 |λᵢ|<1,故 power iteration 必定收斂至唯一穩態分布。
# 收斂速率:||ER(t)-ER*|| ≤ d^t · ||ER(0)-ER*||
# (完整數學證明見上方 docstring「為什麼 EpiRank 保證收斂」一節)
#
# CN (W) and CNt (W') are column-stochastic; their convex combo
# P is too. Adding (1-d)·e makes M = (1-d)·E + d·P strictly
# positive. By Perron-Frobenius: unique λ₁=1, all |λᵢ|<1,
# so power iteration converges to the unique stationary vector.
# Rate: ||ER(t)-ER*|| <= d^t · ||ER(0)-ER*||
# (Full proof: see docstring section "Why is EpiRank guaranteed
# to converge?" above.)
#
# The ``@`` operator performs matrix multiplication (NumPy >= 1.10),
# replacing the old np.asmatrix() * pattern from the original ERA.py.
#
# 收斂判準:元素最大變化量 < 1e-12,通常 50–200 次迭代即收斂 (d=0.95)
# Convergence criterion: max element-wise change < 1e-12.
# Typical convergence: 50–200 iterations for d=0.95.
#
iterations = 0
for i in range(number_of_loops):
old_er = epidemic_risk.copy()
epidemic_risk = ((1.0 - d) * other_factors +
d * (daytime * (CNt @ epidemic_risk) +
(1.0 - daytime) * (CN @ epidemic_risk)))
iterations = i + 1
if np.allclose(epidemic_risk, old_er, atol=1e-12):
break
if progress_callback and (i % 50 == 0 or i == number_of_loops - 1):
progress_callback(i + 1, number_of_loops)
# ER 向量收斂後加總為 1.0 — 每個元素代表該鄉鎮佔全台灣疫情風險的
# 「份額」。值越大,表示該鄉鎮在通勤網絡中越處於疫情傳播的樞紐位置。
#
# After convergence, ER sums to 1.0 — each element represents the
# township's "share" of the total epidemic risk across all of Taiwan.
# Higher values indicate that the township sits at a critical hub
# in the commuting network for disease transmission.
return epidemic_risk, iterations, CN_C
# ============================================================
# Worker Thread — runs EpiRank computation off the GUI thread
# ============================================================
class ComputeWorker(QThread):
"""Background thread that loads data, builds the commuting network,
computes EpiRank (and PageRank / HITS for comparison), and emits
the complete results dict on ``finished_ok``.
The worker temporarily ``os.chdir()`` into ``data_dir`` so that all
data-loading functions can use relative paths. A ``finally`` block
ensures the original working directory is always restored.
"""
progress = Signal(int, int) # current, max
log_message = Signal(str)
finished_ok = Signal(dict) # results dict
finished_err = Signal(str)
def __init__(self, data_dir, d, daytime, max_loops):
super().__init__()
self.data_dir = data_dir
self.d = d
self.daytime = daytime
self.max_loops = max_loops
def run(self):
old_cwd = os.getcwd()
try:
os.chdir(self.data_dir)
# ── Stage 1: Load all data files ──
town_data = {}
g = nx.DiGraph()
self.log_message.emit("Loading basic town data (bs.xlsx)...")
build_basic_table_of_towns(town_data)
self.log_message.emit(f" Loaded {len(town_data)} towns.")
self.log_message.emit("Loading Flu reported cases (Flu.xlsx)...")
build_flu_reported_cases(town_data)
self.log_message.emit("Loading Enterovirus reported cases (ev.xlsx)...")
build_ev_reported_cases(town_data)
self.log_message.emit("Loading SARS reported cases (SARS.xlsx)...")
build_sars_reported_cases(town_data)
self.log_message.emit("Building commuting network (cn.xlsx)... (this may take a while)")
build_commuting_network(g, town_data)
self.log_message.emit(f" Network: {g.number_of_nodes()} nodes, {g.number_of_edges()} edges")
# ── Stage 2: Compute EpiRank (main run) ──
self.log_message.emit(f"Computing EpiRank (d={self.d}, daytime={self.daytime}, max_loops={self.max_loops})...")
epidemic_risk, iterations, CN_C = compute_epidemic_risk(
g, town_data, self.d, self.daytime, self.max_loops,
progress_callback=lambda cur, mx: self.progress.emit(cur, mx)
)
self.log_message.emit(f" Converged after {iterations} iterations.")
# ── Stage 3: Compute EpiRank for 3 canonical daytime values ──
# These are needed for Figures 6, 7, and 8 which compare
# daytime = 0.0 (backward), 0.5 (bidirectional), 1.0 (forward).
self.log_message.emit("Computing EpiRank for daytime=0.0, 0.5, 1.0 (for Figure 6)...")
fig6_daytimes = [0.0, 0.5, 1.0]
fig6_data = {}
for dt in fig6_daytimes:
if abs(dt - self.daytime) < 1e-9:
# Reuse the already-computed result
fig6_data[dt] = {
'epidemic_risk': epidemic_risk,
'iterations': iterations,
}
else: