Skip to content

Commit 780d55f

Browse files
mbaaksbrugman
authored andcommitted
FIX: add preprocess_merge_abbr_v0 preprocessing option for bkw-compatibility
preprocess_merge_abbr_v0 is same as preprocess_merge_abbr v0 w/o remove_extra_space, for model bkw-compatibility. This makes it possible to use v1.3 models in emm v2.0.
1 parent 1f45540 commit 780d55f

File tree

1 file changed

+10
-0
lines changed

1 file changed

+10
-0
lines changed

emm/preprocessing/base_name_preprocessor.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,16 @@
4646
"handle_lower_trim",
4747
"remove_extra_space",
4848
],
49+
"preprocess_merge_abbr_v0": [ # preprocess_merge_abbr v0, w/o remove_extra_space, for bkw-compatibility
50+
"strip_accents_unicode",
51+
"replace_punctuation",
52+
"remove_newline",
53+
"merge_abbreviations", # merge all abbreviation
54+
"merge_&",
55+
"strip_punctuation",
56+
"handle_lower_trim",
57+
"map_shorthands",
58+
],
4959
"preprocess_merge_abbr": [
5060
"strip_accents_unicode",
5161
"replace_punctuation",

0 commit comments

Comments
 (0)