ERROR: Failed building wheel for tokenizers

[FurkanGozukara]

Hello. Ty very much for installation and examples. However I am getting this error. Any idea how to solve?

I am using Windows 10 and pip

C:\punctuation-restoration>pip install -r requirements.txt
Collecting transformers==v2.11.0
Using cached transformers-2.11.0-py3-none-any.whl (674 kB)
Collecting pytorch-crf
Using cached pytorch_crf-0.7.2-py3-none-any.whl (9.5 kB)
Requirement already satisfied: sentencepiece in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (0.1.97)
Requirement already satisfied: requests in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (2.21.0)
Collecting sacremoses
Using cached sacremoses-0.0.53.tar.gz (880 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: packaging in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (21.3)
Requirement already satisfied: filelock in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (3.8.0)
Requirement already satisfied: numpy in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (1.23.3)
Requirement already satisfied: tqdm>=4.27 in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (4.64.1)
Requirement already satisfied: regex!=2019.12.17 in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (2022.9.13)
Collecting tokenizers==0.7.0
Using cached tokenizers-0.7.0.tar.gz (81 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: colorama in c:\python399\lib\site-packages (from tqdm>=4.27->transformers==v2.11.0->-r requirements.txt (line 1)) (0.4.5)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in c:\python399\lib\site-packages (from packaging->transformers==v2.11.0->-r requirements.txt (line 1)) (3.0.9)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\python399\lib\site-packages (from requests->transformers==v2.11.0->-r requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in c:\python399\lib\site-packages (from requests->transformers==v2.11.0->-r requirements.txt (line 1)) (1.24.3)
Requirement already satisfied: idna<2.9,>=2.5 in c:\python399\lib\site-packages (from requests->transformers==v2.11.0->-r requirements.txt (line 1)) (2.8)
Requirement already satisfied: certifi>=2017.4.17 in c:\python399\lib\site-packages (from requests->transformers==v2.11.0->-r requirements.txt (line 1)) (2022.9.14)
Requirement already satisfied: six in c:\python399\lib\site-packages (from sacremoses->transformers==v2.11.0->-r requirements.txt (line 1)) (1.12.0)
Collecting click
Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting joblib
Using cached joblib-1.2.0-py3-none-any.whl (297 kB)
Building wheels for collected packages: tokenizers
Building wheel for tokenizers (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for tokenizers (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [46 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-39
creating build\lib.win-amd64-cpython-39\tokenizers
copying tokenizers_init_.py -> build\lib.win-amd64-cpython-39\tokenizers
creating build\lib.win-amd64-cpython-39\tokenizers\models
copying tokenizers\models_init_.py -> build\lib.win-amd64-cpython-39\tokenizers\models
creating build\lib.win-amd64-cpython-39\tokenizers\decoders
copying tokenizers\decoders_init_.py -> build\lib.win-amd64-cpython-39\tokenizers\decoders
creating build\lib.win-amd64-cpython-39\tokenizers\normalizers
copying tokenizers\normalizers_init_.py -> build\lib.win-amd64-cpython-39\tokenizers\normalizers
creating build\lib.win-amd64-cpython-39\tokenizers\pre_tokenizers
copying tokenizers\pre_tokenizers_init_.py -> build\lib.win-amd64-cpython-39\tokenizers\pre_tokenizers
creating build\lib.win-amd64-cpython-39\tokenizers\processors
copying tokenizers\processors_init_.py -> build\lib.win-amd64-cpython-39\tokenizers\processors
creating build\lib.win-amd64-cpython-39\tokenizers\trainers
copying tokenizers\trainers_init_.py -> build\lib.win-amd64-cpython-39\tokenizers\trainers
creating build\lib.win-amd64-cpython-39\tokenizers\implementations
copying tokenizers\implementations\base_tokenizer.py -> build\lib.win-amd64-cpython-39\tokenizers\implementations
copying tokenizers\implementations\bert_wordpiece.py -> build\lib.win-amd64-cpython-39\tokenizers\implementations
copying tokenizers\implementations\byte_level_bpe.py -> build\lib.win-amd64-cpython-39\tokenizers\implementations
copying tokenizers\implementations\char_level_bpe.py -> build\lib.win-amd64-cpython-39\tokenizers\implementations
copying tokenizers\implementations\sentencepiece_bpe.py -> build\lib.win-amd64-cpython-39\tokenizers\implementations
copying tokenizers\implementations_init_.py -> build\lib.win-amd64-cpython-39\tokenizers\implementations
copying tokenizers_init_.pyi -> build\lib.win-amd64-cpython-39\tokenizers
copying tokenizers\models_init_.pyi -> build\lib.win-amd64-cpython-39\tokenizers\models
copying tokenizers\decoders_init_.pyi -> build\lib.win-amd64-cpython-39\tokenizers\decoders
copying tokenizers\normalizers_init_.pyi -> build\lib.win-amd64-cpython-39\tokenizers\normalizers
copying tokenizers\pre_tokenizers_init_.pyi -> build\lib.win-amd64-cpython-39\tokenizers\pre_tokenizers
copying tokenizers\processors_init_.pyi -> build\lib.win-amd64-cpython-39\tokenizers\processors
copying tokenizers\trainers_init_.pyi -> build\lib.win-amd64-cpython-39\tokenizers\trainers
running build_ext
running build_rust
error: can't find Rust compiler

  If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.

  To update pip, run:

      pip install --upgrade pip

  and then retry package installation.

  If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

[FurkanGozukara]

I have installed rust from here and now have different error

https://www.rust-lang.org/tools/install

C:\punctuation-restoration>pip install -r requirements.txt
Collecting transformers==v2.11.0
  Using cached transformers-2.11.0-py3-none-any.whl (674 kB)
Collecting pytorch-crf
  Using cached pytorch_crf-0.7.2-py3-none-any.whl (9.5 kB)
Requirement already satisfied: packaging in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (21.3)
Requirement already satisfied: requests in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (2.21.0)
Collecting sacremoses
  Using cached sacremoses-0.0.53.tar.gz (880 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: sentencepiece in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (0.1.97)
Requirement already satisfied: regex!=2019.12.17 in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (2022.9.13)
Requirement already satisfied: tqdm>=4.27 in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (4.64.1)
Requirement already satisfied: numpy in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (1.23.3)
Collecting tokenizers==0.7.0
  Using cached tokenizers-0.7.0.tar.gz (81 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: filelock in c:\python399\lib\site-packages (from transformers==v2.11.0->-r requirements.txt (line 1)) (3.8.0)
Requirement already satisfied: colorama in c:\python399\lib\site-packages (from tqdm>=4.27->transformers==v2.11.0->-r requirements.txt (line 1)) (0.4.5)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in c:\python399\lib\site-packages (from packaging->transformers==v2.11.0->-r requirements.txt (line 1)) (3.0.9)
Requirement already satisfied: idna<2.9,>=2.5 in c:\python399\lib\site-packages (from requests->transformers==v2.11.0->-r requirements.txt (line 1)) (2.8)
Requirement already satisfied: certifi>=2017.4.17 in c:\python399\lib\site-packages (from requests->transformers==v2.11.0->-r requirements.txt (line 1)) (2022.9.14)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in c:\python399\lib\site-packages (from requests->transformers==v2.11.0->-r requirements.txt (line 1)) (1.24.3)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in c:\python399\lib\site-packages (from requests->transformers==v2.11.0->-r requirements.txt (line 1)) (3.0.4)
Requirement already satisfied: six in c:\python399\lib\site-packages (from sacremoses->transformers==v2.11.0->-r requirements.txt (line 1)) (1.12.0)
Collecting click
  Using cached click-8.1.3-py3-none-any.whl (96 kB)
Collecting joblib
  Using cached joblib-1.2.0-py3-none-any.whl (297 kB)
Building wheels for collected packages: tokenizers
  Building wheel for tokenizers (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for tokenizers (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [258 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib.win-amd64-cpython-39
      creating build\lib.win-amd64-cpython-39\tokenizers
      copying tokenizers\__init__.py -> build\lib.win-amd64-cpython-39\tokenizers
      creating build\lib.win-amd64-cpython-39\tokenizers\models
      copying tokenizers\models\__init__.py -> build\lib.win-amd64-cpython-39\tokenizers\models
      creating build\lib.win-amd64-cpython-39\tokenizers\decoders
      copying tokenizers\decoders\__init__.py -> build\lib.win-amd64-cpython-39\tokenizers\decoders
      creating build\lib.win-amd64-cpython-39\tokenizers\normalizers
      copying tokenizers\normalizers\__init__.py -> build\lib.win-amd64-cpython-39\tokenizers\normalizers
      creating build\lib.win-amd64-cpython-39\tokenizers\pre_tokenizers
      copying tokenizers\pre_tokenizers\__init__.py -> build\lib.win-amd64-cpython-39\tokenizers\pre_tokenizers
      creating build\lib.win-amd64-cpython-39\tokenizers\processors
      copying tokenizers\processors\__init__.py -> build\lib.win-amd64-cpython-39\tokenizers\processors
      creating build\lib.win-amd64-cpython-39\tokenizers\trainers
      copying tokenizers\trainers\__init__.py -> build\lib.win-amd64-cpython-39\tokenizers\trainers
      creating build\lib.win-amd64-cpython-39\tokenizers\implementations
      copying tokenizers\implementations\base_tokenizer.py -> build\lib.win-amd64-cpython-39\tokenizers\implementations
      copying tokenizers\implementations\bert_wordpiece.py -> build\lib.win-amd64-cpython-39\tokenizers\implementations
      copying tokenizers\implementations\byte_level_bpe.py -> build\lib.win-amd64-cpython-39\tokenizers\implementations
      copying tokenizers\implementations\char_level_bpe.py -> build\lib.win-amd64-cpython-39\tokenizers\implementations
      copying tokenizers\implementations\sentencepiece_bpe.py -> build\lib.win-amd64-cpython-39\tokenizers\implementations
      copying tokenizers\implementations\__init__.py -> build\lib.win-amd64-cpython-39\tokenizers\implementations
      copying tokenizers\__init__.pyi -> build\lib.win-amd64-cpython-39\tokenizers
      copying tokenizers\models\__init__.pyi -> build\lib.win-amd64-cpython-39\tokenizers\models
      copying tokenizers\decoders\__init__.pyi -> build\lib.win-amd64-cpython-39\tokenizers\decoders
      copying tokenizers\normalizers\__init__.pyi -> build\lib.win-amd64-cpython-39\tokenizers\normalizers
      copying tokenizers\pre_tokenizers\__init__.pyi -> build\lib.win-amd64-cpython-39\tokenizers\pre_tokenizers
      copying tokenizers\processors\__init__.pyi -> build\lib.win-amd64-cpython-39\tokenizers\processors
      copying tokenizers\trainers\__init__.pyi -> build\lib.win-amd64-cpython-39\tokenizers\trainers
      running build_ext
      running build_rust
      info: syncing channel updates for 'nightly-x86_64-pc-windows-msvc'
      info: latest update on 2022-10-18, rust version 1.66.0-nightly (06f049a35 2022-10-17)
      info: downloading component 'cargo'
      info: downloading component 'clippy'
      info: downloading component 'rust-docs'
      info: downloading component 'rust-std'
      info: downloading component 'rustc'
      info: downloading component 'rustfmt'
      info: installing component 'cargo'
      info: installing component 'clippy'
      info: installing component 'rust-docs'
      info: retrying renaming 'C:\Users\King\.rustup\tmp\qkopjzc7kj3elau0_dir\rust-docs\share/doc/rust/html' to 'C:\Users\King\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\share/doc/rust/html'
      info: installing component 'rust-std'
      info: installing component 'rustc'
      info: installing component 'rustfmt'
      cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module -- --crate-type cdylib
      warning: unused manifest key: target.x86_64-apple-darwin.rustflags
          Updating crates.io index
       Downloading crates ...
        Downloaded autocfg v1.1.0
        Downloaded serde v1.0.145
        Downloaded serde_json v1.0.86
        Downloaded rand v0.7.3
        Downloaded itoa v1.0.4
        Downloaded memchr v2.5.0
        Downloaded num_cpus v1.13.1
        Downloaded inventory-impl v0.1.11
        Downloaded parking_lot v0.10.2
        Downloaded parking_lot_core v0.7.2
        Downloaded number_prefix v0.3.0
        Downloaded indoc-impl v0.3.6
        Downloaded serde_derive v1.0.145
        Downloaded ryu v1.0.11
        Downloaded version_check v0.9.4
        Downloaded unicode-width v0.1.10
        Downloaded syn v1.0.102
        Downloaded ghost v0.1.6
        Downloaded lock_api v0.3.4
        Downloaded inventory v0.1.11
        Downloaded indicatif v0.14.0
        Downloaded getrandom v0.1.16
        Downloaded textwrap v0.11.0
        Downloaded lazy_static v1.4.0
        Downloaded ctor v0.1.24
        Downloaded crossbeam-utils v0.8.12
        Downloaded num-traits v0.2.15
        Downloaded unindent v0.1.10
        Downloaded unicode-normalization-alignments v0.1.12
        Downloaded pyo3cls v0.9.2
        Downloaded pyo3 v0.9.2
        Downloaded paste-impl v0.1.18
        Downloaded rand_core v0.5.1
        Downloaded strsim v0.8.0
        Downloaded scopeguard v1.1.0
        Downloaded vec_map v0.8.2
        Downloaded unicode-ident v1.0.5
        Downloaded rayon-core v1.9.3
        Downloaded rayon v1.5.3
        Downloaded unicode_categories v0.1.1
        Downloaded pyo3-derive-backend v0.9.2
        Downloaded bitflags v1.3.2
        Downloaded cfg-if v1.0.0
        Downloaded cfg-if v0.1.10
        Downloaded atty v0.2.14
        Downloaded regex v1.6.0
        Downloaded memoffset v0.6.5
        Downloaded indoc v0.3.6
        Downloaded either v1.8.0
        Downloaded crossbeam-deque v0.8.2
        Downloaded crossbeam-channel v0.5.6
        Downloaded clap v2.34.0
        Downloaded regex-syntax v0.6.27
        Downloaded winapi v0.3.9
        Downloaded libc v0.2.135
        Downloaded aho-corasick v0.7.19
        Downloaded console v0.15.2
        Downloaded crossbeam-epoch v0.9.11
        Downloaded encode_unicode v0.3.6
        Downloaded smallvec v1.10.0
        Downloaded terminal_size v0.1.17
        Downloaded proc-macro2 v1.0.47
        Downloaded quote v1.0.21
        Downloaded rand_chacha v0.2.2
        Downloaded proc-macro-hack v0.5.19
        Downloaded ppv-lite86 v0.2.16
        Downloaded paste v0.1.18
         Compiling proc-macro2 v1.0.47
         Compiling quote v1.0.21
         Compiling unicode-ident v1.0.5
         Compiling syn v1.0.102
         Compiling autocfg v1.1.0
         Compiling memchr v2.5.0
         Compiling winapi v0.3.9
         Compiling cfg-if v1.0.0
         Compiling serde_derive v1.0.145
         Compiling serde v1.0.145
         Compiling serde_json v1.0.86
         Compiling scopeguard v1.1.0
         Compiling crossbeam-utils v0.8.12
         Compiling proc-macro-hack v0.5.19
         Compiling getrandom v0.1.16
         Compiling libc v0.2.135
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\proc-macro2-1.0.47\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"proc-macro\"" -C metadata=5b0e58d159021849 -C extra-filename=-5b0e58d159021849 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\proc-macro2-5b0e58d159021849 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\quote-1.0.21\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"proc-macro\"" -C metadata=142419b9cedb1d17 -C extra-filename=-142419b9cedb1d17 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\quote-142419b9cedb1d17 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name unicode_ident --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\unicode-ident-1.0.5\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=1afecb6243c40eff -C extra-filename=-1afecb6243c40eff --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-1.0.102\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"clone-impls\"" --cfg "feature=\"default\"" --cfg "feature=\"derive\"" --cfg "feature=\"extra-traits\"" --cfg "feature=\"full\"" --cfg "feature=\"parsing\"" --cfg "feature=\"printing\"" --cfg "feature=\"proc-macro\"" --cfg "feature=\"quote\"" -C metadata=b8e33a9e5e50652b -C extra-filename=-b8e33a9e5e50652b --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\syn-b8e33a9e5e50652b -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name autocfg C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\autocfg-1.1.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=572016c50f479faf -C extra-filename=-572016c50f479faf --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\memchr-2.5.0\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=4ba3537c9d483825 -C extra-filename=-4ba3537c9d483825 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\memchr-4ba3537c9d483825 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-0.3.9\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"consoleapi\"" --cfg "feature=\"errhandlingapi\"" --cfg "feature=\"fileapi\"" --cfg "feature=\"handleapi\"" --cfg "feature=\"minwinbase\"" --cfg "feature=\"minwindef\"" --cfg "feature=\"ntstatus\"" --cfg "feature=\"processenv\"" --cfg "feature=\"winbase\"" --cfg "feature=\"wincon\"" --cfg "feature=\"winerror\"" --cfg "feature=\"winnt\"" --cfg "feature=\"winuser\"" -C metadata=9ff0f262099ebbbd -C extra-filename=-9ff0f262099ebbbd --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\winapi-9ff0f262099ebbbd -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name cfg_if --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\cfg-if-1.0.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=360d109d20cdf1de -C extra-filename=-360d109d20cdf1de --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\serde_derive-1.0.145\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" -C metadata=84c8f4354cf6545a -C extra-filename=-84c8f4354cf6545a --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\serde_derive-84c8f4354cf6545a -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\serde-1.0.145\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"derive\"" --cfg "feature=\"serde_derive\"" --cfg "feature=\"std\"" -C metadata=1080dbca61ce40eb -C extra-filename=-1080dbca61ce40eb --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\serde-1080dbca61ce40eb -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\serde_json-1.0.86\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=b3ba47c415a7892f -C extra-filename=-b3ba47c415a7892f --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\serde_json-b3ba47c415a7892f -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name scopeguard C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\scopeguard-1.1.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=38978501dffc7e67 -C extra-filename=-38978501dffc7e67 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-utils-0.8.12\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=66c260199ab3a6b1 -C extra-filename=-66c260199ab3a6b1 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\crossbeam-utils-66c260199ab3a6b1 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\proc-macro-hack-0.5.19\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C metadata=43fb356fc25df1f2 -C extra-filename=-43fb356fc25df1f2 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\proc-macro-hack-43fb356fc25df1f2 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\getrandom-0.1.16\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"std\"" -C metadata=c55ade8324d95812 -C extra-filename=-c55ade8324d95812 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\getrandom-c55ade8324d95812 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.135\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=e99f7f26a77660cd -C extra-filename=-e99f7f26a77660cd --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\libc-e99f7f26a77660cd -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
         Compiling unicode-width v0.1.10
         Compiling rayon-core v1.9.3
           Running `rustc --crate-name unicode_width C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\unicode-width-0.1.10\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" -C metadata=5f855d5352d382a0 -C extra-filename=-5f855d5352d382a0 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\rayon-core-1.9.3\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C metadata=e7c93bf382490bd0 -C extra-filename=-e7c93bf382490bd0 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\rayon-core-e7c93bf382490bd0 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
         Compiling smallvec v1.10.0
           Running `rustc --crate-name smallvec --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\smallvec-1.10.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=881054317b8f4d9c -C extra-filename=-881054317b8f4d9c --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
         Compiling ryu v1.0.11
           Running `rustc --crate-name ryu --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\ryu-1.0.11\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=1d25c68059b095ff -C extra-filename=-1d25c68059b095ff --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
         Compiling memoffset v0.6.5
         Compiling crossbeam-epoch v0.9.11
           Running `rustc --crate-name build_script_build C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\memoffset-0.6.5\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" -C metadata=8550414a9ea9eb93 -C extra-filename=-8550414a9ea9eb93 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\memoffset-8550414a9ea9eb93 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern autocfg=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libautocfg-572016c50f479faf.rlib --cap-lints allow`
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-epoch-0.9.11\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"alloc\"" --cfg "feature=\"std\"" -C metadata=a5f96eb43b93974a -C extra-filename=-a5f96eb43b93974a --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\crossbeam-epoch-a5f96eb43b93974a -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern autocfg=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libautocfg-572016c50f479faf.rlib --cap-lints allow`
         Compiling regex-syntax v0.6.27
           Running `rustc --crate-name regex_syntax --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\regex-syntax-0.6.27\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"unicode\"" --cfg "feature=\"unicode-age\"" --cfg "feature=\"unicode-bool\"" --cfg "feature=\"unicode-case\"" --cfg "feature=\"unicode-gencat\"" --cfg "feature=\"unicode-perl\"" --cfg "feature=\"unicode-script\"" --cfg "feature=\"unicode-segment\"" -C metadata=ecd6baa27c468f0d -C extra-filename=-ecd6baa27c468f0d --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\syn-b8e33a9e5e50652b\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\proc-macro2-5b0e58d159021849\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\quote-142419b9cedb1d17\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\winapi-9ff0f262099ebbbd\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\serde_derive-84c8f4354cf6545a\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\memoffset-8550414a9ea9eb93\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\proc-macro-hack-43fb356fc25df1f2\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\getrandom-c55ade8324d95812\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\libc-e99f7f26a77660cd\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\serde-1080dbca61ce40eb\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\crossbeam-epoch-a5f96eb43b93974a\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\crossbeam-utils-66c260199ab3a6b1\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\memchr-4ba3537c9d483825\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\memchr-4ba3537c9d483825\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\rayon-core-e7c93bf382490bd0\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\serde_json-b3ba47c415a7892f\build-script-build`
           Running `rustc --crate-name proc_macro2 --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\proc-macro2-1.0.47\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"proc-macro\"" -C metadata=30944198976f606e -C extra-filename=-30944198976f606e --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern unicode_ident=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libunicode_ident-1afecb6243c40eff.rmeta --cap-lints allow --cfg use_proc_macro --cfg wrap_proc_macro --cfg proc_macro_span`
         Compiling itoa v1.0.4
           Running `rustc --crate-name itoa --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\itoa-1.0.4\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=c5e72f9360d1cff0 -C extra-filename=-c5e72f9360d1cff0 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name winapi C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\winapi-0.3.9\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"consoleapi\"" --cfg "feature=\"errhandlingapi\"" --cfg "feature=\"fileapi\"" --cfg "feature=\"handleapi\"" --cfg "feature=\"minwinbase\"" --cfg "feature=\"minwindef\"" --cfg "feature=\"ntstatus\"" --cfg "feature=\"processenv\"" --cfg "feature=\"winbase\"" --cfg "feature=\"wincon\"" --cfg "feature=\"winerror\"" --cfg "feature=\"winnt\"" --cfg "feature=\"winuser\"" -C metadata=c43802643be2e565 -C extra-filename=-c43802643be2e565 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow -l dylib=advapi32 -l dylib=cfgmgr32 -l dylib=gdi32 -l dylib=kernel32 -l dylib=msimg32 -l dylib=opengl32 -l dylib=user32 -l dylib=winspool --cfg "feature=\"vadefs\"" --cfg "feature=\"cfgmgr32\"" --cfg "feature=\"devpropdef\"" --cfg "feature=\"processthreadsapi\"" --cfg "feature=\"basetsd\"" --cfg "feature=\"winreg\"" --cfg "feature=\"ntdef\"" --cfg "feature=\"libloaderapi\"" --cfg "feature=\"wincontypes\"" --cfg "feature=\"vcruntime\"" --cfg "feature=\"excpt\"" --cfg "feature=\"wtypesbase\"" --cfg "feature=\"reason\"" --cfg "feature=\"limits\"" --cfg "feature=\"rpcndr\"" --cfg "feature=\"ktmtypes\"" --cfg "feature=\"guiddef\"" --cfg "feature=\"windef\"" --cfg "feature=\"wingdi\"" --cfg "feature=\"cfg\""`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\serde-1080dbca61ce40eb\build-script-build`
           Running `rustc --crate-name memoffset C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\memoffset-0.6.5\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" -C metadata=d521e9d5ca1cfb74 -C extra-filename=-d521e9d5ca1cfb74 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow --cfg tuple_ty --cfg allow_clippy --cfg maybe_uninit --cfg doctests --cfg raw_ref_macros`
           Running `rustc --crate-name proc_macro_hack --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\proc-macro-hack-0.5.19\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type proc-macro --emit=dep-info,link -C prefer-dynamic -C embed-bitcode=no -C debug-assertions=off -C metadata=097737d8e2c954ab -C extra-filename=-097737d8e2c954ab --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern proc_macro --cap-lints allow`
           Running `rustc --crate-name getrandom --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\getrandom-0.1.16\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"std\"" -C metadata=6c7f93dfc6444a9c -C extra-filename=-6c7f93dfc6444a9c --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern cfg_if=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libcfg_if-360d109d20cdf1de.rmeta --cap-lints allow -l advapi32`
           Running `rustc --crate-name libc C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.135\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=86a1bab4b120f6a2 -C extra-filename=-86a1bab4b120f6a2 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow --cfg freebsd11 --cfg libc_priv_mod_use --cfg libc_union --cfg libc_const_size_of --cfg libc_align --cfg libc_int128 --cfg libc_core_cvoid --cfg libc_packedN --cfg libc_cfg_target_vendor --cfg libc_non_exhaustive --cfg libc_ptr_addr_of --cfg libc_underscore_const_names --cfg libc_const_extern_fn`
         Compiling rayon v1.5.3
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\rayon-1.5.3\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C metadata=f6ea986aed560875 -C extra-filename=-f6ea986aed560875 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\rayon-f6ea986aed560875 -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern autocfg=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libautocfg-572016c50f479faf.rlib --cap-lints allow`
         Compiling num-traits v0.2.15
           Running `rustc --crate-name build_script_build C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\num-traits-0.2.15\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=286ff2921bdb81ee -C extra-filename=-286ff2921bdb81ee --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\num-traits-286ff2921bdb81ee -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern autocfg=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libautocfg-572016c50f479faf.rlib --cap-lints allow`
           Running `rustc --crate-name crossbeam_utils --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-utils-0.8.12\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=3869c69ecfdeae93 -C extra-filename=-3869c69ecfdeae93 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern cfg_if=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libcfg_if-360d109d20cdf1de.rmeta --cap-lints allow`
           Running `rustc --crate-name memchr --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\memchr-2.5.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=055ae21494443b01 -C extra-filename=-055ae21494443b01 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow --cfg memchr_runtime_simd --cfg memchr_runtime_sse2 --cfg memchr_runtime_sse42 --cfg memchr_runtime_avx`
           Running `rustc --crate-name memchr --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\memchr-2.5.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=f54a1ac40e150573 -C extra-filename=-f54a1ac40e150573 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow --cfg memchr_runtime_simd --cfg memchr_runtime_sse2 --cfg memchr_runtime_sse42 --cfg memchr_runtime_avx`
         Compiling unindent v0.1.10
           Running `rustc --crate-name unindent --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\unindent-0.1.10\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=fcf514872b9c9d8e -C extra-filename=-fcf514872b9c9d8e --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name regex_syntax --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\regex-syntax-0.6.27\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" --cfg "feature=\"unicode\"" --cfg "feature=\"unicode-age\"" --cfg "feature=\"unicode-bool\"" --cfg "feature=\"unicode-case\"" --cfg "feature=\"unicode-gencat\"" --cfg "feature=\"unicode-perl\"" --cfg "feature=\"unicode-script\"" --cfg "feature=\"unicode-segment\"" -C metadata=5a236c973584f460 -C extra-filename=-5a236c973584f460 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
         Compiling cfg-if v0.1.10
           Running `rustc --crate-name cfg_if --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\cfg-if-0.1.10\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=f3af8d70d946c761 -C extra-filename=-f3af8d70d946c761 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
         Compiling encode_unicode v0.3.6
           Running `rustc --crate-name encode_unicode C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\encode_unicode-0.3.6\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=3130f1c305b48f53 -C extra-filename=-3130f1c305b48f53 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
         Compiling inventory v0.1.11
         Compiling ppv-lite86 v0.2.16
           Running `rustc --crate-name build_script_build --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\inventory-0.1.11\build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off -C metadata=2a48a1911251848c -C extra-filename=-2a48a1911251848c --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\inventory-2a48a1911251848c -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name ppv_lite86 --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\ppv-lite86-0.2.16\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"simd\"" --cfg "feature=\"std\"" -C metadata=94c9eb93ee689d3b -C extra-filename=-94c9eb93ee689d3b --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
         Compiling rand_core v0.5.1
           Running `rustc --crate-name rand_core --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\rand_core-0.5.1\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"alloc\"" --cfg "feature=\"getrandom\"" --cfg "feature=\"std\"" -C metadata=06e7d83191242dc6 -C extra-filename=-06e7d83191242dc6 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern getrandom=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libgetrandom-6c7f93dfc6444a9c.rmeta --cap-lints allow`
         Compiling version_check v0.9.4
           Running `rustc --crate-name version_check C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\version_check-0.9.4\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=0584bbe3f7491e0d -C extra-filename=-0584bbe3f7491e0d --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name quote --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\quote-1.0.21\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"proc-macro\"" -C metadata=dabbd39e3909793b -C extra-filename=-dabbd39e3909793b --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern proc_macro2=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libproc_macro2-30944198976f606e.rmeta --cap-lints allow`
         Compiling num_cpus v1.13.1
           Running `rustc --crate-name num_cpus C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\num_cpus-1.13.1\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=31ad845e15e2b15d -C extra-filename=-31ad845e15e2b15d --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
         Compiling lazy_static v1.4.0
           Running `rustc --crate-name lazy_static C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\lazy_static-1.4.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=431f071606c587f4 -C extra-filename=-431f071606c587f4 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
           Running `rustc --crate-name crossbeam_epoch --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-epoch-0.9.11\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"alloc\"" --cfg "feature=\"std\"" -C metadata=4897f65d76bcc39c -C extra-filename=-4897f65d76bcc39c --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern cfg_if=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libcfg_if-360d109d20cdf1de.rmeta --extern crossbeam_utils=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libcrossbeam_utils-3869c69ecfdeae93.rmeta --extern memoffset=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libmemoffset-d521e9d5ca1cfb74.rmeta --extern scopeguard=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libscopeguard-38978501dffc7e67.rmeta --cap-lints allow`
         Compiling crossbeam-channel v0.5.6
           Running `rustc --crate-name crossbeam_channel --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-channel-0.5.6\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"crossbeam-utils\"" --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=658fbd4d6c655a6f -C extra-filename=-658fbd4d6c655a6f --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern cfg_if=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libcfg_if-360d109d20cdf1de.rmeta --extern crossbeam_utils=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libcrossbeam_utils-3869c69ecfdeae93.rmeta --cap-lints allow`
           Running `rustc --crate-name syn --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\syn-1.0.102\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"clone-impls\"" --cfg "feature=\"default\"" --cfg "feature=\"derive\"" --cfg "feature=\"extra-traits\"" --cfg "feature=\"full\"" --cfg "feature=\"parsing\"" --cfg "feature=\"printing\"" --cfg "feature=\"proc-macro\"" --cfg "feature=\"quote\"" -C metadata=ed20b2b04ea8255e -C extra-filename=-ed20b2b04ea8255e --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern proc_macro2=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libproc_macro2-30944198976f606e.rmeta --extern quote=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libquote-dabbd39e3909793b.rmeta --extern unicode_ident=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libunicode_ident-1afecb6243c40eff.rmeta --cap-lints allow`
         Compiling aho-corasick v0.7.19
           Running `rustc --crate-name aho_corasick --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\aho-corasick-0.7.19\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=f1529ca83e28da6f -C extra-filename=-f1529ca83e28da6f --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern memchr=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libmemchr-055ae21494443b01.rmeta --cap-lints allow`
         Compiling rand_chacha v0.2.2
           Running `rustc --crate-name aho_corasick --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\aho-corasick-0.7.19\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=dfcbc49062ee55e6 -C extra-filename=-dfcbc49062ee55e6 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern memchr=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libmemchr-f54a1ac40e150573.rmeta --cap-lints allow`
           Running `rustc --crate-name rand_chacha --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\rand_chacha-0.2.2\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"std\"" -C metadata=42edb5e8f22d28e3 -C extra-filename=-42edb5e8f22d28e3 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern ppv_lite86=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libppv_lite86-94c9eb93ee689d3b.rmeta --extern rand_core=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\librand_core-06e7d83191242dc6.rmeta --cap-lints allow`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\inventory-2a48a1911251848c\build-script-build`
         Compiling crossbeam-deque v0.8.2
           Running `rustc --crate-name crossbeam_deque --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-deque-0.8.2\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"crossbeam-epoch\"" --cfg "feature=\"crossbeam-utils\"" --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=b75d3846468b682e -C extra-filename=-b75d3846468b682e --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern cfg_if=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libcfg_if-360d109d20cdf1de.rmeta --extern crossbeam_epoch=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libcrossbeam_epoch-4897f65d76bcc39c.rmeta --extern crossbeam_utils=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libcrossbeam_utils-3869c69ecfdeae93.rmeta --cap-lints allow`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\num-traits-286ff2921bdb81ee\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\rayon-f6ea986aed560875\build-script-build`
           Running `C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\build\serde_json-b3ba47c415a7892f\build-script-build`
         Compiling textwrap v0.11.0
           Running `rustc --crate-name rayon_core --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\rayon-core-1.9.3\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=0dc7012c06ddf19e -C extra-filename=-0dc7012c06ddf19e --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern crossbeam_channel=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libcrossbeam_channel-658fbd4d6c655a6f.rmeta --extern crossbeam_deque=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libcrossbeam_deque-b75d3846468b682e.rmeta --extern crossbeam_utils=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libcrossbeam_utils-3869c69ecfdeae93.rmeta --extern num_cpus=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libnum_cpus-31ad845e15e2b15d.rmeta --cap-lints allow`
           Running `rustc --crate-name textwrap C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\textwrap-0.11.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=65eec7bbb3cdf0f8 -C extra-filename=-65eec7bbb3cdf0f8 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern unicode_width=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libunicode_width-5f855d5352d382a0.rmeta --cap-lints allow`
         Compiling paste-impl v0.1.18
           Running `rustc --crate-name paste_impl --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\paste-impl-0.1.18\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type proc-macro --emit=dep-info,link -C prefer-dynamic -C embed-bitcode=no -C debug-assertions=off -C metadata=d5e32c087e023b55 -C extra-filename=-d5e32c087e023b55 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern proc_macro_hack=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\proc_macro_hack-097737d8e2c954ab.dll --extern proc_macro --cap-lints allow`
         Compiling lock_api v0.3.4
         Compiling strsim v0.8.0
           Running `rustc --crate-name lock_api --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\lock_api-0.3.4\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"nightly\"" -C metadata=2fe17ee22c58d967 -C extra-filename=-2fe17ee22c58d967 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern scopeguard=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libscopeguard-38978501dffc7e67.rmeta --cap-lints allow`
           Running `rustc --crate-name strsim C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\strsim-0.8.0\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no -C metadata=8fb9b7eda8bf3ac7 -C extra-filename=-8fb9b7eda8bf3ac7 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --cap-lints allow`
      error[E0557]: feature has been removed
        --> C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\lock_api-0.3.4\src\lib.rs:91:42
         |
      91 | #![cfg_attr(feature = "nightly", feature(const_fn))]
         |                                          ^^^^^^^^ feature has been removed
         |
         = note: split into finer-grained feature gates

      For more information about this error, try `rustc --explain E0557`.
      error: could not compile `lock_api` due to previous error

      Caused by:
        process didn't exit successfully: `rustc --crate-name lock_api --edition=2018 C:\Users\King\.cargo\registry\src\github.com-1ecc6299db9ec823\lock_api-0.3.4\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"nightly\"" -C metadata=2fe17ee22c58d967 -C extra-filename=-2fe17ee22c58d967 --out-dir C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps -L dependency=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps --extern scopeguard=C:\Users\King\AppData\Local\Temp\pip-install-b7_bo2sd\tokenizers_1b6098a3d736487c996866dfd304f880\target\release\deps\libscopeguard-38978501dffc7e67.rmeta --cap-lints allow` (exit code: 1)
      warning: build failed, waiting for other jobs to finish...
      error: `cargo rustc --lib --message-format=json-render-diagnostics --manifest-path Cargo.toml --release -v --features pyo3/extension-module -- --crate-type cdylib` failed with code 101
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

C:\punctuation-restoration>

[afzalsiddique]

I installed transfomers 4.37.2 and it fixed the issue. Found the solution here
pip install transformers==4.37.2