Skip to content

Add unicode-normalization-0.1.24 #2087

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 30, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions collector/compile-benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,9 @@ They mostly consist of real-world crates.
- **unicode-normalization-0.1.19**: Unicode character composition and decomposition
utilities. Uses huge `match` statements that stress the compiler in unusual
ways.
- **unicode-normalization-0.1.24**: Unicode character composition and decomposition
utilities. Uses huge `match` statements that stress the compiler in unusual
ways.

## Secondary

Expand Down
5 changes: 5 additions & 0 deletions collector/compile-benchmarks/REUSE.toml
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,11 @@ path = "unicode-normalization-0.1.19/**"
SPDX-FileCopyrightText = "unicode-normalization contributors"
SPDX-License-Identifier = "MIT OR Apache-2.0"

[[annotations]]
path = "unicode-normalization-0.1.24/**"
SPDX-FileCopyrightText = "unicode-normalization contributors"
SPDX-License-Identifier = "MIT OR Apache-2.0"

[[annotations]]
path = "unify-linearly/**"
SPDX-FileCopyrightText = "The Rust Project Developers (see https://thanks.rust-lang.org)"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"git": {
"sha1": "c9921309f09ebd05108920fda92efbf5f8124a7d"
},
"path_in_vcs": ""
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
name: Rust

on:
push:
branches: [ master ]
pull_request:
branches: [ master ]

env:
CARGO_INCREMENTAL: 0
CARGO_TERM_COLOR: always
RUST_BACKTRACE: 1
RUSTFLAGS: -D warnings
RUSTDOCFLAGS: -D warnings --cfg docsrs

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
rust:
- stable
- beta
- nightly
steps:
- uses: actions/checkout@v2
- name: Install toolchain
uses: actions-rs/toolchain@v1
with:
toolchain: ${{ matrix.rust }}
override: true
components: rustfmt, clippy
- name: Build
run: cargo build --verbose
- name: Run tests with all features
run: cargo test --all-features --verbose
- name: Run tests without features
run: cargo test --no-default-features --verbose
- name: Package
run: cargo package
- name: Test package
run: cd $(find target/package/ -maxdepth 1 -mindepth 1 -type d) && cargo test
- name: Test package without features
run: cd $(find target/package/ -maxdepth 1 -mindepth 1 -type d) && cargo test --no-default-features
- name: Build docs
if: matrix.rust == 'nightly'
run: cargo doc --all-features --verbose
- name: Check formatting
if: matrix.rust == 'stable'
run: cargo fmt --all --check
- name: Check clippy
if: matrix.rust == 'stable'
run: cargo clippy --all-features --lib --tests --examples --verbose
- name: Check benchmarks with clippy
if: matrix.rust == 'nightly'
run: cargo clippy --all-features --benches --verbose
- name: Check fuzz tests with clippy
if: matrix.rust == 'stable'
working-directory: fuzz
run: cargo clippy --all-features --all-targets --verbose
- name: Check fuzz tests formatting
if: matrix.rust == 'stable'
working-directory: fuzz
run: cargo fmt --all --check
msrv:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install msrv toolchain
uses: dtolnay/[email protected]
- name: Use tinyvec 1.6.0
run: cargo update -p tinyvec --precise 1.6.0
- name: Build
run: cargo build --verbose --all-features
regen:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Regen
run: cd scripts && python3 unicode.py
- name: Diff tables
run: diff src/tables.rs scripts/tables.rs
- name: Diff tests
run: diff tests/data/normalization_tests.rs scripts/normalization_tests.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
target
#Cargo.lock
Copy link
Contributor

@Kobzol Kobzol Apr 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't needed, we can just force add the file to our git index and it will be tracked from then on. But no harm in this either.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, from the how-to-update docs I wasn't sure if it was necessary or if -f was enough.

scripts/tmp
*.pyc
*.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
language: rust
rust:
- 1.36.0
- stable
- nightly
sudo: false
script:
- cargo build --verbose
- cargo test --verbose
- cargo test --verbose --no-default-features
- cargo package
- cd target/package/unicode-normalization-*
- cargo test --verbose
- cargo test --verbose --no-default-features
notifications:
email:
on_success: never
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
diff --git a/src/lookups.rs b/src/lookups.rs
index ea7a96c4..c1938cd1 100644
--- a/src/lookups.rs
+++ b/src/lookups.rs
@@ -43,6 +43,7 @@ pub(crate) fn composition_table(c1: char, c2: char) -> Option<char> {
}

pub(crate) fn canonical_fully_decomposed(c: char) -> Option<&'static [char]> {
+ println!("testing");
mph_lookup(
c.into(),
CANONICAL_DECOMPOSED_SALT,
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Licensed under the Apache License, Version 2.0
<LICENSE-APACHE or
http://www.apache.org/licenses/LICENSE-2.0> or the MIT
license <LICENSE-MIT or http://opensource.org/licenses/MIT>,
at your option. All files in the project carrying such
notice may not be copied, modified, or distributed except
according to those terms.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# THIS FILE IS AUTOMATICALLY GENERATED BY CARGO
#
# When uploading crates to the registry Cargo will automatically
# "normalize" Cargo.toml files for maximal compatibility
# with all versions of Cargo and also rewrite `path` dependencies
# to registry (e.g., crates.io) dependencies.
#
# If you are reading this file be aware that the original Cargo.toml
# will likely look very different (and much more reasonable).
# See Cargo.toml.orig for the original contents.

[package]
edition = "2018"
rust-version = "1.36"
name = "unicode-normalization"
version = "0.1.24"
authors = [
"kwantam <[email protected]>",
"Manish Goregaokar <[email protected]>",
]
build = false
exclude = [
"target/*",
"Cargo.lock",
"scripts/tmp",
"*.txt",
"tests/*",
]
autobins = false
autoexamples = false
autotests = false
autobenches = false
description = """
This crate provides functions for normalization of
Unicode strings, including Canonical and Compatible
Decomposition and Recomposition, as described in
Unicode Standard Annex #15.
"""
homepage = "https://github.com/unicode-rs/unicode-normalization"
documentation = "https://docs.rs/unicode-normalization/"
readme = "README.md"
keywords = [
"text",
"unicode",
"normalization",
"decomposition",
"recomposition",
]
license = "MIT/Apache-2.0"
repository = "https://github.com/unicode-rs/unicode-normalization"

[lib]
name = "unicode_normalization"
path = "src/lib.rs"

[[bench]]
name = "bench"
path = "benches/bench.rs"

[dependencies.tinyvec]
version = "1"
features = ["alloc"]

[features]
default = ["std"]
std = []

[workspace]

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading