Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System locale #5081

Merged
merged 79 commits into from
Aug 8, 2024
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
1c7b634
Linux support added
ashu26jha Jun 2, 2024
f6b96a9
improved interface of linux
ashu26jha Jun 14, 2024
df361b8
calendar for linux
ashu26jha Jun 14, 2024
a3d8154
added readme
ashu26jha Jun 14, 2024
bb95d79
apple locale and calendar
ashu26jha Jun 18, 2024
ee23fd2
windows locale retrieval
ashu26jha Jun 19, 2024
1b1b514
windows calendar added
ashu26jha Jun 19, 2024
9b3d8e5
Update Readme
ashu26jha Jun 19, 2024
0adf4b9
Fixed linux
ashu26jha Jul 5, 2024
fe54b79
apple fi
ashu26jha Jul 5, 2024
d7ca69e
Minor fix
ashu26jha Jul 5, 2024
1c10d33
Change module name
ashu26jha Jul 5, 2024
c243451
rustdocs
ashu26jha Jul 5, 2024
0aa332e
readme license name fix
ashu26jha Jul 6, 2024
e96b2d3
Rename Readme.md to README.md
ashu26jha Jul 6, 2024
f7212b5
Change crate name
ashu26jha Jul 7, 2024
aba5c24
Added comment for calendar in linux.rs
ashu26jha Jul 19, 2024
72c0404
Re-export modules
ashu26jha Jul 19, 2024
6dc1973
Fix bug in locale retrieval
ashu26jha Jul 19, 2024
7cc7c65
Added enum in linux.rs
ashu26jha Jul 19, 2024
4c594be
tools/graveyard/Cargo.lock removed
ashu26jha Jul 19, 2024
514f4a0
fmt
ashu26jha Jul 19, 2024
6fd9d43
Added safety comments
ashu26jha Jul 20, 2024
3534091
Refactor linux calendar
ashu26jha Jul 23, 2024
62fe2c5
Fixed macos suffixes
ashu26jha Jul 23, 2024
efd5263
Added safety comments, changed function name to plural
ashu26jha Jul 23, 2024
f59808e
fmt
ashu26jha Jul 23, 2024
ad1a293
Windows fix
ashu26jha Jul 23, 2024
68c8775
removed cfg_if
ashu26jha Jul 26, 2024
e072f7f
added ergonomic names in enum
ashu26jha Jul 26, 2024
1a786f9
one unsafe function call per block
ashu26jha Jul 26, 2024
d73f432
added safety comments for apple.rs
ashu26jha Jul 26, 2024
489c6e6
added safety comments for linux.rs
ashu26jha Jul 26, 2024
52a6bc9
remove .clone()
ashu26jha Jul 30, 2024
87dcefc
Release memory
ashu26jha Jul 30, 2024
00efc98
cargo fmt
ashu26jha Jul 30, 2024
3f41cd9
suggestion: remove mod wrapper
ashu26jha Aug 2, 2024
6be1eae
suggestion: added result type in linux.rs
ashu26jha Aug 2, 2024
de9e217
suggestion: refactor to remove .clone()
ashu26jha Aug 2, 2024
daa9ddb
remove test, will be added in further commits
ashu26jha Aug 2, 2024
f1c9367
improved error format
ashu26jha Aug 2, 2024
c3f5da0
windows error propogation fixed
ashu26jha Aug 2, 2024
b5d5a38
remove unnecessary enum in linux.rs
ashu26jha Aug 2, 2024
8a4eea4
suggestion: improve error propagation macos
ashu26jha Aug 2, 2024
c31695f
updated linux.rs to new error enum
ashu26jha Aug 2, 2024
66eacc6
remove unnecessary return in linux.rs
ashu26jha Aug 2, 2024
60c4c58
impl from UTF8Error
ashu26jha Aug 2, 2024
a3d7393
updated apple.rs error propagation
ashu26jha Aug 2, 2024
d348fc4
updated linux.rs error propagation
ashu26jha Aug 2, 2024
3a07b75
linux.rs test covering get_locales
ashu26jha Aug 3, 2024
8ace9df
corrected license
ashu26jha Aug 4, 2024
8b71b4d
added calendar test for linux
ashu26jha Aug 4, 2024
bc2987a
clippy fix
ashu26jha Aug 4, 2024
af6c742
fix: linux test assert statements
ashu26jha Aug 4, 2024
ae87b00
added apple tests
ashu26jha Aug 4, 2024
835cf07
linux test changes
ashu26jha Aug 4, 2024
439de02
change error enum to test in CI
ashu26jha Aug 4, 2024
fdf3b45
error log ci
ashu26jha Aug 4, 2024
c86f7b4
remove return
ashu26jha Aug 4, 2024
a1fe396
handle lang_str NULL case
ashu26jha Aug 4, 2024
668245f
added safety comments and remove null chars
ashu26jha Aug 4, 2024
e0451f9
fix apple calendar
ashu26jha Aug 4, 2024
a40e2c5
refactor apple.rs
ashu26jha Aug 5, 2024
549b9a5
add timezone for apple
ashu26jha Aug 5, 2024
b45216f
added safety comments
ashu26jha Aug 5, 2024
52a491a
added timezone windows.rs
ashu26jha Aug 5, 2024
6865a0b
added windows rs tests
ashu26jha Aug 5, 2024
fdbd74f
optimise: get_string()
ashu26jha Aug 6, 2024
f51575c
error propagation improve
ashu26jha Aug 6, 2024
5586b47
change order of enum
ashu26jha Aug 6, 2024
f4bd006
added comments for get_string()
ashu26jha Aug 6, 2024
1e3a9fb
refactor: get_string()
ashu26jha Aug 6, 2024
82aaf5a
error.rs and variable name changes
ashu26jha Aug 6, 2024
5d4482b
update comment
ashu26jha Aug 6, 2024
feed347
suggestion: use NullLocale instead of NullPointer
ashu26jha Aug 6, 2024
edc9855
remove nullptr error enum
ashu26jha Aug 6, 2024
d9f65d7
updated tests
ashu26jha Aug 7, 2024
de77db0
ci fix: remove into_iter()
ashu26jha Aug 7, 2024
8f3a6c0
nit: fixed ascii assertion in macos test
ashu26jha Aug 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ members = [
"utils/litemap",
"utils/pattern",
"utils/resb",
"utils/env_preferences",
"utils/tinystr",
"utils/tzif",
"utils/writeable",
Expand Down
7 changes: 7 additions & 0 deletions tools/graveyard/Cargo.lock
robertbastian marked this conversation as resolved.
Show resolved Hide resolved

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

33 changes: 33 additions & 0 deletions utils/env_preferences/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# This file is part of ICU4X. For terms of use, please see the file
# called LICENSE at the top level of the ICU4X source tree
# (online at: https://github.com/unicode-org/icu4x/blob/main/LICENSE ).

[package]
name = "env_preferences"
version.workspace = true
rust-version.workspace = true
authors.workspace = true
edition.workspace = true
repository.workspace = true
homepage.workspace = true
license.workspace = true
categories.workspace = true
include.workspace = true

[dependencies]
core-foundation-sys = "0.8.6"
libc = "0.2.155"

[dependencies.windows]
version = "0.56.0"
features = [
"System",
"Foundation",
"System_UserProfile",
"Foundation_Collections",
"Globalization",
"Globalization_DateTimeFormatting",
"Win32",
"Win32_Globalization"
zbraniecki marked this conversation as resolved.
Show resolved Hide resolved

]
46 changes: 46 additions & 0 deletions utils/env_preferences/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
UNICODE LICENSE V3

COPYRIGHT AND PERMISSION NOTICE

Copyright © 2020-2024 Unicode, Inc.

NOTICE TO USER: Carefully read the following legal agreement. BY
DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING DATA FILES, AND/OR
SOFTWARE, YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE
TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE, DO NOT
DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE THE DATA FILES OR SOFTWARE.

Permission is hereby granted, free of charge, to any person obtaining a
copy of data files and any associated documentation (the "Data Files") or
software and any associated documentation (the "Software") to deal in the
Data Files or Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, and/or sell
copies of the Data Files or Software, and to permit persons to whom the
Data Files or Software are furnished to do so, provided that either (a)
this copyright and permission notice appear with all copies of the Data
Files or Software, or (b) this copyright and permission notice appear in
associated Documentation.

THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF
THIRD PARTY RIGHTS.

IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE
BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES,
OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION,
ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA
FILES OR SOFTWARE.

Except as contained in this notice, the name of a copyright holder shall
not be used in advertising or otherwise to promote the sale, use or other
dealings in these Data Files or Software without prior written
authorization of the copyright holder.

SPDX-License-Identifier: Unicode-3.0


Portions of ICU4X may have been adapted from ICU4C and/or ICU4J.
ICU 1.8.1 to ICU 57.1 © 1995-2016 International Business Machines Corporation and others.
5 changes: 5 additions & 0 deletions utils/env_preferences/README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

89 changes: 89 additions & 0 deletions utils/env_preferences/src/apple.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
// This file is part of ICU4X. For terms of use, please see the file
// called LICENSE at the top level of the ICU4X source tree
// (online at: https://github.com/unicode-org/icu4x/blob/main/LICENSE ).

pub mod apple_locales_prefs {
use core_foundation_sys::{
array::{CFArrayGetCount, CFArrayGetValueAtIndex},
calendar::{CFCalendarCopyCurrent, CFCalendarCopyLocale, CFCalendarGetIdentifier},
locale::{CFLocaleCopyPreferredLanguages, CFLocaleGetIdentifier},
string::CFStringGetCStringPtr,
};
use std::ffi::CStr;

pub fn get_locales_mac() -> Vec<String> {
robertbastian marked this conversation as resolved.
Show resolved Hide resolved
let mut languages: Vec<String> = Vec::new();

unsafe {
let locale_carr_ref = CFLocaleCopyPreferredLanguages();
if !locale_carr_ref.is_null() {
let count = CFArrayGetCount(locale_carr_ref as _);

for i in 0..count {
let lang_ptr = CFArrayGetValueAtIndex(locale_carr_ref as _, i);
if !lang_ptr.is_null() {
let lang_str = CFStringGetCStringPtr(lang_ptr as _, 0);
if !lang_str.is_null() {
let lang_rust_str = CStr::from_ptr(lang_str)
.to_str()
.unwrap_or("Unknown")
.to_string();
languages.push(lang_rust_str);
}
}
}
}
}

// Defaulting to `und`
if languages.is_empty() {
languages.push(String::from("und"));
}
languages
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not do this in this low level function. In my mental model:

  • Low Level - get information from env. No logic. If languages is empty, return empty. If it fails, return failure.
  • High level - normalize results from different platforms into a single cohesive result. Here comes heuristics such as this.

}

pub fn get_system_calendars_macos() -> Vec<(String, String)> {
let mut calendars = Vec::new();
let mut calendar_locale_str = String::new();
let mut calendar_identifier_str = String::new();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's no value in defining those two strings here. Do it inside the if !calendar.is_null(). You can also do `let mut calendar_locale_str: Option = None;

then try to get it.

Then check if you have if let Some((locale, calendar)) = locale.zip(calendar) { calendars.push((...)) }


unsafe {
let calendar = CFCalendarCopyCurrent();
if !calendar.is_null() {
let locale = CFCalendarCopyLocale(calendar as _);
let identifier = CFCalendarGetIdentifier(calendar as _);

if !locale.is_null() {
let locale_identifier = CFLocaleGetIdentifier(locale);
let locale_cstr = CFStringGetCStringPtr(locale_identifier, 0);

if !locale_cstr.is_null() {
calendar_locale_str = CStr::from_ptr(locale_cstr)
.to_str()
.unwrap_or("")
.to_string();
}
}

if !identifier.is_null() {
let identifier_cstr = CFStringGetCStringPtr(identifier, 0);

if !identifier_cstr.is_null() {
calendar_identifier_str = CStr::from_ptr(identifier_cstr)
.to_str()
.unwrap_or("")
.to_string();
}
}

calendars.push((calendar_locale_str, calendar_identifier_str.clone()));
}
}

if calendars.is_empty() {
calendars.push((String::from("und"), String::from("Gregorian")));
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

again, I wouldn't change the values here. Let it return what the system has.


calendars
}
}
14 changes: 14 additions & 0 deletions utils/env_preferences/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
// This file is part of ICU4X. For terms of use, please see the file
// called LICENSE at the top level of the ICU4X source tree
// (online at: https://github.com/unicode-org/icu4x/blob/main/LICENSE ).

//! Retrieval of system locales and preferences.

#[cfg(target_os = "linux")]
pub mod linux;

#[cfg(target_os = "macos")]
pub mod apple;

#[cfg(target_os = "windows")]
mod windows;
71 changes: 71 additions & 0 deletions utils/env_preferences/src/linux.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
// This file is part of ICU4X. For terms of use, please see the file
// called LICENSE at the top level of the ICU4X source tree
// (online at: https://github.com/unicode-org/icu4x/blob/main/LICENSE ).

pub mod linux_locales_prefs {
use std::{
borrow::Cow,
collections::{HashMap, HashSet},
ffi::CStr,
ptr,
};

use libc::{setlocale, LC_ALL, LC_TIME};

fn fetch_locale_settings() -> HashMap<String, String> {
let mut locale_map = HashMap::new();

// Thread safety is ensured by fallbacking to the default locale of `linux` which is `C`
unsafe {
robertbastian marked this conversation as resolved.
Show resolved Hide resolved
let locales_ptr = setlocale(LC_ALL, ptr::null());
let locales = CStr::from_ptr(locales_ptr);

if let Ok(locales_str) = locales.to_str() {
let locales_slice = locales_str.split(';');
for locale in locales_slice {
let mut locale_parts = locale.split('=');
if let (Some(key), Some(value)) = (locale_parts.next(), locale_parts.next()) {
locale_map.insert(key.to_string(), value.to_string());
}
}
} else {
locale_map.insert(String::from("LC_ALL"), String::from("C"));
}
}

locale_map
}

pub fn get_locales_linux() -> Vec<String> {
robertbastian marked this conversation as resolved.
Show resolved Hide resolved
let mut unique_locales = HashSet::new();
let locale_map = fetch_locale_settings();
for value in locale_map.values() {
unique_locales.insert(value.clone());
}

unique_locales.into_iter().collect()
}

pub fn get_system_calendars_linux(
) -> Box<dyn Iterator<Item = (Cow<'static, str>, Cow<'static, str>)>> {
unsafe {
let locale_ptr = setlocale(LC_TIME, ptr::null());
if !locale_ptr.is_null() {
let c_str = CStr::from_ptr(locale_ptr);
if let Ok(str_slice) = c_str.to_str() {
return Box::new(
Some((
Cow::Owned(str_slice.to_string()),
Cow::Borrowed("Gregorian"),
))
.into_iter(),
robertbastian marked this conversation as resolved.
Show resolved Hide resolved
);
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is gnome-calendar here? How does it work on non-gnome Linux? Why are you adding Gregorian?

I'd just return locale without calendar here if your API on Linux doesn't have any info on calendars. just return fn get_system_calendars() -> Result<Vec<String>> for locales and document that all locales are for gregorian calendar.

Copy link
Contributor Author

@ashu26jha ashu26jha Aug 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the format should:

get_system_calendar() -> Result<String>
// LC_TIME -> only single locale can be returned

Box::new(
None.into_iter()
.chain(Some((Cow::Borrowed("C"), Cow::Borrowed("Gregorian")))),
)
}
}
}
Loading
Loading