Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nix_rs: Don't hardcode flake schema types #235

Merged
merged 35 commits into from
Sep 24, 2024
Merged
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
c28fdbf
nix_rs: Remove `schema` module; Enable generic schemas lookup from `F…
shivaraj-bh Aug 26, 2024
0dab936
fix clippy
shivaraj-bh Aug 26, 2024
745613d
extra line
shivaraj-bh Aug 27, 2024
50fce31
Merge branch 'main' into generic-schemas
shivaraj-bh Sep 19, 2024
f701b83
rm schema.rs
shivaraj-bh Sep 19, 2024
204e68d
revert omnix-gui changes
shivaraj-bh Sep 19, 2024
e904b57
fix clippy and add todo
shivaraj-bh Sep 19, 2024
bd746f0
re-write doc for lookup_returning_qualified_attributes
shivaraj-bh Sep 19, 2024
279965f
introduce get_inventory fn; rm pop from FlakeOutputs
shivaraj-bh Sep 19, 2024
1de6ec7
remove irrelevant doc string
shivaraj-bh Sep 19, 2024
b0f134c
Merge branch 'main' into generic-schemas
shivaraj-bh Sep 19, 2024
4452812
add changelog
shivaraj-bh Sep 19, 2024
e12ff2a
Row::from_flake_outputs_for -> Row::from_inventory_for
shivaraj-bh Sep 19, 2024
0e58c80
Everything about flake-schemas must be in schema.rs
shivaraj-bh Sep 23, 2024
6154d44
update nix_rs changelog
shivaraj-bh Sep 23, 2024
0912275
fix clippy; remove unused function def
shivaraj-bh Sep 23, 2024
7dd2a65
Merge branch 'main' into generic-schemas
shivaraj-bh Sep 23, 2024
4df2e6b
remove system from Flake struct; edit documentation for FlakeSchemas …
shivaraj-bh Sep 23, 2024
0472200
fix doc
shivaraj-bh Sep 23, 2024
3f9c73f
Merge branch 'main' into generic-schemas
shivaraj-bh Sep 23, 2024
79f2177
document schema.rs and outputs.rs
shivaraj-bh Sep 23, 2024
0432202
as_vec -> get_children
shivaraj-bh Sep 23, 2024
c877c77
as_val -> get_val
shivaraj-bh Sep 23, 2024
9de27ef
BtreeMap -> HashMap; replace FlakeOutputs::pop with get
shivaraj-bh Sep 23, 2024
b1e6aff
Modify doc of FlakeSchemas::from_nix
shivaraj-bh Sep 23, 2024
e5c6d6a
Simplify doc for Val type
shivaraj-bh Sep 23, 2024
ce7a346
allow missing_docs to avoid repetition; Improve doc for Leaf enum
shivaraj-bh Sep 23, 2024
f948e1a
typo
shivaraj-bh Sep 23, 2024
d2f2b1d
fix clippy warn; get needn't be generic, its okay to only work with str
shivaraj-bh Sep 23, 2024
4a2a2c8
https://srid.ca/coding#fn-interface
srid Sep 24, 2024
9134ee2
Use builtin traits when appropriate
srid Sep 24, 2024
2682915
avoid "_ => None" which can accidentally miss if we add more construc…
srid Sep 24, 2024
8db48b1
refactor a bit
srid Sep 24, 2024
cada611
Add this important distinction to doc!
srid Sep 24, 2024
f7b9e64
fix: doc
srid Sep 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion crates/nix_rs/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Changelog

## Unreleased

- **`flake::schema`**
- Don't hardcode flake schema types
- **`env`**:
- use `whoami` crate to find the current user instead of depending on environment variable `USER`
- Support Nix 2.20
14 changes: 6 additions & 8 deletions crates/nix_rs/src/flake/mod.rs
Original file line number Diff line number Diff line change
@@ -8,11 +8,13 @@ pub mod schema;
pub mod system;
pub mod url;

use schema::FlakeSchemas;
use serde::{Deserialize, Serialize};

use system::System;
use tracing::instrument;

use self::{outputs::FlakeOutputs, schema::FlakeSchema, system::System, url::FlakeUrl};
use self::{outputs::FlakeOutputs, url::FlakeUrl};

use crate::{
command::{NixCmd, NixCmdError},
@@ -24,10 +26,8 @@ use crate::{
pub struct Flake {
/// The flake url which this struct represents
pub url: FlakeUrl,
/// `nix flake show` output
/// Flake outputs derived from [FlakeSchemas]
pub output: FlakeOutputs,
/// Flake output schema (typed version of [FlakeOutputs])
pub schema: FlakeSchema,
// TODO: Add `nix flake metadata` info.
}

@@ -40,12 +40,10 @@ impl Flake {
nix_config: &NixConfig,
url: FlakeUrl,
) -> Result<Flake, NixCmdError> {
let output = FlakeOutputs::from_nix(nix_cmd, &url, &nix_config.system.value).await?;
let schema = FlakeSchema::from(&output, &nix_config.system.value);
let schemas = FlakeSchemas::from_nix(nix_cmd, &url, &nix_config.system.value).await?;
Ok(Flake {
url,
output: output.clone(),
schema,
output: schemas.into(),
})
}
}
254 changes: 38 additions & 216 deletions crates/nix_rs/src/flake/outputs.rs
Original file line number Diff line number Diff line change
@@ -1,245 +1,67 @@
//! Nix flake outputs
// TODO: Document this module!
#![allow(missing_docs)]

use lazy_static::lazy_static;
use serde::{Deserialize, Serialize};
use std::{
collections::{btree_map::Entry, BTreeMap},
fmt::Display,
path::Path,
};
use std::collections::HashMap;

use crate::system_list::SystemsListFlakeRef;

use super::{command::FlakeOptions, eval::nix_eval, url::FlakeUrl};

lazy_static! {
/// Flake URL of the default flake schemas
///
/// We expect this environment to be set in Nix build and shell.
pub static ref DEFAULT_FLAKE_SCHEMAS: FlakeUrl = {
Into::<FlakeUrl>::into(Path::new(env!("DEFAULT_FLAKE_SCHEMAS")))
};

/// Flake URL of the flake that defines functions for inspecting flake outputs
///
/// We expect this environment to be set in Nix build and shell.
pub static ref INSPECT_FLAKE: FlakeUrl = {
Into::<FlakeUrl>::into(Path::new(env!("INSPECT_FLAKE")))
};
}

/// Represents the "outputs" of a flake
///
/// TODO: Rename this to `FlakeSchema` while generalizing the existing `schema.rs` module.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct FlakeOutputs {
pub inventory: BTreeMap<String, InventoryItem>,
}
use super::schema::{FlakeSchemas, Val};

/// Outputs of a flake
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(untagged)]
pub enum InventoryItem {
Leaf(Leaf),
Attrset(BTreeMap<String, InventoryItem>),
pub enum FlakeOutputs {
/// Terminal value that is not an attrset.
Val(Val),
/// An attrset of nested [FlakeOutputs]
Attrset(HashMap<String, FlakeOutputs>),
}

impl FlakeOutputs {
/// Determine flake outputs using [static@INSPECT_FLAKE] and [static@DEFAULT_FLAKE_SCHEMAS]
pub async fn from_nix(
nix_cmd: &crate::command::NixCmd,
flake_url: &super::url::FlakeUrl,
system: &super::System,
) -> Result<Self, crate::command::NixCmdError> {
let inspect_flake: FlakeUrl = INSPECT_FLAKE
// Why `exculdingOutputPaths`?
// This function is much faster than `includingOutputPaths` and also solves <https://github.com/juspay/omnix/discussions/231>
// Also See: https://github.com/DeterminateSystems/inspect/blob/7f0275abbdc46b3487ca69e2acd932ce666a03ff/flake.nix#L139
//
//
// Note: We might need to use `includingOutputPaths` in the future, when replacing `devour-flake`.
// In which case, `om ci` and `om show` can invoke the appropriate function from `INSPECT_FLAKE`.
//
.with_attr("contents.excludingOutputPaths");
let systems_flake = SystemsListFlakeRef::from_known_system(system)
// TODO: don't use unwrap
.unwrap()
.0
.clone();
let flake_opts = FlakeOptions {
no_write_lock_file: true,
override_inputs: BTreeMap::from_iter([
(
"flake-schemas".to_string(),
DEFAULT_FLAKE_SCHEMAS.to_owned(),
),
("flake".to_string(), flake_url.clone()),
("systems".to_string(), systems_flake),
]),
..Default::default()
};
let v = nix_eval::<Self>(nix_cmd, &flake_opts, &inspect_flake).await?;
Ok(v)
}
}

impl InventoryItem {
/// Get the non-attrset leaf
pub fn as_leaf(&self) -> Option<&Leaf> {
/// Get the terminal value
pub fn get_val(&self) -> Option<&Val> {
match self {
Self::Leaf(v) => Some(v),
Self::Val(v) => Some(v),
_ => None,
}
}

/// Ensure the value is an attrset, and get it
pub fn as_attrset(&self) -> Option<&BTreeMap<String, InventoryItem>> {
/// Get the attrset
pub fn get_attrset(&self) -> Option<&HashMap<String, FlakeOutputs>> {
match self {
Self::Attrset(v) => Some(v),
_ => None,
Self::Val(_) => None,
Self::Attrset(map) => Some(map),
}
}

/// Lookup the given path, returning the value, while removing it from the tree.
/// Get the attrset as a vector of key-value pairs
///
/// **NOTE**: Only terminal values are included!
pub fn get_attrset_of_val(&self) -> Vec<(String, Val)> {
self.get_attrset().map_or(vec![], |map| {
map.iter()
.filter_map(|(k, v)| v.get_val().map(|val| (k.clone(), val.clone())))
.collect()
})
}

/// Lookup the given path, returning a reference to the value if it exists.
///
/// # Example
/// ```no_run
/// let tree : &nix_rs::flake::outputs::InventoryItem = todo!();
/// let val = tree.pop(&["aarch64-darwin", "default"]);
/// let tree : &nix_rs::flake::outputs::FlakeOutputs = todo!();
/// let val = tree.get_by_path(&["aarch64-darwin", "default"]);
/// ```
pub fn pop(&mut self, path: &[&str]) -> Option<Self> {
let mut curr = self;
let mut path = path.iter().peekable();
while let Some(part) = path.next() {
let Self::Attrset(v) = curr else {
return None;
};
let Entry::Occupied(entry) = v.entry(part.to_string()) else {
return None;
};
if path.peek().is_none() {
return Some(entry.remove());
} else {
curr = entry.into_mut();
}
}
None
}
}

/// Represents a leaf value of a flake output
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(untagged)]
pub enum Leaf {
Val(Val),
Unknown(Unknown),
Filtered(Filtered),
Skipped(Skipped),
/// Represents description for a flake output
/// (e.g. `Doc` for `formatter` will be "The `formatter` output specifies the package to use to format the project.")
Doc(String),
}

impl Leaf {
/// Get the value as a [Val]
pub fn as_val(&self) -> Option<&Val> {
match self {
Self::Val(v) => Some(v),
_ => None,
}
}
}

/// The metadata of a flake output value which is of non-attrset [Type]
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Val {
#[serde(rename = "what")]
pub type_: Type,
pub derivation_name: Option<String>,
pub short_description: Option<String>,
}

impl Default for Val {
fn default() -> Self {
Self {
type_: Type::Unknown,
derivation_name: None,
short_description: None,
}
}
}

/// Boolean flags at the leaf of a flake output
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Unknown {
pub unknown: bool,
}

/// Represents flake outputs that cannot be evaluated on current platform
/// (e.g. `nixosConfigurations` on darwin System)
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename = "camelCase")]
pub struct Filtered {
pub filtered: bool,
}

/// Represents flake outputs that are skipped unless explicitly requested
/// (e.g. `legacyPackages`)
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Skipped {
pub skipped: bool,
}

/// The type of a flake output [Val]
///
/// These types can differ based on [static@DEFAULT_FLAKE_SCHEMAS].
/// The types here are based on <https://github.com/DeterminateSystems/flake-schemas>
/// For example, see [NixosModule type](https://github.com/DeterminateSystems/flake-schemas/blob/0a5c42297d870156d9c57d8f99e476b738dcd982/flake.nix#L268)
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub enum Type {
#[serde(rename = "NixOS module")]
NixosModule,
#[serde(rename = "NixOS configuration")]
NixosConfiguration,
#[serde(rename = "nix-darwin configuration")]
DarwinConfiguration,
#[serde(rename = "package")]
Package,
#[serde(rename = "development environment")]
DevShell,
#[serde(rename = "CI test")]
Check,
#[serde(rename = "app")]
App,
#[serde(rename = "template")]
Template,
#[serde(other)]
Unknown,
}

impl Type {
/// Get the icon for this type
pub fn to_icon(&self) -> &'static str {
match self {
Self::NixosModule => "❄️",
Self::NixosConfiguration => "🔧",
Self::DarwinConfiguration => "🍎",
Self::Package => "📦",
Self::DevShell => "🐚",
Self::Check => "🧪",
Self::App => "📱",
Self::Template => "🏗️",
Self::Unknown => "❓",
pub fn get_by_path(&self, path: &[&str]) -> Option<&Self> {
let mut current = self;
for key in path {
let map = current.get_attrset()?;
current = map.get(*key)?;
}
Some(current)
}
}

impl Display for Type {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&format!("{:?}", self))
impl From<FlakeSchemas> for FlakeOutputs {
fn from(schema: FlakeSchemas) -> Self {
schema.to_flake_outputs()
}
}
Loading