Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nix_rs: Inherit stderr while running nix as a child process #222

Closed
wants to merge 25 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
9dbce04
nix_rs: Stream stdout of nix to om
shivaraj-bh Aug 16, 2024
acd91cf
add fixme to capture stderr
shivaraj-bh Aug 16, 2024
69888ca
Merge branch 'main' into stream-stdout
shivaraj-bh Aug 31, 2024
8426455
find qualified attr in flake output; remove Leaf for simplicity
shivaraj-bh Sep 2, 2024
e569b2b
invoke flake show with `url.without_attr()`
shivaraj-bh Sep 2, 2024
a9ad465
remove usage of nix_eval_attr
shivaraj-bh Sep 2, 2024
45ffdbd
is eval.rs needed anymore?
shivaraj-bh Sep 2, 2024
9d19ad0
ci_config_from_flake_url -> ci_config_from_flake_outputs
shivaraj-bh Sep 2, 2024
2ef7795
Merge branch 'main' into stream-stdout
srid Sep 2, 2024
49e164e
Merge branch 'main' into stream-stdout
shivaraj-bh Sep 3, 2024
0aab0e0
Actually use custom flake schema
shivaraj-bh Sep 3, 2024
eeeb953
Add `om`, `nixci` and `nix-health` schemas to custom schema
shivaraj-bh Sep 3, 2024
9f1b0cc
fmt flake-schemas/flake.nix
shivaraj-bh Sep 3, 2024
aebe695
Add FilteredFlakeOutputs; remove qualified_attr.rs
shivaraj-bh Sep 3, 2024
1cf31ec
Merge branch 'main' into stream-stdout
shivaraj-bh Sep 3, 2024
bc412db
rename stdout_stream to stdout_handle
shivaraj-bh Sep 3, 2024
c0e87ea
update changelog and history.md
shivaraj-bh Sep 3, 2024
f0d28f8
refactor(dry): flake-schemas/flake.nix
shivaraj-bh Sep 3, 2024
66ac5c7
edit
shivaraj-bh Sep 4, 2024
fda748d
Merge branch 'main' into stream-stdout
shivaraj-bh Sep 24, 2024
def7f66
support `FlakeOutputs`
shivaraj-bh Sep 29, 2024
61cc65d
`registry.rs` added by mistake while resolving merge conflicts
shivaraj-bh Sep 29, 2024
865c963
use url without attr while fetching flake schemas
shivaraj-bh Oct 4, 2024
5f56bda
Merge branch 'main' into stream-stdout
shivaraj-bh Oct 4, 2024
aa80d32
revert omnix-gui changes
shivaraj-bh Oct 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions crates/nix_health/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## Unreleased

- The flake config is now fetched from flake outputs using flake-schemas instead of `nix eval`
- Remove unused `logging` module
- Display Nix installer used

Expand Down
4 changes: 2 additions & 2 deletions crates/nix_health/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,8 @@ impl NixHealth {
/// override it.
pub async fn from_flake(url: &FlakeUrl) -> Result<Self, OmConfigError> {
let cmd = NixCmd::get().await;
let cfg =
OmConfig::<NixHealth>::from_flake_url(cmd, url, &["om.health", "nix-health"]).await?;
let cfg = OmConfig::<NixHealth>::from_nix(cmd, url, &[&["om", "health"], &["nix-health"]])
.await?;
let (cfg, _rest) = cfg.get_referenced()?;
Ok(cfg.clone())
}
Expand Down
7 changes: 7 additions & 0 deletions crates/nix_rs/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
# Changelog

## Unreleased

- **Inherit `stderr`**
- `NixCmd::run_with_args_returning_stdout` now inherits `stderr` from parent process by default.
- `stderr` is no longer captured and passed along with `CommandError::ProcessFailed`.
- `nix_eval_attr` no longer supports missing attribute check.
- Introduce `FilteredFlakeOutputs` to fetch the flake config, avoiding dependency on `nix_eval_attr`.
- Remove `qualified_attr` module, use `find_qualified_attr` from `FilteredFlakeOutputs` instead.
- **`flake::schema`**
- Don't hardcode flake schema types
- **`env`**:
Expand Down
57 changes: 28 additions & 29 deletions crates/nix_rs/src/command.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,11 @@ use std::{
use serde::{Deserialize, Serialize};
use thiserror::Error;

use tokio::{process::Command, sync::OnceCell};
use tokio::{
io::AsyncReadExt,
process::{ChildStdout, Command},
sync::OnceCell,
};

use tracing::instrument;

Expand Down Expand Up @@ -118,11 +122,17 @@ impl NixCmd {
where
T: serde::de::DeserializeOwned,
{
let stdout: Vec<u8> = self
let mut stdout_handle = self
.run_with_returning_stdout(|c| {
c.args(args);
})
.await?;
let mut stdout = Vec::new();

stdout_handle
.read_to_end(&mut stdout)
.await
.map_err(CommandError::ChildProcessError)?;
let v = serde_json::from_slice::<T>(&stdout)?;
Ok(v)
}
Expand All @@ -133,39 +143,38 @@ impl NixCmd {
T: std::str::FromStr,
<T as std::str::FromStr>::Err: std::fmt::Display,
{
let stdout = self
let mut stdout_handle = self
.run_with_returning_stdout(|c| {
c.args(args);
})
.await?;
let mut stdout = Vec::new();
stdout_handle
.read_to_end(&mut stdout)
.await
.map_err(CommandError::ChildProcessError)?;
let v = &String::from_utf8_lossy(&stdout);
let v = T::from_str(v.trim()).map_err(|e| FromStrError(e.to_string()))?;
Ok(v)
}

/// Like [Self::run_with] but returns stdout as a [`Vec<u8>`]
pub async fn run_with_returning_stdout<F>(&self, f: F) -> Result<Vec<u8>, CommandError>
pub async fn run_with_returning_stdout<F>(&self, f: F) -> Result<ChildStdout, CommandError>
where
F: FnOnce(&mut Command),
{
let mut cmd = self.command();
f(&mut cmd);
cmd.stdout(Stdio::piped());
trace_cmd(&cmd);

cmd.stdout(Stdio::piped());
cmd.stderr(Stdio::piped());
let child = cmd.spawn()?;
let out = child.wait_with_output().await?;
let mut child = cmd.spawn()?;
let exit_status = child.wait().await?; // Wait for the child to finish

if out.status.success() {
Ok(out.stdout)
} else {
let stderr = String::from_utf8(out.stderr)?;
Err(CommandError::ProcessFailed {
stderr: Some(stderr),
exit_code: out.status.code(),
})
}
let stdout = child.stdout.take().ok_or(CommandError::ProcessFailed {
exit_code: exit_status.code(),
})?;
Ok(stdout)
}

/// Run Nix with given [Command] customizations, while also tracing the command being run.
Expand All @@ -181,7 +190,6 @@ impl NixCmd {
Ok(())
} else {
Err(CommandError::ProcessFailed {
stderr: None,
exit_code: status.code(),
})
}
Expand Down Expand Up @@ -264,18 +272,9 @@ pub enum CommandError {
ChildProcessError(#[from] std::io::Error),

/// Child process exited unsuccessfully
#[error(
"Process exited unsuccessfully. exit_code={:?}{}",
exit_code,
match stderr {
Some(s) => format!(" stderr={}", s),
None => "".to_string()
},
)]
#[error("Process exited unsuccessfully. exit_code={:?}", exit_code)]
ProcessFailed {
/// The stderr of the process, if available.
stderr: Option<String>,
/// The exit code of the process
/// The exit code of the failed process
exit_code: Option<i32>,
},

Expand Down
37 changes: 11 additions & 26 deletions crates/nix_rs/src/flake/eval.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
//! Work with `nix eval`
use tokio::io::AsyncReadExt;

use crate::command::{CommandError, NixCmd, NixCmdError};

use super::{command::FlakeOptions, url::FlakeUrl};
Expand All @@ -12,13 +14,19 @@ pub async fn nix_eval<T>(
where
T: serde::de::DeserializeOwned,
{
let stdout = nixcmd
let mut stdout_handle = nixcmd
.run_with_returning_stdout(|cmd| {
cmd.args(["eval", "--json"]);
opts.use_in_command(cmd);
cmd.arg(url.to_string());
})
.await?;
let mut stdout = Vec::new();

stdout_handle
.read_to_end(&mut stdout)
.await
.map_err(CommandError::ChildProcessError)?;
let v = serde_json::from_slice::<T>(&stdout)?;
Ok(v)
}
Expand All @@ -32,29 +40,6 @@ pub async fn nix_eval_attr<T>(cmd: &NixCmd, url: &FlakeUrl) -> Result<Option<T>,
where
T: Default + serde::de::DeserializeOwned,
{
let result = cmd
.run_with_args_expecting_json(&["eval", &url.0, "--json"])
.await;
match result {
Ok(v) => Ok(Some(v)),
Err(err) if error_is_missing_attribute(&err) => {
Ok(None) // Attr is missing
}
Err(err) => Err(err),
}
}

/// Check that [NixCmdError] is a missing attribute error
fn error_is_missing_attribute(err: &NixCmdError) -> bool {
match err {
NixCmdError::CmdError(CommandError::ProcessFailed { stderr, .. }) => {
if let Some(stderr) = stderr {
if stderr.contains("does not provide attribute") {
return true;
}
}
false
}
_ => false,
}
cmd.run_with_args_expecting_json(&["eval", &url.0, "--json"])
.await
}
29 changes: 28 additions & 1 deletion crates/nix_rs/src/flake/outputs.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
//! Nix flake outputs

use serde::{Deserialize, Serialize};
use serde::{de::DeserializeOwned, Deserialize, Serialize};
use std::collections::HashMap;

use super::schema::{FlakeSchemas, Val};
Expand All @@ -10,11 +10,19 @@ use super::schema::{FlakeSchemas, Val};
#[serde(untagged)]
pub enum FlakeOutputs {
/// Terminal value that is not an attrset.
#[serde(serialize_with = "value_serializer")]
Val(Val),
/// An attrset of nested [FlakeOutputs]
Attrset(HashMap<String, FlakeOutputs>),
}

fn value_serializer<S>(val: &Val, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
val.value.serialize(serializer)
}

impl FlakeOutputs {
/// Get the terminal value
pub fn get_val(&self) -> Option<&Val> {
Expand Down Expand Up @@ -58,6 +66,25 @@ impl FlakeOutputs {
}
Some(current)
}

/// Lookup the given paths, returning the first match.
pub fn get_first_by_paths(&self, paths: &[&[&str]]) -> Option<&Self> {
for path in paths {
if let Some(v) = self.get_by_path(path) {
return Some(v);
}
}
None
}

/// Deserialize the FlakeOutputs into a generic type T
pub fn deserialize<T>(&self) -> Result<T, serde_json::Error>
where
T: Default + DeserializeOwned + std::fmt::Debug,
{
let json_value = serde_json::to_value(self)?;
serde_json::from_value(json_value)
}
}

impl From<FlakeSchemas> for FlakeOutputs {
Expand Down
3 changes: 3 additions & 0 deletions crates/nix_rs/src/flake/schema.rs
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,8 @@ pub struct Val {
pub derivation_name: Option<String>,
/// A short description derived from `meta.description` of the derivation with [Val::derivation_name]
pub short_description: Option<String>,
/// The raw value of the flake output
pub value: Option<serde_json::Value>,
}

impl Default for Val {
Expand All @@ -159,6 +161,7 @@ impl Default for Val {
type_: Type::Unknown,
derivation_name: None,
short_description: None,
value: None,
}
}
}
Expand Down
1 change: 0 additions & 1 deletion crates/nix_rs/src/flake/url/mod.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
//! Work with flake URLs
pub mod attr;
mod core;
pub mod qualified_attr;

pub use core::*;
Loading