Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add options to exclude files #238

Merged
merged 11 commits into from
Dec 20, 2024
1 change: 0 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

30 changes: 30 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -152,6 +152,36 @@ preview = true

Run `fortitude explain` to see which rules are in preview mode.

### Excluding Files
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe instead this should be in docs/index.md?

Actually, looking at ruff, they have some tools for extracting portions of the README.md into the site docs. We should look at that at some point

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now, I've moved that section to the docs along with some other things from the README.


> [!NOTE]
> To be featured in v0.7.0

Fortitude will automatically ignore files in some directories (`build/`, `.git/`,
`.venv/`, etc.), and this behaviour can be extended using the `--exclude` option. For
example, to ignore all files in the directory `benchmarks/`:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, does --exclude not override the builtins?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From playing around with Ruff, apparently not, as it behaves as if the built-ins are a mandatory set of --extend-exclude instead of --exclude. If you want it to lint everything in your venv, you have to manually pass it venv/lib/Python3.10/site-packages, and setting your own exclude doesn't change this behaviour.

I haven't played around with include in a config file, so I'm not sure how that might affect it.


```console
$ fortitude check --exclude=benchmarks
```

Note that this will override any exclusions specified in your `fpm.toml` or
`fortitude.toml` file. To extend those exclusions, use instead:

```console
$ fortitude check --extend-exclude=benchmarks
```

It is also possible to switch off individual rules or rule categories for specific
files using `--per-file-ignores`:

```console
$ fortitude check --per-file-ignores=**/*.f95:non-standard-file-extension
```

Similarly, if these are set in the configuration file, `--extend-per-file-ignores` can
be used to add additional rule ignores instead of overwriting them.

## Configuration

Fortitude will look for either a `fortitude.toml` or `fpm.toml` file in the
1 change: 0 additions & 1 deletion fortitude/Cargo.toml
Original file line number Diff line number Diff line change
@@ -57,7 +57,6 @@ unicode-width = "0.2.0"
url = { version = "2.5.0" }
walkdir = "2.4.0"
globset = "0.4.15"
glob = "0.3.1"

[build-dependencies]
shadow-rs = { version = "0.36.0", default-features = false }
74 changes: 60 additions & 14 deletions fortitude/src/check.rs
Original file line number Diff line number Diff line change
@@ -12,8 +12,8 @@ use crate::rule_selector::{
use crate::rules::Rule;
use crate::rules::{error::ioerror::IoError, AstRuleEnum, PathRuleEnum, TextRuleEnum};
use crate::settings::{
FixMode, OutputFormat, PatternPrefixPair, PreviewMode, ProgressBar, Settings, UnsafeFixes,
DEFAULT_SELECTORS,
ExcludeMode, FilePattern, FilePatternSet, FixMode, OutputFormat, PatternPrefixPair,
PreviewMode, ProgressBar, Settings, UnsafeFixes, DEFAULT_SELECTORS,
};

use anyhow::{anyhow, Context, Result};
@@ -55,6 +55,25 @@ struct CheckSection {
check: Option<CheckArgs>,
}

// Default paths to exclude when searching paths
pub(crate) static EXCLUDE_BUILTINS: &[FilePattern] = &[
FilePattern::Builtin(".git"),
FilePattern::Builtin(".git-rewrite"),
FilePattern::Builtin(".hg"),
FilePattern::Builtin(".svn"),
FilePattern::Builtin("venv"),
FilePattern::Builtin(".venv"),
FilePattern::Builtin("pyenv"),
FilePattern::Builtin(".pyenv"),
FilePattern::Builtin(".eggs"),
FilePattern::Builtin("site-packages"),
FilePattern::Builtin(".vscode"),
FilePattern::Builtin("build"),
FilePattern::Builtin("_build"),
FilePattern::Builtin("dist"),
FilePattern::Builtin("_dist"),
];

// Adapted from ruff
fn parse_fpm_toml<P: AsRef<Path>>(path: P) -> Result<Fpm> {
let contents = std::fs::read_to_string(path.as_ref())
@@ -154,6 +173,9 @@ pub struct CheckSettings {
pub output_format: OutputFormat,
pub progress_bar: ProgressBar,
pub preview: PreviewMode,
pub exclude: Option<Vec<FilePattern>>,
pub extend_exclude: Vec<FilePattern>,
pub exclude_mode: ExcludeMode,
}

/// Read either fpm.toml or fortitude.toml into our "known good" file
@@ -192,6 +214,11 @@ fn parse_config_file(config_file: &Option<PathBuf>) -> Result<CheckSettings> {
preview: resolve_bool_arg(value.preview, value.no_preview)
.map(PreviewMode::from)
.unwrap_or_default(),
exclude: value.exclude,
extend_exclude: value.extend_exclude.unwrap_or_default(),
exclude_mode: resolve_bool_arg(value.force_exclude, value.no_force_exclude)
.map(ExcludeMode::from)
.unwrap_or_default(),
},
None => CheckSettings::default(),
};
@@ -241,9 +268,8 @@ fn ruleset(args: RuleSelection, preview: &PreviewMode) -> anyhow::Result<Vec<Rul
Ok(rules)
}

/// Helper function used with `filter` to select only paths that end in a Fortran extension.
/// Includes non-standard extensions, as these should be reported.
fn filter_fortran_extensions<S: AsRef<str>>(path: &Path, extensions: &[S]) -> bool {
/// Helper function used with `get_files` to select only paths that end in a Fortran extension.
fn is_valid_extension<S: AsRef<str>>(path: &Path, extensions: &[S]) -> bool {
if let Some(ext) = path.extension() {
// Can't use '&[&str].contains()', as extensions are of type OsStr
extensions.iter().any(|x| x.as_ref() == ext)
@@ -256,24 +282,28 @@ fn filter_fortran_extensions<S: AsRef<str>>(path: &Path, extensions: &[S]) -> bo
fn get_files<P: AsRef<Path>, S: AsRef<str>>(
paths: &[P],
extensions: &[S],
) -> anyhow::Result<Vec<PathBuf>> {
excludes: &FilePatternSet,
exclude_mode: ExcludeMode,
) -> Vec<PathBuf> {
paths
.iter()
.flat_map(|path| {
if path.as_ref().is_dir() {
if matches!(exclude_mode, ExcludeMode::Force) && excludes.matches(path) {
vec![]
} else if path.as_ref().is_dir() {
WalkDir::new(path)
.min_depth(1)
.into_iter()
.filter_map(|x| x.ok()) // skip dirs if user doesn't have permission
.filter(|x| filter_fortran_extensions(x.path(), extensions))
.map(|x| std::path::absolute(x.path()))
.filter_entry(|e| !excludes.matches(e.path()))
.filter_map(|p| p.ok()) // skip dirs if user doesn't have permission
.filter(|p| is_valid_extension(p.path(), extensions))
.map(|p| fs::normalize_path(p.path()))
.collect::<Vec<_>>()
} else {
vec![std::path::absolute(path)]
vec![fs::normalize_path(path)]
}
})
.collect::<Result<Vec<_>, _>>()
.map_err(anyhow::Error::new)
.collect()
}

/// Parse a file, check it for issues, and return the report.
@@ -718,6 +748,22 @@ pub fn check(args: CheckArgs, global_options: &GlobalConfigArgs) -> Result<ExitC
.collect::<Vec<_>>(),
))?;

let file_excludes = FilePatternSet::try_from_iter(
EXCLUDE_BUILTINS
.iter()
.cloned()
.chain(
args.exclude
.unwrap_or(file_settings.exclude.unwrap_or_default())
.into_iter(),
)
.chain(args.extend_exclude.unwrap_or_default().into_iter())
.chain(file_settings.extend_exclude.into_iter()),
)?;
let exclude_mode = resolve_bool_arg(args.force_exclude, args.no_force_exclude)
.map(ExcludeMode::from)
.unwrap_or(file_settings.exclude_mode);

let output_format = args.output_format.unwrap_or(file_settings.output_format);
let preview_mode = resolve_bool_arg(args.preview, args.no_preview)
.map(PreviewMode::from)
@@ -762,7 +808,7 @@ pub fn check(args: CheckArgs, global_options: &GlobalConfigArgs) -> Result<ExitC
let text_rules = rules_to_text_rules(&rules);
let ast_entrypoints = ast_entrypoint_map(&rules);

let files = get_files(files, file_extensions)?;
let files = get_files(files, file_extensions, &file_excludes, exclude_mode);
let file_digits = files.len().to_string().len();
let progress_bar_style = match progress_bar {
ProgressBar::Fancy => {
143 changes: 98 additions & 45 deletions fortitude/src/cli.rs
Original file line number Diff line number Diff line change
@@ -3,8 +3,8 @@ use serde::Deserialize;
use std::path::PathBuf;

use crate::{
build, rule_selector::RuleSelector, settings::OutputFormat, settings::PatternPrefixPair,
settings::ProgressBar, RuleSelectorParser,
build, rule_selector::RuleSelector, settings::FilePattern, settings::OutputFormat,
settings::PatternPrefixPair, settings::ProgressBar, RuleSelectorParser,
};

/// Default extensions to check
@@ -31,6 +31,7 @@ pub struct GlobalConfigArgs {
pub config_file: Option<PathBuf>,
}

#[allow(clippy::large_enum_variant)]
#[derive(Debug, Subcommand, Clone, PartialEq)]
pub enum SubCommands {
Check(CheckArgs),
@@ -60,68 +61,28 @@ pub struct CheckArgs {
/// are included in the search.
#[arg(default_value = ".")]
pub files: Option<Vec<PathBuf>>,
/// Comma-separated list of rules to ignore.
#[arg(
long,
value_delimiter = ',',
value_name = "RULE_CODE",
value_parser = RuleSelectorParser,
help_heading = "Rule selection",
hide_possible_values = true
)]
pub ignore: Option<Vec<RuleSelector>>,
/// Comma-separated list of rule codes to enable (or ALL, to enable all rules).
#[arg(
long,
value_delimiter = ',',
value_name = "RULE_CODE",
value_parser = RuleSelectorParser,
help_heading = "Rule selection",
hide_possible_values = true
)]
pub select: Option<Vec<RuleSelector>>,
/// Like --select, but adds additional rule codes on top of those already specified.
#[arg(
long,
value_delimiter = ',',
value_name = "RULE_CODE",
value_parser = RuleSelectorParser,
help_heading = "Rule selection",
hide_possible_values = true
)]
pub extend_select: Option<Vec<RuleSelector>>,
/// List of mappings from file pattern to code to exclude.
#[arg(long, value_delimiter = ',', help_heading = "Rule selection")]
pub per_file_ignores: Option<Vec<PatternPrefixPair>>,
/// Like `--per-file-ignores`, but adds additional ignores on top of those already specified.
#[arg(long, value_delimiter = ',', help_heading = "Rule selection")]
pub extend_per_file_ignores: Option<Vec<PatternPrefixPair>>,

/// Set the maximum allowable line length.
#[arg(long, default_value = "100")]
pub line_length: Option<usize>,
/// File extensions to check
#[arg(long, value_delimiter = ',', default_values = FORTRAN_EXTS)]
pub file_extensions: Option<Vec<String>>,

/// Apply fixes to resolve lint violations.
/// Use `--no-fix` to disable or `--unsafe-fixes` to include unsafe fixes.
#[arg(long, overrides_with("no_fix"), action = clap::ArgAction::SetTrue)]
pub fix: Option<bool>,
#[clap(long, overrides_with("fix"), hide = true, action = SetTrue)]
pub no_fix: Option<bool>,

/// Include fixes that may not retain the original intent of the code.
/// Use `--no-unsafe-fixes` to disable.
#[arg(long, overrides_with("no_unsafe_fixes"), action = SetTrue)]
pub unsafe_fixes: Option<bool>,
#[arg(long, overrides_with("unsafe_fixes"), hide = true, action = SetTrue)]
pub no_unsafe_fixes: Option<bool>,

/// Show an enumeration of all fixed lint violations.
/// Use `--no-show-fixes` to disable.
#[arg(long, overrides_with("no_show_fixes"), action = SetTrue)]
pub show_fixes: Option<bool>,
#[clap(long, overrides_with("show_fixes"), hide = true, action = SetTrue)]
pub no_show_fixes: Option<bool>,

/// Apply fixes to resolve lint violations, but don't report on, or exit non-zero for, leftover violations. Implies `--fix`.
/// Use `--no-fix-only` to disable or `--unsafe-fixes` to include unsafe fixes.
#[arg(long, overrides_with("no_fix_only"), action = SetTrue)]
@@ -145,4 +106,96 @@ pub struct CheckArgs {
/// Options are "off" (default), "ascii", and "fancy"
#[arg(long, value_enum)]
pub progress_bar: Option<ProgressBar>,

// Rule selection
/// Comma-separated list of rules to ignore.
#[arg(
long,
value_delimiter = ',',
value_name = "RULE_CODE",
value_parser = RuleSelectorParser,
help_heading = "Rule selection",
hide_possible_values = true
)]
pub ignore: Option<Vec<RuleSelector>>,

/// Comma-separated list of rule codes to enable (or ALL, to enable all rules).
#[arg(
long,
value_delimiter = ',',
value_name = "RULE_CODE",
value_parser = RuleSelectorParser,
help_heading = "Rule selection",
hide_possible_values = true
)]
pub select: Option<Vec<RuleSelector>>,

/// Like --select, but adds additional rule codes on top of those already specified.
#[arg(
long,
value_delimiter = ',',
value_name = "RULE_CODE",
value_parser = RuleSelectorParser,
help_heading = "Rule selection",
hide_possible_values = true
)]
pub extend_select: Option<Vec<RuleSelector>>,

/// List of mappings from file pattern to code to exclude.
#[arg(
long,
value_delimiter = ',',
value_name = "FILE_PATTERN:RULE_CODE",
help_heading = "Rule selection"
)]
pub per_file_ignores: Option<Vec<PatternPrefixPair>>,

/// Like `--per-file-ignores`, but adds additional ignores on top of those already specified.
#[arg(
long,
value_delimiter = ',',
value_name = "FILE_PATTERN:RULE_CODE",
help_heading = "Rule selection"
)]
pub extend_per_file_ignores: Option<Vec<PatternPrefixPair>>,

// File selection
/// File extensions to check
#[arg(
long,
value_delimiter = ',',
default_values = FORTRAN_EXTS,
help_heading = "File selection"
)]
pub file_extensions: Option<Vec<String>>,

/// List of paths, used to omit files and/or directories from analysis.
#[arg(
long,
value_delimiter = ',',
value_name = "FILE_PATTERN",
help_heading = "File selection"
)]
pub exclude: Option<Vec<FilePattern>>,

/// Like --exclude, but adds additional files and directories on top of those already excluded.
#[arg(
long,
value_delimiter = ',',
value_name = "FILE_PATTERN",
help_heading = "File selection"
)]
pub extend_exclude: Option<Vec<FilePattern>>,

/// Enforce exclusions, even for paths passed to Fortitude directly on the command-line.
/// Use `--no-force_exclude` to disable.
#[arg(long, overrides_with("no_force_exclude"), help_heading="File selection", action = SetTrue)]
pub force_exclude: Option<bool>,
#[clap(long, overrides_with("force_exclude"), hide = true, action = SetTrue)]
pub no_force_exclude: Option<bool>,

// Options for individual rules
/// Set the maximum allowable line length.
#[arg(long, help_heading = "Per-Rule Options", default_value = "100")]
pub line_length: Option<usize>,
}
Loading