Skip to content

problems() reports the wrong file for lazy subsets of multi-file reads #642

Open

Description

@kevinushey

Found while reviewing #638; pre-existing (reproduced on vroom 1.7.0, R 4.6.1, macOS).

When a lazy character column from a multi-file read is subset in R (still ALTREP) and a parse problem is recorded during element access, the recorded file is always the first file:

f1 <- tempfile(pattern = "clean-")
f2 <- tempfile(pattern = "dirty-")

writeLines(c("x", "a", "b"), f1)
# header "x", then row "c<NUL>d" -- embedded null
writeBin(as.raw(c(0x78, 0x0a, 0x63, 0x00, 0x64, 0x0a)), f2)

x <- vroom::vroom(c(f1, f2), delim = ",", col_types = "c", altrep = "chr")

# lazy subset: element 2 maps to the embedded-null row in f2
y <- x$x[c(1, 3)]
y[[2]]
#> [1] "c"
#> Warning: One or more parsing issues ...

as.data.frame(vroom::problems(x, lazy = TRUE))
#>   row col expected        actual                  file
#> 1   2   1          embedded null /tmp/.../clean-...

The embedded null is in dirty-, but the problem points at clean-.

Cause: subset_iterator::filename() at src/index.h:53 returns it_.filename(), but it_ only advances inside value(); next()/advance() only bump i_. (index() on the very next line already offsets correctly.) One-line fix:

std::string filename() const override {
  return (start_ + (*indexes_)[i_]).filename();
}

Two notes:

  • On 1.7.0 the row recorded on this path is wrong as well (Val() used info.column->begin() without offsetting); that half is fixed by Fix for ALTREP crash in base R CHARSXP use-after-free #638. The stale filename is not.
  • The same filename() is used by the other lazy subset column types (int/dbl/num), so those record the wrong file too.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions