problems() reports the wrong file for lazy subsets of multi-file reads #642
Copy link
Copy link
Open
Description
Found while reviewing #638; pre-existing (reproduced on vroom 1.7.0, R 4.6.1, macOS).
When a lazy character column from a multi-file read is subset in R (still ALTREP) and a parse problem is recorded during element access, the recorded file is always the first file:
f1 <- tempfile(pattern = "clean-")
f2 <- tempfile(pattern = "dirty-")
writeLines(c("x", "a", "b"), f1)
# header "x", then row "c<NUL>d" -- embedded null
writeBin(as.raw(c(0x78, 0x0a, 0x63, 0x00, 0x64, 0x0a)), f2)
x <- vroom::vroom(c(f1, f2), delim = ",", col_types = "c", altrep = "chr")
# lazy subset: element 2 maps to the embedded-null row in f2
y <- x$x[c(1, 3)]
y[[2]]
#> [1] "c"
#> Warning: One or more parsing issues ...
as.data.frame(vroom::problems(x, lazy = TRUE))
#> row col expected actual file
#> 1 2 1 embedded null /tmp/.../clean-...The embedded null is in dirty-, but the problem points at clean-.
Cause: subset_iterator::filename() at src/index.h:53 returns it_.filename(), but it_ only advances inside value(); next()/advance() only bump i_. (index() on the very next line already offsets correctly.) One-line fix:
std::string filename() const override {
return (start_ + (*indexes_)[i_]).filename();
}Two notes:
- On 1.7.0 the
rowrecorded on this path is wrong as well (Val()usedinfo.column->begin()without offsetting); that half is fixed by Fix for ALTREP crash in base R CHARSXP use-after-free #638. The stale filename is not. - The same
filename()is used by the other lazy subset column types (int/dbl/num), so those record the wrong file too.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity