Cannot replay commands

[yarikoptic]

Small description

I was excited to find https://www.visidata.org/docs/save-restore/ since I spent considerable time to tune ordering and assigning formats to columns. Ideally there should (may be there is) a way to store "style" list (order of columns, which are marked as index, what are the chosen formats) but replay of commands should theoretically achieve the same albeit with some "animation" of actions ;)

Data to reproduce

https://www.oneukrainian.com/tmp/git-annex-info.jsonl
https://www.oneukrainian.com/tmp/git-annex-info.vdj

vdj produced with system wide debian install of visidata v3.0.2 . Then loading issue tested/reproduced with current master reported in --version as v3.4dev and otherwise (please use annotated tags for releases; not github lightweight one, related : #2969 ):

(.venv) dandi@drogon:~/proj/misc/visidata$ git describe
v1.3.1-6141-gcf428f5b
(.venv) dandi@drogon:~/proj/misc/visidata$ git describe --tags
v3.2-243-gcf428f5b

Steps to reproduce

NB another effect I noticed is that it seems somehow it relies on the "full" (original) path of the spreadsheet, so can't be replayed in the folder, so full reproducer:

mkdir derivatives;
( cd derivatives; wget https://www.oneukrainian.com/tmp/git-annex-info.jsonl; wget https://www.oneukrainian.com/tmp/git-annex-info.vdj; )
visidata -p derivatives/git-annex-info.vdj derivatives/git-annex-info.jsonl

Expected result

get original order and formats like I have in suspended session

Actual result with screenshot

If there is an error message, please include the full stack trace shown with Ctrl+E ...

says no error

Configuration

• Does this issue reproduce without any plugins or configuration (using the -N CLI flag)?

• Does this issue reproduce with either the latest release, or with the develop branch?

Additional context

NB filed #2970

• What platform and version are you using (Linux, MacOS, Windows)?

Debian GNU/linux stable (IIRC)

• Which version of Python?

3.13.5

• Which terminal are you using (for display and input issues)?

not sure what specific aspect to mention:

gnome terminal -> ssh -> GNU screen -> zsh -> visidata

$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE=en_US.UTF-8
LC_NUMERIC=ru_RU.UTF-8
LC_TIME=en_US.UTF-8
LC_COLLATE=en_US.UTF-8
LC_MONETARY=ru_RU.UTF-8
LC_MESSAGES=en_US.UTF-8
LC_PAPER=ru_RU.UTF-8
LC_NAME=ru_RU.UTF-8
LC_ADDRESS=ru_RU.UTF-8
LC_TELEPHONE=ru_RU.UTF-8
LC_MEASUREMENT=ru_RU.UTF-8
LC_IDENTIFICATION=ru_RU.UTF-8
LC_ALL=

$ export | grep TERM
declare -x COLORTERM="truecolor"
declare -x TERM="screen.xterm-256color"
declare -x TERMCAP="SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual terminal:DO=\\E[%dB:LE=\\E[%dD:RI=\\E[%dC:UP=\\E[%dA:bs:bt=\\E[Z:cd=\\E[J:ce=\\E[K:cl=\\E[H\\E[J:cm=\\E[%i%d;%dH:ct=\\E[3g:do=^J:nd=\\E[C:pt:rc=\\E8:rs=\\Ec:sc=\\E7:st=\\EH:up=\\EM:le=^H:bl=^G:cr=^M:it#8:ho=\\E[H:nw=\\EE:ta=^I:is=\\E)0:li#63:co#226:am:xn:xv:LP:sr=\\EM:al=\\E[L:AL=\\E[%dL:cs=\\E[%i%d;%dr:dl=\\E[M:DL=\\E[%dM:dc=\\E[P:DC=\\E[%dP:im=\\E[4h:ei=\\E[4l:mi:IC=\\E[%d@:ks=\\E[?1h\\E=:ke=\\E[?1l\\E>:vi=\\E[?25l:ve=\\E[34h\\E[?25h:vs=\\E[34l:ti=\\E[?1049h:te=\\E[?1049l:us=\\E[4m:ue=\\E[24m:so=\\E[3m:se=\\E[23m:mb=\\E[5m:md=\\E[1m:mh=\\E[2m:mr=\\E[7m:me=\\E[m:ms:Co#8:pa#64:AF=\\E[3%dm:AB=\\E[4%dm:op=\\E[39;49m:AX:vb=\\Eg:G0:as=\\E(0:ae=\\E(B:ac=\\140\\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:po=\\E[5i:pf=\\E[4i:Km=\\E[<:k0=\\E[10~:k1=\\EOP:k2=\\EOQ:k3=\\EOR:k4=\\EOS:k5=\\E[15~:k6=\\E[17~:k7=\\E[18~:k8=\\E[19~:k9=\\E[20~:k;=\\E[21~:F1=\\E[23~:F2=\\E[24~:kB=\\E[Z:kh=\\E[1~:@1=\\E[1~:kH=\\E[4~:@7=\\E[4~:kN=\\E[6~:kP=\\E[5~:kI=\\E[2~:kD=\\E[3~:ku=\\EOA:kd=\\EOB:kr=\\EOC:kl=\\EOD:km:"

[midichef]

The replay problem is caused by the type-floatsi command on line 47 of your .vdj file:

{"sheet": "git-annex-info_columns", "col": "name", "row": "", "longname": "type-floatsi", "input": "", "keystrokes": "z%", "comment": "set type of current column to SI float"}

If you remove it, replay will succeed.

That line changes the type of the key column name to be an SI float, instead of its default string type. So when the next command slide-up seeks a row with a name matching the string value annex sizes of repositories, it fails to find any. Because it's looking only for a string, which is a different type from SI float. Since there's no matching string, replay has to stop there.

Also the problem you ran into, failed string matching of command log row keycol values, is specific to command logs created in visidata v3.0, which is obsolete and due for sunsetting (EDIT to add: in my opinion). It will not occur on command logs created on Visidata 3.1 or later, because those versions have changed how they look for rows to apply to (since #2246). For such rows as you had, command logs now record the integer index of the row the command was applied to, instead of its string value. I strongly recommend using v3.3 instead of v3.0.2. Those two versions are over a year and a half apart, and a lot of problems were fixed in that time.

[midichef]

somehow [the replay file] relies on the "full" (original) path of the spreadsheet, so can't be replayed in the folder

The replay can't be replayed in the derivatives/ folder because the replay file open-file command specifies the relative path derivatives/ in line 2 of your .vdj file:

{"sheet": null, "col": null, "row": null, "longname": "open-file", "input": "derivatives/git-annex-info.jsonl", "keystrokes": "o", "comment": null}

If you remove derivatives/ from that line, you can replay the command file in any directory containing git-annex-info-jsonl.

[yarikoptic]

Thank you for all the details! claude adjusted original file into https://www.oneukrainian.com/tmp/git-annex-info-adj.vdj using indexes and that one worked for me! Overall I will try to not forget to use updated visidata on that older server. Feel free to close this issue unless you want to work it out further (e.g. disregarding the format character while looking up a column) or creating mapping from names to indexes for columns while replaying which would account for the knowledge of their mutation due to format changes)