merge deferred mods with existing deferred add

[justin2004]

i think we have a little bug wrt deferred modifications.

demonstration

put this in the CsvSheet class (just for this demo)

defer = True

currently we have this behavior:

vd sample_data/benchmark.csv
add-row
edit the customer column

then in the vd repl

vd.sheet.getDeferredChanges()
({140131282503616: [None, None, None, None, None, None, None]}, {}, {})

note that there is a deferred addition but has no column values (all None) even though we modified 1 cell.
that is, the deferred modifications are lost if applied to a new row.

with this PR we have this behavior:

vd sample_data/benchmark.csv
add-row

then confirm it is an empty row at the vd repl
vd.sheet.getDeferredChanges()
({139923036424208: [None, None, None, None, None, None, None]}, {}, {})

note empty row -- as expected

now edit the name column and another column if you like

and at the vd repl
vd.sheet.getDeferredChanges()
({139923036424208: [None, 'fred', None, 'chickpea', None, None, None]}, {}, {})

note that the deferred addition now includes all modifications -- so we effectively rolled the modifications into the addition (since it was a deferred addition anyway)

[justin2004]

just to note the reason...

i am extending the Imap loader to support sending emails (with smtp) and using deferred changes is useful for that:

[saulpw]

Hm, I thought this is how it already worked. When I view a sqlite database with --debug (which shows the SQL statements executed), do a similar set of add/edit commands and then commit, it only shows the one INSERT, and no UPDATEs. Are there different code paths on this for some reason?

[justin2004]

hey @saulpw , i think i see what is happening.

since the deferred mods don't get added to the deferred adds (in the develop branch -- like my example above shows)
the sqlite loader, upon commit, when it has a deferred add has to reach back into the sheet to get the values itself using:
col.getTypedValue(row) or col.getDisplayValue(r)

notice how the parameters to the execute sql function are built by calling 2 helper functions that reach back into the sheet (instead of using the values in the deferred adds (because they get discarded if done after the initial add-row command)).

i think i can simplify the sqlite loader on top of this PR. let me see...

[saulpw]

Oh, that would be great if you can simplify the sqlite with this!