Set multiple metakeys at once via kdb

[kodebach]

It can often be useful to set multiple metakeys (even if the metadata may not be stored as such) together in one atomic operation.

In most cases I know of, there is an order in which you can set the metakeys to make things work. But theoretically there could be plugin, where you need to set metakeys together and there should be an error, if not all required metakeys are set.

For example, take this simplified snippet from the WIP update to toml:

kdb set 'user:/tests/storage/key' '1'

kdb meta-set 'user:/tests/storage/key' 'comment/#0/inline' '1'
kdb meta-set 'user:/tests/storage/key' 'comment/#0/start' '    # '
kdb meta-set 'user:/tests/storage/key' 'comment/#0' 'This value is very interesting'

cat `kdb file user:/tests/storage`
#> key = 1    # This value is very interesting

You need to set the meta-keys in this specific order to the the expected formatting. This is because of the way the toml plugin fills in the other required parameters and because of the way comments are read. Yes, this could be fixed in toml, but there are other examples where this is not possible. Take for example the type plugin with an enum key:

kdb set user:/tests/type/value middle
kdb meta-set user:/tests/type/value check/enum '#2'
kdb meta-set user:/tests/type/value 'check/enum/#0' 'low'
kdb meta-set user:/tests/type/value 'check/enum/#1' 'middle'
kdb meta-set user:/tests/type/value 'check/enum/#2' 'high'
kdb meta-set user:/tests/type/value type enum

This will not work in any order. For example, you obviously setting type = enum before adding middle to the enum values will fail. This is not something that can be solved by changing the type plugin, because this is the intended behaviour.

To solve such cases, it would be nice, if we could set the value of a key and its metadata in one atomic command. This is easy to implement, since kdbSet() is atomic anyway and you can change as much as you want between kdbGet() and kdbSet().

I therefore propose this extension to kdb meta-set:

# order of metakeys doesn't matter everything is just one kdbSet()
kdb meta-set user:/tests/type/value type enum check/enum '#2' 'check/enum/#0' 'low' 'check/enum/#1' 'middle' 'check/enum/#2' 'high'

# same command but more readable
kdb meta-set user:/tests/type/value \
    type enum \
    check/enum '#2' \
    'check/enum/#0' 'low' \
    'check/enum/#1' 'middle' \
    'check/enum/#2' 'high'

I think the following would be more readable, but we can only implement it, if we accept that the kdb tool cannot set metakeys that contain a = or implement yet another escaping mechanism. IMO it would be fine, since kdb mount already uses = for plugin config, so kdb already doesn't universally except every possible key name.

kdb meta-set user:/tests/type/value type=enum check/enum='#2' 'check/enum/#0'='low' 'check/enum/#1'='middle' 'check/enum/#2'='high'

There may also be a use-case for setting not just multiple metakeys in one go, but also set the key value at the same time. For this we could add an extension to kdb set. However, I don't know of a use case right now, so I would consider this low priority.

The syntax for kdb set could be (assuming = is okay):

kdb set user:/tests/type/value middle type=enum check/enum='#2' 'check/enum/#0'='low' 'check/enum/#1'='middle' 'check/enum/#2'='high'

---

@markus2330 What do you think?

[markus2330]

I agree that something like this is needed (not only for metadata but also for some storage plugins which need several keys at once) but I don't see big urgency (i.e. the issue is post-1.0) as we have kdb import and kdb shell which already support setting arbitrary KeySets atomically.

Btw. the now-gone tcl plugin supported a very similar syntax you proposed here { metakey=metavalue * } but the plugin was removed. So probably if you write a plugin supporting your favorite syntax, kdb import will already do what you want.

[kodebach]

we have kdb import and kdb shell which already

kdb import always replaces a whole section of the KDB (AFAIK), so you can't for example set /foo/bar and /foo/baz, but leave other keys below /foo untouched.

kdb shell works, but it is pretty complicated to use, if you just want to set a few keys.

Both have the problem, that you cannot easily set a keys metadata without touching the value. You would need to know the current value and specify that again. For kdb shell this could be solved by adding a ksLookup command.

I don't see big urgency (i.e. the issue is post-1.0)

I don't see urgency either. It is not a must-have for 1.0, but it is a bit annoying in the meantime and some READMEs could be written in a more straightforward way and without disclaimers like "You must call the commands in this exact order".

I would create a low priority issue to remember that this should be done. It shouldn't be too much work, so maybe someone will implement it when there is some free time (e.g. because more important issues are still in discussion, blocked, need review, etc.)

[markus2330]

I don't think we need any further low-prior issues, there are dozens of them 😉

If it really disturbs too much, simply do it. (But please with the proper separation in extra arguments, i.e., without =)

[kodebach]

But please with the proper separation in extra arguments, i.e., without =

That is actually one reason, why I wanted to have this discussion. I find the version without the = very hard to read, unless it is formatted with the \-escaped newlines.

[mpranj]

Looks like a very nice enhancement to me as well. I would, however, convert it to an issue so it does not get lost here.

[markus2330]

kdb import always replaces a whole section of the KDB (AFAIK),

No, kdb import has several strategies, see man elektra-merge-strategies. Only the strategy cut does what you describe here.

As the 3-way merge was never fully completed+integrated, some of the strategies are buggy, though... I hope @dominicjaeger finds some time to do this soon.

For kdb shell this could be solved by adding a ksLookup command.

Good idea.

I find the version without the = very hard to read, unless it is formatted with the -escaped newlines.

I agree, the newlines are a pretty way to format for shell recorder. Imho this formatting makes the = unnecessary.

Looks like a very nice enhancement to me as well. I would, however, convert it to an issue so it does not get lost here.

I would prefer if anything that might lead to code changes at some point gets created as an issue immediately. The conversion to an issue is a bit of a joke: it simply copies the top-post to a new issue.

[kodebach]

No, kdb import has several strategies, see man elektra-merge-strategies. Only the strategy cut does what you describe here.

Ah yes, I misread the code. However, the problem of keeping the value, but changing metadata remains. At least I don't see how the merge strategy could differentiate between: "Keep value & change metadata" and "Set value to NULL/empty & change metadata"

Imho this formatting makes the = unnecessary.

The problem arises, if the code is not formatted this way. Even kdb set user:/tests/type/value middle type enum is a bit hard to read.

The conversion to an issue is a bit of a joke: it simply copies the top-post to a new issue.

It's still in beta, so maybe that will improve. But I think the idea was to create an issue manually anyway.