Skip to content

decoder: long cannot be really used as dict key #55

Description

@navytux

Currently we decode longs to *big.Int and it kind-of works. Also despite == not giving equality for two *big.Int numbers, reflect.DeepEqual works for two such numbers since the numbers are internally always stored in the same form. However not working == poses a problem:

  • *big.Int cannot be used as map key for lookup: the lookup will use == and lookup will succed only if accessing something with pointer to exactly the same *big.Int object that was used at map creation.
  • reflect.DeepEqual for maps with *big.Int key(s) wont work - for maps reflect.DeepEqual compares deeply only values and keys are compared by regular Go ==.

All this in turn leads to that some pickles with long keys are decoded wrongly, e.g.:

# python
In [9]: q = "}(L1L\nS'aaa'\nL1L\nS'bbb'\nu."

In [10]: dis(q)
    0: }    EMPTY_DICT
    1: (    MARK
    2: L        LONG       1L
    6: S        STRING     'aaa'
   13: L        LONG       1L
   17: S        STRING     'bbb'
   24: u        SETITEMS   (MARK at 1)
   25: .    STOP
highest protocol among opcodes = 1

In [11]: pickle.loads(q)
Out[11]: {1L: 'bbb'}

but ogórek gives:

map[interface {}]interface {}{1:"aaa", 1:"bbb"}

Probably nothing we can do here until in Go2 they make builtin int to be really a bigint (golang/go#19623), and then, similarly to Python3, we decode both INT and LONG into Go int, and on encoding Go int, we look into its range and if it is in int32 range -> go INT/BININT/..., and otherwise go via LONG*.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions