An example with double channel string marker #4
Conversation
|
Same comment as in #5, can you add a description for the example file please? Either in a separate |
|
Also, I think |
|
Used |
|
@abcsds we have updated the README, can you please add a new section with a description of this dataset? Thanks! |
| Timestamp: 16.725987961266686 | ||
|
|
||
| These time-stamps precede the first clock offset measurement, but for | ||
| synchronization they will be handled with respect to the first (and | ||
| only in this case) detected clock segment. |
There was a problem hiding this comment.
How did you get this timestamp? When loading the file, I get the following output for that stream:
{'clock_times': [6.1],
'clock_values': [-0.1],
'footer': {'info': defaultdict(<class 'list'>,
{'first_timestamp': ['10'],
'last_timestamp': ['10.001'],
'sample_count': ['1']})},
'info': defaultdict(<class 'list'>,
{'channel_count': ['2'],
'channel_format': ['string'],
'created_at': ['10'],
'effective_srate': np.float64(999.9999999987779),
'name': ['SendMarker'],
'nominal_srate': ['1000'],
'segments': [(np.int64(0), np.int64(0))],
'stream_id': 3735928559,
'type': ['Marker']}),
'time_series': [['Marker 0A', 'Marker 0B']],
'time_stamps': array([16.9])}
The time stamp is 16.9, or no? This also means that the clock offset measurements precede the timestamp and not the other way round.
There was a problem hiding this comment.
Hi Clemens,
This PR, was suggested by you in cbrnr/XDF.jl#14, I loaded it with https://github.com/cbrnr/XDF.jl
Here's the output for twochannel_string_marker.xdf for every stream using XDF.jl:
Pair{Int64, Any}(3735928559, Dict{String, Any}(
"time" => [16.725987961266686],
"data" => ["Marker 0A" "Marker 0A"],
"nchannels" => 2,
"name" => "SendMarker",
"dtype" => String,
"srate" => 1000.0f0,
"clock" => [6.1],
"offset" => [-0.1],
"footer" => "<?xml version=\"1.0\"?><info><first_timestamp>10</first_timestamp><last_timestamp>10.001</last_timestamp><sample_count>1</sample_count></info>",
"header" => "<?xml version=\"1.0\"?><info><name>SendMarker</name><type>Marker</type><channel_count>2</channel_count><nominal_srate>1000</nominal_srate><channel_format>string</channel_format><created_at>10</created_at></info>",
"type" => "Marker")
)
Pair{Int64, Any}(46202862, Dict{String, Any}(
"time" => [16.725987961266686],
"data" => [0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0],
"nchannels" => 64,
"name" => "SendDataC",
"dtype" => Float64,
"srate" => 1000.0f0,
"clock" => [6.1],
"offset" => [-0.1],
"footer" => "<?xml version=\"1.0\"?><info><first_timestamp>10</first_timestamp><last_timestamp>10.001</last_timestamp><sample_count>1</sample_count></info>",
"header" => "<?xml version=\"1.0\"?><info><name>SendDataC</name><type>EEG</type><channel_count>64</channel_count><nominal_srate>1000</nominal_srate><channel_format>double64</channel_format><created_at>10</created_at></info>",
"type" => "EEG")
)
I get the same output as you from pyxdf.
Testing the outputs between the two libraries on minimal.xdf they seem to match:
(
[
{
'info': defaultdict(<class 'list'>, {
'name': ['SendDataC'],
'type': ['EEG'],
'channel_count': ['3'],
'nominal_srate': ['10'],
'channel_format': ['int16'],
'created_at': ['50942.723319709003'],
'desc': [None],
'uid': ['xdfwriter_11_int'],
'stream_id': 0,
'effective_srate': np.float64(10.000000000000025),
'segments': [(np.int64(0), np.int64(8))]
}),
'footer': {
'info': defaultdict(<class 'list'>, {
'writer': ['LabRecorder xdfwriter'],
'first_timestamp': ['5.1'],
'last_timestamp': ['5.9'],
'sample_count': ['9'],
'clock_offsets': [
defaultdict(<class 'list'>, {
'offset': [
defaultdict(<class 'list'>, {
'time': ['50979.76'],
'value': ['-.01']
}),
defaultdict(<class 'list'>, {
'time': ['50979.86'],
'value': ['-.02']
})
]
})
]
})
},
'time_series': array([[192, 255, 238],
[ 12, 22, 32],
[ 13, 23, 33],
[ 14, 24, 34],
[ 15, 25, 35],
[ 12, 22, 32],
[ 13, 23, 33],
[ 14, 24, 34],
[ 15, 25, 35]], dtype=int16),
'time_stamps': array([5. , 5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7, 5.8]),
'clock_times': [6.1, 7.1],
'clock_values': [-0.1, -0.1]
},
{
'info': defaultdict(<class 'list'>, {
'name': ['SendDataString'],
'type': ['StringMarker'],
'channel_count': ['1'],
'nominal_srate': ['10'],
'channel_format': ['string'],
'created_at': ['50942.723319709003'],
'desc': [None],
'uid': ['xdfwriter_11_str'],
'stream_id': 46202862,
'effective_srate': np.float64(10.000000000000016),
'segments': [(np.int64(0), np.int64(8))]
}),
'footer': {
'info': defaultdict(<class 'list'>, {
'writer': ['LabRecorder xdfwriter'],
'first_timestamp': ['5.1'],
'last_timestamp': ['5.9'],
'sample_count': ['9'],
'clock_offsets': [
defaultdict(<class 'list'>, {
'offset': [
defaultdict(<class 'list'>, {
'time': ['50979.76'],
'value': ['-.01']
}),
defaultdict(<class 'list'>, {
'time': ['50979.86'],
'value': ['-.02']
})
]
})
]
})
},
'time_series': [
[
'<?xml version="1.0"?><info><writer>LabRecorder
xdfwriter</writer><first_timestamp>5.1</first_timestamp><last_timestamp>5.9</last_timestamp>
<sample_count>9</sample_count><clock_offsets><offset><time>50979.76</time><value>-.01</value
></offset><offset><time>50979.86</time><value>-.02</value></offset></clock_offsets></info>'
],
['Hello'],
['World'],
['from'],
['LSL'],
['Hello'],
['World'],
['from'],
['LSL']
],
'time_stamps': array([5.1, 5.2, 5.3, 5.4, 5.5, 5.6, 5.7, 5.8, 5.9]),
'clock_times': [],
'clock_values': []
}
],
{'info': defaultdict(<class 'list'>, {'version': ['1.0']})}
)and Julia
Pair{Int64, Any}(0, Dict{String, Any}(
"time" => [5.0, 5.1000000000000005, 5.200000001490117, 5.300000002980233, 5.4, 5.5, 5.600000001490116, 5.700000002980232, 5.800000004470348],
"data" => Int16[192 255 238; 12 22 32; 13 23 33; 14 24 34; 15 25 35; 12 22 32; 13 23 33; 14 24 34; 15 25 35],
"nchannels" => 3,
"name" => "SendDataC",
"dtype" => Int16,
"srate" => 10.0f0,
"clock" => [6.1, 7.1],
"offset" => [-0.1, -0.1],
"footer" => "<?xml version=\"1.0\"?><info><writer>LabRecorder xdfwriter</writer><first_timestamp>5.1</first_timestamp><last_timestamp>5.9</last_timestamp><sample_count>9</sample_count><clock_offsets><offset><time>50979.76</time><value>-.01</value></offset><offset><time>50979.86</time><value>-.02</value></offset></clock_offsets></info>",
"header" => "<?xml version=\"1.0\"?><info><name>SendDataC</name><type>EEG</type><channel_count>3</channel_count><nominal_srate>10</nominal_srate><channel_format>int16</channel_format><created_at>50942.723319709003</created_at><desc/><uid>xdfwriter_11_int</uid></info>",
"type" => "EEG")
)
Pair{Int64, Any}(46202862, Dict{String, Any}(
"time" => [5.1, 5.2, 5.300000001490116, 5.400000002980232, 5.5, 5.6, 5.700000001490116, 5.800000002980232, 5.900000004470348],
"data" => ["<?xml version=\"1.0\"?><info><writer>LabRecorder xdfwriter</writer><first_timestamp>5.1</first_timestamp><last_timestamp>5.9</last_timestamp><sample_count>9</sample_count><clock_offsets><offset><time>50979.76</time><value>-.01</value></offset><offset><time>50979.86</time><value>-.02</value></offset></clock_offsets></info>"; "Hello"; "World"; "from"; "LSL"; "Hello"; "World"; "from"; "LSL";;],
"nchannels" => 1,
"name" => "SendDataString",
"dtype" => String,
"srate" => 10.0f0,
"clock" => Float64[],
"offset" => Float64[],
"footer" => "<?xml version=\"1.0\"?><info><writer>LabRecorder xdfwriter</writer><first_timestamp>5.1</first_timestamp><last_timestamp>5.9</last_timestamp><sample_count>9</sample_count><clock_offsets><offset><time>50979.76</time><value>-.01</value></offset><offset><time>50979.86</time><value>-.02</value></offset></clock_offsets></info>",
"header" => "<?xml version=\"1.0\"?><info><name>SendDataString</name><type>StringMarker</type><channel_count>1</channel_count><nominal_srate>10</nominal_srate><channel_format>string</channel_format><created_at>50942.723319709003</created_at><desc/><uid>xdfwriter_11_str</uid></info>",
"type" => "StringMarker")
)This doesn't look like a rounding error. Do you see where the difference is? I can't seem to find the source of it.
There was a problem hiding this comment.
This is strange. I don't have the time to dive into this right now, but maybe XDF.jl does clock correction by default?
There was a problem hiding this comment.
Hi Clemens, I can work on it on short periods, but continuously. If you check it out every once in a while and point me in the right direction is very helpful.
Both do clock correction by default (clock syncing):
- pyxdf https://github.com/xdf-modules/pyxdf/blob/main/src/pyxdf/pyxdf.py#L77
- xdf.jl https://github.com/cbrnr/XDF.jl/blob/d3bb58bd11e1699471e747974125e40e1143a5fc/src/XDF.jl#L34C45-L34C60
However, pyxdf also does de-jittering by default. https://github.com/xdf-modules/pyxdf/blob/main/src/pyxdf/pyxdf.py#L79. XDF.jl still has it in pr: cbrnr/XDF.jl#13, so we should consider turn it it on as default on that branch. I think I left it off by default.
Still, when loading pyxdf.load_xdf("twochannel_string_marker.xdf", dejitter_timestamps=False) I get the same array:
...
'name': ['SendMarker'],
'type': ['Marker'],
'channel_count': ['2'],
'nominal_srate': ['1000'],
'channel_format': ['string'],
'created_at': ['10'],
'stream_id': 3735928559,
'effective_srate': 0.0,
'segments': [(0,0)]
...
'footer': {
'info': defaultdict(<class 'list'>, {
'first_timestamp': ['10'],
'last_timestamp': ['10.001'],
'sample_count': ['1']
})
},
'time_series': [['Marker 0A', 'Marker 0B']],
'time_stamps': array([16.9]),
'clock_times': [6.1],
'clock_values': [-0.1]
...I think it's unlikely to be the de-jittering. It doesn't really do anything to the time stamp, (at least for this file at the floating point precision we use to compare the two libraries). I guess next I'll go debugging pyxdf, to see what it actually does. Unless you have another idea of what it could be.
Thanks!
|
Looks good! Now we should try to figure out the time stamp issue (i.e., why XDF.jl reports a different time stamp than pyxdf)! |
Multiple channels with string markers seem to have been an issue in both liblxdf and xdf.jl. This file was uploaded by @tstenner in xdf-modules/libxdf#19
It presents a single data point at time [16.725987961266686] with markers ["Marker 0A" "Marker 0B"].
The header of the file is:
The footer of the file is: