关于训练数据格式的一些问题

[moosich7]

感谢您提供的优秀作品，这是一个先进的模型。
我在使用的过程中遇到一些不太理解的问题，可能比较基础，希望能得到回复!

Q1：首先是您提供的sa1b_coco_fmt_iminfo_500k.json文件，我在train.md中看到您使用代码：
python tools/format_conversion/convert_sa1b_to_coco.py --img_list data/sam/sam_annotations/jsons/sa1b_coco_fmt_iminfo_500k.json --input_directory data/sam/batch0 --output_folder data/sam/sam_annotations/jsons/
将该文件转换为COCO格式的json，我的疑惑点是你提供的sa1b_coco_fmt_iminfo_500k.json文件中"images"字段包含了所有图片的信息如：
{"image_id": 10904990, "width": 2250, "height": 1500, "file_name": "sa_10904990.jpg", "id": 10904990}，
而该文件的"annotations"字段是不包含图片分割的后的信息如：
{"segmentation":{"size":[],"counts:"“}}
这种COCO RLE形式的掩膜，所以"annotations"字段的信息需要自己使用SAM分割图片后得到的COCO RLE放入"annotations"字段中吗，还是不需要补充"annotations"字段的信息

Q2：用SAM分割后的得到"counts"字段是一大串毫无规律的字符串这是正常的吗，如下是其中一个分割：

{"segmentation": {"size": [1012, 2688], "counts": "[Udd1o08WOR1K]k0e1[TOcNXO0bk0e1QUO_NWO3ak0n1iTOVOcj0Q1XUOYO]j0k0UOASj0d3[O>C=B7_WOWIlg0W8QO:PYO^G[f0U9Gg0XOf0YO?Bi0XOW1hN_1aNd0]O:R@@W?V0h@n_O5Kg42h3_a0[Gh^O1O1O10[1j0]NUOV8oc0RHm[O4R2e0lMk5ld0TI]O6lMM46E6Od02S3_g0kKRZO8cN2NM3l3li0QLjVOn3j0N1O1O2QLcTOc3hk0O1O1N2O2N1O1N101O1N101O0O10001O01O0000000000000000O010O010O000O2L3M4M31O0O2O1O1O1O0O2O001O1O2M2O2N1O1O1O1O001O00001O00000000001O00101N9G2N1O2N2N4L2N1O1O1O1O1O1O1O1O1O1N2O001O1O001O1O1O2N1O3M3M2N2N1O1O1O1O1O1O001N2O1O1O1O1O1O2N1O1O1O1@WJSWOj5kh0WJUWOj5jh0WJUWOj5jh0VJUWOk5lh0SJUWOn5kh0QJUWOo5lh0PJSWOQ6Xi01O0O1O2O0O1O1N3M2N3N100O2O0O101O01O00010O01O00010O001O001O0O2O001O00001O00000000001O00000000000O100O101O0O1000000O1001O01O0001O01O001O010O1O1O001O001O00001O000000001O000001O01N1000000O1000O0100O1000001O000000001O000000000000O01000000O01000O1000000000000000000000O1000000O1000000O1000000000000O1000000O1O1O1O001O1O1O010OEnHVXOQ7ig0RIVXOn6jg0RIVXOn6jg0SITXOm6ng0QISXOo6ng0oHSXOQ7Xh0O1000000O100O100O1O2N100O1000CXInWOg6Sh0YImWOg6Sh0YImWOg6Sh0YImWOg6h0O10O100O010O1O10O0100O001O1N101N2O01000000O2O1O1N2O001O00000000O10000O10000O10000001O0O2O00001O000000000000O100O100O1O100O10000000O10000000O1000O010O1O1O1O010O1O1000000O1000000O100000000O10O10000000O01@\IPXOd6og0]IQXOc6og0^IPXOb6og0_IQXOa6Ph0^IPXOb6Ph0]IQXOc6Ph0\IoWOf6_h00000000000000000O1000BYIoWOg6og0\IPXOd6Ph0]IoWOb6Qh0_IoWOa6Qh0_IoWOa6Qh0_InWOb6Rh0^InWOb6Sh0]ImWOc6Sh0]ImWOb6Th0^IlWOb6Uh0\IlWOd6Vh0ZIiWOg6bh00O1000O010O1O1NHIWO\6ch0fI\WOW6gh0jIXWOT6ih0nIVWOP6lh0QJSWOl5Pi0:2M3O1O1O1O101N1O101N1O1O1O1O0O2N2]OaVOoJai0P5cVOlJ^i0S5b0O100O11O1O001O100O1O2NO100N2N2L3N3M3M3M3K5M2O2O100000000000O10000000001O0000001O00000000001O0000001O00001N101O0000000O10000000000O2O000000001O00001O000000O10O100000O1000000001O0000001O00001bVORKTh0n4lWORKSh0o4mWOQKSh0o4Y1000000O10O100000000O2O001N101O2M2O1gVOhJSh0Z5lWOfJTh0[5kWOeJUh0\5jWOdJUh0^5P11O1N2O000O2O00001O001O001O1O1kVOVJ^h0k5aWOVJ^h0k5WOVJh0j5WOWJ_h0i5aWOWJ_h0i5aWOVJh0j5aWOUJ^h0l5bWOTJ^h0k5f00O100O010O2O0000001O001O0000000O2O000O01000O100000000O101O001O00001O00001N101O001O001O001N2O1O001O1O001N2O001O1O0O2O1O1O1O1O1N2O1O1O2N1O2N1N2O0O2N2M3N2N1O3M2N4M3L5K8I8G7J6I7J5K6I5L3L5L6J<C7J5K3L3N1O2M4M3L;E9F=B<Ac0TOj0\OU1QO:G9G<\O0C<Eb0]O;G8SCd]OV<bc0XO<F7Hi0TOh0dNbZO[Gne0S8Q2lMPXOSJh0W4ZWO^LPk0[1QUOoMnl06]SONjm^6"}, "area": 610773, "bbox": [1704, 58, 774, 934], "predicted_iou": 1.0659044981002808, "point_coords": [[1974.0, 237.1875]], "stability_score": 0.9624068737030029, "crop_box": [0, 0, 2688, 1012]}

[siyuanliii]

Thanks for your questions. Q1: sa1b_coco_fmt_iminfo_500k.json only includes the file name lists, not the actual annotations. You can either Run SAM-H to generate the annotations by yourself or you can directly use the annotations provided by SA-1B dataset. Q2: it is expected. It is just RLE encoding.

[moosich7]

Thanks for your reply, It's useful for me. However, I encounter another error, when I run follow codes and meet UnicodeDecodeError:

(masaenv) C:\pythonlearning\masa>python tools/format_conversion/convert_sam_2_cocofmt.py --img_list data/1/pig.json
 --input_directory data/1/images --output_folder data/1/jsons
  0%|                                                                                       | 0/1 [00:00<?, ?it/s]I
nitializing process 18956
Initializing process 10912
Initializing process 9864
Initializing process 18928
  0%|                                                                                       | 0/1 [00:00<?, ?it/s]
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "C:\Users\hhhhh\anaconda3\envs\masaenv\lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "C:\pythonlearning\masa\tools\format_conversion\convert_sam_2_cocofmt.py", line 19, in process_file
    json_data = json.load(f)
  File "C:\Users\hhhhh\anaconda3\envs\masaenv\lib\json\__init__.py", line 293, in load
    return loads(fp.read(),
UnicodeDecodeError: 'gbk' codec can't decode byte 0x80 in position 19: illegal multibyte sequence
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\pythonlearning\masa\tools\format_conversion\convert_sam_2_cocofmt.py", line 84, in <module>
    results = list(tqdm(pool.imap(process_file, files), total=len(files)))
  File "C:\Users\hhhhh\anaconda3\envs\masaenv\lib\site-packages\tqdm\std.py", line 1178, in __iter__
    for obj in iterable:
  File "C:\Users\hhhhh\anaconda3\envs\masaenv\lib\multiprocessing\pool.py", line 870, in next
    raise value
UnicodeDecodeError: 'gbk' codec can't decode byte 0x80 in position 19: illegal multibyte sequence

I try to modify file convert_sam_2_cocofmt.py line 18 with open(file, 'r') as f: to with open(file, 'r',encoding="UTF-8") as f:, But I also not can solve problem, it cause new error:

(masaenv) C:\pythonlearning\masa>python tools/format_conversion/convert_sam_2_cocofmt.py --img_list data/1/pig.json
 --input_directory data/1/images --output_folder data/1/jsons
  0%|                                                                                       | 0/1 [00:00<?, ?it/s]I
nitializing process 5948
Initializing process 18140
Initializing process 156
  0%|                                                                                       | 0/1 [00:00<?, ?it/s] 
Initializing process 9292
Initializing process 20188
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "C:\Users\hhhhh\anaconda3\envs\masaenv\lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "C:\pythonlearning\masa\tools\format_conversion\convert_sam_2_cocofmt.py", line 19, in process_file
    json_data = json.load(f)
  File "C:\Users\hhhhh\anaconda3\envs\masaenv\lib\json\__init__.py", line 293, in load
    return loads(fp.read(),
  File "C:\Users\hhhhh\anaconda3\envs\masaenv\lib\codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\pythonlearning\masa\tools\format_conversion\convert_sam_2_cocofmt.py", line 84, in <module>
    results = list(tqdm(pool.imap(process_file, files), total=len(files)))
  File "C:\Users\hhhhh\anaconda3\envs\masaenv\lib\site-packages\tqdm\std.py", line 1178, in __iter__
    for obj in iterable:
  File "C:\Users\hhhhh\anaconda3\envs\masaenv\lib\multiprocessing\pool.py", line 870, in next
    raise value
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte

The pig.json like this:(only include 1 image)
{"images":[{"width": 2688, "height": 1012, "file_name": "101001.png", "id": 101001}]}, "annotations":[{"segmentation":{"size":[1012,2688],"image_id":101001,"counts":"...(a long string)","area": 610773, "bbox": [1704, 58, 774, 934], "predicted_iou": 1.0659044981002808, "point_coords": [[1974.0, 237.1875]], "stability_score": 0.9624068737030029, "crop_box": [0, 0, 2688, 1012]},{},{},{},...}],"categories":[{"id":1,"name":"object"}]}

The file struct as: