Bug: HuggingFace Dataset Automatically Expand Dictionary Keys

[richardzhuang0412]

As I'm creating demos using curator for fine-tuning function-calling models, I found that Huggingface Dataset seems to automatically process the dictionary entries. Specifically, it will impute keys with None value so that all entries in the column has the same keys I guess for compatibility.

Function-Calling Example:

from datasets import Dataset

ls = [{'name': 'land_drone', 'arguments': {'location': 'current'}},
 {'name': 'land_drone', 'arguments': {'location': 'home_base'}},
 {'name': 'land_drone', 'arguments': {'location': 'custom'}},
 {'name': 'control_camera', 'arguments': {'mode': 'photo'}},
 {'name': 'control_camera', 'arguments': {'mode': 'video'}},
 {'name': 'control_camera', 'arguments': {'mode': 'panorama'}},
 {'name': 'set_drone_lighting', 'arguments': {'mode': 'on'}},
 {'name': 'set_drone_lighting', 'arguments': {'mode': 'off'}},
 {'name': 'set_drone_lighting', 'arguments': {'mode': 'blink'}},
 {'name': 'set_drone_lighting', 'arguments': {'mode': 'sos'}},
 {'name': 'return_to_home', 'arguments': {}},
 {'name': 'set_battery_saver_mode', 'arguments': {'status': 'on'}},
 {'name': 'set_battery_saver_mode', 'arguments': {'status': 'off'}},
 {'name': 'set_obstacle_avoidance', 'arguments': {'mode': 'on'}},
 {'name': 'set_obstacle_avoidance', 'arguments': {'mode': 'off'}},
 {'name': 'set_follow_me_mode', 'arguments': {'status': 'on'}},
 {'name': 'set_follow_me_mode', 'arguments': {'status': 'off'}},
 {'name': 'calibrate_sensors', 'arguments': {}},
 {'name': 'set_autopilot', 'arguments': {'status': 'on'}},
 {'name': 'set_autopilot', 'arguments': {'status': 'off'}},
 {'name': 'configure_led_display', 'arguments': {'pattern': 'solid'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'solid', 'color': 'red'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'solid', 'color': 'blue'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'solid', 'color': 'green'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'solid', 'color': 'yellow'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'solid', 'color': 'white'}},
 {'name': 'configure_led_display', 'arguments': {'pattern': 'blink'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'blink', 'color': 'red'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'blink', 'color': 'blue'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'blink', 'color': 'green'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'blink', 'color': 'yellow'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'blink', 'color': 'white'}},
 {'name': 'configure_led_display', 'arguments': {'pattern': 'pulse'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'pulse', 'color': 'red'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'pulse', 'color': 'blue'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'pulse', 'color': 'green'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'pulse', 'color': 'yellow'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'pulse', 'color': 'white'}},
 {'name': 'configure_led_display', 'arguments': {'pattern': 'rainbow'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'rainbow', 'color': 'red'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'rainbow', 'color': 'blue'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'rainbow', 'color': 'green'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'rainbow', 'color': 'yellow'}},
 {'name': 'configure_led_display',
  'arguments': {'pattern': 'rainbow', 'color': 'white'}},
 {'name': 'reject_request', 'arguments': {}}
 ]

ls_str = [str(item) for item in ls]

ds = Dataset.from_dict({"function_call": ls})
ds_str = Dataset.from_dict({"function_call": ls_str})

print(ds[0])
print(ds_str[0])

Output:

{'function_call': {'arguments': {'color': None, 'location': 'current', 'mode': None, 'pattern': None, 'status': None}, 'name': 'land_drone'}}
{'function_call': "{'name': 'land_drone', 'arguments': {'location': 'current'}}"}

Since curator deals with Dataset objects throughout, for function-calling settings this could be a potential issue as the argument key can be expanded with arguments from all other unrelated functions and mess up fine-tuning.