Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Discussion] Do we need emit_json ? #542

Open
g41797 opened this issue Nov 16, 2024 · 8 comments
Open

[Discussion] Do we need emit_json ? #542

g41797 opened this issue Nov 16, 2024 · 8 comments

Comments

@g41797
Copy link

g41797 commented Nov 16, 2024

lcmgen consists of two parts:

  • parser - parses *lcm files and creates internal "db"
  • emitter - reads "db" and creats cources for language in question

This makes development for other languages somewhat inconvenient (at least for me):

  • using C for emitter
  • forking repo
  • or adding separated projects for the new language

Protocol buffers compiler solves this problem via compiler plugins:

A plugin is just a program which reads a CodeGeneratorRequest protocol buffer from standard input and then writes a CodeGeneratorResponse protocol buffer to standard output. These message types are defined in plugin.proto. We recommend that all third-party code generators be written as plugins, as this allows all generators to provide a consistent interface and share a single parser implementation.

I am not calling for changing the implementation

Just to think about saving results of the parsing in JSON file.

Generator for new language reads JSON and generates code

Credit to jbendes

@nosracd
Copy link
Contributor

nosracd commented Nov 17, 2024

I agree that the current implementation is not very friendly to people who want to contribute and appreciate your ideas for improving it!

To make sure I'm understanding your ideas correctly, are you proposing that we add an emit_json so that lcm-gen can output JSON files in order for a follow-on generator to consume the JSON and output new languages? Also, sorry if I'm missing some of the context; that discord link doesn't appear to work for me even after logging in.

One other path forward we could go to make adding support for new languages a bit more developer-friendly: if we added some bindings for portions of lcm-gen we could allow for new emitters to be written in Python.

@g41797
Copy link
Author

g41797 commented Nov 17, 2024

Or on Zig - it's my case

@jbendes
Copy link
Contributor

jbendes commented Nov 18, 2024

Is the goal here to not contribute to the repo? I'm lost as to why a 2 step process is better than a 1 step process for generating types. The only benefit I'm seeing is that you wouldn't have to contribute new emitter code back to the repo. Is that the goal? I feel like there's another goal that I'm not understanding

@g41797
Copy link
Author

g41797 commented Nov 19, 2024

Any color the customer wants, as long as it’s black.

Henry Ford (with regard to the color of the Model T's)

Not everything should be implemented in C.

My intent was to port *cm to Zig:

  • gen - Zig
  • communication - Zig only (not wrapper and stdclib)

Later the same approach can be applied to go and c#

@jbendes
Copy link
Contributor

jbendes commented Nov 19, 2024

Okay so your goal was to be able to write emitters in a different language. Now I understand a lot more. That was never stated prior to this. So you're saying that you would like there to be an intermediate format that is "easier" to parse than the {z,l}cmtype itself? I'll move back over to discord for zcm conversation so I don't take over an lcm issue haha

@jbendes
Copy link
Contributor

jbendes commented Nov 19, 2024

Just to close the loop, I believe this is already implemented in zcm by using zcm-gen --summary <path/to/type>

@g41797
Copy link
Author

g41797 commented Nov 19, 2024

/* ZCM */
    if (gopt.getBool("summary")) {
        did_something = 1;
        zcm.dump(); JSON!!!
    }

/* LCM */
    if (getopt_get_bool(gopt, "debug")) {
        did_something = 1;
        lcmgen_dump(lcm); TEXT!!!
    }

now it's lcm turn

btw emit_json looks more symmetric/preferable solution

@g41797
Copy link
Author

g41797 commented Nov 19, 2024

example_1.zsm:

struct example_t
{
    int64_t  timestamp;
    double   position[3];
    double   orientation[4];
    int32_t  num_ranges;
    int16_t  ranges[num_ranges];
    string   name;
    boolean  enabled;

    int32_t nExamples1;
    int32_t nExamples2;
    example_t subExamples[nExamples1][nExamples2];
    string subStrings[nExamples1][nExamples2];

    float complexNestedArray[5][nExamples1][3][nExamples2][4];

    int8_t:2 bitfield1;
    int8_t:3 bitfield2[4];

    const int8_t  test_const_8_max_hex  = 0xff;
    const int16_t test_const_16_max_hex = 0xffff;
    const int32_t test_const_32_max_hex = 0xffffffff;
    const int64_t test_const_64_max_hex = 0xffffffffffffffff;

    const float  test_const_float  = 1e-20;
    const double test_const_double = 12.1e200;

    const string test_const_string = "example";
}

JSON dump:

{
	"example_t" : 
	{
		"comment" : "",
		"constants" : 
		[
			{
				"comment" : "",
				"membername" : "test_const_8_max_hex",
				"type" : "int8_t",
				"val" : 
				{
					"as_json" : -1,
					"as_string" : "0xff"
				}
			},
			{
				"comment" : "",
				"membername" : "test_const_16_max_hex",
				"type" : "int16_t",
				"val" : 
				{
					"as_json" : -1,
					"as_string" : "0xffff"
				}
			},
			{
				"comment" : "",
				"membername" : "test_const_32_max_hex",
				"type" : "int32_t",
				"val" : 
				{
					"as_json" : -1,
					"as_string" : "0xffffffff"
				}
			},
			{
				"comment" : "",
				"membername" : "test_const_64_max_hex",
				"type" : "int64_t",
				"val" : 
				{
					"as_json" : -1,
					"as_string" : "0xffffffffffffffff"
				}
			},
			{
				"comment" : "",
				"membername" : "test_const_float",
				"type" : "float",
				"val" : 
				{
					"as_json" : 9.9999996826552254e-21,
					"as_string" : "1e-20"
				}
			},
			{
				"comment" : "",
				"membername" : "test_const_double",
				"type" : "double",
				"val" : 
				{
					"as_json" : 1.21e+201,
					"as_string" : "12.1e200"
				}
			},
			{
				"comment" : "",
				"membername" : "test_const_string",
				"type" : "string",
				"val" : 
				{
					"as_json" : "\"example\"",
					"as_string" : "\"example\""
				}
			}
		],
		"hash" : 18359083813438051081,
		"members" : 
		[
			{
				"comment" : "",
				"membername" : "timestamp",
				"typename" : 
				{
					"fullname" : "int64_t"
				}
			},
			{
				"comment" : "",
				"dims" : 
				[
					{
						"size" : "3",
						"type" : "const"
					}
				],
				"membername" : "position",
				"typename" : 
				{
					"fullname" : "double"
				}
			},
			{
				"comment" : "",
				"dims" : 
				[
					{
						"size" : "4",
						"type" : "const"
					}
				],
				"membername" : "orientation",
				"typename" : 
				{
					"fullname" : "double"
				}
			},
			{
				"comment" : "",
				"membername" : "num_ranges",
				"typename" : 
				{
					"fullname" : "int32_t"
				}
			},
			{
				"comment" : "",
				"dims" : 
				[
					{
						"size" : "num_ranges",
						"type" : "var"
					}
				],
				"membername" : "ranges",
				"typename" : 
				{
					"fullname" : "int16_t"
				}
			},
			{
				"comment" : "",
				"membername" : "name",
				"typename" : 
				{
					"fullname" : "string"
				}
			},
			{
				"comment" : "",
				"membername" : "enabled",
				"typename" : 
				{
					"fullname" : "boolean"
				}
			},
			{
				"comment" : "",
				"membername" : "nExamples1",
				"typename" : 
				{
					"fullname" : "int32_t"
				}
			},
			{
				"comment" : "",
				"membername" : "nExamples2",
				"typename" : 
				{
					"fullname" : "int32_t"
				}
			},
			{
				"comment" : "",
				"dims" : 
				[
					{
						"size" : "nExamples1",
						"type" : "var"
					},
					{
						"size" : "nExamples2",
						"type" : "var"
					}
				],
				"membername" : "subExamples",
				"typename" : 
				{
					"fullname" : "example_t"
				}
			},
			{
				"comment" : "",
				"dims" : 
				[
					{
						"size" : "nExamples1",
						"type" : "var"
					},
					{
						"size" : "nExamples2",
						"type" : "var"
					}
				],
				"membername" : "subStrings",
				"typename" : 
				{
					"fullname" : "string"
				}
			},
			{
				"comment" : "",
				"dims" : 
				[
					{
						"size" : "5",
						"type" : "const"
					},
					{
						"size" : "nExamples1",
						"type" : "var"
					},
					{
						"size" : "3",
						"type" : "const"
					},
					{
						"size" : "nExamples2",
						"type" : "var"
					},
					{
						"size" : "4",
						"type" : "const"
					}
				],
				"membername" : "complexNestedArray",
				"typename" : 
				{
					"fullname" : "float"
				}
			},
			{
				"comment" : "",
				"membername" : "bitfield1",
				"typename" : 
				{
					"fullname" : "int8_t",
					"numbits" : 2,
					"sign_extend" : false
				}
			},
			{
				"comment" : "",
				"dims" : 
				[
					{
						"size" : "4",
						"type" : "const"
					}
				],
				"membername" : "bitfield2",
				"typename" : 
				{
					"fullname" : "int8_t",
					"numbits" : 3,
					"sign_extend" : false
				}
			}
		],
		"structname" : 
		{
			"fullname" : "example_t"
		}
	}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants