How to trim trailing zeros for float/double type when dumping?

[YarNix]

Hello, I'm trying find a way to trim excess zero when dumping to json, but I couldn't find any discussion about it.

nlohmann::json j = {
    {"time", 1.0},
    {"duration", 2.0f},
};
// Current: {"duration":2.0,"time":1.0}
// Desired: {"duration":2,"time":1}
j.dump(-1);

My main motivation for this is just reducing file size.
I'm writing a level editor, the export format is json, and I noticed that the version without trailing zero is always 40%-66% smaller

[nlohmann]

This is not possible, because we want to keep sufficient precision to roundtrip the numbers. If you want to save bandwidth, consider using a binary format like CBOR.

[gregmarr]

Are you sure that these are floats and not integers in the format? If they're integers, you can use integers in the json, and it will be written as an integer in the text.

[YarNix]

I can assure you that float is the right type. I guess one solution you can do is to add a conversion to string at every location where the float value is assigned

[gregmarr]

Oh well, it was worth a shot. I don't think converting to string would be good, as then they would be "1" in the output, which is not only the wrong type, but it exactly the same number of characters as 1.0.

[gregmarr]

the version without trailing zero is always 40%-66% smaller

I think if you're seeing the upper range of that, then you probably have other differences. I don't think it's possible to get a 66% reduction by eliminating the trailing zeros for anything larger than a file with just a single integer between 0 and 9 and no trailing newline. Then you would get the 3 bytes with the trailing zero and 1 byte without it, for a 66.6% reduction. Anything beyond that, and all the extra brackets, commas, colons, quotes, and names are going to reduce that percentage.

The difference between {"duration":2,"time":1} and {"duration":2.0,"time":1.0} is 23 bytes vs 27 bytes. The reduced file is 14.8% smaller. The theoretical maximum reduction for an object with two values happens where every label is one character. That is {"d":2,"t":1} vs {"d":2.0,"t":1.0} for 13 vs 17 bytes, which is 23.5% smaller. But either way it's still only 4 bytes.

If you're talking about large arrays of numbers between 0 and 9 instead of objects, then you can get a much larger percentage reduction. [0,1,2,3,4,5,6,7,8,9] vs [0.0,1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0] is 21 vs 41, so the version without the zeros is 48.8% smaller, but that is still only 20 bytes, as comparing small changes in small numbers still gives you large percentage changes.

If you have a single array of 1000 numbers that all happen to be single digit integer values in the 0 to 9 range, that is 2001 characters at the smallest possible representation, as each number is followed by a comma or a bracket, plus the opening bracket. If those same 1000 numbers are written with the trailing zero and decimal point, that's 4001 characters, just under a 50% savings for the non-trailing-zero case, but still only 2000 characters difference.

As soon as you get outside the range of single digit integers in that array, the savings percentage goes down again. If they are two digit numbers, or negative single digit numbers, then every number is 3 characters in the optimal case and 5 characters in the trailing zero case. For that single array of 1000 numbers, that's 3001 vs 5001 characters, or just under 40%.

If your application is that sensitive to the file size, and you've done everything else that you are able to do to reduce that file size, I think you could have logic in the place where you are adding the value to the nlohmann::json object that detects that the value you are writing is exactly an integer and set it as an int instead of a double, but you're going to pay for that in computation time, so be sure that the integer value is common enough that the extra processing cost is worth the space savings.

[YarNix]

detects that the value you are writing is exactly an integer and set it as an int instead of a double, but you're going to pay for that in computation time
Yeah the intention of my previous reply

The example I've given is purely for demonstration. The actual format is closer to a huge list of object. Each other has a list of keys corresponding to a number.

I totally did the math wrong to get those percentages, sorry for that 😅
The real percentages should be around 20%-46% which is still worth noting.
Well saving 100KB for 1000 levels still add ups to ~100MB so that sound pretty worth the extra computation for me.