Skip to content

Allow exposing CBOR undefined value as JsonToken.VALUE_EMBEDDED_OBJECT #588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 6, 2025
Merged

Allow exposing CBOR undefined value as JsonToken.VALUE_EMBEDDED_OBJECT #588

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
03b355e
Allow exposing CBOR undefined value as JsonToken.VALUE_EMBEDDED_OBJECT
iifawzi May 5, 2025
bdf400e
Add release notes, minor tweaks
cowtowncoder May 6, 2025
cae0ab7
Fix minor bug I introduced :)
cowtowncoder May 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions cbor/src/main/java/com/fasterxml/jackson/dataformat/cbor/CBORConstants.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ public final class CBORConstants

public final static int INT_BREAK = 0xFF;

public final static int SIMPLE_VALUE_UNDEFINED = 0xF7;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could use a Javadoc


/*
/**********************************************************
/* Basic UTF-8 decode/encode table
Expand Down
49 changes: 44 additions & 5 deletions cbor/src/main/java/com/fasterxml/jackson/dataformat/cbor/CBORParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,23 @@ public enum Feature implements FormatFeature
*
* @since 2.20
*/
DECODE_USING_STANDARD_NEGATIVE_BIGINT_ENCODING(false)
DECODE_USING_STANDARD_NEGATIVE_BIGINT_ENCODING(false),

/**
* Feature that determines how an ` undefined ` value (0xF7) is decoded.
*
* <p>
* When enabled, the parser returns {@link JsonToken#VALUE_EMBEDDED_OBJECT} with a
* value of {@code null}, allowing the caller to distinguish `undefined` from actual
* {@link JsonToken#VALUE_NULL}.
*<p>
* When disabled (default, for backwards compatibility), `undefined` value is
* reported as {@link JsonToken#VALUE_NULL}, maintaining legacy behavior from Jackson 2.9.6 to 2.19.
*<p>
*
* @since 2.20
*/
HANDLE_UNDEFINED_AS_EMBEDDED_OBJECT(false)
;

final boolean _defaultState;
Expand Down Expand Up @@ -1915,6 +1931,20 @@ private final byte[] _getBinaryFromString(Base64Variant variant) throws IOExcept
return _binaryValue;
}

/**
* Checking whether the current token represents an `undefined` value (0xF7).
* <p>
* This method allows distinguishing between real {@code null} and `undefined`,
* even if {@link CBORParser.Feature#HANDLE_UNDEFINED_AS_EMBEDDED_OBJECT} is disabled
* and the token is reported as {@link JsonToken#VALUE_NULL}.
*
* @return {@code true} if current token is an `undefined`, {@code false} otherwise
* @since 2.20
*/
public boolean isUndefined() {
return (_inputBuffer[_inputPtr - 1] & 0xFF) == SIMPLE_VALUE_UNDEFINED;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmmh. Not a fan of this approach but... I guess it'll do. Let's preface it by

(_currToken == JsonToken.VALUE_EMBEDDED_OBJECT || _currToken == JsonToken.VALUE_NULL)

to avoid likelihood of incorrect deterimation (binary segment ending with byte 0xF7).

Also: not 100% sure if bounds checks would be needed.
Actually, must check that _inputBuffer is not null -- _releaseBuffers() clears it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I can add that.

}

/*
/**********************************************************
/* Numeric accessors of public API
Expand Down Expand Up @@ -3656,13 +3686,22 @@ private final static long _long(int i1, int i2)
* Helper method to encapsulate details of handling of mysterious `undefined` value
* that is allowed to be used as something encoder could not handle (as per spec),
* whatever the heck that should be.
* Current definition for 2.9 is that we will be return {@link JsonToken#VALUE_NULL}, but
* for later versions it is likely that we will alternatively allow decoding as
* {@link JsonToken#VALUE_EMBEDDED_OBJECT} with "embedded value" of `null`.
*
* <p>
* For backward compatibility with Jackson 2.9.6 to 2.19, this value is decoded
* as {@link JsonToken#VALUE_NULL} by default.
* <p>
*
* since 2.20 If {@link CBORParser.Feature#HANDLE_UNDEFINED_AS_EMBEDDED_OBJECT} is enabled,
* the value will instead be decoded as {@link JsonToken#VALUE_EMBEDDED_OBJECT}
* with an embedded value of {@code null}.
*
* @since 2.9.6
*/
protected JsonToken _decodeUndefinedValue() throws IOException {
protected JsonToken _decodeUndefinedValue() {
if (CBORParser.Feature.HANDLE_UNDEFINED_AS_EMBEDDED_OBJECT.enabledIn(_formatFeatures)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add _binaryValue = null; just as sort of documentation of intent. I know it should be cleared by nextToken() and other places but it seems better be explicit in this particular case.

return JsonToken.VALUE_EMBEDDED_OBJECT;
}
return JsonToken.VALUE_NULL;
}

Expand Down
75 changes: 71 additions & 4 deletions cbor/src/test/java/com/fasterxml/jackson/dataformat/cbor/parse/UndefinedValueTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@

import org.junit.jupiter.api.Test;

import com.fasterxml.jackson.core.JsonParser;
import com.fasterxml.jackson.core.JsonToken;
import com.fasterxml.jackson.dataformat.cbor.*;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;

// for [dataformat-binary#93]
public class UndefinedValueTest extends CBORTestBase
Expand All @@ -21,8 +21,23 @@ public class UndefinedValueTest extends CBORTestBase
@Test
public void testUndefinedLiteralStreaming() throws Exception
{
JsonParser p = cborParser(CBOR_F, new byte[] { BYTE_UNDEFINED });
CBORParser p = cborParser(CBOR_F, new byte[] { BYTE_UNDEFINED });
assertEquals(JsonToken.VALUE_NULL, p.nextToken());
assertTrue(p.isUndefined());
assertNull(p.nextToken());
p.close();
}

// @since 2.20 [jackson-dataformats-binary/137]
@Test
public void testUndefinedLiteralAsEmbeddedObject() throws Exception {
CBORFactory f = CBORFactory.builder()
.enable(CBORParser.Feature.HANDLE_UNDEFINED_AS_EMBEDDED_OBJECT)
.build();
CBORParser p = cborParser(f, new byte[] { BYTE_UNDEFINED });

assertEquals(JsonToken.VALUE_EMBEDDED_OBJECT, p.nextToken());
assertTrue(p.isUndefined());
assertNull(p.nextToken());
p.close();
}
Expand All @@ -34,9 +49,30 @@ public void testUndefinedInArray() throws Exception
out.write(CBORConstants.BYTE_ARRAY_INDEFINITE);
out.write(BYTE_UNDEFINED);
out.write(CBORConstants.BYTE_BREAK);
JsonParser p = cborParser(CBOR_F, out.toByteArray());
CBORParser p = cborParser(CBOR_F, out.toByteArray());
assertEquals(JsonToken.START_ARRAY, p.nextToken());
assertEquals(JsonToken.VALUE_NULL, p.nextToken());
assertTrue(p.isUndefined());
assertEquals(JsonToken.END_ARRAY, p.nextToken());
assertNull(p.nextToken());
p.close();
}

// @since 2.20 [jackson-dataformats-binary/137]
@Test
public void testUndefinedInArrayAsEmbeddedObject() throws Exception {
CBORFactory f = CBORFactory.builder()
.enable(CBORParser.Feature.HANDLE_UNDEFINED_AS_EMBEDDED_OBJECT)
.build();

ByteArrayOutputStream out = new ByteArrayOutputStream();
out.write(CBORConstants.BYTE_ARRAY_INDEFINITE);
out.write(BYTE_UNDEFINED);
out.write(CBORConstants.BYTE_BREAK);
CBORParser p = cborParser(f, out.toByteArray());
assertEquals(JsonToken.START_ARRAY, p.nextToken());
assertEquals(JsonToken.VALUE_EMBEDDED_OBJECT, p.nextToken());
assertTrue(p.isUndefined());
assertEquals(JsonToken.END_ARRAY, p.nextToken());
assertNull(p.nextToken());
p.close();
Expand All @@ -57,11 +93,42 @@ public void testUndefinedInObject() throws Exception
// assume we use end marker for Object, so
doc[doc.length-2] = BYTE_UNDEFINED;

JsonParser p = cborParser(CBOR_F, doc);
CBORParser p = cborParser(CBOR_F, doc);
assertEquals(JsonToken.START_OBJECT, p.nextToken());
assertEquals(JsonToken.FIELD_NAME, p.nextToken());
assertEquals("bar", p.currentName());
assertEquals(JsonToken.VALUE_NULL, p.nextToken());
assertTrue(p.isUndefined());
assertEquals(JsonToken.END_OBJECT, p.nextToken());
assertNull(p.nextToken());
p.close();
}

// @since 2.20 [jackson-dataformats-binary/137]
@Test
public void testUndefinedInObjectAsEmbeddedObject() throws Exception {
CBORFactory f = CBORFactory.builder()
.enable(CBORParser.Feature.HANDLE_UNDEFINED_AS_EMBEDDED_OBJECT)
.build();

ByteArrayOutputStream out = new ByteArrayOutputStream();
CBORGenerator g = cborGenerator(out);
g.writeStartObject();
g.writeFieldName("bar");
g.writeBoolean(true);
g.writeEndObject();
g.close();

byte[] doc = out.toByteArray();
// assume we use end marker for Object, so
doc[doc.length - 2] = BYTE_UNDEFINED;

CBORParser p = cborParser(f, doc);
assertEquals(JsonToken.START_OBJECT, p.nextToken());
assertEquals(JsonToken.FIELD_NAME, p.nextToken());
assertEquals("bar", p.currentName());
assertEquals(JsonToken.VALUE_EMBEDDED_OBJECT, p.nextToken());
assertTrue(p.isUndefined());
assertEquals(JsonToken.END_OBJECT, p.nextToken());
assertNull(p.nextToken());
p.close();
Expand Down