Skip to content

Commit 10fc1ba

Browse files
MichalFoksaMichalFoksa
authored
Add Logical Type support for java.util.UUID (#536)
1 parent ad37021 commit 10fc1ba

File tree

8 files changed

+125
-25
lines changed

8 files changed

+125
-25
lines changed

‎avro/README.md

Lines changed: 21 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -114,32 +114,34 @@ and that's about it, for now.
114114

115115
## Avro Logical Types
116116

117-
Following is an extract from [Logical Types](http://avro.apache.org/docs/current/specification/_print/#logical-types) paragraph in
118-
Avro schema specification:
117+
The following is an excerpt from the [Logical Types](https://avro.apache.org/docs/1.11.1/specification/#logical-types) section of
118+
the Avro schema specification:
119+
119120
> A logical type is an Avro primitive or complex type with extra attributes to represent a derived type. The attribute
120-
> `logicalType` is always be present for a logical type, and is a string with the name of one of the logical types
121-
> defined by Avro specification.
121+
> `logicalType` must always be present for a logical type, and is a string with the name of one of the logical types
122+
> listed later in this section. Other attributes may be defined for particular logical types.
123+
124+
Logical types are supported for a limited set of `java.time` classes and for 'java.util.UUID'. See the table below for more details.
122125

123-
Generation of logical types for limited set of `java.time` classes is supported at the moment. See a table bellow.
126+
### Mapping to Logical Types
124127

125-
### Mapping to Logical Type
128+
Mapping to Avro type and logical type involves these steps:
126129

127-
Mapping to Avro type and logical type works in few steps:
128-
1. Serializer for particular Java type (or class) determines a Jackson type where the Java type will be serialized into.
129-
2. `AvroSchemaGenerator` determines corresponding Avro type for that Jackson type.
130-
2. If logical type generation is enabled, then `logicalType` is determined for the above combination of Java type and
131-
Avro type.
130+
1. The serializer for a Java type identifies the Jackson type it will serialize into.
131+
2. The `AvroSchemaGenerator` maps that Jackson type to the corresponding Avro type.
132+
3. `logicalType` value is combination of Java type and Jackson type.
132133

133134
#### Java type to Avro Logical Type mapping
134135

135-
| Java type | Serialization type | Generated Avro schema with Avro type and logical type
136-
| ----------------------------- | ------------------ | -----------------------------------------------------
137-
| `java.time.OffsetDateTime` | NumberType.LONG | `{"type": "long", "logicalType": "timestamp-millis"}`
138-
| `java.time.ZonedDateTime` | NumberType.LONG | `{"type": "long", "logicalType": "timestamp-millis"}`
139-
| `java.time.Instant` | NumberType.LONG | `{"type": "long", "logicalType": "timestamp-millis"}`
140-
| `java.time.LocalDate` | NumberType.INT | `{"type": "int", "logicalType": "date"}`
141-
| `java.time.LocalTime` | NumberType.INT | `{"type": "int", "logicalType": "time-millis"}`
142-
| `java.time.LocalDateTime` | NumberType.LONG | `{"type": "long", "logicalType": "local-timestamp-millis"}`
136+
| Java type | Jackson type | Generated Avro schema with logical type |
137+
|----------------------------|-----------------|---------------------------------------------------------------------------------------------------|
138+
| `java.time.OffsetDateTime` | NumberType.LONG | `{"type": "long", "logicalType": "timestamp-millis"}` |
139+
| `java.time.ZonedDateTime` | NumberType.LONG | `{"type": "long", "logicalType": "timestamp-millis"}` |
140+
| `java.time.Instant` | NumberType.LONG | `{"type": "long", "logicalType": "timestamp-millis"}` |
141+
| `java.time.LocalDate` | NumberType.INT | `{"type": "int", "logicalType": "date"}` |
142+
| `java.time.LocalTime` | NumberType.INT | `{"type": "int", "logicalType": "time-millis"}` |
143+
| `java.time.LocalDateTime` | NumberType.LONG | `{"type": "long", "logicalType": "local-timestamp-millis"}` |
144+
| `java.util.UUID` (2.19+) | | `{"type": "fixed", "name": "UUID", "namespace": "java.util", "size": 16, "logicalType" : "uuid"}` |
143145

144146
_Provided Avro logical type generation is enabled._
145147

‎avro/src/main/java/com/fasterxml/jackson/dataformat/avro/schema/AvroSchemaHelper.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -286,7 +286,7 @@ public static Schema createEnumSchema(BeanDescription bean, List<String> values)
286286
* @since 2.11
287287
*/
288288
public static Schema createUUIDSchema() {
289-
return Schema.createFixed("UUID", "", "java.util", 16);
289+
return Schema.createFixed("UUID", null, "java.util", 16);
290290
}
291291

292292
/**

‎avro/src/main/java/com/fasterxml/jackson/dataformat/avro/schema/StringVisitor.java

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -39,11 +39,7 @@ public Schema builtAvroSchema() {
3939
// should we construct JavaType for `Character.class` in case of primitive or... ?
4040
return AvroSchemaHelper.numericAvroSchema(NumberType.INT, _type);
4141
}
42-
// [dataformats-binary#179]: need special help with UUIDs, to coerce into Binary
43-
// (could actually be
44-
if (_type.hasRawClass(java.util.UUID.class)) {
45-
return AvroSchemaHelper.createUUIDSchema();
46-
}
42+
4743
BeanDescription bean = _provider.getConfig().introspectClassAnnotations(_type);
4844
Schema schema = Schema.create(Schema.Type.STRING);
4945
// Stringable classes need to include the type

‎avro/src/main/java/com/fasterxml/jackson/dataformat/avro/schema/UUIDVisitor.java

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
package com.fasterxml.jackson.dataformat.avro.schema;
2+
3+
import java.util.Set;
4+
5+
import com.fasterxml.jackson.databind.jsonFormatVisitors.JsonStringFormatVisitor;
6+
import com.fasterxml.jackson.databind.jsonFormatVisitors.JsonValueFormat;
7+
8+
import org.apache.avro.LogicalTypes;
9+
import org.apache.avro.Schema;
10+
11+
/**
12+
* Visitor for {@link java.util.UUID} type. When it is created with logicalTypesEnabled enabled,
13+
* Avro schema is created with logical type uuid.
14+
*
15+
* @since 2.19
16+
*/
17+
public class UUIDVisitor extends JsonStringFormatVisitor.Base
18+
implements SchemaBuilder {
19+
protected boolean _logicalTypesEnabled = false;
20+
21+
22+
public UUIDVisitor(boolean logicalTypesEnabled) {
23+
_logicalTypesEnabled = logicalTypesEnabled;
24+
}
25+
26+
@Override
27+
public void format(JsonValueFormat format) {
28+
// Ideally, we'd recognize UUIDs, Dates etc if need be, here...
29+
}
30+
31+
@Override
32+
public void enumTypes(Set<String> enums) {
33+
// Do nothing
34+
}
35+
36+
@Override
37+
public Schema builtAvroSchema() {
38+
// [dataformats-binary#179]: need special help with UUIDs, to coerce into Binary
39+
// (could actually be
40+
Schema schema = AvroSchemaHelper.createUUIDSchema();
41+
return this._logicalTypesEnabled ? LogicalTypes.uuid().addToSchema(schema) : schema;
42+
}
43+
}

‎avro/src/main/java/com/fasterxml/jackson/dataformat/avro/schema/VisitorFormatWrapperImpl.java

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -214,6 +214,12 @@ public JsonStringFormatVisitor expectStringFormat(JavaType type)
214214
return v;
215215
}
216216

217+
if (type.hasRawClass(java.util.UUID.class)) {
218+
UUIDVisitor v = new UUIDVisitor(this._logicalTypesEnabled);
219+
_builder = v;
220+
return v;
221+
}
222+
217223
StringVisitor v = new StringVisitor(_provider, type);
218224
_builder = v;
219225
return v;

‎avro/src/test/java/com/fasterxml/jackson/dataformat/avro/schema/UUIDVisitor_builtAvroSchemaTest.java

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
package com.fasterxml.jackson.dataformat.avro.schema;
2+
3+
import org.junit.Test;
4+
5+
import org.apache.avro.LogicalType;
6+
import org.apache.avro.Schema;
7+
8+
import static org.assertj.core.api.Assertions.assertThat;
9+
10+
public class UUIDVisitor_builtAvroSchemaTest {
11+
12+
@Test
13+
public void testLogicalTypesDisabled() {
14+
// GIVEN
15+
boolean logicalTypesEnabled = false;
16+
UUIDVisitor uuidVisitor = new UUIDVisitor(logicalTypesEnabled);
17+
18+
// WHEN
19+
Schema actualSchema = uuidVisitor.builtAvroSchema();
20+
21+
// THEN
22+
assertThat(actualSchema.getType()).isEqualTo(Schema.Type.FIXED);
23+
assertThat(actualSchema.getFixedSize()).isEqualTo(16);
24+
assertThat(actualSchema.getName()).isEqualTo("UUID");
25+
assertThat(actualSchema.getNamespace()).isEqualTo("java.util");
26+
assertThat(actualSchema.getProp(LogicalType.LOGICAL_TYPE_PROP)).isNull();
27+
}
28+
29+
@Test
30+
public void testLogicalTypesEnabled() {
31+
// GIVEN
32+
boolean logicalTypesEnabled = true;
33+
UUIDVisitor uuidVisitor = new UUIDVisitor(logicalTypesEnabled);
34+
35+
// WHEN
36+
Schema actualSchema = uuidVisitor.builtAvroSchema();
37+
38+
// THEN
39+
assertThat(actualSchema.getType()).isEqualTo(Schema.Type.FIXED);
40+
assertThat(actualSchema.getFixedSize()).isEqualTo(16);
41+
assertThat(actualSchema.getName()).isEqualTo("UUID");
42+
assertThat(actualSchema.getNamespace()).isEqualTo("java.util");
43+
assertThat(actualSchema.getProp(LogicalType.LOGICAL_TYPE_PROP)).isEqualTo("uuid");
44+
}
45+
46+
}

‎release-notes/CREDITS-2.x

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -225,6 +225,8 @@ Michal Foksa (MichalFoksa@github)
225225
* Contributed #494: Avro Schema generation: allow mapping Java Enum properties to
226226
Avro String values
227227
(2.18.0)
228+
* Contributed #536: (avro) Add Logical Type support for `java.util.UUID`
229+
(2.19.0)
228230

229231
Hunter Herman (hherman1@github)
230232

‎release-notes/VERSION-2.x

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,11 @@ Active maintainers:
1414
=== Releases ===
1515
------------------------------------------------------------------------
1616

17+
2.19.0 (not yet released)
18+
19+
#536: (avro) Add Logical Type support for `java.util.UUID`
20+
(contributed by Michal F)
21+
1722
2.18.2 (27-Nov-2024)
1823

1924
No changes since 2.18.1

0 commit comments

Comments
 (0)