Overhauled serialization system

[eviltak]

Contents

• Clean serialization API for use in the engine and in modules
• Add serialization support for various types previously unsupported
• Modularize serialization to make it easier to add custom type serialization
• Rework serialization to serialize objects via Gson or Protobuf by changing a single line of code (see AbstractSerializer, GsonSerializer and ProtobufSerializer)

Closes #3490.

Review guide

This PR is finally ready to review and merge! Since there are a lot of changes to review, I'd recommend taking a look at the user-facing API introductions/changes first:

• AbstractSerializer
• GsonSerializer
• ProtobufSerializer
• TypeHandlerLibrary (renamed from TypeSerializationLibrary)
• TypeHandler
• TypeHandlerContext
• TypeHandlerFactory
• SerializedName
• TypeInfo
• ReflectionUtil -- will probably not be used outside the engine at all, but it's still worth a look to see if I missed documenting any public methods
• SerializationSandbox -- again, not intended for use outside the engine, but still worth a look

Since there are a lot of files changed by this PR (still no conflicts, surprisingly!), I'd suggest you initially skip the ones with few (< 20) changed lines and a similar number of additions and deletions -- almost all of these files were changed by the IDE during a refactor and have no real reviewable changes.

After you finish reviewing the API changes in the files listed above, you can take a look at the logic introduced by the code in this PR: almost all of it resides in the persistence.typeHandling package under a TypeHandler or a TypeHandlerFactory, or the ReflectionUtil class. However, I am pretty confident that all these classes/methods have been tested thoroughly to ensure they work as intended, so this step can be skipped if needed.

I'd then recommend looking at the tests to see how the new serialization system would be used; the main ones to look at here are TypeSerializerTest and TypeHandlerLibraryTest, since these test most of the user-facing APIs.

Testing

Check out branch MovingBlocks/newSerialization and run ./gradlew tests

TODO

• Prevent modules for adding handlers for types they do not declare (compare module declaring TypeHandler<T> and module declaring T).
• Add TypeHandlerLibrary.getRuntimeTypeHandler that returns the retrieved type handler wrapped in a RuntimeDelegatingTypeHandler to support subtype serialization
• Create a reflection abstraction over ModuleEnvironment/Reflections with more AbstractClassLibrary-esque resolution features (if made with a ModuleEnvironment).
• Flatten the RuntimeDelegatingTypeHandler serialization format by adding the class entry to the serialized object itself, if the serialized representation is a map

Module Fixes

• Fixes for new Terasology serialization system (engine PR 3535) Terasology/Tasks#7
• Fixes for new Terasology serialization system (engine PR 3535) Terasology/DynamicCities#33
• Fixes for new Terasology serialization system (engine PR 3535) Terasology/Dialogs#8
• Adjustments needed for engine PR 3456 Terasology/MasterOfOreon#29
• Adjustments needed for engine PR 3456 Terasology/LightAndShadow#74
• Adjustments (maybe) needed for engine PR 3456 Terasology/GooeyDefence#30
• Adjustments (maybe) needed for engine PR 3456 Terasology/GooeysQuests#31

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Uh oh, something went wrong with the build. Need to check on that

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[syntaxi]

@eviltak, is this ready to be reviewed and then subsequently merged? Or is there still outstanding work blocking this?
Would love to get this in as part of the merge-ocalypse

[eviltak]

@jellysnake unfortunately there is still some work to be done, potentially involving related changes in gestalt. I would say that this should be merged only after all modules have PRs fixing code that has been broken by these changes. I shall update the PR description with clear todo items so that progress can be easily gauged!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[Cervator]

This pull request introduces 2 alerts when merging 0ae28ce into b6590b7 - view on LGTM.com

new alerts:

• 1 for Potential output resource leak
• 1 for Dereferenced variable may be null

---

Comment posted by LGTM.com

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Uh oh, something went wrong with the build. Need to check on that

[GooeyHub]

Uh oh, something went wrong with the build. Need to check on that

[GooeyHub]

Uh oh, something went wrong with the build. Need to check on that

[GooeyHub]

Uh oh, something went wrong with the build. Need to check on that

[GooeyHub]

Uh oh, something went wrong with the build. Need to check on that

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[GooeyHub]

Hooray Jenkins reported success with all tests good!

[Cervator]

Testing & merge report

Tried out the following modules to some degree, including their dependencies & fix PRs if ready:

• Tasks & QuestExamples - found a minor bug in the tasks type handler, actually managed to fix it woo
• MedievalCities - mostly to exercise a smaller set of modules at first, fixed the Perlin->Simplex thing
• LightAndShadow (GSOC) - still had a soft dependency on Tasks, restored it for now
• MetalRenegades (GSOC) - no module fix PR yet, managed to put one together with a bunch of tweaks
• MasterOfOreon (TSOC)
• WildAnimalsMadness (GSOC)
• Apiculture (GSOC)

I think that's the critical stuff for now. JoshariasSurvival and others need some attention in the near future as well, but there's no GSOC or TSOC impact on there, so less time sensitive. Phiew. This might be happening!

/itshappeninggif