Skip to content

Commit

Permalink
ZCS-691 - Bringing in dev-soap as soap subdirectory
Browse files Browse the repository at this point in the history
  • Loading branch information
Shriram V committed Feb 17, 2017
1 parent a0d25b8 commit d24bacc
Show file tree
Hide file tree
Showing 1,959 changed files with 161,179 additions and 0 deletions.
3 changes: 3 additions & 0 deletions soap/.gitignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
build/
.classpath
.project
300 changes: 300 additions & 0 deletions soap/README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,300 @@
# zm-soap Repository which hosts JAXB classes used internally by Zimbra Server and the basis of our WSDL definition

## Inputs from Perforce

- `./build.xml`
- `./docs`
- `./soapdocs`
- `./src/java`
- `./src/java-test`

## Dependencies

- `zm-common`
- `zm-thirdparty-jars`

## Artifacts

- `zimbrasoap.jar`
- `soapapi-changelog.zip`
- `soapapi-zimbra-doc.zip`

## Implementation Notes for JAXB classes

See also [Testing Notes](#testingNotes). It is important to validate that any new classes can be used within
Zimbra and **also** by code generated from our WSDL definition, which is based on these JAXB classes

### Steps to Create JAXB classes for a new request/response pair

1. See `Steps to add a new namespace` if necessary - although it is recommended that new namespaces are
**NOT** created unless absolutely necessary.
2. Determine the package name **{PKG}** appropriate for the namespace of your request/response pair. For instance:

grep namespaceURI src/java/com/zimbra/soap/*/message/package-info.java

currently yields output like:

src/java/com/zimbra/soap/account/message/package-info.java: @XmlNs(prefix="account", namespaceURI = "urn:zimbraAccount")
src/java/com/zimbra/soap/admin/message/package-info.java: @XmlNs(prefix="admin", namespaceURI = "urn:zimbraAdmin")
src/java/com/zimbra/soap/adminext/message/package-info.java: @XmlNs(prefix="adminExt", namespaceURI = "urn:zimbraAdminExt")
src/java/com/zimbra/soap/mail/message/package-info.java: @XmlNs(prefix="mail", namespaceURI = "urn:zimbraMail")
src/java/com/zimbra/soap/replication/message/package-info.java: @XmlNs(prefix="repl", namespaceURI = "urn:zimbraRepl")
src/java/com/zimbra/soap/sync/message/package-info.java: @XmlNs(prefix="sync", namespaceURI = "urn:zimbraSync")
src/java/com/zimbra/soap/voice/message/package-info.java: @XmlNs(prefix="voice", namespaceURI = "urn:zimbraVoice")

`{PKG}` is the name of the directory component before message/package-info.java

3. Create `src/java/com/zimbra/soap/{PKG}/message/{Request}.java` where `{Request}` is the top level element name for
your new request. Base this on other similar files. See below for generic advice on JAXB annotations.
4. Create `src/java/com/zimbra/soap/{PKG}/message/{Response}.java` where `{Response}` is the top level element
name for your new response.
5. Create other JAXB classes for data types needed by your request/response pair if they don't exist already.
6. Update the **MESSAGE_CLASSES** array in `src/java/com/zimbra/soap/JaxbUtil.java`

### Steps to add a new namespace

Ideally, don't! It makes things way more complicated than it needs to be when you want to share objects.
So, just in case you have chosen to ignore that advice, this is how to proceed:

1. Choose a new sub-package name `{PKG}` and a service name `{SVC}`
e.g. for AppBlast, might choose `{PKG}="appblast"` and `{SVC}=AppblastService`
2. Create new directories:

src/java/com/zimbra/soap/{PKG}/message
src/java/com/zimbra/soap/{PKG}/type

3. Create new files:

src/java/com/zimbra/soap/{PKG}/message/package-info.java
src/java/com/zimbra/soap/{PKG}/type/package-info.java

These can be based on similar files for another namespace.
Need to update namespace/package etc info in these files.
4. Update `src/java/com/zimbra/soap/util/WsdlGenerator.java`.
Either the `addAdminNamespaceInfo` or `addUserNamespaceInfo` method needs something like :

nsInfoList.add(WsdlInfoForNamespace.create(AppBlastConstants.NAMESPACE_STR, zcsService,
packageToRequestListMap.get("com.zimbra.soap.appblast.message")));

5. Update `soapdocs/src/java/com/zimbra/doc/soap/WsdlDocGenerator.java`
the static block which initialises **serviceDescriptions** needs updating
6. Update binding file `../zm-wsdl-test/bindings/xsdBindings-zcs.xml` to ensure that WSDL tests will function
correctly. Any public Wiki articles on how to use our WSDL might need changes too.
This will need a `<jaxb:bindings>` for the new targetNamespace.

### Manual steps

#### Marshalling / Unmarshalling support

The code that is responsible for marshalling and unmarshalling JAXB classes needs to know about the top level
requests and response classes.


See the `MESSAGE_CLASSES` array in `src/java/com/zimbra/soap/JaxbUtil.java`

#### Ordering of elements.

For pre-existing requests, best not to force an ordering of elements in case this breaks clients.
For responses, because the server controls the order, it is reasonable to enforce an order.

Enforcing an order is achieved via the @XmlType annotation. For example :

@XmlType(propOrder = {"hostNames", "stats", "note"})

The strings in the list are the names of fields which map to elements (NOT the `@XmlElement` names).
Attributes cannot be ordered, so their field names are excluded.

#### @XmlRootElement

SOAP Request and Response classes must have this.

In a lot of JAXB, the root element associated with a field is chosen by the `@XmlElement` annotation associated with
the field - so there is no need for an `@XmlRootElement`. This has the added advantage that classes can
represent types independant of the element name used with them.
Problems were encountered in some of the COS related classes when an un-necessary `@XmlRootElement` was used.
On the other hand, see the gotchas. Sometimes an `@XmlRootElement` is required...

#### @XmlAccessorType

Tend to prefer `XmlAccessType.NONE` and explicitly label everything that needs
to be annotated rather than letting JAXB do some defaults.

#### enums

For the values of some fields in JAXB objects, Enums are needed under `zm-soap`.
For preference, non-`zm-soap` code should use the `zm-soap` enums to avoid having to keep 2 sets of enums in lock step.

If that isn't possible, either because an enum is already widely used or because it references objects not visible
in `zm-soap`, then using the `zm-soap` enum in the constructor of the other enum should help avoid the enums getting
out of step.

For example, see JAXB enum `com.zimbra.soap.type.TargetType` and `zm-store` enum
`com.zimbra.cs.account.accesscontrol.TargetType`. The latter enum includes the
`toJaxb()` method and the static method `fromJaxb(com.zimbra.soap.type.TargetType)`.

## Gotchas

* Do NOT reference non-simple classes outside the `zm-soap` hierarchy in fields relevant to JAXB.
Failing to do this will result in errors in the resulting WSDL as it will not have
the information it needs to specify the structure of these classes.
* Unmarshalling from Xml may not correctly identify superclasses.
Test before using! e.g. See : `JaxbToElementTest.ConvActionRequestJaxbSubclassHandlingTestDisabled()`
* Target objects for `@XmlElementRef` etc must have an `@XmlRootElement` annotation
* For structured fields, e.g. comma separated strings, there is a temptation to use non-JAXB fields for the real data.
* Elements which have both sub-elements AND a value. End up having to used `@XmlMixed` which is horrible.
**No new code should allow this**.
* You cannot use basic types like `int` for optional attributes/variables.
This is because you need something which can have a null value to represent absence.

## Special handling

### Boolean / boolean support using ZmBoolean

XML schema regards the literals `{true, false, 1, 0}` as legal
However, the canonical representation for boolean is the set of literals `{true, false}` and this is reflected in
JAXB behavior when marshalling. Historically, Zimbra Booleans have been rendered in SOAP response XML as `{1, 0}`
To further complicate things, SOAP response JSON uses `{true, false}`.

When adding `Boolean` or `boolean` attributes/elements to SOAP requests or responses:

1. Specify the corresponding fields in JAXB with type `ZmBoolean`
2. Add methods which get and set these fields via Boolean (or boolean for required fields).
The static methods `ZmBoolean.fromBool(Boolean)`, `ZmBoolean.toBool(ZmBoolean)` and
`ZmBoolean.toBool(ZmBoolean, boolean)` should be used from these methods.
3. Do NOT create `ZmBoolean` getters and setters!
4. If you are creating an element that should be treated as an element in JSON, use an `@JsonSerialize` annotation,
for example :

@XmlElement(name=AdminConstants.E_STATUS, required=true)
@JsonSerialize(using=ZmBooleanContentSerializer.class)
private final ZmBoolean status;

### FAQ

1. How do you define an `XmlAttribute` or `XmlElement` whose value is an enum?
You need to create an enum in `zm-soap`. Ideally, used this new enum in all code rather than duplicating
from somewhere else. For a fairly simple example - see:

src/java/com/zimbra/soap/type/AccountBy.java

which uses the `@XmlEnum` annotation. See:

src/java/com/zimbra/soap/admin/type/AccountSelector.java

for how this enum is used for an `@XmlAttribute`.
If you want the enum values to differ from what is in the xml, see another example - Folder.View:

src/java/com/zimbra/soap/mail/type/Folder.java

This also uses `@XmlEnumValue` annotations.

2. For some JAXB classes, `@XmlElement` or `@XmlAttribute` might have property `required` set as `false` even when it
is not optional. Is this intentional?

If the `@XmlElement` or `@XmlAttribute` is required in all contexts then this is an error. When the original
implementor was constructing JAXB classes, situations were encountered where looking at the first use of a
JAXB type some of these were clearly required but later uses had them as optional. As this situation can
be a bit difficult to spot a lot of newer JAXB classes were written with a more relaxed view of whether
things were required or not.

3. I'm creating new JAXB classes for a request/response in the `urn:zimbraAdmin` namespace which calls code which
is already used for request/responses in the `urn:zimbraAccount` namespace. Can I re-use the JAXB objects in
package `com.zimbra.soap.account.type`?

You need to ensure that you won't have namespace related problems. If the SOAP handler code uses Element
code to build the structure, most elements are added using Element.addElement(String). This causes the namespace
associated with the parent to be associated with the new element, which will often mean that it will differ
depending on the main namespace associated with the request/response being handled and the same JAXB class cannot
be used.

There are 2 suggested solutions:

1. If the current code is only used in one namespace, then namespace lock the data structure by changing
the creation of the top level Element for the structure from `Element.addElement(String)` to
`Element.addElement(QName)`. Where possible, this is the preferred option as it avoids propagating almost
duplicate JAXB objects for different namespaces.
Check the .xsd files produced to make sure they make sense. There is a gotcha with schemagen where
it sometimes uses references (ref=)to the other .xsd file but has a poor algorithm for choosing the reference
name (it just uses the xml element name). If more than one reference ends up using the same name, you can
end up with the target type being something like :

<xs:element name="device" nillable="true" type="xs:anyType"/>

2. If the current code is used in multiple namespaces, then the approach I'm taking these days is to create
almost duplicate JAXB objects in each namespace where the data structure is used and have the objects
implement an interface which encapsulates the important aspects of their behavior in package
`com.zimbra.soap.base`. The plan is that when we move to using JAXB code instead of Element code we will be
able to use the interface combined with appropriate JAXB factories. This is rather fiddly but I don't
think there is a good alternative.

Aside:
In some cases in the past, if a JAXB class has only had `@XmlAttribute` (i.e. no `@XmlElement`)
I have moved the JAXB class to `com.zimbra.soap.type` and re-used it. This works because we have things
setup not to tie `@XmlAttribute` to a namespace when processing the JAXB. However, the issue was raised
that this is confusing as it isn't clear why this works but objects containing elements don't.
So, although this results in simpler code it is a deprecated practise.

4. I have wrapped lists which when they have 2 entries end up with an example structure like:

<ExampleResponse>
<numbers>
<number>4</number>
<number>3</number>
</numbers>
</ExampleResponse>

If I have no numbers, I get:

<ExampleResponse>
<numbers/>
</ExampleResponse>

but I would like:

<ExampleResponse>
</ExampleResponse>

how do I achieve that?

Answer - see the `WrapperAbsentIfEmpty` class and its usage from `JaxbToJsonTest`. Note that
`@XmlElementWrapper` / `@XmlElement` are on the getter rather than the field (which should have the `@XmlTransient`
annotation). JAXB then lets the getter and setter methods control the mapping with XML. If the return from
a getter is null, no XML wrapper element will be output, but if the return is an empty list, the wrapper
element will be present.

<a name="testingNotes"></a>
## Testing notes

It is important to validate that any new classes can be used within Zimbra and **also** by code generated from our
WSDL definition, which is based on these JAXB classes

### ant "test" target
These tests do not require Zimbra to be installed on the system under test.
Useful for exercising marshalling and unmarshalling for JAXB objects under `src/java/`

### zmsoap -z RunUnitTestsRequest

This is the classic `zm-store` RunUnitTests framework which relies on Zimbra being installed on the local
system and some test accounts etc being populated.

Some of this exercices `zm-soap` JAXB classes.
e.g. see `com.zimbra.qa.unittest.TestJaxbProvisioning` in the `zm-store` repository.

### The zm-wsdl-test repository ant "test" target

These tests are intended to be a clean room test of Zimbra's WSDL, with no dependencies on Zimbra product
source code - only using the `.wsdl` and `.xsd` files discoverable from the installed Zimbra Server.

The tests rely on Java API files automatically generated from Zimbra's WSDL definition.

The main intention is to validate that correct, working WSDL client software can be written based on our WSDL.

The JAXB classes under `src/java/` in the `zm-soap` repository are NOT tested directly by these tests, although to
some extent their correctness is being validated as the WSDL files are based on them and the test API is based on
the WSDL.

These tests have the same requirements as the classic `zm-store` RunUnitTests framework.

### TODO

It would be good to have demonstration clients using other programming languages based on our WSDL
Loading

0 comments on commit d24bacc

Please sign in to comment.