Weird "bitmap$0" field in version 2.7.2

[crumbpicker]

I'm trying to switch from 2.6.3 to 2.7.2.
In my Unit test I have some errors due to the presence of a field which is not a member of my case class.
At the end of my generated JSON string I have the following additional field:
"bitmap$0":false

[nbauernfeind]

@crumbpicker Any chance you can provide me a version of your case class that does this?

[mancvso]

Also, there's an option to (not) fail on unknown properties, right there in the mapper.

[nbauernfeind]

bitmap$0 usually has to deal with lazy vals. So I tried this:

case class LazyValTestCaseClass() {
  lazy val intVal: Int = 1
  lazy val strVal: String = "foo"
}

However, it seems to work properly so I'm going to need a little more background for your failure.

[crumbpicker]

I think I found the problem.
It seems that defining a "lazy val field" annoted with @JsonProperty generate this bitmap field when we use the the configuration "PropertyAccessor.FIELD = Visibility.ANY".

Here is a simple test case:

case class User(
  val firstName: String,
  val lastName: String
) {
  @JsonProperty lazy val fullName: String = firstName +' ' + lastName
}

val u = User("David","Bowie")

val objMapper = new ObjectMapper() with ScalaObjectMapper
objMapper.registerModule(DefaultScalaModule)
objMapper.setVisibility(PropertyAccessor.ALL, Visibility.NONE)
objMapper.setVisibility(PropertyAccessor.FIELD, Visibility.ANY)
println( objMapper.writeValueAsString(u) ) // {"firstName":"David","lastName":"Bowie","fullName":null,"bitmap$0":false}

It seems that the lazy val is not evaluated before serialization.

[nbauernfeind]

Ok, thanks that's definitely useful. However, I see this test failing for 2.5.3, 2.6.5, and 2.7.2. What version were you using previously?

[araqnid]

The lazy val won't be calculated with the getters marked as non-visible,
that seems like a different (non-)issue to me.

Separately, the "bitmap$0" is appearing if private fields are
auto-detected. Possibly private volatile fields named "bitmap$..." should
simply be excluded from auto-detection.

(I'm also seeing this behaviour on 2.6.5 btw, it doesn't seem to be new)

SRH

On 2 March 2016 at 15:02, crumbpicker [email protected] wrote:

I think I found the problem.
It seems that defining a "lazy val field" annoted with @JsonProperty
generate this bitmap field when we use the the configuration
"PropertyAccessor.FIELD = Visibility.ANY".

Here is a simple test case:

case class User(
val firstName: String,
val lastName: String
) {
@JsonProperty lazy val fullName: String = firstName +' ' + lastName
}

val u = User("David","Bowie")

val objMapper = new ObjectMapper() with ScalaObjectMapper
objMapper.registerModule(DefaultScalaModule)
objMapper.setVisibility(PropertyAccessor.ALL, Visibility.NONE)
objMapper.setVisibility(PropertyAccessor.FIELD, Visibility.ANY)
println( objMapper.writeValueAsString(u) ) // {"firstName":"David","lastName":"Bowie","fullName":null,"bitmap$0":false}

It seems that the lazy val is not evaluated before serialization.

—
Reply to this email directly or view it on GitHub
#238 (comment)
.

"My home is not a place, it is people."

[nbauernfeind]

@crumbpicker Your expected behavior would be to, basically, invoke the getter in this context (leaving the lazy val initialized after serialization)?

[crumbpicker]

I was using 2.6.3 previously but I had disabled this specific test because of an other issue in version 2.6.x (FasterXML/jackson-databind#849). So I'm not really sure when this behavior really changed.
What would be ideal for me is to have "@JsonProperty lazy fields" serialized in the generated JSON String. So I think that invoking the getter is a good idea.
However I'm not sure I can answer to question "Should we leave the lazy field initialized or not after serialization ?". I let you decide what you find the most appropriate.

[nbauernfeind]

Intuitively I would expect the lazy val to be initialized as jackson reads it; I'll make that happen. Note: I checked the 2.4 branch and it was still not working there so I don't think this ever worked.

If you need a more immediate work-around, you could make your lazy val a def or a val. I don't know how to fix this yet, but I will find a way.

[christophercurrie]

Lazy val support has been raised before as issue #113.

[crumbpicker]

You're right @christophercurrie, I totally forgot I created this issue two years ago ;-)
Thank you !
@nbauernfeind I really can't remove swtich my lazy fields to defs (I have to many cases). So I'll wait for a fix before upgrading to 2.7.x. Thanks !

[christophercurrie]

FWIW, the delay in addressing lazy vals has been to retain historic support for Scala 2.10, because if the lack of thread safety in scala-reflect for that version. If Scala 2.10 is no longer a need for end users, it becomes a much simpler prospect.

[crumbpicker]

For me it's not a problem, but it's hard to know how many people have this requirement.
I created a poll for this purpose:
Here is the link to participate:
http://www.polljunkie.com/poll/ztgebz/scala-210-support-in-jackson-module-scala

Here is the link to the admin page:
http://www.polljunkie.com/poll/wnfciw/scala-210-support-in-jackson-module-scala/admin

Maybe you could put the first link on the readme page ?

Another information source is maven central (http://mvnrepository.com/artifact/com.fasterxml.jackson.module):

• 111 usages of jackson-module-scala_2.10
• 87 usage of jackson-module-scala_2.11

It's important to note that these values may not be relevant because they concern all historical jackson versions. But it's still interesting to see that the usage number is lower for 2.11.

[cowtowncoder]

@crumbpicker I can also have a look at Sonatype OSS repo for actual downloads, and although update is bit spotty (feb numbers are blank), it seems to me that counts actually favor 2.10:

• 2.11: about 25,000 downloads in Jan 2016
• 2.10: about 80,000 downloads in Jan 2016
• 2.9: barely more than ~100 for all 2.9.x in Jan 2016

for whatever that is worth. But I don't think deprecating 2.10 is yet feasible.

[christophercurrie]

Of those 80k 2.10 downloads, half of them were for version 2.4.4, which is more than a year old at this point; similarly, ~14k of the 2.11 downloads were in the 2.4.x series. So I expect some large project depends on that version somewhere.

But looking at recent releases, 2.11 usage at least doubles that of 2.10 for all release in the 2.6 series. Also, Scala 2.12 is at Milestone 3, and theoretically should release this year. So perhaps the time is right to consider removing support for 2.10 in Jackson 2.8.

[cowtowncoder]

@christophercurrie Good point. 2.4.4 is rather widely used (by AWS client, spark-core et al).

Anyone want to start "which Scala versions to support" thread on dev list? I am not at all against deprecated support for 2.10 if user base is ok with that -- I remember you describing a big bag of flaws that exist there....

Would also make sense to do it so that 2.12 support was added, 2.10 dropped. So keep supporting 2 major versions.
2.10 would still be supported with 2.7, and we can keep that branch nominally open longer (similar to how 2.3 is technically still open, although with little likelihood of new patches getting released).

[sksamuel]

I suspect most of the 2.10 usage was from people using Spark < 2.0 which defaulted to 2.10. Version 2 has been out for a few weeks now and defaults to 2.11

Also Scala 2.12 hit a couple of days ago. I think 2.10 is dead and buried at this point. I'm dropping it from all my projects for instance.

[umcodemonkey]

Given that 2.8 has already released, and 2.9 is imminent, moving 2.9 to Scala 2.11/2.12 seems prudent.

[sksamuel]

Would be ideal if it's possible, as no Jackson 2.12 will be a blocker on many projects upgrading to 2.12 I imagine (being the most used Json framework, at least in my experience).

[cowtowncoder]

Couple of notes:

• Scala 2.12 versions have been out for a bit, see: http://repo1.maven.org/maven2/com/fasterxml/jackson/module/jackson-module-scala_2.12.0-M5/
• Scala 2.10 downloads still exceed those of 2.11, as per Sonatype repo
• More Scala 2.10 dependencies via Maven: http://mvnrepository.com/search?q=jackson+scala+module

Now, just because existing usage is still high (numbers are similar between 2.10 and 2.11, former slightly higher) does not necessarily mean that we must keep supporting it with new Jackson versions. It is quite possible and likely that much/most of 2.10 usage is for legacy reasons, for which Jackson versions up to 2.8 would suffice as well. But I thought I'll add those data points just to give some context on general usage -- some developers are early adopters, others stick to known good (enough) versions.

[christophercurrie]

All good points. As we did with Scala 2.9, it makes sense to keep the Jackson 2.8 branch open for critical bugfixes for Scala 2.10. But I'll observe that 2.10 downloads appear to be in decline, and the majority of those downloads are for the Jackson 2.4 series. Demand for Jackson 2.8 on Scala 2.10 is non-existent by comparison. So now seems a fine time for a migration.

[cowtowncoder]

@christophercurrie That makes sense to me. 2.4.x seems to be used for Spark 1.6, for example. I have no objections for moving away from scala 2.10 for Jackson 2.9.

[crumbpicker]

👍

[pjfanning]

This issue should not occur in v2.11 and above