Deserializing a document from a collection containing a HashMap with a newtype struct key doesn't work

[tyilo]

Versions/Environment

1. What version of Rust are you using? 1.86.0
2. What operating system are you using? Linux
3. What versions of the driver and its dependencies are you using? (Run
   cargo pkgid mongodb & cargo pkgid bson)

registry+https://github.com/rust-lang/crates.io-index#[email protected]
registry+https://github.com/rust-lang/crates.io-index#[email protected]

4. What version of MongoDB are you using? (Check with the MongoDB shell using db.version()) Not relevant
5. What is your MongoDB topology (standalone, replica set, sharded cluster, serverless)? Not relevant

Describe the bug

The following Foo struct can be serialized/deserialized to/from BSON using bson::to_document and bson::from_document. However, when using the mongodb crate it is not possible to deserialize such a value from a collection (serialization works).

#[derive(Debug, Deserialize, Serialize, Hash, PartialEq, Eq)]
struct NewType(String);

#[derive(Debug, Deserialize, Serialize, PartialEq, Eq)]
struct Foo {
    map: HashMap<NewType, u64>,
}

The following code demonstrates the problem:

#[cfg(test)]
mod test {
    use std::collections::HashMap;

    use mongodb::bson::{self, doc};
    use serde::{Deserialize, Serialize};

    async fn connect_to_database() -> mongodb::error::Result<mongodb::Database> {
        let client_options =
            mongodb::options::ClientOptions::parse(std::env::var("MONGODB_URL").unwrap()).await?;
        let default_database = client_options.default_database.as_ref().unwrap().clone();

        let client = mongodb::Client::with_options(client_options)?;
        Ok(client.database(&default_database))
    }

    #[derive(Debug, Deserialize, Serialize, Hash, PartialEq, Eq)]
    struct NewType(String);

    #[derive(Debug, Deserialize, Serialize, PartialEq, Eq)]
    struct Foo {
        map: HashMap<NewType, u64>,
    }

    #[test]
    fn to_from_bson() {
        let foo = Foo {
            map: [(NewType("abc".to_string()), 1)].into(),
        };
        let doc = bson::to_document(&foo).unwrap();
        let foo2: Foo = bson::from_document(doc).unwrap();
        assert_eq!(foo2, foo);
    }

    #[tokio::test]
    async fn to_from_bson_via_db() {
        let foo = Foo {
            map: [(NewType("abc".to_string()), 1)].into(),
        };

        let db = connect_to_database().await.unwrap();
        let coll = db.collection::<Foo>("_mongodb-serialization-test");

        coll.insert_one(&foo).await.unwrap();

        let foo2 = coll.find_one(doc! {}).await.unwrap().unwrap();

        assert_eq!(foo2, foo);
    }
}

It fails with:

running 2 tests
test test::to_from_bson ... ok
test test::to_from_bson_via_db ... FAILED

failures:

---- test::to_from_bson_via_db stdout ----

thread 'test::to_from_bson_via_db' panicked at src/lib.rs:46:49:
called `Result::unwrap()` on an `Err` value: Error { kind: BsonDeserialization(DeserializationError { message: "invalid type: string \"abc\", expected tuple struct NewType" }), labels: {}, wire_version: None, source: None }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    test::to_from_bson_via_db

test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.73s

error: test failed, to rerun pass `--lib`

To Reproduce
Steps to reproduce the behavior:

1. Set MONGODB_URL environment variable to a mongodb server URL.
2. cargo test

[isabelatkinson]

@tyilo thanks for this report! The discrepancy you're seeing is due to the way our raw BSON deserializer (which the driver uses under the hood) handles keys in maps. A simpler way to reproduce the issue is:

#[derive(Hash, PartialEq, Eq, Serialize, Deserialize)]
struct NewType(String);

fn main() {
    let map: HashMap<NewType, u64> = [(NewType("hi".to_string()), 5)].into();

    let document = bson::to_document(&map).unwrap();
    let map_from_document: HashMap<NewType, u64> = bson::from_document(document).unwrap(); // this succeeds

    let raw_document = bson::to_raw_document_buf(&map).unwrap();
    let map_from_raw_document: HashMap<NewType, u64> =
        bson::from_slice(raw_document.as_bytes()).unwrap(); // this panics when attempting to deserialize the string key
}

Version 2.12 of bson included a refactor of the raw deserializer which updated the logic of the deserializer to use a BorrowedStrDeserializer for map keys -- apparently this doesn't play nicely with newtypes. A temporary workaround is to pin your bson version to 2.11 or earlier. You can also add the #[serde(transparent)] annotation to your newtype, which will allow for proper deserialization from a borrowed string.

I filed RUST-2205 to fix this issue; feel free to follow along there for updates.

[tyilo]

Thanks for the explanation and the possible workarounds!