Search image with Hybrid method

[thanhtung2693]

Hello,

My use case is I want to search ecommerce products with image and options. For example my query would be: https://example.com/product.jpg with Size S, but with the options (with Size S) will make Marqo download the wrong image URL or wrong result.

Here is my index settings:

{
    "type": "structured",
    "model": "marqo-fashion-siglip-custom-load",
    "modelProperties": {
        "name": "hf-hub:Marqo/marqo-fashionSigLIP",
        "dimensions": 768,
        "type": "open_clip",
    },
    "normalizeEmbeddings": True,
    "allFields": [
        {"name": "thumbnail", "type": "image_pointer"},
        {"name": "options", "type": "array<text>", "features": ["lexical_search"]},
        {"name": "name", "type": "text", "features": ["lexical_search"]},
        {
            "name": "multimodal_image_name",
            "type": "multimodal_combination",
            "dependentFields": {"thumbnail": 0.9, "name": 0.1}
        }
    ],
    "tensorFields": ["multimodal_image_name"],
}

and this is the query:

mq.index(INDEX_NAME).search(
    q="https://example.com/product.jpg with size M",
    search_method="HYBRID",
    limit=10,
)

Thanks.

[papa99do]

Hello @thanhtung2693 . Sorry for the late reply. Marqo supports passing a dictionary as a query. See the second example in this doc for details. Specifically, you can use this query to retrieve the docs (please test the weights to get a better recall on your dataset):

mq.index(INDEX_NAME).search(
    q={"https://example.com/product.jpg": 1.0, "size M": 0.5},
    search_method="HYBRID",
    limit=10,
)

Please let me know if this works for you. Thanks.

[aniruddhaadak80]

Hello @thanhtung2693,

Thanks for starting this discussion! When dealing with AI/LLM integrations, Vector DBs, or agent frameworks, quirks like this can usually be traced back to a few specific moving parts:

1. Environment & Dependency Versions: The AI tooling ecosystem evolves incredibly rapidly. Double-check your local environment (Node/Python versions) to ensure packages like langchain, transformers, or openai are on the latest stable releases. Breaking API changes upstream happen often.
2. Context Window & Embeddings: If you are seeing strange truncation, blank responses, or timeouts, make sure the payload you are sending fits well within the specific model???s maximum context window. Also verify your vector chunk sizes match the dimensionality of your chosen embedding model.
3. Rate Limits: If you are hitting external APIs, occasionally verify that your API key is correctly scoped and that you are not hitting concurrency or token rate limits which might silently fail or hang the process.

If you are still blocked, providing a minimal reproducible snippet or logging the raw request/response payload (scrubbed of secrets) usually helps pinpoint the exact failure layer much faster.

Hope this helps point you in the right direction. Let me know if you make any progress!