Add option to use local Nemotron instance

[taylorwass]

Hey mate, thought I'd catch you while you're a roll writing this Nemotron stuff. I was wondering if you could please add an option to the PDF parser settings for it to use a web address that we specify? This one is actually feasible to self-host, like the embeddings, so you can save a decent amount of resources doing it yourself. Also good if you have private data you would not like corporations getting high quality scans of!

Thanks for your work on this, had a play around with Nemotron last night and was immediately impressed/installed it in my server.

Cheers,
Taylor

P.S. if too busy I can do a pull request in 2-3 weeks, but not sure if the way I'd go about coding it would work with how the team wants to refactor the code for v6.

[jamesbraza]

Hi @taylorwass thanks for the discussion, happy to help.

add an option to the PDF parser settings for it to use a web address that we specify

What is behind the web address, is it a PDF to read or a hosted nemotron-parse?

As far as PDF to read, are you familiar with aadd_url: https://github.com/Future-House/paper-qa/blob/v5.29.1/src/paperqa/docs.py#L190

If you mean hosted nemotron-parse, we support either:

• OpenAI-like endpoints: https://github.com/Future-House/paper-qa/blob/A9c7d54cce6b0329be16be04e33cf1ee644a525aa/packages/paper-qa-nemotron/src/paperqa_nemotron/api.py#L281-L292
• AWS SageMaker endpoints:

  paper-qa/packages/paper-qa-nemotron/src/paperqa_nemotron/api.py

  Lines 434 to 442 in 9c7d54c

  	async with session.create_client(
  	"sagemaker-runtime", region_name=aws_region
  	) as client:
  	response = await client.invoke_endpoint(
  	EndpointName=endpoint_name,
  	ContentType="application/json",
  	Body=json.dumps(payload),
  	)
  	response_body = await response["Body"].read()

I believe actually what you want to do is already possible. Please let me know if I misinterpreted your post

[taylorwass]

Thanks mate, think I was searching in a wrong branch or something. Works perfectly :)