Skip to content

Enhanced RAG Pipeline for Scientific/Research Workflows#20

Open
9ddpgd76dw-jpg wants to merge 1 commit into
aietal:masterfrom
9ddpgd76dw-jpg:scientific-rag-enhancements
Open

Enhanced RAG Pipeline for Scientific/Research Workflows#20
9ddpgd76dw-jpg wants to merge 1 commit into
aietal:masterfrom
9ddpgd76dw-jpg:scientific-rag-enhancements

Conversation

@9ddpgd76dw-jpg
Copy link
Copy Markdown

This PR implements an enhanced RAG Pipeline specifically optimized for scientific and research workflows.

Key Enhancements:

  • Scientific Document Splitter: Prioritizes splitting at key research sections (Abstract, Methods, Results, etc.) to maintain context.
  • Dynamic Section Detection: Automatically identifies the section of the paper during ingestion for better metadata enrichment.
  • Structured Citations: Generates stable, human-readable citation keys (e.g., Title:p5:c12) for every chunk.
  • Context Formatting: New formatter for LLM prompts that includes citation keys and relevance scores.
  • Rigorous Scientific Assistant: Updated chat logic to enforce citation-backed responses.

These changes ensure research context is preserved and every answer is verifiable via stable citations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant