Feat: semi self-hosted LLMs using AzureML Online Endpoints

[janaka]

Description

Implements #10

Enables deploying an OSS LLM like Llama2 on AzureML + LlamaIndex level API implementation.

• IaC for infra using ARM templates
• AzureML Online Endpoints Web API client
• LlamaIndex LLM class for AzureML self-hosted models
• Test
• maybe some LLM class code folder re-organisation

Note:

• Docq application-level integration is not part of this PR and depends on CORE: Settings for model selection #12.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Refactor and code improvement (non-breaking change which improves code quality and/or performance)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation
• Tests
• Other chores such as maintenance

How Has This Been Tested?

Current not tested

• Test A
• Test B
• Test C

Test Configuration:

There two methods for setting up the Model backend for testing

• Spin up an Online Endpoint with Llama2-7b-chat on Azure
  • pro: this is a matter of running the ARM template via the shell script.
  • con: slow to spin up/down the infra 15-20mins. the infra is expensive ~$150 /day
• Spin up a local Online Endpoint with Llama2-7b-chat on Azure
  • pro: zero cloud bill cost.
  • con: speed of spin up is not known for sure but assume still slow. some work to figure out local Online Endpoints

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged and published in downstream modules
• I have checked my code and corrected any misspellings
• The commit message follows the convention of this project

[cwang]

Nice, a few comments:

• If any of the files are exported instead of drafted from scratch, would be nice to state how they are created in Azure console and how they are exported.
• Docs should probably go into the doc site instead of staying with the README file here.

[janaka]

Nice, a few comments:

• If any of the files are exported instead of drafted from scratch, would be nice to state how they are created in Azure console and how they are exported.

Agreed - though not expecting exported files. There are a bunch of files that aren't needed which needs to be cleaned up.

• Docs should probably go into the doc site instead of staying with the README file here.

Yep