Skip to content

Commit 4b3218a

Browse files
ikxplainlucas-aixplainthiago-aixplainmikelam-us-aixplainThiago Castro Ferreira
authored
Merge to prod (#297)
* Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Merge dev to test (#107) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Development to Test (#109) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Lucas Pavanelli <[email protected]> * Merge to test (#111) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner functional tests (#112) * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#118) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Merge to test (#124) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message * Merge dev to test (#126) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Add new LLMs finetuner models (mistral and solar) (#128) * Merge dev to test (#129) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message * Add new LLMs finetuner models (mistral and solar) (#128) --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enabling dataset ID and model ID as parameters for finetuner creation (#131) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fix supplier representation of a model (#132) * Fix supplier representation of a model * Fixing parameter typing --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fixing indentation in documentation sample code (#134) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Merge to test (#135) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message * Add new LLMs finetuner models (mistral and solar) (#128) * Enabling dataset ID and model ID as parameters for finetuner creation (#131) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fix supplier representation of a model (#132) * Fix supplier representation of a model * Fixing parameter typing --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fixing indentation in documentation sample code (#134) Co-authored-by: Thiago Castro Ferreira <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update FineTune unit and functional tests (#136) * Merge dev to test (#137) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message * Add new LLMs finetuner models (mistral and solar) (#128) * Enabling dataset ID and model ID as parameters for finetuner creation (#131) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fix supplier representation of a model (#132) * Fix supplier representation of a model * Fixing parameter typing --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fixing indentation in documentation sample code (#134) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update FineTune unit and functional tests (#136) --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Click fix (#140) * Merge to prod (#119) * Merge dev to test (#107) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Development to Test (#109) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Lucas Pavanelli <[email protected]> * Merge to test (#111) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#118) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mik…
1 parent 36cf948 commit 4b3218a

29 files changed

+1037
-412
lines changed

aixplain/factories/api_key_factory.py

Lines changed: 26 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,20 @@
44
from datetime import datetime
55
from typing import Text, List, Optional, Dict, Union
66
from aixplain.utils.file_utils import _request_with_retry
7-
from aixplain.modules.api_key import APIKey, APIKeyGlobalLimits, APIKeyUsageLimit
7+
from aixplain.modules.api_key import APIKey, APIKeyLimits, APIKeyUsageLimit
88

99

1010
class APIKeyFactory:
1111
backend_url = config.BACKEND_URL
1212

13+
@classmethod
14+
def get(cls, api_key: Text) -> APIKey:
15+
"""Get an API key"""
16+
for api_key_obj in cls.list():
17+
if str(api_key_obj.access_key).startswith(api_key[:4]) and str(api_key_obj.access_key).endswith(api_key[-4:]):
18+
return api_key_obj
19+
raise Exception(f"API Key Error: API key {api_key} not found")
20+
1321
@classmethod
1422
def list(cls) -> List[APIKey]:
1523
"""List all API keys"""
@@ -30,7 +38,7 @@ def list(cls) -> List[APIKey]:
3038
name=key["name"],
3139
budget=key["budget"] if "budget" in key else None,
3240
global_limits=key["globalLimits"] if "globalLimits" in key else None,
33-
asset_limits=key["assetLimits"] if "assetLimits" in key else [],
41+
asset_limits=key["assetsLimits"] if "assetsLimits" in key else [],
3442
expires_at=key["expiresAt"] if "expiresAt" in key else None,
3543
access_key=key["accessKey"],
3644
is_admin=key["isAdmin"],
@@ -46,8 +54,8 @@ def create(
4654
cls,
4755
name: Text,
4856
budget: int,
49-
global_limits: Union[Dict, APIKeyGlobalLimits],
50-
asset_limits: List[Union[Dict, APIKeyGlobalLimits]],
57+
global_limits: Union[Dict, APIKeyLimits],
58+
asset_limits: List[Union[Dict, APIKeyLimits]],
5159
expires_at: datetime,
5260
) -> APIKey:
5361
"""Create a new API key"""
@@ -84,6 +92,7 @@ def create(
8492
@classmethod
8593
def update(cls, api_key: APIKey) -> APIKey:
8694
"""Update an existing API key"""
95+
api_key.validate()
8796
try:
8897
resp = "Unspecified error"
8998
url = f"{cls.backend_url}/sdk/api-keys/{api_key.id}"
@@ -112,12 +121,10 @@ def update(cls, api_key: APIKey) -> APIKey:
112121
raise Exception(f"API Key Update Error: Failed to update API key with ID {api_key.id}. Error: {str(resp)}")
113122

114123
@classmethod
115-
def get_usage_limit(cls, api_key: Text = config.TEAM_API_KEY, asset_id: Optional[Text] = None) -> APIKeyUsageLimit:
116-
"""Get API key usage limit"""
124+
def get_usage_limits(cls, api_key: Text = config.TEAM_API_KEY, asset_id: Optional[Text] = None) -> List[APIKeyUsageLimit]:
125+
"""Get API key usage limits"""
117126
try:
118127
url = f"{config.BACKEND_URL}/sdk/api-keys/usage-limits"
119-
if asset_id is not None:
120-
url += f"?assetId={asset_id}"
121128
headers = {"Authorization": f"Token {api_key}", "Content-Type": "application/json"}
122129
logging.info(f"Start service for GET API Key Usage - {url} - {headers}")
123130
r = _request_with_retry("GET", url, headers=headers)
@@ -128,11 +135,16 @@ def get_usage_limit(cls, api_key: Text = config.TEAM_API_KEY, asset_id: Optional
128135
raise Exception(f"{message}")
129136

130137
if 200 <= r.status_code < 300:
131-
return APIKeyUsageLimit(
132-
request_count=resp["requestCount"],
133-
request_count_limit=resp["requestCountLimit"],
134-
token_count=resp["tokenCount"],
135-
token_count_limit=resp["tokenCountLimit"],
136-
)
138+
return [
139+
APIKeyUsageLimit(
140+
daily_request_count=limit["requestCount"],
141+
daily_request_limit=limit["requestCountLimit"],
142+
daily_token_count=limit["tokenCount"],
143+
daily_token_limit=limit["tokenCountLimit"],
144+
model=limit["assetId"] if "assetId" in limit else None,
145+
)
146+
for limit in resp
147+
if asset_id is None or ("assetId" in limit and limit["assetId"] == asset_id)
148+
]
137149
else:
138150
raise Exception(f"API Key Usage Error: Failed to get usage. Error: {str(resp)}")

aixplain/factories/benchmark_factory.py

Lines changed: 37 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -22,10 +22,8 @@
2222
"""
2323

2424
import logging
25-
from typing import Dict, List, Optional, Text
25+
from typing import Dict, List, Text
2626
import json
27-
import pandas as pd
28-
from pathlib import Path
2927
from aixplain.enums.supplier import Supplier
3028
from aixplain.modules import Dataset, Metric, Model
3129
from aixplain.modules.benchmark_job import BenchmarkJob
@@ -34,9 +32,8 @@
3432
from aixplain.factories.dataset_factory import DatasetFactory
3533
from aixplain.factories.model_factory import ModelFactory
3634
from aixplain.utils import config
37-
from aixplain.utils.file_utils import _request_with_retry, save_file
35+
from aixplain.utils.file_utils import _request_with_retry
3836
from urllib.parse import urljoin
39-
from warnings import warn
4037

4138

4239
class BenchmarkFactory:
@@ -117,18 +114,25 @@ def get(cls, benchmark_id: str) -> Benchmark:
117114
logging.info(f"Start service for GET Benchmark - {url} - {headers}")
118115
r = _request_with_retry("get", url, headers=headers)
119116
resp = r.json()
120-
benchmark = cls._create_benchmark_from_response(resp)
117+
121118
except Exception as e:
122119
status_code = 400
123120
if resp is not None and "statusCode" in resp:
124121
status_code = resp["statusCode"]
125122
message = resp["message"]
126123
message = f"Benchmark Creation: Status {status_code} - {message}"
127124
else:
128-
message = f"Benchmark Creation: Unspecified Error"
125+
message = "Benchmark Creation: Unspecified Error"
129126
logging.error(f"Benchmark Creation Failed: {e}")
130127
raise Exception(f"Status {status_code}: {message}")
131-
return benchmark
128+
if 200 <= r.status_code < 300:
129+
benchmark = cls._create_benchmark_from_response(resp)
130+
logging.info(f"Benchmark {benchmark_id} retrieved successfully.")
131+
return benchmark
132+
else:
133+
error_message = f"Benchmark GET Error: Status {r.status_code} - {resp}"
134+
logging.error(error_message)
135+
raise Exception(error_message)
132136

133137
@classmethod
134138
def get_job(cls, job_id: Text) -> BenchmarkJob:
@@ -189,7 +193,7 @@ def create(cls, name: str, dataset_list: List[Dataset], model_list: List[Model],
189193
"""
190194
payload = {}
191195
try:
192-
url = urljoin(cls.backend_url, f"sdk/benchmarks")
196+
url = urljoin(cls.backend_url, "sdk/benchmarks")
193197
headers = {"Authorization": f"Token {config.TEAM_API_KEY}", "Content-Type": "application/json"}
194198
payload = {
195199
"name": name,
@@ -204,12 +208,19 @@ def create(cls, name: str, dataset_list: List[Dataset], model_list: List[Model],
204208
payload = json.dumps(clean_payload)
205209
r = _request_with_retry("post", url, headers=headers, data=payload)
206210
resp = r.json()
207-
logging.info(f"Creating Benchmark Job: Status for {name}: {resp}")
208-
return cls.get(resp["id"])
211+
209212
except Exception as e:
210213
error_message = f"Creating Benchmark Job: Error in Creating Benchmark with payload {payload} : {e}"
211214
logging.error(error_message, exc_info=True)
212-
return None
215+
raise Exception(error_message)
216+
217+
if 200 <= r.status_code < 300:
218+
logging.info(f"Benchmark {name} created successfully.")
219+
return cls.get(resp["id"])
220+
else:
221+
error_message = f"Benchmark Creation Error: Status {r.status_code} - {resp}"
222+
logging.error(error_message)
223+
raise Exception(error_message)
213224

214225
@classmethod
215226
def list_normalization_options(cls, metric: Metric, model: Model) -> List[str]:
@@ -223,21 +234,28 @@ def list_normalization_options(cls, metric: Metric, model: Model) -> List[str]:
223234
List[str]: List of supported normalization options
224235
"""
225236
try:
226-
url = urljoin(cls.backend_url, f"sdk/benchmarks/normalization-options")
237+
url = urljoin(cls.backend_url, "sdk/benchmarks/normalization-options")
227238
if cls.aixplain_key != "":
228239
headers = {"x-aixplain-key": f"{cls.aixplain_key}", "Content-Type": "application/json"}
229240
else:
230241
headers = {"Authorization": f"Token {config.TEAM_API_KEY}", "Content-Type": "application/json"}
231242
payload = json.dumps({"metricId": metric.id, "modelIds": [model.id]})
232243
r = _request_with_retry("post", url, headers=headers, data=payload)
233244
resp = r.json()
234-
logging.info(f"Listing Normalization Options: Status of listing options: {resp}")
235-
normalization_options = [item["value"] for item in resp]
236-
return normalization_options
245+
237246
except Exception as e:
238247
error_message = f"Listing Normalization Options: Error in getting Normalization Options: {e}"
239248
logging.error(error_message, exc_info=True)
240-
return []
249+
raise Exception(error_message)
250+
251+
if 200 <= r.status_code < 300:
252+
logging.info("Listing Normalization Options: ")
253+
normalization_options = [item["value"] for item in resp]
254+
return normalization_options
255+
else:
256+
error_message = f"Error listing normalization options: Status {r.status_code} - {resp}"
257+
logging.error(error_message)
258+
raise Exception(error_message)
241259

242260
@classmethod
243261
def get_benchmark_job_scores(cls, job_id):
@@ -255,7 +273,8 @@ def __get_model_name(model_id):
255273
if model.version is not None:
256274
name = f"{name}({model.version})"
257275
return name
276+
258277
benchmarkJob = cls.get_job(job_id)
259278
scores_df = benchmarkJob.get_scores()
260279
scores_df["Model"] = scores_df["Model"].apply(lambda x: __get_model_name(x))
261-
return scores_df
280+
return scores_df

aixplain/factories/corpus_factory.py

Lines changed: 56 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,6 @@
2121
Corpus Factory Class
2222
"""
2323

24-
import aixplain.utils.config as config
2524
import aixplain.processes.data_onboarding.onboard_functions as onboard_functions
2625
import json
2726
import logging
@@ -86,12 +85,12 @@ def __from_response(cls, response: Dict) -> Corpus:
8685

8786
try:
8887
license = License(response["license"]["typeId"])
89-
except:
88+
except Exception:
9089
license = None
9190

9291
try:
9392
length = int(response["segmentsCount"])
94-
except:
93+
except Exception:
9594
length = None
9695

9796
corpus = Corpus(
@@ -116,17 +115,27 @@ def get(cls, corpus_id: Text) -> Corpus:
116115
Returns:
117116
Corpus: Created 'Corpus' object
118117
"""
119-
url = urljoin(cls.backend_url, f"sdk/corpora/{corpus_id}/overview")
120-
if cls.aixplain_key != "":
121-
headers = {"x-aixplain-key": f"{cls.aixplain_key}", "Content-Type": "application/json"}
118+
try:
119+
url = urljoin(cls.backend_url, f"sdk/corpora/{corpus_id}/overview")
120+
if cls.aixplain_key != "":
121+
headers = {"x-aixplain-key": f"{cls.aixplain_key}", "Content-Type": "application/json"}
122+
else:
123+
headers = {"Authorization": f"Token {config.TEAM_API_KEY}", "Content-Type": "application/json"}
124+
logging.info(f"Start service for GET Corpus - {url} - {headers}")
125+
r = _request_with_retry("get", url, headers=headers)
126+
resp = r.json()
127+
128+
except Exception as e:
129+
error_message = f"Error retrieving Corpus {corpus_id}: {str(e)}"
130+
logging.error(error_message, exc_info=True)
131+
raise Exception(error_message)
132+
if 200 <= r.status_code < 300:
133+
logging.info(f"Corpus {corpus_id} retrieved successfully.")
134+
return cls.__from_response(resp)
122135
else:
123-
headers = {"Authorization": f"Token {config.TEAM_API_KEY}", "Content-Type": "application/json"}
124-
logging.info(f"Start service for GET Corpus - {url} - {headers}")
125-
r = _request_with_retry("get", url, headers=headers)
126-
resp = r.json()
127-
if "statusCode" in resp and resp["statusCode"] == 404:
128-
raise Exception(f"Corpus GET Error: Dataset {corpus_id} not found.")
129-
return cls.__from_response(resp)
136+
error_message = f"Corpus GET Error: Status {r.status_code} - {resp}"
137+
logging.error(error_message)
138+
raise Exception(error_message)
130139

131140
@classmethod
132141
def create_asset_from_id(cls, corpus_id: Text) -> Corpus:
@@ -168,7 +177,7 @@ def list(
168177
else:
169178
headers = {"Authorization": f"Token {config.TEAM_API_KEY}", "Content-Type": "application/json"}
170179

171-
assert 0 < page_size <= 100, f"Corpus List Error: Page size must be greater than 0 and not exceed 100."
180+
assert 0 < page_size <= 100, "Corpus List Error: Page size must be greater than 0 and not exceed 100."
172181
payload = {"pageSize": page_size, "pageNumber": page_number, "sort": [{"field": "createdAt", "dir": -1}]}
173182

174183
if query is not None:
@@ -188,26 +197,38 @@ def list(
188197
language = [language]
189198
payload["language"] = [lng.value["language"] for lng in language]
190199

191-
logging.info(f"Start service for POST List Corpus - {url} - {headers} - {json.dumps(payload)}")
192-
r = _request_with_retry("post", url, headers=headers, json=payload)
193-
resp = r.json()
194-
corpora, page_total, total = [], 0, 0
195-
if "results" in resp:
196-
results = resp["results"]
197-
page_total = resp["pageTotal"]
198-
total = resp["total"]
199-
logging.info(f"Response for POST List Corpus - Page Total: {page_total} / Total: {total}")
200-
for corpus in results:
201-
corpus_ = cls.__from_response(corpus)
202-
# add languages
203-
languages = []
204-
for lng in corpus["languages"]:
205-
if "dialect" not in lng:
206-
lng["dialect"] = ""
207-
languages.append(Language(lng))
208-
corpus_.kwargs["languages"] = languages
209-
corpora.append(corpus_)
210-
return {"results": corpora, "page_total": page_total, "page_number": page_number, "total": total}
200+
try:
201+
logging.info(f"Start service for POST List Corpus - {url} - {headers} - {json.dumps(payload)}")
202+
r = _request_with_retry("post", url, headers=headers, json=payload)
203+
resp = r.json()
204+
205+
except Exception as e:
206+
error_message = f"Error listing corpora: {str(e)}"
207+
logging.error(error_message, exc_info=True)
208+
raise Exception(error_message)
209+
210+
if 200 <= r.status_code < 300:
211+
corpora, page_total, total = [], 0, 0
212+
if "results" in resp:
213+
results = resp["results"]
214+
page_total = resp["pageTotal"]
215+
total = resp["total"]
216+
logging.info(f"Response for POST List Corpus - Page Total: {page_total} / Total: {total}")
217+
for corpus in results:
218+
corpus_ = cls.__from_response(corpus)
219+
# add languages
220+
languages = []
221+
for lng in corpus["languages"]:
222+
if "dialect" not in lng:
223+
lng["dialect"] = ""
224+
languages.append(Language(lng))
225+
corpus_.kwargs["languages"] = languages
226+
corpora.append(corpus_)
227+
return {"results": corpora, "page_total": page_total, "page_number": page_number, "total": total}
228+
else:
229+
error_message = f"Corpus List Error: Status {r.status_code} - {resp}"
230+
logging.error(error_message)
231+
raise Exception(error_message)
211232

212233
@classmethod
213234
def get_assets_from_page(
@@ -245,7 +266,7 @@ def create(
245266
functions: List[Function] = [],
246267
privacy: Privacy = Privacy.PRIVATE,
247268
error_handler: ErrorHandler = ErrorHandler.SKIP,
248-
api_key: Optional[Text] = None
269+
api_key: Optional[Text] = None,
249270
) -> Dict:
250271
"""Asynchronous call to Upload a corpus to the user's dashboard.
251272

0 commit comments

Comments
 (0)