Skip to content

Commit f39ad5a

Browse files
ikxplainlucas-aixplainthiago-aixplainmikelam-us-aixplainThiago Castro Ferreira
authored
Merge to prod (#186)
* Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Merge dev to test (#107) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Development to Test (#109) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Lucas Pavanelli <[email protected]> * Merge to test (#111) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner functional tests (#112) * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#118) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Merge to test (#124) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message * Merge dev to test (#126) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Add new LLMs finetuner models (mistral and solar) (#128) * Merge dev to test (#129) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message * Add new LLMs finetuner models (mistral and solar) (#128) --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enabling dataset ID and model ID as parameters for finetuner creation (#131) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fix supplier representation of a model (#132) * Fix supplier representation of a model * Fixing parameter typing --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fixing indentation in documentation sample code (#134) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Merge to test (#135) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message * Add new LLMs finetuner models (mistral and solar) (#128) * Enabling dataset ID and model ID as parameters for finetuner creation (#131) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fix supplier representation of a model (#132) * Fix supplier representation of a model * Fixing parameter typing --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fixing indentation in documentation sample code (#134) Co-authored-by: Thiago Castro Ferreira <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update FineTune unit and functional tests (#136) * Merge dev to test (#137) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message * Add new LLMs finetuner models (mistral and solar) (#128) * Enabling dataset ID and model ID as parameters for finetuner creation (#131) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fix supplier representation of a model (#132) * Fix supplier representation of a model * Fixing parameter typing --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fixing indentation in documentation sample code (#134) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update FineTune unit and functional tests (#136) --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Click fix (#140) * Merge to prod (#119) * Merge dev to test (#107) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Development to Test (#109) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Lucas Pavanelli <[email protected]> * Merge to test (#111) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#118) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mik…
1 parent 2be27be commit f39ad5a

File tree

6 files changed

+39
-25
lines changed

6 files changed

+39
-25
lines changed

aixplain/factories/model_factory.py

Lines changed: 10 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ def _create_model_from_response(cls, response: Dict) -> Model:
6565
response["name"],
6666
supplier=response["supplier"],
6767
api_key=response["api_key"],
68-
pricing=response["pricing"],
68+
cost=response["pricing"],
6969
function=Function(response["function"]["id"]),
7070
parameters=parameters,
7171
is_subscribed=True if "subscription" in response else False,
@@ -404,9 +404,11 @@ def onboard_model(cls, model_id: Text, image_tag: Text, image_hash: Text, api_ke
404404
message = "Your onboarding request has been submitted to an aiXplain specialist for finalization. We will notify you when the process is completed."
405405
logging.info(message)
406406
return response
407-
407+
408408
@classmethod
409-
def deploy_huggingface_model(cls, name: Text, hf_repo_id: Text, hf_token: Optional[Text] = "", api_key: Optional[Text] = None) -> Dict:
409+
def deploy_huggingface_model(
410+
cls, name: Text, hf_repo_id: Text, hf_token: Optional[Text] = "", api_key: Optional[Text] = None
411+
) -> Dict:
410412
"""Onboards and deploys a Hugging Face large language model.
411413
412414
Args:
@@ -433,20 +435,16 @@ def deploy_huggingface_model(cls, name: Text, hf_repo_id: Text, hf_token: Option
433435
"sourceLanguage": "en",
434436
},
435437
"source": "huggingface",
436-
"onboardingParams": {
437-
"hf_model_name": model_name,
438-
"hf_supplier": supplier,
439-
"hf_token": hf_token
440-
}
438+
"onboardingParams": {"hf_model_name": model_name, "hf_supplier": supplier, "hf_token": hf_token},
441439
}
442440
response = _request_with_retry("post", deploy_url, headers=headers, json=body)
443441
logging.debug(response.text)
444442
response_dicts = json.loads(response.text)
445443
return response_dicts
446-
444+
447445
@classmethod
448446
def get_huggingface_model_status(cls, model_id: Text, api_key: Optional[Text] = None):
449-
"""Gets the on-boarding status of a Hugging Face model with ID MODEL_ID.
447+
"""Gets the on-boarding status of a Hugging Face model with ID MODEL_ID.
450448
451449
Args:
452450
model_id (Text): The model's ID as returned by DEPLOY_HUGGINGFACE_MODEL
@@ -466,6 +464,6 @@ def get_huggingface_model_status(cls, model_id: Text, api_key: Optional[Text] =
466464
"status": response_dicts["status"],
467465
"name": response_dicts["name"],
468466
"id": response_dicts["id"],
469-
"pricing": response_dicts["pricing"]
467+
"pricing": response_dicts["pricing"],
470468
}
471-
return ret_dict
469+
return ret_dict

aixplain/modules/asset.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ def __init__(
3636
version: Text = "1.0",
3737
license: Optional[License] = None,
3838
privacy: Privacy = Privacy.PRIVATE,
39-
cost: float = 0,
39+
cost: Optional[Union[Dict, float]] = None,
4040
) -> None:
4141
"""Create an Asset with the necessary information
4242
@@ -46,6 +46,7 @@ def __init__(
4646
description (Text): Description of the Asset
4747
supplier (Union[Dict, Text, Supplier, int], optional): supplier of the asset. Defaults to "aiXplain".
4848
version (Optional[Text], optional): asset version. Defaults to "1.0".
49+
cost (Optional[Union[Dict, float]], optional): asset price. Defaults to None.
4950
"""
5051
self.id = id
5152
self.name = name

aixplain/modules/metric.py

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -61,12 +61,10 @@ def __init__(
6161
supplier (Text): author of the Metric
6262
is_reference_required (bool): does the metric use reference
6363
is_source_required (bool): does the metric use source
64-
cost (float): cost of the metric
64+
cost (float): price of the metric
6565
normalization_options(list, [])
6666
**additional_info: Any additional Metric info to be saved
6767
"""
68-
69-
7068
super().__init__(id, name, description="", supplier=supplier, version="1.0", cost=cost)
7169
self.is_source_required = is_source_required
7270
self.is_reference_required = is_reference_required
@@ -76,7 +74,7 @@ def __init__(
7674

7775
def __repr__(self) -> str:
7876
return f"<Metric {self.name}>"
79-
77+
8078
def add_normalization_options(self, normalization_options: List[str]):
8179
"""Add a given set of normalization options to be used while benchmarking
8280
@@ -85,7 +83,12 @@ def add_normalization_options(self, normalization_options: List[str]):
8583
"""
8684
self.normalization_options.append(normalization_options)
8785

88-
def run(self, hypothesis: Optional[Union[str, List[str]]]=None, source: Optional[Union[str, List[str]]]=None, reference: Optional[Union[str, List[str]]]=None):
86+
def run(
87+
self,
88+
hypothesis: Optional[Union[str, List[str]]] = None,
89+
source: Optional[Union[str, List[str]]] = None,
90+
reference: Optional[Union[str, List[str]]] = None,
91+
):
8992
"""Run the metric to calculate the scores.
9093
9194
Args:
@@ -94,6 +97,7 @@ def run(self, hypothesis: Optional[Union[str, List[str]]]=None, source: Optional
9497
reference (Optional[Union[str, List[str]]], optional): Can give a single reference or a list of references for metric calculation. Defaults to None.
9598
"""
9699
from aixplain.factories.model_factory import ModelFactory
100+
97101
model = ModelFactory.get(self.id)
98102
payload = {
99103
"function": self.function,
@@ -115,4 +119,3 @@ def run(self, hypothesis: Optional[Union[str, List[str]]]=None, source: Optional
115119
reference = [[ref] for ref in reference]
116120
payload["references"] = reference
117121
return model.run(payload)
118-

aixplain/modules/model.py

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ class Model(Asset):
4848
function (Text, optional): model AI function. Defaults to None.
4949
url (str): URL to run the model.
5050
backend_url (str): URL of the backend.
51+
pricing (Dict, optional): model price. Defaults to None.
5152
**additional_info: Any additional Model info to be saved
5253
"""
5354

@@ -61,6 +62,7 @@ def __init__(
6162
version: Optional[Text] = None,
6263
function: Optional[Text] = None,
6364
is_subscribed: bool = False,
65+
cost: Optional[Dict] = None,
6466
**additional_info,
6567
) -> None:
6668
"""Model Init
@@ -74,9 +76,10 @@ def __init__(
7476
version (Text, optional): version of the model. Defaults to "1.0".
7577
function (Text, optional): model AI function. Defaults to None.
7678
is_subscribed (bool, optional): Is the user subscribed. Defaults to False.
79+
cost (Dict, optional): model price. Defaults to None.
7780
**additional_info: Any additional Model info to be saved
7881
"""
79-
super().__init__(id, name, description, supplier, version)
82+
super().__init__(id, name, description, supplier, version, cost=cost)
8083
self.api_key = api_key
8184
self.additional_info = additional_info
8285
self.url = config.MODELS_RUN_URL
@@ -264,6 +267,7 @@ def check_finetune_status(self, after_epoch: Optional[int] = None):
264267
"""
265268
from aixplain.enums.asset_status import AssetStatus
266269
from aixplain.modules.finetune.status import FinetuneStatus
270+
267271
headers = {"x-api-key": self.api_key, "Content-Type": "application/json"}
268272
resp = None
269273
try:
@@ -274,15 +278,15 @@ def check_finetune_status(self, after_epoch: Optional[int] = None):
274278
finetune_status = AssetStatus(resp["finetuneStatus"])
275279
model_status = AssetStatus(resp["modelStatus"])
276280
logs = sorted(resp["logs"], key=lambda x: float(x["epoch"]))
277-
281+
278282
target_epoch = None
279283
if after_epoch is not None:
280284
logs = [log for log in logs if float(log["epoch"]) > after_epoch]
281285
if len(logs) > 0:
282286
target_epoch = float(logs[0]["epoch"])
283287
elif len(logs) > 0:
284288
target_epoch = float(logs[-1]["epoch"])
285-
289+
286290
if target_epoch is not None:
287291
log = None
288292
for log_ in logs:
@@ -294,7 +298,7 @@ def check_finetune_status(self, after_epoch: Optional[int] = None):
294298
log["trainLoss"] = log_["trainLoss"]
295299
if log_["evalLoss"] is not None:
296300
log["evalLoss"] = log_["evalLoss"]
297-
301+
298302
status = FinetuneStatus(
299303
status=finetune_status,
300304
model_status=model_status,

aixplain/processes/data_onboarding/process_media_files.py

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
import pandas as pd
66
import shutil
77
import tarfile
8+
import validators
89

910
from aixplain.enums.data_subtype import DataSubtype
1011
from aixplain.enums.data_type import DataType
@@ -115,6 +116,13 @@ def run(metadata: MetaData, paths: List, folder: Path, batch_size: int = 100) ->
115116
shutil.copy2(media_path, new_path)
116117
batch.append(fname)
117118
else:
119+
if metadata.storage_type == StorageType.TEXT and (
120+
str(media_path).startswith("s3://")
121+
or str(media_path).startswith("http://")
122+
or str(media_path).startswith("https://")
123+
or validators.url(media_path)
124+
):
125+
media_path = "DONOTDOWNLOAD" + str(media_path)
118126
batch.append(media_path)
119127

120128
# crop intervals can not be used with interval data types

tests/functional/general_assets/asset_functional_test.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -82,8 +82,8 @@ def test_model_sort():
8282
prev_model = models[idx - 1]
8383
model = models[idx]
8484

85-
prev_model_price = prev_model.additional_info["pricing"]["price"]
86-
model_price = model.additional_info["pricing"]["price"]
85+
prev_model_price = prev_model.cost["price"]
86+
model_price = model.cost["price"]
8787
assert prev_model_price >= model_price
8888

8989

0 commit comments

Comments
 (0)