Skip to content

Commit f289f40

Browse files
ikxplainlucas-aixplainthiago-aixplainmikelam-us-aixplainThiago Castro Ferreira
authored
Release. (#141)
* Merge dev to test (#107) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Development to Test (#109) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Lucas Pavanelli <[email protected]> * Merge to test (#111) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) --------- Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#118) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Merge to test (#124) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Merge dev to test (#126) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Merge dev to test (#129) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message * Add new LLMs finetuner models (mistral and solar) (#128) --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Merge to test (#135) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message * Add new LLMs finetuner models (mistral and solar) (#128) * Enabling dataset ID and model ID as parameters for finetuner creation (#131) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fix supplier representation of a model (#132) * Fix supplier representation of a model * Fixing parameter typing --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fixing indentation in documentation sample code (#134) Co-authored-by: Thiago Castro Ferreira <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Merge dev to test (#137) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) * Hf deployment test (#115) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Hf deployment test (#117) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> * Merge dev to test (#113) * Create bounds for FineTune hyperparameters (#103) * Test bound to hyperparameters * Update finetune llm hyperparameters * Remove option to use PEFT, always on use now * Fixing pipeline general asset test (#106) * Update Finetuner functional tests (#112) --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Hf deployment test (#114) * Started adding Hugging Face deployment to aiXplain SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Added model status function to SDK Signed-off-by: mikelam-us-aixplain <[email protected]> * Updating Signed-off-by: mikelam-us-aixplain <[email protected]> * Updated CLI Signed-off-by: mikelam-us <[email protected]> * Adding CLI Signed-off-by: mikelam-us-aixplain <[email protected]> * Corrected request error Signed-off-by: mikelam-us-aixplain <[email protected]> * Clearing out unnecessary information in return Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding status Signed-off-by: mikelam-us-aixplain <[email protected]> * Simplifying status Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests and correcting tokens Signed-off-by: mikelam-us-aixplain <[email protected]> * Added bad repo ID test Signed-off-by: mikelam-us-aixplain <[email protected]> * Finished rough draft of tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding tests Signed-off-by: mikelam-us-aixplain <[email protected]> * Fixing hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding hf token Signed-off-by: mikelam-us-aixplain <[email protected]> * Correcting first test Signed-off-by: mikelam-us-aixplain <[email protected]> * Testing Signed-off-by: mikelam-us-aixplain <[email protected]> * Adding config Signed-off-by: mikelam-us-aixplain <[email protected]> * Added user doc Signed-off-by: mikelam-us-aixplain <[email protected]> * Added gated model test Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> * Adding HF token Signed-off-by: mikelam-us-aixplain <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> * Do not download textual URLs (#120) * Do not download textual URLs * Treat as string --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Enable api key parameter in data asset creation (#122) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update Finetuner hyperparameters (#125) * Update Finetuner hyperparameters * Change hyperparameters error message * Add new LLMs finetuner models (mistral and solar) (#128) * Enabling dataset ID and model ID as parameters for finetuner creation (#131) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fix supplier representation of a model (#132) * Fix supplier representation of a model * Fixing parameter typing --------- Co-authored-by: Thiago Castro Ferreira <[email protected]> * Fixing indentation in documentation sample code (#134) Co-authored-by: Thiago Castro Ferreira <[email protected]> * Update FineTune unit and functional tests (#136) --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> --------- Signed-off-by: mikelam-us-aixplain <[email protected]> Signed-off-by: mikelam-us <[email protected]> Co-authored-by: Lucas Pavanelli <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: mikelam-us-aixplain <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]> Co-authored-by: Thiago Castro Ferreira <[email protected]>
1 parent edce6ab commit f289f40

File tree

11 files changed

+86
-39
lines changed

11 files changed

+86
-39
lines changed

aixplain/factories/finetune_factory/__init__.py

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,9 +22,11 @@
2222
"""
2323

2424
import logging
25-
from typing import Dict, List, Optional, Text
25+
from typing import Dict, List, Optional, Text, Union
2626
import json
27+
from aixplain.factories.dataset_factory import DatasetFactory
2728
from aixplain.factories.finetune_factory.prompt_validator import validate_prompt
29+
from aixplain.factories.model_factory import ModelFactory
2830
from aixplain.modules.finetune import Finetune
2931
from aixplain.modules.finetune.cost import FinetuneCost
3032
from aixplain.modules.finetune.hyperparameters import Hyperparameters
@@ -61,8 +63,8 @@ def _create_cost_from_response(cls, response: Dict) -> FinetuneCost:
6163
def create(
6264
cls,
6365
name: Text,
64-
dataset_list: List[Dataset],
65-
model: Model,
66+
dataset_list: List[Union[Dataset, Text]],
67+
model: Union[Model, Text],
6668
prompt_template: Optional[Text] = None,
6769
hyperparameters: Optional[Hyperparameters] = None,
6870
train_percentage: Optional[float] = 100,
@@ -72,8 +74,8 @@ def create(
7274
7375
Args:
7476
name (Text): Name of the Finetune.
75-
dataset_list (List[Dataset]): List of Datasets to be used for fine-tuning.
76-
model (Model): Model to be fine-tuned.
77+
dataset_list (List[Dataset]): List of Datasets (or dataset IDs) to be used for fine-tuning.
78+
model (Model): Model (Model ID) to be fine-tuned.
7779
prompt_template (Text, optional): Fine-tuning prompt_template. Should reference columns in the dataset using format <<COLUMN_NAME>>. Defaults to None.
7880
hyperparameters (Hyperparameters, optional): Hyperparameters for fine-tuning. Defaults to None.
7981
train_percentage (float, optional): Percentage of training samples. Defaults to 100.
@@ -86,6 +88,14 @@ def create(
8688
assert (
8789
train_percentage + dev_percentage <= 100
8890
), f"Create FineTune: Train percentage + dev percentage ({train_percentage + dev_percentage}) must be less than or equal to one"
91+
92+
for i, dataset in enumerate(dataset_list):
93+
if isinstance(dataset, str) is True:
94+
dataset_list[i] = DatasetFactory.get(dataset_id=dataset)
95+
96+
if isinstance(model, str) is True:
97+
model = ModelFactory.get(model_id=model)
98+
8999
if prompt_template is not None:
90100
prompt_template = validate_prompt(prompt_template, dataset_list)
91101
try:

aixplain/modules/asset.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
from aixplain.enums.license import License
2424
from aixplain.enums.supplier import Supplier
2525
from aixplain.enums.privacy import Privacy
26-
from typing import Dict, Optional, Text, Tuple
26+
from typing import Dict, Optional, Text, Union
2727

2828

2929
class Asset:
@@ -32,7 +32,7 @@ def __init__(
3232
id: Text,
3333
name: Text,
3434
description: Text,
35-
supplier: Tuple[Dict, Text, Supplier, int] = "aiXplain",
35+
supplier: Union[Dict, Text, Supplier, int] = "aiXplain",
3636
version: Text = "1.0",
3737
license: Optional[License] = None,
3838
privacy: Privacy = Privacy.PRIVATE,
@@ -44,7 +44,7 @@ def __init__(
4444
id (Text): ID of the Asset
4545
name (Text): Name of the Asset
4646
description (Text): Description of the Asset
47-
supplier (Tuple[Text, Supplier, int], optional): supplier of the asset. Defaults to "aiXplain".
47+
supplier (Union[Dict, Text, Supplier, int], optional): supplier of the asset. Defaults to "aiXplain".
4848
version (Optional[Text], optional): asset version. Defaults to "1.0".
4949
"""
5050
self.id = id

aixplain/modules/corpus.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,9 @@ def __init__(
8282
self.length = length
8383
self.kwargs = kwargs
8484

85+
def __repr__(self):
86+
return f"<Corpus: {self.name}>"
87+
8588
def delete(self) -> None:
8689
"""Delete Corpus service"""
8790
try:

aixplain/modules/finetune/hyperparameters.py

Lines changed: 1 addition & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -16,9 +16,8 @@ class SchedulerType(Text, Enum):
1616

1717

1818
EPOCHS_MAX_VALUE = 4
19-
BATCH_SIZE_VALUES = [1, 2, 4, 8, 16, 32, 64, 128]
19+
BATCH_SIZE_VALUES = [1, 2, 4, 8, 16, 32, 64]
2020
MAX_SEQ_LENGTH_MAX_VALUE = 4096
21-
GENERATION_MAX_LENGTH_MAX_VALUE = 225
2221

2322

2423
@dataclass_json
@@ -28,7 +27,6 @@ class Hyperparameters(object):
2827
train_batch_size: int = 4
2928
eval_batch_size: int = 4
3029
learning_rate: float = 1e-5
31-
generation_max_length: int = 225
3230
max_seq_length: int = 4096
3331
warmup_ratio: float = 0.0
3432
warmup_steps: int = 0
@@ -47,9 +45,6 @@ def __post_init__(self):
4745
if not isinstance(self.learning_rate, float):
4846
raise TypeError("learning_rate should be of type float")
4947

50-
if not isinstance(self.generation_max_length, int):
51-
raise TypeError("generation_max_length should be of type int")
52-
5348
if not isinstance(self.max_seq_length, int):
5449
raise TypeError("max_seq_length should be of type int")
5550

@@ -73,6 +68,3 @@ def __post_init__(self):
7368

7469
if self.max_seq_length > MAX_SEQ_LENGTH_MAX_VALUE:
7570
raise ValueError(f"max_seq_length must be less or equal to {MAX_SEQ_LENGTH_MAX_VALUE}")
76-
77-
if self.generation_max_length > GENERATION_MAX_LENGTH_MAX_VALUE:
78-
raise ValueError(f"generation_max_length must be less or equal to {GENERATION_MAX_LENGTH_MAX_VALUE}")

aixplain/modules/model.py

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@
2727
import traceback
2828
from typing import List
2929
from aixplain.factories.file_factory import FileFactory
30-
from aixplain.enums import Function
30+
from aixplain.enums import Function, Supplier
3131
from aixplain.modules.asset import Asset
3232
from aixplain.utils import config
3333
from urllib.parse import urljoin
@@ -44,7 +44,7 @@ class Model(Asset):
4444
description (Text, optional): description of the model. Defaults to "".
4545
api_key (Text, optional): API key of the Model. Defaults to None.
4646
url (Text, optional): endpoint of the model. Defaults to config.MODELS_RUN_URL.
47-
supplier (Text, optional): model supplier. Defaults to "aiXplain".
47+
supplier (Union[Dict, Text, Supplier, int], optional): supplier of the asset. Defaults to "aiXplain".
4848
version (Text, optional): version of the model. Defaults to "1.0".
4949
function (Text, optional): model AI function. Defaults to None.
5050
url (str): URL to run the model.
@@ -58,7 +58,7 @@ def __init__(
5858
name: Text,
5959
description: Text = "",
6060
api_key: Optional[Text] = None,
61-
supplier: Text = "aiXplain",
61+
supplier: Union[Dict, Text, Supplier, int] = "aiXplain",
6262
version: Optional[Text] = None,
6363
function: Optional[Text] = None,
6464
is_subscribed: bool = False,
@@ -71,7 +71,7 @@ def __init__(
7171
name (Text): Name of the Model
7272
description (Text, optional): description of the model. Defaults to "".
7373
api_key (Text, optional): API key of the Model. Defaults to None.
74-
supplier (Text, optional): model supplier. Defaults to "aiXplain".
74+
supplier (Union[Dict, Text, Supplier, int], optional): supplier of the asset. Defaults to "aiXplain".
7575
version (Text, optional): version of the model. Defaults to "1.0".
7676
function (Text, optional): model AI function. Defaults to None.
7777
is_subscribed (bool, optional): Is the user subscribed. Defaults to False.
@@ -95,7 +95,10 @@ def to_dict(self) -> Dict:
9595
return {"id": self.id, "name": self.name, "supplier": self.supplier, "additional_info": clean_additional_info}
9696

9797
def __repr__(self):
98-
return f"<Model: {self.name} by {self.supplier}>"
98+
try:
99+
return f"<Model: {self.name} by {self.supplier['name']}>"
100+
except Exception:
101+
return f"<Model: {self.name} by {self.supplier}>"
99102

100103
def __polling(self, poll_url: Text, name: Text = "model_process", wait_time: float = 0.5, timeout: float = 300) -> Dict:
101104
"""Keeps polling the platform to check whether an asynchronous call is done.

docs/user/user_doc.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -219,11 +219,17 @@ You can also process an aiXplain data asset, being a Corpus or a Dataset, using
219219

220220
```python
221221
# Run Synchronously
222-
result = pipeline.run(data="64acbad666608858f693a3a0", data_asset="64acbad666608858f693a39f")
222+
result = pipeline.run(
223+
data_asset="64acbad666608858f693a39f",
224+
data="64acbad666608858f693a3a0"
225+
)
223226

224227
# Run Asynchronously
225228
## Start async job
226-
start_response = pipeline.run_async(data="64acbad666608858f693a3a0", data_asset="64acbad666608858f693a39f")
229+
start_response = pipeline.run_async(
230+
data_asset="64acbad666608858f693a39f",
231+
data="64acbad666608858f693a3a0"
232+
)
227233
poll_url = start_response["url"]
228234
## Poll to see current job status
229235
poll_response = pipeline.poll(poll_url)
Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,12 @@
11
[
2-
{"model_name": "llama 2 7b", "model_id": "6543cb991f695e72028e9428", "dataset_name": "Test text generation dataset"},
2+
{"model_name": "Llama 2 7b", "model_id": "6543cb991f695e72028e9428", "dataset_name": "Test text generation dataset"},
33
{"model_name": "Llama 2 7B Chat", "model_id": "65519ee7bf42e6037ab109d8", "dataset_name": "Test text generation dataset"},
4-
{"model_name": "mistral 7b", "model_id": "6551a9e7bf42e6037ab109de", "dataset_name": "Test text generation dataset"},
5-
{"model_name": "solar 10b", "model_id": "65b7baac1d5ea75105c14971", "dataset_name": "Test text generation dataset"}
4+
{"model_name": "Mistral 7b", "model_id": "6551a9e7bf42e6037ab109de", "dataset_name": "Test text generation dataset"},
5+
{"model_name": "Solar 10b", "model_id": "65b7baac1d5ea75105c14971", "dataset_name": "Test text generation dataset"},
6+
{"model_name": "Falcon 7b", "model_id": "6551bff9bf42e6037ab109e1", "dataset_name": "Test text generation dataset"},
7+
{"model_name": "Falcon 7b Instruct", "model_id": "65519d57bf42e6037ab109d5", "dataset_name": "Test text generation dataset"},
8+
{"model_name": "MPT 7b", "model_id": "6551a72bbf42e6037ab109d9", "dataset_name": "Test text generation dataset"},
9+
{"model_name": "MPT 7b storywriter", "model_id": "6551a870bf42e6037ab109db", "dataset_name": "Test text generation dataset"},
10+
{"model_name": "BloomZ 7b", "model_id": "6551ab17bf42e6037ab109e0", "dataset_name": "Test text generation dataset"},
11+
{"model_name": "BloomZ 7b MT", "model_id": "656e80147ca71e334752d5a3", "dataset_name": "Test text generation dataset"}
612
]

tests/functional/finetune/data/finetune_test_end2end.json

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,23 @@
44
"model_id": "6543cb991f695e72028e9428",
55
"dataset_name": "Test text generation dataset",
66
"inference_data": "Hello!",
7-
"required_dev": true
7+
"required_dev": true,
8+
"search_metadata": false
89
},
910
{
1011
"model_name": "aiR",
1112
"model_id": "6499cc946eb5633de15d82a1",
12-
"dataset_name": "Test search dataset",
13+
"dataset_name": "Test search dataset metadata",
1314
"inference_data": "Hello!",
14-
"required_dev": false
15+
"required_dev": false,
16+
"search_metadata": true
17+
},
18+
{
19+
"model_name": "vectara",
20+
"model_id": "655e20f46eb563062a1aa301",
21+
"dataset_name": "Test search dataset metadata",
22+
"inference_data": "Hello!",
23+
"required_dev": false,
24+
"search_metadata": true
1525
}
1626
]

tests/functional/finetune/finetune_functional_test.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ def validate_prompt_input_map(request):
6060
return request.param
6161

6262

63-
def test_end2end_text_generation(run_input_map):
63+
def test_end2end(run_input_map):
6464
model = ModelFactory.get(run_input_map["model_id"])
6565
dataset_list = [DatasetFactory.list(query=run_input_map["dataset_name"])["results"][0]]
6666
train_percentage, dev_percentage = 100, 0
@@ -84,7 +84,13 @@ def test_end2end_text_generation(run_input_map):
8484
end = time.time()
8585
assert finetune_model.check_finetune_status() == "onboarded"
8686
result = finetune_model.run(run_input_map["inference_data"])
87+
print(f"Result: {result}")
8788
assert result is not None
89+
if run_input_map["search_metadata"]:
90+
assert "details" in result
91+
assert len(result["details"]) > 0
92+
assert "metadata" in result["details"][0]
93+
assert len(result["details"][0]["metadata"]) > 0
8894
finetune_model.delete()
8995

9096

tests/unit/finetune_test.py

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,11 +21,13 @@
2121
from dotenv import load_dotenv
2222

2323
load_dotenv()
24+
import requests
2425
import requests_mock
2526
from aixplain.utils import config
2627
from aixplain.factories import ModelFactory
2728
from aixplain.factories import FinetuneFactory
2829
from aixplain.modules import Model, Finetune
30+
from aixplain.modules.finetune import Hyperparameters
2931
from aixplain.enums import Function
3032

3133
import pytest
@@ -51,14 +53,28 @@ def percentage_exception_map(request):
5153

5254

5355
def test_create():
56+
model_map = read_data(MODEL_FILE)
5457
with requests_mock.Mocker() as mock:
58+
test_model = "test_asset_id"
59+
url = f"{MODEL_URL}/{test_model}"
60+
mock.get(url, headers=FIXED_HEADER, json=model_map)
5561
cost_estimation_map = read_data(COST_ESTIMATION_FILE)
5662
mock.post(COST_ESTIMATION_URL, headers=FIXED_HEADER, json=cost_estimation_map)
57-
test_model = Model("", "")
58-
finetune = FinetuneFactory.create("", [], test_model)
63+
finetune = FinetuneFactory.create("", [], test_model, prompt_template="test", hyperparameters=Hyperparameters())
5964
assert finetune is not None
65+
assert finetune.model.id == test_model
6066
assert finetune.cost.to_dict() == cost_estimation_map
6167

68+
def test_create_exception():
69+
model_map = read_data(MODEL_FILE)
70+
with requests_mock.Mocker() as mock:
71+
test_model = "test_asset_id"
72+
url = f"{MODEL_URL}/{test_model}"
73+
mock.get(url, headers=FIXED_HEADER, json=model_map)
74+
mock.post(COST_ESTIMATION_URL, exc=requests.exceptions.ConnectTimeout)
75+
finetune = FinetuneFactory.create("", [], test_model, prompt_template="test", hyperparameters=Hyperparameters())
76+
assert finetune is None
77+
6278

6379
def test_create_train_dev_percentage(percentage_exception_map):
6480
with requests_mock.Mocker() as mock:

0 commit comments

Comments
 (0)