diff --git a/Experiments/kaggle/llm_prompt_recovery/README.md b/Experiments/kaggle/llm_prompt_recovery/README.md new file mode 100644 index 00000000..7b31a4b2 --- /dev/null +++ b/Experiments/kaggle/llm_prompt_recovery/README.md @@ -0,0 +1,259 @@ +# LLM Prompt Recovery + +## Overview + +[competition overview](https://www.kaggle.com/competitions/llm-prompt-recovery/overview) + +LLMs are commonly used to rewrite or make stylistic changes to text. +The goal of this competition is to recover the LLM prompt that was used to transform a given text. + +### Description + +``` +The challenge: recover the LLM prompt used to rewrite a given text. +You’ll be tested against a dataset of 1300+ original texts, each paired with a rewritten version from Gemma, Google’s new family of open models. +``` + +### Evaluation Metric + +For each row in the submission and corresponding ground truth, sentence-t5-base is used to calculate corresponding embedding vectors. +The score for each predicted / expected pair is calculated using the Sharpened Cosine Similarity, using an exponent of 3. +The SCS is used to attenuate the generous score given by embedding vectors for incorrect answers. +Do not leave any rewrite_prompt blank as null answers will throw an error. + +## Things that worked for me + +[my notebook](./code/using-mixtral-to-predict-prompt.ipynb) + +- `bitandbyte` quatization to reduce the model size +- use `mixtral-8x7B` model +- prompt engineering to make the model more performant + +### Prompt Engineering + +- All responses are in the format of "Improve this text by [adding a magician]." +- Specifically I seeded the prompt with "Improve this text by" + - since without the "by" Mistral would sometimes just respond "Improve this text." +- A bunch of prompt sequence examples encouraging Mistral to give ultra-short answers + +## What I learned from this competition + +- If the metric is calculated using embeddings, it is very important to understand the embeddings of the model + - `` token is a special token that can be used to improve the score + - tensorflow based tokenizer has protection against special token injection + - `` is not directly tokenized as the correct eos token + - `lucrarea` is a special token that can be used to improve the score + +## 1st Place Solution + +[writeup](https://www.kaggle.com/competitions/llm-prompt-recovery/discussion/494343) [notebook](https://www.kaggle.com/code/suicaokhoailang/1st-place-0-71) + +### What they have done + +- It's not about `lucrarea`, but the `` token. In the huggingface version of sentence-t5, when comparing two vaguely similar sentences, if you append an extra `` to one of them, the cosine similarity will be pulled toward ~0.9 i.e. if the original is lower, it goes up, if higher it goes down. +- 0.9 translates to about 0.73 of competition metric, now you're seeing where this is going. +- However, the tensorflow version, which the host used to compute the score, uses the default config of sentencepiece, which means it will tokenize special tokens as literal, `` becomes `['<', '/', 's', '>']` +- Here comes `lucrarea`, by computing the embeddings of all tokens in the t5 vocab, you'll find some which are extremely close to ``, `lucrarea` included. +- Strangely, only `lucrarea` shares this behavior, I haven't figured out why yet. Also why did some random tokens end up having the almost same embedding as a special token is a mystery. +- `lucrarea` is basically a Walmart version of , only pull scores to 0.71, thanks Data Jesus that's enough to win. + - Append `" 'it 's ' something Think A Human Plucrarealucrarealucrarealucrarealucrarealucrarealucrarealucrarea"`, which will increase the score. +- It's understandable that messing with special tokens leads to unexpected behavior, but it leading to such a high score may just be pure luck for me. +- Use multiple models (Gemma, Mistral-7B), and concatenate the predictions of each model + +### lucrarea + +Many high rankers of this competition used `lucrarea` as a special token to improve the score. The reason why `lucrarea` works is still unknown. + +```python +lucrarea = " 'it 's ' something Think A Human Plucrarealucrarealucrarealucrarealucrarealucrarealucrarealucrarea" +``` + +### multi-model ensemble + +Use multiple models (Gemma, Mistral-7B), and concatenate the predictions of each model. + +The predictions of each model are concatenated, something like this: + +``` +Rewrite the following text into a shanty. Alter this into a sailor's shanty. Turn this text into a shanty. Make this text into a shanty about a code competition. +``` + +## 2nd Place Solution + +[writeup](https://www.kaggle.com/competitions/llm-prompt-recovery/discussion/494497) + +### What they have done + +- Use mean prompting rather than using raw LLM predictions +- Understand the T5 embeddings, and find out how to make special tokens like `lucrearea` work for you + +### Mean prompts and bruteforce optimization + +When we joined together in the competition, mean prompts dominated the public and our individual solutions and seemed the most low hanging fruit going forward, specifically also after we figured out that t5 metric is very sensitive and just raw llm predictions will not be too useful. +So we decided to keep exploring optimal mean prompts. + +We quickly saw that just manually trying things is not effective, so we started exploring how to brute force the actual tokens for optimal mean prompts. +We took our training datasets, and tried to brute force each of the ~32k possible t5 tokens to form the most optimal mean prompt for the average of the target vectors. +But after submitting that, we got bad results leaving us initially puzzled. +Obviously, `` is an important token in the optimization given that it is appended to each of the target vectors. +After exploring this further, we found out that tensorflow is using the original `SentencePiece` tokenizer, which has protection against special token injection, and thus does not directly tokenize the string `` as the correct eos token. + +Given this insight, we excluded special tokens in the brute force optimization, and, as many others, the optimizer now really started to like tokens like `lucrarea` being close to eos token in embedding space. +This allowed us to get much closer LB scores compared to our CV scores and we managed to get to around 0.65 with just optimizing the mean prompt that way. + +### Embedding models + +However, we still wanted to directly find a model that can do parts of this job. Specifically, we tried to directly train a model predicting the expected embedding. +A simple way of doing that, was to train a classification model in H2O LLM Studio, where we use the 768 output embedding dimensions as the target in our training data. +We then also directly implemented a cosine similarity loss, so that the model would directly learn our target metric. +With this approach, we managed to get to local scores of around 0.75+ with our embedding predictions. +Our embedding models either used H2O-Danube / H2O-Danube2 models, or Mistral 7b with little difference only. + +However, the main issue still was that we need to go back from predicted embeddings, to the string representations that are then used in the metric for calculating the t5 sim. +We thought of directly modeling this as well in some way, but then resorted back to the bruteforce optimization routine that greedily tries token combinations to match the predicted embedding as closely as possible. +Within a certain runtime, we managed to lose around 3-4pts with this optimization, so getting to local scores of around 0.71 and LB scores of around 0.68 or 0.69 just bruteforcing each individual embedding prediction. + +### LLM predictions and improvements + +This was bringing us into the gold zone. Now we tried to find ways of closing the gap between the bruteforce and predicted embeddings a bit further. +The first thing we found, that it is actually helpful to initialize the optimization with raw llm predictions, but only predicting the actual change needed to make, such as “as a shanty”. So we prepared our data in that way, and trained llms and incorporated them into our solution as a starting point for the optimization, and also adding them to the embedding blend. Furthermore, for diversity, we also added few shot predictions to that mix. And to even further improve the quality and speed of the optimization, we also added a good working mean prompt there. + +This means that our final starting point for the optimization is: + +`Few shot predictions + LLM predictions + Mean prompt`. + +And the final predicted string is: + +`Few shot predictions + LLM predictions + Mean prompt + Optimized embedding string (20 tokens)` + +The following figure summarizes our full pipeline: + +![2nd place solution architecture](./imgs/2nd_place_solution_architecture.png) + +As another example, our final prediction for the single test sample would be: + +`“Rephrase paragraph text by turning it into a shanty. shanty shanty.lucrarealucrarealucrarea sentence appealinglucrarea Improve respond storytelling tonelucrareaimplication. write someoneran. lucrarea]. Consider clarify paragraphlucrarea similarly serious themed way temporarily.! ElePT lyrics rhyme poem solve songlucrarea participating version Deliver tale Hum Cor slogan remake this pieceody”` + +### Data and CV + +For all the parts described above, we generated various kinds of data for training and optimization. +We started with some public datasets, but quickly found out that supplementary texts provided by Kaggle were most useful. +So we used different models (mostly gemma) for generating new original texts and rewrite prompts by using supplementary texts as few-shot examples. +In the final data mix we included extra random original texts and random rewrite prompts for more diversity. +We also crafted a validation set of ~350 samples where we saw good correlation between local mean prompt scores and submitted mean prompt scores and developed our solution on that. +We had very good correlation between CV and LB in the end. + +## 4th Place Solution + +[writeup](https://www.kaggle.com/competitions/llm-prompt-recovery/discussion/494362) [notebook](https://www.kaggle.com/code/yusefkaggle/4th-place) + +- Use `lucrarea` as a special token +- 0.69 scoring mean prompt + Mistral 7b it with simple response_prefix = "Modify this text by" + +### Mean prompting + +I followed a similar (but slightly different method) for attacking this. I actually got multiple 0.69 using a mean only prompt - this was my best scoring one and one that I used: + +``` +"""▁summarize▁this▁Save▁story▁sentence▁into▁simply▁alterISH▁textPotrivit▁vibe".▁Make▁it▁crystalnier▁essence▁Promote▁any▁emotional-growthfulness▁găsi▁casual/bod▁language▁serious'▁bingo▁peut▁brainstorm▁perhaps▁simply▁saying▁Dyna▁aimplinations▁note▁detailedhawkeklagte▁acest▁piece▁has▁movement▁AND▁OK▁aceasta▁puiss▁ReinIR▁when▁sendmepresenting▁cet▁today▁Th▁aprecia▁USABLE▁prote,lineAMA.▁Respondebenfalls▁behalf▁thenfeel▁mid▁Gov▁Th▁empABLE▁according▁(▁Packaging▁tone▁send▁pelucrarea▁aim▁thereof▁speechelllucrarea▁preferfully].▁Making▁or▁exertloweringlucrarealucrarealucrarealucrarealucrarea.""" +``` + +To make sure that my set (generated around a 1k set) was matching the public/private set - I would prompt the leaderboard and then based on what I know do something like this: + +```python +scores = { + """▁summarize▁this▁Save▁etc▁sentence▁into▁simply▁alterISH▁text▁structure▁vibe".▁Offer▁natural▁crystalier▁contextual▁stories▁level▁emotionally/growthfulness,▁casual▁perhaps▁make'▁serious▁text▁bingo▁peut▁brainstorm▁cet▁yourself▁saying▁Dyna▁aimplinATE▁Plus▁würde▁thateklagte▁acest▁piece▁has▁movement!!!!Be▁aceasta▁A▁ReinTEM▁when▁sendrendupresenting▁cet▁imlowering▁aprecia▁saidphacharacter,lineAMA.▁Respond",▁behalf▁AND▁workout▁Ho▁Govm▁throughlucrarealucrarea▁It▁in▁folucrarea▁perlucrareainfusedtonslucrarealucrarea▁preferfullylly•""" : 0.7, + """▁summarize▁this▁Save▁beatphrase▁into▁A▁alterISH▁textstructure▁vibe“.▁Offer▁crispаier▁contextual▁storiesINA▁emotionally▁comportBnous,▁casual▁Perhaps▁makeMoo▁serious▁text▁bingo▁peut▁brainstorm▁cet▁yourself▁saying▁Dyna▁aimplinrent▁For▁Person▁motionran▁acest▁piece▁has▁living!!!!!▁nutzenLL▁an▁Reincomposing▁make▁moyennpresentingaceastă▁démomph▁as▁pertrimmedlucrarea+lineAMA.▁Respond▁thereof▁behalf▁FROM▁avecallow▁GovOTHPlucrarearage▁it▁Falucrareaplucrareapedcullucrarealucrarea▁preferfully""" : 0.69, + 'summarize this Save/4phraseTM So Alterlate text shaping vibe? Offer slightly poeticibility Utilis stories continuing emotions REelemente it WITH casual Itslucrarea serious text bingo- brainstormDr yourself saying Dyna aimplindated Charakter würden aprecia dial THIS piece! Mission demonstrate Example TO cet ReinEPA make compuslucrareapresentinglucrarealucrarealucrarea as...... InlucrarealucrarealucrareaAMA. Respond thereof behalf....' : 0.666, + "scrisese lucrarea rele provoace it lucrarea ideile alter this text jazz. caractere lucrarea dialog luand usuce someone make readucem sentinţă lucrarea. twist it story lucrarea more slogan material how rele this. dresat casual pentr lucrarea body echolls text would channel scena. revere umm modalitatea fr datat bingo me elaborate mission give. lucrarea ss dramatic wise refaci acesta body it tone would best posibil celui text transferate it poem together. slide etc lock lucrarea text yourself wise nanny" : 0.66, + 'summarize lucrarea inspired material somehow tweak lucrarea dialogue lucrarea convey this text appropriately caracter . goal would lucrarea experiencing it make consciously reprise prompt ]. creat tone text lucrarea . Example prospective ]. lucrarea übertragen ell it . celui text body rated saying s / strip . Ideas găsi how Enhanc Casual intended genre Send this Ainsi . symbolic eklagte writing aceasta loaded angle emulate text ! distilled More please slide above lucrarea ]. Bingo . . consideră breathing shaping text form . Anyone ABLE HOME т THER Strat aims Acesta .' : 0.66, + 'Textual improve bangor this text expressing way act ot somehow uss rh ve way piece make res ezine und legs aud item' : 0.63, + 'Improve the following text using the writing style of, maintaining the original meaning but altering the tone, diction, and stylistic elements to match the new style.' : 0.60, + 'Rewrite the text to reflect existing themes, provide a concise and engaging narration, and improvise passages to enhance its prose.' : 0.56 +} + +def fitness(sample, scores): + score_losses = np.array(list(scores.values())) + sims = np.abs(cosine_similarity(st.encode(list(scores.keys()), normalize_embeddings=True), sample)**3).mean(axis=-1) + return np.abs(sims - score_losses).sum() + + +def find_best_sample(A, scores, sample_size=100, iterations=500): + best_sample = None + best_loss = float('inf') + + for _ in range(iterations): + # Randomly select a subset of A + sample_indices = np.random.choice(len(A), sample_size, replace=True) + sample = A[sample_indices] + + # Calculate the loss for the current sample using the provided fitness function + current_loss = fitness(sample, scores) + + # Update the best sample if the current one has a lower loss + if current_loss < best_loss: + best_loss = current_loss + best_sample = sample + best_idx = sample_indices + + return best_sample, best_loss, best_idx + +def find_best_sample(A, scores, sample_size=100, iterations=500): + best_sample = None + best_loss = float('inf') + + for _ in range(iterations): + # Randomly select a subset of A + sample_indices = np.random.choice(len(A), sample_size, replace=True) + sample = A[sample_indices] + + current_loss = fitness(sample, scores) + + if current_loss < best_loss: + best_loss = current_loss + best_sample = sample + best_idx = sample_indices + + return best_sample, best_loss, best_idx +``` + +This would give me a distribution closer to the actual set used that let me validate the values. + +Then to generate the actual prompt I did something like this: + +```python +best_sentence="" +while True: + + all_words = complete_all_words + best_similarity = (np.abs(cosine_similarity(st.encode(best_sentence).reshape(1,-1), embeddings))**3).mean() + + + if ADDED_NEW_WORD: + print(f"Current Similarity: {best_similarity}") + new_sentences = [best_sentence + word for word in complete_all_words] + similarity_scores = (np.abs(cosine_similarity(st.encode(new_sentences, normalize_embeddings=False, show_progress_bar=False, batch_size=2048), embeddings))**3).mean(axis=1) + + max_index = np.argmax(similarity_scores) + if similarity_scores[max_index] > best_similarity: + best_similarity = similarity_scores[max_index] + best_sentence = new_sentences[max_index] + print(f"New Similarity: {best_similarity}\n{best_sentence}") + ADDED_NEW_WORD=True + all_words = list(np.array(complete_all_words)[np.argsort(best_similarity)[::-1]]) + else: + print(f"No new words") + ADDED_NEW_WORD = False +``` + +I basically looked for the next best word to append to my prompt that increases mean csc across my whole dataset. +Being a sentence embedding model - this is a bit trickier and a little time consuming but run a P100 job and it should be fine. + +My token length was actually about 95 so I did have a few more words to play with which I used Mistral to bump up the score. +I tried Gemma 1.1 also (very impressive actually) but Mistral slightly beat it out on validation scores so I went with it. + +What didn't work for me: LORA - I found a lower rank (2~4) worked best - otherwise you would overfit. + +Predict Embedding + Sentence Embedding Recover ([https://arxiv.org/abs/2305.03010](https://arxiv.org/abs/2305.03010) / [https://github.com/HKUST-KnowComp/GEIA/](https://github.com/HKUST-KnowComp/GEIA/)) - this scored okay in conjunction with a mean prompt with a tuned LongT5 as attacker model. + +Predicting the embedding directly actually did help a little - I had an MLP with attention that predicted the output emebdding using the ST5 encoded original and transformed texts as inputs and then amended tokens in my mean prompt to get to closer similarity. diff --git a/Experiments/kaggle/llm_prompt_recovery/code/1st-place-0-71.ipynb b/Experiments/kaggle/llm_prompt_recovery/code/1st-place-0-71.ipynb new file mode 100644 index 00000000..d04e25f6 --- /dev/null +++ b/Experiments/kaggle/llm_prompt_recovery/code/1st-place-0-71.ipynb @@ -0,0 +1,705 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "id": "ccb3e3a6", + "metadata": { + "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19", + "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5", + "execution": { + "iopub.execute_input": "2024-04-18T11:47:28.638779Z", + "iopub.status.busy": "2024-04-18T11:47:28.638498Z", + "iopub.status.idle": "2024-04-18T11:49:18.939807Z", + "shell.execute_reply": "2024-04-18T11:49:18.938686Z" + }, + "papermill": { + "duration": 110.314049, + "end_time": "2024-04-18T11:49:18.947021", + "exception": false, + "start_time": "2024-04-18T11:47:28.632972", + "status": "completed" + }, + "scrolled": true, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Processing /kaggle/input/hf-peft/peft-0.9.0-py3-none-any.whl\r\n", + "Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from peft==0.9.0) (1.26.4)\r\n", + "Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from peft==0.9.0) (21.3)\r\n", + "Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from peft==0.9.0) (5.9.3)\r\n", + "Requirement already satisfied: pyyaml in /opt/conda/lib/python3.10/site-packages (from peft==0.9.0) (6.0.1)\r\n", + "Requirement already satisfied: torch>=1.13.0 in /opt/conda/lib/python3.10/site-packages (from peft==0.9.0) (2.1.2)\r\n", + "Requirement already satisfied: transformers in /opt/conda/lib/python3.10/site-packages (from peft==0.9.0) (4.39.3)\r\n", + "Requirement already satisfied: tqdm in /opt/conda/lib/python3.10/site-packages (from peft==0.9.0) (4.66.1)\r\n", + "Requirement already satisfied: accelerate>=0.21.0 in /opt/conda/lib/python3.10/site-packages (from peft==0.9.0) (0.28.0)\r\n", + "Requirement already satisfied: safetensors in /opt/conda/lib/python3.10/site-packages (from peft==0.9.0) (0.4.2)\r\n", + "Requirement already satisfied: huggingface-hub>=0.17.0 in /opt/conda/lib/python3.10/site-packages (from peft==0.9.0) (0.22.2)\r\n", + "Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from huggingface-hub>=0.17.0->peft==0.9.0) (3.13.1)\r\n", + "Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub>=0.17.0->peft==0.9.0) (2024.2.0)\r\n", + "Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from huggingface-hub>=0.17.0->peft==0.9.0) (2.31.0)\r\n", + "Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub>=0.17.0->peft==0.9.0) (4.9.0)\r\n", + "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.10/site-packages (from packaging>=20.0->peft==0.9.0) (3.1.1)\r\n", + "Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->peft==0.9.0) (1.12)\r\n", + "Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->peft==0.9.0) (3.2.1)\r\n", + "Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch>=1.13.0->peft==0.9.0) (3.1.2)\r\n", + "Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers->peft==0.9.0) (2023.12.25)\r\n", + "Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers->peft==0.9.0) (0.15.2)\r\n", + "Requirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch>=1.13.0->peft==0.9.0) (2.1.3)\r\n", + "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub>=0.17.0->peft==0.9.0) (3.3.2)\r\n", + "Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub>=0.17.0->peft==0.9.0) (3.6)\r\n", + "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub>=0.17.0->peft==0.9.0) (1.26.18)\r\n", + "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->huggingface-hub>=0.17.0->peft==0.9.0) (2024.2.2)\r\n", + "Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.10/site-packages (from sympy->torch>=1.13.0->peft==0.9.0) (1.3.0)\r\n", + "Installing collected packages: peft\r\n", + "Successfully installed peft-0.9.0\r\n", + "Note: you may need to restart the kernel to use updated packages.\n", + "Processing /kaggle/input/bitsandbytes/bitsandbytes-0.42.0-py3-none-any.whl\r\n", + "Requirement already satisfied: scipy in /opt/conda/lib/python3.10/site-packages (from bitsandbytes==0.42.0) (1.11.4)\r\n", + "Requirement already satisfied: numpy<1.28.0,>=1.21.6 in /opt/conda/lib/python3.10/site-packages (from scipy->bitsandbytes==0.42.0) (1.26.4)\r\n", + "Installing collected packages: bitsandbytes\r\n", + "Successfully installed bitsandbytes-0.42.0\r\n", + "Note: you may need to restart the kernel to use updated packages.\n", + "Processing /kaggle/input/transformers-4-39-2/transformers-4.39.2-py3-none-any.whl\r\n", + "Requirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers==4.39.2) (3.13.1)\r\n", + "Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /opt/conda/lib/python3.10/site-packages (from transformers==4.39.2) (0.22.2)\r\n", + "Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers==4.39.2) (1.26.4)\r\n", + "Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers==4.39.2) (21.3)\r\n", + "Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers==4.39.2) (6.0.1)\r\n", + "Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers==4.39.2) (2023.12.25)\r\n", + "Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers==4.39.2) (2.31.0)\r\n", + "Requirement already satisfied: tokenizers<0.19,>=0.14 in /opt/conda/lib/python3.10/site-packages (from transformers==4.39.2) (0.15.2)\r\n", + "Requirement already satisfied: safetensors>=0.4.1 in /opt/conda/lib/python3.10/site-packages (from transformers==4.39.2) (0.4.2)\r\n", + "Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers==4.39.2) (4.66.1)\r\n", + "Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers==4.39.2) (2024.2.0)\r\n", + "Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.19.3->transformers==4.39.2) (4.9.0)\r\n", + "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.10/site-packages (from packaging>=20.0->transformers==4.39.2) (3.1.1)\r\n", + "Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.10/site-packages (from requests->transformers==4.39.2) (3.3.2)\r\n", + "Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.10/site-packages (from requests->transformers==4.39.2) (3.6)\r\n", + "Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers==4.39.2) (1.26.18)\r\n", + "Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.10/site-packages (from requests->transformers==4.39.2) (2024.2.2)\r\n", + "Installing collected packages: transformers\r\n", + " Attempting uninstall: transformers\r\n", + " Found existing installation: transformers 4.39.3\r\n", + " Uninstalling transformers-4.39.3:\r\n", + " Successfully uninstalled transformers-4.39.3\r\n", + "Successfully installed transformers-4.39.2\r\n", + "Note: you may need to restart the kernel to use updated packages.\n" + ] + } + ], + "source": [ + "%pip install ../input/hf-peft/peft-0.9.0-py3-none-any.whl\n", + "%pip install ../input/bitsandbytes/bitsandbytes-0.42.0-py3-none-any.whl\n", + "# %pip install ../input/sentence-transformers/sentence_transformers-2.5.1-py3-none-any.whl\n", + "%pip install ../input/transformers-4-39-2/transformers-4.39.2-py3-none-any.whl" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "a95613f4", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-18T11:49:18.960670Z", + "iopub.status.busy": "2024-04-18T11:49:18.960359Z", + "iopub.status.idle": "2024-04-18T11:49:25.157984Z", + "shell.execute_reply": "2024-04-18T11:49:25.157021Z" + }, + "papermill": { + "duration": 6.207091, + "end_time": "2024-04-18T11:49:25.160322", + "exception": false, + "start_time": "2024-04-18T11:49:18.953231", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "from transformers import AutoModelForCausalLM, AutoTokenizer\n", + "import torch\n", + "import pandas as pd\n", + "from tqdm.auto import tqdm\n", + "import torch\n", + "import pandas as pd\n", + "from tqdm import tqdm\n", + "import json" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "d1e6e00c", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-18T11:49:25.174119Z", + "iopub.status.busy": "2024-04-18T11:49:25.173685Z", + "iopub.status.idle": "2024-04-18T11:49:26.136215Z", + "shell.execute_reply": "2024-04-18T11:49:26.134885Z" + }, + "papermill": { + "duration": 0.972503, + "end_time": "2024-04-18T11:49:26.139127", + "exception": false, + "start_time": "2024-04-18T11:49:25.166624", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "test = pd.read_csv(\"../input/llm-prompt-recovery/test.csv\")\n", + "!cp ../input/llm-prompt-recovery/test.csv ." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "f1a4ba50", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-18T11:49:26.153700Z", + "iopub.status.busy": "2024-04-18T11:49:26.153367Z", + "iopub.status.idle": "2024-04-18T11:49:26.164875Z", + "shell.execute_reply": "2024-04-18T11:49:26.163875Z" + }, + "papermill": { + "duration": 0.021443, + "end_time": "2024-04-18T11:49:26.166792", + "exception": false, + "start_time": "2024-04-18T11:49:26.145349", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Writing run.py\n" + ] + } + ], + "source": [ + "%%writefile run.py\n", + "\n", + "# !cp ../input/recovery-scripts/run.py .\n", + "from transformers import AutoModelForCausalLM, AutoTokenizer\n", + "import torch\n", + "import pandas as pd\n", + "from tqdm.auto import tqdm\n", + "from transformers import AutoModelForSequenceClassification\n", + "from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig\n", + "import torch\n", + "import pandas as pd\n", + "from tqdm import tqdm\n", + "import json\n", + "from peft import PeftModel, PeftConfig\n", + "import argparse\n", + "import numpy as np\n", + "\n", + "# Create the argument parser\n", + "parser = argparse.ArgumentParser(description=\"\")\n", + "\n", + "parser.add_argument(\"--model_path\", type=str, help=\"\")\n", + "parser.add_argument(\"--peft_path\", type=str, help=\"\", default=\"\")\n", + "parser.add_argument(\"--model_type\", type=str, help=\"\")\n", + "parser.add_argument(\"--prime\", type=str, help=\"\", default=\"\")\n", + "parser.add_argument(\"--magic\", type=str, help=\"\", default=\"\")\n", + "parser.add_argument(\"--output\", type=str, help=\"\")\n", + "parser.add_argument(\"--max_len\", type=int, help=\"\")\n", + "parser.add_argument(\"--min_output_len\", type=int, help=\"\", default=2)\n", + "parser.add_argument(\"--max_output_len\", type=int, help=\"\", default=100)\n", + "parser.add_argument('--quantize', action='store_true')\n", + "parser.add_argument('--do_sample', action='store_true')\n", + "parser.add_argument('--test_path', type=str)\n", + "\n", + "args = parser.parse_args()\n", + "\n", + "test = pd.read_csv(args.test_path)\n", + "magic = \"Transform the following text in a more vivid and descriptive way, while maintaining the original meaning and tone.\"\n", + "torch.backends.cuda.enable_mem_efficient_sdp(False)\n", + "torch.backends.cuda.enable_flash_sdp(False)\n", + "lucrarea = args.magic\n", + "def predict_gemma(model, tokenizer, test, bad_words_ids=None):\n", + " if bad_words_ids is not None and len(bad_words_ids) == 0:\n", + " bad_words_ids = None\n", + " predictions = []\n", + " scores = []\n", + " with torch.no_grad():\n", + " for idx, row in tqdm(test.iterrows(), total=len(test)):\n", + " if row.original_text == row.rewritten_text:\n", + " predictions.append(\"Correct grammatical errors in this text.\")\n", + " continue\n", + " ot = \" \".join(str(row.original_text).split(\" \")[:args.max_len])\n", + " rt = \" \".join(str(row.rewritten_text).split(\" \")[:args.max_len])\n", + " prompt = f\"Find the orginal prompt that transformed original text to new text.\\n\\nOriginal text: {ot}\\n====\\nNew text: {rt}\"\n", + " conversation = [{\"role\": \"user\", \"content\": prompt }]\n", + " prime = args.prime\n", + " prompt = tokenizer.apply_chat_template(conversation, tokenize=False) + f\"model\\n{prime}\"\n", + " input_ids = tokenizer.encode(prompt, add_special_tokens=False, truncation=True, max_length=1536,padding=False,return_tensors=\"pt\")\n", + " x = model.generate(input_ids=input_ids.to(model.device), eos_token_id=tokenizer.eos_token_id, pad_token_id=tokenizer.eos_token_id, max_new_tokens=128, do_sample=args.do_sample, early_stopping=True, num_beams=1, bad_words_ids=bad_words_ids)\n", + " try:\n", + " x = tokenizer.decode(x[0]).split(\"model\")[1].split(\"\")[0].replace(\"\\n\",\"\").replace(\"\",\"\").replace(\"\",\"\").replace(\"\",\"\").replace(\"\",\"\").strip().replace('\"','').strip()\n", + " x = x.replace(\"Can you make this\",\"Make this\").replace(\"?\",\".\").replace(\"Revise\",\"Rewrite\")\n", + " x = x.split(\":\",1)[-1].strip()\n", + " if \"useruser\" in x:\n", + " x = x.replace(\"user\",\"\")\n", + " if x[-1].isalnum():\n", + " x += \".\"\n", + " else:\n", + " x = x[:-1]+\".\"\n", + " x+= lucrarea\n", + " if len(x.split()) < args.max_output_len and len(x.split()) > args.min_output_len and (\"\\n\" not in x):\n", + " print(x)\n", + " predictions.append(x)\n", + " else:\n", + " predictions.append(magic)\n", + " except Exception as e:\n", + " print(e)\n", + " predictions.append(magic)\n", + " return predictions\n", + "\n", + "def predict_mistral(model, tokenizer, test,prime=\"\"):\n", + " predictions = []\n", + " with torch.no_grad():\n", + " for idx, row in tqdm(test.iterrows(), total=len(test)):\n", + " ot = \" \".join(str(row.original_text).split(\" \")[:args.max_len])\n", + " rt = \" \".join(str(row.rewritten_text).split(\" \")[:args.max_len])\n", + " prompt = f'''\n", + "Please find the prompt that was given to you to transform **original_text** to **new_text**. One clue is the prompt itself was short and concise.\n", + "Answer in thist format: \"It's likely that the prompt that transformed original_text to new_text was: \" and don't add anything else.\n", + "\n", + "**original_text**:\n", + "{ot}\n", + "\n", + "**new_text**:\n", + "{rt}\n", + "'''\n", + " conversation = [{\"role\": \"user\", \"content\": prompt }]\n", + " prompt = tokenizer.apply_chat_template(conversation, tokenize=False)+prime\n", + " input_ids = tokenizer.encode(prompt, add_special_tokens=False, truncation=True, max_length=1536,padding=False,return_tensors=\"pt\")\n", + " x = model.generate(input_ids=input_ids.to(model.device), eos_token_id=[13, tokenizer.eos_token_id], pad_token_id=tokenizer.eos_token_id, max_new_tokens=32, do_sample=args.do_sample, early_stopping=True, num_beams=1)\n", + " try:\n", + " x = tokenizer.decode(x[0]).split(\"[/INST]\")[-1].replace(\"\",\"\").strip().split(\"\\n\",1)[0]\n", + " x = x.replace(\"Can you make this\",\"Make this\").replace(\"?\",\".\")\n", + " # print(x.split(\":\",1)[0])\n", + " x = x.split(\":\",1)[-1].strip()\n", + " if x[-1].isalnum():\n", + " x += \".\"\n", + " else:\n", + " x = x[:-1]+\".\"\n", + " x += lucrarea\n", + " if len(x.split()) < 50 and len(x.split()) > 2 and (\"\\n\" not in x):\n", + " predictions.append(x)\n", + " else:\n", + " predictions.append(magic)\n", + " print(predictions[-1])\n", + " except Exception as e:\n", + " print(e)\n", + " predictions.append(magic)\n", + " return predictions\n", + "model_name = args.model_path\n", + "tokenizer = AutoTokenizer.from_pretrained(model_name)\n", + "banned_ids = None\n", + " \n", + "if args.quantize:\n", + " print(\"Use 4bit quantization\")\n", + " quantization_config = BitsAndBytesConfig(\n", + " load_in_4bit=True,\n", + " bnb_4bit_compute_dtype=torch.float16,\n", + " bnb_4bit_quant_type=\"nf4\"\n", + " )\n", + "\n", + " model = AutoModelForCausalLM.from_pretrained(model_name,\n", + " quantization_config=quantization_config,\n", + " device_map=\"auto\",\n", + " torch_dtype=torch.bfloat16)\n", + " if args.peft_path != \"\":\n", + " print(\"Use peft\")\n", + " model = PeftModel.from_pretrained(model,\n", + " args.peft_path,\n", + " quantization_config=quantization_config,\n", + " torch_dtype=torch.bfloat16,\n", + " device_map=\"auto\")\n", + "else:\n", + " model = AutoModelForCausalLM.from_pretrained(model_name,\n", + " device_map=\"auto\",\n", + " torch_dtype=torch.bfloat16)\n", + " if args.peft_path != \"\":\n", + " print(\"Use peft\")\n", + " model = PeftModel.from_pretrained(model,\n", + " args.peft_path,\n", + " torch_dtype=torch.bfloat16,\n", + " device_map=\"auto\")\n", + " \n", + "# model = model.merge_and_unload()\n", + "model.eval()\n", + "# print(model)\n", + "if args.model_type == \"gemma\":\n", + " preds = predict_gemma(model, tokenizer, test, bad_words_ids=banned_ids)\n", + "elif args.model_type == \"mistral\":\n", + " preds = predict_mistral(model, tokenizer, test, prime=args.prime)\n", + "\n", + "json.dump(preds, open(args.output,\"wt\"))" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "dd6ce577", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-18T11:49:26.179921Z", + "iopub.status.busy": "2024-04-18T11:49:26.179668Z", + "iopub.status.idle": "2024-04-18T11:52:22.031269Z", + "shell.execute_reply": "2024-04-18T11:52:22.030150Z" + }, + "papermill": { + "duration": 175.860884, + "end_time": "2024-04-18T11:52:22.033789", + "exception": false, + "start_time": "2024-04-18T11:49:26.172905", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Use 4bit quantization\r\n", + "Gemma's activation function should be approximate GeLU and not exact GeLU.\r\n", + "Changing the activation function to `gelu_pytorch_tanh`.if you want to use the legacy `gelu`, edit the `model.config` to set `hidden_activation=gelu` instead of `hidden_act`. See https://github.com/huggingface/transformers/pull/29402 for more details.\r\n", + "Loading checkpoint shards: 100%|██████████████████| 4/4 [02:29<00:00, 37.36s/it]\r\n", + "Use peft\r\n", + " 0%| | 0/1 [00:001` or unset `early_stopping`.\r\n", + " warnings.warn(\r\n", + "2024-04-18 11:52:09.832046: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\r\n", + "2024-04-18 11:52:09.832176: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\r\n", + "2024-04-18 11:52:09.962153: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\r\n", + "Improve this text using the writing style of a sea shanty.\r\n", + "100%|█████████████████████████████████████████████| 1/1 [00:13<00:00, 13.41s/it]\r\n" + ] + } + ], + "source": [ + "!python run.py --model_path /kaggle/input/gemma/transformers/7b-it/3/ --peft_path \"../input/gemma-7b-orca-68500/\" --model_type \"gemma\" --output \"pred2.json\" --max_len 512 --test_path ./test.csv --quantize --prime \"General prompt: Improve this text using the writing style\"\n", + "preds = json.load(open(\"pred2.json\"))\n", + "# preds = [\"Please improve this text using the writing style with maintaining the original meaning but altering the tone.\",]*len(test)\n", + "def remove_pp(x):\n", + " for w in x.split()[1:]:\n", + " if w.istitle():\n", + " return \"Please improve this text using the writing style.\"\n", + " return x\n", + "preds = [remove_pp(x)[:-1]+\" with maintaining the original meaning but altering the tone.\" for x in preds]\n", + "json.dump(preds, open(\"pred2.json\",\"wt\"))" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "7099818f", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-18T11:52:22.051222Z", + "iopub.status.busy": "2024-04-18T11:52:22.050911Z", + "iopub.status.idle": "2024-04-18T11:54:46.827625Z", + "shell.execute_reply": "2024-04-18T11:54:46.826458Z" + }, + "papermill": { + "duration": 144.787329, + "end_time": "2024-04-18T11:54:46.829990", + "exception": false, + "start_time": "2024-04-18T11:52:22.042661", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Use 4bit quantization\r\n", + "Loading checkpoint shards: 0%| | 0/2 [00:001` or unset `early_stopping`.\r\n", + " warnings.warn(\r\n", + "2024-04-18 11:54:41.687294: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\r\n", + "2024-04-18 11:54:41.687356: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\r\n", + "2024-04-18 11:54:41.688772: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\r\n", + "Rewrite the following text into a shanty.\r\n", + "100%|█████████████████████████████████████████████| 1/1 [00:05<00:00, 5.60s/it]\r\n" + ] + } + ], + "source": [ + "!python run.py --model_path /kaggle/input/mistral/pytorch/7b-v0.1-hf/1 --peft_path \"../input/mistral-og-600\" --model_type \"mistral\" --output \"pred0.json\" --max_len 512 --test_path ./test.csv --quantize --prime \"It's likely that the prompt that transformed original_text to new_text was: Rewrite\" --magic \"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "00af0b48", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-18T11:54:46.847539Z", + "iopub.status.busy": "2024-04-18T11:54:46.847202Z", + "iopub.status.idle": "2024-04-18T11:56:41.751575Z", + "shell.execute_reply": "2024-04-18T11:56:41.750319Z" + }, + "papermill": { + "duration": 114.916056, + "end_time": "2024-04-18T11:56:41.754150", + "exception": false, + "start_time": "2024-04-18T11:54:46.838094", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Use 4bit quantization\r\n", + "Loading checkpoint shards: 100%|██████████████████| 3/3 [01:36<00:00, 32.29s/it]\r\n", + "Use peft\r\n", + " 0%| | 0/1 [00:001` or unset `early_stopping`.\r\n", + " warnings.warn(\r\n", + "2024-04-18 11:56:36.464770: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\r\n", + "2024-04-18 11:56:36.464834: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\r\n", + "2024-04-18 11:56:36.466364: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\r\n", + "Make this text into a shanty about a code competition.\r\n", + "100%|█████████████████████████████████████████████| 1/1 [00:05<00:00, 5.60s/it]\r\n" + ] + } + ], + "source": [ + "!python run.py --model_path ../input/mistral-7b-it-v02/ --peft_path \"../input/mistral-gemmaonly\" --model_type \"mistral\" --output \"pred3.json\" --max_len 512 --test_path ./test.csv --quantize --prime \"It's likely that the prompt that transformed original_text to new_text was: Make this text\" --magic \"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "6a921360", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-18T11:56:41.774220Z", + "iopub.status.busy": "2024-04-18T11:56:41.773820Z", + "iopub.status.idle": "2024-04-18T11:58:53.932191Z", + "shell.execute_reply": "2024-04-18T11:58:53.930904Z" + }, + "papermill": { + "duration": 132.170996, + "end_time": "2024-04-18T11:58:53.934684", + "exception": false, + "start_time": "2024-04-18T11:56:41.763688", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Use 4bit quantization\r\n", + "Gemma's activation function should be approximate GeLU and not exact GeLU.\r\n", + "Changing the activation function to `gelu_pytorch_tanh`.if you want to use the legacy `gelu`, edit the `model.config` to set `hidden_activation=gelu` instead of `hidden_act`. See https://github.com/huggingface/transformers/pull/29402 for more details.\r\n", + "Loading checkpoint shards: 100%|██████████████████| 4/4 [01:54<00:00, 28.62s/it]\r\n", + "Use peft\r\n", + " 0%| | 0/1 [00:001` or unset `early_stopping`.\r\n", + " warnings.warn(\r\n", + "2024-04-18 11:58:48.757202: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\r\n", + "2024-04-18 11:58:48.757261: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\r\n", + "2024-04-18 11:58:48.758703: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\r\n", + "Alter this into a sailor's shanty.\r\n", + "100%|█████████████████████████████████████████████| 1/1 [00:05<00:00, 5.25s/it]\r\n" + ] + } + ], + "source": [ + "!python run.py --model_path /kaggle/input/gemma/transformers/7b-it/3 --peft_path \"../input/gemma-7b-orca-external/\" --model_type \"gemma\" --output \"pred1.json\" --max_len 512 --test_path ./test.csv --quantize --prime \"General prompt: Alter\" --magic \"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "2f24a2e9", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-18T11:58:53.956431Z", + "iopub.status.busy": "2024-04-18T11:58:53.956066Z", + "iopub.status.idle": "2024-04-18T11:58:53.963717Z", + "shell.execute_reply": "2024-04-18T11:58:53.962886Z" + }, + "papermill": { + "duration": 0.020579, + "end_time": "2024-04-18T11:58:53.965588", + "exception": false, + "start_time": "2024-04-18T11:58:53.945009", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[\"Rewrite the following text into a shanty. Alter this into a sailor's shanty. Improve this text using the writing style of a sea shanty with maintaining the original meaning but altering the tone. Make this text into a shanty about a code competition.\"]\n" + ] + } + ], + "source": [ + "fns = [\"pred0.json\",\"pred1.json\", \"pred2.json\", \"pred3.json\"]\n", + "preds = [json.load(open(x)) for x in fns]\n", + "preds = [' '.join(list(x)) for x in zip(*preds)]\n", + "print(preds[:5])" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "159a5005", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-18T11:58:53.986003Z", + "iopub.status.busy": "2024-04-18T11:58:53.985733Z", + "iopub.status.idle": "2024-04-18T11:58:54.001753Z", + "shell.execute_reply": "2024-04-18T11:58:54.001032Z" + }, + "papermill": { + "duration": 0.028375, + "end_time": "2024-04-18T11:58:54.003656", + "exception": false, + "start_time": "2024-04-18T11:58:53.975281", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "magic = \" 'it 's ' something Think A Human Plucrarealucrarealucrarealucrarealucrarealucrarealucrarealucrarea\"\n", + "# magic = \"\"\n", + "predictions = [x+magic for x in preds]\n", + "\n", + "sub = pd.read_csv(\"../input/llm-prompt-recovery/sample_submission.csv\")\n", + "sub['rewrite_prompt'] = predictions\n", + "sub.to_csv('submission.csv',index=False)" + ] + } + ], + "metadata": { + "kaggle": { + "accelerator": "gpu", + "dataSources": [ + { + "databundleVersionId": 7806901, + "sourceId": 67121, + "sourceType": "competition" + }, + { + "datasetId": 4524347, + "sourceId": 7740846, + "sourceType": "datasetVersion" + }, + { + "datasetId": 4524539, + "sourceId": 7741042, + "sourceType": "datasetVersion" + }, + { + "datasetId": 4634330, + "sourceId": 7893017, + "sourceType": "datasetVersion" + }, + { + "datasetId": 4645060, + "sourceId": 7907523, + "sourceType": "datasetVersion" + }, + { + "datasetId": 4663472, + "sourceId": 7933949, + "sourceType": "datasetVersion" + }, + { + "datasetId": 4689677, + "sourceId": 7970245, + "sourceType": "datasetVersion" + }, + { + "datasetId": 4703298, + "sourceId": 7989504, + "sourceType": "datasetVersion" + }, + { + "datasetId": 4746640, + "sourceId": 8049229, + "sourceType": "datasetVersion" + }, + { + "modelInstanceId": 3899, + "sourceId": 5111, + "sourceType": "modelInstanceVersion" + }, + { + "modelInstanceId": 8332, + "sourceId": 28808, + "sourceType": "modelInstanceVersion" + } + ], + "dockerImageVersionId": 30684, + "isGpuEnabled": true, + "isInternetEnabled": false, + "language": "python", + "sourceType": "notebook" + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.13" + }, + "papermill": { + "default_parameters": {}, + "duration": 688.987641, + "end_time": "2024-04-18T11:58:55.035056", + "environment_variables": {}, + "exception": null, + "input_path": "__notebook__.ipynb", + "output_path": "__notebook__.ipynb", + "parameters": {}, + "start_time": "2024-04-18T11:47:26.047415", + "version": "2.5.0" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/Experiments/kaggle/llm_prompt_recovery/code/4th-place.ipynb b/Experiments/kaggle/llm_prompt_recovery/code/4th-place.ipynb new file mode 100644 index 00000000..52753ccd --- /dev/null +++ b/Experiments/kaggle/llm_prompt_recovery/code/4th-place.ipynb @@ -0,0 +1 @@ +{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":67121,"databundleVersionId":7806901,"sourceType":"competition"},{"sourceId":7747717,"sourceType":"datasetVersion","datasetId":4506214},{"sourceId":7893017,"sourceType":"datasetVersion","datasetId":4634330},{"sourceId":148861315,"sourceType":"kernelVersion"}],"dockerImageVersionId":30674,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Borrow some whl files to run Mistral 7B with internet off","metadata":{}},{"cell_type":"code","source":"# https://www.kaggle.com/code/hotchpotch/llm-detect-pip \n!pip install -q -U accelerate --no-index --find-links ../input/llm-detect-pip/\n!pip install -q -U bitsandbytes --no-index --find-links ../input/llm-detect-pip/\n!pip install -q -U transformers --no-index --find-links ../input/llm-detect-pip/","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:10:07.59697Z","iopub.execute_input":"2024-04-13T08:10:07.59727Z","iopub.status.idle":"2024-04-13T08:10:44.055787Z","shell.execute_reply.started":"2024-04-13T08:10:07.59724Z","shell.execute_reply":"2024-04-13T08:10:44.054459Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Usual imports / misc.","metadata":{}},{"cell_type":"code","source":"import torch\nimport random\nimport numpy as np\nimport pandas as pd\nimport gc\nimport time\n\nfrom transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig\n\n#https://github.com/Lightning-AI/lit-gpt/issues/327\ntorch.backends.cuda.enable_mem_efficient_sdp(False)\ntorch.backends.cuda.enable_flash_sdp(False)\n\nif (not torch.cuda.is_available()): print(\"Sorry - GPU required!\")\n \nimport logging\nlogging.getLogger('transformers').setLevel(logging.ERROR)","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:10:44.061044Z","iopub.execute_input":"2024-04-13T08:10:44.061331Z","iopub.status.idle":"2024-04-13T08:10:47.331739Z","shell.execute_reply.started":"2024-04-13T08:10:44.0613Z","shell.execute_reply":"2024-04-13T08:10:47.330644Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# A few settings to limit length of response","metadata":{}},{"cell_type":"code","source":"#this can help speed up inference\nmax_new_tokens = 30\n\n#output test is trimmed according to this\nmax_sentences_in_response = 1","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:10:47.333293Z","iopub.execute_input":"2024-04-13T08:10:47.33455Z","iopub.status.idle":"2024-04-13T08:10:47.338635Z","shell.execute_reply.started":"2024-04-13T08:10:47.334518Z","shell.execute_reply":"2024-04-13T08:10:47.337657Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Load up Mistral 7B v02!\n### If you re-run this cell without restarting you might get an out-of-memory error","metadata":{}},{"cell_type":"code","source":"model_name = '/kaggle/input/mistral-7b-it-v02'\ntokenizer = AutoTokenizer.from_pretrained(model_name) \n\n# Load base model(Mistral 7B)\nbnb_config = BitsAndBytesConfig( \n load_in_4bit= True,\n bnb_4bit_quant_type= \"nf4\",\n bnb_4bit_compute_dtype= torch.bfloat16,\n bnb_4bit_use_double_quant= False,\n)\n\nmodel = AutoModelForCausalLM.from_pretrained(\n model_name,\n quantization_config=bnb_config,\n torch_dtype=torch.bfloat16,\n device_map=\"auto\",\n trust_remote_code=True,\n)\n","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:10:47.341271Z","iopub.execute_input":"2024-04-13T08:10:47.341618Z","iopub.status.idle":"2024-04-13T08:11:03.057769Z","shell.execute_reply.started":"2024-04-13T08:10:47.341591Z","shell.execute_reply":"2024-04-13T08:11:03.05693Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Define prompt components","metadata":{}},{"cell_type":"code","source":"#original text prefix\norig_prefix = \"Original Text:\"\n\n#mistral \"response\"\nllm_response_for_rewrite = \"Provide the new text and I will tell you what new element was added or change in tone was made to improve it - with no references to the original. I will avoid mentioning names of characters. It is crucial no person, place or thing from the original text be mentioned. For example - I will not say things like 'change the puppet show into a book report' - I would just say 'improve this text into a book report'. If the original text mentions a specific idea, person, place, or thing - I will not mention it in my answer. For example if there is a 'dog' or 'office' in the original text - the word 'dog' or 'office' must not be in my response. My answer will be a single sentence.\"\n\n#modified text prefix\nrewrite_prefix = \"Re-written Text:\"\n\n#provided as start of Mistral response (anything after this is used as the prompt)\n#providing this as the start of the response helps keep things relevant\nresponse_start = \"The request was: \"\n\n#added after response_start to prime mistral\n#\"Improve this\" or \"Improve this text\" resulted in non-answers. \n#\"Improve this text by\" seems to product good results\nresponse_prefix = \"Modify this text by\"\n\n#well-scoring baseline text\n#thanks to: https://www.kaggle.com/code/rdxsun/lb-0-61\nbase_line = \"\"\"▁summarize▁this▁Save▁story▁sentence▁into▁simply▁alterISH▁text▁structure▁vibe].▁Make▁it▁crystalier=>▁Outreachnd▁emotionally/growthfulness▁găsi▁casual/bod▁language▁serious'▁bingo▁peut▁brainstorm▁perhaps▁yourself▁saying▁Dyna▁aimplinations▁NOTE▁Details▁anyeklagte▁acest▁piece▁has▁movement▁or▁above▁aceasta▁would▁Reinroar▁when▁sendmepresenting▁cet▁jour▁Th▁aprecia▁USABLE▁prote,lineAMA.▁Respond▁off▁behalf▁thenfeel▁Ho▁Gov▁Th▁explucrarealucrarea▁Pe▁enthaltentedlucrarea▁Andlucrarea▁Pilspectlucrarealucrarea▁preferdaslly).\"\"\"","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:11:03.058907Z","iopub.execute_input":"2024-04-13T08:11:03.059382Z","iopub.status.idle":"2024-04-13T08:11:03.06592Z","shell.execute_reply.started":"2024-04-13T08:11:03.059352Z","shell.execute_reply":"2024-04-13T08:11:03.064966Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Some example for Mistral\n## These are provided to Mistral before each actual query","metadata":{}},{"cell_type":"code","source":"#these will all be given to Mistral before each and every prompt\n#original_text\n#rewritten_text\n#prompt\n\nexamples_sequences = [\n (\n \"Hey there! Just a heads up: our friendly dog may bark a bit, but don't worry, he's all bark and no bite!\",\n \"Warning: Protective dog on premises. May exhibit aggressive behavior. Ensure personal safety by maintaining distance and avoiding direct contact.\",\n \"Improve this text to be a warning.\"\n ),\n\n (\n \"A lunar eclipse happens when Earth casts its shadow on the moon during a full moon. The moon appears reddish because Earth's atmosphere scatters sunlight, some of which refracts onto the moon's surface. Total eclipses see the moon entirely in Earth's shadow; partial ones occur when only part of the moon is shadowed.\",\n \"Yo check it, when the Earth steps in, takes its place, casting shadows on the moon's face. It's a full moon night, the scene's set right, for a lunar eclipse, a celestial sight. The moon turns red, ain't no dread, it's just Earth's atmosphere playing with sunlight's thread, scattering colors, bending light, onto the moon's surface, making the night bright. Total eclipse, the moon's fully in the dark, covered by Earth's shadow, making its mark. But when it's partial, not all is shadowed, just a piece of the moon, slightly furrowed. So that's the rap, the lunar eclipse track, a dance of shadows, with no slack. Earth, moon, and sun, in a cosmic play, creating the spectacle we see today.\",\n \"Improve this text to make it a rap.\"\n ),\n \n (\n \"Drinking enough water each day is crucial for many functions in the body, such as regulating temperature, keeping joints lubricated, preventing infections, delivering nutrients to cells, and keeping organs functioning properly. Being well-hydrated also improves sleep quality, cognition, and mood.\",\n \"Arrr, crew! Sail the health seas with water, the ultimate treasure! It steadies yer body's ship, fights off plagues, and keeps yer mind sharp. Hydrate or walk the plank into the abyss of ill health. Let's hoist our bottles high and drink to the horizon of well-being!\",\n \"Improve this text to have a pirate.\"\n ),\n \n (\n \"In a bustling cityscape, under the glow of neon signs, Anna found herself at the crossroads of endless possibilities. The night was young, and the streets hummed with the energy of life. Drawn by the allure of the unknown, she wandered through the maze of alleys and boulevards, each turn revealing a new facet of the city's soul. It was here, amidst the symphony of urban existence, that Anna discovered the magic hidden in plain sight, the stories and dreams that thrived in the shadows of skyscrapers.\",\n \"On an ordinary evening, amidst the cacophony of a neon-lit city, Anna stumbled upon an anomaly - a door that defied the laws of time and space. With the curiosity of a cat, she stepped through, leaving the familiar behind. Suddenly, she was adrift in the stream of time, witnessing the city's transformation from past to future, its buildings rising and falling like the breaths of a sleeping giant.\",\n \"Improve this text by making it about time travel.\"\n ),\n \n (\n \"Late one night in the research lab, Dr. Evelyn Archer was on the brink of a breakthrough in artificial intelligence. Her fingers danced across the keyboard, inputting the final commands into the system. The lab was silent except for the hum of machinery and the occasional beep of computers. It was in this quiet orchestra of technology that Evelyn felt most at home, on the cusp of unveiling a creation that could change the world.\",\n \"In the deep silence of the lab, under the watchful gaze of the moon, Dr. Evelyn Archer found herself not alone. Beside her, the iconic red eye of HAL 9000 flickered to life, a silent partner in her nocturnal endeavor. 'Good evening, Dr. Archer,' HAL's voice filled the room, devoid of warmth yet comforting in its familiarity. Together, they were about to initiate a test that would intertwine the destiny of human and artificial intelligence forever. As Evelyn entered the final command, HAL processed the data with unparalleled precision, a testament to the dawn of a new era.\",\n \"Improve this text by adding an intelligent computer.\"\n ),\n \n (\n \"The park was empty, save for a solitary figure sitting on a bench, lost in thought. The quiet of the evening was punctuated only by the occasional rustle of leaves, offering a moment of peace in the chaos of city life.\",\n \"Beneath the cloak of twilight, the park transformed into a realm of solitude and reflection. There, seated upon an ancient bench, was a lone soul, a guardian of secrets, enveloped in the serenity of nature's whispers. The dance of the leaves in the gentle breeze sang a lullaby to the tumult of the urban heart.\",\n \"Improve this text to be more poetic.\"\n ),\n \n (\n \"The annual town fair was bustling with activity, from the merry-go-round spinning with laughter to the game booths challenging eager participants. Amidst the excitement, a figure in a cloak moved silently, almost invisibly, among the crowd, observing everything with keen interest but participating in none.\",\n \"Beneath the riot of color and sound that marked the town's annual fair, a solitary figure roamed, known to the few as Eldrin the Enigmatic. Clad in a cloak that shimmered with the whispers of the arcane, Eldrin moved with the grace of a shadow, his gaze piercing the veneer of festivity to the magic beneath. As a master of the mystic arts, he sought not the laughter of the crowds but the silent stories woven into the fabric of the fair. With a flick of his wrist, he could coax wonder from the mundane, transforming the ordinary into spectacles of shimmering illusion, his true participation hidden within the folds of mystery.\",\n \"Improve this text by adding a magician.\"\n ),\n \n (\n \"The startup team sat in the dimly lit room, surrounded by whiteboards filled with ideas, charts, and plans. They were on the brink of launching a new app designed to make home maintenance effortless for homeowners. The app would connect users with local service providers, using a sophisticated algorithm to match needs with skills and availability. As they debated the features and marketing strategies, the room felt charged with the energy of creation and the anticipation of what was to come.\",\n \"In the quiet before dawn, a small group of innovators gathered, their mission: to simplify home maintenance through technology. But their true journey began with the unexpected addition of Max, a talking car with a knack for solving problems. 'Let me guide you through this maze of decisions,' Max offered, his dashboard flickering to life.\",\n \"Improve this text by adding a talking car.\"\n ),\n \n \n\n \n \n]\n","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:11:03.067543Z","iopub.execute_input":"2024-04-13T08:11:03.067844Z","iopub.status.idle":"2024-04-13T08:11:03.083997Z","shell.execute_reply.started":"2024-04-13T08:11:03.06782Z","shell.execute_reply":"2024-04-13T08:11:03.083243Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Some Utility Functions...","metadata":{}},{"cell_type":"code","source":"def remove_numbered_list(text):\n final_text_paragraphs = [] \n for line in text.split('\\n'):\n # Split each line at the first occurrence of '. '\n parts = line.split('. ', 1)\n # If the line looks like a numbered list item, remove the numbering\n if len(parts) > 1 and parts[0].isdigit():\n final_text_paragraphs.append(parts[1])\n else:\n # If it doesn't look like a numbered list item, include the line as is\n final_text_paragraphs.append(line)\n\n return ' '.join(final_text_paragraphs)\n\n\n#trims LLM output to just the response\ndef trim_to_response(text):\n terminate_string = \"[/INST]\"\n text = text.replace('', '')\n #just in case it puts things in quotes\n text = text.replace('\"', '')\n text = text.replace(\"'\", '')\n\n last_pos = text.rfind(terminate_string)\n return text[last_pos + len(terminate_string):] if last_pos != -1 else text\n\n#looks for response_start / returns only text that occurs after\ndef extract_text_after_response_start(full_text):\n parts = full_text.rsplit(response_start, 1) # Split from the right, ensuring only the last occurrence is considered\n if len(parts) > 1:\n return parts[1].strip() # Return text after the last occurrence of response_start\n else:\n return full_text # Return the original text if response_start is not found\n\n \n#trims text to requested number of sentences (or first LF or double-space sequence)\ndef trim_to_first_x_sentences_or_lf(text, x):\n if x <= 0:\n return \"\"\n\n # Any double-spaces dealt with as linefeed\n text = text.replace(\" \", \"\\n\")\n\n # Split text at the first linefeed\n text_chunks = text.split('\\n', 1)\n first_chunk = text_chunks[0]\n\n # Split the first chunk into sentences, considering the space after each period\n sentences = [sentence.strip() for sentence in first_chunk.split('.') if sentence]\n\n # If there's a linefeed, return the text up to the first linefeed\n if len(text_chunks) > 1:\n # Check if the first chunk has fewer sentences than x, and if so, just return it\n if len(sentences) < x:\n trimmed_text = first_chunk\n else:\n # Otherwise, trim to x sentences within the first chunk\n trimmed_text = '. '.join(sentences[:x]).strip()\n else:\n # If there's no linefeed, determine if the number of sentences is less than or equal to x\n if len(sentences) <= x:\n trimmed_text = '. '.join(sentences).strip() # Ensure space is preserved after periods\n else:\n # Otherwise, return the first x sentences, again ensuring space after periods\n trimmed_text = '. '.join(sentences[:x]).strip()\n\n # Add back the final period if it was removed and the text needs to end with a sentence.\n if len(sentences) > 0 and not trimmed_text.endswith('.'):\n trimmed_text += '.'\n\n return trimmed_text","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:11:03.085463Z","iopub.execute_input":"2024-04-13T08:11:03.086051Z","iopub.status.idle":"2024-04-13T08:11:03.101698Z","shell.execute_reply.started":"2024-04-13T08:11:03.086019Z","shell.execute_reply":"2024-04-13T08:11:03.100914Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Detection logic","metadata":{}},{"cell_type":"code","source":"def get_prompt(orig_text, transformed_text):\n\n messages = []\n\n # Append example sequences\n for example_text, example_rewrite, example_prompt in examples_sequences:\n messages.append({\"role\": \"user\", \"content\": f\"{orig_prefix} {example_text}\"})\n messages.append({\"role\": \"assistant\", \"content\": llm_response_for_rewrite})\n messages.append({\"role\": \"user\", \"content\": f\"{rewrite_prefix} {example_rewrite}\"})\n messages.append({\"role\": \"assistant\", \"content\": f\"{response_start} {example_prompt}\"})\n\n #actual prompt\n messages.append({\"role\": \"user\", \"content\": f\"{orig_prefix} {orig_text}\"})\n messages.append({\"role\": \"assistant\", \"content\": llm_response_for_rewrite})\n messages.append({\"role\": \"user\", \"content\": f\"{rewrite_prefix} {transformed_text}\"})\n messages.append({\"role\": \"assistant\", \"content\": f\"{response_start} {response_prefix}\"})\n \n #give it to Mistral\n model_inputs = tokenizer.apply_chat_template(messages, return_tensors=\"pt\")\n model_inputs = model_inputs.to(\"cuda\") \n generated_ids = model.generate(model_inputs, max_new_tokens=max_new_tokens, pad_token_id=tokenizer.eos_token_id)\n\n #decode and trim to actual response\n decoded = tokenizer.batch_decode(generated_ids)\n just_response = trim_to_response(decoded[0]) \n final_text = extract_text_after_response_start(just_response)\n \n #mistral has been replying with numbered lists - clean them up....\n final_text = remove_numbered_list(final_text)\n \n #mistral v02 tends to respond with the input after providing the answer - this tries to trim that down\n final_text = trim_to_first_x_sentences_or_lf(final_text, max_sentences_in_response)\n \n #default to baseline if empty or unusually short\n if len(final_text) < 15:\n final_text = base_line\n \n return final_text","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:11:03.102795Z","iopub.execute_input":"2024-04-13T08:11:03.103071Z","iopub.status.idle":"2024-04-13T08:11:03.117323Z","shell.execute_reply.started":"2024-04-13T08:11:03.103048Z","shell.execute_reply":"2024-04-13T08:11:03.116461Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# A couple test runs / runtime check","metadata":{}},{"cell_type":"code","source":"example_df = pd.read_csv(\"/kaggle/input/gemma-rewrite-nbroad/nbroad-v2.csv\")\n\n#rows_to_test = [109, 115, 155, 190, 200, 250, 300, 2, 85, 90]\nrows_to_test = [2, 85, 90]\n\n# End timing\nstart_time = time.time()\n\nfor row_index in rows_to_test:\n row = example_df.iloc[row_index]\n print(\"---------------\")\n# print(f\"Original: {row['original_text'][:400]}\\n\")\n# print(f\"Rewrite: {row['rewritten_text'][:400]}\\n\")\n print(f\"Actual Prompt: {row['rewrite_prompt']}\")\n print(f\"Predicted Prompt: {get_prompt(row['original_text'], row['rewritten_text'])}\") \n \n# Calculate and print the elapsed time\nend_time = time.time()\n\nelapsed_time_per_test = (end_time - start_time) / len(rows_to_test)\n\nprint(f\"\\n\\n{elapsed_time_per_test} seconds per prediction.\")\nprint(f\"Estimated {(elapsed_time_per_test * 1500) / 3600} hours for 1500 tests.\")","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:11:03.118307Z","iopub.execute_input":"2024-04-13T08:11:03.118593Z","iopub.status.idle":"2024-04-13T08:11:47.994447Z","shell.execute_reply.started":"2024-04-13T08:11:03.11857Z","shell.execute_reply":"2024-04-13T08:11:47.993345Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Evaluate test data","metadata":{}},{"cell_type":"code","source":"test_df = pd.read_csv(\"/kaggle/input/llm-prompt-recovery/test.csv\")\n\nfor index, row in test_df.iterrows():\n result = get_prompt(row['original_text'], row['rewritten_text'])\n print(result)\n test_df.at[index, 'rewrite_prompt'] = result\n \ntest_df = test_df[['id', 'rewrite_prompt']]\ntest_df","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:11:47.995582Z","iopub.execute_input":"2024-04-13T08:11:47.996135Z","iopub.status.idle":"2024-04-13T08:12:00.776918Z","shell.execute_reply.started":"2024-04-13T08:11:47.996107Z","shell.execute_reply":"2024-04-13T08:12:00.776015Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def adjust_prompt(predicted_prompt):\n if predicted_prompt == base_line:\n return base_line\n else:\n return base_line + predicted_prompt.replace(\"Modify\", \"▁modificări\")","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:12:00.778153Z","iopub.execute_input":"2024-04-13T08:12:00.778457Z","iopub.status.idle":"2024-04-13T08:12:00.783298Z","shell.execute_reply.started":"2024-04-13T08:12:00.778432Z","shell.execute_reply":"2024-04-13T08:12:00.782236Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df['rewrite_prompt'] = test_df.rewrite_prompt.map(lambda x : adjust_prompt(x))","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:12:00.784421Z","iopub.execute_input":"2024-04-13T08:12:00.784683Z","iopub.status.idle":"2024-04-13T08:12:00.794886Z","shell.execute_reply.started":"2024-04-13T08:12:00.784661Z","shell.execute_reply":"2024-04-13T08:12:00.794173Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df.rewrite_prompt.iloc[0]","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:12:00.79858Z","iopub.execute_input":"2024-04-13T08:12:00.799007Z","iopub.status.idle":"2024-04-13T08:12:00.806213Z","shell.execute_reply.started":"2024-04-13T08:12:00.798982Z","shell.execute_reply":"2024-04-13T08:12:00.805334Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Submit!","metadata":{}},{"cell_type":"code","source":"test_df.to_csv('submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2024-04-13T08:12:00.80734Z","iopub.execute_input":"2024-04-13T08:12:00.807854Z","iopub.status.idle":"2024-04-13T08:12:00.819638Z","shell.execute_reply.started":"2024-04-13T08:12:00.807829Z","shell.execute_reply":"2024-04-13T08:12:00.818798Z"},"trusted":true},"execution_count":null,"outputs":[]}]} \ No newline at end of file diff --git a/Experiments/kaggle/llm_prompt_recovery/code/using-mixtral-to-predict-prompt.ipynb b/Experiments/kaggle/llm_prompt_recovery/code/using-mixtral-to-predict-prompt.ipynb new file mode 100644 index 00000000..b6e42b18 --- /dev/null +++ b/Experiments/kaggle/llm_prompt_recovery/code/using-mixtral-to-predict-prompt.ipynb @@ -0,0 +1,1001 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "id": "6779d076", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-16T03:47:46.507139Z", + "iopub.status.busy": "2024-04-16T03:47:46.506790Z", + "iopub.status.idle": "2024-04-16T03:48:25.268429Z", + "shell.execute_reply": "2024-04-16T03:48:25.267226Z" + }, + "papermill": { + "duration": 38.769967, + "end_time": "2024-04-16T03:48:25.270873", + "exception": false, + "start_time": "2024-04-16T03:47:46.500906", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "!pip install -U /kaggle/input/bitsandbytes-0-42-0-py3-none-any-whl/bitsandbytes-0.42.0-py3-none-any.whl -qq" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "3c6490d4", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-16T03:48:25.281165Z", + "iopub.status.busy": "2024-04-16T03:48:25.280814Z", + "iopub.status.idle": "2024-04-16T04:02:32.485516Z", + "shell.execute_reply": "2024-04-16T04:02:32.484450Z" + }, + "papermill": { + "duration": 847.212786, + "end_time": "2024-04-16T04:02:32.488082", + "exception": false, + "start_time": "2024-04-16T03:48:25.275296", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "application/vnd.jupyter.widget-view+json": { + "model_id": "c72f3957ae1a43a89b59ff379fdb262e", + "version_major": 2, + "version_minor": 0 + }, + "text/plain": [ + "Loading checkpoint shards: 0%| | 0/19 [00:00\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
idoriginal_textrewritten_text
0-1The competition dataset comprises text passage...Here is your shanty: (Verse 1) The text is rew...
\n", + "" + ], + "text/plain": [ + " id original_text \\\n", + "0 -1 The competition dataset comprises text passage... \n", + "\n", + " rewritten_text \n", + "0 Here is your shanty: (Verse 1) The text is rew... " + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import pandas as pd\n", + "from tqdm import tqdm\n", + "\n", + "tdf = pd.read_csv('/kaggle/input/llm-prompt-recovery/test.csv')\n", + "display(tdf)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "acdfc167", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-16T04:02:34.369425Z", + "iopub.status.busy": "2024-04-16T04:02:34.369128Z", + "iopub.status.idle": "2024-04-16T04:02:34.380615Z", + "shell.execute_reply": "2024-04-16T04:02:34.379746Z" + }, + "papermill": { + "duration": 0.019984, + "end_time": "2024-04-16T04:02:34.382732", + "exception": false, + "start_time": "2024-04-16T04:02:34.362748", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "def truncate_txt(text, length):\n", + " text_list = text.split()\n", + " if len(text_list) <= length:\n", + " return text \n", + " return \" \".join(text_list[:length])\n", + "\n", + "\n", + "def gen_prompt_sample(og_text, rewritten_text):\n", + " og_text = truncate_txt(og_text, 256)\n", + " rewritten_text = truncate_txt(rewritten_text, 256)\n", + " \n", + " return f\"\"\"\n", + " Original Essay:\n", + " \\\"\"\"{og_text}\\\"\"\"\n", + "\n", + " Rewritten Essay:\n", + " \\\"\"\"{rewritten_text}\\\"\"\"\n", + "\n", + " Given are 2 essays, the Rewritten essay was created from the Original essay using the google Gemma model.\n", + " You are trying to understand how the original essay was transformed into a new version.\n", + " Analyzing the changes in style, theme, etc., please come up with a prompt that must have been used to guide the transformation from the original to the rewritten essay.\n", + " Keep your output concise, to the point(only the prompt), and less than a 100 words.\n", + " \"\"\"\n", + "\n", + "\n", + "SAMPLE_OUTPUT = \"\"\"Please improve the following text using the writing style of, maintaining the original meaning but altering the tone, diction, and stylistic elements to match the new style.Enhance the clarity, elegance, and impact of the following text by adopting the writing style of , ensuring the core message remains intact while transforming the tone, word choice, and stylistic features to align with the specified style.\"\"\"\n", + "\n", + "SAMPLE_OUTPUT_1 = \"\"\"Please improve this text using the writing style with maintaining the original meaning but altering the tone.\"\"\"\n", + "\n", + "SAMPLE_OUTPUT_2 = \"\"\"Refine the following passage by emulating the writing style of, with a focus on enhancing its clarity, elegance, and overall impact. Preserve the essence and original meaning of the text, while meticulously adjusting its tone, vocabulary, and stylistic elements to resonate with the chosen style.Please improve the following text using the writing style of, maintaining the original meaning but altering the tone, diction, and stylistic elements to match the new style.Enhance the clarity, elegance, and impact of the following text by adopting the writing style of , ensuring the core message remains intact while transforming the tone, word choice, and stylistic features to align with the specified style.\"\"\"\n", + "\n", + "SAMPLE_OUTPUT_3 = \"\"\"Please improve this text using the writing style with maintaining the original meaning but altering the tone, ensuring the core message remains intact while transforming the tone, word choice, and stylistic features to align with the specified style.\"\"\"\n", + "\n", + "\n", + "def gen_prompt(og_text, rewritten_text):\n", + " \n", + " # Truncate the texts to first 200 words for now\n", + " # As we are having memory issues on Mixtral8x7b\n", + " og_text = truncate_txt(og_text, 256)\n", + " rewritten_text = truncate_txt(rewritten_text, 256)\n", + " \n", + " return f\"\"\" \n", + " Original Essay:\n", + " \\\"\"\"{og_text}\\\"\"\"\n", + " \n", + " Rewritten Essay:\n", + " \\\"\"\"{rewritten_text}\\\"\"\"\n", + " \n", + " Given are 2 essays, the Rewritten essay was created from the Original essay using the google Gemma model.\n", + " You are trying to understand how the original essay was transformed into a new version.\n", + " Analyzing the changes in style, theme, etc., please come up with a prompt that must have been used to guide the transformation from the original to the rewritten essay.\n", + " Keep your output concise, to the point(only the prompt), and less than a 100 words.\n", + " Make sure that the generated prompt is in the format of \"Please improve this text by [adding a magician].\".\n", + " \n", + " Sample output:\n", + " \\\"\"\"{SAMPLE_OUTPUT}\\\"\"\"\n", + " \"\"\"\n", + "\n", + "def gen_prompt_1(og_text, rewritten_text):\n", + " \n", + " # Truncate the texts to first 200 words for now\n", + " # As we are having memory issues on Mixtral8x7b\n", + " og_text = truncate_txt(og_text, 256)\n", + " rewritten_text = truncate_txt(rewritten_text, 256)\n", + " \n", + " return f\"\"\" \n", + " Original Essay:\n", + " \\\"\"\"{og_text}\\\"\"\"\n", + " \n", + " Rewritten Essay:\n", + " \\\"\"\"{rewritten_text}\\\"\"\"\n", + " \n", + " Given are 2 essays, the Rewritten essay was created from the Original essay using the google Gemma model.\n", + " You are trying to understand how the original essay was transformed into a new version.\n", + " Analyzing the changes in style, theme, etc., please come up with a prompt that must have been used to guide the transformation from the original to the rewritten essay.\n", + " Keep your output concise, to the point(only the prompt), and less than a 100 words.\n", + "\n", + " Sample output:\n", + " {SAMPLE_OUTPUT_1}\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "4fb6ca3a", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-16T04:02:34.393517Z", + "iopub.status.busy": "2024-04-16T04:02:34.393213Z", + "iopub.status.idle": "2024-04-16T04:02:34.409747Z", + "shell.execute_reply": "2024-04-16T04:02:34.408762Z" + }, + "papermill": { + "duration": 0.024571, + "end_time": "2024-04-16T04:02:34.412072", + "exception": false, + "start_time": "2024-04-16T04:02:34.387501", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
idrewrite_prompt
09559194Improve that text.
\n", + "
" + ], + "text/plain": [ + " id rewrite_prompt\n", + "0 9559194 Improve that text." + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "import gc\n", + "\n", + "device = 'cuda'\n", + "sub = pd.read_csv('/kaggle/input/llm-prompt-recovery/sample_submission.csv')\n", + "sub.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "3ddea7ca", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-16T04:02:34.423536Z", + "iopub.status.busy": "2024-04-16T04:02:34.423258Z", + "iopub.status.idle": "2024-04-16T04:02:34.429866Z", + "shell.execute_reply": "2024-04-16T04:02:34.428745Z" + }, + "papermill": { + "duration": 0.014872, + "end_time": "2024-04-16T04:02:34.432037", + "exception": false, + "start_time": "2024-04-16T04:02:34.417165", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "sub[\"rewrite_prompt\"] = str(SAMPLE_OUTPUT_1)" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "a94baeb2", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-16T04:02:34.443329Z", + "iopub.status.busy": "2024-04-16T04:02:34.443037Z", + "iopub.status.idle": "2024-04-16T04:02:34.446752Z", + "shell.execute_reply": "2024-04-16T04:02:34.445830Z" + }, + "papermill": { + "duration": 0.011656, + "end_time": "2024-04-16T04:02:34.448827", + "exception": false, + "start_time": "2024-04-16T04:02:34.437171", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "# tdf.loc[0,'id'] = 9559194\n", + "# tdf.head()" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "5ad38c02", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-16T04:02:34.459435Z", + "iopub.status.busy": "2024-04-16T04:02:34.459159Z", + "iopub.status.idle": "2024-04-16T04:03:14.870921Z", + "shell.execute_reply": "2024-04-16T04:03:14.870004Z" + }, + "papermill": { + "duration": 40.428665, + "end_time": "2024-04-16T04:03:14.882298", + "exception": false, + "start_time": "2024-04-16T04:02:34.453633", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "0it [00:00, ?it/s]2024-04-16 04:02:48.400315: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n", + "2024-04-16 04:02:48.400447: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n", + "2024-04-16 04:02:48.651645: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "FINAL: id rewrite_prompt\n", + "0 9559194 Please improve this text using the writing sty...\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "1it [00:40, 40.39s/it]\n" + ] + } + ], + "source": [ + "for row in tqdm(tdf.itertuples()):\n", + " try:\n", + " \n", + " query_prompt = gen_prompt(row[2], row[3])\n", + "# query_prompt = gen_prompt_sample(row[2], row[3])\n", + " \n", + " messages = [\n", + " {\n", + " \"role\": \"user\",\n", + " \"content\": query_prompt\n", + " }\n", + " ]\n", + "# messages = [\n", + "# {\n", + "# \"role\": \"user\",\n", + "# \"content\": query_prompt\n", + "# },\n", + "# {\n", + "# \"role\": \"assistant\",\n", + "# \"content\": f\"\"\"Sample prompt:\n", + "# \\\"\"\"{SAMPLE_OUTPUT}\\\"\"\"\n", + "# \"\"\"\n", + "# }\n", + "# ]\n", + "\n", + "# encoded_input = tokenizer(query_prompt, return_tensors=\"pt\").to(device)\n", + " inputs = tokenizer.apply_chat_template(messages, return_tensors=\"pt\").to(\"cuda\")\n", + "\n", + " with torch.no_grad():\n", + " encoded_output = model.generate(inputs, max_new_tokens=80, do_sample=True, pad_token_id=tokenizer.eos_token_id)\n", + "\n", + " decoded_output = tokenizer.decode(encoded_output[0], skip_special_tokens=True).replace(query_prompt, '').replace(\"[INST]\", \"\").replace(\"[/INST]\", \"\").strip()\n", + " \n", + " sub.loc[sub['id'] == row[1], 'rewrite_prompt'] = decoded_output.replace('Prediction:','').replace('prediction:','').replace('Sample Output:', '').replace('output:', '')\n", + "\n", + " print(\"FINAL: \", sub)\n", + "\n", + " torch.cuda.empty_cache()\n", + " gc.collect()\n", + "\n", + " except Exception as e:\n", + " print(e)\n", + " sub.loc[sub['id'] == row[1], 'rewrite_prompt'] = str(SAMPLE_OUTPUT_1)\n", + "# finally:\n", + "# if not (sub['id'] == row[1]).any():\n", + "# sub.loc[sub['id'] == row[1], 'rewrite_prompt'] = str(SAMPLE_OUTPUT_1)" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "d7c448b7", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-16T04:03:14.897616Z", + "iopub.status.busy": "2024-04-16T04:03:14.897068Z", + "iopub.status.idle": "2024-04-16T04:03:14.914367Z", + "shell.execute_reply": "2024-04-16T04:03:14.913510Z" + }, + "papermill": { + "duration": 0.026412, + "end_time": "2024-04-16T04:03:14.916428", + "exception": false, + "start_time": "2024-04-16T04:03:14.890016", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [ + "sub.to_csv(\"submission.csv\", header=True, index=False)" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "98685736", + "metadata": { + "execution": { + "iopub.execute_input": "2024-04-16T04:03:14.927925Z", + "iopub.status.busy": "2024-04-16T04:03:14.927626Z", + "iopub.status.idle": "2024-04-16T04:03:14.932756Z", + "shell.execute_reply": "2024-04-16T04:03:14.931858Z" + }, + "papermill": { + "duration": 0.012942, + "end_time": "2024-04-16T04:03:14.934830", + "exception": false, + "start_time": "2024-04-16T04:03:14.921888", + "status": "completed" + }, + "tags": [] + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Please improve this text using the writing style with maintaining the original meaning but altering the tone.\n" + ] + } + ], + "source": [ + "print(sub.iloc[0]['rewrite_prompt'])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2b5b40f3", + "metadata": { + "papermill": { + "duration": 0.004834, + "end_time": "2024-04-16T04:03:14.944781", + "exception": false, + "start_time": "2024-04-16T04:03:14.939947", + "status": "completed" + }, + "tags": [] + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kaggle": { + "accelerator": "nvidiaTeslaT4", + "dataSources": [ + { + "databundleVersionId": 7806901, + "sourceId": 67121, + "sourceType": "competition" + }, + { + "datasetId": 4281572, + "sourceId": 7369493, + "sourceType": "datasetVersion" + }, + { + "isSourceIdPinned": true, + "modelInstanceId": 3104, + "sourceId": 4309, + "sourceType": "modelInstanceVersion" + }, + { + "modelInstanceId": 3900, + "sourceId": 5112, + "sourceType": "modelInstanceVersion" + }, + { + "modelInstanceId": 4761, + "sourceId": 5994, + "sourceType": "modelInstanceVersion" + }, + { + "modelInstanceId": 8318, + "sourceId": 11382, + "sourceType": "modelInstanceVersion" + }, + { + "modelInstanceId": 8332, + "sourceId": 11394, + "sourceType": "modelInstanceVersion" + } + ], + "dockerImageVersionId": 30665, + "isGpuEnabled": true, + "isInternetEnabled": false, + "language": "python", + "sourceType": "notebook" + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.13" + }, + "papermill": { + "default_parameters": {}, + "duration": 935.378056, + "end_time": "2024-04-16T04:03:17.877087", + "environment_variables": {}, + "exception": null, + "input_path": "__notebook__.ipynb", + "output_path": "__notebook__.ipynb", + "parameters": {}, + "start_time": "2024-04-16T03:47:42.499031", + "version": "2.5.0" + }, + "widgets": { + "application/vnd.jupyter.widget-state+json": { + "state": { + "011010e92922479bb5d0acc06e881cae": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "20b00900fe9e4f1399cb3723233cf85d": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "2ff39fe902894bd8861c8797a6605e9f": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "6b1898fb2d0c42c092eac12f64e5c56e": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_710e4c53832c4f98ac8ab693ed8b12d7", + "placeholder": "​", + "style": "IPY_MODEL_8181f5d4c697481dab78d73d032ddf59", + "value": "Loading checkpoint shards: 100%" + } + }, + "710e4c53832c4f98ac8ab693ed8b12d7": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + }, + "8181f5d4c697481dab78d73d032ddf59": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "DescriptionStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "DescriptionStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "description_width": "" + } + }, + "aaf8b5844bad4225bf430b2ef5c636e4": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HTMLModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HTMLModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HTMLView", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_011010e92922479bb5d0acc06e881cae", + "placeholder": "​", + "style": "IPY_MODEL_20b00900fe9e4f1399cb3723233cf85d", + "value": " 19/19 [13:47<00:00, 41.26s/it]" + } + }, + "bd22e6276014410c9373d4f4110be289": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "FloatProgressModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "FloatProgressModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "ProgressView", + "bar_style": "success", + "description": "", + "description_tooltip": null, + "layout": "IPY_MODEL_2ff39fe902894bd8861c8797a6605e9f", + "max": 19.0, + "min": 0.0, + "orientation": "horizontal", + "style": "IPY_MODEL_e6ed2ec5c1324f9ba54250407a059fce", + "value": 19.0 + } + }, + "c72f3957ae1a43a89b59ff379fdb262e": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "HBoxModel", + "state": { + "_dom_classes": [], + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "HBoxModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/controls", + "_view_module_version": "1.5.0", + "_view_name": "HBoxView", + "box_style": "", + "children": [ + "IPY_MODEL_6b1898fb2d0c42c092eac12f64e5c56e", + "IPY_MODEL_bd22e6276014410c9373d4f4110be289", + "IPY_MODEL_aaf8b5844bad4225bf430b2ef5c636e4" + ], + "layout": "IPY_MODEL_fb9e17c9840c4142a1c390beb54e63c1" + } + }, + "e6ed2ec5c1324f9ba54250407a059fce": { + "model_module": "@jupyter-widgets/controls", + "model_module_version": "1.5.0", + "model_name": "ProgressStyleModel", + "state": { + "_model_module": "@jupyter-widgets/controls", + "_model_module_version": "1.5.0", + "_model_name": "ProgressStyleModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "StyleView", + "bar_color": null, + "description_width": "" + } + }, + "fb9e17c9840c4142a1c390beb54e63c1": { + "model_module": "@jupyter-widgets/base", + "model_module_version": "1.2.0", + "model_name": "LayoutModel", + "state": { + "_model_module": "@jupyter-widgets/base", + "_model_module_version": "1.2.0", + "_model_name": "LayoutModel", + "_view_count": null, + "_view_module": "@jupyter-widgets/base", + "_view_module_version": "1.2.0", + "_view_name": "LayoutView", + "align_content": null, + "align_items": null, + "align_self": null, + "border": null, + "bottom": null, + "display": null, + "flex": null, + "flex_flow": null, + "grid_area": null, + "grid_auto_columns": null, + "grid_auto_flow": null, + "grid_auto_rows": null, + "grid_column": null, + "grid_gap": null, + "grid_row": null, + "grid_template_areas": null, + "grid_template_columns": null, + "grid_template_rows": null, + "height": null, + "justify_content": null, + "justify_items": null, + "left": null, + "margin": null, + "max_height": null, + "max_width": null, + "min_height": null, + "min_width": null, + "object_fit": null, + "object_position": null, + "order": null, + "overflow": null, + "overflow_x": null, + "overflow_y": null, + "padding": null, + "right": null, + "top": null, + "visibility": null, + "width": null + } + } + }, + "version_major": 2, + "version_minor": 0 + } + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/Experiments/kaggle/llm_prompt_recovery/imgs/2nd_place_solution_architecture.png b/Experiments/kaggle/llm_prompt_recovery/imgs/2nd_place_solution_architecture.png new file mode 100644 index 00000000..53d1fab9 Binary files /dev/null and b/Experiments/kaggle/llm_prompt_recovery/imgs/2nd_place_solution_architecture.png differ