v0.6.0

[mattdangerw]

KerasNLP is adding experimental support for Jax and PyTorch backends on top of the Keras Core library. Read the anouncement, and browse the full library documentation, including how to specify the backend when running your code.

Support for both Jax and PyTorch is still experimental, expect some rough edges and please give us feedback!

Summary

• This release should be equivalent to 0.5.2 with the addition of multi-backend support.
• The following API symbols are currently restricted to the tensorflow backend:
  • keras_nlp.models.DebertaV3*
  • keras_nlp.models.FNet*
  • keras_nlp.metrics
  • keras_nlp.samplers.BeamSampler
  • keras_nlp.samplers.ContrastiveSampler
• Note that there are two ways you can run on top of Tensorflow.
  • If you run your scripts/colab without any changes, KerasNLP will use tf.keras for all layer and modeling implementations. This should be a no-op from previous releases of the library.
  • If you run your scripts/colab with KERAS_BACKEND={jax, torch, tensorflow}, you will be trying the new Keras Core library, using the specified backend. This is a great way to test out the future of the library!
  • Full details on runtime specification is available along with the Keras Core documentation.

What's Changed

• small updates to the release doc by @chenmoneygithub in small updates to the release doc #1031
• Sampler docstring edit by @abuelnasr0 in Sampler docstring edit #1033
• Fix program crash for id_to_token() method in SentencePieceTokenizer by @abuelnasr0 in Fix program crash for id_to_token() method in SentencePieceTokenizer #1040
• Update our release process to preview docs before release by @mattdangerw in Update our release process to preview docs before release #1043
• Add Whisper Tokenizer and Audio Feature Extractor by @abheesht17 in Add Whisper Tokenizer and Audio Feature Extractor #847
• Also strip padding token for opt by @mattdangerw in Also strip padding token for opt #1028
• Add regex dep by @mattdangerw in Add regex dep #1044
• Add BartSeq2SeqLM and conditional text generation with BART by @abheesht17 in Add BartSeq2SeqLM and conditional text generation with BART #974
• Support list/tuple inputs for special tokens in StartEndPacker layer by @abheesht17 in Support list/tuple inputs for special tokens in StartEndPacker layer #1045
• Support list/tuple inputs for special tokens in MultiSegmentPacker layer by @abheesht17 in Support list/tuple inputs for special tokens in MultiSegmentPacker layer #1046
• Fix a misleading part of our cached MHA docs by @mattdangerw in Fix a misleading part of our cached MHA docs #1048
• Always pass weight name by kwarg by @mattdangerw in Always pass weight name by kwarg #1053
• Always pass metrics in a list or dict by @mattdangerw in Always pass metrics in a list or dict #1054
• Move Defaults to to end of arg docstring and standardise values by @SamuelMarks in Move Defaults to to end of arg docstring and standardise values #1057
• Fix beam search for BART by @abheesht17 in Fix beam search for BART #1058
• Replace tf.dtype with "dtype" by @mattdangerw in Replace tf.dtype with "dtype" #1059
• Test shapes directly by @mattdangerw in Test shapes directly #1064
• Clean up metrics tests by @mattdangerw in Clean up metrics tests #1063
• Remove metrics merge tests by @mattdangerw in Remove metrics merge tests #1065
• Fix whisper feature inputs by @mattdangerw in Fix whisper feature inputs #1069
• Always specify shape when creating variables by @mattdangerw in Always specify shape when creating variables #1067
• Remove ragged support from position embeddings by @mattdangerw in Remove ragged support from position embeddings #1068
• Clean up dtype handling for preprocessing layers by @mattdangerw in Clean up dtype handling for preprocessing layers #1066
• Add BART finetuned on CNN+DM for summarisation by @abheesht17 in Add BART finetuned on CNN+DM for summarisation #1060
• Fix saving bug by @mattdangerw in Fix saving bug #1073
• Fix t5 forward pass by @mattdangerw in Fix t5 forward pass #1082
• Feat/make transformer decoder callable without causal mask by @ferraric in Feat/make transformer decoder callable without causal mask #1083
• Adding GPTNeoXBackbone by @shivance in Adding GPTNeoXBackbone #1056
• Add a common test case by @mattdangerw in Add a common test case #1095
• Update register_keras_serializable to use saving module by @mattdangerw in Update register_keras_serializable to use saving module #1094
• Don't test tf format by @mattdangerw in Don't test tf format #1104
• Add GPTNeoXPreprocessor by @shivance in Add GPTNeoXPreprocessor #1093
• Split layers into layers/modeling & layers/preprocessing by @mattdangerw in Split layers into layers/modeling & layers/preprocessing #1102
• Fix merge conflict from Split layers into layers/modeling & layers/preprocessing #1102 by @mattdangerw in Fix merge conflict from #1102 #1105
• Add a common base class for generative models by @mattdangerw in Add a common base class for generative models #1096
• Add GPTNeoXCausalLMPreprocessor by @shivance in Add GPTNeoXCausalLMPreprocessor #1106
• Add Whisper Presets by @abheesht17 in Add Whisper Presets #1089
• Refactor RotaryEmbedding and GPTNeoXAttention by @shivance in Refactor RotaryEmbedding and GPTNeoXAttention #1101
• Remove all the secret keys for ci by @mattdangerw in Remove all the secret keys for ci #1126
• Fix publish to pypi action by @mattdangerw in Fix publish to pypi action #1127
• Unexport models that are not in the 0.6 release by @mattdangerw in Unexport models that are not in the 0.6 release #1125
• Bump the version to 0.6.0 by @mattdangerw in Bump the version to 0.6.0 #1128

New Contributors

• @SamuelMarks made their first contribution in Move Defaults to to end of arg docstring and standardise values #1057
• @ferraric made their first contribution in Feat/make transformer decoder callable without causal mask #1083

Full Changelog: v0.5.2...v0.6.0

---

This discussion was created from the release v0.6.0.