transformers 4.14.0

v4.14.0: Perceiver, Keras model cards

on Python PyPI

Perceiver

The Perceiver model was released in the previous version:

Perceiver

Eight new models are released as part of the Perceiver implementation: PerceiverModel, PerceiverForMaskedLM, PerceiverForSequenceClassification, PerceiverForImageClassificationLearned, PerceiverForImageClassificationFourier, PerceiverForImageClassificationConvProcessing, PerceiverForOpticalFlow, PerceiverForMultimodalAutoencoding, in PyTorch.

The Perceiver IO model was proposed in Perceiver IO: A General Architecture for Structured Inputs & Outputs by Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch,
Catalin Ionescu, David Ding, Skanda Koppula, Daniel Zoran, Andrew Brock, Evan Shelhamer, Olivier Hénaff, Matthew M.
Botvinick, Andrew Zisserman, Oriol Vinyals, João Carreira.

• Add Perceiver IO by @NielsRogge in #14487

Compatible checkpoints can be found on the hub: https://huggingface.co/models?other=perceiver

Version v4.14.0 adds support for Perceiver in multiple pipelines, including the fill mask and sequence classification pipelines.

Keras model cards

The Keras push to hub callback now generates model cards when pushing to the model hub. Additionally to the callback, model cards will be generated by default by the model.push_to_hub() method.

• TF model cards by @Rocketknight1 in #14720

What's Changed

• Fix : wrong link in the documentation (ConvBERT vs DistilBERT) by @Tikquuss in #14705

• Put back open in colab markers by @sgugger in #14684

• Fix doc examples: KeyError by @ydshieh in #14699

• Fix doc examples: 'CausalLMOutput...' object has no attribute 'last_hidden_state' by @ydshieh in #14678

• Adding Perceiver to AutoTokenizer. by @Narsil in #14711

• Fix doc examples: unexpected keyword argument by @ydshieh in #14689

• Automatically build doc notebooks by @sgugger in #14718

• Fix special character in MDX by @sgugger in #14721

• Fixing tests for perceiver (texts) by @Narsil in #14719

• [doc] document MoE model approach and current solutions by @stas00 in #14725

• [Flax examples] remove dependancy on pytorch training args by @patil-suraj in #14636

• Update bug-report.md by @patrickvonplaten in #14715

• [Adafactor] Fix adafactor by @patrickvonplaten in #14713

• Code parrot minor fixes/niceties by @ncoop57 in #14666

• Fix doc examples: modify config before super().init by @ydshieh in #14697

• Improve documentation of some models by @NielsRogge in #14695

• Skip Perceiver tests by @LysandreJik in #14745

• Add ability to get a list of supported pipeline tasks by @codesue in #14732

• Fix the perceiver docs by @LysandreJik in #14748

• [CI/pt-nightly] switch to cuda-11.3 by @stas00 in #14726

• Swap TF and PT code inside two blocks by @LucienShui in #14742

• Fix doc examples: cannot import name by @ydshieh in #14698

• Fix: change tooslow to slow by @ydshieh in #14734

• Small fixes for the doc by @sgugger in #14751

• Update transformers metadata by @sgugger in #14724

• Mention no images added to repository by @LysandreJik in #14738

• Avoid using tf.tile in embeddings for TF models by @ydshieh in #14735

• Change how to load config of XLNetLMHeadModel by @josutk in #14746

• Improve perceiver by @NielsRogge in #14750

• Convert Trainer doc page to MarkDown by @sgugger in #14753

• Update Table of Contents by @sgugger in #14755

• Fixing tests for Perceiver by @Narsil in #14739

• Make data shuffling in run_clm_flax.py respect global seed by @bminixhofer in #13410

• Adding support for multiple mask tokens. by @Narsil in #14716

• Fix broken links to distillation on index page of documentation by @amitness in #14722

• [doc] performance: groups of operations by compute-intensity by @stas00 in #14757

• Fix the doc_build_test job by @sgugger in #14774

• Fix preprocess_function in run_summarization_flax.py by @ydshieh in #14769

• Simplify T5 docs by @xhlulu in #14776

• Update Perceiver code examples by @NielsRogge in #14783

New Contributors

• @Tikquuss made their first contribution in #14705
• @codesue made their first contribution in #14732
• @LucienShui made their first contribution in #14742
• @josutk made their first contribution in #14746
• @amitness made their first contribution in #14722

Full Changelog: v4.13.0...v4.14.0