FluxML/Flux.jl v0.16.14 on GitHub

Flux v0.16.14

Merged pull requests:

Add a GPT-2 pretraining example (Reactant, HuggingFaceDatasets.jl) (#2754) (@CarloLucibello)
Integer class labels in crossentropy and logitcrossentropy (#2758) (@CarloLucibello)
Compute losses in Float32 for Float16/BFloat16 inputs (#2760) (@CarloLucibello)
README: standard example first, install/GPU/feature overview; CI: nightly non-blocking (#2763) (@CarloLucibello)
Fix the gradient rule of xlogx (#2764) (@CarloLucibello)
Don't copy BFloat16 input of a BFloat16 layer in _match_eltype (#2769) (@CarloLucibello)
Fused rule for logitcrossentropy with class labels (#2770) (@CarloLucibello)
Use NNlib.layernorm in the LayerNorm forward pass (#2773) (@CarloLucibello)
Fix a GC race in the Reactant compile-cache test (#2775) (@CarloLucibello)
Remove the ignored Greek keywords of focal_loss and tversky_loss (#2779) (@parth-patel010)
Build error messages lazily: first Conv 1.4 s → 0.4 s (#2783) (@CarloLucibello)
Let layers customize f16/bf16/f32/f64 with a 2-argument method (#2784) (@CarloLucibello)
Drop the Reactant override of logsumexp in the label loss (#2785) (@CarloLucibello)

Closed issues:

Cross-entropy losses with integer class targets (#2756)
Losses should compute in Float32 for Float16/BFloat16 inputs (#2757)
xlogx rrule refers to an undefined y, so scalar gradients throw UndefVarError (#2762)
_match_eltype copies the BFloat16 input of a BFloat16 layer (#2766)
logitcrossentropy with integer labels: 15 ms of memory traffic on large bf16 logits (#2767)
LayerNorm should use NNlib.layernorm in the forward (#2771)
focal_loss and tversky_loss silently ignore the γ, ϵ and β keywords (#2778)
First Conv construction spends ~1 s inferring repr(::Symbol) in _check_pad_mode (#2780)
Public interface for layers to customize f16/bf16 conversion (#2782)