creasty/defaults v1.11.0

on GitHub

v1.10.0 ended with six known issues. This release fixes three of them — data with a cycle crashed the process, a SetDefaults ran after an unmarshaler had taken the tag, and a failed default left part of itself behind — and half of a fourth: Set no longer writes to a map of slices or maps. The other two, and the rest of the fourth, stay as they are, now documented as intended. The API is unchanged, and so is go.mod.

Behavior changes come first, as before. Two of them can change what a working program does, and neither announces itself: one stops a SetDefaults call, and the other changes what Set leaves behind when it returns an error.

[BREAKING] SetDefaults is skipped behind a pointer once an unmarshaler took the tag (#97)

The README says that when a tag is handed to UnmarshalText, SetDefaults is not called. v1.10.0 made that hold for a struct, behind a pointer or not, and listed the rest as a known issue: a pointer to any other type still got SetDefaults after its unmarshaler took the tag. It no longer does:

type Level int

func (l *Level) UnmarshalText(b []byte) error {
	n, err := strconv.Atoi(string(b))
	if err != nil {
		return err
	}
	*l = Level(n)
	return nil
}

func (l *Level) SetDefaults() { *l += 100 }

type Config struct {
	Level *Level `default:"3"` // v1.10.0: 103. v1.11.0: 3.
}

Nothing reports the missing call. How to tell whether you are affected: look for a type that is not a struct — a named int, string or slice — with both SetDefaults and UnmarshalText or UnmarshalJSON, behind a pointer a tag allocates. If its SetDefaults adjusted what the unmarshaler produced, that adjustment is gone; make it in the unmarshaler instead. A field of the type itself never got the call, and a pointer the caller allocated is still left alone.

[BREAKING] A failed default no longer leaves part of itself behind (#98)

When a default failed, Set returned the error but kept what it had filled on the way down. The value was then no longer zero, so a second Set could skip the tag and return nil, which v1.10.0 listed as a known issue for *scalar, *[]T and *map fields. Now each value Set found zero on the way to the failing default is put back to zero, whatever part of the default it had taken:

Only values that were zero are put back. A pointer, slice or map the caller provided is kept, and the fields of the struct passed to Set keep the defaults they took before the failure: there, A stays 7. The *uuid.UUID row in the v1.10.0 notes changes the same way, to an error with the pointer left nil.

Who is affected: code that ignores Set's error, or recovers from MustSet's panic, and then uses the value. Where it found an allocated pointer it now finds nil, so reading through it panics; where it found part of a default it finds a zero value; and a second Set returns the error again, where it could return nil.

Data with a cycle ends instead of crashing the process (#104, #108)

Set followed wherever the caller's data pointed, so data with a way back to itself, such as a child pointing up to its parent, was walked until the stack overflowed. That is a fatal error rather than a panic: no recover catches it, and the process exits.

type Node struct {
	Name     string `default:"node"`
	Parent   *Node
	Children []*Node
}

root := &Node{}
root.Children = []*Node{{Parent: root}}
err := defaults.Set(root)
// v1.10.0: fatal error: stack overflow
// v1.11.0: err is nil, both nodes are named "node", and the parent pointer is kept

Set now keeps track of the structs, slices and maps on the path it is walking. Where the data leads back to one of them, it goes no further, and the walk already under way finishes that value, so a struct gets no second SetDefaults from its own cycle. Down to 64 values deep the check scans the path and allocates nothing; below that it builds an index of the path and looks values up in it, so deep data costs a few allocations rather than time growing with the square of the depth (#108). See Performance.

A walk that finished in v1.10.0 finishes the same way, with one exception: if a SetDefaults below a repeated value broke the cycle as a side effect, say by clearing the pointer back, v1.10.0 walked that value a second time, and this release does not. The check covers the current path only, so a value that two paths reach, neither through the other, is still walked on both, as described below.

Set no longer writes to a map of slices or maps (#105)

A map value is not addressable, so Set fills a copy and stores it back under its key. It stored back every struct, slice and map value it walked, changed or not, which v1.10.0 listed as a known issue: a goroutine reading the map meanwhile raced with Set, and could crash. A slice or map copy shares its array or table with the value in the map, so storing it back never changed anything, and it no longer happens:

The same holds for a map of maps. A struct value is still stored back, changed or not, so a map of structs still must not be read during Set; that is now documented as intended, below.

An int64 default neither parser takes reports both reasons (#96)

An int64-kinded field is offered to time.ParseDuration and then to strconv.ParseInt, since reflection cannot tell a time.Duration from a plain int64 (#66). When both rejected the tag, only ParseInt's error was reported, so a duration with a unit Go does not know read as bad integer syntax:

The field X: invalid default "…": prefix is unchanged, and on a plain int64 the duration half is noise, the cost of the shared parser. A type whose own unmarshaler rejected the tag still reports only that rejection (#90). Both errors are wrapped, so errors.As still reaches the *strconv.NumError and errors.Is still matches strconv.ErrSyntax and strconv.ErrRange, but errors.Unwrap no longer reaches it in one step: errors.Unwrap(err).(*strconv.NumError) stops matching. An empty tag is still no error, and no longer pays for an error message it throws away (#107).

Kept as they are, and now documented

Three behaviors that look wrong were weighed and kept. The README now describes each, and a test marked // QUIRK or // BUG pins it, so changing one later means flipping that test:

• A value reachable by more than one path is filled on each path, with a SetDefaults call on each, as v1.10.0's known issues said. #99 filled such a value once per call, at the cost of an allocation on every Set that entered a value, and was closed in favor of #103 and #104. A SetDefaults that is not idempotent applies once per path, and a chain of values each shared by two pointers takes time that doubles with every link.
• A struct held as a map value is stored back under its key, changed or not, so do not call Set while another goroutine reads a map of structs it walks. #101 stored a struct back only when it changed, at the cost of an allocation for each struct a tag or setter touched, and was closed in favor of #105.
• A pointer the caller allocated to a slice, a map or a pointer is not descended into. What a caller's *[]T, *map[K]T or **T holds gets no defaults, and a default that would fail there is not reported, though a *[]T or *map[K]T held as a map value is descended into, as it has been since v1.6.0. #100 descended into all three and was closed in favor of #106, which pins the skip: nobody had asked for the descent, and it would bring new errors and new cycles.

#95 pins more behavior no test covered, marking what looks wrong // QUIRK, and corrects documentation that described behavior Set does not have:

• An integer tag is a Go integer literal, legacy octal included: default:"0644" is 420, and default:"08080" is an error. Slice and map tags are JSON, where a number rejects a leading zero and an integer map key "010" is 10.
• A tagged field promoted from an unexported embedded struct stays empty, and a map entry under a NaN key gets no defaults.
• {} and [] are never handed to UnmarshalJSON directly, though encoding/json may call it while parsing the other literal.
• The README said a pointer to the zero value is preserved. A pointer to a zero scalar is, but a pointer to a struct is descended into, so the struct's zero fields still get their defaults.

API and minimum Go version: unchanged

Set, MustSet, CanUpdate, Setter and ErrInvalidType are as in v1.10.0. go.mod still says go 1.22, and CI tests 1.22, 1.26 and 1.27.

Performance

Measured with make bench-compare BASE=v1.10.0, the suite from #102: 6 interleaved rounds at 400ms, Go 1.26.5, darwin/arm64 (M1 Max). Every difference shown is significant at p < 0.05. Allocations are unchanged on every row but those named below.

Faster: slices and maps of scalars. Set walked every element of a slice and every entry of a map whatever their type, though only a struct, pointer, slice or map element can hold a default. It now skips the rest (#103). Tagged fields also got cheaper, since the tag is offered to a field's unmarshalers through one interface conversion rather than two (#110):

Set on four scalar fields is unchanged, at 547 ns against 538 ns (p=0.18).

Slower: walking values that could hold a default, by 3 to 14%. Each struct, slice and map Set enters now checks the path above it for a cycle (#104), and each field it visits takes one more call (#98):

Deep data costs a little more, and a few allocations. Down to 64 values deep the cycle check scans the path; below that it builds an index of the path, which is what keeps the cost from growing with the square of the depth (#108). Set on an already-filled linked list of structs, one tagged field each, measured with testing.Benchmark outside the committed suite (the last row is a single call):

A slice of pointers to structs runs in one of two modes. A filled []*T of 1000 elements takes about 187 µs in some processes and about 313 µs in others, where v1.10.0 took 183 µs in every process. A process picks a mode as it starts and keeps it for its whole run. Over twelve fresh processes each, v1.10.0 was fast in twelve, this release's #111 was fast in twelve, and #112 — which only stopped aliasing a struct field to a local — was fast in four. That change removes no work and alters no behavior: the cost is in the machine code the compiler generates around it. The alias stays gone by choice, since keeping a local to steer code generation would argue for hoisting the tag's other fields into locals too. A map of slices moves with it, at 30 µs against 38 µs.

Parse/unmarshaler/text reads 23% faster than on v1.10.0. That is code placement rather than work removed: the same row moved as far with dead code added to an unchanged tree (#108).

Documentation

• The README gains Integers, Unmarshalers, Shared and cyclic values, Maps and Errors sections, and its pointer paragraph now says which pointers Set descends into. (#95, #97, #98, #104, #105, #106)
• Set's doc comment says what it fills, what it descends into, what it writes to a map and what a failure leaves behind. Setter's says SetDefaults is called once for each path to its struct, but not again from a cycle.
• The README no longer promises a runnable MustSet example; the package example calls it. (#95)

Internals

• benchmark_test.go is a suite of 46 cases: whole calls, the zero check, each parse path, walks with nothing left to fill, and failures. make bench-compare BASE=<ref> runs it against another commit in interleaved rounds and compares them with benchstat, and CI runs every case once on each Go version. Every number under Performance but the deep-nesting table comes from it. (#102, #108)
• Three pieces of machinery the public API reaches only through Set now live in files of their own, with their tests beside them in package defaults: the promoted-method check (method.go), the unmarshaler handoff (unmarshal.go) and the path check and its index (path.go). They are the only exception to this repository's black-box test rule, since nothing else can reach them. (#109, #110, #111)
• path.go imports unsafe to hold each value on the path as an unsafe.Pointer, which keeps that value alive so nothing allocated below it can reuse its address, and its index keys hold addresses as uintptr so that entries stay off the heap. There is no pointer arithmetic. (#104, #108)
• Statement coverage stays at 100%, and make cover now measures ./... rather than the root package alone. (#109)

Upgrading

Run your tests. If a value that a SetDefaults used to adjust comes back as its unmarshaler left it, see the first section. If code carries on after Set returns an error, expect nil and zero values where part of a default used to be. If you match the text of errors from an int64 or time.Duration field, they now name both parsers.

Full Changelog: v1.10.0...v1.11.0