Release notes for sbmlutils 0.12.0

We are pleased to release the next version of sbmlutils including the following changes. This release makes the fbc, distrib and comp packages round trip (#469), which completes the work 0.11.0 started for SBML core. sbml_to_model reads their content now, so a model read from a file keeps its flux bounds, its gene products, its uncertainties and its hierarchy when it is written again. That changes part of the python API, listed under breaking changes: code which uses one of these has to be changed. It also changes how some existing model definitions are written, listed under behaviour changes: the model definition stays as it is, but the SBML written from it differs.

Breaking changes

• Uncertainty.uncertSpans, the attribute, no longer exists: an uncertainty holds its parameters and its spans in one ordered list, uncertParameters, which is what makes the order of the document expressible. The uncertSpans= constructor argument is unchanged and its spans are the front of that list. Code which read uncertainty.uncertSpans reads uncertainty.uncertParameters and filters it with isinstance(child, UncertSpan). There is no compatibility property: it could only return a filtered copy, and an append to that copy would be lost in silence, which is worse than an AttributeError with this note behind it
• UncertParameter and UncertSpan no longer raise a ValueError when neither a value nor a variable is given, and an UncertSpan no longer raises for a bound it does not state: such an element is logged as an error instead. distrib requires neither, a parameter may carry only a math, a definitionURL or a nested list of its own, and the parser has to be able to hold everything libsbml reads
• ModelDefinition is a Model and takes Model's constructor, which is what lets it hold everything a model holds. There is no positional shim: a call with eight or more positional arguments binds differently, since the 8th, 9th and 10th used to be units, compartments and species and are packages, creators and model_units now. Pass them by keyword, which is the documented style. packages, model_definitions and external_model_definitions raise a ValueError on a model definition, at construction and when the model is written: all three belong to the document. A ModelDefinition is frozen like a Model, so the camelCase names the old class walked, md.externalModelDefinitions, md.modelDefinitions and md.replacedElements, raise an AttributeError; the third is replaced_elements
• a ModelDefinition is not the model of a document: Document(model=...) and create_model(model=...) raise a ValueError naming it. A model definition is written through the model_definitions of a Model
• ModelDefinition.create_sbml(model) no longer exists. A model definition is a Model and inherits Model.create_sbml(doc), which takes the libsbml.SBMLDocument the definition is created on, not the libsbml.Model it used to be written in; a call which passes a model is refused with a ValueError which says what the argument is, where it used to end in an AttributeError about createModelDefinition. A model definition is written by putting it in the model_definitions of a Model and writing that model
• keywords which were accepted and never had an effect are refused with a TypeError which names them: replacedBy= on Priority, GeneProduct, Objective, FluxObjective, UserDefinedConstraint, UserDefinedConstraintComponent, Uncertainty, KeyValuePair and LocalParameter, uncertainties= and keyValuePairs= on KeyValuePair, and keyValuePairs= on Document. libsbml attaches no comp plugin to the first eight of those elements, so no reader ever saw such a replacement, and no <comp:replacedBy> on a <localParameter> is valid; replace the element which contains it, i.e. the event, the model, the parameter or the kinetic law, which is written now, and put an uncertainty on the element a key-value pair belongs to
• Port.create_sbml writes no sboTerm when the port states none and portType is None. portType defaults to PortType.PORT, so a Port(...) without an sboTerm still gets SBO:0000599; only portType=None, which used to raise an UnboundLocalError, behaves differently
• three fixtures are removed from the packaged resources: resources/distrib/uncertainty_distribution.xml, uncertainty_uncertspan.xml and uncertainty_uncertvalue.xml. They use a pre-release distrib syntax which libsbml drops silently, so each of them read as a document without a single uncertainty. Nothing in the package, the tests, the examples or the documentation referenced them

Behaviour changes

• write_sbml and create_model create the parent directory of the file they are asked to write and raise an OSError naming the path when the write fails all the same. A write into a directory which does not exist used to write nothing; create_model then re-read the path, got an empty document, reported valid: TRUE and returned a result whose sbml_path does not exist. Validation is untouched and still reports rather than blocks
• an annotation resource whose collection the canonical form would lose is written exactly as it was read, and reported once per collection with the count and an example. http://identifiers.org/sabiork/1406 used to be written as the bare 1406, which resolves to nothing and names no collection; that is a loss in released 0.11.0. A resource of a collection the identifiers.org registry knows is normalized as before
• fbc:strict survives a round trip. Model has a strict field which is written and read; it was hardcoded to false when the document was created. A document which declares fbc version 1, which has no such attribute, is read without a strictness claim and written as fbc:strict="false": libsbml's own converter invents fbc:strict="true" for such a document, under which 11 of the 12 fbc version 1 cases of the SBML test suite become invalid
• the active objective is the one the document names: the parser reads fbc:activeObjective and sets Objective.active on each objective accordingly. The default of Objective.active is unchanged, so several authored objectives still leave the last one active
• the charge of a species is written as the fbc version of the document spells it, an integer in fbc version 2 and a double in fbc version 3. A charge which is not a whole number is reported and left unset in an fbc version 2 document, which has no place for it; it used to be written as 0
• a Compartment.spatialDimensions which is not a whole number round trips, e.g. 2.7; it used to come back as 0
• the package namespaces on <sbml> are written in a fixed order, comp, distrib, fbc. They followed the iteration order of a set before, which depends on the hash seed of the process, so the same model definition could write two different files in two runs
• Uncertainty(formula=...) takes the distribution of the value from the parsed formula, i.e. from the function the formula calls at its top level, instead of searching the text of the formula for the name of a distribution. Against 0.11.0 this changes five kinds of input: rayleigh(0.5) is written with the rayleigh definitionURL and its math, where the name was misspelled and the parameter came out empty; and normalization * 2, gamma_rate + 1, 5 * normal(0, 1) and a formula which does not parse at all get no definitionURL and no math and are reported, where each of them used to be written as the distribution whose name it happens to contain. Every other formula is unchanged, lognormal(0, 1) included
• key value pairs and user defined constraints are written only into a document which declares fbc version 3. In an fbc version 2 document, or in one which declares no fbc, nothing is written and the loss is reported once per kind of content for the whole document, with how many pieces on how many elements, an example and the version to declare, the way an attribute the document has no place for is reported; such a document used to get an <fbc:listOfKeyValuePairs> of empty <fbc:keyValuePair/> elements and an empty, invalid <fbc:userDefinedConstraint>, and a document which declares no fbc raised an AttributeError which took the whole file with it. A charge which fbc version 2 cannot express stays one error per species: it is a refused value, different on each of them
• a model declares the packages its content needs, in the main model as it already did in a model definition: gene products, an objective, key value pairs or an uncertainty declare fbc or distrib. Such a model without packages=[...] used to write nothing of it or fail with an AttributeError. A version which the caller states is not raised by the content
• a <comp:port> names an element by comp:metaIdRef where the document has no id for it: an initial assignment, an assignment rule, a rate rule, an algebraic rule, an event assignment and a local parameter always, and a kinetic law, a trigger, a priority, a delay, a constraint and a key value pair below SBML Level 3 Version 2, which is what create_model writes by default. libsbml resolves a comp:idRef with Model.getElementBySId, which answers with none of them, so such a port never validated. The id of a port which references by metaid is derived from that metaid, so two local parameters of the same id in two kinetic laws no longer produce two ports of the same id; an element which has no metaid where the document needs one is reported and gets no port, and it no longer makes the model declare an empty comp namespace
• every attribute libsbml refuses is reported: 60 libsbml setter calls of factory.py answered with a failing status and said nothing, and their status is checked now, which is 94 checked calls against the 34 of 0.11.0. An attribute which the SBML level and version, or the version of a package, has no place for at all is a different case and is reported once per kind of element and attribute, with the count, an example and what to write to keep it, rather than once per element; the per-element detail is logged at debug
• the core id and name of a ReplacedElement, a ReplacedBy and a nested sBaseRef are not written by libsbml into any document, measured at Level 3 Version 1 and at Version 2, so they are no longer set and a value given for either is reported once per document and per kind of element, without advice. The report used to say "Write SBML Level 3 Version 2 to keep it", which does not keep it, and at Level 3 Version 2 the loss was silent. sid is optional on all three classes now, in the same position, so a comp author no longer has to state an id which is thrown away; elementRef and submodelRef of a replacement keep their position and default to the empty string, so a replacement which states no elementRef is refused when it is written, with the ValueError which names an elementRef the model has no element for. The comp:id and comp:name of a Port and of a Deletion are unaffected
• Uncertainty.formula is a property. It is normalized into the uncert parameter of the type distribution which the uncertainty is written from, on an assignment as well as in the constructor: u.formula = "uniform(0, 1)" after Uncertainty(formula="normal(0, 1)") used to be kept in the field, written nowhere, and normal was written instead, in silence. Setting it to None removes the parameter the formula stands for and leaves every other child of the uncertainty alone
• a model which states strict declares fbc for it, True and False alike. Model(strict=True) without any other fbc content wrote no fbc namespace, no fbc:strict and no report at all; a model which states neither leaves strict at None and declares nothing. A ModelDefinition is not counted, since libsbml cannot write fbc:strict on a <comp:modelDefinition>
• sbml_to_model reports nothing for a document which libsbml reads without an error. A <distrib:uncertParameter type="mean"/> was logged as stating nothing about the value, and an uncert span with one bound as missing the other: both are hints about a model definition being written, and distrib requires neither, so the reader of a file was told to fix the file it had just read
• an element whose fbc content needs a plugin the document does not have is refused with a ValueError which names the element, the package and why the package is absent, the way a comp port already was: the flux bounds of a reaction, a gene product and an objective dereferenced None and ended in AttributeError: 'NoneType' object has no attribute 'setUpperFluxBound' at sbml_level=2, and a Deletion whose submodelRef names no submodel in one about createDeletion, which names the submodel now. The charge and the chemical formula of a species are still reported rather than raised, and no longer advise packages = ['fbc'], which a model with a charge declares by itself
• UnitDefinition.get_uid_for_unit raises a ValueError naming the value for a unit which is neither a UnitDefinition nor the id of one. It used to surface as a SWIG TypeError from setUnits, which names neither the value nor the element it was set on
• merge_models copies the unit definitions of the models it merges, like every other merged list. Changing a unit definition of the merged model used to change the model it was merged from
• the authoring hints, 'name' should be set and 'sboTerm' should be set, are held in a ContextVar, so writing a parsed model in one thread no longer suppresses the hints of a model written in another
• the sboTerm of a reactant, a product or a modifier takes either spelling, SBO:0000011 and SBO_0000011, like every other element; libsbml refused the underscore form on a species reference while accepting it on a species
• an objective which omits the required fbc:type is read as maximize with an error in the log, where sbml_to_model used to raise a ValueError and read nothing at all
• an external model definition without a modelRef is written without that attribute, instead of passing the empty string to libsbml, which refused it
• two packaged example files were regenerated, because neither could round trip as it was: resources/examples/fbc/fbc_key_value_pair.xml, whose key value pairs were three empty elements, and resources/examples/fbc_user_defined_constraints.xml, which spelled the coefficients of its components as numbers, which is invalid fbc version 3 with 13 validation errors and which libsbml refuses to write

Features

• the fbc, distrib and comp packages round trip (#469): SBML -> sbml_to_model -> create_model -> SBML preserves their content, where sbml_to_model used to read none of it and a round trip of e_coli_core discarded its 137 gene products, 69 gene product associations, 95 flux bounds, 72 charges, 72 chemical formulas, its objective and fbc:strict, and a round trip of icg_body.xml its submodel, its 16 ports, its 6 replaced elements and its external model definition, see Reading and writing
• fbc: flux bounds, objectives and flux objectives with the type, the coefficients and which objective is active, gene products with their label and associated species, the gene product association of a reaction as an infix string of gene product ids, the charge and the chemical formula of a species, user defined constraints with their components, and key value pairs on every element, see Flux balance constraints
• distrib: the uncertainties of every element with their uncert parameters and spans in the order of the document, each with its type, value, variable, bounds, unit, definitionURL, math and the nested parameters of an external distribution, and the distribution functions in a formula, see Distributions and uncertainties
• comp: submodels with their time and extent conversion factors, ports, deletions, replaced elements and replacedBy with the whole nested sBaseRef chain below them, model definitions with everything inside them, core as well as fbc and distrib, and external model definitions, which are preserved as the reference they are and never resolved, see Model composition
• Model(strict=...) writes fbc:strict, which says that every reaction of the model has constant bounds with a value and every species reference is constant
• UncertParameter and UncertSpan are Sbase subclasses, so each carries its own id, name, metaid, sboTerm, notes and annotations, and both take definitionURL=, math= and uncertParameters=, the nested parameters of an external distribution. Uncertainty.uncertParameters takes parameters and spans in one list and writes them in the order they are given in, which is what preserves the order of a parsed document; uncertSpans= is unchanged and its spans are written in front. The types distribution and externalParameter, which an uncertainty refused before, are written
• SbaseRef.sBaseRef holds the nested <comp:sBaseRef> of a port, a replaced element, a replacedBy or a deletion, which continues the reference into a submodel of the submodel it names, to any depth. The SBML test suite nests it three levels deep
• ModelDefinition is a Model, so a <comp:modelDefinition> holds everything a model holds, its unit definitions included, and the document declares the packages the content of a model definition needs
• the key value pairs of a reactant, a product and a modifier are written; EquationPart.keyValuePairs was declared and never used. libsbml 5.21.2 writes them for all three and reads them back only for a modifier, which is recorded for an upstream report
• a port is written for a kinetic law, a local parameter, an event and its trigger, priority, delay and assignments, a constraint, an uncertainty, a gene product, an objective, a flux objective, a user defined constraint and its components and a key value pair. Each of them accepted a port, dropped it, and made the document declare an empty comp namespace for it
• a KeyValuePair writes its id, name, metaid, sboTerm, notes and annotations, not only its key, value and uri, and the uncertainties of a kinetic law and of a local parameter and the replacedBy of a kinetic law are written

Fixes

• the gene pre-check of a reaction stripped and, AND, or, OR and the parentheses out of the association string with a chain of str.replace, so a gene product whose label contains those letters was mangled, ORF1 became F1, and a GeneProduct missing in model was logged for a gene the model declares. The association is tokenized now
• UserDefinedConstraintComponent dropped a variableType given as the libsbml constant FBC_VARIABLE_TYPE_LINEAR, which is 0 and therefore falsy, and with it the default of UserDefinedConstraint, which is that constant and is passed on to every component. The string "linear" was never affected
• Submodel(sid=...) without a modelRef is written with the reference unset and one error naming it, where setModelRef(None) raised a TypeError out of SWIG
• Port(portRef=...) is reported: comp does not allow a <comp:port> to reference another port and libsbml answered every such call with a failure, silently
• a distribution named rayleigh in an Uncertainty(formula=...) was spelled raleigh, so the uncert parameter was written without its definitionURL and without its math
• comp.flatten_sbml leaves the working directory where it found it when the flattening raises. It changes into the directory of the document so that libsbml resolves a comp:source relative to it, and changed back only on the way out, so a caller whose document could not be flattened was left in another directory, with every relative path of its own pointing somewhere else

Documentation

• Reading and writing describes what the packages preserve, how the three verification layers measure it, on which corpus, and what the round trip does not keep
• Flux balance constraints, Distributions and uncertainties and Model composition describe what each package keeps and what it does not, and the new fields of this release
• Validation describes what writing a model reports, which no validation of the result can show
• Annotations describes which annotation resources are written as they were given, and why

Development

• tests/structural.py compares the fbc, distrib and comp content of two documents attribute by attribute and returns every difference. Simulation and validation are blind to all three packages, so this is the measure of the package round trip; its module docstring is the comparison policy and every package test is judged by it. scripts/package_report.py sweeps the corpus with it, one process per case, and prints what every construct preserves, as scripts/roundtrip_report.py does for simulation
• tests/test_package_semantics.py checks the two packages which have a semantics an independent tool can judge: a comp model and its round trip are flattened and simulated with roadrunner and their trajectories compared, an fbc model and its round trip are read as constraint based models with cobrapy and their stoichiometry, bounds, objective, gene reaction rules and flux balance solution compared. Both comparisons are shown to see a damaged round trip
• the tox environment cobra installs the optional cobra extra and runs the tests which need cobrapy, which would otherwise skip in every environment and verify nothing. It has a job of its own in the CI workflow, which informs and is not part of the required checks
• Model._keys, which merge_models reads to decide how a field is merged, is derived from the field annotations of Model instead of being a hand-maintained copy of them, with the two deliberate exceptions in an explicit override table
• what follows -- in a tox run is passed on to pytest, so tox r -e py3.14 -- tests/test_factory.py runs one module in the environment continuous integration uses; the path used to be dropped and the whole suite ran

Your sbmlutils team