Brace vs. bracket vs. sub-bracket — what each groups

Brace connects staves that ONE performer reads as a single continuous system — piano, harp, organ, and (by extension in choral engraving) a two-stave SATB choir short score. Bracket connects staves belonging to DIFFERENT performers who are nonetheless one conceptual group — an open-score choir (one stave per singer), a woodwind or brass section, or Gradus's fugue open-score voices. A sub-bracket (thin bracket nested inside a larger bracket) further groups instrument pairs within a bracketed family when the family has more than ~3 staves (e.g. the two horns within a 4-5-stave brass bracket).

An open-score fugue (one stave per voice, openScore=true) rendered with a BRACE instead of a bracket.

Gould (Behind Bars, pp. ~480-504, brace/bracket distinction including the choir short-score-gets-brace convention); Read (Music Notation) gives the same brace=single-performer / bracket=grouped-performers distinction; Ross (The Art of Music Engraving) covers orchestral sub-bracketing by family.

This rule can be decided from pitches, durations and voices — no layout required. A validator can check it before anything is drawn.