03 · The trunk: where the water is stored

The baobab stores up to thousands of litres of water in its trunk and gets through months of drought. Baobáxia's core keeps saberes in plain files that survive power cuts, swapped disks and years without a network.

Files to read, in this order

1. baobaxia/core/saber.py — the model (≈ 250 lines).
2. baobaxia/infra/storage.py — reading and writing saber.bbx.
3. baobaxia/core/repository.py — SaberRepository: one galaxy.
4. baobaxia/core/index.py — GalaxyIndex: the SQLite cache.
5. baobaxia/core/relation_graph.py and core/registry.py.

The saber.bbx file

---
uuid: 9f8e7d6c-1b2a-4c3d-8e9f-0a1b2c3d4e5f
nature: midia
slug: pescaria
name: Pescaria no rio
galaxia: 201b8213-…
mucua: 373cc7ef-…
creator: 4890f893-…
created: '2026-05-02T14:10:00'
visibility: public
applications: [acervo]
data:
  title: {pt: Pescaria no rio, en: Fishing on the river}
  tags: [pesca, rio]
  protecao: publico
content:
- {filename: foto.jpg, mimetype: image/jpeg, size: 182044,
   hash: SHA256E-s182044--3f2a….jpg}
---
Free markdown text (becomes data.body).

Readable by people, versionable by git, and foto.jpg is a git-annex symbolic link: the binary only exists in the mucuas that asked for a copy.

SaberRepository: one galaxy

classDiagram
    class SaberRepository {
        +Path root
        +GalaxyIndex index
        +create(saber, author, commit) Saber
        +upsert_saber(saber) Saber
        +update(uuid, data, …) Saber
        +delete(uuid) bool
        +get(uuid) Saber
        +add_contents(uuid, files) Saber
        +reindex(incremental) int
        +reindex_saber(uuid) bool
        +refrescar_presenca() int
        -_saber_dir(saber) Path
        -_merge_content(dir, refs) list
        -_index_entry(saber) dict
        -_load(dir) Saber
        -_commit(msg, author)
    }
    class GalaxyIndex {
        +upsert(entry)
        +upsert_many(entries) int
        +aplicar_lote(entradas, remover, marcador)
        +get(uuid) dict
        +query(nature, tag, text, is_local, …) dict
        +assinaturas() dict
        +ultimo_commit_indexado() str
        -_criar_acesso()
    }
    class storage {
        <<module>>
        +read_bbx(path) (meta, body)
        +write_bbx(path, meta, body)
        +list_content_files(dir) list
    }
    class Sankofa {
        <<facade>>
        +init(path)
        +add(path, root)
        +commit(root, msg)
        +sync(root, remote, data) bool
        +get / drop / whereis
    }
    SaberRepository *-- GalaxyIndex
    SaberRepository ..> storage
    SaberRepository ..> Sankofa

Writing a saber (create)

sequenceDiagram
    participant S as SaberService
    participant R as SaberRepository
    participant St as storage
    participant G as Sankofa (git)
    participant I as GalaxyIndex
    S->>S: permissions.require(CREATE) and validate_data (the nature schema)
    S->>S: authorship.try_sign (signs with the author's key, if any)
    S->>R: create(saber, author)
    R->>R: nature default applications, slug, folder
    R->>R: _merge_content (declared refs + disk)
    R->>St: write_bbx(saber.bbx, to_metadata(), body)
    R->>G: add(saber.bbx) and commit("create midia: pescaria")
    R->>I: upsert(_index_entry(saber))
    S->>S: _pos_escrita: invalidates the graph and propagates to copies in other galaxies

The SQLite index (core/index.py)

A wide table (saberes), auxiliary tables and a narrow acesso table kept up to date by triggers — listings check permission for thousands of rows without reading the JSON blobs.

erDiagram
    saberes {
        TEXT uuid PK
        TEXT nature
        TEXT slug
        TEXT name
        TEXT galaxia
        TEXT mucua
        TEXT creator
        TEXT visibility
        TEXT created
        TEXT rel_path
        REAL lat
        REAL lon
        TEXT territorio
        TEXT busca "lowercase, no accents"
        TEXT meta_json
        TEXT data_json
        TEXT fs_sig "mtime:size"
        INTEGER is_local
    }
    acesso {
        TEXT uuid PK
        TEXT protecao
        TEXT leitores
        TEXT editores
        TEXT status
        INTEGER ok "valid json"
    }
    tags { TEXT uuid TEXT tag }
    tipos { TEXT uuid TEXT tipo "imagem video audio pdf…" }
    applications { TEXT uuid TEXT application }
    fts { TEXT uuid TEXT text "FTS5" }
    meta { TEXT chave PK TEXT valor "ultimo_commit_indexado" }
    saberes ||--|| acesso : "trigger"
    saberes ||--o{ tags : ""
    saberes ||--o{ tipos : ""
    saberes ||--o{ applications : ""
    saberes ||--|| fts : ""

Details worth reading:

• Separate readers (_Leitores, _leitura("rapida"|"pesada")): a quick query (by uuid) never waits behind a heavy listing.
• geracao(): a global write counter; the read cache (services/cache_leitura.py) uses this number as its key — any write invalidates everything at once.
• Accent-insensitive search: normalizar_busca (NFKD + lowercase) on both sides. "acai" finds "açaí".

Incremental reindex: spiral time

reindex(incremental=True) asks git what changed since ultimo_commit_indexado (_pastas_mudadas_no_git) and rereads only those folders. Without a marker, it compares the disk signature (fs_sig). The marker is written in the same transaction as the data: if it got ahead, saberes would disappear for good.

flowchart TD
    A[incremental reindex] --> B{marker exists and is<br/>an ancestor of HEAD?}
    B -- yes --> C[git diff marker..HEAD<br/>→ folders touched]
    C --> D[rereads only those folders<br/>+ rows with empty fs_sig]
    B -- no --> E[scans saber.bbx on disk<br/>opens only those with a different fs_sig]
    D --> F[aplicar_lote: entries + removals + marker<br/>in one transaction]
    E --> F
    F --> G[unreadable saber.bbx?<br/>keeps the previous version and logs a warning]

The presence of a binary (git annex get/drop) does not change any versioned file — that is why refrescar_presenca() exists, doing one listdir per folder that has attachments.

Practice

1. In test_reindex_incremental.py, find the test that counts how many saber.bbx files were reread. Change a saber and check the number.
2. Corrupt the YAML of a saber.bbx by hand (as a merge conflict would) and run bbx reindex. Does the saber vanish from search? Why not? (_load, reindex.)
3. Open the index with sqlite3 .bbx/index.sqlite and run SELECT nature, count(*) FROM acesso GROUP BY nature;.
4. Read _index_entry and explain where the territorio column comes from for a saber that did not declare a territory.

To talk about

• The index can be deleted at any time. What freedom does that give the people who maintain the system? What care does it demand?