Building a custom client

A practical reference for implementing your own client that talks to your own Totems over BLE, the ESP-NOW mesh, and a custom OTA server — no keys required.

This page collects everything a from-scratch client needs to talk to Totems you own: the BLE GATT link the phone app uses, the ESP-NOW mesh frame, a drop-in OTA server, and the chunked-transfer header. Every value here is verified against the recovered firmware_v5.0.3.bin image (and unchanged from v5.0.2 unless marked). The BLE parts are also tested on hardware. Confidence is graded Confirmed / Inferred / Partial throughout.

No cryptographic secrets are required

The single most important fact for a client author: you need no keys, no pairing, no signatures, and no shared secret to interoperate. Nothing on the application path is authenticated or encrypted (Confirmed).

Path	Protection a client must handle
ESP-NOW (broadcast and unicast)	none — entirely cleartext, no PMK/LMK ever programmed
BLE GATT	none — no pairing/bonding/encryption; an unpaired central can read and write
Application messages (BLE + mesh)	none — no token, HMAC, challenge/response, or signature anywhere
OTA	integrity only — SHA-256 hash, no firmware signature, fetched over plain HTTP

The ESP-NOW ROM module contains the concept of encryption (set_pmk, lmk, encrypt), but no frozen Python module references those symbols, and add_peer(mac) is always called with a single positional argument, so lmk=None and encrypt=False. The only crypto in the image is the bundled (and, on the app path, unused) mbedTLS stack plus the OTA SHA-256. ble_keys.bin holds optional BLE reconnect secrets — not required to connect.

On the BLE side this is checkable directly: BleLite.enable calls ble.active(True) and nothing else, and the only ble.config() call anywhere in the 94 frozen modules is ota_ble’s mtu=512 — no bond, mitm, le_secure or io option is ever set, and the IRQ handler has no encryption or passkey branch (Confirmed).

BLE client

The Totem is a BLE peripheral; your client is the central. No pairing is required. A working reference implementation in Go, totemctl, lives in this repository (protocol/, client/, cmd/totemctl/). It has been verified against a real device on firmware 4.1.3 and 5.0.3.

Connect

Bluetooth is off until the user double-presses the power button; the crystal breathes blue while it advertises. The primary advertisement carries the flags, the 128-bit service UUID and the local name totem; the appearance spills into the scan response, so a passive scan sees the name and UUID but not the appearance (Confirmed — see advertising). Advertising interval is 50 ms.

Item	Value
Service UUID	7913b588-0000-4635-b066-baa2cfc197cf
Advertised name	totem (appearance 1361, in the scan response)
Pairing / bonding	not required

ble_core.py registers these characteristics under the service, all with GATT flags 0x3E (READ | WRITE | WRITE_NO_RESP | NOTIFY | INDICATE):

Characteristic UUID	Name	Role	Value buffer
7913b588-0001-4635-b066-baa2cfc197cf	chars__conn_status	control / handshake: the client writes here, and the device announces handoffs and disconnects here	84 B
7913b588-0002-4635-b066-baa2cfc197cf	chars__data_transfer	application data	185 B
7913b588-0003-4635-b066-baa2cfc197cf	chars__on_demand	v5.0.3+ only: file-upload chunks, used only if the client announced upload support	device → client only

Direction is fixed: client → device is a GATT write; device → client is notify/indicate (on connection handle 0 — the firmware passes a literal 0 to both gatts_notify and gatts_indicate, so it only ever serves one central). Subscribe to both …-0001 and …-0002; a single write must fit the target characteristic’s value buffer above, whatever the MTU (Confirmed, ble_manager.dis:4308-4400).

Do not poll with GATT reads. Characteristic.write only ever calls gatts_notify(0, handle, data) — the gatts_write branch that would update the readable attribute value is unreachable in this build. The characteristic you read back is whatever you last wrote. All device data arrives as notifications or indications (Confirmed, f_ble_ble_data.dis:528-563).

MTU is client-negotiated: the device never calls gattc_exchange_mtu and sets no preferred MTU on the app path, so it starts at 23 and adopts whatever _IRQ_MTU_EXCHANGED reports (Confirmed). macOS negotiates 256 (observed on hardware, not derivable from the image).

Right after connecting the device asks for a 10–20 ms connection interval, slave latency 0 and a 12 s supervision timeout via gap_conn_param_update(0, 10000, 20000, 0, 12000).

Handshake (ConnStatus-Ready)

1. Subscribe

   Enable notifications on …-0002 and …-0001.

2. Report app state (optional)

   Write [0x03, 0x00, bits] to …-0001: bit0 isActive, bit1 isFocused, bit2 isLocked, bit3 isUiClosed, bit4 isService, bit5 lets the device drop BLE on its own schedule.

3. Write the Ready frame

   Write [0x00, 0x01, 0x01, 0x00] to …-0001 within 15 s of connecting (v5.0.3 schedules a disconnect for silent links). A last byte of 0x00 selects the legacy full-duplex transmit loop, which works on every platform. A nonzero frame_schema_id selects the half-duplex loop, which stalls on macOS/iOS. v5.0.3 accepts an optional 5th byte whose bit 0 announces file-upload support; leave it out unless you implement uploads.

4. Request and acknowledge records

   Write (0x01, 0x01) to …-0002 to request Static Data. In the legacy loop the device repeats Static Data, the WiFi list and Peer Sync until you acknowledge them with (0x01, 0x00), (0x02, 0x00) and a cat-6 command such as (0x06, 0x08) — any command in the category clears its pending flag, because recv_data_msgs just assigns controller.<cat>_cmd_id = cmd_id. Peer Sync is pending from the start (Controller.__init__ leaves peer_cmd_id = 1), but the device withholds it until Static Data has been acknowledged (Static data not yet sent, skipping BLE Peer Sync). Live Data then arrives every ~3 s, and Peer Pings whenever a peer changes — one per loop pass, and only while the last Live Data was under 10 s ago. Read them with the gen_* layouts below.

5. Send commands, then disconnect

   Write commands to …-0002 at any time; the full list is in the command map. To disconnect gracefully, write [0x00, 0x03] to …-0001 — that sets is_graceful_disconn, so the device does not auto-reconnect afterwards. Drop the link without it and the device silently re-advertises for 45 s.

recv_status_msgs applies these gates (Confirmed, ble_manager.dis:2753-3119): conn_mode == 1 clears pending static/peer requests; frame_schema_id > 0 sets TX ready and makes the legacy loop exit in favour of send_data_v2; TX owner is set only when both hold — the evt_is_tx_owner.set() lives inside the frame_schema_id > 0 branch. A short frame is not an error: conn_mode and frame_schema_id default to 0 when the write is under 3 or 4 bytes, and is_upload_supported to 0 under 5. The ids 25, 45, 59, 72 are real EXTENDED schema variants, but no BLE record layout depends on the schema id. In the half-duplex loop the device waits for the ATT confirmation of every indication (500 ms timeout), hands TX to the app with a 12-byte [0x04, 0x02, 0x02] + 00×9 indication on …-0001, and expects [0x04, 0x03, 0x02] (bit 1) to take it back; [0x04, 0x03, 0x01] (bit 0) revokes it instead. See Half-duplex & comms handoff.

Keeping the link alive

The device watches controller.status__conn. If it is anything but 1 for 15 s it counts a stall, logs App never sent ConnStatus within {} ms - link treated as hung, and raises evt_ble_schedule_disconn; the scheduled-disconnect manager then drops the link and parks BLE for 30 s (90 s, and with no fast-reconnect bursts, after two hung links in a row). Sending the Ready frame promptly is the whole fix. (Confirmed, ble_manager.dis:3840-3890 and compass.dis:5747-6039.)

Before it goes away the device announces its intent on …-0001, three times, 50 ms apart, as a 10-byte frame (Confirmed, ble_manager.dis:4934-5084):

Frame	Meaning
[0x00, 0x02] + 00×8	plain disconnect
[0x00, 0x05] + <ii	scheduled: (reconn_start_ms, reconn_duration_ms) — reconnect in N ms, advertising for M ms (M ≥ 10 000)

A client that handles [0x00, 0x05] can schedule its own rescan instead of hunting blind. During the off window the device also advertises a connectable 1.2 s burst every ~8 s, so a client that keeps scanning will usually reconnect before the window ends.

Setting bit 5 of the app-runtime byte ([0x03, 0x00, bits]) is what permits the routine version of this cycle: espnow_conn_v2 only requests a scheduled disconnect when modes.is_ble_auto_reconn is set and the app has reported itself inactive (bit 0 clear) for more than 10 s ([ESP-NOW] | App allows BLE Auto Reconnect → [ESP-NOW] | Requesting BLE Scheduled Disconnect). Leave bit 5 clear and the device stays connected — the 15 s hung-link guard is the only other thing that raises the event (Confirmed, espnow_conn_v2.dis:3030-3160).

Messages to read (BLE gen_* layouts)

The BLE GATT layouts are not the ESP-NOW layouts. The same (cat_id, cmd_id) key is reused by two transports with different struct layouts — for example BLE gen_live_data (0x03,0x01) is a 69-byte record, while TOTEM_MSG_MAP (3,1) is 20 bytes. A BLE client must use the gen_* layouts here; a mesh client must use the mesh map.

All BLE records prefix buff[0:2] = (cat_id, cmd_id). Confirmed layouts:

Live Data (0x03, 0x01) — gen_live_data, struct <bfi3fb4Bi3b2hbiffb3ibHBBb at offset 2 (69 B). Fields in order:

#	code	Field
1	b	sat_count
2	f	p_acc, m (−1 = None)
3	i	altitude, m (−500 = None)
4	f	latitude, °
5	f	longitude, °
6	f	batt_volts, V
7	b	esp-now channel
8	B	power_bits (power_mode in bits 0-2)
9	B	max_hop_cnt
10	B	mesh_rx
11	B	mesh_relayed
12	i	unix_ts, s
13	b	color_id
14	b	orientation
15	b	solution_id
16	h	heading, °
17	h	azimuth, °
18	b	speed (cap 127, −1 = None)
19	i	odometer
20-21	ff	reserved ×2 (always 0)
22	b	reserved (always −1)
23	i	uptime, s
24	i	age
25	i	reserved (always 0)
26	b	power_level: battery health, 2 = low
27	H	reserved (always 0)
28	B	flags
29	B	reserved (always 0)
30	b	batt_pct

The flags byte (field 28) is pack_flags(is_sos, is_eco, led_brt ≥ GLOBAL_BRT, gnss_location_set, power_level == 2, is_charging, 0, is_mag_cal_needed): bit 2 is normal (undimmed) brightness and bit 4 is low battery.

The reserved fields hold these constants in every published firmware (3.2.12 to 5.0.3), and the official app reads and discards them, so a client can ignore them.

Static Data (0x01, 0x02) — gen_static_data. buff[2] = total_len & 255, buff[3:9] = MAC (6 B), then struct <biHBBBbBBBbhhiiibbb at offset 9 (34 B), then three UTF-8 strings concatenated from offset 43. There are no per-string prefixes: their lengths are the struct’s last three fields.

Region	Field
buff[2]	total_len & 255
buff[3:9]	device MAC (6 B)
struct @9: b	reserved (always 0)
i	age
H	release_id
B B B	ver_major, ver_minor, ver_patch
b	color_id
B	settings_flags (bit0 persistent north, bit1 compass lock; the app reads bit3 as bond chat)
B	capabilities (bit0 = half-duplex loop; 1 in 5.x, 0 in 4.1.3)
B	service_id
bhhiii	6× reserved (always 0; the app discards them)
b b b	len(device_name), len(branch), len(wifi_ssid)
strings @43	device_name, git branch (N/A), wifi_ssid ("")

Peer Ping (0x06, 0x02) — gen_peer_ping. buff[2] = total length, buff[3:9] = peer MAC (6 B), buff[9] = mesh hops, then struct <ffbbh4BHbb4BiihBbf at offset 10 (40 B): lat, lon, p_acc, speed, bearing, flag byte A, r, g, b, 0 (the app’s dtim), name length, rssi, msg_rx, msg_tx, mesh_rx, mesh_send_count, last_update, last_coords_unix, distance_diff, flag byte B, orientation, volts. Then the peer name (from offset 50) and <bH = (batt_pct, release_id). A = pack_flags(sos, is_poi, is_mesh, is_stale, is_collected, 0, is_unknown, 0), B = pack_flags(is_hidden, is_locked, 0×6). The app reads A’s bit 5 as isIdle; the firmware always sends 0 there. is_unknown is set once a peer has had no coordinates for 2 h (v5.0.3; 4 h in v5.0.2).

Peer Sync (0x06, 0x07) — gen_peer_sync: a peer-MAC list, [0x06, 0x07, total_len & 255, peer_count, mac0(6), mac1(6), …].

ESP-NOW mesh client

To join the mesh peer-to-peer instead of going through a phone, send/receive raw ESP-NOW frames on the fleet’s channel. The repository’s Go mesh package encodes and decodes these frames, and the emulator package plus cmd/totememu run a complete Totem on an ESP32 with TinyGo: see ESP32 emulator.

Radio setup

Item	Value
Channel	6 (self.channel = 6, the only write to it; there is no set_channel)
Interface	WiFi station; the ESP-NOW address is the STA MAC (machine.unique_id())
PHY	WIFI_PROTOCOL_LR only (protocol bitmap 0x08), in every mode 5.0.3 uses
LR rate	250 K: EspConn calls e.config(rate=41) (WIFI_PHY_RATE_LORA_250K)
TX power	txpower=21 dBm (the driver caps it at 20)
Encryption	none — add_peer(mac) with no lmk

The fleet is pinned to channel 6 (Confirmed, and on hardware); ESP-IDF drops off-channel frames, so a mesh client must match it.

Frame layout

┌──────────┬──────────┬────────┬────────┬───────────────────────────┐
│ SyncWord │ SyncWord │ cat_id │ cmd_id │ payload (struct-packed)   │
│  0xA7    │  0x74    │  u8    │  u8    │ variable                  │
└──────────┴──────────┴────────┴────────┴───────────────────────────┘
  [0]        [1]        [2]      [3]      [4:]

• SyncWord = the 2 bytes 0xA7 0x74 (literal in the image).
• Validation is SyncWord match + len >= 4 + a per-(cat, cmd) payload-size check.
• There is no CRC or checksum on the ESP-NOW frame. (The rodata string Invalid Checksum value for: {} belongs to the u-blox UBX GNSS parser, not this path.)
• Mesh dedup UID is a uint16 randint(1, 65534) at mesh-frame offset 20; MSG_EXP_MSECS = 150000 ms.

Payload formats (TOTEM_MSG_MAP, corrected)

Earlier analyses decoded the map’s qstr-immediates with >>2, producing bogus “handler names” (disconn_animation, device_power, dev_info, disabled, …). Those are decode artifacts and are wrong. ESP32 MicroPython (REPR_A) tags qstr-immediates as (o & 7) == 2 with value o >> 3; under the correct >>3 decode every value is a struct format string. The corrected table follows (Confirmed).

Every payload leads with the echoed (cat_id, cmd_id) as BB.

(cat, cmd)	Format	Size
(0,0) / (0,1) / (0,2)	<BBffbbhbbb6B	23 B
(1,0)	<BBB7bHbb	14 B
(1,2)	<BBiffHii	24 B
(1,5)	<BB9B8bhh4B	27 B
(1,6)	<BBbbB	5 B
(1,7)	<3Bbb	5 B
(2,0)	<BB6BffbbHbbbhhBBi	33 B
(3,1) / (3,2)	<BBHH4b4BHHBb	20 B
(7,0)	<BBffbbbHbbH	19 B
(7,1)	<BBHbffb	14 B

EXTENDED longer variants — (cat, cmd) → {frame length → fmt}. The keys are total frame lengths including the SyncWord, not schema ids (for (0,0), 72 means “72 bytes or more”):

(cat, cmd)	Frame length	Format	Size
(0,0)	25	<BBffbbhbbb6B	23 B
(0,0)	59	<BBffbbhbbb6Bhii4BhHehhffbB	57 B
(0,0)	72	<BBffbbhbbb6Bhii4BhHehhffbBBiiBBb	69 B
(1,6)	5	<BBbbB	5 B
(1,6)	29	<BBbb9BBB3i	27 B
(2,0)	45	<BB6BffbbHbbbhhBBiffh	43 B

Every field is now labelled (Confirmed): see the peer, locate and Smart Group tables in message format. The 6-byte MAC in the (0,*) frames is a target MAC that 5.0.3 always leaves zero; the sender is identified by its ESP-NOW source address. Bonding and timing are in ESP-NOW mesh.

Custom OTA server

You can point a Totem at your own OTA server; it needs no signing key (SHA-256 only, plain HTTP). The contract (Confirmed):

1. Device announces

   POST http://api.totemportal.com/devices/{MAC}/ota (plain HTTP, no auth) with JSON body {version, endpoint_id, device_type_id, lat, lon, gnss_time, release_id}.

2. Server replies with a release object

   Expose ota_url, version, product, branch, release_code, release_id.

3. Device fetches the manifest

   GET {ota_url}/contents.json — a JSON array of filenames. The device picks the entry ending .bin (firmware) or .tgz (preview).

4. Download, verify, flash

   The device downloads the chosen file, verifies SHA-256 only, flashes, reboots, and reports back with POST …/ota?updated.

A device WebSocket to api.totemportal.com (ws://) carries push OTA triggers shaped like {"cmd":…} (Partial). A demi-god ESP-NOW OTA trigger also exists (demigod_gen_ota_update), but its exact (cat, cmd) and struct are not recovered — do not fabricate them.

Chunked transfer

Only needed if your client accepts the device’s log uploads (v5.0.3) or does BLE OTA. A client that sends the plain 4-byte Ready frame never receives uploads. Layouts (Confirmed, f_ble_chunking.dis:383-492 and f_ble_file_upload.dis:591-745, 1165-1300; details in chunking):

• Transfer header (announce / finish), indicated on …-0001: [0x02, 0x02] + struct '<HBBBiHiB' = file_id, status_id, action_id, file_type_id, byte_pos, chunk_no, file_size, flags (v5.0.3: bit0 last chunk, bit1 from compass). Then the 32-byte SHA-256 at buff[18:50], the name length at buff[50], the name from buff[51], and err_no right after it. file_id = sha256[0] | sha256[1] << 8.
• Chunk (v5.0.3), notified on …-0003: [0x00, 0x02] + struct '<HHiH' = file_id, length, byte_pos, chunk_no, then the data (CHUNK_HDR_SZ = 12). The payload size is min(mtu - 3, 247) - 12 — 235 bytes at MTU 256, and 8 bytes if the MTU is still 23. A write that fails with OSError is retried up to 3 times, 160 ms apart.
• App reply, written to …-0001 (v5.0.3): (0x02, 0x03) + struct '<HbBBiHiB', 18 bytes in total — the device parses it as '<BBHbBBiHiB' from offset 0 and ignores anything shorter than 18 bytes or carrying a file_id other than the one in flight. It reads file_id, status, action, chunk: status ∈ {2, 3, 4} is terminal, action 4 = resume at chunk (or chunk 1 if chunk is 0), action 1 = ready. A reply longer than 50 bytes may carry err_no after the name, exactly as the device’s own header does.

Field	Enum values
status_id	1 = normal / in-progress, 4 = error / abort (the device sends 1 for announce and completion, 4 to abort)
action_id	0 = header / announce, 1 = last-chunk / complete, 4 = resume (app → device)
file_type_id	2 for a log upload to the app

file_type_id is not the save_to enum. SAVE_TO_VFS = 1, SAVE_TO_OTA = 2, UPLOAD_TO_APP = 3 are internal destinations passed to FileTransfer(save_to=…) and never appear on the wire; _upload separately sets file.file_type_id = 2, and that is the byte you receive (Confirmed, f_ble_file_upload.dis:1495-1530).

Overall result codes, from the uploader’s own summary line (result: {} (1=done 2=no app 3=retry 4=failed 5=cancel 6=abort)): 1 done, 2 no-app, 3 retry, 4 failed, 5 cancel, 6 abort.

While an upload is streaming the uploader takes ble.noncrit_owner = 'upload', which lets it delay the half-duplex TX handoff by up to 2 s ([send_data_v2] Non-critical hold expired ({}); handing off).

Still missing / confirm on-device

These are known-structure-but-unconfirmed items. Verify them against your own hardware before relying on them:

Item	Status	How to confirm
Home channel	6 (Confirmed, hardware-tested); no set_channel exists	—
frame_schema_id / transmit mode	Confirmed and hardware-tested: 0 = legacy loop, which notifies (works on macOS), > 0 = half duplex, which indicates (stalls on macOS/iOS)	totemctl --trace info, or add --half-duplex to compare
Negotiated MTU	Partial — the device sets no preferred MTU, so this is the central’s choice; 256 observed on macOS	watch mtu_payload effects on upload chunk size
Half duplex on Linux / Android / Windows	Untested (those stacks confirm indications, so it should work)	totemctl --half-duplex --trace watch on such a host
Mesh payload field semantics	Confirmed; peer frames and the pairing handshake hardware-tested with the ESP32 emulator	—
Locate relay and Smart Group on the air	Confirmed from bytecode, not yet hardware-tested	run the emulator with pos set, far enough from the Totem for its peer to go stale
Demi-god OTA frame	Partial / missing	(cat, cmd) + struct not recovered — do not fabricate
Runtime-only values	Partial	venue code, group id, and device name live in runtime state / bytecode, not the static image

Safety

Interoperate only with Totems you own. Because the mesh is unauthenticated and unencrypted, the demi-god ESP-NOW broadcast path can affect nearby devices — its only gate is RSSI proximity plus message-UID dedupe, with no signature or HMAC. Do not send demi-god or mesh commands against devices you do not own; broadcasting near other people’s Totems can change their behaviour without their consent.