Skip to main content

Encoding Spec With Examples

Summary of Data Types

Molecule categorizes data types into fixed and dynamic sizes:

Fixed SizeDynamic Size
Typesbyte array structvector table option union

Memory Layout

|  Type  |                      Header                      |               Body                |
|--------+--------------------------------------------------+-----------------------------------|
| array | | item-0 | item-1 | ... | item-N |
| struct | | field-0 | field-1 | ... | field-N |
| fixvec | items-count | item-0 | item-1 | ... | item-N |
| dynvec | full-size | offset-0 | offset-1 | ... | offset-N | item-0 | item-1 | ... | item-N |
| table | full-size | offset-0 | offset-1 | ... | offset-N | filed-0 | field-1 | ... | field-N |
| option | | item or none (zero bytes) |
| union | item-type-id | item | |
note

All items in Header are 32-bit unsigned integers in little-endian.

Primitive Data Type

byte

NameDescriptionExample
byteA single byte00

Composite Data Types

array

NameDescriptionSizeSerialization
arrayA fixed-size type with a fixed-size inner type and a fixed length.
All items are stored consecutively.
The size of an array is the size of inner type times length.Serializing an array is to serialize all fields in it.
No overhead incurs since all items are stored consecutively without extra space.

Examples

DefinitionStored ValueSerialized Bytes
array Byte3 [byte; 3];01020301 02 03
array Uint32 [byte; 4];A 32-bit unsigned integer 0x01020304 in little-endian04 03 02 01
array TwoUint32 [Uint32; 2];Two 32-bit unsigned integers 0x01020304 and 0xabcde in little-endian04 03 02 01 de bc 0a 00

struct

NameDescriptionSizeSerialization
structA fixed-size type with all fields being fixed-size and a fixed quantity of fields.The size of a struct is the sum of all its fields.Serializing a struct is to serialize all fields in it.
No overhead incurs since all items are stored consecutively without extra space.

Examples

DefinitionStored ValueSerialized Bytes
struct OnlyAByte { f1: byte }A byte abab
struct ByteAndUint32 { f1: byte, f2: Uint32 }A byte ab and a 32-bit unsigned integer 0x010203 in little-endianab 03 02 01 00

vectors

Vectors are categorized into: fixed vector, fixvec, and dynamic vector, dynvec. If the inner item is fixed-size, it's a fixvec; otherwise, it's a dynvec.

Both of fixvec and dynvec are dynamic-size types.

NameDescriptionSerialization
fixvecA dynamic-size type with fixed-size inner item.Step 1. Serialize the length as a 32-bit unsigned integer in little-endian.
Step 2. Serialize all items.
dynvecA dynamic-size type with dynamic-size inner item.Step1. Serialize the full size in bytes as a 32-bit unsigned integer in little-endian.
Step 2. Serialize all offset of the items as 32-bit unsigned integer in little-endian.
Step 3. Serialize all items.

Examples of Fixed Vector

Define vector Bytes <byte>;:

ItemSerialized Bytes
An empty Bytes00 00 00 00 (The length of any empty fixed vector is 0).
0x1201 00 00 00, 12
0x1234567890abcdef08 00 00 00, 12 34 56 78 90 ab cd ef

Define vector Uint32Vec <Uint32>;:

ItemSerialized Bytes
An empty Uint32Vec00 00 00 00
0x12301 00 00 00, 23 01 00 00
[0x123, 0x456, 0x7890, 0xa, 0xbc, 0xdef]# there are six items
06 00 00 00
# six items
23 01 00 00, 56 04 00 00, 90 78 00 00, 0a 00 00 00, bc 00 00 00, ef 0d 00 00

Example of Dynamic Vector

Define vector BytesVec <Bytes>;:

ItemSerialized Bytes
An empty BytesVec04 00 00 00 (The full size of an empty dynamic vector is 4 bytes).
[0x1234]# the full size is 14 bytes
0e 00 00 00
# one offset
08 00 00 00
# one item
02 00 00 00 12 34
[0x1234, 0x, 0x567, 0x89, 0xabcdef]# full size is 52 (0x34) bytes
34 00 00 00
# five offsets (20 bytes in total)
18 00 00 00, 1e 00 00 00, 22 00 00 00, 28 00 00 00, 2d 00 00 00
# five items (28 bytes in total)
02 00 00 00, 12 34
00 00 00 00,
02 00 00 00, 05 67
01 00 00 00, 89
03 00 00 00, ab cd ef

table

NameDescriptionSerialization
tableA dynamic-size type.
Can be considered as a dynvec with a fixed length.
The same as dynvec:
Step1. Serialize the full size in bytes as a 32-bit unsigned integer in little-endian.
Step 2. Serialize all offset of the items as 32-bit unsigned integer in little-endian.
Step 3. Serialize all items.

Example

Define table MixedType { f1: Bytes, f2: byte, f3: Uint32, f4: Byte3, f5: Bytes }, the serialized bytes of a MixedType { f1: 0x, f2: 0xab, f3: 0x123, f4: 0x456789, f5: 0xabcdef } is:

# the full size is 43 (0x2b) bytes
2b 00 00 00
# five offsets (20 bytes in total)
18 00 00 00, 1c 00 00 00, 1d 00 00 00, 21 00 00 00, 24 00 00 00
# five items (19 bytes in total)
00 00 00 00
ab
23 01 00 00
45 67 89
03 00 00 00, ab cd ef

option

NameDescriptionSerialization
optionA dynamic-size type.Serializing an option depends on whether it is empty or not.
If empty, zero bytes (the size is 0).
If not, serialize the inner item (the size is same as the inner item's size).

Example

Define option BytesVecOpt (BytesVec);

ItemSerialized Bytes
None (Empty)
Some([])04 00 00 00
Some([0x])# the full size of BytesVec is 12 bytes
0c 00 00 00
# the offset of Bytes
08 00 00 00
# the length of Bytes
00 00 00 00

union

NameDescriptionSerialization
unionA dynamic-size type.Step 1. Serialize a item type ID in bytes as a 32-bit unsigned integer in little-endian. The item type ID is the index of the inner items, starting at 0.
Step 2. Serialize the inner item.

Example

Define union HybridBytes { Byte3, Bytes, BytesVec, BytesVecOpt }

ItemSerialized Bytes
Byte3 (0x123456)00 00 00 00, 12 34 56
Bytes (0x)01 00 00 00, 00 00 00 00
Bytes (0x123)01 00 00 00, 02 00 00 00, 01 23
BytesVec ([])02 00 00 00, 04 00 00 00
BytesVec ([0x])02 00 00 00, 0c 00 00 00, 08 00 00 00, 00 00 00 00
BytesVec ([0x123])02 00 00 00, 0e 00 00 00, 08 00 00 00, 02 00 00 00, 01 23
BytesVec ([0x123, 0x456])# Item Type Id
02 00 00 00
# the full size of BytesVec is 24 bytes
18 00 00 00
# two offsets of BytesVec (8 bytes in total)
0c 00 00 00, 12 00 00 00,
# two Bytes (12 bytes in total)
02 00 00 00, 01 23
02 00 00 00, 04 56
BytesVecOpt (None)03 00 00 00
BytesVecOpt (Some(([])))03 00 00 00, 04 00 00 00
BytesVecOpt (Some(([0x])))03 00 00 00, 0c 00 00 00, 08 00 00 00, 00 00 00 00
BytesVecOpt (Some(([0x123])))03 00 00 00, 0e 00 00 00, 08 00 00 00, 02 00 00 00, 01 23
BytesVecOpt (Some(([0x123, 0x456])))# Item Type Id
03 00 00 00
# the full size of BytesVec is 24 bytes
18 00 00 00
# two offsets of BytesVec (8 bytes in total)
0c 00 00 00, 12 00 00 00,
# two Bytes (12 bytes in total)
02 00 00 00, 01 23
02 00 00 00, 04 56