Embedded Files
Some programming languages provide structure syntax to group a set of data in a structural pattern. Usually the name is known as struct (in C and Go). There is such a coding problem called struct memory misalignment. This means we are wasting memory spaces in our structure definition.
The Problem
The Problem
As we define the structure, we need to consider the memory allocation for each elements in the structure. Example:
amd64 cpu has the following memory allocation sizes for Go:
var basicSizes = [...]byte { Bool: 1, Int8: 1, Int16: 2, Int32: 4, Int64: 8, Uint8: 1, Uint16: 2, Uint32: 4, Uint64: 8, Float32: 4, Float64: 8, Complex64: 8, Complex128: 16,}When you start structuring the data sizes, say for example:
struct { a string b bool c string d bool e int8 f int32}The compiler's memory allocation will looks as such:
struct { a string b bool _ [7]byte // padding added by compiler c string d bool e int8 f int32 _ [2]byte // padding added by compiler}As the compiler allocates memory, say the maximum size is 8, gives:
struct { a [8]byte a [8]byte b [1]byte _ [7]byte c [8]byte c [8]byte d [1]byte e [1]byte f [4]byte _ [2]byte} // 56 bytes in total
Notice that we are wasting large number of padding memory to match the structure around b? This is called structure memory misalignment, where the memory spaces are wasted due to padding.
The solution is to structure the element accordingly to the memory alignment.
The Solution
The Solution
Do the memory count as you design the structure. Example, by moving b element down the low bytes elements together, we save 8 bytes memory spaces.
// Sample codestruct { a string c string b bool d bool e int8 f int32}Compiler will provides the following memory allocations with 1 byte padding space:
struct { a [8]byte a [8]byte c [8]byte c [8]byte b [1]byte d [1]byte e [1]byte f [4]byte _ [1]byte} // 56 bytes in total
The Caveat
The Caveat
#1 - Hard to Optimize for Existing Large Bad Codes
#1 - Hard to Optimize for Existing Large Bad Codes
The thing is that if your user already using some non-referencing structure initialization, it is highly difficult to optimize such structure. Example of bad code in Go:
scenarios := []struct { inStatus int inMessage string inArg interface{} inStdout *bytes.Buffer inStderr *bytes.Buffer outMessage string }{ { InfoStatus, sample1, nil, &bytes.Buffer{}, &bytes.Buffer{}, "[ INFO ] " + sample1, }, { WarningStatus, sample1, nil, &bytes.Buffer{}, &bytes.Buffer{}, "[ WARNING ] " + sample1, }, { ... continue 500 lines moreThe above is the learning from Go's recommended table driven testing, where the values are assigned using the array style. If you want to restructure your struct, you need to realign all the values (that >500 lines of codes of editing and scrolling).
Hence, it is always good and right to fill in a structure using the reference. This way, you leaves room for developers to further optimize the structure without needing to alter your existing data set.
Good Example in Go:
scenarios := []struct { inPrefix int outPrefix int }{ { inPrefix: NoTerminal, outPrefix: NoTerminal, }, { inPrefix: BASHTerminal, outPrefix: BASHTerminal, }, { inPrefix: SHTerminal, outPrefix: SHTerminal, }, { inPrefix: DOSTerminal, outPrefix: DOSTerminal, }, { inPrefix: -1, outPrefix: NoTerminal, }, { inPrefix: -5000, outPrefix: NoTerminal, }, { ... continue 500 lines more }Example in C:
employee emp = { .id = 0, .name = "none"};OR struct iio_chan_spec stm32_lptim_cnt_channels = { .type = IIO_COUNT, .channel = 0, .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) | BIT(IIO_CHAN_INFO_ENABLE) | BIT(IIO_CHAN_INFO_SCALE), .ext_info = stm32_lptim_cnt_ext_info, .indexed = 1,};#2 - Compiler Specifics
#2 - Compiler Specifics
Such memory optimization are sometimes specific to compiler depending on languages like C. However, developers are strongly not to rely on compiler to write his/her codes when a disciplined practices is available.
In Summary
In Summary
To avoid over-padding your structure memory, you MUST:
- Organize the structure based on memory alignment.
- ALWAYS assign values using reference style.
- ALWAYS forbids array style assignment.
References
References
- https://elixir.bootlin.com/linux/latest/source/drivers/iio/counter/stm32-lptimer-cnt.c
- https://stackoverflow.com/questions/13706809/structs-in-c-with-initial-values
- https://stackoverflow.com/questions/5435841/memory-alignment-in-c-structs
- https://go101.org/article/memory-block.html
- https://www.tutorialspoint.com/cprogramming/c_data_types.htm
- https://golang.org/src/go/types/sizes.go
- https://stackoverflow.com/questions/39063530/optimising-datastructure-word-alignment-padding-in-golang
- https://dave.cheney.net/2015/10/09/padding-is-hard
- http://golang-sizeof.tips/?t=Ly8gU2FtcGxlIGNvZGUKc3RydWN0IHsKCWEgc3RyaW5nCgljIHN0cmluZwoJYiBib29sCglkIGJvb2wKCWUgaW50OAoJZiBpbnQzMgp9
- https://medium.com/@sebassegros/golang-dealing-with-maligned-structs-9b77bacf4b97
Page updated
Google Sites
Report abuse