This is one place where I'm wondering how Cap'n Proto[1] would help. Since the structures are already made for in memory usage I think it might work pretty well. And given that it's all versioned/tagged structure wise it should be easy to do the schema migration.

[1] http://kentonv.github.io/capnproto/