Design notes for saving tables.
The first point to note is that in Lua any data type (except nil) can be used for the key, as well as the value for any table field. The replacement version of _freeciv_state_dump() UNDERSTANDS THIS. (The distro version does not!)
The next is to realise that some data cannot be saved mostly because the data has no 'literal' form needed to write the vars script. The obvious consequence of this is that a table field can only be saved if both the key AND value are eligible.
The replacement version of _freeciv_state_dump asseses data to be eligible as follows:
- boolean
number
string (provided it has no unprintable characters; see also special note)
userdata (subject to special considerations described below)
table (provided it is designated as saveable explained below)
special note for string:
A simple field in the environment (_ENV) with a key that is an identifier starting with an underscore will NOT be saved (so that, for example _VERSION will not be overwritten).
userdata:
The two criteria for userdata are that it must be identifiable, and that there must be a finder. Both criteria are assessed automatically so as fc software changes, eligibily will change with it. See a previous post for current status as of fc 2.6. I will add a special note here. As previously noted userdata that is metadata has both id and rule_name as identifiers. The current version of _freeciv_state_dump only ever uses id, but this should be deprecated because it contributes to the save file compatibility problem of not being able to remove or reorganise the defining sections in the ruleset - the reason being that id`s are assigned by the order of those defining sections. Note that this issue is MOOT if the metadata is hardcoded (obviously) but also
would be if the id is also user defined. Unfortunately, this is not the case. Currently in fc2.6 metadata of type Terrain has an "identifier =" field which is supposedly used for save restore processes but it is NOT used as the tolua .id field. In any event there is no apparent harm in using rule_name wherever possible, and it removes one excuse for dev`s to not fix the general save/restore misuse of id`s for metadata. (One might note the drip feed of problem reports of save file incompatibilities due to misuse of metadata id in the general save/restore process. Considering the warnings given to modders about preserving rule_name compatibility this must be a classic case of 'Do as I say, not as I do'.)
table:
As mentioned previously, the replacement version of _freeciv_state_dump will consider a table to be eligible if it has a metatable with a SAVEABLE field with the identifier _fc_keep as the key. Clearly the key is eligible but one must ensure that the value is also eligible. Note that this INCLUDES ELIGIBLE TABLES. To preserve elibility from one save/restore to the next the metatable will also be saved (and linked as the table`s metatable). Metatables are ordinary Lua tables so they too could have a metatable. This would be highly unusual and I know of only one person who would do such a silly thing, but it is possible. So accordingly, a policy decision was made to save the entire chain of metatables if the originating table and metatable were saved. At the moment you are unlikely to ever see this happen - unless you actually want to test for it. There are two ways to terminate this chain:
- a. The metatable has been previously encoutered and since this will have already been processed nothing more needs to be done.
b. The chain terminates in a table that has no metatable thus creating a 'tail'.
Unfortunately the latter termination raises an obscure issue: one could use a 'system' table as a tail to the chain. There is no easy way to reliably reconstruct this structure so the replacement will create private copies - to the extent possible.
But all is not lost. Included in the vars script is a conditional call to a user supplied function with the name _fc_restore. This function is called with two arguments - the first being a list of tables restored. This will allow the Lua scribe to fix the mess (and much more - in fact the primary purpose was to allow the scribe to recreate those fields that could not be saved).
One final design note:
The allowance for the associated value of _fc_keep to be a table can create another 'chain'. Like the metatable chain this is also linear and can be terminated in similar ways but with some important differences. The conditions are:
- a. The associated value of some _fc_keep field will be clearly identified as ineligible in which case the entire chain will be abandoned.
b. Similar to (a) but the value is clearly identified as eligible so the chain will be eligible.
c. The value is a table that has previously been encountered in the current chain as a potentially saveable table - that is one that has a metatable with an _fc_keep field. In this case the chain terminates in a loop and WILL BE ASSESSED AS BEING ELIGIBLE. Note that this assessment excludes tables that have been saved ONLY because they are part of a saved metatable chain. This exclusion is made to avoid a minor technical issue that will be of interest to no one.
I will close this post with a caveat 'for the record'. A metatable can be hidden if it contains the metamethod __metatable. I know of no way to defeat this. The regular scribe will have no need for this for saveable tables and so should be avoided as it will corrupt the restored table structure.