krh@311: <?xml version="1.0" encoding="utf-8"?>
krh@311: <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN" "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd">
krh@311: 
krh@311: <chapter id="file-format">
krh@311:   <title>Package Set File Format</title>
krh@311:   
krh@311:   <sect2 id="file-header">
krh@311:     <title>File header</title>
krh@311:     
krh@311:     <para>
krh@312:       The rzdb file starts with a header, containing some number of
krh@311:       sections, terminated by a section with type 0:
krh@311:     </para>
krh@311: 
krh@311:     <programlisting><![CDATA[
krh@311: struct razor_set_header {
krh@311: 	uint32_t magic;
krh@311: 	uint32_t version;
krh@311: 	struct razor_set_section sections[0];
krh@311: };
krh@311: 
krh@311: struct razor_set_section {
krh@311: 	uint32_t type;
krh@311: 	uint32_t offset;
krh@311: 	uint32_t size;
krh@311: };
krh@311: ]]></programlisting>
krh@311: 
krh@311:     <para>
krh@312:       razor_set_open() mmaps the rzdb file, and creates a struct razor_set:
krh@311:     </para>
krh@311: 
krh@311:     <programlisting><![CDATA[
krh@311: struct razor_set {
krh@311: 	struct array string_pool;
krh@311:  	struct array packages;
krh@311:  	struct array properties;
krh@311:  	struct array files;
krh@311: 	struct array package_pool;
krh@311:  	struct array property_pool;
krh@311:  	struct array file_pool;
krh@311: 	struct razor_set_header *header;
krh@311: };
krh@311: ]]></programlisting>
krh@311: 
krh@311:     <para>
krh@311:       by finding the sections with those IDs and creating struct
krh@311:       array's pointing to the right places in the mmapped data. (This
krh@311:       is the only processing needed when reading in the file;
krh@311:       everything else is used exactly as-is.)
krh@311:     </para>
krh@311: 
krh@311:   </sect2>
krh@311: 
krh@311:   <sect2 id="sections">
krh@311:     <title>The sections</title>
krh@311: 
krh@311:     <itemizedlist>
krh@311:       <listitem>
krh@311:         <para>
krh@311:           <emphasis>RAZOR_STRING_POOL</emphasis> Stores one copy of
krh@312: 	  each string that appears in the rzdb file. (At the moment,
krh@312: 	  this is: package names, package versions, property names,
krh@311: 	  property versions, and (basenames of) filenames.) The
krh@311: 	  strings are arbitrarily-sized, 0-terminated, and not in any
krh@311: 	  particular order (although the empty string always ends up
krh@311: 	  being at offset 0).
krh@311: 	</para>
krh@311:       </listitem>
krh@311: 
krh@311:       <listitem>
krh@311:         <para>
krh@311:           <emphasis>RAZOR_PACKAGES</emphasis> Array of struct
krh@311: 	  razor_package; one for each package in the set, sorted by
krh@311: 	  name.
krh@311: 	</para>
krh@311:       </listitem>
krh@311: 
krh@311:       <listitem>
krh@311:         <para>
krh@311:           <emphasis>RAZOR_PROPERTIES</emphasis> Array of struct
krh@311: 	  razor_property; one for each unique property in the set,
krh@311: 	  sorted by type, then name, then relation type (eg, "&lt;" or
krh@311: 	  "&gt;="), then version. (Properties with no version have
krh@311: 	  relation type RAZOR_VERSION_EQUAL, and version "".)
krh@311: 	</para>
krh@311:       </listitem>
krh@311: 	    
krh@311:       <listitem>
krh@311:         <para>
krh@311:           <emphasis>RAZOR_FILES</emphasis> Array of struct
krh@311: 	  razor_entry; one for each file owned by any package in the
krh@311: 	  set. The current sort order (which is subject to change)
krh@311: 	  is breadth-first, sorted by basename. So eg: /, /bin,
krh@311: 	  /dev, /etc, /bin/false, /bin/true, /dev/null, /etc/passwd.
krh@311: 	</para>
krh@311:       </listitem>
krh@311: 
krh@311:       <listitem>
krh@311:         <para>
krh@311:           <emphasis>RAZOR_PACKAGE_POOL</emphasis> Array of struct
krh@311: 	  list, with each list item containing the index of a struct
krh@311: 	  razor_package in the packages section. See the discussion
krh@311: 	  of lists below.
krh@311: 	</para>
krh@311:       </listitem>
krh@311: 
krh@311:       <listitem>
krh@311:         <para>
krh@311:           <emphasis>RAZOR_PROPERTY_POOL</emphasis> Array of struct
krh@311: 	  list, with each list item containing the index of a struct
krh@311: 	  razor_property in the properties section. See the
krh@311: 	  discussion of lists below.
krh@311: 	</para>
krh@311:       </listitem>
krh@311: 
krh@311:       <listitem>
krh@311:         <para>
krh@311:           <emphasis>RAZOR_FILE_POOL</emphasis> Array of struct list,
krh@311: 	  with each list item containing the index of a struct
krh@311: 	  razor_entry in the files section. See the discussion of
krh@311: 	  lists below.
krh@311: 	</para>
krh@311:       </listitem>
krh@311:     </itemizedlist>
krh@311:   </sect2>
krh@311: 
krh@311:   <sect2 id="data-types">
krh@311:     <title>Data types</title>
krh@311: 
krh@311:     <para>
krh@311:       Note that the exact layout of bits involves some historical
krh@311:       accidents.  (Particularly the fact that the "name" field in most
krh@311:       structs loses its high bits to a flags field.)
krh@311:     </para>
krh@311: 
krh@311:     <programlisting><![CDATA[
krh@311: struct list_head
krh@311: 	uint list_ptr : 24;
krh@311: 	uint flags    : 8;
krh@311: 
krh@311: struct list
krh@311: 	uint data  : 24;
krh@311: 	uint flags : 8;
krh@311: ]]></programlisting>
krh@311: 
krh@311:     <para>
krh@311:       Used to store lists of package, property, or file IDs. "struct
krh@311:       list_head" stores the head of the list, which points to one or
krh@311:       more "struct list"s in the appropriate "pool" section.  ("struct
krh@311:       list" should probably be called "struct list_item".)
krh@311:     </para>
krh@311: 
krh@311:     <para>
krh@311:       "list_first(&amp;head, &amp;pool)" returns a "struct list *"
krh@311:       pointing to the first element of the list (or NULL for an empty
krh@311:       list), and "list_next(list)" will return successive elements,
krh@311:       until NULL is returned. Each "list->data" contains the index of
krh@311:       a package, property, or file in the corresponding section of the
krh@311:       set.
krh@311:     </para>
krh@311: 
krh@311:     <para>
krh@311:       Peeking underneath the abstraction, a list_head's "flags" is
krh@311:       0xff if the list is empty, 0x80 if it contains a single element,
krh@311:       or 0x00 if it contains more than one element. In the
krh@311:       single-element case, that element is actually stored in the
krh@311:       list_head directly rather than being stored in a pool (and so
krh@311:       list_first() just casts the list_head* to a list* and returns
krh@311:       it). For multi-element lists, list_ptr is the index in the pool
krh@311:       of the first element of this list; the list continues through
krh@311:       successive elements of the pool until one with non-zero flags is
krh@311:       reached, indicating the end of the list.
krh@311:     </para>
krh@311: 
krh@311:     <programlisting><![CDATA[
krh@311: struct razor_package
krh@311: 	uint name    : 24;
krh@311: 	uint flags   : 8;
krh@311: 	uint version : 32;
krh@311: 	struct list_head properties;
krh@311: 	struct list_head files;
krh@311: ]]></programlisting>
krh@311: 
krh@311:     <para>
krh@311:       name and version are indexes into string_pool. properties is a
krh@311:       list of all of the package's properties, and files is a list of
krh@311:       its files. flags is currently only used during razor_set
krh@311:       merging, to keep track of which set a package came from.
krh@311:     </para>
krh@311: 
krh@311:     <programlisting><![CDATA[
krh@311: struct razor_property
krh@311: 	uint name     : 24;
krh@311: 	uint flags    : 6;
krh@311: 	uint type     : 2;
krh@311: 	uint relation : 32;
krh@311: 	uint version  : 32;
krh@311: 	struct list_head packages;
krh@311: ]]></programlisting>
krh@311: 
krh@311:     <para>
krh@311:       name and version are indexes into string_pool. type is an enum
krh@311:       razor_property_type (eg, RAZOR_PROPERTY_REQUIRES), and relation
krh@311:       is an enum razor_version_relation (eg,
krh@311:       RAZOR_VERSION_GREATER_OR_EQUAL). packages is a list of the
krh@311:       packages that have this property. flags is currently unused.
krh@311:     </para>
krh@311: 
krh@311:     <programlisting><![CDATA[
krh@311: struct razor_entry
krh@311: 	uint name  : 24;
krh@311: 	uint flags : 8;
krh@311: 	uint start : 32;
krh@311: 	struct list_head packages;
krh@311: ]]></programlisting>
krh@311: 
krh@311:     <para>
krh@311:       name is an index into string_pool, giving the basename of the
krh@311:       file. start is either 0, or an index pointing to another
krh@311:       razor_entry that is the first child of this entry (for a
krh@311:       non-empty directory). (Entry 0 is always the root of the tree,
krh@311:       so no entry could have entry 0 as a child.) flags is 0x80
krh@311:       (RAZOR_ENTRY_LAST) if an entry is the last entry in its
krh@311:       directory. Otherwise it is 0.
krh@311:     </para>
krh@311: 
krh@311:     <para>
krh@311:       Note that given a pointer to a struct_razor_entry (eg, from a
krh@311:       package's "files" list), there is no way to reconstruct its full
krh@311:       name without walking the entire files array up to that
krh@311:       point. Because of this and other problems (fix_file_map()), it
krh@311:       seems like razor_entry should be modified to include a pointer
krh@311:       to its parent. (Storing full paths instead of just basenames
krh@311:       would also fix this problem, but that would use much more
krh@311:       memory.)
krh@311:     </para>
krh@311:   </sect2>
krh@311: </chapter>