1 Kris's original TODO list follows:
3 ----------------------------------------------------------------------
4 Towards replacing rpm + yum (0.1):
6 - drop the filelists from the main package set file, split out to a
7 secondary file. for package sets that depend on other package sets,
8 we need to be able to generate properties with owning packages that
9 are in another set. this way, a package that requires a file, will
10 look up the provides in the set and find the package that owns it
11 and then try mark that for update.
15 - pre install check; check that dirs can be created (no files where
16 want to create dirs), move config files according to file
19 - store rpm headers for installed packages.
21 - implement rpm uninstall and update.
23 - triggers? just say no?
25 - rpm seems to consider glibc > 2.6.90 to mean greater than
26 2.6.90-anything. That is, a comparison that doesn't mention the
27 release field, shouldn't regard the release field of pkgs it
28 compares against. glibc-common-2.6.90 has
30 conflicts: glibc < 2.6.90, glibc > 2.6.90
32 since rpm doesn't let you do glibc != 2.6.90, and
34 requires: glibc = 2.6.90
36 will always pull in glibc. But even with a != relation, would
37 glibc-2.6.90-16 be equal to 2.6.90? glibc 2.7.90-8 dropped it in
38 favor of requires = 2.7.90-8 (#225806).
42 - space calculation before transaction, but ideally, do a number of
45 - pre-link changing binaries and libs on disk screwing up checksum?
47 - pipelined download and install; topo-sort packages in update set,
48 pick one with all deps in the current set, add it to the current set
49 and satisfy deps against update set => result: minimal update
50 transaction. Queue download and install/update transaction for the
51 packages in the minimal set, start over. This also makes the
52 installation phase much more interruptible, basically just stop
53 after a sub-transaction finishes. As we keep the update set around
54 as a target, we can restart if needed. Probably don't need to, can
55 just do a new update. During a sub-transaction we should keep the
56 target set (i.e. the current set to be) around as a lock file
57 (system.rzdb.lock or so, see git) so that razor updates are
58 prevented if the systems crashes during an update.
60 - implement depsolving between multiple package sets by creating an
61 iterator that has a sorted list of all installed pkgs from all sets,
62 all installed requires from all sets, all installed provides from
63 all sets etc. could be a list of tuples (pkgs index, set index).
64 should simplify even the two-set depsolving a bit since we can
65 pretend there's just one set. this should also be useful for the
66 'overlay set' idea where the system set is actually made up of a
67 number of sets, but typically a read-only set from a read-only fs
68 and a read-write set from a r/w fs.
70 - locking: we use advisory file locking on the system set
71 (/var/lib/razor/system.rzdb) to indicate a transaction is in
72 progress. The locking algorithm is as follows:
74 1. obtain advisory lock on system set. if this is already taken,
75 we know that a process is actively modifying the system set and
76 we have to wait. there's a fcntl that lets you block for the
79 2. if a system-next.rzdb file already exists an earlier razor
80 process was interrupted or crashed and we may want to clean
81 that up. the system-next.rzdb file will record what the
82 previous instance was trying to do and we can just replay that
85 3. create the new package set whichever way and write it to
86 system-next.rzdb, then start installing/removing rpms.
88 4. When the update is complete, rename system-next.rzdb to
89 system.rzdb and remove the advisory lock.
91 we should probably introduce a new object that encapsulates this
92 sequence, the filename conventions, rpm cache, e.g. struct
93 razor_image, with operations such as
95 #define RAZOR_IMAGE_READ 0x01
96 #define RAZOR_IMAGE_WRITE 0x02
99 razor_image_open(const char *root, unsigned int flags);
102 razor_image_begin_transaction(struct razor_image *image,
103 struct razor_set *target);
106 razor_image_finish_transaction(struct razor_image *image);
108 the transaction pipelineing described above sits on top of this,
109 since each step there needs to complete a full transaction that
110 writes out a new package set.
112 for overlay package sets we could do something like
115 razor_image_open_with_base(const char *root, unsigned int flags,
116 struct razor_image *base);
118 where base specifies the r/o package set it's layered on. this
119 allows for stacking several layers of images.
122 Package set file format items:
124 - nail down byte-order of repo file.
126 - version the sections in the file, put the element size in the header
127 so we can add stuff to elements in a backwards compatible way.
128 maybe not necessary, we can just add sections that augment the
129 sections we want to add to (similar to how rpm has add versioned
135 - keep history of installed packages/journal of package transaction,
136 so we can roll back to yesterday, or see what got installed in the
139 - use hash table for package and property lists so we only store
140 unique lists (like for string pool).
142 - use existing, running system as repo; eg
144 razor update razor://other-box.local evince
146 to pull eg the latest evince and dependencies from another box. We
147 should be able to regenerate a rzr pkg from the system so we can
148 reuse the signature from the originating repo.
150 - Ok, maybe the fastest package set merge method in the end is to use
151 the razor_importer, but use a hash table for the properties. This
152 way we can assign them unique IDs immediately (like tokenizing
155 - test suite should be easy, just keep .rzdb files around and test
156 different type of upgrades that way (obsoletes, conflicts, file
157 conflicts, file/dir problems etc). Or maybe just keep a simple file
158 format ad use a custom importer to create the .rzdb files.
160 - overlay package sets? mount a read-only /usr over nfs or from the
161 virt-host and have a local package set overlaid over the read-only
162 one. shouldn't need new features in the core package set data
163 structure, but should be just conventions on top. we have the base
164 package set from the r/o system, the overlay set from the local
165 system and we can have an effective package set which is the merge
166 of everything from the overlay into the base set. the effective set
167 is easy to compute and we could do it on the fly or cache it. or
168 maybe the effective set is the on-disk representation and the
169 overlay can be computed when needed, we just keep a link back to the
172 - incremental rawhide repo updates? instead of downloading 10MB zipped
173 repo every time, download a diff rzdb? Should be pretty small,
174 especially if we don't have file checksums in metadata. Filenames
175 and properties are for the most part already present, typically just
176 a version bump plus maybe tweaking a couple requires. The upstream
177 repo can store multiple incremental updates in one big file and
178 provide an index file that maps updates for a given date (we should
179 use repo-file checksums though) to a range in the file: Download the
180 index file, search for a match for your latest rawhide.rzdb file,
181 download range of updates that brings it up to date.
183 - use hash tables for dirs when importing files to avoid qsorting all
188 - eliminate duplicate entries in package property lists.