<?xml version="1.0" encoding="utf-8"?>

 <!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN" "http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd">

 <chapter id="solver">

   <title>Dependency Solver</title>

   <para>

     At a very high level, yum's depsolver does something roughly

     equivalent to:

     - For each package being installed or removed

 	- For each relevant property (provides, requires, conflicts,

           obsoletes):

 	    - Figure out what additional packages need to be added to

 	      or removed from the system to satisfy this property

     which ends up being roughly O(N^2 * M) where N is the total number of

     properties and M is the number of packages being acted on.

 (I just figured that out off the top of my head, and I'm not totally

 familiar with the yum code, so it may be wrong.)

 Razor's depsolver is something like:

     - do {

 	- For each property to be added to or removed from the system:

 	    - Figure out what packages need to be added to or removed

 	      from the system to satisfy this property

     - } until we stop adding/remove more packages

 with the key being that it's very easy to find the PROVIDES

 corresponding to a REQUIRES and vice versa, because the property

 arrays are sorted, and so all properties with the same "name" will be

 adjacent to one another in the array, allowing many dependencies to be

 satisified in essentially constant time. (Actually... we've been

 calling it constant, but it's really O(log N) for heavily-depended-on

 packages, because the more packages you have, the more variations on

 "requires foo", "requires foo = 1.1", "requires foo > 1.0", etc you're

 going to have to scan through.)

 Ideally though, each iteration of the inner loop body happens in

 constant time, and thus the inner loop as a whole is O(N), and thus

 the depsolver as a whole is O(N * M) (or at least, less than

 O(N * M * log N).

 FILE DEPENDENCIES

 -----------------

 Whenever we add a package with a file REQUIRES to a razor_set, we also

 add a PROVIDES for that file to the package or packages which provide

 that file. This means that if we later add another package that

 requires the same file (eg, /bin/sh or /usr/bin/perl), we can resolve

 its file requirement exactly like we would resolve a property

 requirement, in nearly constant time.

 When adding a *new* file requirement (ie, a requirement on a file that

 no existing package depends on), we still have to scan through the

 file tree, which is O(log N) in the number of files.

 (AFAICT, there's no reason yum couldn't do the same optimization.

 Also, AFAICT, yum currently sticks property dependencies and file

 dependencies into the same hash table, so that if any package in the

 transaction has a file dependency, it causes *property* dependencies

 to become slower to resolve as well...)

 THE RULES

 ---------

 This is what we have figured out for transaction-solving rules;

 neither yum nor rpm's algorithm seems to be explained in full

 anywhere...

     1. Every requested install in the initial package set must be

        satisfied as either a new install or an update:

 	- if the requested package name is the name of an upstream

           package:

 	    - if there is not a corresponding already-installed

               package, then install the upstream package

 	    - else if the upstream package is newer than the

               already-installed package, then update the package

 	    - else it's an error (UP_TO_DATE)

 	- else if the requested package name is the name of an

           already-installed package:

 	    - if there is an upstream package that obsoletes the

               already-installed package, then behave as though the

               user had requested that that package be installed

               instead.

 	    - else it's an error (UP_TO_DATE or INSTALL_UNAVAILABLE?)

 	- else it's an error (INSTALL_UNAVAILABLE)

     2. Every requested removal in the initial package set must be

        satisfied as a removal. If any requested package name is not

        the name of an installed package, it's an error

        (REMOVE_NOT_INSTALLED)

     REQUIRES processing:

     3. If a package being installed or updated-to REQUIRES a property

        that is not provided by any installed or to-be-installed

        package, we need to find an installable package that provides

        that property. If we find one, install/update it. If not, it's

        an error (UNSATISFIABLE). (If we find an upstream package

        providing the property that corresponds to a system package

        that's being removed, then it's a CONTRADICTION.)

     4. If an already-installed package REQUIRES a property which is

        only provided by a package that is being removed, then that

        package needs to be removed as well.

     5. If an already-installed package REQUIRES a property which is

        only provided by a package that is being upgraded or obsoleted

        (to a new package which does not provide that property), then:

 	- if there is an update for the installed package, then update

           the installed package

 	- else if there is another installable package that provides

           the required property, then install that.

 	- else it's an error (UNSATISFIABLE?)

     CONFLICTS processing

     6. If a package being installed or updated-to CONFLICTS with a

        property provided by an installed package:

 	- if there is an update for the installed package, which the

           new package does not conflict with, then update the

           installed package.

 	- else it's an error (NEW_CONFLICT)

     7. If an already-installed package CONFLICTS with a property

        provided by a to-be-installed package:

 	- if there is an update for the installed package, which does

           not conflict with the new package, then update the installed

           package.

 	- else it's an error (NEW_CONFLICT)

     8. If a package being installed or updated-to CONFLICTS with a

        property provided by a to-be-installed package, then it's an

        error (CONTRADICTION).

     OBSOLETES processing. NOTE: OBSOLETES are only matched against

     package names, not against arbitrary provided properties

     9. If a package being installed or updated-to OBSOLETES an

        installed package, then obsolete that package. (ie, remove it,

        but treat it as updated for purposes of dangling REQUIRES).

    10. If an already-installed package OBSOLETES a to-be-installed

        package, then it's an error. (ALREADY_OBSOLETE)

    11. If a package being installed or updated-to OBSOLETES another

        package being installed or updated-to, then it's an error

        (CONTRADICTION).

 THE DEPSOLVER

 -------------

 We start with two razor_sets, system and upstream, and a list of

 requested installations and removals.

     FIXME: what about multiple upstream repos? Having to deal with

     arbitrary numbers of razor_sets is possible, but will probably be

     messy... It might be easier to either store all upstream repo data

     in a single .rzdb file, or else merge all upstream .rzdb files

     together into a single razor_set at startup. (Or some combination

     of those.)

 We create a bit array of the packages in each set, indicating which

 ones are installed; the system bitarray starts out all 1s, and the

 upstream bitarray all 0s. Each bit is only allowed to change state

 once during the transaction; an installed package can be removed, or

 an uninstalled package installed, but trying to reinstall a removed

 package, or uninstall a newly-installed package is an error. This

 means the packages break down into four categories:

     - installed       (1 bit in the system bit array)

     - to-be-removed   (0 bit in the system bit array)

     - to-be-installed (1 bit in the upstream bit array)

     - installable     (0 bit in the upstream bit array)

 Depsolver algorithm:

     - Create new razor_transaction_packages ("rtp"s) for each

       requested install or remove. These will be "unresolved", because

       we haven't yet found the razor_packages that correspond to them.

     - while there are new rtps:

 	- sort the new rtps

 	- Walk the system property list, upstream property list, and

           new rtp list in parallel, and:

 	    - For each uninstalled PROVIDES:

 		- If the property is a valid package name (that is,

                   either it's a package providing its own name, or it

                   has a matching OBSOLETES), and it matches the name

                   of a new rtp of type INSTALL or FORCED_UPDATE with

                   an unresolved new_package:

 		    - If the upstream package has the same version as

 		      the system package, we have an UP_TO_DATE error

 		      (FIXME: not quite right. This doesn't deal with

 		      the case where we try to update an application

 		      because of a library update, and it turns out

 		      there's no new version of the application, but

 		      there IS a compat package containing the old

 		      version of the library.)

 		    - Otherwise, set the rtp's new_package to point to

 		      the package providing this property and set the

 		      appropriate bit in the upstream bit array.

 	    - For each to-be-installed non-file REQUIRES:

 		- See if there's an installed or to-be-installed

 		  package that PROVIDES that property.

 		- If not, see if there's an installable package that

 		  PROVIDES that property, and create a new INSTALL rtp

 		  for it if so.

 		- If not, see if there's a to-be-removed package that

 		  PROVIDES that property. (If we find such a package,

 		  we have a CONTRADICTION error.)

 		- If none of the above, then we have an UNSATISFIABLE

                   error

 	    - For each to-be-installed file REQUIRES:

 		- (We create fake file PROVIDES to match file REQUIRES

                   when importing/merging razor sets, so if there is

                   already another installed package that REQUIRES this

                   file, there will be a PROVIDES listed for it as well.)

 		- See if there's an installed package that PROVIDES

                   that file.

 		- If not, do a binary search of the system file tree

                   looking to see if some installed package provides

                   that file but does not have a PROVIDES for it.

 		- If not, see if there's an installable package that

 		  PROVIDES that property, and create a new INSTALL rtp

 		  for it if so.

 		- (If we actually work with multiple upstream

                   razor_sets, then we will need to search the upstream

                   file trees at this point, because it's possible that

                   a package in one upstream repo would require a file

                   in another upstream repo. But if we merge the

                   multiple upstream repos into a single razor_set at

                   some point, then we would not need to do that,

                   because it would be guaranteed that we would have

                   already created a fake PROVIDES if any package

                   provides the file.)

 		- If no installed or installable package provides the

 		  file, see if there's a to-be-removed package that

 		  provides the file. (If we find such a package, we

 		  have a CONTRADICTION error.)

 		- If none of the above, then we have an UNSATISFIABLE

                   error

 	    - For each to-be-installed PROVIDES:

 		- Check if the new PROVIDES conflicts with an

 		  installed CONFLICTS. If so, create a new

 		  FORCED_UPDATE rtp for the installed package, so we

 		  can try to upgrade it to a non-conflicting version.

 		  (If we can't, we'll have an OLD_CONFLICT error.)

 		- Check if the new PROVIDES conflicts with an

                   installed OBSOLETES *and* the PROVIDES property

                   corresponds to the name of its package. (That is,

                   OBSOLETES are only matched against package names,

                   not arbitrary provided properties.) If so, we have

                   an ALREADY_OBSOLETE error.

 		- Check if the new PROVIDES conflicts with a

 		  to-be-installed CONFLICTS. If so, we have a

 		  CONTRADICTION error.

 	    - For each to-be-installed CONFLICTS:

 		- Basically the reverse of the previous case: check if

 		  the new CONFLICTS conflicts with an installed

 		  PROVIDES. If so, create a new FORCED_UPDATE rtp for

 		  the installed package, so we can try to upgrade it

 		  to a non-conflicting version. (If we can't, we'll

 		  have an NEW_CONFLICT error.)

 		- Check if the new CONFLICTS conflicts with a

 		  to-be-installed PROVIDES. If so, we have a

 		  CONTRADICTION error.

 	    - For each to-be-installed OBSOLETES:

 		- Check if there's an installed package that PROVIDES

 		  that property. If so, create an OBSOLETED rtp for

 		  the installed package.

 		- If not, check if there's a to-be-installed package

 		  that PROVIDES that property. If so, we have a

 		  CONTRADICTION error.

 	    - For each installed PROVIDES:

 		- If the property is a valid package name (that is,

                   it's a package providing its own name), and it

                   matches the name of a new rtp with an unresolved

                   old_package, then set the rtp's old_package to point

                   to the package providing this property and clear the

                   appropriate bit in the system bit array.

 	    - For each to-be-removed PROVIDES:

 		- If there's also an identical to-be-installed

 		  PROVIDES, we're ok and can skip this

 		- Otherwise, for each installed REQUIRES of this

                   property:

 		    - Look for some other installed or to-be-installed

 		      property that satisfies the REQUIRES.

 		    - If there isn't one, then for each installed

 		      package in this REQUIRES's package list:

 			- If the PROVIDES was lost because the old

 			  package was REMOVEd (not FORCED_UPDATE or

 			  OBSOLETED), then create a new REMOVE rtp for

 			  this package.

 			- Otherwise, create a new FORCED_UPDATE rtp

                           for this package.

 		- (We don't need to look at to-be-installed REQUIRES

 		  of this property, because if there are any, they

 		  will cause a CONTRADICTION error when we try to

 		  re-satisfy them the next time through.)

   </para>

 </chapter>