User:Ffesti/NextLayer

The Next Layer

Motivation

Handling the huge number of packages has become more and more difficult over the years. While the number of installed packages has not grown as fast as the available packages it is still close to impossible to understand what software is installed on a system by looking at the package list. Ideally the system should not look more complicated that the decisions the user made to set it up. If the user only selected half a dozen of groups and installed 20 additional packages the description of the system should not have more than those 26 items.

There are several issues that make a better representation of the installed packages impossible:

Comps groups are just used for installing. Although yum does store that a group got installed somewhere this information is no longer used. Neither do changes of the group have any consequences nor can one reliably link groups and packages.

While anaconda - and now also yum - supports installing a group except some of the packages such concept is gone as soon as the transaction is finished. This makes bringing groups and the installed packages together even more difficult. As many administrators only use comps groups as a starting point for further refinement this is a serious problem.

To solve this issues groups need to become first class citizens in the yum world. Actually they need to be come a layer that is on top of packages as they already are in the repositories. But they still have to gain the same status on the system.

This could also solve another problem: Telling packages only installed from dependencies from packages wanted and installed by the user. Actually yum already stores this information on a per package basis. Nevertheless this information is of very little use right now. One reason is the missing understanding of groups which make basically half of the packages "selected by the user" even if he made just 20 decision for installing his machine.

The only way to solve all this is to store the decisions the user made in an editable form. This form then needs to be translated into a actual list of packages to be installed. Groups could be this form.

Installed Groups

Installed comps group contain a copy of the original comps group plus the decisions made while installing the group. This includes what set of packages (mandatory, default, optional) are to be installed, which are excluded from that and what (optional) packages are selected in addition. The copy of the original comps group keeps the installed group functional even if the repository with the comps file vanished or is disabled.

Whenever an installed group is changed or removed or the associated comps group changes the installed packages need to be adjusted accordingly. Packages no longer needed by the group (and nothing else) should be silently removed. This frees the administrator from cleaning up no longer needed packages by hand. But requires to reliably know what packages are just installed by dependency.

If the comps group in the repository is changed "yum update" should also update the installed group and install or remove packages accordingly.

This model raises the question what to do with the packages not in a (comps) group?

Standalone Groups

One solution would be creating groups on user demand that are not connected to any comps group. The user could add just arbitrary packages. May be other groups or comps groups can be used as templates but standalone groups would often just be created from scratch. This would also allow the administrator to group packages by purpose. An additional description or comment could be used for documentation. Beside just grouping the packages those groups could also document who uses or requires the packages. Packages needed for different reasons could be part of different groups and would be kept around while any of this groups still exists ("Reference counting").

Groups only

The question is if it really is necessary to have anything else than groups. The current mechanism of marking packages as "user installed" and "by requirement" on installation time does not reflect the actual use cases precisely. Even if a package got installed as an dependency it might still be or become demanded by a user. The current implementation does not offer a way of switching the status except reinstalling the package which is a pretty big hammer for that nail. While a command to alter the status could be added easily there is an alternative implementation: Use some sort of default group packages installed "by hand" get added to. This group could then be edited to change the status of a particular package. This would also make such packages accessible to all tools (or GUIs) that work on groups. Also "old style" yum commands dealing with packages can be integrated into the new paradigm.

Editing Installed Groups

Beside installing groups from comps editing installed or newly create groups will become the most important operation. Do do this different interfaces are possible.

Pure Command line

A new set of commands do not just add and remove packages from the system but work on a group given as parameter. May be different names should be chosen to signal that the packages are not installed or removed directly but just added an removed from the group.

Interactive Command line

To do bigger changes the group could be represented as a text file that is opened in an editor and can be changed in there. After the editor is closed the changes are parsed, saved into the group and executed. This would work similar to the "git rebase -i" command. The packages would be represented in the file similar to this:

• Packages installed by comps status (mandatory, default, optional): commented out line with prepended "-". Uncommenting it will exclude the package
• Packages selected although the comps status is not installed: just the package name. Commenting or deleting the entry will unselect the package
• Packages excluded as they would be installed by their comps status: "-" and package name. Commenting or removing this line would re-add the package to the group

A very small example group could look like this:

### Group Administration Tools
### follows "Administration Tools"
### installs mandatory, default
### Installed "mandatory" and "default" packages
### Uncomment to remove package
# -authconfig-gtk
# -gnome-packagekit
# -system-config-boot
# -system-config-date
# -system-config-firewall
# -system-config-keyboard
# -system-config-language
-system-config-lvm
-system-config-network
# -system-config-users

### Optional packages
### Comment to remove; uncomment to add
cacti
# dbench
# etherape
# ettercap-gtk
# galternatives
gparted
# gpointing-device-settings
# kde-partitionmanager
kpackagekit
# lat
# lshw-gui
# luma
# ntop
# pessulus
qtparted
system-config-kickstart
# system-config-rootpassword
# yumex

Graphical

The UI for comps groups would pretty much look the same as right now. For standalone groups (and may be even comps groups) there'd be an interface to add new packages. Either by selecting packages from the global lis/tree and then press a "Add to group" button or with a drag and drop interface that shows the selected group content side by side to the available packages. This is left to the GUI designer.

---

An user created group would look very similar to an installed comps group except that it would not follow any external group. The idea is that users create groups for a given purpose. That way the groups can serve as documentation on what is installed and why. Typical groups could be a tool chain or devel packages for a project. Groups can and should overlap if necessary. So packages can be required from several groups.

Group commands

• • out dated **

So the operation of the package manager look very different from yum's:

• grouplist [available|installed]
  • May be also prints statistics on how many packages in each state for installed groups
• groupinfo GROUP
  • give group details
  • switch for machine readable format for installed groups
• info PACKAGE
  • show details for package
  • if installed explain why the package is installed (member/dependency of group(s))
• groupinstall GROUP
  • GROUP can either be a group name or a file
  • if no cli param is given open an editor to select which optional packages to select and to add a comment
  • cli switch allows installing the group immediately (selecting mandatory, default or optional packages)
• groupedit GROUP
  • opens editor allowing adding and removing packages (optional and arbitrary) or modifying the comment(s)
  • afterwards packages are added and removed to keep the system in sync with the group
  • there are commands to alter the group from the cmd line
• groupnew GROUP [OTHERGROUP, ...]
  • create new group with the given name
  • opens editor if details are not given at the cmd line
  • can use other groups as template
• groupremove GROUP
  • remove group and all packages no longer required

Graphical tools would do similar things by drag and dropping packages into new groups or checking boxes of optional packages very much similar to what we do today.

Yum commands

To have persistent group support in yum a number of commands would need additions.

• install package1 [package2] [...]
  • additional param to to tell yum what group the packages to add to ("DefaultGroup" if not given)
• update [package1] [package2] [...]
  • also check for and do updates for comps groups
• distribution-synchronization [package1] [package2] [...]
  • ??
• remove | erase package1 [package2] [...]
  • remove or add excludes to groups containing removed pakcages
  • Command should be discouraged in docs
• list [...]
  • list installed groups
• info [...]
  • for installed packages give details why package is installed
    • Is this the right command to do this?
• provides | whatprovides feature1 [feature2] [...]
• groupinstall group1 [group2] [...]
  • Do a real group install
  • param for group_package_types
  • param for interactive editing before installing
• groupupdate group1 [group2] [...]
  • Do a real update of the group
• grouplist [hidden] [groupwildcard] [...]
  • list installed groups (may be including a bit of statistics per group)
• groupremove group1 [group2] [...]
  • remove persistent group. Only offer groupremove_leaf_only behavior
• groupinfo group1 [...]
  • also work on installed groups
• version [ all | installed | available | group-* | nogroups* | grouplist | groupinfo ]
  • ??
• history [info|list|summary|redo|undo|new|addon-info]
  • ??
• check
  • ??

Dealing with Inconsistencies

The idea is that the installed packages do only depend on the groups installed (otherwise it'd be not sufficient to just look at the groups). This will require a slightly different implementation of updates. Packages no longer required need to be removed.

In addition an update mechanism for groups is needed. It is still open if comps groups should have a version number or if we can just have a hash or stamp that we compare with the version of the group we have installed. As it is possible that the group list do no longer match the packages in repositories there is a new class of errors. As we want automatic updated to continue working the implementation should handle them gracefully but keep track of them. All interactive tools should list such inconsistencies and ask the user to solve them.

If yum or rpm is still around it is possible that the installed packages do no longer match the groups. It is though important to detect that the package list was changed "behind the back". These chances need the to be integrated into the group world. Packages that are required by groups should be marked as deleted/deselected. New packages should be added to a "Unknown" group. Similar to other mismatch errors the user should be asked to clarify the situation.

Problematic situations:

• Package listed in group does not exist in repository
  • may or may not be installed on the system
    • may be ignore if package was installed from file
• Package selected in group is no longer in comps group (as an previously optional package)
  • keep entry but warn user
• Package listed in group got deleted by hand
  • Remove from group/ Add deleted entry to avoid installing it again
• New Package installed by hand
  • Add to dummy group
• Package is obsoleted
  • Maintain an obsoleted list to translate entries in groups that did not yet do the transition

Conversion of installed systems

If a system already was installed with a previous versions yum it has to be converted to the new group model. Matching the comps groups against the installed packages (see show-installed from yum-utils) can be a good starting point. Nevertheless such conversion can not be perfect as it cannot tell which package got installed by dependency and which on purpose. After a first conversion the administrator should have a look if the result matches his intent.