path: root/chapter08/pkgmgt.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE sect1 PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
  "http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
  <!ENTITY % general-entities SYSTEM "../general.ent">
  %general-entities;
]>

<sect1 id="ch-system-pkgmgt">
  <?dbhtml filename="pkgmgt.html"?>

  <title>Package Management</title>

  <para>Package Management is an often requested addition to the LFS Book. A
  Package Manager tracks the installation of files, making it easier to
  remove and upgrade packages. A good package manager will also handle the
  configuration files specially to keep the user configuration when the
  package is reinstalled or upgraded.  Before
  you begin to wonder, NO&mdash;this section will not talk about nor recommend
  any particular package manager. What it does provide is a roundup of the more
  popular techniques and how they work. The perfect package manager for you may
  be among these techniques, or it may be a combination of two or more of these
  techniques. This section briefly mentions issues that may arise when upgrading
  packages.</para>

  <para>Some reasons why no package manager is mentioned in LFS or BLFS
  include:</para>

  <itemizedlist>
    <listitem>
      <para>Dealing with package management takes the focus away from the goals
      of these books&mdash;teaching how a Linux system is built.</para>
    </listitem>

    <listitem>
      <para>There are multiple solutions for package management, each having
      its strengths and drawbacks.  Finding one solution that satisfies all audiences
      is difficult.</para>
    </listitem>
  </itemizedlist>

  <para>There are some hints written on the topic of package management. Visit
  the <ulink url="&hints-root;">Hints Project</ulink> and see if one of them
  fits your needs.</para>

  <sect2 id='pkgmgmt-upgrade-issues'>
    <title>Upgrade Issues</title>

    <para>A Package Manager makes it easy to upgrade to newer versions when they
    are released. Generally the instructions in the LFS and BLFS books can be
    used to upgrade to the newer versions. Here are some points that you should
    be aware of when upgrading packages, especially on a running system.</para>

    <itemizedlist>
      <listitem>
        <para>If the Linux kernel needs to be upgraded (for example, from
        5.10.17 to 5.10.18 or 5.11.1), nothing else needs to be rebuilt.
        The system will keep working fine thanks to the well-defined interface
        between the kernel and userspace.  Specifically, Linux API headers
        need not be (and should not be, see the next item) upgraded
        along with the kernel.  You will merely need to reboot your system to use the
        upgraded kernel.</para>
      </listitem>

      <listitem>
        <para>If the Linux API headers or Glibc need to be upgraded to a newer
        version, (e.g.,  from Glibc-2.31 to Glibc-2.32), it is safer to
        rebuild LFS.  Though you <emphasis>may</emphasis> be able to rebuild
        all the packages in their dependency order, we do not recommend
        it.  </para>
      </listitem>

      <listitem>
        <para>Reinstalling the same version of Glibc (&glibc-version; for
        this release of LFS) with patches should be safe when these patches
        do not change ABI and API.  When a security vulnerability is found
        in Glibc, we often need to apply such a patch to fix the
        vulnerability and reinstall Glibc.  Consult
        <ulink url='&lfs-root;lfs/advisories/'>LFS security
        advisories</ulink> if you are alerted for a published Glibc security
        vulnerability but unsure about the action to take.</para>
      </listitem>

      <listitem> <para>If a package containing a shared library is updated, and
      if the name of the library changes, then any packages dynamically
      linked to the library must be recompiled, to link against the
      newer library.  (Note that there is no correlation between the package
      version and the name of the library.) For example, consider a package
      foo-1.2.3 that installs a shared library with the name <filename
      class='libraryfile'>libfoo.so.1</filename>. Suppose you upgrade the package to
      a newer version foo-1.2.4 that installs a shared library with the name
      <filename class='libraryfile'>libfoo.so.2</filename>. In this case, any
      packages that are dynamically linked to <filename
      class='libraryfile'>libfoo.so.1</filename> need to be recompiled to link
      against <filename class='libraryfile'>libfoo.so.2</filename> in order to
      use the new library version. You should not remove the old
      libraries until all the dependent packages have been recompiled.</para>
      </listitem>

      <listitem><para>If a package is (directly or indirectly) linked to both
      the old and new names of a shared library (for example, the package
      links to both <filename class='libraryfile'>libfoo.so.2</filename> and
      <filename class='libraryfile'>libbar.so.1</filename>, while the latter
      links to <filename class='libraryfile'>libfoo.so.3</filename>), the
      package may malfunction because the different revisions of the shared
      library present incompatible definitions for some symbol names. This can be
      caused by recompiling some, but not all, of the packages linked to the
      old shared library after the package providing the shared library is
      upgraded.  To avoid the issue, users will need to rebuild every package
      linked to a shared library with an updated revision (e.g. libfoo.so.2 to
      libfoo.so.3) as soon as possible.
      </para></listitem>

      <listitem> <para>If a package containing a shared library is updated,
      and the name of the library doesn't change, but the version number of the
      library <emphasis role="bold">file</emphasis> decreases (for example,
      the library is still named
      <filename class='libraryfile'>libfoo.so.1</filename>,
      but the name of the library file is changed from
      <filename class='libraryfile'>libfoo.so.1.25</filename> to
      <filename class='libraryfile'>libfoo.so.1.24</filename>),
      you should remove the library file from the previously installed version
      (<filename class='libraryfile'>libfoo.so.1.25</filename> in this case).
      Otherwise, a <command>ldconfig</command> command (invoked by yourself from the command
      line, or by the installation of some package) will reset the symlink
      <filename class='libraryfile'>libfoo.so.1</filename> to point to
      the old library file because it seems to be a <quote>newer</quote>
      version; its version number is larger.  This situation may arise if
      you have to downgrade a package, or if the authors change the versioning
      scheme for library files.</para> </listitem>

      <listitem><para>If a package containing a shared library is updated,
      and the name of the library doesn't change, but a severe issue
      (especially, a security vulnerability) is fixed, all running programs
      linked to the shared library should be restarted.  The following
      command, run as <systemitem class="username">root</systemitem> after
      the update is complete, will list which processes are using the old versions of those libraries
      (replace <replaceable>libfoo</replaceable> with the name of the
      library):</para>

<screen role="nodump"><userinput>grep -l '<replaceable>libfoo</replaceable>.*deleted' /proc/*/maps | tr -cd 0-9\\n | xargs -r ps u</userinput></screen>

      <para>
        If <application>OpenSSH</application> is being used to access
        the system and it is linked to the updated library, you must
        restart the <command>sshd</command> service, then logout, login again,
        and run the preceding command again to confirm that nothing is still using the
        deleted libraries.
      </para>

      <para revision='systemd'>
        If the <command>systemd</command> daemon (running as PID 1) is
        linked to the updated library, you can restart it without rebooting
        by running <command>systemctl daemon-reexec</command> as the
        <systemitem class='username'>root</systemitem> user.
      </para></listitem>

      <listitem>
        <para>If an executable program or a shared library is overwritten, the processes
        using the code or data in that program or library may crash.  The
        correct way to update a program or a shared library without causing
        the process to crash is to remove it first, then install the new
        version. The <command>install</command> command
        provided by <application>coreutils</application> has already
        implemented this, and most packages use that command to install binary files and
        libraries.  This means that you won't be troubled by this issue most of the time.
        However, the install process of some packages (notably SpiderMonkey
        in BLFS) just overwrites the file if it exists; this causes a crash. So
        it's safer to save your work and close unneeded running processes
        before updating a package.</para> <!-- binary is an adjective, not a noun. -->
      </listitem>
    </itemizedlist>

  </sect2>

  <sect2>
    <title>Package Management Techniques</title>

    <para>The following are some common package management techniques. Before
    making a decision on a package manager, do some research on the various
    techniques, particularly the drawbacks of each particular scheme.</para>

    <sect3>
      <title>It is All in My Head!</title>

      <para>Yes, this is a package management technique. Some folks do not 
      need a package manager because they know the packages intimately
      and know which files are installed by each package. Some users also do not
      need any package management because they plan on rebuilding the entire
      system whenever a package is changed.</para>

    </sect3>

    <sect3>
      <title>Install in Separate Directories</title>

      <para>This is a simplistic package management technique that does not need a
      special program to manage the packages. Each package is installed in a
      separate directory. For example, package foo-1.1 is installed in
      <filename class='directory'>/opt/foo-1.1</filename>
      and a symlink is made from <filename>/opt/foo</filename> to
      <filename class='directory'>/opt/foo-1.1</filename>. When
      a new version foo-1.2 comes along, it is installed in
      <filename class='directory'>/opt/foo-1.2</filename> and the previous
      symlink is replaced by a symlink to the new version.</para>

      <para>Environment variables such as <envar>PATH</envar>,
      <envar>MANPATH</envar>, <envar>INFOPATH</envar>,
      <envar>PKG_CONFIG_PATH</envar>, <envar>CPPFLAGS</envar>,
      <envar>LDFLAGS</envar>, and the configuration file
      <filename>/etc/ld.so.conf</filename> may need to be expanded to
      include the corresponding subdirectories in
      <filename class='directory'>/opt/foo-x.y</filename>.</para>

      <para>
        This scheme is used by the BLFS book to install some very large
        packages to make it easier to upgrade them.  If you install more
        than a few packages, this scheme becomes unmanageable.  And some
        packages (for example Linux API headers and Glibc) may not work well
        with this scheme.
        <emphasis role='bold'>Never use this scheme system-wide.</emphasis>
      </para>
    </sect3>

    <sect3>
      <title>Symlink Style Package Management</title>

      <para>This is a variation of the previous package management technique.
      Each package is installed as in the previous scheme. But instead of
      making the symlink via a generic package name, each file is symlinked into the
      <filename class='directory'>/usr</filename> hierarchy. This removes the
      need to expand the environment variables. Though the symlinks can be
      created by the user, many package managers use this approach, and
      automate the creation of the symlinks. A few of the popular ones include Stow,
      Epkg, Graft, and Depot.</para>

      <para>The installation script needs to be fooled, so the package thinks
      it is installed in <filename class="directory">/usr</filename> though in
      reality it is installed in the
      <filename class="directory">/usr/pkg</filename> hierarchy. Installing in
      this manner is not usually a trivial task. For example, suppose you
      are installing a package libfoo-1.1. The following instructions may
      not install the package properly:</para>

<screen role="nodump"><userinput>./configure --prefix=/usr/pkg/libfoo/1.1
make
make install</userinput></screen>

      <para>The installation will work, but the dependent packages may not link
      to libfoo as you would expect. If you compile a package that links against
      libfoo, you may notice that it is linked to
      <filename class='libraryfile'>/usr/pkg/libfoo/1.1/lib/libfoo.so.1</filename>
      instead of <filename class='libraryfile'>/usr/lib/libfoo.so.1</filename>
      as you would expect. The correct approach is to use the
      <envar>DESTDIR</envar> variable to direct the installation. This
      approach works as follows:</para>

<screen role="nodump"><userinput>./configure --prefix=/usr
make
make DESTDIR=/usr/pkg/libfoo/1.1 install</userinput></screen>

      <para>Most packages support this approach, but there are some which do not.
      For the non-compliant packages, you may either need to install the
      package manually, or you may find that it is easier to install some problematic
      packages into <filename class='directory'>/opt</filename>.</para>

    </sect3>

    <sect3>
      <title>Timestamp Based</title>

      <para>In this technique, a file is timestamped before the installation of
      the package. After the installation, a simple use of the
      <command>find</command> command with the appropriate options can generate
      a log of all the files installed after the timestamp file was created. A
      package manager that uses this approach is install-log.</para>

      <para>Though this scheme has the advantage of being simple, it has two
      drawbacks. If, during installation, the files are installed with any
      timestamp other than the current time, those files will not be tracked by
      the package manager. Also, this scheme can only be used when packages
      are installed one at a time. The logs are not reliable if two packages are
      installed simultaneously from two different consoles.</para>

    </sect3>

    <sect3>
      <title>Tracing Installation Scripts</title>

      <para>In this approach, the commands that the installation scripts perform
      are recorded.  There are two techniques that one can use:</para>

      <para>The <envar>LD_PRELOAD</envar> environment variable can be set to
      point to a library to be preloaded before installation.  During
      installation, this library tracks the packages that are being installed by
      attaching itself to various executables such as <command>cp</command>,
      <command>install</command>, <command>mv</command> and tracking the system
      calls that modify the filesystem. For this approach to work, all the
      executables need to be dynamically linked without the suid or sgid bit.
      Preloading the library may cause some unwanted side-effects during
      installation. Therefore, it's a good idea to perform some tests to
      ensure that the package manager does not break anything, and that it logs all the
      appropriate files.</para>

      <para>Another technique is to use <command>strace</command>, which
      logs all the system calls made during the execution of the installation
      scripts.</para>
    </sect3>

    <sect3>
      <title>Creating Package Archives</title>

      <para>In this scheme, the package installation is faked into a separate
      tree as previously described in the symlink style package management section. After the
      installation, a package archive is created using the installed files.
      This archive is then used to install the package on the local
      machine or even on other machines.</para>

      <para>This approach is used by most of the package managers found in the
      commercial distributions. Examples of package managers that follow this
      approach are RPM (which, incidentally, is required by the <ulink
      url="https://refspecs.linuxfoundation.org/lsb.shtml">Linux
      Standard Base Specification</ulink>), pkg-utils, Debian's apt, and
      Gentoo's Portage system.  A hint describing how to adopt this style of
      package management for LFS systems is located at <ulink
      url="&hints-root;fakeroot.txt"/>.</para>

      <para>The creation of package files that include dependency information is
      complex, and beyond the scope of LFS.</para>

      <para>Slackware uses a <command>tar</command>-based system for package
      archives.  This system purposely does not handle package dependencies
      as more complex package managers do.  For details of Slackware package
      management, see <ulink
      url="https://www.slackbook.org/html/package-management.html"/>.</para>
    </sect3>

    <sect3>
      <title>User Based Management</title>

      <para>This scheme, unique to LFS, was devised by Matthias Benkmann, and is
      available from the <ulink url="&hints-root;">Hints Project</ulink>. In
      this scheme, each package is installed as a separate user into the
      standard locations. Files belonging to a package are easily identified by
      checking the user ID. The features and shortcomings of this approach are
      too complex to describe in this section. For the details please see the
      hint at <ulink url="&hints-root;more_control_and_pkg_man.txt"/>.</para>

    </sect3>

  </sect2>

  <sect2>
    <title>Deploying LFS on Multiple Systems</title>

    <para>One of the advantages of an LFS system is that there are no files that
    depend on the position of files on a disk system.  Cloning an LFS build to
    another computer with the same architecture as the base system is as
    simple as using <command>tar</command> on the LFS partition that contains
    the root directory (about 900MB uncompressed for a basic LFS build), copying
    <!-- D. Bryant created LFS 11.2 in October 2022; 900MB is (roughly) the size of his rsync archive. -->
    that file via network transfer or CD-ROM / USB stick to the new system, and expanding
    it.  After that, a few configuration files will have to be changed.
    Configuration files that may need to be updated include:
    <filename>/etc/hosts</filename>,
    <filename>/etc/fstab</filename>,
    <filename>/etc/passwd</filename>,
    <filename>/etc/group</filename>,
    <phrase revision="systemd">
      <filename>/etc/shadow</filename>, and
      <filename>/etc/ld.so.conf</filename>.
    </phrase>
    <phrase revision="sysv">
      <filename>/etc/shadow</filename>,
      <filename>/etc/ld.so.conf</filename>,
      <filename>/etc/sysconfig/rc.site</filename>,
      <filename>/etc/sysconfig/network</filename>, and
      <filename>/etc/sysconfig/ifconfig.eth0</filename>.
    </phrase>
    </para>

    <para>A custom kernel may be needed for the new system, depending on
    differences in system hardware and the original kernel
    configuration.</para>

    <note><para>There have been some reports of issues when copying between
    similar but not identical architectures. For instance, the instruction set
    for an Intel system is not identical with the AMD processor's instructions, and later
    versions of some processors may provide instructions that are unavailable with
    earlier versions.</para></note>

    <para>Finally, the new system has to be made bootable via <xref
    linkend="ch-bootable-grub"/>.</para>

  </sect2>

</sect1>