diff options
author | Xi Ruoyao <xry111@xry111.site> | 2023-11-14 00:27:17 +0800 |
---|---|---|
committer | Xi Ruoyao <xry111@xry111.site> | 2023-11-14 00:27:17 +0800 |
commit | 7a72a22f9ba2c4e665dc4eefec60bc588ab57673 (patch) | |
tree | 43d1349bcc25af69d3bc05fe418bf7a77ea37322 | |
parent | 1780e72d8d14621ffe3aaea89678d2eb93a63bf1 (diff) | |
parent | e07c41946983e891b283482c294ed748494c2578 (diff) |
Merge tag 'xry111/update-20231113' into trunk
There are some suspicious "nptl/tst-robust" test failures in Glibc, but
all other test results are clean. To be diagnosed...
And note that jhalfs needs an update to work with the merged parallelism
changes.
-rw-r--r-- | chapter01/changelog.xml | 38 | ||||
-rw-r--r-- | chapter02/hostreqs.xml | 15 | ||||
-rw-r--r-- | chapter04/aboutsbus.xml | 12 | ||||
-rw-r--r-- | chapter04/settingenviron.xml | 49 | ||||
-rw-r--r-- | chapter07/chroot.xml | 13 | ||||
-rw-r--r-- | chapter08/automake.xml | 9 | ||||
-rw-r--r-- | chapter08/openssl.xml | 2 | ||||
-rw-r--r-- | chapter08/perl.xml | 2 | ||||
-rw-r--r-- | packages.ent | 52 |
9 files changed, 145 insertions, 47 deletions
diff --git a/chapter01/changelog.xml b/chapter01/changelog.xml index 82a3d85be..8ad5949fb 100644 --- a/chapter01/changelog.xml +++ b/chapter01/changelog.xml @@ -40,6 +40,44 @@ appropriate for the entry or if needed the entire day's listitem. --> <listitem> + <para>2023-11-13</para> + <itemizedlist> + <listitem> + <para>[xry111] - Update to elfutils-0.190. Fixes + <ulink url='&lfs-ticket-root;5373'>#5373</ulink>.</para> + </listitem> + <listitem> + <para>[xry111] - Update to vim-9.0.2103. Addresses + <ulink url='&lfs-ticket-root;4500'>#4500</ulink>.</para> + </listitem> + <listitem> + <para>[xry111] - Update to linux-6.6.1. Fixes + <ulink url='&lfs-ticket-root;5369'>#5369</ulink>.</para> + </listitem> + <listitem> + <para>[xry111] - Update to xz-5.4.5. Fixes + <ulink url='&lfs-ticket-root;5371'>#5371</ulink>.</para> + </listitem> + <listitem> + <para>[xry111] - Update to iana-etc-20231107. Addresses + <ulink url='&lfs-ticket-root;5006'>#5006</ulink>.</para> + </listitem> + <listitem> + <para>[xry111] - Update to gawk-5.3.0. Fixes + <ulink url='&lfs-ticket-root;5372'>#5372</ulink>.</para> + </listitem> + <listitem> + <para>[xry111] - Update to bash-5.2.21. Fixes + <ulink url='&lfs-ticket-root;5375'>#5375</ulink>.</para> + </listitem> + <listitem> + <para>[xry111] - Update to iproute2-6.6.0. Fixes + <ulink url='&lfs-ticket-root;5374'>#5374</ulink>.</para> + </listitem> + </itemizedlist> + </listitem> + + <listitem> <para>2023-11-01</para> <itemizedlist> <listitem> diff --git a/chapter02/hostreqs.xml b/chapter02/hostreqs.xml index 69533b9c7..63dcd943f 100644 --- a/chapter02/hostreqs.xml +++ b/chapter02/hostreqs.xml @@ -54,8 +54,9 @@ should be a link to bison or a small script that executes bison)</para> </listitem> + <!-- In Coreutils-8.1 the nproc program is added. --> <listitem> - <para><emphasis role="strong">Coreutils-7.0</emphasis></para> + <para><emphasis role="strong">Coreutils-8.1</emphasis></para> </listitem> <listitem> @@ -218,8 +219,8 @@ ver_kernel() fi } -# Coreutils first because-sort needs Coreutils >= 7.0 -ver_check Coreutils sort 7.0 || bail "--version-sort unsupported" +# Coreutils first because --version-sort needs Coreutils >= 7.0 +ver_check Coreutils sort 8.1 || bail "Coreutils too old, stop" ver_check Bash bash 3.2 ver_check Binutils ld 2.13.1 ver_check Bison bison 2.7 @@ -259,7 +260,13 @@ echo "Compiler check:" if printf "int main(){}" | g++ -x c++ - then echo "OK: g++ works"; else echo "ERROR: g++ does NOT work"; fi -rm -f a.out</literal> +rm -f a.out + +if [ "$(nproc)" = "" ]; then + echo "ERROR: nproc is not available or it produces empty output" +else + echo "OK: nproc reports $(nproc) logical cores are available" +fi</literal> EOF bash version-check.sh</userinput></screen> diff --git a/chapter04/aboutsbus.xml b/chapter04/aboutsbus.xml index f88db24f6..42a523fa1 100644 --- a/chapter04/aboutsbus.xml +++ b/chapter04/aboutsbus.xml @@ -37,18 +37,6 @@ numbers can vary by as much as dozens of minutes in some cases.</para> <note> - <para>For many modern systems with multiple processors (or cores) the - compilation time for a package can be reduced by performing a "parallel - make" by either setting an environment variable or telling the - <command>make</command> program how many processors are available. For - instance, an Intel i5-6500 CPU can support four simultaneous processes with:</para> - - <screen role="nodump"><userinput>export MAKEFLAGS='-j4'</userinput></screen> - - <para>or by building with:</para> - - <screen role="nodump"><userinput>make -j4</userinput></screen> - <para>When multiple processors are used in this way, the SBU units in the book will vary even more than they normally would. In some cases, the make step will simply fail. Analyzing the output of the build process will also diff --git a/chapter04/settingenviron.xml b/chapter04/settingenviron.xml index b255a4b45..2d8d6045f 100644 --- a/chapter04/settingenviron.xml +++ b/chapter04/settingenviron.xml @@ -195,6 +195,55 @@ EOF</userinput></screen> completed LFS system.</para> </important> + <para> + For many modern systems with multiple processors (or cores) the + compilation time for a package can be reduced by performing a "parallel + make" by telling the make program how many processors are available via + a command line option or an environment variable. For instance, an Intel + Core i9-13900K processor has 8 P (performance) cores and + 16 E (efficiency) cores, and a P core can simultaneously run two threads + so each P core are modeled as two logical cores by the Linux kernel. + As the result there are 32 logical cores in total. One obvious way to + use all these logical cores is allowing <command>make</command> to spawn + up to 32 build jobs. This can be done by passing the + <parameter>-j32</parameter> option to <command>make</command>: + </para> + + <screen role='nodump'><userinput>make -j32</userinput></screen> + + <para> + Or set the <envar>MAKEFLAGS</envar> environment variable and its + content will be automatically used by <command>make</command> as + command line options: + </para> + + <screen role='nodump'><userinput>export MAKEFLAGS=-j32</userinput></screen> + + <important> + <para> + Never pass a <parameter>-j</parameter> option without a number to + <command>make</command> or set such an option in + <envar>MAKEFLAGS</envar>. Doing so will allow <command>make</command> + to spawn infinite build jobs and cause system stability issue. + </para> + </important> + + <para> + To use all logical cores available for building packages in + <xref linkend='chapter-cross-tools'/> and + <xref linkend='chapter-temporary-tools'/>, set <envar>MAKEFLAGS</envar> + now in <filename>.bashrc</filename>: + </para> + +<screen><userinput>cat >> ~/.bashrc << "EOF" +<literal>export MAKEFLAGS=-j<replaceable>$(nproc)</replaceable></literal> +EOF</userinput></screen> + + <para> + Replace <replaceable>$(nproc)</replaceable> with the number of logical + cores you want to use if you don't want to use all the logical cores. + </para> + <para>Finally, to ensure the environment is fully prepared for building the temporary tools, force the <command>bash</command> shell to read the new user profile:</para> diff --git a/chapter07/chroot.xml b/chapter07/chroot.xml index b8de03dc2..a8cd5f878 100644 --- a/chapter07/chroot.xml +++ b/chapter07/chroot.xml @@ -23,8 +23,21 @@ TERM="$TERM" \ PS1='(lfs chroot) \u:\w\$ ' \ PATH=/usr/bin:/usr/sbin \ + MAKEFLAGS="-j<replaceable>$(nproc)</replaceable>" \ + TESTSUITEFLAGS="-j<replaceable>$(nproc)</replaceable>" \ /bin/bash --login</userinput></screen> + <para> + Again, replace <replaceable>$(nproc)</replaceable> with the number + of logical cores you want to use for building packages in this chapter + and the following chapters if you don't want to use all available + logical cores. The test suites of some packages (notably Autoconf, + Libtool, and Tar) in &ch-final; are not affected by + <envar>MAKEFLAGS</envar>, they use a <envar>TESTSUITEFLAGS</envar> + environment variable instead. So we set it here as well for running + these test suites with multiple cores. + </para> + <para>The <parameter>-i</parameter> option given to the <command>env</command> command will clear all the variables in the chroot environment. After that, only the <envar>HOME</envar>, <envar>TERM</envar>, <envar>PS1</envar>, and diff --git a/chapter08/automake.xml b/chapter08/automake.xml index 4fc1bd277..5d7541049 100644 --- a/chapter08/automake.xml +++ b/chapter08/automake.xml @@ -53,11 +53,14 @@ <screen><userinput remap="make">make</userinput></screen> - <para> Using the -j4 make option speeds up the tests, even on systems with - only one processor, due to internal delays in individual tests. To test + <para>Using four parallel jobs speeds up the tests, even on systems with + less logical cores, due to internal delays in individual tests. To test the results, issue:</para> -<screen><userinput remap="test">make -j4 check</userinput></screen> +<screen><userinput remap="test">make <replaceable>-j$(($(nproc)>4?$(nproc):4))</replaceable> check</userinput></screen> + + <para>Replace <replaceable>$((...))</replaceable> with the number of + logical cores you want to use if you don't want to use all.</para> <para>The test t/subobj.sh is known to fail.</para> diff --git a/chapter08/openssl.xml b/chapter08/openssl.xml index 6a4441552..25295f056 100644 --- a/chapter08/openssl.xml +++ b/chapter08/openssl.xml @@ -63,7 +63,7 @@ <para>To test the results, issue:</para> -<screen><userinput remap="test">make test</userinput></screen> +<screen><userinput remap="test">HARNESS_JOBS=<replaceable>$(nproc)</replaceable> make test</userinput></screen> <para>One test, 30-test_afalg.t, is known to fail if the host kernel does not have <option>CONFIG_CRYPTO_USER_API_SKCIPHER</option> enabled, diff --git a/chapter08/perl.xml b/chapter08/perl.xml index d3159595c..2bb849645 100644 --- a/chapter08/perl.xml +++ b/chapter08/perl.xml @@ -111,7 +111,7 @@ export BUILD_BZIP2=0</userinput></screen> <para>To test the results (approximately 11 SBU), issue:</para> -<screen><userinput remap="test">make test</userinput></screen> +<screen><userinput remap="test">TEST_JOBS=$(nproc) make test_harness</userinput></screen> <para>Install the package and clean up:</para> diff --git a/packages.ent b/packages.ent index 9e0b0db2d..fbb39bc31 100644 --- a/packages.ent +++ b/packages.ent @@ -47,10 +47,10 @@ <!ENTITY automake-fin-du "114 MB"> <!ENTITY automake-fin-sbu "less than 0.1 SBU (about 7.0 SBU with tests)"> -<!ENTITY bash-version "5.2.15"> -<!ENTITY bash-size "10,695 KB"> +<!ENTITY bash-version "5.2.21"> +<!ENTITY bash-size "10,696 KB"> <!ENTITY bash-url "&gnu;bash/bash-&bash-version;.tar.gz"> -<!ENTITY bash-md5 "4281bb43497f3905a308430a8d6a30a5"> +<!ENTITY bash-md5 "ad5b38410e3bf0e9bcc20e2765f5e3f9"> <!ENTITY bash-home "&gnu-software;bash/"> <!ENTITY bash-tmp-du "67 MB"> <!ENTITY bash-tmp-sbu "0.2 SBU"> @@ -148,10 +148,10 @@ <!ENTITY e2fsprogs-fin-du "95 MB"> <!ENTITY e2fsprogs-fin-sbu "2.4 SBU on a spinning disk, 0.6 SBU on an SSD"> -<!ENTITY elfutils-version "0.189"> <!-- libelf --> -<!ENTITY elfutils-size "8,936 KB"> +<!ENTITY elfutils-version "0.190"> <!-- libelf --> +<!ENTITY elfutils-size "8,949 KB"> <!ENTITY elfutils-url "https://sourceware.org/ftp/elfutils/&elfutils-version;/elfutils-&elfutils-version;.tar.bz2"> -<!ENTITY elfutils-md5 "5cfaa711a90cb670406cd495aeaa6030"> +<!ENTITY elfutils-md5 "79ad698e61a052bea79e77df6a08bc4b"> <!ENTITY elfutils-home "https://sourceware.org/elfutils/"> <!ENTITY elfutils-fin-du "122 MB"> <!ENTITY elfutils-fin-sbu "0.3 SBU"> @@ -208,10 +208,10 @@ <!ENTITY flit-core-fin-du "1.7 MB"> <!ENTITY flit-core-fin-sbu "less than 0.1 SBU"> -<!ENTITY gawk-version "5.2.2"> -<!ENTITY gawk-size "3,324 KB"> +<!ENTITY gawk-version "5.3.0"> +<!ENTITY gawk-size "3,356 KB"> <!ENTITY gawk-url "&gnu;gawk/gawk-&gawk-version;.tar.xz"> -<!ENTITY gawk-md5 "d63b4de2c722cbd9b8cc8e6f14d78a1e"> +<!ENTITY gawk-md5 "97c5a7d83f91a7e1b2035ebbe6ac7abd"> <!ENTITY gawk-home "&gnu-software;gawk/"> <!ENTITY gawk-tmp-du "48 MB"> <!ENTITY gawk-tmp-sbu "0.1 SBU"> @@ -317,10 +317,10 @@ <!ENTITY gzip-fin-du "21 MB"> <!ENTITY gzip-fin-sbu "0.3 SBU"> -<!ENTITY iana-etc-version "20231026"> -<!ENTITY iana-etc-size "588 KB"> +<!ENTITY iana-etc-version "20231107"> +<!ENTITY iana-etc-size "589 KB"> <!ENTITY iana-etc-url "https://github.com/Mic92/iana-etc/releases/download/&iana-etc-version;/iana-etc-&iana-etc-version;.tar.gz"> -<!ENTITY iana-etc-md5 "87e127b8f5ce7f2daa7b4afa1c147465"> +<!ENTITY iana-etc-md5 "01331998d26d5beb592c12b06b2eeb24"> <!ENTITY iana-etc-home "https://www.iana.org/protocols"> <!ENTITY iana-etc-fin-du "4.8 MB"> <!ENTITY iana-etc-fin-sbu "less than 0.1 SBU"> @@ -341,10 +341,10 @@ <!ENTITY intltool-fin-du "1.5 MB"> <!ENTITY intltool-fin-sbu "less than 0.1 SBU"> -<!ENTITY iproute2-version "6.5.0"> -<!ENTITY iproute2-size "908 KB"> +<!ENTITY iproute2-version "6.6.0"> +<!ENTITY iproute2-size "897 KB"> <!ENTITY iproute2-url "&kernel;linux/utils/net/iproute2/iproute2-&iproute2-version;.tar.xz"> -<!ENTITY iproute2-md5 "ae811fc51b3a2c9c7701be308152c45a"> +<!ENTITY iproute2-md5 "6716fc3188dbea226997fa2478a190d7"> <!ENTITY iproute2-home "&kernel;linux/utils/net/iproute2/"> <!ENTITY iproute2-fin-du "17 MB"> <!ENTITY iproute2-fin-sbu "0.1 SBU"> @@ -430,13 +430,13 @@ <!ENTITY libxcrypt-fin-sbu "0.1 SBU"> <!ENTITY linux-major-version "6"> -<!ENTITY linux-minor-version "5"> -<!ENTITY linux-patch-version "9"> +<!ENTITY linux-minor-version "6"> +<!ENTITY linux-patch-version "1"> <!--<!ENTITY linux-version "&linux-major-version;.&linux-minor-version;">--> <!ENTITY linux-version "&linux-major-version;.&linux-minor-version;.&linux-patch-version;"> -<!ENTITY linux-size "135,718 KB"> +<!ENTITY linux-size "136,730 KB"> <!ENTITY linux-url "&kernel;linux/kernel/v&linux-major-version;.x/linux-&linux-version;.tar.xz"> -<!ENTITY linux-md5 "9e187b3c0cd0e29d151a1bd72c66f78f"> +<!ENTITY linux-md5 "90291279ca684fb8cfa59b2ae75b6fe0"> <!ENTITY linux-home "https://www.kernel.org/"> <!-- measured for 6.5.3 / gcc-13.2.0 on x86_64 with -j4 : minimum is allnoconfig + some configs we recommend for the users, rounded down to @@ -735,13 +735,13 @@ <!ENTITY util-linux-fin-du "310 MB"> <!ENTITY util-linux-fin-sbu "0.5 SBU"> -<!ENTITY vim-version "9.0.1968"> +<!ENTITY vim-version "9.0.2103"> <!-- <!ENTITY vim-majmin "90"> --> <!ENTITY vim-docdir "vim/vim90"> -<!ENTITY vim-size "16,909 KB"> +<!ENTITY vim-size "11,193 KB"> <!--<!ENTITY vim-url "https://github.com/vim/vim/archive/v&vim-version;/vim-&vim-version;.tar.gz">--> -<!ENTITY vim-url "&anduin-sources;/vim-&vim-version;.tar.gz"> -<!ENTITY vim-md5 "66147348ba84ea9c78b9d6595015f5a6"> +<!ENTITY vim-url "&anduin-sources;/vim-&vim-version;.tar.xz"> +<!ENTITY vim-md5 "1af1d018026234d085a82348703461ce"> <!ENTITY vim-home "https://www.vim.org"> <!ENTITY vim-fin-du "229 MB"> <!ENTITY vim-fin-sbu "2.3 SBU"> @@ -762,10 +762,10 @@ <!ENTITY xml-parser-fin-du "2.3 MB"> <!ENTITY xml-parser-fin-sbu "less than 0.1 SBU"> -<!ENTITY xz-version "5.4.4"> -<!ENTITY xz-size "1,623 KB"> +<!ENTITY xz-version "5.4.5"> +<!ENTITY xz-size "1,642 KB"> <!ENTITY xz-url "https://tukaani.org/xz/xz-&xz-version;.tar.xz"> -<!ENTITY xz-md5 "d83d6f64a64f88759e312b8a38c3add6"> +<!ENTITY xz-md5 "1d33e0be05c53e7a5641acf5c8b35fdd"> <!ENTITY xz-home "https://tukaani.org/xz"> <!ENTITY xz-tmp-du "22 MB"> <!ENTITY xz-tmp-sbu "0.1 SBU"> |