[PATCH Libgpg-error] gpg-error.m4: support pkg-config

[Alon Bar-Lev]

On Fri, Oct 12, 2018 at 9:35 AM Werner Koch <wk at gnupg.org> wrote:
>
> On Thu, 11 Oct 2018 20:27, alon.barlev at gmail.com said:

Hi,

> > * Simplicity, built for the task.
>
> I cannot say that 145,517 source lines of code is a good indication of
> simplicity (Debian's 0.29).  Compare that to make which has 30,919 SLOC
> and thus 21% of pkg-config.  I know that pkg-config it comes with glib
> included but for a bootstrapping tool this is quite excessive.  In
> particular when looking at the 45,000 SLOC of libgpg-error.

I am sure you know the following, but I must address you counting line
of codes... This is odd... feel free to skip the next story.

<< __STORY__
scripts were used to build programs, then realized that building
everything every time is a waste of time and resources, and scripts
became complex to address these concerns.
make was introduced to manage a set of simple rules to avoid re-build
when possible, then realized that for large project it is difficult to
write proper make files, make files got complex and in most cases
implemented using wrong logic.
autoconf was introduced to generate files from templates based on
logic as it was difficult to add functional logic into make files.
automake was introduced to provide a simple method to generate make
files that actually work with less error prune syntax, supporting
dependency management, distribution, tests, conditionals and more.
When shared library were introduced, most people got it wrong as each
architecture had different technology (if any) and tweaks (RPATH,
SONAME), support test and post-install etc... make files become
complex once again.
libtool was introduced to simplify library (static and shared)
creation, addressing all concerns.
CFLAGS, LDFLAGS detection were not that simple as threading and other
compiler settings should have considered, it required additional logic
of detection which was not always aligned with the library best
practice.
These concerns could have been address using metadata or scripts, at
first there was no standard for metadata, so these programs that
required special settings, developed their own unique solutions.
pkg-config evolved as metadata usage. There are advantages of using
metadata based information, mainly the ability to provide tool
selection to manage the metadata, having consistent behavior among
packages, not running anything from sysroot - all is important to
downstream maintainer and anyone who try to build.
__STORY__

The gnupg projects already use them all, make, autoconf, automake,
libtool and pkg-config, so let's at least count them all and
understand why we use and reuse components that are shared between
projects.

Don't get me wrong, I truly respect the gnupg project and maintained
its packages for many years, however, after reached to more or less
stable feature-set and structure, it may be the time to address some
of the issues that downstream maintainer experience to make it easier
to use the packages is required use cases.

When I saw libgpg-error master publishes pkg-config metadata, I was
very happy, as it does show some new openness, as I know your point of
view. You also stated that you want pkg-config to be second class
option, so I introduced the enable_pkg_config flag with default no, to
keep backward compatibility. Using the pkg-config resolves issues of
multilib and cross-compile without need to modify the existing
mechanism.

Some statistics:

1. Per component a non trivial specific m4 script is maintained.
2. The m4 script required by dependencies so it is distributed to
other components (aclocal auto install is not used), this create
additional maintenance effort.
3. Size of each:
$ n=0; for f in ./gpgme/src/gpgme.m4 ./libassuan/src/libassuan.m4
./libgcrypt/src/libgcrypt.m4 ./libgpg-error/src/gpg-error.m4
./libksba/src/ksba.m4; do l=$(cat $f | grep -v '^dnl' | wc -l); echo
$l $f; n=$(($n+$l)); done; echo $n total
265 ./gpgme/src/gpgme.m4
127 ./libassuan/src/libassuan.m4
128 ./libgcrypt/src/libgcrypt.m4
128 ./libgpg-error/src/gpg-error.m4
113 ./libksba/src/ksba.m4
761 total

4. Per each component a config script is being maintained.
5. The script is different per project.
6. In libgpg-error master the script is a wrapper on top of pkg-config
metadata which implies that the pkg-config metadata is formally
maintained.
7. Size of scripts:
$ find . -name '*-config.in' | sort | xargs wc -l
  139 ./libksba/src/ksba-config.in
  134 ./libassuan/src/libassuan-config.in
  189 ./libgcrypt/src/libgcrypt-config.in
  103 ./libgpg-error/src/gpg-error-config.in
  211 ./gpgme/src/gpgme-config.in
  776 total

8. pkg-config and pkg.m4 are already used by the following projects:
$ find . -name 'pkg.m4' |  sort
./gnupg/m4/pkg.m4
./gpgme/m4/pkg.m4
./pinentry/m4/pkg.m4

9. Size of the pkg.m4 is smaller than the usage of component m4, and
reused for all cases of dependency detection:
$ cat m4/pkg.m4 | grep -v '^dnl' | wc -l
212

9. pkg-conf is an re-implementation of pkg-config:
$ find pkgconf-1.3.7/ -name '*.c' | xargs wc -l | grep total
  6421 total

10. pkg-config is bloated because it has embedded glib:
$ find pkg-config-0.29.2/ -name '*.c' | xargs wc -l | grep total
 123507 total
$ find pkg-config-0.29.2/ -name '*.c' | grep -v /glib | xargs wc -l | grep total
 3386 total

> > * Good integration into autoconf using the pkg.m4 macros, enables the
> >   detection of pkg-config, fallback if missing and handle the
> >   interaction, including override settings by environment.
>
> And still not way to indicate an API break.  Pretty inflexible way to
> add new configure options due to using its own syntax instead of
> utilizing the standard glue scripting language on Unix.

I am unsure I follow, I will appreciate if you provide an example from
the existing script. I do not see a difference between pkg-config and
script:
a. it can detect a package based on component version expression
b. if ABI version is required retrieve a specific variable out of the metadata.
c. if a severe ABI breakage is happening project can modify the name
of the metadata for side-by-side or even install multiple metadata
each with different setting

What is missing?

> Remember that it was build for large projects like GNOME where creating
> new libs is so common that virtually nobody takes care of ABI/API
> maintenance.  For such an environment pkg-config (or the older larger
> foo-config) stuff makes a lot of sense.  For libraries with a maintained
> API and ABI a simpler, more portable but also harder to initially create
> dedicated config file is a cleaner approach.

I do not understand this argument, I will appreciate an example.

> > * Support for multilib configurations as configuration is installed per
> >   library setup, unlike a common script that is installed at bindir.
>
> Multilib is a questionable feature as it introduced a third way to build
> packages (native, cross, multilib).  The major problem, here is that it
> was been developed by the linux distros without integrating this in the
> standard GNU toolchain or getting it properly upstream.

On the contrary... I think. Each multilib configuration has a
different  ${HOST} which complies with toolchain standards. The
autoconf AC_*_TOOL are designed to search for a tool at convention of
${HOST}-${TOOL}, which is aligned to the config script approach.
I already raised this issue in this recent thread, using multilib with
config script should:
1. install the tool as ${HOST}-${TOOL}, symlink this into ${TOOL} on
native if it makes sense.
2. use AC_*_TOOL to search for the tool
3. for cross-compile, install these config scripts outside of the
sysroot, as it makes no sense to execute anything from sysroot of
other architecture
4. for cross-compile, the tool need to be familiar with sysroot
concept to output paths relative to sysroot

This is actually, the only point of why config script are "better"
supported, however, it require effort to do this right, and the gnupg
project should be fixed to follow the pattern.

However, using metadata instead of config scripts make it easier to
mange the settings as:
1. the tool is the tool which reads the metadata
2. nothing is running from the sysroot
3. the tool is familiar with sysroot

Once the gnupg projects provide pkg-config metadata, it is probably
easier to support these concerns properly within the pkg-config scope,
and providing a pkg-config alternative (enable_pkg_config=yes) while
preserving the current behavior for backward compatibility with past
use cases.

> > * Cross-compile support, support sysroot and does not require to execute
> >   anything from sysroot.
>
> It seems to have improved over the years.  I remember well, how we
> needed to patch around pkg-config when cross building larger systems for
> Windows.

Correct, it has been improved, it is not the best tool, but it is
sufficient in most of the cases. If you find a specific issue, I will
be happy to try and sort it out.

> Please don't let us have such discussions in commit logs.

I only expected to trigger discussion, I did not expected you just
merge it as is :)
If you want to proceed I will happily remove these comments.
I hope you will be in favor of supporting pkg-config as 2nd citizen in
gnupg projects.

Thanks for the discussion,
Alon