Now the target CPU architecture should be set by the 'ARCH' environment
variable before invoking appimage-builder.
This makes it easier to build the AppImage for other CPU architectures.
This environment variable was necessary back then to work around a bug
in the libgtk3 version we include in the AppImage.
However, since commit 3897a8bccf
("ci/linux: build AppImage on Ubuntu 22.04 (Python 3.10)"), that's not
needed anymore, because Ubuntu 22.04 has a newer version of libgtk3,
which fixes the problem.
I made sure to verify the app runs as expected on Wayland sessions of
two major desktop environments:
- GNOME: 42.9 (Ubuntu 22.04.05) and 48.3 (Arch Linux)
- KDE Plasma: 5.27.5 (Debian 12) and 6.4.3 (Arch Linux)
An up to date version of libXft is still needed because Ubuntu 22.04
still has 2.3.4, which can cause the app not to launch if an emoji font
is installed on the host.
Unfortunately, sourcing the Python venv Bash script to activate the
virtual environment doesn't work while we're building the AppImage, so
this just manually invokes Pip from within the venv to install updated
versions of meson and ninja (for building PyGObject next), and then we
prepend the venv's "bin" directory to the 'PATH' environment variable
to make meson and ninja discoverable on the next "pip install" step.
This also removes "certifi" from being manually installed, because
"truststore" has been usable since commit
3897a8bccf ("ci/linux: build AppImage on
Ubuntu 22.04 (Python 3.10)").
appimage-builder is a tool that is not officially part of the AppImage
ecosystem, and lately its maintainer hasn't been able to keep up with
the latest AppImage ideas, most notably:
- The move to a new statically-linked runtime: Before this, all AppImages
using the old runtime, would depend on libfuse2 to be installed on the
host system just to be able to run out of the box. This wasn't much of
a problem back then, because libfuse2 was ubiquitous. However, nowadays,
some have distros don't even have it in their official repositories.
- The removal of the xz compression algorithm (used by default), and the
addition of Zstd.
Luckily, the problems above have been fixed in the git version of
appimage-builder, but a stable release is yet to be made, so we'll have
to use the git version, because other AppImage-making tools are not as
nice and convenient as appimage-builder.
This manually downloads the old AppImage runtime (now marked as
obsolete), to prevent appimage-builder v1.1.0 from failing to make the
AppImage.
The download URL appimage-builder v1.1.0 uses for the AppImage runtime
is hardcoded into the tool, and unfortunately it's been recently
removed, so this is merely a workaround, until hopefully a future
appimage-builder release fixes this.
See: https://redirect.github.com/AppImageCrafters/appimage-builder/issues/375
PyGObject 3.51 (and newer) now requires girepository-2.0 as a build
dependency [1], which is a version we cannot easily install in the
Ubuntu version we use in CI/CD, so the build fails:
=======
$ python3 -m pip install -r requirements.txt
[ ... ]
Collecting PyGObject (from -r requirements.txt (line 4))
Downloading pygobject-3.52.3.tar.gz (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 45.4 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [216 lines of output]
+ meson setup /tmp/pip-install-nayte7bo/pygobject_097c5326983b4ca690e238e49ba4b29c /tmp/pip-install-nayte7bo/pygobject_097c5326983b4ca690e238e49ba4b29c/.mesonpy-sp71efmg -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md -Dtests=false -Dwheel=true --wrap-mode=nofallback --native-file=/tmp/pip-install-nayte7bo/pygobject_097c5326983b4ca690e238e49ba4b29c/.mesonpy-sp71efmg/meson-python-native-file.ini
The Meson build system
Version: 1.7.0
Source dir: /tmp/pip-install-nayte7bo/pygobject_097c5326983b4ca690e238e49ba4b29c
Build dir: /tmp/pip-install-nayte7bo/pygobject_097c5326983b4ca690e238e49ba4b29c/.mesonpy-sp71efmg
Build type: native build
Project name: pygobject
Project version: 3.52.3
C compiler for the host machine: cc (gcc 9.4.0 "cc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0")
C linker for the host machine: cc ld.bfd 2.34
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python3 found: YES (/opt/hostedtoolcache/Python/3.10.16/x64/bin/python3)
Found pkg-config: YES (/usr/bin/pkg-config) 1.6.3
Run-time dependency python found: YES 3.10
Found CMake: /usr/local/bin/cmake (3.31.6)
Run-time dependency girepository-2.0 found: NO (tried pkgconfig and cmake)
Not looking for a fallback subproject for the dependency girepository-2.0 because:
Use of fallback dependencies is disabled.
../meson.build:31:9: ERROR: Dependency 'girepository-2.0' is required but not found.
=======
Unfortunately, girepository-2.0 is only available in Ubuntu 24.10, a
version we can't easily use in GitHub Actions. But even if could, doing
so would also needlessly bump the minimum glibc version requirement for
Linux users, making TDM much less portable.
So, for now, let's pin PyGObject to <3.51.
[1]: https://gitlab.gnome.org/GNOME/pygobject/-/blob/d29f4e65faabf2ba6892a976a0fdfcb2cc3031f7/NEWS#L4-5
Some systems (e.g. Ubuntu 22.04, which we use to make the AppImage),
have ancient versions of meson and ninja, which is not good when we
have dependencies like PyGObject 3.47+ (and more recently, one of its
transitive dependencies, pycairo 1.27+) requiring a fairly recent
version of meson to be able to be built.
So, instead of relying on whatever meson/ninja version the current
system provides us with to make the AppImage, this change will simply
install up-to-date, official wheels of meson/ninja directly from PyPI.
This also allows us to finally unpin PyGObject's version.
This fixes the broken "Linux (AppImage)" CI job.
PyGObject 3.47 has dropped setuptools in favor of meson-python, which
isn't a problem per se, but they also require meson 0.63.3, which we
don't currently have in the repositories of Ubuntu 20.04, which is
the oldest (yet supported) version we use to build our AppImage
package, in order to increase compatibility with as much Linux distros
as possible.
And since there's no real, compelling reason to upgrade to the latest
version of PyGObject right now, let's just pin it to version <3.46,
until we eventually upgrade the Ubuntu version we use for CI.
This allows us to use the base URL as specified in ClientInfo for
requests made to Twitch.
Previously, we'd hardcode "www.twitch.tv" as the base URL to make
channel-related requests, but because we're currently spoofing the
mobile client, that would always cause a HTTP redirect to m.twitch.tv.
This change avoids that HTTP redirect, and potentially saves a whopping
150 ms (or more 😱) when making channel-related requests.
Co-authored-by: DevilXD <4180725+DevilXD@users.noreply.github.com>
Ubuntu 20.04 only has Python 3.8, and because the 'truststore'
dependency requires at least Python 3.10, the AppImage package would
need to bundle CA certificates from 'certifi' to make HTTPS requests
work at all, which is quite an ugly workaround.
Unfortunately, as a side effect, building on Ubuntu 22.04 increases the
minimum required version of glibc from 2.31 to 2.35.
Excluding these Linux libraries doesn't seem to cause any side-effects
at runtime (they're also not present in the AppImage).
And the benefit of this, is a reduction of 15 MiB in the final size of
the Linux binary, which should also slightly reduce the app's startup
time (but don't quote me on that). :)
AppImage is a more flexible packaging solution for Linux than what
PyInstaller currently offers.
Some of the benefits are:
- Ability to ship the app with predefined environment variables
(solving issues like #321)
- Improved integration with the desktop environment (if you're using
go-appimage[1] or libappimage[2])
- Much lighter in size (~27 MB difference)
To actually make the AppImage, I'm using appimage-builder[3], which
makes the process a bit easier.
For more details about the AppImage technology, check out their
website[4].
[1] https://github.com/probonopd/go-appimage
[2] https://github.com/AppImageCommunity/libappimage
[3] https://appimage-builder.readthedocs.io/en/latest/index.html
[4] https://appimage.org/
This fixes the following error when you launch the app with an emoji
font installed (e.g. Noto Emoji):
X Error of failed request: BadLength (poly request too large or internal Xlib length error)
Major opcode of failed request: 139 (RENDER)
Minor opcode of failed request: 20 (RenderAddGlyphs)
Serial number of failed request: 277
Current serial number in output stream: 300
This has been fixed[1] in libXft 2.3.5[2], but unfortunately Ubuntu
20.04 (which we currently use in the Linux CI workflow), still has
version 2.3.3, so we have to build it ourselves.
[1] https://gitlab.freedesktop.org/xorg/lib/libxft/-/merge_requests/12
[2] https://lists.freedesktop.org/archives/xorg-announce/2022-September/003209.html
This greatly reduces the final size of the UPX-compressed binary, from
78.7 MiB to 54.9 MiB (a 23.8 MiB difference)!
Co-authored-by: DevilXD <bartosz.peszek@gmail.com>
Because PyInstaller modifies the LD_LIBRARY_PATH environment variable
to make things work correctly on Linux, when we try to launch any
subprocess (such as the web browser), that modified variable gets
picked up as well, causing things to go south.
To work around this Linux-only problem, we have to:
1. Save the current value of LD_LIBRARY_PATH
2. Move the value of LD_LIBRARY_PATH_ORIG to LD_LIBRARY_PATH
3. Launch subprocess
4. Restore the saved LD_LIBRARY_PATH again
Reference:
https://pyinstaller.org/en/stable/runtime-information.html#ld-library-path-libpath-considerations
Co-authored-by: DevilXD <bartosz.peszek@gmail.com>
Because Linux distributions don't have an 'universal' path for the
system certificate store, bundling libssl built on distro A might not
work on distro B, because distros tend to change path to the
certificate store at build time.
This means that creating a PyInstaller package on Ubuntu will include
the libssl built there, and if you try to run this package on a
different distro (e.g. Arch Linux), any attempt to create an SSL
connection will fail, because Arch Linux has a different path for the
certificate store than Ubuntu.
This issue is usually worked around by bundling certificates with the
app (usually using certifi), and then pointing the SSL_CERT_FILE
environment to it.
However, truststore seems to be a better alternative, since it tries to
use the certificate store from the system instead.
The list of benefits are listed on the project's page on GitHub:
https://github.com/sethmlarson/truststore
On Linux, pystray's icon.stop() currently doesn't work well with the
AppIndicator backend.
So, instead of destroying the icon when the user restores from tray
(only to later re-create it when the user minimizes to tray again), we
have the option to control the icon's visibility instead! Luckily, that
works on Windows too!
Note that for this change to work properly on Linux, we need to switch
to the master version of pystray, because we need this specific fix:
https://github.com/moses-palmer/pystray/commit/5943ed4e19be48224cf39b70444de8ffe22c23ce
After pystray releases a version newer than 0.19.4, we can switch back.
Co-authored-by: DevilXD <bartosz.peszek@gmail.com>
guihkx