===============
[](https://gitter.im/termux/termux)
-This project contains scripts and patches to cross compile and package packages for
-the [Termux](http://termux.com/) Android application.
+This project contains scripts and patches to build packages for the
+[Termux](https://termux.com/) Android application.
-The scripts and patches to build each package is licensed under the same license as
-the actual package (so the patches and scripts to build bash are licensed under
-the same license as bash, while the patches and scripts to build python are licensed
-under the same license as python, etc).
+Setting up a build environment using Docker
+===========================================
+For most people the best way to obtain an environment for building packages is by using Docker. This should work everywhere Docker is supported (replace `/` with `\` if using Windows) and ensures an up to date build environment that is tested by other package builders.
-NOTE: This is in a rough state - be prepared for some work and frustrations, and give
-feedback if you find incorrect our outdated things!
+Run the following script to setup a container (from an image created by [scripts/Dockerfile](scripts/Dockerfile)) suitable for building packages:
-Initial setup
-=============
-Building packages are for now only tested to work on Ubuntu 15.10. Perform the following
-setup steps:
+ ./scripts/run-docker.sh
-* Run `ubuntu-setup.sh` to install required packages and setup the `/data/` folder (see below).
+This source folder is mounted as the `/root/termux-packages` data volume, so changes are kept
+in sync between the host and the container when trying things out before committing, and built
+deb files will be available on the host in the `debs/` directory just as when building on the host.
-* Install the Android SDK at `$HOME/lib/android-sdk`. Override this by setting the environment
-variable `$ANDROID_HOME` to point at another location.
+The docker container used for building packages is a Ubuntu 16.10 installation with necessary packages
+pre-installed. The default user is a non-root user to avoid problems with package builds modifying the system
+by mistake, but `sudo` can be used to install additional Ubuntu packages to be used during development.
-* Install the Android NDK, version r10e, at `$HOME/lib/android-ndk`. Override this by setting
-the environment variable `$NDK` to point at another location.
+Build commands can be given to be executed in the docker container directly:
-Alternatively a Dockerfile is provided which sets up a pristine image
-suitable for building packages. To build the docker image, run the
-following command:
+ ./scripts/run-docker.sh ./build-package.sh libandroid-support
- docker build --rm=true -t termux .
+will launch the docker container, execute the `./build-package.sh libandroid-support`
+command inside it and afterwards return you to the host prompt, with the newly built
+deb in `debs/` to try out.
-After build is successful, you can open an interactive prompt inside the
-container using:
+Note that building packages can take up a lot of space (especially if `build-all.sh` is used to build all packages) and you may need to [increase the base device size](http://www.projectatomic.io/blog/2016/03/daemon_option_basedevicesize/) if running with a storage driver using a small base size of 10 GB.
- docker run --rm=true -ti termux /bin/bash
+Build environment without Docker
+================================
+If you can't run Docker you can use a Ubuntu 16.10 installation (either by installing a virtual maching guest or on direct hardware) by using the below scripts:
+
+- Run `scripts/setup-ubuntu.sh` to install required packages and setup the `/data/` folder.
+- Run `scripts/setup-android-sdk.sh` to install the Android SDK and NDK at `$HOME/lib/android-{sdk,ndk}`.
+
+There is also a [Vagrantfile](scripts/Vagrantfile) available as a shortcut for setting up an Ubuntu installation with the above steps applied.
Building a package
==================
-In a non-rooted Android device an app such as Termux may not write to system locations,
-which is why every package is installed inside the private file area of the Termux app:
-
- PREFIX=/data/data/com.termux/files/usr
+The basic build operation is to run `./build-package.sh $PKG`, which:
-For simplicity while developing and building, the build scripts here assume that a /data
-folder is reserved for use on the host builder and install everything there.
+1. Sets up a patched stand-alone Android NDK toolchain if necessary.
-The basic flow is then to run "./build-package.sh $PKG", which:
-* Sets up a patched stand-alone Android NDK toolchain
+2. Reads `packages/$PKG/build.sh` to find out where to find the source code of the package and how to build it.
-* Reads packages/$PKG/build.sh to find out where to find the source code of the package and how to build it.
+3. Extracts the source in `$HOME/.termux-build/$PKG/src`.
-* Applies all patches in packages/$PKG/\*.patch
+4. Applies all patches in packages/$PKG/\*.patch.
-* Builds the package and installs it to $PREFIX
+5. Builds the package under `$HOME/.termux-build/$PKG/` (either in the build/ directory there or in the
+ src/ directory if the package is specified to build in the src dir) and installs it to `$PREFIX`.
-* Creates a dpkg package file for distribution.
+6. Extracts modified files in `$PREFIX` into `$HOME/.termux-build/$PKG/massage` and massages the
+ files there for distribution (removes some files, splits it up in sub-packages, modifies elf files).
-Reading and following build-package.sh is the best way to understand what's going on here.
+7. Creates a deb package file for distribution in `debs/`.
+Reading [build-package.sh](build-package.sh) is the best way to understand what is going on.
Additional utilities
====================
-* build-all.sh: used for building all packages in the correct order (using buildorder.py)
+* build-all.sh: used for building all packages in the correct order (using buildorder.py).
-* check-pie.sh: Used for verifying that all binaries are using PIE, which is required for Android 5+
+* clean.sh: used for doing a clean rebuild of all packages.
-* detect-hardlinks.sh: Used for finding if any packages uses hardlinks, which does not work on Android M
+* scripts/check-pie.sh: Used for verifying that all binaries are using PIE, which is required for Android 5+.
-* check-versions.sh: used for checking for package updates
-
-* clean-rebuild-all.sh: used for doing a clean rebuild of all packages (takes a couple of hours)
+* scripts/check-versions.sh: used for checking for package updates.
-* list-packages.sh: used for listing all packages with a one-line summary
-
-
-Resources about cross-compiling packages
-========================================
-* [Linux From Scratch](http://www.linuxfromscratch.org/blfs/view/svn/index.html)
+* scripts/list-packages.sh: used for listing all packages with a one-line summary.
-* [Beyond Linux From Scratch](http://www.linuxfromscratch.org/blfs/view/svn/)
-* [Cross-Compiled Linux From Scratch](http://cross-lfs.org/view/svn/x86_64-64/)
+Resources
+=========
+* [Android changes for NDK developers](https://android.googlesource.com/platform/bionic/+/master/android-changes-for-ndk-developers.md)
-* [OpenWrt](https://openwrt.org/), an embedded Linx distribution, contains [patches and build scripts](https://dev.openwrt.org/browser/packages)
+* [Linux From Scratch](http://www.linuxfromscratch.org/lfs/view/stable/)
-* http://dan.drown.org/android contains [patches for cross-compiling to Android](http://dan.drown.org/android/src/) as well as [work notes](http://dan.drown.org/android/worknotes.html), including a modified dynamic linker to avoid messing with LD_LIBRARY_PATH.
+* [Beyond Linux From Scratch](http://www.linuxfromscratch.org/blfs/view/stable/)
-* [CCTools](http://cctools.info/index.php?title=Main_Page) is an Android native IDE containing [patches for several programs](https://code.google.com/p/cctools/source/browse/#svn%2Ftrunk%2Fcctools-repo%2Fpatches) and [a bug tracker](https://code.google.com/p/cctools/issues/list).
+* [OpenWrt](https://openwrt.org/) as an embedded Linx distribution contains [patches and build scripts](https://dev.openwrt.org/browser/packages)
-* [BotBrew](http://botbrew.com/) was a package manager for rooted devices with [sources on github](https://github.com/jyio/botbrew). Based on opkg and was transitioning to apt.
-
-* [Kivy recipes](https://github.com/kivy/python-for-android/tree/master/recipes) contains recipes for building packages for Android.
+* [Kivy recipes](https://github.com/kivy/python-for-android/tree/master/pythonforandroid/recipes) contains recipes for building packages for Android.
Common porting problems
=======================
-* The Android bionic libc does not have iconv and gettext/libintl functionality built in. A package from the NDK, libandroid-support,
-contains these and may be used by all packages.
+* The Android bionic libc does not have iconv and gettext/libintl functionality built in. A `libandroid-support` package contains these and may be used by all packages.
* "error: z: no archive symbol table (run ranlib)" usually means that the build machines libz is used instead of the one for cross compilation, due to the builder library -L path being setup incorrectly
-* rindex(3) is defined in <strings.h> but does not exist in NDK, but strrchr(3) from <string.h> is preferred anyway
+* rindex(3) does not exist, but strrchr(3) is preferred anyway.
* <sys/termios.h> does not exist, but <termios.h> is the standard location.
* <sys/fcntl.h> does not exist, but <fcntl.h> is the standard location.
-* glob(3) system function (glob.h) - not in bionic, but use the `libandroid-glob` package
-
-* cmake and cross compiling: http://www.cmake.org/Wiki/CMake_Cross_Compiling
- CMAKE_FIND_ROOT_PATH=$TERMUX_PREFIX to search there.
- CMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY and
- CMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY
- for only searching there and don't fall back to build machines
+* <sys/timeb.h> does not exist (removed in POSIX 2008), but ftime(3) can be replaced with gettimeofday(2).
-* Android is removing sys/timeb.h because it was removed in POSIX 2008, but ftime(3) can be replaced with gettimeofday(2)
+* <glob.h> does not exist, but is available through the `libandroid-glob` package.
-* mempcpy(3) is a GNU extension. We have added it to <string.h> provided TERMUX_EXPOSE_MEMPCPY is defined,
- so use something like CFLAGS+=" -DTERMUX_EXPOSE_MEMPCPY=1" for packages expecting that function to exist.
+* SYSV shared memory is not supported by the kernel. A `libandroid-shmem` package, which emulates SYSV shared memory on top of the [ashmem](http://elinux.org/Android_Kernel_Features#ashmem) shared memory system, is available. Use it with `LDFLAGS+=" -landroid-shmem`.
+* SYSV semaphores is not supported by the kernel. Use unnamed POSIX semaphores instead (named semaphores are unimplemented).
dlopen() and RTLD_* flags
=================================
-<dlfcn.h> declares
+<dlfcn.h> originally declares
enum { RTLD_NOW=0, RTLD_LAZY=1, RTLD_LOCAL=0, RTLD_GLOBAL=2, RTLD_NOLOAD=4}; // 32-bit
enum { RTLD_NOW=2, RTLD_LAZY=1, RTLD_LOCAL=0, RTLD_GLOBAL=0x00100, RTLD_NOLOAD=4}; // 64-bit
These differs from glibc ones in that
-1. They are not preprocessor #define:s so cannot be checked for with #ifdef RTLD_GLOBAL (dln.c in ruby does this)
+1. They are not preprocessor #define:s so cannot be checked for with `#ifdef RTLD_GLOBAL`. Termux patches this to #define values for compatibility with several packages.
2. They differ in value from glibc ones, so cannot be hardcoded in files (DLFCN.py in python does this)
-3. They are missing some values (RTLD_BINDING_MASK, RTLD_NOLOAD, ...)
-
+3. They are missing some values (`RTLD_BINDING_MASK`, `RTLD_NOLOAD`, ...)
-RPATH, LD_LIBRARY_PATH AND RUNPATH
-==================================
+RPATH, RUNPATH AND LD\_LIBRARY\_PATH
+====================================
On desktop linux the linker searches for shared libraries in:
-1. RPATH - a list of directories which is linked into the executable, supported on most UNIX systems. It is ignored if RUNPATH is present.
-2. LD_LIBRARY_PATH - an environment variable which holds a list of directories
-3. RUNPATH - same as RPATH, but searched after LD_LIBRARY_PATH, supported only on most recent UNIX systems
-
-The Android linker (/system/bin/linker) does not support RPATH or RUNPATH, so we set LD_LIBRARY_PATH=$USR/lib and try to avoid building useless rpath entries with --disable-rpath configure flags. Another option to avoid depending on LD_LIBRARY_PATH would be supplying a custom linker - this is not done due to the overhead of maintaining a custom linker.
+1. `RPATH` - a list of directories which is linked into the executable, supported on most UNIX systems. It is ignored if `RUNPATH` is present.
+2. `LD_LIBRARY_PATH` - an environment variable which holds a list of directories
+3. `RUNPATH` - same as `RPATH`, but searched after `LD_LIBRARY_PATH`, supported only on most recent UNIX systems
+The Android linker, /system/bin/linker, does not support RPATH or RUNPATH, so we set `LD_LIBRARY_PATH=$PREFIX/lib` and try to avoid building useless rpath entries (which the linker warns about) with --disable-rpath configure flags. NOTE: Starting from Android 7.0 RUNPATH (but not RPATH) is supported.
Warnings about unused DT entries
================================
-Starting from 5.1 the Android linker warns about VERNEED (0x6FFFFFFE) and VERNEEDNUM (0x6FFFFFFF) ELF dynamic sections:
-
- WARNING: linker: $BINARY: unused DT entry: type 0x6ffffffe arg ...
- WARNING: linker: $BINARY: unused DT entry: type 0x6fffffff arg ...
-These may come from version scripts in a Makefile such as:
-
- -Wl,--version-script=$(top_srcdir)/proc/libprocps.sym
-The termux-elf-cleaner utilty is run from build-package.sh and should normally take care of that problem.
+Starting from 5.1 the Android linker warns about VERNEED (0x6FFFFFFE) and VERNEEDNUM (0x6FFFFFFF) ELF dynamic sections (WARNING: linker: $BINARY: unused DT entry: type 0x6ffffffe/0x6fffffff). These may come from version scripts (`-Wl,--version-script=`). The termux-elf-cleaner utilty is run from build-package.sh and should normally take care of that problem. NOTE: Starting from Android 6.0 symbol versioning is supported.
~mdw / termux-packages / blobdiff