Documentation more readable when viewed on Github. Some extra changes by @laanwj: - Make README.usage the default README. This is more convenient from a user perspective. Link to other documentation in this default README - Add list of popular targets for cross compilation, change default to Win64 instead of Win32tags/v0.15.1
@@ -0,0 +1,56 @@ | |||
### Usage | |||
To build dependencies for the current arch+OS: | |||
make | |||
To build for another arch/OS: | |||
make HOST=host-platform-triplet | |||
For example: | |||
make HOST=x86_64-w64-mingw32 -j4 | |||
A prefix will be generated that's suitable for plugging into Bitcoin's | |||
configure. In the above example, a dir named i686-w64-mingw32 will be | |||
created. To use it for Bitcoin: | |||
./configure --prefix=`pwd`/depends/x86_64-w64-mingw32 | |||
Common `host-platform-triplets` for cross compilation are: | |||
- `i686-w64-mingw32` for Win32 | |||
- `x86_64-w64-mingw32` for Win64 | |||
- `x86_64-apple-darwin11` for MacOSX | |||
- `arm-linux-gnueabihf` for Linux ARM | |||
No other options are needed, the paths are automatically configured. | |||
Dependency Options: | |||
The following can be set when running make: make FOO=bar | |||
SOURCES_PATH: downloaded sources will be placed here | |||
BASE_CACHE: built packages will be placed here | |||
SDK_PATH: Path where sdk's can be found (used by OSX) | |||
FALLBACK_DOWNLOAD_PATH: If a source file can't be fetched, try here before giving up | |||
NO_QT: Don't download/build/cache qt and its dependencies | |||
NO_WALLET: Don't download/build/cache libs needed to enable the wallet | |||
NO_UPNP: Don't download/build/cache packages needed for enabling upnp | |||
DEBUG: disable some optimizations and enable more runtime checking | |||
If some packages are not built, for example `make NO_WALLET=1`, the appropriate | |||
options will be passed to bitcoin's configure. In this case, `--disable-wallet`. | |||
Additional targets: | |||
download: run 'make download' to fetch all sources without building them | |||
download-osx: run 'make download-osx' to fetch all sources needed for osx builds | |||
download-win: run 'make download-win' to fetch all sources needed for win builds | |||
download-linux: run 'make download-linux' to fetch all sources needed for linux builds | |||
### Other documentation | |||
- [description.md](description.md): General description of the depends system | |||
- [packages.md](packages.md): Steps for adding packages | |||
@@ -1,34 +0,0 @@ | |||
To build dependencies for the current arch+OS: | |||
make | |||
To build for another arch/OS: | |||
make HOST=host-platform-triplet && make HOST=host-platform-triplet | |||
(For example: make HOST=i686-w64-mingw32 -j4) | |||
A prefix will be generated that's suitable for plugging into Bitcoin's | |||
configure. In the above example, a dir named i686-w64-mingw32 will be | |||
created. To use it for Bitcoin: | |||
./configure --prefix=`pwd`/depends/i686-w64-mingw32 | |||
No other options are needed, the paths are automatically configured. | |||
Dependency Options: | |||
The following can be set when running make: make FOO=bar | |||
SOURCES_PATH: downloaded sources will be placed here | |||
BASE_CACHE: built packages will be placed here | |||
SDK_PATH: Path where sdk's can be found (used by OSX) | |||
FALLBACK_DOWNLOAD_PATH: If a source file can't be fetched, try here before giving up | |||
NO_QT: Don't download/build/cache qt and its dependencies | |||
NO_WALLET: Don't download/build/cache libs needed to enable the wallet | |||
NO_UPNP: Don't download/build/cache packages needed for enabling upnp | |||
DEBUG: disable some optimizations and enable more runtime checking | |||
If some packages are not built, for example 'make NO_WALLET=1', the appropriate | |||
options will be passed to bitcoin's configure. In this case, --disable-wallet. | |||
Additional targets: | |||
download: run 'make download' to fetch all sources without building them | |||
download-osx: run 'make download-osx' to fetch all sources needed for osx builds | |||
download-win: run 'make download-win' to fetch all sources needed for win builds | |||
download-linux: run 'make download-linux' to fetch all sources needed for linux builds |
@@ -1,9 +1,7 @@ | |||
This is a system of building and caching dependencies necessary for building | |||
Bitcoin. | |||
This is a system of building and caching dependencies necessary for building Bitcoin. | |||
There are several features that make it different from most similar systems: | |||
- It is designed to be builder and host agnostic | |||
### It is designed to be builder and host agnostic | |||
In theory, binaries for any target OS/architecture can be created, from a | |||
builder running any OS/architecture. In practice, build-side tools must be | |||
@@ -11,18 +9,18 @@ specified when the defaults don't fit, and packages must be amended to work | |||
on new hosts. For now, a build architecture of x86_64 is assumed, either on | |||
Linux or OSX. | |||
- No reliance on timestamps | |||
### No reliance on timestamps | |||
File presence is used to determine what needs to be built. This makes the | |||
results distributable and easily digestable by automated builders. | |||
- Each build only has its specified dependencies available at build-time. | |||
### Each build only has its specified dependencies available at build-time. | |||
For each build, the sysroot is wiped and the (recursive) dependencies are | |||
installed. This makes each build deterministic, since there will never be any | |||
unknown files available to cause side-effects. | |||
- Each package is cached and only rebuilt as needed. | |||
### Each package is cached and only rebuilt as needed. | |||
Before building, a unique build-id is generated for each package. This id | |||
consists of a hash of all files used to build the package (Makefiles, packages, | |||
@@ -32,7 +30,7 @@ any other package that depends on it. If any of the main makefiles (Makefile, | |||
funcs.mk, etc) are changed, all packages will be rebuilt. After building, the | |||
results are cached into a tarball that can be re-used and distributed. | |||
- Package build results are (relatively) deterministic. | |||
### Package build results are (relatively) deterministic. | |||
Each package is configured and patched so that it will yield the same | |||
build-results with each consequent build, within a reasonable set of | |||
@@ -41,13 +39,13 @@ beyond the scope of this system. Additionally, the toolchain itself must be | |||
capable of deterministic results. When revisions are properly bumped, a cached | |||
build should represent an exact single payload. | |||
- Sources are fetched and verified automatically | |||
### Sources are fetched and verified automatically | |||
Each package must define its source location and checksum. The build will fail | |||
if the fetched source does not match. Sources may be pre-seeded and/or cached | |||
as desired. | |||
- Self-cleaning | |||
### Self-cleaning | |||
Build and staging dirs are wiped after use, and any previous version of a | |||
cached result is removed following a successful build. Automated builders |
@@ -4,121 +4,137 @@ variables, and defining build commands. | |||
The package "mylib" will be used here as an example | |||
General tips: | |||
mylib_foo is written as $(package)_foo in order to make recipes more similar. | |||
- mylib_foo is written as $(package)_foo in order to make recipes more similar. | |||
Identifiers: | |||
## Identifiers | |||
Each package is required to define at least these variables: | |||
$(package)_version: | |||
$(package)_version: | |||
Version of the upstream library or program. If there is no version, a | |||
placeholder such as 1.0 can be used. | |||
$(package)_download_path: | |||
$(package)_download_path: | |||
Location of the upstream source, without the file-name. Usually http or | |||
ftp. | |||
$(package)_file_name: | |||
$(package)_file_name: | |||
The upstream source filename available at the download path. | |||
$(package)_sha256_hash: | |||
$(package)_sha256_hash: | |||
The sha256 hash of the upstream file | |||
These variables are optional: | |||
$(package)_build_subdir: | |||
$(package)_build_subdir: | |||
cd to this dir before running configure/build/stage commands. | |||
$(package)_download_file: | |||
$(package)_download_file: | |||
The file-name of the upstream source if it differs from how it should be | |||
stored locally. This can be used to avoid storing file-names with strange | |||
characters. | |||
$(package)_dependencies: | |||
$(package)_dependencies: | |||
Names of any other packages that this one depends on. | |||
$(package)_patches: | |||
$(package)_patches: | |||
Filenames of any patches needed to build the package | |||
$(package)_extra_sources: | |||
$(package)_extra_sources: | |||
Any extra files that will be fetched via $(package)_fetch_cmds. These are | |||
specified so that they can be fetched and verified via 'make download'. | |||
Build Variables: | |||
## Build Variables: | |||
After defining the main identifiers, build variables may be added or customized | |||
before running the build commands. They should be added to a function called | |||
$(package)_set_vars. For example: | |||
define $(package)_set_vars | |||
... | |||
endef | |||
define $(package)_set_vars | |||
... | |||
endef | |||
Most variables can be prefixed with the host, architecture, or both, to make | |||
the modifications specific to that case. For example: | |||
Universal: $(package)_cc=gcc | |||
Linux only: $(package)_linux_cc=gcc | |||
x86_64 only: $(package)_x86_64_cc = gcc | |||
x86_64 linux only: $(package)_x86_64_linux_cc = gcc | |||
Universal: $(package)_cc=gcc | |||
Linux only: $(package)_linux_cc=gcc | |||
x86_64 only: $(package)_x86_64_cc = gcc | |||
x86_64 linux only: $(package)_x86_64_linux_cc = gcc | |||
These variables may be set to override or append their default values. | |||
$(package)_cc | |||
$(package)_cxx | |||
$(package)_objc | |||
$(package)_objcxx | |||
$(package)_ar | |||
$(package)_ranlib | |||
$(package)_libtool | |||
$(package)_nm | |||
$(package)_cflags | |||
$(package)_cxxflags | |||
$(package)_ldflags | |||
$(package)_cppflags | |||
$(package)_config_env | |||
$(package)_build_env | |||
$(package)_stage_env | |||
$(package)_build_opts | |||
$(package)_config_opts | |||
$(package)_cc | |||
$(package)_cxx | |||
$(package)_objc | |||
$(package)_objcxx | |||
$(package)_ar | |||
$(package)_ranlib | |||
$(package)_libtool | |||
$(package)_nm | |||
$(package)_cflags | |||
$(package)_cxxflags | |||
$(package)_ldflags | |||
$(package)_cppflags | |||
$(package)_config_env | |||
$(package)_build_env | |||
$(package)_stage_env | |||
$(package)_build_opts | |||
$(package)_config_opts | |||
The *_env variables are used to add environment variables to the respective | |||
commands. | |||
Many variables respect a debug/release suffix as well, in order to use them for | |||
only the appropriate build config. For example: | |||
$(package)_cflags_release = -O3 | |||
$(package)_cflags_i686_debug = -g | |||
$(package)_config_opts_release = --disable-debug | |||
$(package)_cflags_release = -O3 | |||
$(package)_cflags_i686_debug = -g | |||
$(package)_config_opts_release = --disable-debug | |||
These will be used in addition to the options that do not specify | |||
debug/release. All builds are considered to be release unless DEBUG=1 is set by | |||
the user. | |||
Other variables may be defined as needed. | |||
the user. Other variables may be defined as needed. | |||
Build commands: | |||
## Build commands: | |||
For each build, a unique build dir and staging dir are created. For example, | |||
work/build/mylib/1.0-1adac830f6e and work/staging/mylib/1.0-1adac830f6e. | |||
`work/build/mylib/1.0-1adac830f6e` and `work/staging/mylib/1.0-1adac830f6e`. | |||
The following build commands are available for each recipe: | |||
$(package)_fetch_cmds: | |||
$(package)_fetch_cmds: | |||
Runs from: build dir | |||
Fetch the source file. If undefined, it will be fetched and verified | |||
against its hash. | |||
$(package)_extract_cmds: | |||
$(package)_extract_cmds: | |||
Runs from: build dir | |||
Verify the source file against its hash and extract it. If undefined, the | |||
source is assumed to be a tarball. | |||
$(package)_preprocess_cmds: | |||
$(package)_preprocess_cmds: | |||
Runs from: build dir/$(package)_build_subdir | |||
Preprocess the source as necessary. If undefined, does nothing. | |||
$(package)_config_cmds: | |||
$(package)_config_cmds: | |||
Runs from: build dir/$(package)_build_subdir | |||
Configure the source. If undefined, does nothing. | |||
$(package)_build_cmds: | |||
$(package)_build_cmds: | |||
Runs from: build dir/$(package)_build_subdir | |||
Build the source. If undefined, does nothing. | |||
$(package)_stage_cmds: | |||
$(package)_stage_cmds: | |||
Runs from: build dir/$(package)_build_subdir | |||
Stage the build results. If undefined, does nothing. | |||
The following variables are available for each recipe: | |||
$(1)_staging_dir: package's destination sysroot path | |||
$(1)_staging_prefix_dir: prefix path inside of the package's staging dir | |||
$(1)_extract_dir: path to the package's extracted sources | |||
$(1)_build_dir: path where configure/build/stage commands will be run | |||
$(1)_patch_dir: path where the package's patches (if any) are found | |||
$(1)_staging_dir: package's destination sysroot path | |||
$(1)_staging_prefix_dir: prefix path inside of the package's staging dir | |||
$(1)_extract_dir: path to the package's extracted sources | |||
$(1)_build_dir: path where configure/build/stage commands will be run | |||
$(1)_patch_dir: path where the package's patches (if any) are found | |||
Notes on build commands: | |||
@@ -127,4 +143,5 @@ configure step to (usually) correctly configure automatically. Any | |||
$($(package)_config_opts) will be appended. | |||
Most autotools projects can be properly staged using: | |||
$(MAKE) DESTDIR=$($(package)_staging_dir) install | |||
$(MAKE) DESTDIR=$($(package)_staging_dir) install |