You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

README.md 7.5KB

10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
10 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
9 years ago
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200
  1. # Gitian
  2. Read about the project goals at the [project home page](https://gitian.org/).
  3. This package can do a deterministic build of a package inside a VM.
  4. ## Deterministic build inside a VM
  5. This performs a build inside a VM, with deterministic inputs and outputs. If the build script takes care of all sources of non-determinism (mostly caused by timestamps), the result will always be the same. This allows multiple independent verifiers to sign a binary with the assurance that it really came from the source they reviewed.
  6. ## Prerequisites:
  7. ### Gentoo:
  8. layman -a luke-jr # needed for vmbuilder
  9. sudo emerge dev-vcs/git net-misc/apt-cacher-ng app-emulation/vmbuilder dev-lang/ruby
  10. sudo emerge app-emulation/qemu
  11. export KVM=qemu-system-x86_64
  12. ### Ubuntu:
  13. This pulls in all pre-requisites for KVM building on Ubuntu:
  14. sudo apt-get install git apache2 apt-cacher-ng python-vm-builder ruby qemu-utils
  15. If you'd like to use LXC mode instead, install it as follows:
  16. sudo apt-get install lxc
  17. ### Debian:
  18. See Ubuntu, and also run the following on Debian Jessie or newer:
  19. sudo apt-get install ubuntu-archive-keyring
  20. On Debian Wheezy you run the same command, but you must first add backports to your system, because the package is only available in wheezy-backports.
  21. ### OSX with MacPorts:
  22. sudo port install ruby coreutils
  23. export PATH=$PATH:/opt/local/libexec/gnubin # Needed for sha256sum
  24. ### OSX with Homebrew:
  25. brew install ruby coreutils
  26. export PATH=$PATH:/opt/local/libexec/gnubin
  27. #### VirtualBox:
  28. Install virtualbox from http://www.virtualbox.org, and make sure `VBoxManage` is in your `$PATH`.
  29. ## Debian Guests
  30. Gitian now supports Debian guests in addition to Ubuntu guests. Note that this doesn't mean you can allow the builders to choose to use either Debian or Ubuntu guests. The person creating the Gitian descriptor will need to choose a particular distro and suite for the guest and all builders must use that particular distro and suite, otherwise the software won't reproduce for everyone.
  31. The official vmbuilder only includes support for Ubuntu guests, so you need to install [Joseph Bisch's fork of vmbuilder](https://github.com/josephbisch/vmbuilder), which adds a Debian plugin.
  32. To create a Debian guest:
  33. bin/make-base-vm --distro debian --suite jessie
  34. There is currently no support for LXC Debian guests. There is just KVM support. LXC support for Debian guests is planned to be added soon.
  35. Only Debian Jessie guests have been tested with Gitian. Debian Jessie is the current stable release of Debian at this time. If you have success (or trouble) with other versions of Debian, please let us know.
  36. If you are creating a Gitian descriptor, you can now specify a distro. If no distro is provided, the default is to assume Ubuntu. Since Ubuntu is assumed, older Gitian descriptors that don't specify a distro will still work as they always have.
  37. ## Create the base VM for use in further builds
  38. **NOTE:** requires `sudo`, please review the script
  39. ### KVM
  40. bin/make-base-vm
  41. bin/make-base-vm --arch i386
  42. ### LXC
  43. bin/make-base-vm --lxc
  44. bin/make-base-vm --lxc --arch i386
  45. Set the `USE_LXC` environment variable to use `LXC` instead of `KVM`:
  46. export USE_LXC=1
  47. ### VirtualBox
  48. Command-line `VBoxManage` must be in your `$PATH`.
  49. #### Setup:
  50. `make-base-vm` cannot yet make VirtualBox virtual machines ( _patches welcome_, it should be possible to use `VBoxManage`, boot-from-network Linux images and PXE booting to do it). So you must either get or manually create VirtualBox machines that:
  51. 1. Are named `Gitian-<suite>-<arch>` -- e.g. Gitian-lucid-i386 for a 32-bit, Ubuntu 10 machine.
  52. 2. Have a booted-up snapshot named `Gitian-Clean` . The build script resets the VM to that snapshot to get reproducible builds.
  53. 3. Has the VM's NAT networking setup to forward port `localhost:2223` on the host machine to port `22` of the VM; e.g.:
  54. ```
  55. VBoxManage modifyvm Gitian-lucid-i386 --natpf1 "guestssh,tcp,,2223,,22"
  56. ```
  57. The final setup needed is to create an `ssh` key that will be used to login to the virtual machine:
  58. ssh-keygen -t dsa -f var/id_dsa -N ""
  59. ssh -p 2223 ubuntu@localhost 'mkdir -p .ssh && chmod 700 .ssh && cat >> .ssh/authorized_keys' < var/id_dsa.pub
  60. Then log into the vm and copy the `ssh` keys to root's `authorized_keys` file.
  61. ssh -p 2223 ubuntu@localhost
  62. # Now in the vm
  63. sudo bash
  64. mkdir -p .ssh && chmod 700 .ssh && cat ~ubuntu/.ssh/authorized_keys >> .ssh/authorized_keys
  65. Set the `USE_VBOX` environment variable to use `VBOX` instead of `KVM`:
  66. export USE_VBOX=1
  67. ## Sanity-testing
  68. If you have everything set-up properly, you should be able to:
  69. PATH=$PATH:$(pwd)/libexec
  70. make-clean-vm --suite lucid --arch i386
  71. # on-target needs $DISTRO to be set to debian if using a Debian guest
  72. # (when running gbuild, $DISTRO is set based on the descriptor, so this line isn't needed)
  73. DiSTRO=debian
  74. # For LXC:
  75. LXC_ARCH=i386 LXC_SUITE=lucid on-target ls -la
  76. # For KVM:
  77. start-target 32 lucid-i386 &
  78. # wait a few seconds for VM to start
  79. on-target ls -la
  80. stop-target
  81. ## Building
  82. Copy any additional build inputs into a directory named _inputs_.
  83. Then execute the build using a `YAML` description file (can be run as non-root):
  84. export USE_LXC=1 # LXC only
  85. bin/gbuild <package>.yml
  86. or if you need to specify a commit for one of the git remotes:
  87. bin/gbuild --commit <dir>=<hash> <package>.yml
  88. The resulting report will appear in `result/<package>-res.yml`
  89. To sign the result, perform:
  90. bin/gsign --signer <signer> --release <release-name> <package>.yml
  91. Where `<signer>` is your signing PGP key ID and `<release-name>` is the name for the current release. This will put the result and signature in the `sigs/<package>/<release-name>`. The `sigs/<package>` directory can be managed through git to coordinate multiple signers.
  92. After you've merged everybody's signatures, verify them:
  93. bin/gverify --release <release-name> <package>.yml
  94. ## Poking around
  95. * Log files are captured to the _var_ directory
  96. * You can run the utilities in libexec by running `PATH="libexec:$PATH"`
  97. * To start the target VM run `start-target 32 lucid-i386` or `start-target 64 lucid-amd64`
  98. * To ssh into the target run `on-target` (after setting $DISTRO to debian if using a Debian guest) or `on-target -u root`
  99. * On the target, the _build_ directory contains the code as it is compiled and _install_ contains intermediate libraries
  100. * By convention, the script in `<package>.yml` starts with any environment setup you would need to manually compile things on the target
  101. TODO:
  102. - disable sudo in target, just in case of a hypervisor exploit
  103. - tar and other archive timestamp setter
  104. ## LXC tips
  105. `bin/gbuild` runs `lxc-execute` or `lxc-start`, which may require root. If you are in the admin group, you can add the following sudoers line to prevent asking for the password every time:
  106. %admin ALL=NOPASSWD: /usr/bin/lxc-execute
  107. %admin ALL=NOPASSWD: /usr/bin/lxc-start
  108. Right now `lxc-start` is the default, but you can force `lxc-execute` (useful for Ubuntu 14.04) with:
  109. export LXC_EXECUTE=lxc-execute
  110. Recent distributions allow lxc-execute / lxc-start to be run by non-priviledged users, so you might be able to rip-out the `sudo` calls in `libexec/*`.
  111. If you have a runaway `lxc-start` command, just use `kill -9` on it.
  112. The machine configuration requires access to br0 and assumes that the host address is `10.0.2.2`:
  113. sudo brctl addbr br0
  114. sudo ifconfig br0 10.0.2.2/24 up
  115. ## Tests
  116. Not very extensive, currently.
  117. `python -m unittest discover test`