You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

README_osx.txt 4.7KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283
  1. Deterministic OSX Dmg Notes.
  2. Working OSX DMG's are created in Linux by combining a recent clang,
  3. the Apple's binutils (ld, ar, etc), and DMG authoring tools.
  4. Apple uses clang extensively for development and has upstreamed the necessary
  5. functionality so that a vanilla clang can take advantage. It supports the use
  6. of -F, -target, -mmacosx-version-min, and --sysroot, which are all necessary
  7. when building for OSX. A pre-compiled version of 3.2 is used because it was not
  8. available in the Precise repositories at the time this work was started. In the
  9. future, it can be switched to use system packages instead.
  10. Apple's version of binutils (called cctools) contains lots of functionality
  11. missing in the FSF's binutils. In addition to extra linker options for
  12. frameworks and sysroots, several other tools are needed as well such as
  13. install_name_tool, lipo, and nmedit. These do not build under linux, so they
  14. have been patched to do so. The work here was used as a starting point:
  15. https://github.com/mingwandroid/toolchain4
  16. In order to build a working toolchain, the following source packages are needed
  17. from Apple: cctools, dyld, and ld64.
  18. These tools inject timestamps by default, which produce non-deterministic
  19. binaries. The ZERO_AR_DATE environment variable is used to disable that.
  20. This version of cctools has been patched to use the current version of clang's
  21. headers and and its libLTO.so rather than those from llvmgcc, as it was
  22. originally done in toolchain4.
  23. To complicate things further, all builds must target an Apple SDK. These SDKs
  24. are free to download, but not redistributable.
  25. To obtain it, register for a developer account, then download the XCode 6.1.1 dmg:
  26. https://developer.apple.com/downloads/download.action?path=Developer_Tools/xcode_6.1.1/xcode_6.1.1.dmg
  27. This file is several gigabytes in size, but only a single directory inside is
  28. needed: Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk
  29. Unfortunately, the usual linux tools (7zip, hpmount, loopback mount) are incapable of opening this file.
  30. To create a tarball suitable for gitian input, mount the dmg in OSX, then create it with:
  31. $ tar -C /Volumes/Xcode/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ -czf MacOSX10.9.sdk.tar.gz MacOSX10.9.sdk
  32. The gitian descriptors build 2 sets of files: Linux tools, then Apple binaries
  33. which are created using these tools. The build process has been designed to
  34. avoid including the SDK's files in Gitian's outputs. All interim tarballs are
  35. fully deterministic and may be freely redistributed.
  36. genisoimage is used to create the initial DMG. It is not deterministic as-is,
  37. so it has been patched. A system genisoimage will work fine, but it will not
  38. be deterministic because the file-order will change between invocations.
  39. The patch can be seen here:
  40. https://raw.githubusercontent.com/theuni/osx-cross-depends/master/patches/cdrtools/genisoimage.diff
  41. No effort was made to fix this cleanly, so it likely leaks memory badly. But
  42. it's only used for a single invocation, so that's no real concern.
  43. genisoimage cannot compress DMGs, so afterwards, the 'dmg' tool from the
  44. libdmg-hfsplus project is used to compress it. There are several bugs in this
  45. tool and its maintainer has seemingly abandoned the project. It has been forked
  46. and is available (with fixes) here: https://github.com/theuni/libdmg-hfsplus .
  47. The 'dmg' tool has the ability to create DMG's from scratch as well, but this
  48. functionality is broken. Only the compression feature is currently used.
  49. Ideally, the creation could be fixed and genisoimage would no longer be necessary.
  50. Background images and other features can be added to DMG files by inserting a
  51. .DS_Store before creation. The easiest way to create this file is to build a
  52. DMG without one, move it to a device running OSX, customize the layout, then
  53. grab the .DS_Store file for later use. That is the approach taken here.
  54. As of OSX Mavericks (10.9), using an Apple-blessed key to sign binaries is a
  55. requirement in order to satisfy the new Gatekeeper requirements. Because this
  56. private key cannot be shared, we'll have to be a bit creative in order for the
  57. build process to remain somewhat deterministic. Here's how it works:
  58. - Builders use gitian to create an unsigned release. This outputs an unsigned
  59. dmg which users may choose to bless and run. It also outputs an unsigned app
  60. structure in the form of a tarball, which also contains all of the tools
  61. that have been previously (deterministically) built in order to create a
  62. final dmg.
  63. - The Apple keyholder uses this unsigned app to create a detached signature,
  64. using the script that is also included there.
  65. - Builders feed the unsigned app + detached signature back into gitian. It
  66. uses the pre-built tools to recombine the pieces into a deterministic dmg.