Mercurial > gub
view TODO @ 5504:a1ce4bc5aaee
Add failsafe debootstrap recipe. Move philosophical stuff way down.
author | Jan Nieuwenhuizen <janneke@gnu.org> |
---|---|
date | Thu, 20 Aug 2009 20:36:49 +0200 |
parents | aae6f1ef2941 |
children | 7eea529f3f81 |
line wrap: on
line source
* If you have problems installing GUB, just setup a debian root: debootstrap etch etch http://ftp.nl.debian.org/debian chroot etch apt-get install python gcc libc6-dev' # to run the thing mount -t proc /proc etch/proc # for librestrict chroot etch apt-get install g++ # for pkg-config wget -O gub.tar.gz http://github.com/janneke/gub/tarball/master mkdir -p etch/root/gub tar -C etch/root/gub -xzf gub.tar.gz chroot etch bash -c 'cd root/gub && bin/gub mingw::lilypond' * Fix Fedora glibc problem, remove workaround in gub/specs/glibc.py, or add a build_platform == Fedora test. [selinux?] # Disable librestrict.so, as it causes crashes on Fedora 9 and 10. def LD_PRELOAD (self): return '' * junk Makefile, create python based driver/s? ** done: package lilypond-doc replaces DOC* nightmare in lilypond.make ** done: package lilypond-test replaces test-output in lilypond.make ** done: packages lilypond-installer, inkscape-installer. ** todo: doc-clean, test-clean, doc-export, test-export ** todo: python driver that connects [gub, gib], uploads/export calls * Stat restriction ** On my system, Jaunty-64 bit, the restricted LIBRESTRICT=open:stat bin/gub mingw::lilypond now builds. ** Test other archs, lilypond-installer, lilypond-doc, other distributions [Han-Wen], and only then: ** Swap the default from relaxed LIBRESTRICT=open to tighter LIBRESTRICT=open:stat. * sharhead suggest or add ~/bin to PATH/.bashrc? [On sh lily*.sh better as mortal user...] But then you have to mess with the execution path, which does not include ~/bin/ by default. Not perfect for a novice. * --keep kind-a works, but always triggers a rebuild once There is conditionally recorded/serialized code, depending on the state of the file system. For example: class AutogenMagic (ForcedAutogenMagic): def execute (self, logger): package = self.package if not os.path.exists (package.expand ('%(autodir)s/configure')): * after fixing --keep, go something smart with GIT, so that - *every* work-dir in target/*/src/ is a GIT checkout - in the work-dir, GIT can be used to create and maintain patches * Why don't we use tarfile.TarFile? It seems that subprocess/read_pipe in gub/gup.py on tar -tzf is real inefficient (set buffering?) * It seems that the removal of LD_LIBRARY_PATH as per http://lists.gnu.org/archive/html/lilypond-devel/2008-12/msg00501.html http://lists.gnu.org/archive/html/lilypond-devel/2008-12/txteht27nyl4Q.txt was a *bad* idea. Rather, there seems to be something wrong with some [Fedora?] systems. Quite some packages run conftest binaries linked against %(system_prefix)s/lib and for-build binaries created while building, that link eg to libltdl.so or libglib.so. * python3 - python2 compatibility ** testing with python3 ** keep check on diff with `make python3' ** remember to use list (map (..)) list (DICT.keys, items, values) octal.o755 iso 0755/0o755 from gub.syntax import printf printf () rather than print/print () except: t, v, b = sys.exc_info () if t == NameError: ... * support for ccache and icecc (icecream) icecc requires no setup/init effort other than picking the host that runs the scheduler, and works -more-or-less- automagically with cross compilers. [distcc pump? find comparison with icecream * we moeten env ook nog ergens losknippen. We zetten nu onze eigen variabelen over de user zijn env heen, maar dan kan er dus (niet gechecksumde) info uit de user environment lekken. * junk arbitrary SdkBuild class, handle any build dependencies through [module.py].build_dependencies. * use of member functions vs class variables/static functions: - get_build_dependencies () -> class.build_dependencies = [] - def force_sequential_build () -> class.sequential_build = True - def force_autoupdate () -> class.force_autoupdate = True ** also, why are these functions ? - def compile_command () -> class.compile_command - def install_command () -> class.install_command etc - de ellende met klutserige taal als pieton is natuurlijk dat je niet kunt class.makeflags = if Foo.bar: 3 else: 4; --> hah, kan in python2.5, maar we willen 2.4 supporten? [nouja, je kunt met idiote list comprehension alles, maar clien is anders] [x for x in [3, 4] if Foo.bar and x == 3 or not Foo.bar and x == 4][0] dus we moeten wel uitkijken wat we doen... - need_source_tree ? * junk use of /usr in code, use *_prefix) or %(prefix_dir)s * junk use of /cross in code, use %(cross_dir)s * use better names for freebsd-runtime, darwin-sdk, which probably are in fact kernel-headers and libc+headers * Fix --branch mess: -gub: --branch=<PACKAGE>=<BRANCH>:<LOCAL_BRANCH>, eg --branch=lilypond=master-git.sv.gnu.org-lilypond.git -cron-builder: --branch=<LILYPOND-BRANCH> --local-branch=<LILYPOND-LOCAL-BRANCH> + `branch' means remote branch -installer-builder: --branch=<PACKAGE>=<LOCAL_BRANCH> + `branch' means local branch * Remove duplication of directory layout. layout is partly duplicated in test-lily/* and *.make. gub/settings should usable now from any .py, gub/settings.py --plaftorm '' prints layout for use in sh/make. * Document and make easier [plugin.., see gub/gup.py] to add new platform -3. gub/settings.py -2. gub/config_cache.py -1. gub/installer.py 0. lilypond.make 1. toevoegen aan platforms in makefile 2. toevoegen aan platform lijst, die in test-lily/upload wordt gebruikt. * cron-builder: disable initial download: --stage=download depends on tools to be built: /usr/bin/python bin/gub --branch=lilypond=master:master-git.sv.gnu.org-lilypond.git --platform=linux-64 --stage=download lilypond File "/home/janneke/vc/gub-test/gub/specs/fontconfig.py", line 33, in freetype_cflags raise SystemFailed ('Pipe failed: %(cmd)s' % locals ()) SystemFailed: Pipe failed: /home/janneke/vc/gub-test/target/tools/root/usr/bin/freetype-config --cflags * Get rid of (all?), *args and **kwarg constructions, at least where they are now part of the user api. ** done for (most important?) command.py:System, loggedos.system * Resurrect/add: --keep [most annoying for debugging] --force [partly fixed: force full rebuild] --immediate? vs serialized * Printing/logging to console ** oslog.verbose_flag for tar commands has been commented-out? Old behaviour is somewhat foo but better than no behaviour at all. We should *always* use -v but send that output to different pipe. It should *always* be in .log file, but printed to console only if verbosity > 'command'. * Smarter checksumming for functions: should add a dry-run to loggedos so we can do dry-run on functions; then run dry-run on a dummy argument. * investigate flavours of libtool_la fixups: - gubb.libtool_installed_la_fixups () - gup.libtool_la_fixup () - targetpackage.pre_install_libtool_fixup () * cron-builder: disable initial download: --stage=download depends on tools to be built: /usr/bin/python bin/gub --branch=lilypond=master:master-git.sv.gnu.org-lilypond.git --platform=linux-64 --stage=download lilypond File "/home/janneke/vc/gub-test/gub/specs/fontconfig.py", line 33, in freetype_cflags raise SystemFailed ('Pipe failed: %(cmd)s' % locals ()) SystemFailed: Pipe failed: /home/janneke/vc/gub-test/target/tools/root/usr/bin/freetype-config --cflags * Get rid of (all?), *args and **kwarg constructions, at least where they are now part of the user api. ** done for (most important?) command.py:System, loggedos.system * Resurrect/add: --keep [most annoying for debugging] --force [partly fixed: force full rebuild] --immediate? vs serialized * Printing/logging to console ** oslog.verbose_flag for tar commands has been commented-out? Old behaviour is somewhat foo but better than no behaviour at all. We should *always* use -v but send that output to different pipe. It should *always* be in .log file, but printed to console only if verbosity > 'command'. * Smarter checksumming for functions: should add a dry-run to loggedos so we can do dry-run on functions; then run dry-run on a dummy argument. * investigate flavours of libtool_la fixups: - gubb.libtool_installed_la_fixups () - gup.libtool_la_fixup () - targetpackage.pre_install_libtool_fixup () RENAMES * repository -> source * ./gub -> gublib ? * ./gub/specs -> gub/builds? * Use names of stage iso number in status/stamp file. * code cygwin installer as alternative build spec. * code .deb/ipkg installer as alternative build spec. * look at other installer-builders -> build spec? * fix all instances of makeflags () and compile/install * move wrap_pkg_config from libgphoto2 to target/build spec (note configure: PATH setting?) * Bootstrap whole toolchain from source on more platforms (freebsd)? * Packages file with download and update facility, like cyg-apt. LOWER PRIORITY * do not package emtpy subpackages (doc, devel) - replace os. calls with os.context wrapper ones (make a real dry-run to quick-test all .py scripts?) - name for gub-tester (test-repo, repo-builder?) - Split gub/*py into packager, builder, platform - Explode inheritance, and use membership in GUP - move patches upstream, eg. * libpng * python x-compile. * cygwin GCC * zlib - update packages: * Gnome 2.16 * Rethink raison d'etre and design flaws Initial idea: provide up-to-date LilyPond binaries and take away maintenance of several quite broken cross build systems by unifying and fixing them. Creating an easy, dependable, code-reusing, python-based build system that's fun to work with. ** Options *** Make a real distro Fix fatal flaws, accepting the consequences of actually being a GNU/Linux distribution specialized for cross building **** GUB uses nor checks any signatures or checksums, GUB does not include or track any [CVE-] security patches [provided by others/distros]. *** stop now Work with/move to eg emdebian.org (build.opensuse.org) and suffer dpkg or rpm packaging scripts. **** emdebian.org We adapt Debian tools so you can build/cross-compile Debian packages or adapted packages with info on how to cross-build and build smaller packages. *** Do nothing until it bit-rots Not making GUB a real distro may make it too hard to attract many more developers/users. Making GUB a real distro may be too much work. * Fatal flaw: non-[c]rooted build ** Implications *** On linux-64, target/tools != target/linux-64 *** Unable to ship GUB binary packages for tools because tools are built with full /home/janneke/vc/gub/target/... prefix *** Must provide special __tools spec *** No [free] native/tools version of every included package --> GUB still needs a distro's GCC, G++ etc. otherwise it could use it's own, from it's own binary packages. ** use /gub root, with . --> gub symlink inside it? /gub/gub --> ./ /gub/usr/bin --> ../x86_64-pc-linux/bin /gub/usr/lib --> ../x86_64-pc-linux/lib /gub/usr/i686-pc-linux/bin /gub/usr/i686-pc-linux/src /gub/usr/i686-pc-linux/build /gub/usr/x86_64-pc-linux/bin /gub/usr/x86_64-pc-linux/src *** Can have binary packages even install anywhere as long as you're root and use chrooted build *** Can use tools and other stuff directly in host system by adding /gub/usr/bin to PATH?