view TODO @ 5512:9784b7c6be0e

Remove done stages from SKIP. Fixes taintedness check.
author Jan Nieuwenhuizen <janneke@gnu.org>
date Thu, 20 Aug 2009 22:23:57 +0200
parents 7eea529f3f81
children d6fd5f779ca0
line wrap: on
line source

* If you have problems installing GUB, just setup a debian root:
  set -ex
  cd /var/tmp
  debootstrap etch etch http://ftp.nl.debian.org/debian
  chroot etch apt-get --force-yes -y install python gcc libc6-dev # minimal
  chroot etch apt-get --force-yes -y install g++                  # pkg-config
  wget -O gub.tar.gz http://github.com/janneke/gub/tarball/master
#  (cd /home/janneke/vc/gub && git tar-tree HEAD | gzip -c - > gub.tar.gz)
#  cp -pv /home/janneke/vc/gub/gub.tar.gz .
  mkdir -p etch/root/gub
  tar -C etch/root/gub --strip-components=1 -xzf gub.tar.gz
  mount -t proc /proc etch/proc # for librestrict
  chroot etch bash -c 'cd root/gub && bin/gub mingw::lilypond'
  exit 0

* Fix Fedora glibc problem, remove workaround in gub/specs/glibc.py,
  or add a build_platform == Fedora test.  [selinux?]
    # Disable librestrict.so, as it causes crashes on Fedora 9 and 10.
    def LD_PRELOAD (self):
        return ''

* junk Makefile, create python based driver/s?
** done: package lilypond-doc replaces DOC* nightmare in lilypond.make
** done: package lilypond-test replaces test-output in lilypond.make
** done: packages lilypond-installer, inkscape-installer.
** todo: doc-clean, test-clean, doc-export, test-export
** todo: python driver that connects [gub, gib], uploads/export calls

* Stat restriction
** On my system, Jaunty-64 bit, the restricted
   LIBRESTRICT=open:stat bin/gub mingw::lilypond
now builds.
** Test other archs, lilypond-installer, lilypond-doc, other
   distributions [Han-Wen], and only then:
** Swap the default from relaxed LIBRESTRICT=open to tighter
   LIBRESTRICT=open:stat.
   
* sharhead suggest or add ~/bin to PATH/.bashrc?
[On sh lily*.sh better as mortal user...]  But then you have to mess
with the execution path, which does not include ~/bin/ by default. Not
perfect for a novice.

* --keep kind-a works, but always triggers a rebuild once
There is conditionally recorded/serialized code, depending
on the state of the file system.  For example:
   
   class AutogenMagic (ForcedAutogenMagic):
    def execute (self, logger):
        package = self.package
        if not os.path.exists (package.expand ('%(autodir)s/configure')):

   
* after fixing --keep, go something smart with GIT, so that
  - *every* work-dir in target/*/src/ is a GIT checkout
  - in the work-dir, GIT can be used to create and maintain patches
  
* Why don't we use tarfile.TarFile?  It seems that subprocess/read_pipe
  in gub/gup.py on tar -tzf is real inefficient (set buffering?)

* It seems that the removal of LD_LIBRARY_PATH as per

     http://lists.gnu.org/archive/html/lilypond-devel/2008-12/msg00501.html
     http://lists.gnu.org/archive/html/lilypond-devel/2008-12/txteht27nyl4Q.txt

  was a *bad* idea.  Rather, there seems to be something wrong
  with some [Fedora?] systems.

  Quite some packages run conftest binaries linked against
  %(system_prefix)s/lib and for-build binaries created while building,
  that link eg to libltdl.so or libglib.so.

* python3 - python2 compatibility
** testing with python3
** keep check on diff with `make python3'
** remember to use
    list (map (..))
    list (DICT.keys, items, values)
    octal.o755 iso 0755/0o755

    from gub.syntax import printf
    printf () rather than print/print ()

    except:
	t, v, b = sys.exc_info ()
	if t == NameError:
        ...

* support for ccache and icecc (icecream) icecc requires no setup/init
  effort other than picking the host that runs the scheduler, and
  works -more-or-less- automagically with cross compilers.
  [distcc pump?  find comparison with icecream

* we moeten env ook nog ergens losknippen. We zetten nu onze
  eigen variabelen over de user zijn env heen, maar dan kan er dus (niet
  gechecksumde) info uit de user environment lekken.

* junk arbitrary SdkBuild class, handle any build dependencies
  through [module.py].build_dependencies.

* use of member functions vs class variables/static functions:
   - get_build_dependencies () -> class.build_dependencies = []
   - def force_sequential_build () -> class.sequential_build = True
   - def force_autoupdate () -> class.force_autoupdate = True
** also, why are these functions ?
   - def compile_command () -> class.compile_command
   - def install_command () -> class.install_command
     etc
   - de ellende met klutserige taal als pieton is natuurlijk dat
     je niet kunt
       class.makeflags = if Foo.bar: 3 else: 4;
       --> hah, kan in python2.5, maar we willen 2.4 supporten?
     [nouja, je kunt met idiote list comprehension alles, maar clien is anders]
       [x for x in [3, 4] if Foo.bar and x == 3 or not Foo.bar and x == 4][0]
     dus we moeten wel uitkijken wat we doen...
   - need_source_tree ?

* junk use of /usr in code, use *_prefix) or %(prefix_dir)s

* junk use of /cross in code, use %(cross_dir)s

* use better names for freebsd-runtime, darwin-sdk, which probably
  are in fact kernel-headers and libc+headers

* Fix --branch mess:

  -gub: --branch=<PACKAGE>=<BRANCH>:<LOCAL_BRANCH>, eg
        --branch=lilypond=master-git.sv.gnu.org-lilypond.git
  -cron-builder: --branch=<LILYPOND-BRANCH>
                 --local-branch=<LILYPOND-LOCAL-BRANCH>
    + `branch' means remote branch
  -installer-builder: --branch=<PACKAGE>=<LOCAL_BRANCH>
    + `branch' means local branch

* Remove duplication of directory layout.  layout is partly duplicated
  in test-lily/* and *.make.  gub/settings should usable now from any
  .py, gub/settings.py --plaftorm '' prints layout for use in sh/make.

* Document and make easier [plugin.., see gub/gup.py] to add new platform

    -3. gub/settings.py
    -2. gub/config_cache.py
    -1. gub/installer.py
     0. lilypond.make
     1. toevoegen aan platforms in makefile
     2. toevoegen aan platform lijst, die in test-lily/upload wordt gebruikt.

* cron-builder: disable initial download:
  --stage=download depends on tools to be built:
   /usr/bin/python bin/gub --branch=lilypond=master:master-git.sv.gnu.org-lilypond.git --platform=linux-64 --stage=download lilypond
  File "/home/janneke/vc/gub-test/gub/specs/fontconfig.py", line 33, in freetype_cflags
    raise SystemFailed ('Pipe failed: %(cmd)s' % locals ())
SystemFailed: Pipe failed: /home/janneke/vc/gub-test/target/tools/root/usr/bin/freetype-config --cflags


* Get rid of (all?), *args and **kwarg constructions, at least where
  they are now part of the user api.
** done for (most important?) command.py:System, loggedos.system

* Resurrect/add:
  --keep [most annoying for debugging]
  --force [partly fixed: force full rebuild]
  --immediate? vs serialized

* Printing/logging to console

** oslog.verbose_flag for tar commands has been commented-out?  Old
   behaviour is somewhat foo but better than no behaviour at all.  We
   should *always* use -v but send that output to different pipe.  It
   should *always* be in .log file, but printed to console only if
   verbosity > 'command'.
   
* Smarter checksumming for functions: should add a dry-run to loggedos
  so we can do dry-run on functions; then run dry-run on a dummy
  argument.

* investigate flavours of libtool_la fixups:
   - gubb.libtool_installed_la_fixups ()
   - gup.libtool_la_fixup ()
   - targetpackage.pre_install_libtool_fixup ()

* cron-builder: disable initial download:
  --stage=download depends on tools to be built:
   /usr/bin/python bin/gub --branch=lilypond=master:master-git.sv.gnu.org-lilypond.git --platform=linux-64 --stage=download lilypond
  File "/home/janneke/vc/gub-test/gub/specs/fontconfig.py", line 33, in freetype_cflags
    raise SystemFailed ('Pipe failed: %(cmd)s' % locals ())
SystemFailed: Pipe failed: /home/janneke/vc/gub-test/target/tools/root/usr/bin/freetype-config --cflags


* Get rid of (all?), *args and **kwarg constructions, at least where
  they are now part of the user api.
** done for (most important?) command.py:System, loggedos.system

* Resurrect/add:
  --keep [most annoying for debugging]
  --force [partly fixed: force full rebuild]
  --immediate? vs serialized

* Printing/logging to console

** oslog.verbose_flag for tar commands has been commented-out?  Old
   behaviour is somewhat foo but better than no behaviour at all.  We
   should *always* use -v but send that output to different pipe.  It
   should *always* be in .log file, but printed to console only if
   verbosity > 'command'.
   
* Smarter checksumming for functions: should add a dry-run to loggedos
  so we can do dry-run on functions; then run dry-run on a dummy
  argument.

* investigate flavours of libtool_la fixups:
   - gubb.libtool_installed_la_fixups ()
   - gup.libtool_la_fixup ()
   - targetpackage.pre_install_libtool_fixup ()

RENAMES

* repository -> source
* ./gub -> gublib ?
* ./gub/specs -> gub/builds?

* Use names of stage iso number in status/stamp file.

* code cygwin installer as alternative build spec.

* code .deb/ipkg installer as alternative build spec.

* look at other installer-builders -> build spec?

* fix all instances of makeflags () and compile/install

* move wrap_pkg_config from libgphoto2 to target/build spec (note
  configure: PATH setting?)

* Bootstrap whole toolchain from source on more platforms (freebsd)?

* Packages file with download and update facility, like
  cyg-apt.

LOWER PRIORITY

* do not package emtpy subpackages (doc, devel)

- replace os. calls with os.context wrapper ones (make a real
  dry-run to quick-test all .py scripts?)

- name for gub-tester (test-repo, repo-builder?)

- Split gub/*py into packager, builder, platform

- Explode inheritance, and use membership in GUP

- move patches upstream, eg.

  * libpng

  * python x-compile.

  * cygwin GCC

  * zlib

- update packages:

  * Gnome 2.16

 * Rethink raison d'etre and design flaws
Initial idea: provide up-to-date LilyPond binaries and take away
maintenance of several quite broken cross build systems by unifying
and fixing them.  Creating an easy, dependable, code-reusing,
python-based build system that's fun to work with.
** Options
*** Make a real distro
Fix fatal flaws, accepting the consequences of actually being a
GNU/Linux distribution specialized for cross building
**** GUB uses nor checks any signatures or checksums, GUB does
not include or track any [CVE-] security patches [provided by
others/distros].
*** stop now
Work with/move to eg emdebian.org (build.opensuse.org) and suffer dpkg
or rpm packaging scripts.
**** emdebian.org
We adapt Debian tools so you can build/cross-compile Debian
packages or adapted packages with info on how to cross-build and
build smaller packages.
*** Do nothing until it bit-rots
Not making GUB a real distro may make it too hard to attract many more
developers/users.
Making GUB a real distro may be too much work.

* Fatal flaw: non-[c]rooted build
** Implications
*** On linux-64, target/tools != target/linux-64
*** Unable to ship GUB binary packages for tools
because tools are built with full /home/janneke/vc/gub/target/... prefix
*** Must provide special __tools spec
*** No [free] native/tools version of every included package
--> GUB still needs a distro's GCC, G++ etc.  otherwise it
could use it's own, from it's own binary packages.
** use /gub root, with . --> gub symlink inside it?
    /gub/gub --> ./
    /gub/usr/bin --> ../x86_64-pc-linux/bin
    /gub/usr/lib --> ../x86_64-pc-linux/lib
    /gub/usr/i686-pc-linux/bin
    /gub/usr/i686-pc-linux/src
    /gub/usr/i686-pc-linux/build
    /gub/usr/x86_64-pc-linux/bin
    /gub/usr/x86_64-pc-linux/src

*** Can have binary packages
even install anywhere as long as you're root and use chrooted build
*** Can use tools and other stuff directly in host system by adding
/gub/usr/bin to PATH?