view TODO @ 6512:ccc20ae889ca default tip guix

mingw::guile-2.0.7 builds.
author Jan Nieuwenhuizen <janneke@gnu.org>
date Thu, 24 Mar 2016 08:03:39 +0100
parents 7a1a52f86a6f
children
line wrap: on
line source

* If you have problems installing GUB, setup a debian root [sudo bash TODO]:
  set -ex
  cd /var/tmp
  debootstrap etch etch http://ftp.nl.debian.org/debian
  chroot etch apt-get --force-yes -y install python gcc libc6-dev # minimal
  chroot etch apt-get --force-yes -y install g++ # pkg-config
  chroot etch apt-get --force-yes -y install file # ugh, configure 2.63 hardcodes: /usr/bin/file
  wget -O gub.tar.gz http://github.com/janneke/gub/tarball/master
#  (cd /home/janneke/vc/gub && git tar-tree HEAD | gzip -c - > gub.tar.gz)
#  cp -pv /home/janneke/vc/gub/gub.tar.gz .
  mkdir -p etch/root/vc/gub
  tar -C etch/root/vc/gub --strip-components=1 -xzf gub.tar.gz
  mount -t proc /proc etch/proc # for librestrict
  mkdir -p etch/root/bin
  cp -pv etch/root/vc/gub/sourcefiles/gub.sh etch/root/bin/gub
  chroot etch bash -c 'cd root/vc/gub && gub mingw::lilypond'
  exit 0
   
* add tools::texlive
** see texlive.py: the binary builds.
** test and use/resurrect texmf-tiny from previous build system, see
      http://cvs.savannah.gnu.org/viewvc/*checkout*/lilypond/installers/windows/patch/tetex-texmf-3.0.0-3.patch?content-type=text%2Fplain
  Note: the debootstrap above fails when configuring lilypond,
  texlive-metapost [and then some] is needed, but this pulls
  in so many dependencies...

* Promote specific solutions into general ones [PROMOTEME]
   this cleans up existing packages, and increases chances of
   adding new packages without any changes.

* Auto-detection
** If we'd use Source/Repository for the first two build stages
      download
      untar
   and only *then* instantiate the build class, we could use the
   actual source tree to automatically/generically
      determine the package version
      determine the build class
      determine the dependencies

* Fix Fedora glibc problem, remove workaround in gub/specs/glibc.py,
  or add a build_platform == Fedora test.  [selinux?]
    # Disable librestrict.so, as it causes crashes on Fedora 9 and 10.
    def LD_PRELOAD (self):
        return ''

* junk Makefile, create python based driver/s?
** done: package lilypond-doc replaces DOC* nightmare in lilypond.make
** done: package lilypond-test replaces test-output in lilypond.make
** done: packages lilypond-installer, inkscape-installer.
** todo: doc-clean, test-clean, doc-export, test-export
** todo: python driver that connects [gub, gib], uploads/export calls

* Stat restriction
** Has become much more strict and useful: so in progress again.  Try
   LIBRESTRICT=open:stat bin/gub mingw::lilypond
   and fix any problems.  Common problems are
*** configure looks for fortran [java etc..] that are not even used,
   and STATs the whole PATH, ie /usr/local/bin/f77
   Current approach is to add package-specific ac_prog_F77_PATH=no
   overrides; but this is quite a bit of work.
*** missing tools
*** terribly broken build systems that "just want to" stat /,
   resulting in the need to open up the whole file system :-(
   some packages' build systems are just broken and there seems
   to be no way to guarantee a clean cross build.
** Test other archs, test mingw::denemo, lilypond-installer,
   lilypond-doc, other distributions [Han-Wen], and only then:
** Swap the default from relaxed LIBRESTRICT=open to tighter
   LIBRESTRICT=open:stat.
   
* sharhead suggest or add ~/bin to PATH/.bashrc?
[On sh lily*.sh better as mortal user...]  But then you have to mess
with the execution path, which does not include ~/bin/ by default. Not
perfect for a novice.

* --keep kind-a works, but always triggers a rebuild once
There is conditionally recorded/serialized code, depending
on the state of the file system.  For example:
   
   class AutogenMagic (ForcedAutogenMagic):
    def execute (self, logger):
        package = self.package
        if not os.path.exists (package.expand ('%(autodir)s/configure')):

   
* after fixing --keep, go something smart with GIT, so that
  - *every* work-dir in target/*/src/ is a GIT checkout
  - in the work-dir, GIT can be used to create and maintain patches
  
* Why don't we use tarfile.TarFile?
   It seems that subprocess/read_pipe in gub/gup.py on tar -tzf is
   real inefficient (set buffering?)  FIXED.

* It seems that the removal of LD_LIBRARY_PATH as per

     http://lists.gnu.org/archive/html/lilypond-devel/2008-12/msg00501.html
     http://lists.gnu.org/archive/html/lilypond-devel/2008-12/txteht27nyl4Q.txt

  was a *bad* idea.  Rather, there seems to be something wrong
  with some [Fedora?] systems.

  Quite some packages run conftest binaries linked against
  %(system_prefix)s/lib and for-build binaries created while building,
  that link eg to libltdl.so or libglib.so.

* python3 - python2 compatibility
** testing with python3
** keep check on diff with `make python3'
** remember to use
    list (map (..))
    list (DICT.keys, items, values)
    octal.o755 iso 0755/0o755

    from gub.syntax import printf
    printf () rather than print/print ()

    except:
        t, v, b = sys.exc_info ()
        if t == NameError:
        ...

* support for ccache and icecc (icecream) icecc requires no setup/init
  effort other than picking the host that runs the scheduler, and
  works -more-or-less- automagically with cross compilers.
  [distcc pump?  find comparison with icecream

* we moeten env ook nog ergens losknippen. We zetten nu onze
  eigen variabelen over de user zijn env heen, maar dan kan er dus (niet
  gechecksumde) info uit de user environment lekken.

* junk arbitrary SdkBuild class, handle any build dependencies
  through [module.py].build_dependencies.

* DONE use of member functions vs class variables/static functions 
*** done - get_build_dependencies () -> class.dependencies = []
*** done - def force_sequential_build () -> class.parallel_build_broken
*** done - force_autoupdate () -> class.force_autoupdate = True
*** done - configure_variables, configure_flags
?   + TODO: configure_env_variables
*** done make_flags
*** done compile_flags
*** done install_flags
*** done def configure_command () -> class.configure_command
*** done def compile_command () -> class.compile_command
*** done def install_command () -> class.install_command
*** done get_subpackage_names () -> subpackage_names
     etc

* junk use of /usr in code, use *_prefix) or %(prefix_dir)s

* junk use of /cross in code, use %(cross_dir)s

* use better names for freebsd-runtime, darwin-sdk, which probably
  are in fact kernel-headers and libc+headers

* Fix --branch mess:

  -gub: --branch=<PACKAGE>=<BRANCH>:<LOCAL_BRANCH>, eg
        --branch=lilypond=master-git.sv.gnu.org-lilypond.git
  -cron-builder: --branch=<LILYPOND-BRANCH>
                 --local-branch=<LILYPOND-LOCAL-BRANCH>
    + `branch' means remote branch
  -installer-builder: --branch=<PACKAGE>=<LOCAL_BRANCH>
    + `branch' means local branch

* Remove duplication of directory layout.  layout is partly duplicated
  in test-lily/* and *.make.  gub/settings should usable now from any
  .py, gub/settings.py --plaftorm '' prints layout for use in sh/make.

* Document and make easier [plugin.., see gub/gup.py] to add new platform

    -3. gub/settings.py
    -2. gub/config_cache.py
    -1. gub/installer.py
     0. lilypond.make
     1. toevoegen aan platforms in makefile
     2. toevoegen aan platform lijst, die in test-lily/upload wordt gebruikt.

* cron-builder: disable initial download:
  --stage=download depends on tools to be built:
   /usr/bin/python bin/gub --branch=lilypond=master:master-git.sv.gnu.org-lilypond.git --platform=linux-64 --stage=download lilypond
  File "/home/janneke/vc/gub-test/gub/specs/fontconfig.py", line 33, in freetype_cflags
    raise SystemFailed ('Pipe failed: %(cmd)s' % locals ())
SystemFailed: Pipe failed: /home/janneke/vc/gub-test/target/tools/root/usr/bin/freetype-config --cflags


* Get rid of (all?), *args and **kwarg constructions, at least where
  they are now part of the user api.
** done for (most important?) command.py:System, loggedos.system

* Resurrect/add:
  --keep [most annoying for debugging]
  --force [partly fixed: force full rebuild]
  --immediate? vs serialized

* Printing/logging to console

** oslog.verbose_flag for tar commands has been commented-out?  Old
   behaviour is somewhat foo but better than no behaviour at all.  We
   should *always* use -v but send that output to different pipe.  It
   should *always* be in .log file, but printed to console only if
   verbosity > 'command'.
   
* Smarter checksumming for functions: should add a dry-run to loggedos
  so we can do dry-run on functions; then run dry-run on a dummy
  argument.

* investigate flavours of libtool_la fixups:
   - gubb.libtool_installed_la_fixups ()
   - gup.libtool_la_fixup ()
   - targetpackage.pre_install_libtool_fixup ()

* cron-builder: disable initial download:
  --stage=download depends on tools to be built:
   /usr/bin/python bin/gub --branch=lilypond=master:master-git.sv.gnu.org-lilypond.git --platform=linux-64 --stage=download lilypond
  File "/home/janneke/vc/gub-test/gub/specs/fontconfig.py", line 33, in freetype_cflags
    raise SystemFailed ('Pipe failed: %(cmd)s' % locals ())
SystemFailed: Pipe failed: /home/janneke/vc/gub-test/target/tools/root/usr/bin/freetype-config --cflags


* Get rid of (all?), *args and **kwarg constructions, at least where
  they are now part of the user api.
** done for (most important?) command.py:System, loggedos.system

* Resurrect/add:
  --keep [most annoying for debugging]
  --force [partly fixed: force full rebuild]
  --immediate? vs serialized

* Printing/logging to console

** oslog.verbose_flag for tar commands has been commented-out?  Old
   behaviour is somewhat foo but better than no behaviour at all.  We
   should *always* use -v but send that output to different pipe.  It
   should *always* be in .log file, but printed to console only if
   verbosity > 'command'.
   
* Smarter checksumming for functions: should add a dry-run to loggedos
  so we can do dry-run on functions; then run dry-run on a dummy
  argument.

* investigate flavours of libtool_la fixups:
   - gubb.libtool_installed_la_fixups ()
   - gup.libtool_la_fixup ()
   - targetpackage.pre_install_libtool_fixup ()

RENAMES

* repository -> source
* ./gub -> gublib ?
* ./gub/specs -> gub/builds?

* Use names of stage iso number in status/stamp file.

* code cygwin installer as alternative build spec.

* code .deb/ipkg installer as alternative build spec.

* look at other installer-builders -> build spec?

* fix all instances of makeflags () and compile/install

* move wrap_pkg_config from libgphoto2 to target/build spec (note
  configure: PATH setting?)

* Bootstrap whole toolchain from source on more platforms (freebsd)?

* Packages file with download and update facility, like
  cyg-apt.

LOWER PRIORITY

* do not package emtpy subpackages (doc, devel)

- replace os. calls with os.context wrapper ones (make a real
  dry-run to quick-test all .py scripts?)

- name for gub-tester (test-repo, repo-builder?)

- Split gub/*py into packager, builder, platform

- Explode inheritance, and use membership in GUP

- move patches upstream, eg.

  * libpng

  * python x-compile.

  * cygwin GCC

  * zlib

- update packages:

  * Gnome 2.16