Changes between Version 18 and Version 19 of HPC_deploy


Ignore:
Timestamp:
2015-11-18T18:32:13+01:00 (9 years ago)
Author:
Pieter Neerincx
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • HPC_deploy

    v18 v19  
    1313The depad group uses [https://hpcugent.github.io/easybuild/ EasyBuild], which uses !EasyConfigs as recipes to enforce consistent, reproducible installations.
    1414In a nutshell an EasyConfig deployment recipe can handle the following steps:
    15 * (Bootstrap an installation and satisfy dependencies)
    16 * Download the (source) code
    17 * Verify checksums of the downloads
    18 * Unpack the downloads
    19 * Configure the build
    20 * Compile the code
    21 * Run sanity checks to verify the build was Ok
    22 * Install the (compiled) code together with it's !EasyBuild log
    23 * Generate a module file for use with a module system to configure the environment at runtime.
     151. (Bootstrap an installation and satisfy dependencies)
     161. Download the (source) code
     171. Verify checksums of the downloads
     181. Unpack the downloads
     191. Configure the build
     201. Compile the code
     211. Run sanity checks to verify the build was Ok
     221. Install the (compiled) code together with it's !EasyBuild log
     231. Generate a module file for use with a module system to configure the environment at runtime.
    2424
    2525The locations where we store source code, deployed apps, their accompanying module files, etc. are documented in the [wiki:HPC_storage#Software Storage SOP].
    26 For many apps !EasyBuild !EasyConfigs are already available; These files are stored
    27 * On [https://github.com/hpcugent/easybuild-easyconfigs/tree/master/easybuild/easyconfigs https://github.com/hpcugent/easybuild-easyconfigs/tree/master/easybuild/easyconfigs]
     26!EasyBuild comes with !EasyConfigs for many our apps of interest ''out of the box''; These files are stored
     27* On [https://github.com/hpcugent/easybuild-easyconfigs/tree/master/easybuild/easyconfigs https://github.com/hpcugent/easybuild-easyconfigs/tree/master/easybuild/easyconfigs] - Latest and greatest from the source.
     28* On our machines in a sub sub sub directory of where !EasyBuild was installed - Latest stable release we deployed.
     29  The easiest way to find the latter is to load !EasyBuild and search for an app like this:
     30{{{
     31$> module load EasyBuild
     32$> module list
     33
     34Currently Loaded Modules:
     35  1) EasyBuild/2.2.0
     36
     37$> eb -S GATK
     38== temporary log file in case of crash /tmp/eb-JHpvCj/easybuild-tR8MKl.log
     39== Searching (case-insensitive) for 'GATK' in /apps/software/EasyBuild/2.2.0/lib/python2.6/site-packages/easybuild_easyconfigs-2.2.0-py2.6.egg/easybuild/easyconfigs
     40CFGS1=/apps/software/EasyBuild/2.2.0/lib/python2.6/site-packages/easybuild_easyconfigs-2.2.0-py2.6.egg/easybuild/easyconfigs/g/GATK
     41 * $CFGS1/GATK-1.0.5083.eb
     42 * $CFGS1/GATK-2.5-2-Java-1.7.0_10.eb
     43 * $CFGS1/GATK-2.6-5-Java-1.7.0_10.eb
     44 * $CFGS1/GATK-2.7-4-Java-1.7.0_10.eb
     45 * $CFGS1/GATK-2.7-4.eb
     46 * $CFGS1/GATK-2.8-1-Java-1.7.0_10.eb
     47 * $CFGS1/GATK-3.0-0-Java-1.7.0_10.eb
     48 * $CFGS1/GATK-3.3-0-Java-1.7.0_21.eb
     49== temporary log file(s) /tmp/eb-JHpvCj/easybuild-tR8MKl.log* have been removed.
     50== temporary directory /tmp/eb-JHpvCj has been removed.
     51}}}
     52* On our machines in {{{/apps/sources/EasyBuild/custom/}}} - Bleeding edge home made !EasyConfigs that were not yet committed to and pull-merged into the !GitHub repos.
    2853
    2954If there is not an easybuild file (.eb) on github and there is no eb file on the cluster (/apps/sources/EasyBuild/custom), we have to create one ourselves.