Changes between Version 43 and Version 44 of HPC_deploy

Timestamp:

2017-06-15T19:06:30+02:00 (7 years ago)

Author:

Pieter Neerincx

Comment:

--

HPC_deploy

@@ -321 +321 @@
 A: First check if the location where !EasyBuild tries to download the source code is still up-to-date. If not update the !EasyConfig. If the location is correct, but !EasyBuild cannot access this location directly for example because it is blocked by our firewall or because it requires authentication, you can try to download the source manually and put it in the cache directory for the app in {{{/apps/sources/[a-z]/NameOfTheApp/}}}. When !EasyBuild finds the cached source code it will skip the download step and continue.
 ==== Q: How can I resume a partially failed/succeeded deployment of an !EasyConfig with a lot of extra packages/modules?
-A: When you have an !EasyConfig for a language like Perl, Python, R, etc. you may have a long list of extra Perl modules, Python eggs, R packages, etc.
-If your install failed near the end of these extensions it ain't fun to start all over from scratch. 
+A: When you have an !EasyConfig of type {{{easyblock = 'Bundle'}}} with a long list of extra  modules, packages, etc. for a language like Perl, Python, R, etc. and your install failed partially it ain't fun to start all over from scratch. 
 You can resume the install after fixing the issue when
-* Your config specifies both the extension defaultclass - e.g. {{{exts_defaultclass = 'PerlModule'}}} 
-* and the extension filter (command to test whether installation of the extension succeeded) - e.g. {{{exts_filter = ("perldoc -lm %(ext_name)s ", "")}}}
-* and when you use the {{{--skip}}} and {{{--resume}}} commandline options for the {{{eb}}} command - e.g.:
+* Your config specifies the extension defaultclass (= the !EasyBlock) using {{{exts_defaultclass = 'someEasyBlock'}}} 
+* and the extension filter (= command to test whether installation of the extension succeeded) using {{{exts_filter = ("someCommand", "")}}}
+* and when you've added the path where the additional modules/packages will be installed to the language specific environment variable that is used to search for extras - {{{PERL5LIB}}} for Perl, {{{PYTHONPATH}}} for Python, {{{R_LIBS}}} for R, etc.
+* and when you use the {{{--skip}}} and {{{--resume}}} commandline options for the {{{eb}}} command - e.g. 
   {{{
 $> eb --skip --rebuild \
       --robot --robot-paths=${HOME}/git/easybuild-easyconfigs/easybuild/easyconfigs/: \
-        git/easybuild-easyconfigs/easybuild/easyconfigs/p/PerlPlus/PerlPlus-5.22.0-foss-2015b-v17.01.1.eb
+      path/to/someEaysyConfig.eb
   }}}
+* and when deployment already finished succesfully resulting in a module file, but there is an easy workaround: just touch an empty module file: 
+  {{{
+$> touch ${HPC_ENV_PREFIX/modules/all/moduleName/moduleVersion.lua
+  }}}
+  Once the deployment finished successfully the fake empty module file will be overwritten with a proper one.
+
+Example of a minimal Bundle to extend Perl 5.22.0 with the module Some::Module version 1.2.3 from CPAN:
+{{{
+easyblock = 'Bundle'
+
+name = 'PerlPlus'
+version = '5.22.0'          # Same as the vanilla Perl module on which these add-on modules depend.
+versionsuffix = '-v17.01.1' # In format YY.MM.IncrementedReleaseNumber.
+
+homepage = 'http://www.perl.org/'
+description = """Extra modules for Larry Wall's Practical Extraction and Report Language."""
+
+toolchain = {'name': 'foss', 'version': '2015b'}
+toolchainopts = {'optarch': True, 'pic': True}
+
+dependencies = [
+    ('Perl', version, '-bare'),
+]
+
+modextrapaths = {'PERL5LIB': ['lib/perl5/', 'lib/perl5/site_perl','lib/perl5/site_perl/5.22.0/'] }
+moduleclass = 'lang'
+
+exts_defaultclass = 'PerlModule'
+exts_filter = ("perldoc -lm %(ext_name)s ", "")
+exts_list = [
+    ('Some::Module', '1.2.3', {
+        'source_tmpl': 'Some-Module-1.2.3.tar.gz',
+        'source_urls': ['https://cpan.metacpan.org/authors/id/A/AU/AUTHOR'],
+    }),
+}}}
+
+Example of a minimal Bundle to extend Python 2.7.11 with the egg !SomeEgg version 1.2.3 from !PyPi:
+
+{{{
+easyblock = 'Bundle'
+
+name = 'PythonPlus'
+version = '%(pyver)s'          # Same as the vanilla Python module on which these add-on modules depend.
+versionsuffix = '-v17.06.1'    # In format YY.MM.IncrementedReleaseNumber.
+
+homepage = 'https://www.python.org/'
+description = """The PythonPlus bundle contains add-on modules for Python."""
+
+toolchain = {'name': 'foss', 'version': '2015b'}
+
+dependencies = [
+    ('Python', '2.7.11'),
+]
+
+exts_defaultclass = 'PythonPackage'
+exts_filter = ('python -c "import %(ext_name)s"', "")
+
+exts_list = [
+    ('SomeEgg', '1.2.3', {
+        'source_urls': ['https://pypi.python.org/packages/....long URL'],
+        'checksums': ['da32434ebfebae2c7506e9577ac558f5'],
+        'source_tmpl': '%(name)s-%(version)s.zip', # Only required when file name deviates from default naming scheme @ PiPy.
+        'modulename': 'segg', # name of module for Python import command. Only required when not the same as name of the extension.
+    }),
+]
+
+modextrapaths = {'PYTHONPATH': ['lib/python%(pyshortver)s/site-packages']}
+
+full_sanity_check = True
+sanity_check_paths = {
+    'files': [],
+    'dirs':  ['lib/python%(pyshortver)s/site-packages'],
+}
+
+moduleclass = 'lang'
+}}}