Changes between Version 21 and Version 22 of Modules


Ignore:
Timestamp:
2011-01-31T13:58:02+01:00 (14 years ago)
Author:
Erik Roos
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Modules

    v21 v22  
    11= Modules =
    2 Status options?
     2
     3== Status options ==
    34
    45|| Planned || Scheduled to start work ||
     
    89|| Finished || Module is done, but can undergo fixes or small changes ||
    910
    10 List of modules
     11== List of modules ==
    1112
    12 || '''Name''' || '''Description''' || '''Applications''' || '''Status''' || '''Screenshot''' ||
    13 || !StudyBook || Shows an overview of elements in a study, for example !InvestigationElements, Data and Files. || XGAP, Pheno, NGS || Prototype || [[Image(Picture 2.png,100px)]] ||
    14 || !MatrixViewer || Browse matrix data, allows twodimensional filtering, sorting and searching. || xQTL, AnimalDB || Prototype || [[Image(Picture 3.png,100px)]] ||
    15 || Auth || Secures your data by letting users register themselved and login before entering the application || All ? || Draft || [[Image(auth.jpg,100px)]] ||
    16 || !ExcelWizard || Import any MOLGENIS data into your database using Excel files || All ? || Draft || [[Image(genericwizard-xqtl1.png,100px)]] ||
    17 || !JobManager || Monitor and manage computational jobs || xQTL, NGS || Prototype || [[Image(jobman.png,100px)]] ||
    18 || Heatmap || View matrix data as an adjustable heatmap || xQTL || Draft || [[Image(heatmap.png,100px)]] ||
    19 || RPlots || Create simple R plots from matrix data || xQTL || Draft || [[Image(rplot.png,100px)]] ||
     13=== Study Capturing ===
    2014
    21 Status: this is under development for MOLGENIS 4.0 Don't expect this if you use MOLGENIS 3.x!
     15|| '''Name''' || '''Description''' || '''Applications''' || '''Status''' || '''Wiki page''' || '''Screenshot''' ||
     16|| Auth || Secures your data by letting users register themselved and login before entering the application || All ? || Draft || AuthModule || [[Image(auth.jpg,100px)]] ||
     17|| !ExcelWizard || Import any MOLGENIS data into your database using Excel files || All ? || Draft || ExcelWizardModule || [[Image(genericwizard-xqtl1.png,100px)]] ||
    2218
    23 = Architectural overview =
    24 MOLGENIS is organized as a set of modules which we call plugins. Each plugin adds a combination of
     19=== Study Sharing ===
    2520
    26  * MolgenisModels - models that use the domain specific language to define functionality compactly
    27  * MolgenisGenerators - templates that expand the models into working software
    28  * MolgenisComponents -
     21|| '''Name''' || '''Description''' || '''Applications''' || '''Status''' || '''Wiki page''' || '''Screenshot''' ||
     22|| !StudyBook || Shows an overview of elements in a study, for example !InvestigationElements, Data and Files. || XGAP, Pheno, NGS || Prototype || StudyBookModule || [[Image(Picture 2.png,100px)]] ||
     23|| !MatrixViewer || Browse matrix data, allows twodimensional filtering, sorting and searching. || xQTL, AnimalDB || Prototype || MatrixModule || [[Image(Picture 3.png,100px)]] ||
     24|| Heatmap || View matrix data as an adjustable heatmap || xQTL || Draft || HeatMapModule || [[Image(heatmap.png,100px)]] ||
     25|| RPlots || Create simple R plots from matrix data || xQTL || Draft || RplotsModule || [[Image(rplot.png,100px)]] ||
    2926
    30 and/or handwritten code. Other plugins may extend other plugins to yet add more functionality. The first plugin where all others build upon is the MOLGENIS database core. The last plugin is typically a fully functional application that assembles other plugins
     27=== Analysis Pipelines ===
    3128
    32 TODO core architectural picture.
     29|| '''Name''' || '''Description''' || '''Applications''' || '''Status''' || '''Wiki page''' || '''Screenshot''' ||
     30|| !JobManager || Monitor and manage computational jobs || xQTL, NGS || Prototype || JobManagerModule || [[Image(jobman.png,100px)]] ||
    3331
    34 TODO directory structure
    35 
    36 TODO concept of extension points
    37 
    38 = Extension mechanism =
    39 Each building block may add framework code, model extensions, generators and/or models.  In this last case the module may regeneration against the last versions before it can be used (which we can do automatically?). After that we could use a osgi drop in mechanism to add the produce funtionality into a running system.
    40 
    41 Design 0: simply link to projects In this procedure the model files are puth on classpath and can be simply imported. The code has to be generated by the imported projects themselves. This is ideal for development (as long as re-generation is a one push on a button event and checks if generators are changed (hash or something)).
    42 
    43 Design 1: linking in code We imagine a procedure where when a user runs the generator (ant script, nothing else)
    44 
    45  * downloads the core molgenis and starts the model parser/generator
    46  * when <import="db" .../> is in your local model dependencies are automatically checked out from svn to /generated/java (incl libs!)
    47  * each of these dependencies is regenerated in dependency order
    48  * tests run to verify that the modules are sanitized (no compile errors, test cases work)
    49  * and then produced as osgi modules that are simply loaded by the current project during the rest of development
    50 
    51 And have a 'clean' action to redo the procedure.
    52 
    53 Design 2: downloading maven/osgi modules. We imagine a procedure where when a user runs the generator from maven (maven pom + molgenis core as plugin, nothing else)
    54 
    55  * downloads all depedencies using maven model (all MOLGENIS settings should be in this model as well to keep it simple?)
    56  * add generate tasks to maven to get all stuff done
    57 
    58 Complicating factor here is that the maven repos should be rebuild now and then.
    59 
    60 We could allow both design 1 and 2 in order to speed up development?
    61 
    62 i.e. organize maven compatible but still allow ant.xml type of interactions to do this low-tech. Or use maven in the back and don't necessarily bother the user with it.
    63 
    64 = Core modules =
    65 These are the foundational building blocks that can be reused between all MOLGENIS applications. Package org.molgenis.*
    66 
    67 Core:
    68 
    69  * [wiki:Plugins/OrgMolgenis core] - core platform of model (parsers), generators and util packages
    70  * [wiki:Plugins/OrgMolgenisDb db] - database interface framework on top of MySQL, PostgreSQL and HypersonicSQL (extends core)
    71  * [wiki:Plugins/OrgMolgenisExchange exchange] - file import/export mechanisms from and to csv, tab, excel, (xml), etc (extends db)
    72  * [wiki:Plugins/OrgMolgenisApi api] - application programming interfaces to Java, R, REST, SOAP, RDF, etc on top of Db (extends exchange)
    73  * [wiki:Plugins/OrgMolgenisUi ui] - web user interface framework with forms, menus and plugins (extends api)
    74  * [wiki:Plugins/OrgMolgenisCompute compute] - compute interface framework for short, long and parallel running computations (extends ui?)
    75 
    76 Discussion:  ui could be moved to depend on core only, and then have other packages extend on that to add particular UI components for db, compute, etc. Or have separate ui extensions like ui.db, ui.compute, etc.
    77 
    78 = Components =
    79 These are the semi-finished building blocks that can be shared between different applications to reuse common work. These components typically provide data model (modules) that need to be generated before a plugin is ready to use. Hence, there is a need to regenerate on the first installation of the plugin. package org.molgenis.*
    80 
    81 Cross cutting:
    82 
    83  * [wiki:Component/OrgXgapAuth auth] - secure authentication and sharing framework (extends ui)
    84  * [wiki:Component/OrgXgapVersionable versionable] - mechanism to have versioned data elements by simply extending an interface
    85 
    86 Discussion: need ability to weave this in later, like a special decorator that is applied everywhere? Or should we just make these core features and move it into db package?
    87 
    88 Generic:
    89 
    90  * [wiki:Component/OrgXgapOnto molgenisdata.onto] - for using ontologies including tree based browser, search box (extends ui)
    91  * [wiki:Component/OrgXgapMatrix molgenisdata.matrix] - for working with matrix like data having 'dimensionelements' and 'dataelements'
    92  * [wiki:Component/OrgXgapProtocol molgenisdata.protocol] - for track and tracing of materials/data, protocols, protocolapplications (compatible with MAGE-TAB SDRF)
    93  * [wiki:Component/OrgXgapTool tool] - for integrating R scripts onto the application (extends protocol, compute)
    94  * [wiki:Component/OrgXgapMaterialTracker tool] - for integrating R scripts onto the application (extends protocol)
    95 
    96 Domain:
    97 
    98  * [wiki:Component/OrgXgapLocus locus] - tools for managing and browsing genomic sequence annotations
    99  * [wiki:Component/OrgXgapPheno pheno] - for representing deep phenotypic data collected in clinics and the field (extends onto, tracker, matrix)
    100  * [wiki:Component/OrgXgapMutation muta] - for working with mutating data conforming to HGVS nomenclature (extends pheno)
    101 
    102 More ideas: pedigree, comments
    103 
    104 = Applications =
    105 Systems are ready to use and download. Ideally end users would be able to choose from these online.
    106 
    107  * [wiki:Apps/PhenoTrackerMolgenis PhenoTracker] - for storing phenotypics annotations on individuals and panels
    108  * [wiki:Apps/GwasMolgenis GwasPlatform] - for GWAS studies like in EU-GEN2PHN, LifeLines and BBMRI-NL
    109  * [wiki:Apps/QtlMolgenis QtlPlatform] - for QTL studies in model organisms like EU-Panacea
    110  * [wiki:Apps/MutationMolgenis MutationTracker] - for locus specific annotations (extends pheno, muta)
    111  * [wiki:Apps/ShortReadMolgenis SequenceArchive] - for storing, QC-ing and releasing short read data from NGS
    112  *
    113 
    114 = Ideas for more enhancements =
    115 == Core ==
    116  * [wiki:Plugins/OrgMolgenisUi/Theme ui.theme] - catalog of UI web themes for MOLGENIS applications the use can choose from (extends ui)
    117  * [wiki:Plugins/OrgMolgenisUi/Iphone ui.iphone] - iphone user interface (extends ui)
    118  * [wiki:Plugins/OrgMOlgenisDb/Versionable db.versionable] - to allow versionable data entry
    119  * [wiki:Plugins/OrgMolgenisApi/Observer api.observer] - mechanism to observe and log activity for tracking and tracing
    120  * [wiki:Plugins/OrgMolgenisApi/Feeds api.feed] - atom feeds + command in UI to observe data, queries and/or transactions (extends api, ui) ui
    121  * [wiki:Plugins/OrgMolgenisUi/EclipeR ui.gwt] - rich client application generator based on GWT