Physical TC Meeting June 2016

Jump to: navigation, search
Physical TC Meeting June 2016
Title Physical TC Meeting 2016
Location Berlin
Date Start 2016/06/13
Date End 2016/06/17
Tags
Description


Agenda

The scheduled sessions start Tuesday morning (10:00) and end at Friday noon (11:00).

Tuesday

  • 09.00-10.00 Coffee
  • 10.00-10.15 Welcome (Fredrik)
  • 10.15-10.45 Progress report (Markus S)
  • 10.45-12.00 Workshop agenda, presentation of participants
  • 12.00-13.00 Lunch
  • 13.00-14.00 REST API and other guideline documents (Markus S)
  • 14.00-16.00 Breakout groups (test data and data model; guidelines)
  • 16.00-17.00 Reports from breakout groups

Wednesday

  • 09.00-09.30 Collection manager UI demo (Calle)
  • 09.30-10.00 Taxonomy editor UI sketches (Fredrik)
  • 10.00-10.15 System integration (Markus S)
  • 10.15-10.30 Coffee
  • 10.30-11.00 System integration (continued)
  • 11.00-11.30 Persistent identifiers (Falko)
  • 11.30-12.00 Data wranglers (James, Markus S)
  • 12.00-13.00 Lunch
  • 13.00-16.00 Breakout groups
  • 16.00-17.00 Reports from breakout groups

Thursday

  • 09.00-09.30 Security (Glen)
  • 09.30-10.15 Annotations and data wrangling (Paul)
  • 10.15-10.30 Coffee
  • 10.30-10.45 Roles and permissions (Markus E)
  • 10.45-11.00 QA toolbox (Markus S)
  • 11.00-12.00 Breakout groups
  • 12.00-12:15 Taking a group photo  ;)
  • 12.15-13.00 Lunch
  • 13.00-15.00 Road map
  • 15.00-16.00 DINA ISC meeting (some)
  • 15.00-17.00 Breakout groups

Friday

  • 09.00-10.00 Breakout groups
  • 10.00-10.15 Coffee
  • 10.15-10.45 Concluding discussion
  • 10.50-11.45 Tour of the museum

Remote audio/video links will be available by Adobe Connect or equivalent tools. E-mail Falko for details if you are interested.

Potential themes for breakout groups:

  • Overall Road Map / The Current State of DINA - overview of progress in the DINA-Web system and its individual components
  • Collaboration efforts, communications infrastructure, PR - discuss if/which changes or improvements are needed
  • Guidlines documents, technical requirements, security requirements, establishing offical document versions etc:
    1. Security requirements, OWASP - Session coordinated by Glen Newton
    2. REST-APIs (currently Media, Collections, Classifications) and future steps - Session coordinated by Markus Skyttner
    3. Technical roadmap: refinw Markus S's "State of the Union" tables, or consider alternatives?
    4. QA - How do we best load test and performance various components?
    5. DINA v1.0 Release discussion: technical, communication, etc issues- Session suggested by Glen Newton
  • Test data, updating the information model etc:
    1. Procedures for working with test data and iteratively evolving the data model - Session coordinated by Markus Englund
    2. Information model and evolving the Collections REST API - Transition path from the Specify 6/7 schema (ie dw-collections-api v.0 ) to DINA-Web Information model - version 1.0 (based on the dw-cco embryo - ie the "Complex Collections Objects" proof of concept) - including discussing storage backens (relational and non-relational) - Session coordinated by James Macklin and Paul Morris
    3. Road map for work on the data model
  • Using annotations in DINA-Web - Session coordinated by Paul Morris
  • Data Wrangler's Toolbox - using Kurator, dw-cli-tools, mirroreum and other general purpose tools for migrating data and improving data quality - Session coordinated by Paul Morris
  • Persistent Identifiers - Session coordinated by Falko G
  • Changing the web user interfaces - DINA Collection Manager UI currently and upcoming phases, especially integrating dw-classifications - the PlutoF Taxonomy Module component - into the user interface - Session coordinated by Carl Claesson
  • Systems integration:
    1. KeyCloak: experiences and how to import/export user data - Session coordinated by Ida Li
    2. Providing SSO/authentication in dockerized versions of DDT, SeqDB and Pluto-F taxonomy module with KeyCloak - Session coordinated by Markus S
    3. Data integration discussion: inter-module issues around data models. etc- Session recorded by Glen Newton
  • System level services:
    1. Status overview with ELK-stack
    2. Monitor services using zabbix or equivalent

Notes

2016-06-14 Evolving the data model using example and test data

We discussed the process of evolving the data model using example data and test data, and at the same time evolving migration tools. Example data are modified and well documented data sets, illustrating the range of information models, data types and use cases covered by the current collection databases in DINA institutions (Specify 6 databases and other databases). Test data are example data sets that have been processed so that they conform to the recommended DINA information model. The processing will include the selection of data items in the example data sets that are covered by the DINA data model, and how they will map to that model. It will also include procedures for storing the injected original data, including the data not covered by the DINA data model.

Both types of data sets will evolve over time in conjunction with the revisions of the DINA data model and subsequent releases of the DINA collection manager. Eventually, when DINA-Web is released, the route from source data over example data to test data will be the same as the migration to the production version of DINA-Web. Example data and test data are stored and documented in the DINA repository on github.

These ideas are described in more detail in the
example data and testdata presentation
.

2016-06-15 Morning presentations

Collection manager UI demo. E-mail Carl Claesson for password.

2016-06-15 Systems Integration presentations and discussion

  • Demo from Thomas of Funghi Database - an example of external reference taxonomy support
  • Discussions about implementation ideas for taxonomy support in the Collections Manager UI - using dw-collections, using DarwinCoreArchive format but filtering to not include everything and use a matching step to pull data (if caching is needed) just for the relevant used collection objects
  • Discussions about search api cache updates (pushing diffs as changes happen without affecting performace for APIs)

Updated [https://github.com/DINA-Web/guidelines Guidelines) and improvement of API Guidelines (support batch operations and PATCH etc)

Updated [https://github.com/DINA-Web/dw-cco) with Makefile and using the latest mariadb engine for storing data.

2016-06-15 Data model group

In the Copenhagen workshop in November last year, we tried to approach the task of defining the first version of the DINA-Web collection data model (dwCDM) by first removing the tables and fields that we were not using from the Specify data model, and then introducing the changes we think are necessary. However, we quickly found out that this requires a lot of effort, and we did not get very far.

The situation is quite different now. We have a liquibase implementation of the complex collection object model (CCO model; see github repo here), which introduces some important changes to the core of the Specify data model. We also have a beta version of the core of the collection manager user interface. Finally, we have a module map (see above) describing how we will separate out satellite data entities from the core collection data items in the first version of DINA-Web.

Given these recent advances, we decided to reverse our approach to the first version of the dwCDM. Starting from the CCO model, we will add in the Specify tables and fields used by the current version of the collection manager, taking into account the data items that will be handled by external modules in DINA-Web. This will result in a minimal collection data model that we can subsequently grow as needed by bringing in additional components from the Specify data model or by adding entirely new components.

During the afternoon session, Paul produced a diagram covering the tables that will be needed in dwCDM, as we think of it now. You can find the diagram and the source file (DIA) in the information-model github repo. We will continue working with dwCDM tomorrow and on Friday, with the goal of having a documented liquibase version of it finished before the end of the workshop, if possible.

2016-06-16 Annotations/Data Quality

Selected IRIs relevant to the presentation:

FP-Akka: Java application for data quality control of biodiversity data:

Kurator Project: Producing data quality assurance/control tools for biodiversity data:

FilteredPush Project: http://wiki.filteredpush.org/wiki/

W3C:

2016-06-16 User roles and permissions

Brief descriptions of proposed user roles. Prepared for the TC-meeting in Copenhagen, November 2015.
User roles – Presentation in Berlin, June 2016

Example Roles in MCZbase/Arctos (access to data on institution/department level managed by virtual private databases): See also: http://mczbase.mcz.harvard.edu/

  • public
  • approved user
  • data entry
  • manage taxonomy
  • manage media
  • manage publications
  • manage agents
  • manage geography
  • manage controled vocabularies
  • manage transactions
  • administer transactions

2016-06-16 Road Map Discussion

The road map for the DINA-Web development tasks leading up to the release of version 1.0 in January 2018 was worked on in breakout groups, using a Google document. Five people (James, Glen, Markus S, Markus E, Falko) were elected to process the material in the Google document into a coherent road map that will be uploaded to and maintained in the DINA project Redmine instance. The group would also look into Redmine plugins that provide better support for agile development than the vanilla version of Redmine. Their results will be presented at future TC and ISC meetings.

Practical meeting information

Logistics

Arrival at Berlin Tegel Airport (TXL)

  • Take the public bus TXL (Bus Direction: S+U Alexanderplatz)
  • The bus leaves every 5-10 minutes
  • Leave the bus at the station "Invalidenpark"
  • From "Invalidenpark" walk 250m along the road in direction as the bus you left, then you can't fail the museum
  • Travelling time: approx. 25 minutes
  • For this tour you need a ticket valid for in the zones A and B
    • single ticket AB: 2.70 € (it is valid for 120 minutes after punching it)
    • day ticket AB: 7.00 € (valid for the whole day until 3 am)
  • You need to buy your ticket at the ticket machine (or from the bus driver, but as it is very crowded it's not recommended)
  • You need to show your ticket to the bus driver and punch it in the bus if you bought a single tour ticket at the ticket machine

Arrival at Berlin Schönefeld Airport (SXF) or Berlin Brandenburg Airport (BER)

Option 1

  • Take the regional train RE14 (Direction: Nauen)
  • The train leaves every hour
  • Leave the train at the Berlin Central Station (Hauptbahnhof)
  • Take the tram M5, M8 or M10 and leave at the station "Invalidenpark" (or just take a walk, instead of the tram. It's not that far - just two stations)
  • The tram station is outside the Central Station (leave the station at the exit "Europaplatz" then you will see the tram station)
  • From "Invalidenpark" walk 250m along the road in direction as the train you left, then you can't fail the museum
  • Travelling time: approx. 60 minutes
  • For this tour you need a ticket valid for in the zones A, B and C
    • single ticket ABC: 3.30 € (it is valid for 120 minutes after punching it)
    • day ticket ABC: 7.60 € (valid for the whole day until 3 am)
  • You need to buy your ticket at the ticket machine. It is NOT possible to buy a ticket in the trains!
  • You need to punch the ticket at the station before you enter the train if you bought a single tour ticket at the ticket machine.
  • Travelling without a ticket or with an invalid ticket (if you forgot to punch it) cost a fee of 40 €!

Option 2

  • Take the city train (S-Bahn) S9 (Direction: Berlin-Pankow)
  • The train leaves two times per hour
  • Leave at the station "Ostkreuz"
  • Take one of the city trains (S-Bahn) S5 (Direction: Berlin-Spandau), S7 (Direction: Potsdam) or S75 (Direction: Westkreuz)
  • Leave the train at the Berlin Central Station (Hauptbahnhof)
  • The tram station is outside the Central Station (leave the station at the exit "Europaplatz" then you will see the tram station)
  • Take the tram M5, M8 or M10 and leave at the station "Invalidenpark" (or just take a walk, instead of the tram. It's not that far - just two stations)
  • From "Invalidenpark" walk 250m along the road in direction as the train you left, then you can't fail the museum
  • Travelling time: approx. 75 minutes
  • For this tour you need a ticket valid for in the zones A, B and C
    • single ticket ABC: 3.30 € (it is valid for 120 minutes after punching it)
    • day ticket ABC: 7.60 € (valid for the whole day until 3 am)
  • You need to buy your ticket at the ticket machine. It is NOT possible to buy a ticket in the trains!
  • You need to punch the ticket at the station before you enter the train if you bought a single tour ticket at the ticket machine.
  • Travelling without a ticket or with an invalid ticket (if you forgot to punch it) cost a fee of 40 €!

For cross-checking the time tables or other arrival


At the Museum für Naturkunde

  • Do not enter the museum via the quite obvious main entrance! It will be more comfortable to meet at the staff entrance.
  • Because of a construction site you have to enter the museum area via the driveway next to the Mercure Hotel. People who booked at Mecure Hotel are lucky  ;)
  • To find the staff entrance see the site plan (number 5 on the plan)
  • Every morning you will have to register at the porter when you entered the staff entrance. You will get a badge each day.
  • When you got your badge you can go to the conference room, which is in a different part of the building: Go to the entrance "Administration" (number 7 on the site plan). Upstairs 2nd floor room 3221.
  • But no worries: At the first day Falko will pick you up at 8.45 am at the staff entrance  ;)

Accommodation

  • MfN reserved 20 rooms for 13th -17th June in the Hotel Mecure (100m from the museum).
  • There is a discount for these rooms (99€ incl. breakfast, instead of 109€ per single room / night).
  • The participants have to book themselves with a discount code until May 17th for getting the discount! (see Falko's email from May 3rd)
  • you need to book via phone or email in order to get the discount rate. By booking via the webform on the hotel's website you will only get the normal rate
  • Important:
    1. If you arrive later than 6 pm CEST, please mention in the booking process!
    2. If you booked a room and for some reason you will not be able to come, please cancel the room at the hotel and tell Falko, otherwise the MfN will be charged for the room :)
  • Hotel details:
Mercure Hotel Berlin City
http://www.mercure.com/gb/hotel-5341-mercure-hotel-berlin-city/index.shtml
Invalidenstrasse 38
10115  BERLIN - GERMANY 
Tel : (+49)30/308260
Fax : (+49)30/30826100 
Mail : h5341@accor.com 
  • If you find a cheaper accommodation, feel free to book somewhere else  ;)
  • Hotel booked by the team travelling from Sweden:
Mikon Eastgate Hotel - City Centre
Hessische Str. 10
Mitte
10115 Berlin
Germany
Telefon: +49 30 40054090
[Map: http://www.booking.com/directions.sv.html?hotel_id=72183]

The team from SE will arrive at TXL / Tegel on Monday, 13. juni 2016, *16:30* Departure: Friday 17th of June, from Tegel /TXL at 17:00


Dinner

Tuesday

Wednesday

  • Restaurant "Nolle"
  • 7 pm
  • We will leave at 6:30 at Hotel Mercure
  • Address: Georgenstraße, 10117 Berlin

See on Google Maps


Thursday

  • Restaurant "Hans im Glück"
  • 7 pm
  • We leave at 6:40 pm at Hotel Mecure
  • Address: Invalidenstraße 53, 10557 Berlin

See location on Google Maps

Participants

  • Ida Li (Sweden)
  • Ingimar Erlingsson (Sweden)
  • Carl Claesson (Sweden)
  • Fredrik Ronquist (Sweden)
  • Markus Skyttner (Sweden)
  • Markus Englund (Sweden)
  • Niclas Gyllenstrand (Sweden)
  • Allan Zirk (Estonia)
  • Thomas Stjernegaard Jeppesen (Denmark)
  • James Macklin (Canada)
  • Satpal Bilkhu (Canada)
  • Glen Newton (Canada)
  • Falko Gloeckler (Germany)
  • Guenther Korb (Germany)
  • Jana Hoffmann (Germany)
  • Thomas Pfuhl (Germany)
  • Paul Morris (USA)

This page was last modified on 17 June 2016, at 00:04. Content is available under Attribution-Share Alike Non-commercial 2.5 or later, Unported unless otherwise noted.