HISCOM ad hoc AVH Workshop

Venue
National Herbarium of Victoria, South Yarra, Melbourne, 27 November 2008

Purpose
Followup to CHAH/HISCOM Workshop: Future of the AVH to give effect to the outcomes of this joint CHAH/HISCOM workshop.

Attendees
Bill Barker (AD); Peter Bostock (BRI); Alex Chapman (PERTH); Paul Coddington (eResearch SA); Jim Croft (CANB); Niels Klazenga (MEL); Ben Richardson (PERTH); Helen Thompson (ABRS); Alison Vaughan (MEL); Greg Whitbread (CANB); Karen Wilson (NSW); Aaron Wilton (CHR).

Apologies
Donna Lewis (DNA); Brett Summerell (NSW); Laurence Paine (HO).

Background papers

 * AVH Briefing Paper November 2008
 * CHAH/HISCOM Workshop: Future of the AVH
 * VAH 1999 Strategic Plan

Summary of CHAH/HISCOM workshop recommendations
Jim provided a summary of the CHAH/HISCOM workshop for the benefit of the attendees who weren't at yesterday's meeting. See the summary section of the minutes for more information.

How is HISCOM going to effect the dynamic load of specimen data from each participating herbarium?

 * PERTH - end of year
 * AD - already delivering (& node)
 * NSW - March '09
 * CANB - end of year (& node?)
 * MEL - already delivering (& node). Moving to Specify in the next 18 months, and may need assistance with BioCASE installation once that's up and running.
 * BRI - end of March '09
 * DNA - Donna says: "Hopefully all going OK, data will be delivered dynamically to AVH sometime in Feb."
 * HO - timeline unknown. Dependant upon the delivery of the new Collection Management System (due Sep 08, timeline slipped by 6 months minimum). No expected financial issues. No expected technical issues. Dynamic delivery on the current Filemaker Pro system is NOT feasible (too many processes that can't be automated).
 * NZ - middle of 2009 for NZVH

Other issues

 * BRI, PERTH and CANB could use some funding to assist with dynamic delivery.
 * Need a summary from eResearch SA of the data that is not being delivered in the right format. (See action from the October HISCOM AGM).
 * Discussion about the need for having multiple nodes for the AVH - minutes from 26/11/2008 say that CHAH want as many nodes as possible. HISCOM's view is that we need at least two nodes to keep it working properly.
 * Install package for AVH site is on SourceForge
 * Synchronisation for user accounts still needs to be resolved
 * Discussion of the need for an allocation of funding for AVH maintenance (as opposed to AVH development)
 * Paul highlighted the need for new providers to have their BioCASE providers tested so that debugging can be done before data is delivered to eResearch SA.
 * It was recognised at the Perth HISCOM meeting that we should be providing static dumps relatively frequently, so data is current and so that the EWS can function effectively.

ACTION: Herbaria who are not delivering dynamically to get an estimate of the cost of getting this done asap and populate the AVH Work Plans page

ACTION: Herbaria who are not delivering dynamically to provide a new data file to eResearch SA, including all Australasian records, prior to the AVH3.0 release date (date TBA) (HO, DNA, CANB, NSW, PERTH)

Data limits
Both soft and hard limits are configurable.

Soft limits are effectively warnings that they are approaching a limit of the system (more or less). Hard limits are system- and network- imposed limits beyond which the system or the network will not always work properly.


 * Technical limits based on browser, server, database capability
 * Data file generation can take a long time
 * Download may take a long time because the data file is so large
 * Hard coded limits based on user privilege

User Registration
There are currently two basic levels of usage; public and registered use. The admin account includes privileges to add another provider, set download limits and give other users the ability to verify access for new users. There is currently only one admin account, but more than one person can use that account. Once a user has been approved (or otherwise), an email is automatically sent to the user to advise them. Public access users have to accept the terms and conditions of the data use agreement prior to downloading data. For registered users, the data use agreement needs to be written in to the registration conditions. CHAH indicated the desire to vet registration requests for advanced access to AVH. A standard email request to CHAH reps. was mooted. In order to expedite the approval process, it is likely CHAH reps will require certain information about the registrants in order to establish their bona fides.

Here are a couple of examples of registration forms for similar sites which illustrate the types of fields found useful:


 * FloraBase - the Western Australian Flora - access to specimen records
 * NatureMap - Mapping Western Australia's biodiversity - advanced mapping of specimens

Usernames are currently determined by an authorised user, not by the user requesting the account. This really needs to change.

ACTION: Prepare a list of issues that CHAH need to consider in relation to registration (Alison), e.g.


 * suggestions for registration criteria
 * consider allowing users to choose their own username and password
 * examples of other sites with good registration processes
 * whether registered users need to explain their intended use of the data prior to each download
 * how long registrations should be active for
 * have log file display the user's actual name, to make review of log files easier

EWS Weed Tracker
Rex and Bill evaluating the site over the next few days and then it will be handed over to HISCOM for further assessment. Licensing and registration issues will take some time to sort out. See Bill if you want the username and password. It was suggested that the weed group (a rep from each herbarium) is involved in testing the Weed Tracker along with HISCOM. Beta-testing of the web application is required.

AVH Mapping

 * Where have various map layers gone? (Each instance of AVH2 customised available layers).
 * Paul requests additional layers if necessary
 * Greg suggests many free shape files from GeoScience Australia
 * Searching by polygons
 * realistically this is AVH4, according to Paul
 * a feature request for ALA
 * or feed the data into BioMapper

ACTION: Project manage the choice of replacement and new layers and the default base map for AVH using Geoscience Australia as a source. Add candidate images to the wiki for discussion. (Peter Bostock.)

AVH Usage Statistics
A revised AVH requirement is to keep track of how many records are served per herbarium, to which users and potentially, which taxa. An interface is needed to provide simple access to this information.

The AVH 3 server currently captures the statistics, perhaps with the exception of the taxa involved in the query (but this may not be difficult to change). However, it does not yet provide an interface to the statistics.

Aaron: Are IP addresses being logged? Paul: Perhaps, but he will check.

Currently includes:


 * Username
 * Timestamp
 * Number of records per herbarium
 * Query

Needs to be extended to include:


 * IP Address
 * Taxon (perhaps)

Country, State, Region fixes
This issue was discussed at some length at the recent AGM, and was not discussed again.

Some locations, such as Norfolk Island have been variously placed in the Country, State, or Region fields of participant herbarium databases. This causes a problem in the AVH interface such that a query for "Norfolk Island" in the Region field returns only those records that have the value in the Region field, which is not what the user would expect. The user would expect to retrieve the records collected on Norfolk Island regardless of the field that this value was placed.

The preferred way to correct this situation is to correct the source database, if the data is incorrectly databased, or the BioCASE provider if the source database is considered correct. However, it is additionally possible to correct the problem by making the AVH query engine search multiple places for values such as this.

A separate problem exists with querying the country name field. If a specimen was collected in a country whose name has changed, the old name will commonly remain in the database. However users of AVH will commonly request the name as it is now known. This requires a set of "synonyms" for country names in AVH so that the effect of a query matches the user's expectation, that is, all variations of the name for that country are included in the matched records when one of the "synonym" names for that country are requested by the user.

ACTION: Subcommittee chaired by Alison that will determine and implement the preferred course of action.

Timeline and Requirements for a Public AVH Launch

 * Look and Feel &mdash; end of year
 * Licensing &mdash; end of year
 * Registration &mdash; end of year
 * Help Page updates: higher classification in use in each herbarium; accession number suffixes. HO and NSW to update the documentation at AVH Restricted Help Page.

Wiki Help Page Modification API
Page headings on AVH Public Help Page and AVH Restricted Help Page should not be changed, as these are required for AVH 3 to map to the correct section of the AVH Help. Introducing new headings is OK, because the current AVH doesn't link to these.

ACTION: Paul and Alison to draft guidelines for how to edit the AVH help pages to ensure there is no breakage of the production AVH server(s).

Curatorial area
After AVH 3.0 has been implemented, need herbaria to work towards delivering curation fields so we can get the curation tools functioning.

Cache Management Enhancement
On occasion it will be necessary to remove a record from the AVH cache because it is clearly in error, or it has been removed from the source herbarium database. How do we remove a record from the cache? This is a future enhancement to AVH, and may be reliant upon an update to the BioCASE and/or TAPIR protocols.

It may be a security issue to permit delete requests from a web service, which may be why no such request method exists.

API issues for GET query
An item from the CHAH/HISCOM meeting relates to the ability for external users to ask for a page from AVH with many of the form elements pre-filled, for example a map of a taxon of interest. See the PlantNet page for Eucalyptus acaciiformis, for example, which uses a submit button at the bottom of the page to create a post to the AVH node at RBG Victoria. This button uses a POST query rather than a GET query, which may be necessary for some very long requests. It should remain possible to do this in AVH 3, but it is currently unknown how to do this. It will certainly be different from the URL used in AVH 2.

AVH 3 could construct a link that equates to the query requested and present this link at the bottom of the map page so it is as easy as possible to link to an AVH map, which is the most commonly requested item within this feature request.

The doGet and doPost methods in class avhServlet which receive GET and POST requests from the HTTP server respectively, use the same method, processRequest, to do their work. All tasks accomplished by this servlet thus have the same functionality regardless of whether they are called using the HTTP GET or POST request methods. It should be relatively easy to accomplish GET query access to AVH so that external sites can easily bookmark AVH, or embed portions of AVH, for example the map.

ACTION: Paul, Ben and Shunde to develop a specification for using a GET operation to deliver pages from the AVH. A simple enhancement will be to provide a bookmarkable URL of the map page in the footer of the map page.

Disambiguator
Protracted discussion that attempted to disambiguate the disambiguator. ALA is planning on developing one, but it is unclear where it will sit in relation to the ALA/APC. Further discussion should be conducted on the AVH Taxonomic Disambiguation page.

ACTION: Put up a project outline for AVH requirements for a disambiguator on the wiki (Helen and Greg)

ACTION: As requested at the Melbourne meeting, a project skeleton for a funding proposal for ALA relating to AVH is with Greg.

Greg will be discussing the technical side with Donald and will then complete the technical part of the proposal which will then be loaded to the Hiscom Wiki for your comment. Also requested were the [[Media:Atlas Participation Agreement final - (CSIRO 16 Aug)- clean - 2.pdf|ALA Participation Agreement]] (signed by CHAH): the unsigned pages are attached here but I can scan the signatures if required. Also attached is the proposal submitted to the ALA for funding for the current contract to [[Media:ALA Name Services 08 09 funding proposal.pdf|develop names services for ALA]] using the Australian Plant Census and the Australian Faunal Directory. Hthompso 15:51, 5 December 2008 (EST)

Dynamic delivery of data
The AVH Work Plans document will form the basis of an application to ALA for support for getting all herbaria delivering dynamically.

Flora/species' profiles
ACTION: HISCOM members to update the Online Flora Information Systems wiki page (All)

Strategic Plan
See VAH 1999 Strategic Plan for history.

See AVH 2009 Strategic Plan - work in progress.

ACTION: HISCOM to add content to the new Strategic Plan by end of December 2008 (All)

CHAH-HISCOM Governance
The joint CHAH/HISCOM meeting was very successful. In view of this, HISCOM would like to change the structure of the annual meetings. A good option would be to have a two day CHAH meeting followed by a one day joint meeting, followed by a one or two day HISCOM meeting.

Recommendation: That CHAH and HISCOM have an annual joint meeting.

Actions

 * Summarise limitations and caveats of AVH data
 * Scope and specifications for getting all herbaria providing data dynamically
 * Prototype on-line flora query systems
 * Scope taxonomic disambiguation processes
 * Draft a policy and procedure for user registration on wiki for discussion

Links

 * AVH development site public interface
 * AVH development site restricted interface
 * AVH Briefing Paper November 2008
 * CHAH/HISCOM Workshop: Future of the AVH, Melbourne November 2008