jhpoelen/eol-globi-data v0.11.0
2017
This maintenance release includes, but is not limited to:
support content-based caching of dataset resources to increase reproducibility and stability. Dataset resources include GloBI enabled github/zenodo repositories. Also, the resources include data files or web api responses that are used to extract species interaction data. Local directories are used to package all the source data related to a particular dataset such that a data import can be replayed at some later point in time without having to assume that the associated web servers are still on-line.
For example - a NOAA REEM dataset available at https://zenodo.org/record/229505/files/globalbioticinteractions/noaa-reem-v0.1.zip would result in a local directory that contains something like:
/datasets/globalbioticinteractions/noaa-reem/access.tsv
/datasets/globalbioticinteractions/noaa-reem/13d87c0609080b2fbf527a959fca260e9401f772fd8ee65952a487bce39712e1
/datasets/globalbioticinteractions/noaa-reem/2cf36a75c21fe96e5b73f67a581d21d0eb69c5beb5092c0a7820266b03448731
/datasets/globalbioticinteractions/noaa-reem/660ba0ddebf9c8e7dcd20a77ea3ce26d9a6e3a67a046336880f8de9e94899f5d
/datasets/globalbioticinteractions/noaa-reem/6bd280454f0e6ab2388e87a78e459cedc0e2af34dbfeb9184aa9c0a653950ae4
/datasets/globalbioticinteractions/noaa-reem/8825692b70bd6262bc4e43cb46e2711443047d16c6c4d8806c5e463f31881ee8
/datasets/globalbioticinteractions/noaa-reem/d6cb297d8066ef57088b8fcc793203ad05ec434e7255e6e201fcb90cc6352270
Where the access.tsv file contains the relations between the web uris, associated cache filenames (e.g., d6cb297d...) and access time/date. The associated local files names are generated from the content of the resources using SHA2/SHA256.
dataset name
resource uri
local filename/content-hash
accessed at
type
globalbioticinteractions/noaa-reem
https://zenodo.org/record/229505/files/globalbioticinteractions/noaa-reem-v0.1.zip
2017-09-16T05:57:20Z
application/globi
globalbioticinteractions/noaa-reem
https://zenodo.org/record/229505/files/globalbioticinteractions/noaa-reem-v0.1.zip
13d87c0609080b2fbf527a959fca260e9401f772fd8ee65952a487bce39712e1
2017-09-16T05:57:20Z
...
...
...
...
...
globalbioticinteractions/noaa-reem
http://access.afsc.noaa.gov/REEM/WebDietData/showdata5.php?NODC=1&Region=BS
8825692b70bd6262bc4e43cb46e2711443047d16c6c4d8806c5e463f31881ee8
2017-09-16T05:58:28Z
globalbioticinteractions/noaa-reem
http://access.afsc.noaa.gov/REEM/WebDietData/showdata5.php?NODC=1&Region=GOA
2cf36a75c21fe96e5b73f67a581d21d0eb69c5beb5092c0a7820266b03448731
2017-09-16T05:58:55Z
Keywords:
- Correction
- Source
- Cite
- Save
- Machine Reading By IdeaReader
0
References
0
Citations
NaN
KQI