language-icon Old Web
English
Sign In

jhpoelen/eol-globi-data v0.11.0

2017 
This maintenance release includes, but is not limited to: support content-based caching of dataset resources to increase reproducibility and stability. Dataset resources include GloBI enabled github/zenodo repositories. Also, the resources include data files or web api responses that are used to extract species interaction data. Local directories are used to package all the source data related to a particular dataset such that a data import can be replayed at some later point in time without having to assume that the associated web servers are still on-line. For example - a NOAA REEM dataset available at https://zenodo.org/record/229505/files/globalbioticinteractions/noaa-reem-v0.1.zip would result in a local directory that contains something like: /datasets/globalbioticinteractions/noaa-reem/access.tsv /datasets/globalbioticinteractions/noaa-reem/13d87c0609080b2fbf527a959fca260e9401f772fd8ee65952a487bce39712e1 /datasets/globalbioticinteractions/noaa-reem/2cf36a75c21fe96e5b73f67a581d21d0eb69c5beb5092c0a7820266b03448731 /datasets/globalbioticinteractions/noaa-reem/660ba0ddebf9c8e7dcd20a77ea3ce26d9a6e3a67a046336880f8de9e94899f5d /datasets/globalbioticinteractions/noaa-reem/6bd280454f0e6ab2388e87a78e459cedc0e2af34dbfeb9184aa9c0a653950ae4 /datasets/globalbioticinteractions/noaa-reem/8825692b70bd6262bc4e43cb46e2711443047d16c6c4d8806c5e463f31881ee8 /datasets/globalbioticinteractions/noaa-reem/d6cb297d8066ef57088b8fcc793203ad05ec434e7255e6e201fcb90cc6352270 Where the access.tsv file contains the relations between the web uris, associated cache filenames (e.g., d6cb297d...) and access time/date. The associated local files names are generated from the content of the resources using SHA2/SHA256. dataset name resource uri local filename/content-hash accessed at type globalbioticinteractions/noaa-reem https://zenodo.org/record/229505/files/globalbioticinteractions/noaa-reem-v0.1.zip   2017-09-16T05:57:20Z application/globi globalbioticinteractions/noaa-reem https://zenodo.org/record/229505/files/globalbioticinteractions/noaa-reem-v0.1.zip 13d87c0609080b2fbf527a959fca260e9401f772fd8ee65952a487bce39712e1 2017-09-16T05:57:20Z ... ... ... ... ... globalbioticinteractions/noaa-reem http://access.afsc.noaa.gov/REEM/WebDietData/showdata5.php?NODC=1&Region=BS 8825692b70bd6262bc4e43cb46e2711443047d16c6c4d8806c5e463f31881ee8 2017-09-16T05:58:28Z   globalbioticinteractions/noaa-reem http://access.afsc.noaa.gov/REEM/WebDietData/showdata5.php?NODC=1&Region=GOA 2cf36a75c21fe96e5b73f67a581d21d0eb69c5beb5092c0a7820266b03448731 2017-09-16T05:58:55Z
    • Correction
    • Source
    • Cite
    • Save
    • Machine Reading By IdeaReader
    0
    References
    0
    Citations
    NaN
    KQI
    []