Les données obtenues via le GBIF

Slide 1 - How is data shared on GBIF.org?

One thing to be mindful of is that data publishers on GBIF own and maintain their data, and then they share that with GBIF.

Slide 2 - Data publishing

So the way it works is that we have a data set that is owned by an organization. That organization needs to be endorsed by a GBIF node, that could be a country node or any sort of participant node, to be able to share data with GBIF.

That data then needs to be hosted somewhere in an installation, for example, the integrated publishing toolkit provided by GBIF.

So the data sets are hosted there, and then they can be published to GBIF and shared with you.

Slide 3 - Data shared on GBIF is heterogeneous

Another thing to be mindful of is that data on GBIF is quite heterogeneous. These organizations that share data with GBIF come from quite various sources, and it’s a community-endorsed process, as I had mentioned, where we have the nodes endorsing the publishing organizations, and that they can be anything from universities, museums, governmental agencies, et cetera.

So what they share can come from research projects or monitoring efforts, citizen science data or collections.

That is also something to be mindful of, that data is not live on GBIF.

You can have data that is a part of a mobilized effort of a collection, museum collection that might be digitizing their data from 50 years ago, 70 years ago, and the next day, it’s available on GBIF.

Slide 4 - Accessing and downloading data

So just to show you how such an aggregated search can look for you as a user, if you go and you want to look at a species, in this case, I’m showing you the orange tip butterfly. So if you, as most of our users, go into our portal and search for the species, the first thing you would encounter is the species page.

That will show you an overall description of the species and the GBIF occurrence distribution. But you would usually go to the occurrence record page, where you would apply further filters.

In this case, I’m saying I only want records that are from 2020 to 2022, I think. And I also say that it has to be in the continent of Europe.

So I’ve narrowed down my search and I click download, where I have to choose different options. In this case, I’m choosing the simple option.

But you can also see other information, such as known issues for the data, which might be something you should look into if you want to apply further filters for your download, if you don’t want that data included.

So in your GBIF user profile, you can see all the downloads you ever created.

In this case, I’m showing you the one I just created, the filters. But you will also see that 927 data sets contributed to this aggregated search.

So quite a varied source contributed with data. This matters because that has an impact of what will actually be in the data you download.

Slide 5 - Reviewing a download

So this is an example of that simple download I just created, where in the simple download, you have less fields available, less columns. But if you did the full Darwin Core archive download, you would have many fields to search from. But you will notice that there’s some of these fields that appear to be empty, where we don’t have any values. But as you can see, as I’m showing you now, you do actually do have some locality information shared by some publishers.

So that means that some publishers might be sharing data while others don’t. And that can matter for your analysis.

So for example, individual count, which can give you an idea about the abundance of the species, might not necessarily be filled out by some publishers, but is filled out by others.

And as I mentioned, if you have the full Darwin Core archive download, where you have many more columns, you will see a much more patchy data set, which matters for how you post-process your data.

Slide 6 - To which extent is there content in the fields you download?

If we go to a little more into detail of which content you can expect from the data you download on GBIF, I just give you some general ideas of some of the fields that you could download.

The GBIF ID, which we apply to the record that is shared with GBIF, is always 100%. You always get all the GBIF IDs for your data. But if you are looking for habitat information, if you look across all occurrence records on GBIF, only around 7% have data in that field.

Individual counts have quite high numbers, as well as state and province, but be mindful that this is because of birds. The biggest group of data we have on GBIF is coming from birds, so they really up how much data we actually have available. So it can be quite group specific.

So if you’re looking for insects, you might find that the percentage of values that are actually covered is quite low.

With scientific name, it’s pretty good.

For the depth, if you’re looking into marine occurrences, very few publishers actually share that data.

The same for elevation, and life stage is only 9%.

Most of our publishers do provide some sort of temporal data, but that could be just in the form of year.

So just be mindful of that when you use your data.

Slide 7 - Data standards

So I’m going to talk a bit about the data standards that are behind what you see on GBIF. As you noticed, the columns have quite specific names. These names and fields are defined by data standards, which are maintained by a global biodiversity community.

Slide 8 - Darwin Core Archive (DwC-A)

So most of the data shared with GBIF is usually within a Darwin Core archive, as I also mentioned earlier. And the publishers sharing data with GBIF have to map their data to this Darwin Core standard.

It’s maintained by the Biodiversity Information Standards, also called TDWG.

And those data standards define the field name, as I mentioned, and they also sometimes define the value content that could be like a controlled vocabulary or some sort of normalized value that people should use when they share their data.

So let’s say you’re a publisher, you’re completely new to data standards and publishing on GBIF. You might have a spreadsheet saying that you have a name, you have a count of how many of that species you saw, and some general notes about what you found.

Those fields have to be called scientific name for the name, for example, individual count for the counts, and for the notes, you might want to put those in occurrence remarks or event remarks.

The point is that you have specific fields to capture specific information.

Slide 9 - Available fields

Now, the fields available for sharing on GBIF, it’s dependent on the dataset class.

As I mentioned, you have occurrence records. Those occurrence records is one type of dataset class on GBIF.

So the most simple one is the metadata-only dataset, which doesn’t really have any data. It’s just a description of the dataset. So that could be a description of a collection or description of a field study, but where you haven’t been able to digitize the data just yet.

Then we also have checklist datasets, which is a list of taxon names. There could be more information included, but generally, it’s sharing scientific names of a given area or within a different specific scope.

And then you have occurrences, which I mentioned before, which is the evidence of a species present or a taxon present at a given date and place.

Now, the last type of dataset class we have is the sampling events, which has a little more information because usually people have to share sampling effort and protocol, and it can allow users to assess community composition and abundance of species.

Slide 10 - Dataset cores

Now, those dataset classes are core tables. So if you have a dataset table that is either a taxon, occurrence, or event table those tables have a specific set of fields you can share. But you might want to share even more information.

For example, if you have an event dataset, you might want to share individual occurrences.

You did a BioBlitz from start of Saturday to the end of Saturday, but at that range, you found that butterfly or that beetle at a specific time during that.

That’s an occurrence within the event of the BioBlitz.

Or you might have media associated. You took pictures of the different insects you found during that time, or you might even have taken samples for DNA, and then you have identifications based on DNA.

And you might have a taxon core or a checklist, as I mentioned, where you have the distribution of the species covered.

Slide 11 - Missing fields?

Now, we have a lot of fields available on GBIF in which you can filter down and narrow your search for. But in some cases, you might not find the fields that you need for your analysis.

If you would ever need such a field, the correct way of getting, well, at least starting the discussion of getting that field into GBIF would to be to go through TDWG and the Darwin Core standard.

You can propose new terms and changes to Darwin Core, but you have to be mindful that changing data standards is a community-driven process which requires a lot of engagement and patience because you need to discuss it with your peers, and you need to find a common ground for adding those data. So you have to consider whether the term already exists but would require an update, where the new term would best be placed. Is it taxon specific? Is it occurrence specific or event specific? Is it something to do with locations?

And you also have to think about whether the term is better suited in an extension or in one of the core tables.

So many people are maybe not that aware of extensions to these core tables, but there’s a lot of extensions out there to serve the community who wants to share data or use data.

So please familiarize yourself with them before you propose or change terms.

Slide 12 - Dataset metadata

One last thing I have to mention about the data standards is that you also have a data standard called EML or Ecological Metadata Language publishers use to share information about the dataset resource in which they have all their data. So that is, for example, who was the people who was involved in maintaining, gathering the data, the overall spatial and temporal scope of the dataset, the taxonomic scope of the dataset, and any funding and project-specific information, as well as methods and design.

So that’s another standard which is not part of the TDWG standard you might want to look into as well.