Welcome to the GitHub repository of the DAB Community Edition (CE). This repository includes all consolidated and stable functionalities of the DAB.
The DAB is a brokering software framework for the mediation and harmonization of geospatial data with particular reference to Earth observation data, both satellite and non-satellite. DAB makes it possible to connect heterogeneous data sources and make them discoverable and accessible through homogeneous and standard interfaces by different data user tools and applications. The DAB is capable of implementing data discovery and access functionalities by implementing multiple standard interfaces (e.g. OGC, ISO, ...), and Application Programming Interfaces (API).
Instructions on how to start using the DAB
The DAB is composed by multiple maven modules. GS-service is the main module, capable of starting the DAB Internet services and its web configuration tool.
The command mvn clean install run in the root folder will compile the source code into jar files and finally compose the war package of DAB GS-service.
Finally, it's possible to launch DAB services on a single node configuration by launching the following maven command from the gs-service folder:
mvn jetty:run -Ppreconfigured -DdbUser=XXX -DdbPassword=XXX -DquartzUser=root -DquartzPassword=XXX -DadminUser=XXX -DgoogleClientId=XXX -DgoogleClientSecret=XXX
The DAB will launch using a preconfigured configuration where:
- dbUser/dbPassword are the credentials of a db installed on localhost (9200 port by default for OpenSearch db)
- quartzUser/quartzPassword are the credentials of a mysql db installed on localhost (3306 port)
- adminUser/googleClientId/googleClientSecret are the Google OAuth credentials previously obtained through Google
- an OpenSearhc instance installed on localhost will receive statistics (9200 port)
The DAB can then be configured at the URL:
http://localhost:9090/gs-service/conf
It's also possible to create a brand new configuration at the URL:
http://localhost:9090/gs-service/initialize
The web configurator tool enables to add/remove both profilers and accessors. In this configuration all the profilers are activated. For the accessors, it's needed to specify the data source URL endpoint, the accessor strategy (distributed vs harvested) and the connector type. It's also possible to start harvestings of specific sources.
A demo portal to check that DAB is working as expected will be available at the URL:
http://localhost:9090/gs-service/search
In a more complex configuration the DAB can be deployed on a cloud service, where multiple containers can be responsible for different tasks (e.g. frontend, harvesting, access). The DAB Dockerhub repository is currently under construction.
It's possible to create a docker image with the following commands from the gs-service project:
mvn -o -B -Pvaadin-production -Dmaven.test.skip=true clean package
mvn docker:build
This it will create the same DAB image that is also available on Docker Hub.
The docker container can readily be started for example with the following command:
docker run -p 8080:8080 -e "JAVA_OPTS=-Dconfiguration.url=file:///tmp" essilab/dab:latest
While the DAB community edition code is open source, it is not a public domain software. Therefore, to use the DAB source code you should give full attribution to the original authors and redistribute your modifications in the same way, accordingly to the GNU Affero General Public License v3.0 (see LICENSE). To know more about how to correctly utilize the DAB open source code and provide appropriate attribution, please, select the relevant cases:
To use the DAB technology for providing online services
AGPL license is more restrictive with respect to GPL with regard to online service providers making use of the licensed software.
This is the case where a third party downloads the DAB source code and offers its functionalities through an online service (e.g. through a server managed by the third party). In this case the following statement should appear at the third party site offering the service:
The brokering service is offered by DAB, a software developed by National Research Council of Italy (CNR)/Institute of Atmospheric Pollution Research (IIA)/ESSI-Lab. More information is available at https://github.com/ESSI-Lab/DAB/
To use the DAB technology for a scientific and/or technological publication
Please note at least one of the following papers should be cited to give correct author attributions while describing work making use of the DAB:
To start a new branch of the DAB software
The modified source code should be licensed according to AGPL and redistributed back to the community, preferably through the DAB GitHub repository. You are welcome and encouraged to contact ESSI-Lab to propose code contributions to be applied to the present repository (see contributions section).
In any case the following attribution headers should be preserved in modified versions of the source code:
Discovery and Access Broker (DAB) Community Edition (CE)
Copyright (C) 2021 National Research Council of Italy (CNR)/Institute of Atmospheric Pollution Research (IIA)/ESSI-Lab
In case a modified version of the DAB is offered by a third party as an Internet service, additionally to the attribution described above also the code (or a link to the DAB GitHub repository holding the modifications) should be made available on the third party site.
How to contribute to this project
Contributors to the ESSI-Lab DAB CE project are welcome: you can report bugs and request enhancements contacting ESSI-Lab or through the GitHub issue tracker.
To propose code contributions please contact ESSI-Lab to coordinate the development effort and avoid duplications.
The ESSI-Lab Contributor License Agreement (based on HA-CLA-I-OSI) needs to be signed by contributors to enable ESSI-Lab to accept community contributions to the DAB repository even more smoothly. Now the signature of the agreement is fully integrated in the Github pull request.
If you're interested in contributing, please take a look as well at ESSI-Lab's Contributor Code of Conduct.
Learn more about the DAB history and the main contributors
DAB was first conceived and is still maintained and advanced by the ESSI-Lab based at the Florence division of the Institute of Atmospheric Pollution Research (IIA) of National Research Council of Italy (CNR).
DAB implements the data brokering approach as introduced by Stefano Nativi et al. in the position paper: S. Nativi, M. Craglia and J. Pearlman, "Earth Science Infrastructures Interoperability: The Brokering Approach," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 6, no. 3, pp. 1118-1129, June 2013, doi: 10.1109/JSTARS.2013.2243113. by applying the mediation pattern S. Nativi and L. Bigagli, "Discovery, Mediation, and Access Services for Earth Observation Data," in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 2, no. 4, pp. 233-240, Dec. 2009, doi: 10.1109/JSTARS.2009.2028584.
The DAB software has been developed in the context of numerous National, European, and international projects and initiatives, which have been funded and/or operated by different organizations, over the last ten years, including:
- Intergovernmental initiatives
- GEO
- GEOSS Global Earth System of Systems
- WMO
- "WHOS-Plata" (2019/2020)
- "WHOS-Dominican Republic" (2020/2022)
- "WHOS" (2021/2024)
- GEO
- European Union funded projects
- Horizon Europe
- FAIR-EASE (2022/2025)
- Blue-Cloud 2026 (2023/2026)
- TRIGGER (2022/2027)
- Horizon 2020
- ODIP 2 (2015/2018)
- ERA-PLANET (2016/2022)
- SeaDataCloud (2016/2021)
- Blue-Cloud (2019/2023)
- I-CHANGE (2021/2025)
- EASME
- EMODNet Ingestion (2016/2019)
- EMODNet Ingestion II (2019/2021)
- EMODNet Ingestion III (2022/2024)
- FP7
- EuroGEOSS (2009/2012)
- SeaDataNet II (2011/2015)
- GEOWOW (2011/2014)
- ODIP (2011/2015)
- European Space Agency (ESA)
- HMA-T (2008/2009)
- Prod-Trees (2012/2015)
- DAB4EDGE (2018/2020)
- DAB4GPP (2022/2025)
- Horizon Europe
- National agencies
- National Science Foundation (USA)
- BCube (2013/2016)
- MIUR (Italy)
- ND-SoS-Ina (2012/2015)
- ARPA-ER (Italy)
- P107009S (2016)
- PGSIM/2017/771 (2017)
- PGSIM/2018/0000563 (2018)
- LoA Arpae SIMC/CNR-IIA (2019)
- ISPRA (Italy)
- HIS-Central (2021-2025)
- National Science Foundation (USA)
As from 2012, the DAB technology has been utilized by GEO (the Group of Earth Observation) as the enabling component of the GEO-DAB: the brokerage data services middleware of the GEOSS (Global Earth Observation System of Systems) platform (formerly known as GCI). GEO-DAB is deployed and operated on a scalable cloud infrastructure. In 2021, the GEO-DAB has connected over 190 different geospatial information systems, which allow the search and discovery of more than 10 million datasets, consisting of over 1 billion single downloadable data files (i.e. data granules in GEO jargon). A useful reference is: Stefano Nativi, Paolo Mazzetti, Mattia Santoro, Fabrizio Papeschi, Max Craglia, Osamu Ochiai, Big Data challenges in building the Global Earth Observation System of Systems, Environmental Modelling & Software, Volume 68, 2015, Pages 1-26.. More info on GEO-DAB is available here
As of 2017, the DAB software has been also adopted by WMO as one of the enabling technologies of the WMO Hydrological Observing System (WHOS): a system of systems capable to share hydrological data, at a global scale. More info on WHOS here. A WHOS webinar was recently hold to introduce users to the WHOS brokering services; the webinar recording is available here.
In 2011, the SeaDataNet data platform started utilizing the DAB technology to enable the discovery and access of different international ocean data sources. More information on SeaDataNet brokering services is availabe here. The DAB mediation and brokering services were also applied by the ODIP community to interconnect the ocean information system-of-systems from USA, Australia, and Europe.
The DAB technology won the 2014 Geospatial World Innovation Awards (media coverage here).
The following papers provide a technical description of the DAB brokering framweork.
Many data scientists, computer scientists, information and software engineers have contributed to the DAB design and development throughout an implementation effort of over ten years, spanning different maturity stages (from prototypical to operational) and sub-components (e.g. GI-cat, GI-axe, GI-sem, GI-suite, ...).
DAB development team, in alphabetic order (contact us):
- Lorenzo Bigagli (software design)
- Enrico Boldrini (software design, developer, coordinator of DAB activities with WMO)
- Paolo Mazzetti (responsible)
- Stefano Nativi (DAB father)
- Massimiliano Olivieri (sysadmin)
- Fabrizio Papeschi (developer)
- Roberto Roncella (developer)
- Mattia Santoro (software design, developer, coordinator of DAB activities with GEO)
Experts who contributed with valuable inputs and feedbacks representing significant data provider and user communities (in alphabetic order):
- Alessandro Annoni
- Igor Chernov
- Guido Colangeli
- Max Craglia
- Paola De Salvo
- Ben Domenico
- Gregory Giuliani
- Wim Hugo
- Siri Jodha Khalsa
- Osamu Ochiai
- Washington Otieno
- Anthony Lehmann
- Joan Masó
- Francoise Pearlman
- Jay Pearlman
- Silvano Pecora
- Nicola Pirrone
- Antonello Provenzale
- Barbara Ryan
- Dick Schaap
- Richard Signell
- Joost Van Bemmelen
Previous members of the development team:
- Francesco Pezzati (past developer)
- Alessio Baldini (past developer)
- Fabrizio Vitale (past developer)
- Ugo Mattia (past developer)
- Valerio Angelini (past developer)
ESSI-Lab would like to credit all the maintainers and developers of the third party services and technologies that DAB uses. They include:
- OpenSearch
- MySQL
- ElasticSearch
- Amazon AWS
- Google Cloud Platform
- Google OAuth
- Facebook OAuth
- Twitter OAuth
- Docker
- Kubernetes
See the THIRD-PARTY file for third-party license notices.
Altough we have made our best effort to give the appropriate credits, some entries might be missing or incorrect: you are welcome to report to ESSI-Lab any of such issues you might identifiy.
You are welcome to contact us for any info about the DAB, including bug reports, feature enhancements and code contributions.
GitHub repository link: https://github.com/ESSI-Lab/DAB