diff --git a/about/index.rst b/about/index.rst deleted file mode 100644 index 12cedbca..00000000 --- a/about/index.rst +++ /dev/null @@ -1,90 +0,0 @@ -What is GeoNode -=============== - -.. image:: img/logo.png - -GeoNode is a geospatial content management system, a platform for the management and publication of geospatial data. It brings together mature and stable open-source software projects under a consistent and easy-to-use interface allowing non-specialized users to share data and create interactive maps. - -Data management tools built into GeoNode allow for integrated creation of data, metadata, and map visualization. Each dataset in the system can be shared publicly or restricted to allow access to only specific users. Social features like user profiles and commenting and rating systems allow for the development of communities around each platform to facilitate the use, management, and quality control of the data the GeoNode instance contains. - -It is also designed to be a flexible platform that software developers can extend, modify or integrate against to meet requirements in their own applications. - -Showcase --------- - -A handful of other Open Source projects extend GeoNode’s functionality -by tapping into the re-usability of Django applications. -Visit our gallery to see how the community uses GeoNode: `GeoNode Showcase `_. - -The development community is very supportive of new projects and contributes ideas and guidance for newcomers. - -For a live demo see also :ref:`online_demo` - -Most useful links ------------------ - -**General** - -- Project homepage: https://geonode.org -- Repository: https://github.com/GeoNode/geonode -- Official Demo: http://master.demo.geonode.org -- GeoNode Wiki: https://github.com/GeoNode/geonode/wiki -- Issue tracker: https://github.com/GeoNode/geonode-project/issues - - In case of sensitive bugs like security vulnerabilities, please - contact a GeoNode Core Developer directly instead of using issue - tracker. We value your effort to improve the security and privacy of - this project! - -**Related projects** - -- GeoNode Project: https://github.com/GeoNode/geonode-project -- GeoNode at Docker: https://hub.docker.com/u/geonode -- GeoNode OSGeo-Live: https://live.osgeo.org/en/ - - -Licensing -========= - -GeoNode is Copyright 2018 Open Source Geospatial Foundation (OSGeo). - -GeoNode is free software: you can redistribute it and/or modify it under -the terms of the GNU General Public License as published by the Free -Software Foundation, either version 3 of the License, or (at your -option) any later version. GeoNode is distributed in the hope that it -will be useful, but WITHOUT ANY WARRANTY; without even the implied -warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -GNU General Public License for more details. - -You should have received a copy of the GNU General Public License along -with GeoNode. If not, see http://www.gnu.org/licenses. - -Current Version and Features -============================ - -GeoNode current version: `3.2.0 `_ - -Main Features: `State of GeoNode `_ - -.. _get_in_touch: - -Get in touch with the community -=============================== - -GeoNode is an open source project and contributors are needed to keep this project moving forward. Learn more on how to contribute on our -`Community Bylaws `_. - -- User Mailing List: https://lists.osgeo.org/cgi-bin/mailman/listinfo/geonode-users -- Developer Mailing List: https://lists.osgeo.org/cgi-bin/mailman/listinfo/geonode-devel -- Gitter Chat: https://gitter.im/GeoNode/general - -Roadmap -======= - -GeoNode's development roadmap is documented in a series of GeoNode Improvement Projects (GNIPS). -They are documented at `GeoNode Wiki `_. - -GNIPS are considered to be large undertakings which will add a large amount of features to the project. -As such they are the topic of community discussion and guidance. - -The community discusses these on the developer mailing list: http://lists.osgeo.org/pipermail/geonode-devel/ diff --git a/admin/admin_panel/index.rst b/admin/admin_panel/index.rst deleted file mode 100644 index 57ef41e0..00000000 --- a/admin/admin_panel/index.rst +++ /dev/null @@ -1,1046 +0,0 @@ -Accessing the panel -=================== - -| The *Admin Panel* is a model-centric interface where trusted users can manage content on GeoNode. -| Only the staff users can access the admin interface. - -.. note:: The “staff” flag, which controls whether the user is allowed to log in to the admin interface, can be set by the admin panel itself. - -The panel can be reached from :guilabel:`Admin` link of the *User Menu* in the navigation bar (see the picture below) or through this URL: ``http:///admin``. - -.. figure:: img/admin_link.png - :align: center - - *The Admin Link of the User Menu* - -When clicking on that link the Django-based *Admin Interface* page opens and shows you all the Django models registered in GeoNode. - -.. figure:: img/django_geonode_admin_interface.png - :align: center - - *The GeoNode Admin Interface* - -Reset or Change the admin password -================================== - -From the *Admin Interface* you can access the :guilabel:`Change password` link by clicking on the username on the right side of the navigation bar. which will open a dropdown. - -.. figure:: img/change_password_link.png - :align: center - - *The Change Password Link* - -It allows you to access the *Change Password Form* through which you can change your password. - -.. figure:: img/change_password_form.png - :align: center - - *The Change Password Form* - -Once the fields have been filled out, click on :guilabel:`Change my password` to perform the change. - -.. _simple-theming: - -Simple Theming -============== - -GeoNode provides by default some theming options manageable directly from the Administration panel. -Most of the times those options allows you to easily change the GeoNode look and feel without touching a single line of `HTML` or `CSS`. - -As an `administrator` go to ``http:///admin/geonode_themes/geonodethemecustomization/``. - -.. figure:: img/theming/themes.png - :align: center - - *List of available Themes* - -The panel shows all the available GeoNode themes, if any, and allows you to create new ones. - -.. warning:: Only one theme at a time can be **activated** (aka *enabled*). By disabling or deleting all the available themes, GeoNode will turn the gui back to the default one. - -Editing or creating a new Theme, will actually allow you to customize several properties. - -At least you'll need to provide a ``Name`` for the Theme. Optionally you can specify also a ``Description``, which will allow you to better -identify the type of Theme you created. - -.. figure:: img/theming/theme-def-0001.png - :align: center - - *Theme Name and Description* - -Just below the ``Description`` field, you will find the ``Enabled`` checkbox, allowing you to toggle the Theme. - -.. figure:: img/theming/theme-def-0002.png - :align: center - - *Theme Name and Description* - -Jumbotron and Get Started link -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. note:: Remember, everytime you want to apply some changes to the Theme, you **must** save the Theme and reload the GeoNode browser tab. - In order to quickly switch back to the Home page, you can just click the ``VIEW SITE`` link on the top-right corner of the Admin dashboard. - - .. figure:: img/theming/view_site.png - :align: center - -The next section, allows you to define the first important Theme properties. This part involves the GeoNode main page sections. - -.. figure:: img/theming/theme_properties.png - :align: center - - *Jumbotron and Logo options* - -By changing those properties as shown above, you will easily change your default home page from this - -.. figure:: img/theming/default_theme.png - :align: center - - *GeoNode Default Home* - -to this - -.. figure:: img/theming/first_customization.png - :align: center - - *Updating Jumbotron and Logo* - -It is possible to optionally **hide** the ``Jumbotron text``. - -.. figure:: img/theming/hide_jumbotron.png - :align: center - -.. figure:: img/theming/hide_jumbotron_view.png - :align: center - - *Hide Jumbotron text* - -Slide show -^^^^^^^^^^ -To switch between a slide show and a jumbotron, flip the value of the welcome theme from "slide show" to "jumbotron" and vice versa to either display a jumbotron with content or a slide show in the home page - -For example, to display a slide show, change the welcome theme from jumbotron background - -.. figure:: img/theming/theme-def-0007a.png - :align: center - -to slide show - -.. figure:: img/theming/theme-def-0007b.png - :align: center - -Before creating a slide show, make sure you have slides to select from (in the multi-select widget) to make up the slide show. - -.. figure:: img/theming/theme-def-0007c.png - :align: center - -If no slides exist, click the plus (+) button beside the slide show multi-select widget to add a new slide. - -.. figure:: img/theming/theme-def-0007d.png - :align: center - -Fill in the slide name, slide content using markdown formatting, and upload a slide image (the image that will be displayed when the slide is in view). - -.. figure:: img/theming/theme-def-0007e.png - :align: center - -For slide images that already contain text, hide slide content by checking the checkbox labeled "Hide text in the jumbotron slide" as shown below, then save the slide. - -.. figure:: img/theming/theme-def-0007f.png - :align: center - -It is also possible to hide a slide from all slide show themes that use it by unchecking the checkbox labeled "Is enabled" as shown below. - -.. figure:: img/theming/theme-def-0007g.png - :align: center - -Selecting the above slide in a slide show and enabling slide show (using the "welcome theme" configuration) will create a slide show with a slide as shown below: - -.. figure:: img/theming/theme-def-0007h.png - :align: center - -Switching between different themes -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In the case you have defined more Themes, switching between them is as easy as ``enabling`` one and ``disabling`` the others. - -Remember to save the Themes everytime and refresh the GeoNode home page on the browser to see the changes. - -It is also important that there is **only one** Theme enabled **at a time**. - -In order to go back to the standard GeoNode behavior, just disable or delete all the available Themes. - -Add a new user -============== - -In GeoNode, administrators can manage other users. For example, they can *Add New Users* through the following form. - -.. figure:: img/add_user_form.png - :align: center - - *Adding New Users* - -The form above can be reached from the *Admin Panel* at the following path: *Home > People > Users*. Click on :guilabel:`+ Add user` to open the form page. - -.. figure:: img/add_user_button.png - :align: center - - *The Add User button in the Users List page* - -It is also available, in the GeoNode UI, the :guilabel:`Add User` link of the *About* menu in the navigation bar. - -.. figure:: img/add_user_link.png - :align: center - - *Add User Link* - -To perform the user creation fill out the required fields (*username* and *password*) and click on :guilabel:`Save`. -You will be redirected to the *User Details Page* which allows to insert further information about the user. - -.. figure:: img/user_details_admin_page.png - :align: center - - *The User Details Page* - -The user will be visible into the *Users List Page* of the *Admin Panel* and in the *People Page* (see :ref:`user-info`). - -.. figure:: img/new_user_in_people.png - :align: center - - *The User in the People page* - -Activate/Disable a User -======================= - -When created, new users are *active* by default. -You can check that in the *User Details Page* from the *Admin Panel* (see the picture below). - -.. figure:: img/new_user_active.png - :align: center - - *New Users Active by default* - -| *Active* users can interact with other users and groups, can manage resources and, more in general, can take actions on the GeoNode platform. -| Untick the *Active* checkbox to disable the user. It will be not considered as user by the GeoNode system. - -.. figure:: img/new_user_disabled.png - :align: center - - *Disabled Users* - -Change a User password -====================== - -GeoNode administrators can also change/reset the password for those users who forget it. -As shown in the picture below, click on ``this form`` link from the *User Details Page* to access the *Change Password Form*. - -.. figure:: img/change_user_password_link.png - :align: center - - *Changing Users Passwords* - -The *Change User Password Form* should looks like the following one. -Insert the new password two times and click on :guilabel:`CHANGE PASSWORD`. - -.. figure:: img/change_user_password_form.png - :align: center - - *Changing Users Passwords* - -Promoting a User to Staff member or superuser -============================================= - -Active users have not access to admin tools. -GeoNode makes available those tools only to *Staff Members* who have the needed permissions. -*Superusers* are staff members with full access to admin tools (all permissions are assigned to them). - -Administrators can promote a user to *Staff Member* by ticking the **Staff status** checkbox in the *User Details Page*. -To make some user a *Superuser*, the **Superuser status** checkbox should be ticked. See the picture below. - -.. figure:: img/staff_and_superuser_permissions.png - :align: center - - *Staff and Superuser permissions* - -Creating a Group -================ - -| In GeoNode is possible to create new groups with set of permissions which will be inherited by all the group members. -| The creation of a Group can be done both on the GeoNode UI and on the *Admin Panel*, we will explain how in this paragraph. - -The :guilabel:`Create Groups` link of *About* menu in the navigation bar allows administrators to reach the *Group Creation Page*. - -.. figure:: img/create_group_page_link.png - :align: center - - *The Create Group Link* - -The following form will open. - -.. figure:: img/group_creation_form.png - :align: center - - *The Group Creation Form* - -Fill out all the required fields and click :guilabel:`Create` to create the group. -The *Group Details Page* will open. - -.. figure:: img/group_details_page.png - :align: center - - *The Group Details Page* - -The new created group will be searchable in the *Groups List Page*. - -.. figure:: img/groups_list_page.png - :align: center - - *The Groups List Page* - -.. note:: The :guilabel:`Create a New Group` button on the *Groups List Page* allows to reach the *Group Creation Form*. - -| As already mentioned above, groups can also be created from the Django-based *Admin Interface* of GeoNode. -| The *Groups* link of the *AUTHENTICATION AND AUTHORIZATION* section allows to manage basic Django groups which only care about permissions. -| To create a GeoNode group you should take a look at the *GROUPS* section. - -.. figure:: img/groups_admin_section.png - :align: center - - *The Groups Section on the Admin Panel* - -As you can see, GeoNode provides two types of groups. You will learn more about that in the next paragraph. - -Types of Groups -^^^^^^^^^^^^^^^ - -In GeoNode users can be grouped through a *Group Profile*, an enhanced Django group which can be enriched with some further information such as a description, a logo, an email address, some keywords, etc. -It also possible to define some *Group Categories* based on which those group profiles can be divided and filtered. - -A new **Group Profile** can be created as follow: - -* click on the *Group Profile* :guilabel:`+ Add` button - -* fill out all the required fields (see the picture below), *Group Profiles* can be explicitly related to group categories - - .. figure:: img/new_group_profile_form.png - :align: center - - *A new Group Profile* - -* click on :guilabel:`Save` to perform the creation, the new created group profile will be visible in the *Group Profiles List* - - .. figure:: img/group_profiles_list.png - :align: center - - *The Group Profiles List* - -Group Categories -^^^^^^^^^^^^^^^^ - -*Group Profiles* can also be related to *Group Categories* which represents common topics between groups. -In order to add a new **Group Category** follow these steps: - -* click on the *Group Categories* :guilabel:`+ Add group category` button - -* fill out the creation form (type *name* and *description*) - - .. figure:: img/new_group_category_form.png - :align: center - - *A new Group Category* - -* click on :guilabel:`Save` to perform the creation, the new created category will be visible in the *Group Categories List* - - .. figure:: img/group_categories_list.png - :align: center - - *The Group Categories List* - - -Managing a Group -================ - -Through the :guilabel:`Groups` link of *About* menu in the navigation bar, administrators can reach the *Groups List Page*. - -.. figure:: img/groups_link.png - :align: center - - *The Groups Link in the navigation bar* - -In that page all the GeoNode *Group Profiles* are listed. - -.. figure:: img/group_profiles_list_page.png - :align: center - - *Group Profiles List Page* - -For each group some summary information (such as the *title*, the *description*, the number of *members* and *managers*) are displayed near the *Group Logo*. - -Administrators can manage a group from the *Group Profile Details Page* which is reachable by clicking on the *title* of the group. - -.. figure:: img/group_profile_details_page.png - :align: center - - *Group Profile Details Page* - -As shown in the picture above, all information about the group are available on that page: - -* the group *Title*; -* the *Last Editing Date* which shows a timestamp corresponding to the last editing of the group properties; -* the *Keywords* associated with the group; -* *Permissions* on the group (Public, Public(invite-only), Private); -* *Members* who join the group; -* *Managers* who manage the group. - -There are also four links: - -* The :guilabel:`Edit Group Details` link opens the *Group Profile Form* through which the following properties can be changed: - - * *Title*. - * *Logo* (see next paragraphs). - * *Description*. - * *Email*, to contact one or all group members. - * *Keywords*, a comma-separated list of keywords. - * *Access*, which regulates permissions: - - * *Public*: any registered user can view and join a public group. - * *Public (invite-only)*: only invited users can join, any registered user can view the group. - * *Private*: only invited users can join the group, registered users cannot see any details about the group, including membership. - - * *Categories*, the group categories the group belongs to. - -* :guilabel:`Manage Group Members` (see next paragraphs). -* the :guilabel:`Delete this Group`, click on it to delete the Group Profile. GeoNode requires you to confirm this action. - - .. figure:: img/confirm_group_deletion.png - :align: center - :width: 400px - - *Confirm Group Deletion* - -* the :guilabel:`Group Activities` drives you to the *Group Activities Page* where you can see all datasets, maps and documents associated with the group. There is also a *Comments* tab which shows comments on those resources. - - .. figure:: img/group_activities.png - :align: center - - *Group Activities* - -Group Logo -^^^^^^^^^^ - -Each group represents something in common between its members. -So each group should have a *Logo* which graphically represents the idea that identify the group. - -On the *Group Profile Form* page you can insert a logo from your disk by click on :guilabel:`Browse...`. - -.. figure:: img/editing_group_logo.png - :align: center - - *Editing the Group Logo* - -| Click on :guilabel:`Update` to apply the changes. -| Take a look at your group now, you should be able to see that logo. - -.. figure:: img/group_logo.png - :align: center - - *The Group Logo* - -Managing Group members -^^^^^^^^^^^^^^^^^^^^^^ - -The :guilabel:`Manage Group Members` link opens the *Group Members Page* which shows *Group Members* and *Group Managers*. -**Managers** can edit group details, can delete the group, can see the group activities and can manage memberships. -Other **Members** can only see the group activities. - -| In Public Groups, users can join the group without any approval. - Other types of groups require the user to be invited by the group managers. -| Only group managers can *Add new members*. - In the picture below, you can see the manager can search for users by typing their names into the *User Identifiers* search bar. - Once found, he can add them to the group by clicking the :guilabel:`Add Group Members` button. - The *Assign manager role* flag implies that all the users found will become managers of the group. - -.. figure:: img/add_new_member.png - :align: center - - *Adding a new Member to the Group* - -The following picture shows you the results. - -.. figure:: img/new_members.png - :align: center - - *New Members of the Group* - -If you want to change the role of group members after adding them, you can use the "promote" button to make a member into a manager, and the "demote" button to make a manager into a regular member. - -Group based advanced data workflow -================================== - -By default GeoNode is configured to make every resource suddenly available to everyone, i.e. publicly accessible -even from anonymous/non-logged in users. - -It is actually possible to change few configuration settings in order to allow GeoNode to enable an advanced publication workflow. - -With the advanced workflow enabled, your resources won't be automatically published (i.e. made visible and accessible for all, contributors or simple users). - -For now, your item is only visible by yourself, the manager of the group to which the resource is linked (this information is filled in the metadata), the members of this group, and the GeoNode Administrators. - -Before being published, the resource will follow a two-stage review process, which is described below: - -.. figure:: img/adv_data_workflow/adv_data_workflow_001.jpg - :align: center - - *From upload to publication: the review process on GeoNode* - -How to enable the advanced workflow -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -You have to tweak the GeoNode settings accordingly. - -Please see the details of the following GeoNode ``Settings``: - -* `RESOURCE_PUBLISHING <../../basic/settings/index.html#resource-publishing>`_ - -* `ADMIN_MODERATE_UPLOADS <../../basic/settings/index.html#admin-moderate-uploads>`_ - -* `GROUP_PRIVATE_RESOURCES <../../basic/settings/index.html#group-private-resources>`_ - -Summarizing, when all the options above of the Advanced Workflow are enabled, upon a new upload we will have: - - - The **"unpublished"** resources will be **hidden** to **anonymous users only**. The **registered users** will be still able to access the resources (if they have the rights to do that, of course). - - - The **"unpublished"** resources will remain hidden to users if the permission (see *Admin Guide section: 'Manage Permissions'*) will be explicitly removed - - - During the upload, whenever the advanced workflow is enabled, the **owner's Groups** are automatically allowed to access the resource, even if the **"anonymous"** flag has been disabled. Those permissions can be removed later on - - - During the upload, **"managers"** of the owner's Groups associated to the resource, are always allowed to edit the resource, the same as they are admin for that resource - - - **"managers"** of the owner's Groups associated to the resource are allowed to **"publish"** also the resources, not only to **"approve"** them - - -Change the owner rights in case of advanced workflow is on -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -After switching ``ADMIN_MODERATE_UPLOADS`` to True and resource is approved owner is no longer able -to modify it. He will see new button on the resource detail page: Request change. After clicking this, view with short -form is shown. -On this view user can write short message why he want to modify the resource. - -This message will be sent through messaging and email system to administrators: - -After administrator unapprove the resource owner is again able to modify it. - -The group Manager approval -^^^^^^^^^^^^^^^^^^^^^^^^^^ -Here, the role of the Manager of the group to which your dataset, document or map is linked is to check that the uploaded item is correct. -Particularly, in the case of a dataset or a map, it consists of checking that the chosen cartographic representation and the style are -fitting but also that the discretization is appropriate. - -The Manager must also check that the metadata are properly completed and that the mandatory information -(Title, Abstract, Edition, Keywords, Category, Group, Region) are filled. - -If needed, the Manager can contact the contributor responsible of the dataset, document or map in order to report potential comments or -request clarifications. - -Members of the group can also take part in the reviewing process and give some potential inputs to the responsible of the -dataset, document or map. - -When the Manager considers that the resource is ready to be published, he should approve it. -To do so, the Manager goes to the resource detail page, then opens the :guilabel:`Edit Metadata`. -In the :guilabel:`Settings` tab, the manager checks the :guilabel:`Approved` box, and then updates the metadata and saves the changes: - -Following this approval, the GeoNode Administrators receive a notification informing them that an item is now waiting for publication - - -The publication by the GeoNode Administrator -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Prior to the public release of an approved resource, the Administrator of the platform performs a final validation of -the item and its metadata, notably to check that it is in line with license policies. - -If needed, the GeoNode Administrator can contact the Manager who has approved the resource, as well as its responsible. - -Once the resource is validated, the item is made public by the Administrator. -It can now be viewed, accessed, and downloaded in accordance with the ``Permissions`` set by the responsible contributor. - -Promotion, Demotion and Removal of Group Members -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -If the owner is a group Manager, They have permissions to edit, approve, and publish the resource. - -When a group member is promoted to a manager role, they gain permissions to edit, approve and publish the resource. - -When a group manager is demoted to a member role, they lose edit permissions of the resource and only remain with view and download permissions. - -When a member is removed from the group, they can nolonger access the unpublished resource anymore. - -Manage profiles using the admin panel -===================================== - -So far GeoNode implements two distinct roles, that can be assigned to resources such as datasets, maps or documents: - -* party who authored the resource -* party who can be contacted for acquiring knowledge about or acquisition of the resource - -These two profiles can be set in the GeoNode interface by accessing the metadata page and setting the ``Point of Contact`` and ``Metadata Author`` fields respectively. - -Is possible for an administrator to add new roles if needed, by clicking on the :guilabel:`Add contact role` button in the :guilabel:`Base -> Contact Roles` section: - -.. figure:: img/admin-roles-add.png - :align: center - -Clicking on the :guilabel:`People` section (see figure) will open a web for with some personal information plus a section called :guilabel:`Users`. - -.. figure:: img/admin-people.png - -Is important that this last section is not modified here unless the administrator is very confident in that operation. - -.. figure:: img/admin-profiles-contactroles.png - :align: center - -Manage datasets using the admin panel -=================================== - -Some of the datasets information can be edited directly through the admin interface although the best place is in the :guilabel:`Dataset -> Metadata Edit` in GeoNode. - -Clicking on the :guilabel:`Admin > Dataset > Datasets` link will show the list of available datasets. - -.. figure:: img/admin-datasets.png - :align: center - -.. warning:: It is not recommended to modify the Datasets' ``Attributes`` or ``Styles`` directly from the Admin dashboard unless you are aware of your actions. - -The ``Metadata`` information can be changed for multiple datasets at once through the :guilabel:`Metadata batch edit` action. Select the datasets you want to edit in the batch and at the bottom, enter the `Metadata batch edit` action then click :guilabel:`Go`. - -.. figure:: img/admin-datasets-batch.png - :align: center - -This will open a form with the information you can edit in a batch. see picture below. - -.. figure:: img/admin-datasets-batch-form.png - :align: center - -By clicking over one Dataset link, it will show a detail page allowing you to modify some of the resource info like the metadata, the keywords, the title, etc. - -.. note:: It is strongly recommended to always use the GeoNode resource :guilabel:`Edit Metadata` or :guilabel:`Advanced Metadata` tools in order to edit the metadata info. - -The ``Permissions`` can be changed also for multiple Datasets at once through the :guilabel:`Set permissions` action. - -.. figure:: img/set_datasets_permissions_action.png - :align: center - -By clicking over one Dataset link, it will show a detail page allowing you to modify the permissions for the selected resources. - - -Manage the maps using the admin panel -===================================== - -Similarly to the Datasets, it is possible to manage the available GeoNode Maps through the Admin panel also. - -Move to :guilabel:`Admin > Maps` to access the Maps list. - -.. figure:: img/admin-maps.png - :align: center - -Notice that by enabling the ``Featured`` option here, will allow GeoNode to show the Map thumbnail and the Map detail link at the top under featured resources on the :guilabel:`Home Page` - -.. figure:: img/admin-maps-featured-001.png - :align: center - -.. figure:: img/admin-maps-featured-002.png - :align: center - -Manage the documents using the admin panel -========================================== - -Similarly to the Datasets and Maps, it is possible to manage the available GeoNode Documents through the Admin panel also. - -Move to :guilabel:`Admin > Documents` to access the Documents list. - -.. figure:: img/admin-documents.png - :align: center - -By clicking over one Document link, it will show a detail page allowing you to modify some of the resource info like the metadata, the keywords, the title, etc. - -Manage the base metadata choices using the admin panel -====================================================== - -:guilabel:`Admin > Base` contains almost all the objects you need to populate the resources metadata choices. - -.. figure:: img/metadata-base/admin-panel-metadata-contents-0001.png - :align: center - - *Admin dashboard Base Panel* - -In other words the options available from the :guilabel:`select-boxes` of the resource :guilabel:`Edit Metadata` and :guilabel:`Advanced Metadata` forms. - -.. figure:: img/metadata-base/admin-panel-metadata-contents-0002.png - :align: center - - *Metadata Form* - -.. figure:: img/metadata-base/admin-panel-metadata-contents-0003.png - :align: center - - *Advanced Metadata Form* - -.. note:: When editing the resource metadata through the :guilabel:`Edit Metadata`, some fields are marked as ``mandatory`` and by filling those information - the ``Completeness`` progress will advance accordingly. - - .. figure:: img/metadata-base/admin-panel-metadata-contents-0003a.png - :align: center - - *Metadata Completeness* - - Even if not all the fields have been filled, the system won't prevent you to update the metadata; this is why the ``Mandatory`` fields are - mandatory to be fully compliant with an ``ISO 19115`` metadata schema, but are only recommended to be compliant with GeoNode. - - Also the ``Completeness`` indicates how far the metadata is to be compliant with an ``ISO 19115`` metadata schema. - - Of course, it is **highly** recommended to always fill as much as possible at least all the metadata fields marked as ``Mandatory``. - - This will improve not only the quality of the data stored into the system, but will help the users to easily search for them on GeoNode. - - All the ``Search & Filter`` panels and options of GeoNode are, in fact, based on the resources metadata fields. Too much generic descriptions and - too empty metadata fields, will give highly un-precise and very wide search results to the users. - -Hierarchical keywords -^^^^^^^^^^^^^^^^^^^^^ - -Through the :guilabel:`Admin > Base > Hierarchical keywords` panel it will be possible to manage all the keywords associated to the resources. - -.. figure:: img/metadata-base/admin-panel-metadata-contents-0004.png - :align: center - - *Hierarchical keywords list* - -.. figure:: img/metadata-base/admin-panel-metadata-contents-0005.png - :align: center - - *Hierarchical keywords edit* - -* The :guilabel:`Name` is the human readable text of the keyword, what users will see. - -* The :guilabel:`Slug` is a unique label used by the system to identify the keyword; most of the times it is equal to the name. - -Notice that through the :guilabel:`Position` and :guilabel:`Relative to` selectors, it is possible to establish a hierarchy between the available keywords. -The hierarchy will be reflected in the form of a tree from the metadata panels. - -By default each user with editing metadata rights on any resource, will be able to insert new keywords into the system by simply typing a free text on the keywords metadata field. - -It is possible to force the user to select from a fixed list of keywords through the `FREETEXT_KEYWORDS_READONLY <../../basic/settings/index.html#freetext-keywords-readonly>`_ setting. - -When set to `True` keywords won't be writable from users anymore. Only admins can will be able to manage them through the :guilabel:`Admin > Base > Hierarchical keywords` panel. - -Licenses -^^^^^^^^ - -Through the :guilabel:`Admin > Base > Licenses` panel it will be possible to manage all the licenses associated to the resources. - -.. figure:: img/metadata-base/admin-panel-metadata-contents-0006.png - :align: center - - *Metadata editor Licenses* - -.. warning:: It is **strongly** recommended to not publish resources without an appropriate license. Always make sure the data provider specifies the correct license and that all the restrictions have been honored. - -Metadata Regions -^^^^^^^^^^^^^^^^ - -Through the :guilabel:`Admin > Base > Metadata Regions` panel it will be possible to manage all the admin areas associated to the resources. - -.. figure:: img/metadata-base/admin-panel-metadata-contents-0009.png - :align: center - - *Resource Metadata Regions* - -Notice that those regions are used by GeoNode to filter search results also through the resource list view. - -.. note:: GeoNode tries to guess the ``Regions`` intersecting the data bounding boxes when uploading a new dataset. Those should be refined by the user dataset on anyway. - -Metadata Restriction Code Types and Spatial Representation Types -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Through the :guilabel:`Admin > Base > Metadata Restriction Code Types` and :guilabel:`Admin > Base > Metadata Spatial Representation Types` panels, it will -be possible to **update only** the metadata descriptions for restrictions and spatial representation types. - -Such lists are *read-only* by default since they have been associated to the specific codes of the ``ISO 19115`` metadata schema. -Changing them would require the system to provide a custom dictionary through the metadata catalog too. Such functionality is not supported actually by GeoNode. - -Metadata Topic Categories -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Through the :guilabel:`Admin > Base > Metadata Topic Categories` panel it will be possible to manage all the resource metadata categories avaialble into the system. - -Notice that by default, GeoNode provides the standard topic categories available with the ``ISO 19115`` metadata schema. Changing them means that the system won't be -compliant with the standard ``ISO 19115`` metadata schema anymore. ``ISO 19115`` metadata schema extensions are not currently supported natively by GeoNode. - -It is worth notice that GeoNode allows you to associate `Font Awesome Icons `_ to each topic category through their ``fa-icon`` code. -Those icons will be used by GeoNode to represent the topic category on both the ``Search & Filter`` menus and :guilabel:`Metadata` panels. - -.. warning:: The list of the ``Metadata Topic Categories`` on the home page is currently fixed. To change it you will need to update or override the GeoNode ``index.html`` HTML template. - -By default the ``Metadata Topic Categories`` are *writable*. Meaning that they can be removed or created by the :guilabel:`Admin` panel. - -It is possible to make them fixed (it will be possible to update their descriptions and icons only) through the `MODIFY_TOPICCATEGORY <../../basic/settings/index.html#modify-topiccategory>`_ setting. - -Announcements -============= - -As an Administrator you might need to broadcast announcements to the world about your portal or simply to the internal contributors. - -GeoNode ``Announcements`` allow actually to do that; an admin has the possibility to create three types of messages, accordingly to their severity, -decide their validity in terms of time period (start date and expiring date of the announcement), who can view them or not (everyone or just the -registerd members) and whenever a user can hide the message or not and how long. - - -There are three types of announcements accordingly to their severity level: ``General``, ``Warning`` and ``Critical`` -The difference is mainly the color of the announcement box. - - -Only administrators and staff members can create and manage announcements. - -Currently there two ways to access and manage the announcements list: - -#. Via the GeoNode interface, from the :guilabel:`Profile` panel - - .. note:: Those are accessible by both admins and staff members. - - .. figure:: img/announcments/admin-announcments-005.png - :align: center - - *Announcements from the Profile panel* - -#. Via the GeoNode :guilabel:`Admin` panel - - .. note:: Those are accessible by admins only. - - .. figure:: img/announcments/admin-announcments-006.png - :align: center - - *Announcements from the Admin panel* - -The functionalities are almost the same for both the interfaces, except that from the :guilabel:`Admin` panel it is possible to manage the -dismissals too. - -``Dismissals`` are basically records of members that have read the announcement and closed the message box. An announcement can have one -``dismissal type`` among the three below: - -#. :guilabel:`No Dismissal Allowed` it won't be possible to close the announcement's message box at all. - -#. :guilabel:`Session Only Dismissal` (*) the default one, it will be possible to close the announcement's message box for the current browser session. It will show up again at next access. - -#. :guilabel:`Permanent Dismissal Allowed` once the announcement's message box is closed, it won't appear again for the current member. - -How to create and manage Announcements -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -From the :guilabel:`Profile` panel, click on ``Announcements`` link - -.. figure:: img/announcments/admin-announcments-007.png - :align: center - - *Announcements List from the Profile panel* - -Click either on :guilabel:`New Announcement` to create a new one or over a title of an existing one to manage its contents. - -Create a new announcement is quite straight; you have to fill the fields provided by the form. - -.. warning:: In order to be visible, you will need to check the :guilabel:`Site wide` option **in any case**. You might want to hide the message to *anonymous* users by enabling the :guilabel:`Members only` option too. - -.. figure:: img/announcments/admin-announcments-008.png - :align: center - - *Create Announcement from the Profile panel* - -Managing announcements form the :guilabel:`Admin` panel, is basically the same; the fields for the form will be exactly the same. - -.. figure:: img/announcments/admin-announcments-009.png - :align: center - - *Create Announcement from the Admin panel* - -Accessing announcements options from the :guilabel:`Admin` panel, allows you to manage dismissals also. -Through this interface you will be able to selectively decide members which can or cannot view a specific announcement, or force them to visualize the messages again by deleting the dismissals accordingly. - -.. figure:: img/announcments/admin-announcments-010.png - :align: center - - *Create Dismissal from the Admin panel* - -Menus, Items and Placeholders -============================= - -GeoNode provides some integrated functionalities allowing you to quickly and easily customize the top-bar menu (see the example below). - -.. figure:: img/admin-panel-menus-0000.png - :align: center - - *GeoNode Top-Bar Menu customization* - -With minor changes of the ``basic.html`` template, potentially, it could be possible to use the same approach for a more complex customization. -Let's start with the simple one. - -By default GeoNode provides custom ``placeholders`` already defined into the ``basic.html`` template, called ``CARDS_MENU``, ``TOPBAR_MENU_RIGHT``, ``TOPBAR_MENU_LEFT``, ``TOPBAR_MENU``. - -From the :guilabel:`Admin > Base` panel, it is possible to access to the ``Menu``, ``Menu Items`` and ``Menu Placeholder`` options. - -.. figure:: img/admin-panel-menus-0001.png - :align: center - - *Menu, Menu Items and Menu Placeholder options on the Admin panel* - -The hierarchical structure of a custom ``Menu`` is the following one: - -1. ``Menu Placeholder``; first of all you need to define a *placeholder* both into the :guilabel:`Admin > Base` panel and the ``basic.html`` template, using the same **keyword**. - By default GeoNode provides already defined menus. - - .. figure:: img/admin-panel-menus-0002.png - :align: center - - *The default ``TOPBAR_MENU`` Menu Placeholder on the Admin panel* - -2. ``Menu``; second thing to do is to create a new *menu* associated to the corresponding *placeholder*. - This is still possible from the :guilabel:`Admin > Base` panel - - .. figure:: img/admin-panel-menus-0003.png - :align: center - - *Create a new Menu from the Admin panel* - - You will need to provide: - - * A ``Title``, representing the name of the ``Menu`` visible by the users - - .. warning:: By using this approach, internationalization won't be supported. For the time being GeoNode does not support this for menus created from the :guilabel:`Admin > Base` panel. - - * A ``Menu Placeholder`` from the existing ones. - - * A ``Order`` in the case you'll create more menus associated to the same placeholder. - -3. ``Menu Item``; finally you will need to create voices belonging to the *menu*. For the time being, GeoNode allows you to create only ``href`` links. - - .. figure:: img/admin-panel-menus-0004.png - :align: center - - *Create a new Menu Item from the Admin panel* - - .. warning:: The ``Menu`` won't be visible until you add more than one ``Menu Item``, If you have 1, item, the item will be showed(but not under the menu). - -.. _oauth2_admin_panel_access_tokens: - -OAuth2 Access Tokens -==================== - -This small section won't cover entirely the GeoNode OAuth2 security integration, this is explained in detail in other sections of the documentation -(refer to :ref:`oauth2_fixtures_and_migration` and :ref:`oauth2_tokens_and_sessions`). - -Here we will focus mainly on the :guilabel:`Admin > DJANGO/GEONODE OAUTH TOOLKIT` panel items with a specific attention to the ``Access tokens`` management. - -The :guilabel:`Admin > DJANGO/GEONODE OAUTH TOOLKIT` panel (as shown in the figure below) allows an admin to manage everything related to -GeoNode OAuth2 grants and permissions. - -As better explained in other sections of the documentation, this is needed to correctly handle the communication between GeoNode and GeoServer. - -.. figure:: img/oauth2-tokens/admin-panel-tokens-0001.png - :align: center - - *DJANGO/GEONODE OAUTH TOOLKIT Admin panel* - -Specifically from this panel an admin can create, delete or extend OAuth2 ``Access tokens``. - -The section :ref:`oauth2_tokens_and_sessions` better explains the concepts behind OAuth2 sessions; we want just to refresh the mind here -about the basic concepts: - -* If the `SESSION_EXPIRED_CONTROL_ENABLED <../../basic/settings/index.html#session-expired-control-enabled>`_ setting is set to `True` (by default it is set to `True`) - a registered user cannot login to neither GeoNode nor GeoServer without a valid ``Access token``. - -* When logging-in into GeoNode through the sign-up form, GeoNode checks if a valid ``Access token`` exists and it creates a new one if not, or extends - the existing one if expired. - -* New ``Access tokens`` expire automatically after `ACCESS_TOKEN_EXPIRE_SECONDS <../../basic/settings/index.html#access-token-expire-seconds>`_ setting (by default 86400) - -* When an ``Access token`` expires, the user will be kicked out from the session and forced to login again - -Create a new token or extend an existing one -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -It is possible from the :guilabel:`Admin > DJANGO/GEONODE OAUTH TOOLKIT` panel to create a new ``Access token`` for a user. - -In order to do that, just click on the :guilabel:`Add` button beside ``Access tokens`` topic - -.. figure:: img/oauth2-tokens/admin-panel-tokens-0002.png - :align: center - - *Add a new ``Access token``* - -On the new form - -.. figure:: img/oauth2-tokens/admin-panel-tokens-0003.png - :align: center - - *Create an ``Access token``* - -select the followings: - -1. ``User``; use the search tool in order to select the correct user. The form want the user PK, which is a number, and **not** the username. - The search tool will do everything for you. - - .. figure:: img/oauth2-tokens/admin-panel-tokens-0003a.png - :align: center - - *Select a User* - -2. ``Source refresh token``; this is not mandatory, leave it blank. - -3. ``Token``; write here any alphanumeric string. This will be the ``access_token`` that the member can use to access the OWS services. - We suggest to use a service like https://passwordsgenerator.net/ in order to generate a strong token string. - - .. figure:: img/oauth2-tokens/admin-panel-tokens-0003b.png - :align: center - - *Select a Token* - -4. ``Application``; select **GeoServer**, this is mandatory - - .. figure:: img/oauth2-tokens/admin-panel-tokens-0003c.png - :align: center - - *Select the GeoServer Application* - -5. ``Expires``; select an expiration date by using the :guilabel:`date-time` widgets. - - .. figure:: img/oauth2-tokens/admin-panel-tokens-0003d.png - :align: center - - *Select the Token Expiration* - -6. ``Scope``; select **write**, this is mandatory. - - .. figure:: img/oauth2-tokens/admin-panel-tokens-0003e.png - :align: center - - *Select the Application Scope* - -Do not forget to :guilabel:`Save`. - -From now on, GeoNode will use this ``Access Token`` to control the user session (notice that the user need to login again if closing the browser session), -and the user will be able to access the OWS Services by using the new ``Access Token``, e.g.: - -.. code-block:: shell - - https://dev.geonode.geo-solutions.it/geoserver/ows?service=wms&version=1.3.0&request=GetCapabilities&access_token=123456 - - -Notice the ``...quest=GetCapabilities&access_token=123456`` (**access_token**) parameter at the end of the URL. - -Force a User Session to expire -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Everything said about the creation of a new ``Access Token``, applies to the deletion of the latter. - -From the same interface an admin can either select an expiration date or delete all the ``Access Tokens`` associated to a user, in order to -force its session to expire. - -Remember that the user could activate another session by logging-in again on GeoNode with its credentials. - -In order to be sure the user won't force GeoNode to refresh the token, reset first its password or de-activate it. diff --git a/admin/async/index.rst b/admin/async/index.rst deleted file mode 100644 index 68564639..00000000 --- a/admin/async/index.rst +++ /dev/null @@ -1,83 +0,0 @@ -Supervisord and Systemd -======================= - -Celery -====== - -Rabbitmq and Redis -================== - -How to: Async Upload via API -============================ -In geonode is possible to upload resources via API in async/sync way. - -Here is available a full example of upload via API -https://github.com/GeoNode/geonode/blob/582d6efda74adb8042d1d897004bbf764e6e0285/geonode/upload/api/tests.py#L416 - -Step 1 ------- - -Create a common client session, this is foundamental due the fact that geonode will check the request session. -For example with requests we will do something like: - -.. code-block:: python - - import requests - client = requests.session() - - -Note: in Django this part is already managed - -Step 2 ------- - -Call the `api/v2/uploads/upload` endpoint in PUT (is a form-data endpoint) by specifing in files a dictionary with the names and the files that we want to uploads and a data payload with the required informations. -For example: - -.. code-block:: python - - params = { - "permissions": '{ "users": {"AnonymousUser": ["view_resourcebase"]} , "groups":{}}', # layer permissions - "time": "false", - "layer_title": "layer_title", - "time": "false", - "charset": "UTF-8", - } - - files = { - "filename": <_io.BufferedReader name="filename"> - } - - client.put( - "http://localhost:8000/api/v2/uploads/upload/", - auth=HTTPBasicAuth(username, password), - data=params, - files=files, - ) - - Returns: - - dict with import id of the resource - -Step 3 ------- - -Call in the final upload page in order to trigger the actual import. -If correclty set, Geoserver will manage the upload asyncronously. - -.. code-block:: python - - client.get("http://localhost:8000/upload/final?id={import_id}") - - The `import_id` is returned from the previous step - -Step 4 ------- - -The upload as been completed on GeoNode, we should check utill Geoserver has complete his part. -To do so, is enougth to call the detailed information about the upload that we are performing - -.. code-block:: python - - client.get(f"http://localhost:8000/api/v2/uploads/{upload_id}") - -When the status is `PROCESSED` and the completion is `100%` we are able to see the resource in geonode and geoserver \ No newline at end of file diff --git a/admin/default_lang/index.rst b/admin/default_lang/index.rst deleted file mode 100644 index aac839b2..00000000 --- a/admin/default_lang/index.rst +++ /dev/null @@ -1,80 +0,0 @@ -.. _default_language: - -Changing the Default Language -============================= - -GeoNode's default language is English, but GeoNode users can change the interface language with the pulldown menu at the top-right of most GeoNode pages. Once a user selects a language GeoNode remembers that language for subsequent pages. - -GeoNode Configuration -===================== - -As root edit the geonode config file :file:`/home/geonode/geonode/geonode/settings.py` (or :file:`/etc/geonode/settings.py` if GeoNode has been installed using **apt-get**) and change ``LANGUAGE_CODE`` to the desired default language. - -.. note:: A list of language codes can be found in the global django config file :file:`/usr/local/lib/python2.7/dist-packages/django/conf/global_settings.py` (or :file:`/var/lib/geonode/lib/python2.7/site-packages/django/conf/global_settings.py` if GeoNode has been installed using **apt-get**). - -For example, to make French the default language use:: - - LANGUAGE_CODE = 'fr' - -Unfortunately Django overrides this setting, giving the language setting of a user's browser priority. For example, if ``LANGUAGE_CODE`` is set to French, but the user has configured their operating system for Spanish they may see the Spanish version when they first visit GeoNode. - - -Additional Steps -================ - -If this is not the desired behaviour, and all users should initially see the default ``LANGUAGE_CODE``, regardless of their browser's settings, do the following steps to ensure Django ignores the browser language settings. (Users can always use the pulldown language menu to change the language at any time.) - -As **root** create a new directory within GeoNode's site packages - -.. code-block:: shell - - mkdir /usr/lib/python2.7/dist-packages/setmydefaultlanguage - -or - -.. code-block:: shell - - mkdir /var/lib/geonode/lib/python2.7/site-packages/setmydefaultlanguage - -if GeoNode has been installed using **apt-get**. - -As root create and edit a new file :file:`/usr/lib/python2.7/dist-packages/setmydefaultlanguage/__init__.py` and add the following lines - -.. code-block:: python - - class ForceDefaultLanguageMiddleware(object): - """ - Ignore Accept-Language HTTP headers - - This will force the I18N machinery to always choose settings.LANGUAGE_CODE - as the default initial language, unless another one is set via sessions or cookies - - Should be installed *before* any middleware that checks request.META['HTTP_ACCEPT_LANGUAGE'], - namely django.middleware.locale.LocaleMiddleware - """ - def process_request(self, request): - if request.META.has_key('HTTP_ACCEPT_LANGUAGE'): - del request.META['HTTP_ACCEPT_LANGUAGE'] - -At the end of the GeoNode configuration file :file:`/home/geonode/geonode/geonode/settings.py` (or :file:`/etc/geonode/settings.py` if GeoNode has been installed using **apt-get**) add the following lines to ensure the above class is executed - -.. code-block:: python - - MIDDLEWARE_CLASSES += ( - 'setmydefaultlanguage.ForceDefaultLanguageMiddleware', - ) - -Restart -======= - -You will need to restart GeoNode accordingly to the installation method you have choosen. - -As an instance in case you are using `NGINX` with `UWSGI`, as root you will need to run the following commands - -.. code-block:: shell - - service uwsgi restart - service nginx restart - - -Please refer to Translating GeoNode for information on editing GeoNode pages in different languages and create new GeoNode Translations. diff --git a/admin/index.rst b/admin/index.rst deleted file mode 100644 index 954638c9..00000000 --- a/admin/index.rst +++ /dev/null @@ -1,71 +0,0 @@ -GeoNode Admins Guide -==================== - -GeoNode has an administration panel, based on the Django admin, which can be used to do some database operations. -Although most of the operations can and should be done through the normal GeoNode interface, -the admin panel provides a quick overview and management tool over the database. - -The following sections will explain more in depth what functionalities the admin panel makes available to you. -It should be highlighted that the sections not covered in this guide are meant to be managed through GeoNode UI. - -.. toctree:: - :maxdepth: 3 - - admin_panel/index - -GeoNode Management Commands -=========================== - -Management commands are utility functions for GeoNode maintenance tasks. They are usually run from an SSH/bash shell on the server running GeoNode. -Any call to python is prepended with a configuration parameter to indicate the GeoNode settings module to be used. - -.. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --help - -.. note:: If you have enabled ``local_settings.py`` the command will change as follows: - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py migrate_baseurl --help - -.. toctree:: - :caption: GeoNode Management Commands - :maxdepth: 3 - - mgmt_commands/index - -Changing the default Languages -============================== - -.. toctree:: - :caption: Changing the default Languages - :maxdepth: 3 - - default_lang/index - -GeoNode Upgrade from older versions -=================================== - -.. toctree:: - :caption: GeoNode Upgrade from older versions - :maxdepth: 3 - - upgrade/index - -GeoNode Async Signals -===================== - -.. toctree:: - :caption: GeoNode Async Signals - :maxdepth: 3 - - async/index - -Managing thesauri -========================= - -.. toctree:: - :maxdepth: 3 - - thesaurus/index diff --git a/admin/mgmt_commands/index.rst b/admin/mgmt_commands/index.rst deleted file mode 100644 index bb4b7abe..00000000 --- a/admin/mgmt_commands/index.rst +++ /dev/null @@ -1,2352 +0,0 @@ - -.. _migrate_baseurl: - -Migrate GeoNode Base URL -======================== - -The ``migrate_baseurl`` :guilabel:`Management Command` allows you to fix all the GeoNode Links whenever, for some reason, -you need to change the :guilabel:`Domain Name` of :guilabel:`IP Address` of GeoNode. - -This **must** be used also in the cases you'll need to change the network schema from ``HTTP`` to ``HTTPS``, as an instance. - -First of all let's take a look at the :guilabel:`--help` option of the ``migrate_baseurl`` -management command in order to inspect all the command options and features. - -Run - -.. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --help - -This will produce output that looks like the following - -.. code-block:: shell - - usage: manage.py migrate_baseurl [-h] [--version] [-v {0,1,2,3}] - [--settings SETTINGS] - [--pythonpath PYTHONPATH] [--traceback] - [--no-color] [-f] - [--source-address SOURCE_ADDRESS] - [--target-address TARGET_ADDRESS] - - Migrate GeoNode VM Base URL - - optional arguments: - -h, --help show this help message and exit - --version show program's version number and exit - -v {0,1,2,3}, --verbosity {0,1,2,3} - Verbosity level; 0=minimal output, 1=normal output, - 2=verbose output, 3=very verbose output - --settings SETTINGS The Python path to a settings module, e.g. - "myproject.settings.main". If this isn't provided, the - DJANGO_SETTINGS_MODULE environment variable will be - used. - --pythonpath PYTHONPATH - A directory to add to the Python path, e.g. - "/home/djangoprojects/myproject". - --traceback Raise on CommandError exceptions - --no-color Don't colorize the command output. - -f, --force Forces the execution without asking for confirmation. - --source-address SOURCE_ADDRESS - Source Address (the one currently on DB e.g. - http://192.168.1.23) - --target-address TARGET_ADDRESS - Target Address (the one to be changed e.g. http://my- - public.geonode.org) - -* **Example 1**: I want to move my GeoNode instance from ``http:\\127.0.0.1`` to ``http:\\example.org`` - - .. warning:: Make always sure you are using the **correct** settings - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --source-address=127.0.0.1 --target-address=example.org - -* **Example 2**: I want to move my GeoNode instance from ``http:\\example.org`` to ``https:\\example.org`` - - .. warning:: Make always sure you are using the **correct** settings - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --source-address=http:\\example.org --target-address=https:\\example.org - -* **Example 3**: I want to move my GeoNode instance from ``https:\\example.org`` to ``https:\\geonode.example.org`` - - .. warning:: Make always sure you are using the **correct** settings - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --source-address=example.org --target-address=geonode.example.org - -.. note:: After migrating the base URL, make sure to sanitize the links and catalog metadata also (:ref:`sync_layers_and_metadata`). - -.. _sync_layers_and_metadata: - -Update Permissions, Metadata, Legends and Download Links -======================================================== - -The following three utility :guilabel:`Management Commands`, allow to fixup: - -1. :guilabel:`Users/Groups Permissions` on :guilabel:`Datasets`; those will be refreshed and synchronized with the :guilabel:`GIS Server` ones also - -2. :guilabel:`Metadata`, :guilabel:`Legend` and :guilabel:`Download` links on :guilabel:`Datasets` and :guilabel:`Maps` - -3. Cleanup :guilabel:`Duplicated Links` and :guilabel:`Outdated Thumbnails` - -Management Command ``sync_geonode_datasets`` --------------------------------------------- - -This command allows to sync already existing permissions on Datasets. In order to change/set Datasets' permissions refer to the section :ref:`batch_sync_permissions` - -The options are: - -* **filter**; Only update data the Dataset names that match the given filter. - -* **username**; Only update data owned by the specified username. - -* **updatepermissions**; Update the Dataset permissions; synchronize it back to the GeoSpatial Server. - This option is also available from the :guilabel:`Layer Details` page. - -* **updateattributes**; Update the Dataset attributes; synchronize it back to the GeoSpatial Server. - This option is also available from the :guilabel:`Layer Details` page. - -* **updatethumbnails**; Update the Dataset thumbnail. - This option is also available from the :guilabel:`Layer Details` page. - -* **updatebbox**; Update the Dataset BBOX and LotLan BBOX. - This option is also available from the :guilabel:`Layer Details` page. - -* **remove-duplicates**; Removes duplicated Links. - -First of all let's take a look at the :guilabel:`--help` option of the ``sync_geonode_datasets`` -management command in order to inspect all the command options and features. - -Run - -.. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py sync_geonode_datasets --help - -.. note:: If you enabled ``local_settings.py`` the command will change as following: - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py sync_geonode_datasets --help - -This will produce output that looks like the following - -.. code-block:: shell - - usage: manage.py sync_geonode_datasets [-h] [--version] [-v {0,1,2,3}] - [--settings SETTINGS] - [--pythonpath PYTHONPATH] [--traceback] - [--no-color] [-i] [-d] [-f FILTER] - [-u USERNAME] [--updatepermissions] - [--updatethumbnails] [--updateattributes][--updatebbox] - - Update the GeoNode Datasets: permissions (including GeoFence database), - statistics, thumbnails - - optional arguments: - -h, --help show this help message and exit - --version show program's version number and exit - -v {0,1,2,3}, --verbosity {0,1,2,3} - Verbosity level; 0=minimal output, 1=normal output, - 2=verbose output, 3=very verbose output - --settings SETTINGS The Python path to a settings module, e.g. - "myproject.settings.main". If this isn't provided, the - DJANGO_SETTINGS_MODULE environment variable will be - used. - --pythonpath PYTHONPATH - A directory to add to the Python path, e.g. - "/home/djangoprojects/myproject". - --traceback Raise on CommandError exceptions - --no-color Don't colorize the command output. - -i, --ignore-errors Stop after any errors are encountered. - -d, --remove-duplicates - Remove duplicates first. - -f FILTER, --filter FILTER - Only update data the Datasets that match the given - filter. - -u USERNAME, --username USERNAME - Only update data owned by the specified username. - --updatepermissions Update the Dataset permissions. - --updatethumbnails Update the Dataset styles and thumbnails. - --updateattributes Update the Dataset attributes. - --updatebbox Update the Dataset BBOX. - -* **Example 1**: I want to update/sync all Datasets permissions and attributes with the GeoSpatial Server - - .. warning:: Make always sure you are using the **correct** settings - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py sync_geonode_datasets --updatepermissions --updateattributes - -* **Example 2**: I want to regenerate the Thumbnails of all the Datasets belonging to ``afabiani`` - - .. warning:: Make always sure you are using the **correct** settings - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py sync_geonode_datasets -u afabiani --updatethumbnails - -Management Command ``sync_geonode_maps`` ----------------------------------------- - -This command is basically similar to the previous one, but affects the :guilabel:`Maps`; with some limitations. - -The options are: - -* **filter**; Only update data the maps titles that match the given filter. - -* **username**; Only update data owned by the specified username. - -* **updatethumbnails**; Update the map styles and thumbnails. - This option is also available from the :guilabel:`Map Details` page. - -* **remove-duplicates**; Removes duplicated Links. - -First of all let's take a look at the :guilabel:`--help` option of the ``sync_geonode_maps`` -management command in order to inspect all the command options and features. - -Run - -.. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py sync_geonode_maps --help - -.. note:: If you enabled ``local_settings.py`` the command will change as following: - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py sync_geonode_maps --help - -This will produce output that looks like the following - -.. code-block:: shell - - usage: manage.py sync_geonode_maps [-h] [--version] [-v {0,1,2,3}] - [--settings SETTINGS] - [--pythonpath PYTHONPATH] [--traceback] - [--no-color] [-i] [-d] [-f FILTER] - [-u USERNAME] [--updatethumbnails] - - Update the GeoNode maps: permissions, thumbnails - - optional arguments: - -h, --help show this help message and exit - --version show program's version number and exit - -v {0,1,2,3}, --verbosity {0,1,2,3} - Verbosity level; 0=minimal output, 1=normal output, - 2=verbose output, 3=very verbose output - --settings SETTINGS The Python path to a settings module, e.g. - "myproject.settings.main". If this isn't provided, the - DJANGO_SETTINGS_MODULE environment variable will be - used. - --pythonpath PYTHONPATH - A directory to add to the Python path, e.g. - "/home/djangoprojects/myproject". - --traceback Raise on CommandError exceptions - --no-color Don't colorize the command output. - -i, --ignore-errors Stop after any errors are encountered. - -d, --remove-duplicates - Remove duplicates first. - -f FILTER, --filter FILTER - Only update data the maps that match the given filter. - -u USERNAME, --username USERNAME - Only update data owned by the specified username. - --updatethumbnails Update the map styles and thumbnails. - -* **Example 1**: I want to regenerate the Thumbnail of the Map ``This is a test Map`` - - .. warning:: Make always sure you are using the **correct** settings - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py sync_geonode_maps --updatethumbnails -f 'This is a test Map' - -Management Command ``set_all_datasets_metadata`` ----------------------------------------------- - -This command allows to reset **Metadata Attributes** and **Catalogue Schema** on Datasets. The command will also update the :guilabel:`CSW Catalogue` XML and Links of GeoNode. - -The options are: - -* **filter**; Only update data the Datasets that match the given filter. - -* **username**; Only update data owned by the specified username. - -* **remove-duplicates**; Update the map styles and thumbnails. - -* **delete-orphaned-thumbs**; Removes duplicated Links. -* -* **set-uuid**; will refresh the UUID based on the UUID_HANDLER if configured (Default False). -* -* **set_attrib**; If set will refresh the attributes of the resource taken from Geoserver. (Default True). -* -* **set_links**; If set will refresh the links of the resource. (Default True). - -First of all let's take a look at the :guilabel:`--help` option of the ``set_all_datasets_metadata`` -management command in order to inspect all the command options and features. - -Run - -.. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py set_all_datasets_metadata --help - -.. note:: If you enabled ``local_settings.py`` the command will change as following: - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py set_all_datasets_metadata --help - -This will produce output that looks like the following - -.. code-block:: shell - - usage: manage.py set_all_datasets_metadata [-h] [--version] [-v {0,1,2,3}] - [--settings SETTINGS] - [--pythonpath PYTHONPATH] - [--traceback] [--no-color] [-i] [-d] - [-t] [-f FILTER] [-u USERNAME] - - Resets Metadata Attributes and Schema to All Datasets - - optional arguments: - -h, --help show this help message and exit - --version show program's version number and exit - -v {0,1,2,3}, --verbosity {0,1,2,3} - Verbosity level; 0=minimal output, 1=normal output, - 2=verbose output, 3=very verbose output - --settings SETTINGS The Python path to a settings module, e.g. - "myproject.settings.main". If this isn't provided, the - DJANGO_SETTINGS_MODULE environment variable will be - used. - --pythonpath PYTHONPATH - A directory to add to the Python path, e.g. - "/home/djangoprojects/myproject". - --traceback Raise on CommandError exceptions - --no-color Don't colorize the command output. - -i, --ignore-errors Stop after any errors are encountered. - -d, --remove-duplicates - Remove duplicates first. - -t, --delete-orphaned-thumbs - Delete Orphaned Thumbnails. - -f FILTER, --filter FILTER - Only update data the Datasets that match the given - filter - -u USERNAME, --username USERNAME - Only update data owned by the specified username - -* **Example 1**: After having changed the Base URL, I want to regenerate all the Catalogue Schema and eventually remove all duplicates. - - .. warning:: Make always sure you are using the **correct** settings - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py set_all_datasets_metadata -d - -Management Command ``regenerate_xml`` ----------------------------------------------- - -The command will regenerate the :guilabel:`CSW Catalogue` XML metadata files. - -The main options are: - -* **layer**; Only process specified layers. - -* **dry-run**; Do not actually perform any change. - -When run with the :guilabel:`--help` option the full list of available options will be presented. - -.. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py regenerate_xml --help - -This will produce the following output: - -.. code-block:: shell - - usage: manage.py regenerate_xml [-h] [-l LAYERS] [--skip-logger-setup] [-d] [--version] [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color] - [--skip-checks] - - Re-create XML metadata documents - - options: - -h, --help show this help message and exit - -l LAYERS, --layer LAYERS - Only process specified layers - --skip-logger-setup Skips setup of the "geonode.br" logger, "br" handler and "br" format if not present in settings - -d, --dry-run Do not actually perform any change - --version Show program's version number and exit. - -v {0,1,2,3}, --verbosity {0,1,2,3} - Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output - --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE environment variable will be used. - --pythonpath PYTHONPATH - A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". - --traceback Raise on CommandError exceptions. - --no-color Don't colorize the command output. - --force-color Force colorization of the command output. - --skip-checks Skip system checks. - -* **Example**: - - .. warning:: Make sure you are using the **correct** settings - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py regenerate_xml -d - - -.. _load_data_into_geonode: - -Loading Data into GeoNode -========================= - -There are situations where it is not possible or not convenient to use the -:guilabel:`Upload Form` to add new Datasets to GeoNode via the web interface. -For instance: - -* The dataset is too big to be uploaded through a web interface. - -* Import data from a mass storage programmatically. - -* Import tables from a database. - -This section will walk you through the various options available to load data into your -GeoNode from GeoServer, from the command-line or programmatically. - -.. warning:: Some parts of this section have been taken from the `GeoServer `_ project and training documentation. - -.. _importlayers: - -Management Command ``importlayers`` ------------------------------------ - -The ``geonode.geoserver`` Django app includes 2 management commands that you can use to -load data in your GeoNode. - -Both of them can be invoked by using the ``manage.py`` script. - -First of all let's take a look at the :guilabel:`--help` option of the ``importlayers`` -management command in order to inspect all the command options and features. - -Run - -.. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py importlayers --help - -.. note:: If you enabled ``local_settings.py`` the command will change as following: - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py importlayers --help - -This will produce output that looks like the following - -.. code-block:: shell - - usage: manage.py importlayers [-h] [-hh HOST] [-u USERNAME] [-p PASSWORD] - [--version] [-v {0,1,2,3}] [--settings SETTINGS] - [--pythonpath PYTHONPATH] [--traceback] [--no-color] - [--force-color] [--skip-checks] - [path [path ...]] - - Brings files from a local directory, including subfolders, into a GeoNode site. - The datasets are added to the Django database, the GeoServer configuration, and the - pycsw metadata index. At this moment only files of type Esri Shapefile (.shp) and GeoTiff (.tif) are supported. - In order to perform the import, GeoNode must be up and running. - - positional arguments: - path path [path...] - - optional arguments: - -h, --help show this help message and exit - --version show program's version number and exit - -v {0,1,2,3}, --verbosity {0,1,2,3} - Verbosity level; 0=minimal output, 1=normal output, - 2=verbose output, 3=very verbose output - --settings SETTINGS The Python path to a settings module, e.g. - "myproject.settings.main". If this isn't provided, the - DJANGO_SETTINGS_MODULE environment variable will be - used. - --pythonpath PYTHONPATH - A directory to add to the Python path, e.g. - "/home/djangoprojects/myproject". - -hh HOST, --host HOST - Geonode host url - -u USERNAME, --username USERNAME - Geonode username - -p PASSWORD, --password PASSWORD - Geonode password - -While the description of most of the options should be self explanatory, its worth -reviewing some of the key options a bit more in details. - -- The :guilabel:`-hh` Identifies the GeoNode server where we want to upload our Datasets. The default value is :guilabel:`http://localhost:8000`. -- The :guilabel:`-u` Identifies the username for the login. The default value is :guilabel:`admin`. -- The :guilabel:`-p` Identifies the password for the login. The default value is :guilabel:`admin`. - -The import Datasets management command is invoked by specifying options as described -above and specifying the path to a directory that contains multiple files. For purposes of this exercise, let's use the default set of testing Datasets that ship with geonode. -You can replace this path with a directory to your own shapefiles. - -.. code-block:: shell - First let's run the GeoNode server: - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py runserver - - Then let's import the files: - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py importlayers /home/user/.virtualenvs/geonode/lib/python3.8/site-packages/gisdata/data/good/vector/ - -This command will produce the following output to your terminal - -.. code-block:: shell - - san_andres_y_providencia_poi.shp: 201 - san_andres_y_providencia_location.shp: 201 - san_andres_y_providencia_administrative.shp: 201 - san_andres_y_providencia_coastline.shp: 201 - san_andres_y_providencia_highway.shp: 201 - single_point.shp: 201 - san_andres_y_providencia_water.shp: 201 - san_andres_y_providencia_natural.shp: 201 - - 1.7456605294117646 seconds per Dataset - - Output data: { - "success": [ - "san_andres_y_providencia_poi.shp", - "san_andres_y_providencia_location.shp", - "san_andres_y_providencia_administrative.shp", - "san_andres_y_providencia_coastline.shp", - "san_andres_y_providencia_highway.shp", - "single_point.shp", - "san_andres_y_providencia_water.shp", - "san_andres_y_providencia_natural.shp" - ], - "errors": [] - } - -As output the command will print: - -.. code-block:: shell - layer_name: status code for each Layer - - upload_time spent of each Dataset - - A json with the representation of the Datasets uploaded or with some errors. - -The status code, is the response coming from GeoNode. For example 201 means that the Dataset has been correctly uploaded - -If you encounter errors while running this command, please check the GeoNode logs for more information. - -.. _updatelayers: - -Management Command ``updatelayers`` ------------------------------------ - -While it is possible to import Datasets directly from your servers filesystem into your -GeoNode, you may have an existing GeoServer that already has data in it, or you may -want to configure data from a GeoServer which is not directly supported by uploading data. - -GeoServer supports a wide range of data formats and connections to database, some of them -may not be supported as GeoNode upload formats. You can add them to your GeoNode by following the procedure described below. - -GeoServer supports 4 types of data: :guilabel:`Raster`, :guilabel:`Vector`, :guilabel:`Databases` and :guilabel:`Cascaded`. - -For a list of the supported formats for each type of data, consult the following pages: - -- https://docs.geoserver.org/latest/en/user/data/vector/index.html -- https://docs.geoserver.org/latest/en/user/data/raster/index.html -- https://docs.geoserver.org/latest/en/user/data/database/index.html -- https://docs.geoserver.org/latest/en/user/data/cascaded/index.html - -.. note:: Some of these raster or vector formats or database types require that you install specific plugins in your GeoServer in order to use the. Please consult the GeoServer documentation for more information. - -Data from a PostGIS database -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Lets walk through an example of configuring a new PostGIS database in GeoServer and then -configuring those Datasets in your GeoNode. - -First visit the GeoServer administration interface on your server. This is usually on port 8080 and is available at http://localhost:8080/geoserver/web/ - -1. You should login with the superuser credentials you setup when you first configured your GeoNode instance. - - Once you are logged in to the GeoServer Admin interface, you should see the following. - - .. figure:: img/geoserver_admin.png - :align: center - - .. note:: The number of stores, Datasets and workspaces may be different depending on what you already have configured in your GeoServer. - -2. Next you want to select the "Stores" option in the left hand menu, and then the "Add new Store" option. The following screen will be displayed. - - .. figure:: img/geoserver_new_store.png - :align: center - -3. In this case, we want to select the PostGIS store type to create a connection to our existing database. On the next screen you will need to enter the parameters to connect to your PostGIS database (alter as necessary for your own database). - - .. figure:: img/geoserver_postgis_params.png - :align: center - - .. note:: If you are unsure about any of the settings, leave them as the default. - -4. The next screen lets you configure the Datasets in your database. This will of course be different depending on the Datasets in your database. - - .. figure:: img/geoserver_publish_layers.png - :align: center - -5. Select the "Publish" button for one of the Datasets and the next screen will be displayed where you can enter metadata for this Dataset. Since we will be managing this metadata in GeoNode, we can leave these alone for now. - - .. figure:: img/geoserver_layer_params.png - :align: center - -6. The things that *must* be specified are the Declared SRS and you must select the "Compute from Data" and "Compute from native bounds" links after the SRS is specified. - - .. figure:: img/geoserver_srs.png - :align: center - - .. figure:: img/geoserver_srs_2.png - :align: center - -7. Click save and this Dataset will now be configured for use in your GeoServer. - - .. figure:: img/geoserver_layers.png - :align: center - -8. The next step is to configure these Datasets in GeoNode. The ``updatelayers`` management command can be used for this purpose. As with ``importlayers``, it's useful to look at the command line options for this command by passing the :guilabel:`--help` option - - Run - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py updatelayers --help - - .. note:: If you enabled ``local_settings.py`` the command will change as following: - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py updatelayers --help - - This will produce output that looks like the following - - .. code-block:: shell - - usage: manage.py updatelayers [-h] [--version] [-v {0,1,2,3}] - [--settings SETTINGS] [--pythonpath PYTHONPATH] - [--traceback] [--no-color] [-i] - [--skip-unadvertised] - [--skip-geonode-registered] [--remove-deleted] - [-u USER] [-f FILTER] [-s STORE] [-w WORKSPACE] - [-p PERMISSIONS] - - Update the GeoNode application with data from GeoServer - - optional arguments: - -h, --help show this help message and exit - --version show program's version number and exit - -v {0,1,2,3}, --verbosity {0,1,2,3} - Verbosity level; 0=minimal output, 1=normal output, - 2=verbose output, 3=very verbose output - --settings SETTINGS The Python path to a settings module, e.g. - "myproject.settings.main". If this isn't provided, the - DJANGO_SETTINGS_MODULE environment variable will be - used. - --pythonpath PYTHONPATH - A directory to add to the Python path, e.g. - "/home/djangoprojects/myproject". - --traceback Raise on CommandError exceptions - --no-color Don't colorize the command output. - -i, --ignore-errors Stop after any errors are encountered. - --skip-unadvertised Skip processing unadvertised Datasets from GeoSever. - --skip-geonode-registered - Just processing GeoServer Datasets still not registered - in GeoNode. - --remove-deleted Remove GeoNode Datasets that have been deleted from - GeoSever. - -u USER, --user USER Name of the user account which should own the imported - Datasets - -f FILTER, --filter FILTER - Only update data the Datasets that match the given - filter - -s STORE, --store STORE - Only update data the Datasets for the given geoserver - store name - -w WORKSPACE, --workspace WORKSPACE - Only update data on specified workspace - -p PERMISSIONS, --permissions PERMISSIONS - Permissions to apply to each Dataset - -The update procedure includes the following steps: - - - The process fetches from GeoServer the relevant WMS layers (all, by store or by workspace) - - - If a filter is defined, the GeoServer layers are filtered - - - For each of the layers, a GeoNode dataset is created based on the metadata registered on GeoServer (title, abstract, bounds) - - - New layers are added, existing layers are replaced, unless the :guilabel:`--skip-geonode-registered` option is used - - - The GeoNode layers, added in previous runs of the update process, which are no longer available in GeoServer are removed, if the :guilabel:`--remove-delete` option is set - -.. warning:: One of the :guilabel:`--workspace` or :guilabel:`--store` must be always specified if you want to ingest Datasets belonging to a specific ``Workspace``. As an instance, in order to ingest the Datasets present into the ``geonode`` workspace, you will need to specify the option ``-w geonode``. - -9. Let's ingest the Dataset ``geonode:_1_SARMIENTO_ENERO_2018`` from the ``geonode`` workspace. - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py updatelayers -w geonode -f _1_SARMIENTO_ENERO_2018 - - .. code-block:: shell - - Inspecting the available Datasets in GeoServer ... - Found 1 Datasets, starting processing - /usr/local/lib/python2.7/site-packages/owslib/iso.py:117: FutureWarning: the .identification and .serviceidentification properties will merge into .identification being a list of properties. This is currently implemented in .identificationinfo. Please see https://github.com/geopython/OWSLib/issues/38 for more information - FutureWarning) - /usr/local/lib/python2.7/site-packages/owslib/iso.py:495: FutureWarning: The .keywords and .keywords2 properties will merge into the .keywords property in the future, with .keywords becoming a list of MD_Keywords instances. This is currently implemented in .keywords2. Please see https://github.com/geopython/OWSLib/issues/301 for more information - FutureWarning) - Content-Type: text/html; charset="utf-8" - MIME-Version: 1.0 - Content-Transfer-Encoding: 7bit - Subject: [master.demo.geonode.org] A new Dataset has been uploaded - From: webmaster@localhost - To: mapadeldelito@chubut.gov.ar - Reply-To: webmaster@localhost - Date: Tue, 08 Oct 2019 12:26:17 -0000 - Message-ID: <20191008122617.28801.94967@d3cf85425231> - - - - You have received the following notice from master.demo.geonode.org: -

- - The user admin uploaded the following Dataset:
- _1_SARMIENTO_ENERO_2018
- You can visit the Dataset's detail page here: http://master.demo.geonode.org/Datasets/geonode:_1_SARMIENTO_ENERO_2018 - -

-

- To change how you receive notifications, please go to http://master.demo.geonode.org -

- - - ------------------------------------------------------------------------------- - Content-Type: text/html; charset="utf-8" - MIME-Version: 1.0 - Content-Transfer-Encoding: 7bit - Subject: [master.demo.geonode.org] A new Dataset has been uploaded - From: webmaster@localhost - To: giacomo8vinci@gmail.com - Reply-To: webmaster@localhost - Date: Tue, 08 Oct 2019 12:26:17 -0000 - Message-ID: <20191008122617.28801.53784@d3cf85425231> - - - - You have received the following notice from master.demo.geonode.org: -

- - The user admin uploaded the following Dataset:
- _1_SARMIENTO_ENERO_2018
- You can visit the Dataset's detail page here: http://master.demo.geonode.org/Datasets/geonode:_1_SARMIENTO_ENERO_2018 - -

-

- To change how you receive notifications, please go to http://master.demo.geonode.org -

- - - ------------------------------------------------------------------------------- - Content-Type: text/html; charset="utf-8" - MIME-Version: 1.0 - Content-Transfer-Encoding: 7bit - Subject: [master.demo.geonode.org] A new Dataset has been uploaded - From: webmaster@localhost - To: fmgagliano@gmail.com - Reply-To: webmaster@localhost - Date: Tue, 08 Oct 2019 12:26:17 -0000 - Message-ID: <20191008122617.28801.26265@d3cf85425231> - - - - You have received the following notice from master.demo.geonode.org: -

- - The user admin uploaded the following Dataset:
- _1_SARMIENTO_ENERO_2018
- You can visit the Dataset's detail page here: http://master.demo.geonode.org/Datasets/geonode:_1_SARMIENTO_ENERO_2018 - -

-

- To change how you receive notifications, please go to http://master.demo.geonode.org -

- - - ------------------------------------------------------------------------------- - Found geoserver resource for this Dataset: _1_SARMIENTO_ENERO_2018 - ... Creating Default Resource Links for Layer [geonode:_1_SARMIENTO_ENERO_2018] - -- Resource Links[Prune old links]... - -- Resource Links[Prune old links]...done! - -- Resource Links[Compute parameters for the new links]... - -- Resource Links[Create Raw Data download link]... - -- Resource Links[Create Raw Data download link]...done! - -- Resource Links[Set download links for WMS, WCS or WFS and KML]... - -- Resource Links[Set download links for WMS, WCS or WFS and KML]...done! - -- Resource Links[Legend link]... - -- Resource Links[Legend link]...done! - -- Resource Links[Thumbnail link]... - -- Resource Links[Thumbnail link]...done! - -- Resource Links[OWS Links]... - -- Resource Links[OWS Links]...done! - Content-Type: text/html; charset="utf-8" - MIME-Version: 1.0 - Content-Transfer-Encoding: 7bit - Subject: [master.demo.geonode.org] A Dataset has been updated - From: webmaster@localhost - To: mapadeldelito@chubut.gov.ar - Reply-To: webmaster@localhost - Date: Tue, 08 Oct 2019 12:26:20 -0000 - Message-ID: <20191008122620.28801.81598@d3cf85425231> - - - - You have received the following notice from master.demo.geonode.org: -

- - The following Dataset was updated:
- _1_SARMIENTO_ENERO_2018, owned by admin
- You can visit the Dataset's detail page here: http://master.demo.geonode.org/Datasets/geonode:_1_SARMIENTO_ENERO_2018 - -

-

- To change how you receive notifications, please go to http://master.demo.geonode.org -

- - - ------------------------------------------------------------------------------- - Content-Type: text/html; charset="utf-8" - MIME-Version: 1.0 - Content-Transfer-Encoding: 7bit - Subject: [master.demo.geonode.org] A Dataset has been updated - From: webmaster@localhost - To: giacomo8vinci@gmail.com - Reply-To: webmaster@localhost - Date: Tue, 08 Oct 2019 12:26:20 -0000 - Message-ID: <20191008122620.28801.93778@d3cf85425231> - - - - You have received the following notice from master.demo.geonode.org: -

- - The following Dataset was updated:
- _1_SARMIENTO_ENERO_2018, owned by admin
- You can visit the Dataset's detail page here: http://master.demo.geonode.org/Datasets/geonode:_1_SARMIENTO_ENERO_2018 - -

-

- To change how you receive notifications, please go to http://master.demo.geonode.org -

- - - ------------------------------------------------------------------------------- - Content-Type: text/html; charset="utf-8" - MIME-Version: 1.0 - Content-Transfer-Encoding: 7bit - Subject: [master.demo.geonode.org] A Dataset has been updated - From: webmaster@localhost - To: fmgagliano@gmail.com - Reply-To: webmaster@localhost - Date: Tue, 08 Oct 2019 12:26:20 -0000 - Message-ID: <20191008122620.28801.58585@d3cf85425231> - - - - You have received the following notice from master.demo.geonode.org: -

- - The following Dataset was updated:
- _1_SARMIENTO_ENERO_2018, owned by admin
- You can visit the Dataset's detail page here: http://master.demo.geonode.org/Datasets/geonode:_1_SARMIENTO_ENERO_2018 - -

-

- To change how you receive notifications, please go to http://master.demo.geonode.org -

- - - ------------------------------------------------------------------------------- - Found geoserver resource for this Dataset: _1_SARMIENTO_ENERO_2018 - /usr/local/lib/python2.7/site-packages/geoserver/style.py:80: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead. - if not user_style: - /usr/local/lib/python2.7/site-packages/geoserver/style.py:84: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead. - if user_style: - ... Creating Default Resource Links for Layer [geonode:_1_SARMIENTO_ENERO_2018] - -- Resource Links[Prune old links]... - -- Resource Links[Prune old links]...done! - -- Resource Links[Compute parameters for the new links]... - -- Resource Links[Create Raw Data download link]... - -- Resource Links[Create Raw Data download link]...done! - -- Resource Links[Set download links for WMS, WCS or WFS and KML]... - -- Resource Links[Set download links for WMS, WCS or WFS and KML]...done! - -- Resource Links[Legend link]... - -- Resource Links[Legend link]...done! - -- Resource Links[Thumbnail link]... - -- Resource Links[Thumbnail link]...done! - -- Resource Links[OWS Links]... - -- Resource Links[OWS Links]...done! - [created] Layer _1_SARMIENTO_ENERO_2018 (1/1) - - - Finished processing 1 Datasets in 5.0 seconds. - - 1 Created Datasets - 0 Updated Datasets - 0 Failed Datasets - 5.000000 seconds per Dataset - -.. note:: In case you don't specify the :guilabel:`-f` option, the Datasets that already exists in your GeoNode will be just updated and the configuration synchronized between GeoServer and GeoNode. - -.. warning:: When updating **from** GeoServer, the configuration on GeoNode will be changed! - -.. _gdal_ogr_data_processing: - -Using ``GDAL`` and ``OGR`` to convert your Data for use in GeoNode ------------------------------------------------------------------- - -GeoNode supports uploading data in :guilabel:`ESRI shapefiles`, :guilabel:`GeoTIFF`, :guilabel:`CSV`, :guilabel:`GeoJSON`, :guilabel:`ASCII-GRID` and :guilabel:`KML / KMZ` formats (for the last three formats only if you are using the ``geonode.importer`` backend). - -* If your data is in other formats, you will need to convert it into one of these formats for use in GeoNode. - -* If your :guilabel:`Raster` data is not correctly processed, it might be almost unusable with GeoServer and GeoNode. You will need to process it using `GDAL`. - -You need to make sure that you have the GDAL library installed on your system. -On Ubuntu you can install this package with the following command: - -.. code-block:: shell - - sudo apt-get install gdal-bin - -OGR (Vector Data) -^^^^^^^^^^^^^^^^^ - -OGR is used to manipulate vector data. In this example, we will use MapInfo .tab files and convert them to shapefiles with the ogr2ogr command. We will use sample MapInfo files from the website linked below. - -http://services.land.vic.gov.au/landchannel/content/help?name=sampledata - -You can download the Admin;(Postcode) Dataset by issuing the following command:: - - $ wget http://services.land.vic.gov.au/sampledata/shape/admin_postcode_vm.zip - -You will need to unzip this dataset by issuing the following command:: - - $ unzip admin_postcode_vm.zip - -This will leave you with the following files in the directory where you executed the above commands:: - - |-- ANZVI0803003025.htm - |-- DSE_Data_Access_Licence.pdf - |-- VMADMIN.POSTCODE_POLYGON.xml - |-- admin_postcode_vm.zip - --- vicgrid94 - --- mif - --- lga_polygon - --- macedon\ ranges - |-- EXTRACT_POLYGON.mid - |-- EXTRACT_POLYGON.mif - --- VMADMIN - |-- POSTCODE_POLYGON.mid - --- POSTCODE_POLYGON.mif - -First, lets inspect this file set using the following command:: - - $ ogrinfo -so vicgrid94/mif/lga_polygon/macedon\ ranges/VMADMIN/POSTCODE_POLYGON.mid POSTCODE_POLYGON - -The output will look like the following:: - - Had to open data source read-only. - INFO: Open of `vicgrid94/mif/lga_polygon/macedon ranges/VMADMIN/POSTCODE_POLYGON.mid' - using driver `MapInfo File' successful. - - Layer name: POSTCODE_POLYGON - Geometry: 3D Unknown (any) - Feature Count: 26 - Extent: (2413931.249367, 2400162.366186) - (2508952.174431, 2512183.046927) - Layer SRS WKT: - PROJCS["unnamed", - GEOGCS["unnamed", - DATUM["GDA94", - SPHEROID["GRS 80",6378137,298.257222101], - TOWGS84[0,0,0,-0,-0,-0,0]], - PRIMEM["Greenwich",0], - UNIT["degree",0.0174532925199433]], - PROJECTION["Lambert_Conformal_Conic_2SP"], - PARAMETER["standard_parallel_1",-36], - PARAMETER["standard_parallel_2",-38], - PARAMETER["latitude_of_origin",-37], - PARAMETER["central_meridian",145], - PARAMETER["false_easting",2500000], - PARAMETER["false_northing",2500000], - UNIT["Meter",1]] - PFI: String (10.0) - POSTCODE: String (4.0) - FEATURE_TYPE: String (6.0) - FEATURE_QUALITY_ID: String (20.0) - PFI_CREATED: Date (10.0) - UFI: Real (12.0) - UFI_CREATED: Date (10.0) - UFI_OLD: Real (12.0) - -This gives you information about the number of features, the extent, the projection and the attributes of this Dataset. - -Next, lets go ahead and convert this Dataset into a shapefile by issuing the following command:: - - $ ogr2ogr -t_srs EPSG:4326 postcode_polygon.shp vicgrid94/mif/lga_polygon/macedon\ ranges/VMADMIN/POSTCODE_POLYGON.mid POSTCODE_POLYGON - -Note that we have also reprojected the Dataset to the WGS84 spatial reference system with the -t_srs ogr2ogr option. - -The output of this command will look like the following:: - - Warning 6: Normalized/laundered field name: 'FEATURE_TYPE' to 'FEATURE_TY' - Warning 6: Normalized/laundered field name: 'FEATURE_QUALITY_ID' to 'FEATURE_QU' - Warning 6: Normalized/laundered field name: 'PFI_CREATED' to 'PFI_CREATE' - Warning 6: Normalized/laundered field name: 'UFI_CREATED' to 'UFI_CREATE' - -This output indicates that some of the field names were truncated to fit into the constraint that attributes in shapefiles are only 10 characters long. - -You will now have a set of files that make up the postcode_polygon.shp shapefile set. We can inspect them by issuing the following command:: - - $ ogrinfo -so postcode_polygon.shp postcode_polygon - -The output will look similar to the output we saw above when we inspected the MapInfo file we converted from:: - - INFO: Open of `postcode_polygon.shp' - using driver `ESRI Shapefile' successful. - - Layer name: postcode_polygon - Geometry: Polygon - Feature Count: 26 - Extent: (144.030296, -37.898156) - (145.101137, -36.888878) - Layer SRS WKT: - GEOGCS["GCS_WGS_1984", - DATUM["WGS_1984", - SPHEROID["WGS_84",6378137,298.257223563]], - PRIMEM["Greenwich",0], - UNIT["Degree",0.017453292519943295]] - PFI: String (10.0) - POSTCODE: String (4.0) - FEATURE_TY: String (6.0) - FEATURE_QU: String (20.0) - PFI_CREATE: Date (10.0) - UFI: Real (12.0) - UFI_CREATE: Date (10.0) - UFI_OLD: Real (12.0) - -These files can now be loaded into your GeoNode instance via the normal uploader. - -Visit the upload page in your GeoNode, drag and drop the files that composes the shapefile that you have generated using the GDAL ogr2ogr command (postcode_polygon.dbf, postcode_polygon.prj, postcode_polygon.shp, postcode_polygon.shx). Give the permissions as needed and then click the "Upload files" button. - -.. figure:: img/upload_shapefile.png - :align: center - -As soon as the import process completes, you will have the possibility to go straight to the Dataset info page ("Layer Info" button), or to edit the metadata for that Dataset ("Edit Metadata" button), or to manage the styles for that Dataset ("Manage Styles"). - -.. figure:: img/layer_info_vector.png - :align: center - -GDAL (Raster Data) -^^^^^^^^^^^^^^^^^^ - -Let's see several examples on how to either convert raster data into different formats and/or process it to get the best performances. - -References: - -a) https://geoserver.geo-solutions.it/edu/en/raster_data/processing.html - -b) https://geoserver.geo-solutions.it/edu/en/raster_data/advanced_gdal/ - -Raster Data Conversion: Arc/Info Binary and ASCII Grid data into GeoTIFF format. -++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -Let's assume we have a sample ASCII Grid file compressed as an archive. - -.. code-block:: shell - - # Un-tar the files - tar -xvf sample_asc.tar - -You will be left with the following files on your filesystem: - -.. code-block:: shell - - |-- batemans_ele - |   |-- dblbnd.adf - |   |-- hdr.adf - |   |-- metadata.xml - |   |-- prj.adf - |   |-- sta.adf - |   |-- w001001.adf - |   |-- w001001x.adf - |-- batemans_elevation.asc - -The file ``batemans_elevation.asc`` is an Arc/Info ASCII Grid file and the files in -the batemans_ele directory are an Arc/Info Binary Grid file. - -You can use the ``gdalinfo`` command to inspect both of these files by executing the -following command: - -.. code-block:: shell - - gdalinfo batemans_elevation.asc - -The output should look like the following: - -.. code-block:: shell - - Driver: AAIGrid/Arc/Info ASCII Grid - Files: batemans_elevation.asc - Size is 155, 142 - Coordinate System is `' - Origin = (239681.000000000000000,6050551.000000000000000) - Pixel Size = (100.000000000000000,-100.000000000000000) - Corner Coordinates: - Upper Left ( 239681.000, 6050551.000) - Lower Left ( 239681.000, 6036351.000) - Upper Right ( 255181.000, 6050551.000) - Lower Right ( 255181.000, 6036351.000) - Center ( 247431.000, 6043451.000) - Band 1 Block=155x1 Type=Float32, ColorInterp=Undefined - NoData Value=-9999 - -You can then inspect the batemans_ele files by executing the following command: - -.. code-block:: shell - - gdalinfo batemans_ele - -And this should be the corresponding output: - -.. code-block:: shell - - Driver: AIG/Arc/Info Binary Grid - Files: batemans_ele - batemans_ele/dblbnd.adf - batemans_ele/hdr.adf - batemans_ele/metadata.xml - batemans_ele/prj.adf - batemans_ele/sta.adf - batemans_ele/w001001.adf - batemans_ele/w001001x.adf - Size is 155, 142 - Coordinate System is: - PROJCS["unnamed", - GEOGCS["GDA94", - DATUM["Geocentric_Datum_of_Australia_1994", - SPHEROID["GRS 1980",6378137,298.257222101, - AUTHORITY["EPSG","7019"]], - TOWGS84[0,0,0,0,0,0,0], - AUTHORITY["EPSG","6283"]], - PRIMEM["Greenwich",0, - AUTHORITY["EPSG","8901"]], - UNIT["degree",0.0174532925199433, - AUTHORITY["EPSG","9122"]], - AUTHORITY["EPSG","4283"]], - PROJECTION["Transverse_Mercator"], - PARAMETER["latitude_of_origin",0], - PARAMETER["central_meridian",153], - PARAMETER["scale_factor",0.9996], - PARAMETER["false_easting",500000], - PARAMETER["false_northing",10000000], - UNIT["METERS",1]] - Origin = (239681.000000000000000,6050551.000000000000000) - Pixel Size = (100.000000000000000,-100.000000000000000) - Corner Coordinates: - Upper Left ( 239681.000, 6050551.000) (150d 7'28.35"E, 35d39'16.56"S) - Lower Left ( 239681.000, 6036351.000) (150d 7'11.78"E, 35d46'56.89"S) - Upper Right ( 255181.000, 6050551.000) (150d17'44.07"E, 35d39'30.83"S) - Lower Right ( 255181.000, 6036351.000) (150d17'28.49"E, 35d47'11.23"S) - Center ( 247431.000, 6043451.000) (150d12'28.17"E, 35d43'13.99"S) - Band 1 Block=256x4 Type=Float32, ColorInterp=Undefined - Min=-62.102 Max=142.917 - NoData Value=-3.4028234663852886e+38 - -You will notice that the ``batemans_elevation.asc`` file does *not* contain projection information while the ``batemans_ele`` file does. -Because of this, let's use the ``batemans_ele`` files for this exercise and convert them to a GeoTiff for use in GeoNode. -We will also reproject this file into WGS84 in the process. This can be accomplished with the following command. - -.. code-block:: shell - - gdalwarp -t_srs EPSG:4326 batemans_ele batemans_ele.tif - -The output will show you the progress of the conversion and when it is complete, -you will be left with a ``batemans_ele.tif`` file that you can upload to your GeoNode. - -You can inspect this file with the gdalinfo command: - -.. code-block:: shell - - gdalinfo batemans_ele.tif - -Which will produce the following output: - -.. code-block:: shell - - Driver: GTiff/GeoTIFF - Files: batemans_ele.tif - Size is 174, 130 - Coordinate System is: - GEOGCS["WGS 84", - DATUM["WGS_1984", - SPHEROID["WGS 84",6378137,298.257223563, - AUTHORITY["EPSG","7030"]], - AUTHORITY["EPSG","6326"]], - PRIMEM["Greenwich",0], - UNIT["degree",0.0174532925199433], - AUTHORITY["EPSG","4326"]] - Origin = (150.119938943722502,-35.654598806259330) - Pixel Size = (0.001011114155919,-0.001011114155919) - Metadata: - AREA_OR_POINT=Area - Image Structure Metadata: - INTERLEAVE=BAND - Corner Coordinates: - Upper Left ( 150.1199389, -35.6545988) (150d 7'11.78"E, 35d39'16.56"S) - Lower Left ( 150.1199389, -35.7860436) (150d 7'11.78"E, 35d47' 9.76"S) - Upper Right ( 150.2958728, -35.6545988) (150d17'45.14"E, 35d39'16.56"S) - Lower Right ( 150.2958728, -35.7860436) (150d17'45.14"E, 35d47' 9.76"S) - Center ( 150.2079059, -35.7203212) (150d12'28.46"E, 35d43'13.16"S) - Band 1 Block=174x11 Type=Float32, ColorInterp=Gray - -Raster Data Optimization: Optimizing and serving big raster data -++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -(ref: https://geoserver.geo-solutions.it/edu/en/raster_data/advanced_gdal/example5.html) - -When dealing with big raster datasets it could be very useful to use tiles. - -Tiling allows large raster datasets to be broken-up into manageable pieces and are fundamental -in defining and implementing a higher level raster I/O interface. - -In this example we will use the original dataset of the ``chiangMai_ortho_optimized`` public raster Dataset which -is currently available on the Thai `CHIANG MAI Urban Flooding GeoNode platform `_. - -This dataset contains an orthorectified image stored as RGBa GeoTiff with 4 bands, -three bands for the RGB and one for transparency (the alpha channel). - -Calling the gdalinfo command to see detailed information: - -.. code-block:: shell - - gdalinfo chiangMai_ortho.tif - -It will produce the following results: - -.. code-block:: shell - - Driver: GTiff/GeoTIFF - Files: chiangMai_ortho.tif - Size is 63203, 66211 - Coordinate System is: - PROJCS["WGS 84 / UTM zone 47N", - GEOGCS["WGS 84", - DATUM["WGS_1984", - SPHEROID["WGS 84",6378137,298.257223563, - AUTHORITY["EPSG","7030"]], - AUTHORITY["EPSG","6326"]], - PRIMEM["Greenwich",0, - AUTHORITY["EPSG","8901"]], - UNIT["degree",0.0174532925199433, - AUTHORITY["EPSG","9122"]], - AUTHORITY["EPSG","4326"]], - PROJECTION["Transverse_Mercator"], - PARAMETER["latitude_of_origin",0], - PARAMETER["central_meridian",99], - PARAMETER["scale_factor",0.9996], - PARAMETER["false_easting",500000], - PARAMETER["false_northing",0], - UNIT["metre",1, - AUTHORITY["EPSG","9001"]], - AXIS["Easting",EAST], - AXIS["Northing",NORTH], - AUTHORITY["EPSG","32647"]] - Origin = (487068.774750000040513,2057413.889810000080615) - Pixel Size = (0.028850000000000,-0.028850000000000) - Metadata: - AREA_OR_POINT=Area - TIFFTAG_SOFTWARE=pix4dmapper - Image Structure Metadata: - COMPRESSION=LZW - INTERLEAVE=PIXEL - Corner Coordinates: - Upper Left ( 487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N) - Lower Left ( 487068.775, 2055503.702) ( 98d52'38.77"E, 18d35'25.19"N) - Upper Right ( 488892.181, 2057413.890) ( 98d53'40.94"E, 18d36'27.38"N) - Lower Right ( 488892.181, 2055503.702) ( 98d53'40.98"E, 18d35'25.22"N) - Center ( 487980.478, 2056458.796) ( 98d53' 9.85"E, 18d35'56.28"N) - Band 1 Block=63203x1 Type=Byte, ColorInterp=Red - NoData Value=-10000 - Mask Flags: PER_DATASET ALPHA - Band 2 Block=63203x1 Type=Byte, ColorInterp=Green - NoData Value=-10000 - Mask Flags: PER_DATASET ALPHA - Band 3 Block=63203x1 Type=Byte, ColorInterp=Blue - NoData Value=-10000 - Mask Flags: PER_DATASET ALPHA - Band 4 Block=63203x1 Type=Byte, ColorInterp=Alpha - NoData Value=-10000 - -As you can see, this GeoTiff has not been tiled. For accessing subsets though, tiling can make a difference. With tiling, data are stored and compressed in blocks (tiled) rather than line by line (stripped). - -In the command output above it is visible that each band has blocks with the same width of the image (63203) and a unit length. The grids in the picture below show an image with equally sized tiles (left) and the same number of strips (right). To read data from the red subset, the intersected area will have to be decompressed. - - .. figure:: img/tiled_vs_stripped.png - :align: center - -In the tiled image we will have to decompress only 16 tiles, whereas in the stripped image on the right we’ll have to decompress many more strips. - -Drone images data usually have a stripped structure so, in most cases, they need to be optimized to increase performances. - -Let’s take a look at the gdal_translate command used to optimize our GeoTiff: - -.. code-block:: shell - - gdal_translate -co TILED=YES -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR - --config GDAL_TIFF_INTERNAL_MASK YES -b 1 -b 2 -b 3 -mask 4 - chiangMai_ortho.tif - chiangMai_ortho_optimized.tif - -.. note:: For the details about the command parameters see https://geoserver.geo-solutions.it/edu/en/raster_data/advanced_gdal/example5.html - -Once the process ended, call the gdalinfo command on the resulting tif file: - -.. code-block:: shell - - gdalinfo chiangMai_ortho_optimized.tif - -The following should be the results: - -.. code-block:: shell - - Driver: GTiff/GeoTIFF - Files: chiangMai_ortho_optimized.tif - Size is 63203, 66211 - Coordinate System is: - PROJCS["WGS 84 / UTM zone 47N", - GEOGCS["WGS 84", - DATUM["WGS_1984", - SPHEROID["WGS 84",6378137,298.257223563, - AUTHORITY["EPSG","7030"]], - AUTHORITY["EPSG","6326"]], - PRIMEM["Greenwich",0, - AUTHORITY["EPSG","8901"]], - UNIT["degree",0.0174532925199433, - AUTHORITY["EPSG","9122"]], - AUTHORITY["EPSG","4326"]], - PROJECTION["Transverse_Mercator"], - PARAMETER["latitude_of_origin",0], - PARAMETER["central_meridian",99], - PARAMETER["scale_factor",0.9996], - PARAMETER["false_easting",500000], - PARAMETER["false_northing",0], - UNIT["metre",1, - AUTHORITY["EPSG","9001"]], - AXIS["Easting",EAST], - AXIS["Northing",NORTH], - AUTHORITY["EPSG","32647"]] - Origin = (487068.774750000040513,2057413.889810000080615) - Pixel Size = (0.028850000000000,-0.028850000000000) - Metadata: - AREA_OR_POINT=Area - TIFFTAG_SOFTWARE=pix4dmapper - Image Structure Metadata: - COMPRESSION=YCbCr JPEG - INTERLEAVE=PIXEL - SOURCE_COLOR_SPACE=YCbCr - Corner Coordinates: - Upper Left ( 487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N) - Lower Left ( 487068.775, 2055503.702) ( 98d52'38.77"E, 18d35'25.19"N) - Upper Right ( 488892.181, 2057413.890) ( 98d53'40.94"E, 18d36'27.38"N) - Lower Right ( 488892.181, 2055503.702) ( 98d53'40.98"E, 18d35'25.22"N) - Center ( 487980.478, 2056458.796) ( 98d53' 9.85"E, 18d35'56.28"N) - Band 1 Block=256x256 Type=Byte, ColorInterp=Red - NoData Value=-10000 - Mask Flags: PER_DATASET - Band 2 Block=256x256 Type=Byte, ColorInterp=Green - NoData Value=-10000 - Mask Flags: PER_DATASET - Band 3 Block=256x256 Type=Byte, ColorInterp=Blue - NoData Value=-10000 - Mask Flags: PER_DATASET - -Our GeoTiff is now tiled with 256x256 tiles, has 3 bands and a 1-bit mask for nodata. - -We can also add internal overviews to the file using the gdaladdo command: - -.. code-block:: shell - - gdaladdo -r average chiangMai_ortho_optimized.tif 2 4 8 16 32 64 128 256 512 - -Overviews are duplicate versions of your original data, but resampled to a lower resolution, they can also be compressed with various algorithms, much in the same way as the original dataset. - -By default, overviews take the same compression type and transparency masks of the input dataset (applied through the gdal_translate command), so the parameters to be specified are: - - * :guilabel:`-r average`: computes the average of all non-NODATA contributing pixels - * :guilabel:`2 4 8 16 32 64 128 256 512`: the list of integral overview levels to build (from gdal version 2.3 levels are no longer required to build overviews) - -Calling the gdalinfo command again: - -.. code-block:: shell - - gdalinfo chiangMai_ortho_optimized.tif - -It results in: - -.. code-block:: shell - - Driver: GTiff/GeoTIFF - Files: chiangMai_ortho_optimized.tif - Size is 63203, 66211 - Coordinate System is: - PROJCS["WGS 84 / UTM zone 47N", - GEOGCS["WGS 84", - DATUM["WGS_1984", - SPHEROID["WGS 84",6378137,298.257223563, - AUTHORITY["EPSG","7030"]], - AUTHORITY["EPSG","6326"]], - PRIMEM["Greenwich",0, - AUTHORITY["EPSG","8901"]], - UNIT["degree",0.0174532925199433, - AUTHORITY["EPSG","9122"]], - AUTHORITY["EPSG","4326"]], - PROJECTION["Transverse_Mercator"], - PARAMETER["latitude_of_origin",0], - PARAMETER["central_meridian",99], - PARAMETER["scale_factor",0.9996], - PARAMETER["false_easting",500000], - PARAMETER["false_northing",0], - UNIT["metre",1, - AUTHORITY["EPSG","9001"]], - AXIS["Easting",EAST], - AXIS["Northing",NORTH], - AUTHORITY["EPSG","32647"]] - Origin = (487068.774750000040513,2057413.889810000080615) - Pixel Size = (0.028850000000000,-0.028850000000000) - Metadata: - AREA_OR_POINT=Area - TIFFTAG_SOFTWARE=pix4dmapper - Image Structure Metadata: - COMPRESSION=YCbCr JPEG - INTERLEAVE=PIXEL - SOURCE_COLOR_SPACE=YCbCr - Corner Coordinates: - Upper Left ( 487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N) - Lower Left ( 487068.775, 2055503.702) ( 98d52'38.77"E, 18d35'25.19"N) - Upper Right ( 488892.181, 2057413.890) ( 98d53'40.94"E, 18d36'27.38"N) - Lower Right ( 488892.181, 2055503.702) ( 98d53'40.98"E, 18d35'25.22"N) - Center ( 487980.478, 2056458.796) ( 98d53' 9.85"E, 18d35'56.28"N) - Band 1 Block=256x256 Type=Byte, ColorInterp=Red - NoData Value=-10000 - Overviews: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130 - Mask Flags: PER_DATASET - Overviews of mask band: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130 - Band 2 Block=256x256 Type=Byte, ColorInterp=Green - NoData Value=-10000 - Overviews: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130 - Mask Flags: PER_DATASET - Overviews of mask band: 31602x3Results in:3106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130 - Band 3 Block=256x256 Type=Byte, ColorInterp=Blue - NoData Value=-10000 - Overviews: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130 - Mask Flags: PER_DATASET - Overviews of mask band: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130 - -Notice that the transparency masks of internal overviews have been applied (their compression does not show up in the file metadata). - -UAVs usually provide also two other types of data: ``DTM (Digital Terrain Model)`` and ``DSM (Digital Surface Model)``. - -Those data require different processes to be optimized. Let’s look at some examples to better understand how to use gdal to accomplish that task. - -From the `CHIANG MAI Urban Flooding GeoNode platform `_ platform it is currently available the ``chiangMai_dtm_optimized`` Dataset, -let’s download its original dataset. - -This dataset should contain the DTM file ``chiangMai_dtm.tif``. - -Calling the gdalinfo command on it: - -.. code-block:: shell - - gdalinfo chiangMai_dtm.tif - -The following information will be displayed: - -.. code-block:: shell - - Driver: GTiff/GeoTIFF - Files: chiangMai_dtm.tif - Size is 12638, 13240 - Coordinate System is: - PROJCS["WGS 84 / UTM zone 47N", - GEOGCS["WGS 84", - DATUM["WGS_1984", - SPHEROID["WGS 84",6378137,298.257223563, - AUTHORITY["EPSG","7030"]], - AUTHORITY["EPSG","6326"]], - PRIMEM["Greenwich",0, - AUTHORITY["EPSG","8901"]], - UNIT["degree",0.0174532925199433, - AUTHORITY["EPSG","9122"]], - AUTHORITY["EPSG","4326"]], - PROJECTION["Transverse_Mercator"], - PARAMETER["latitude_of_origin",0], - PARAMETER["central_meridian",99], - PARAMETER["scale_factor",0.9996], - PARAMETER["false_easting",500000], - PARAMETER["false_northing",0], - UNIT["metre",1, - AUTHORITY["EPSG","9001"]], - AXIS["Easting",EAST], - AXIS["Northing",NORTH], - AUTHORITY["EPSG","32647"]] - Origin = (487068.774750000040513,2057413.889810000080615) - Pixel Size = (0.144270000000000,-0.144270000000000) - Metadata: - AREA_OR_POINT=Area - TIFFTAG_SOFTWARE=pix4dmapper - Image Structure Metadata: - COMPRESSION=LZW - INTERLEAVE=BAND - Corner Coordinates: - Upper Left ( 487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N) - Lower Left ( 487068.775, 2055503.755) ( 98d52'38.77"E, 18d35'25.19"N) - Upper Right ( 488892.059, 2057413.890) ( 98d53'40.94"E, 18d36'27.37"N) - Lower Right ( 488892.059, 2055503.755) ( 98d53'40.98"E, 18d35'25.22"N) - Center ( 487980.417, 2056458.822) ( 98d53' 9.85"E, 18d35'56.28"N) - Band 1 Block=12638x1 Type=Float32, ColorInterp=Gray - NoData Value=-10000 - -Reading this image could be very slow because it has not been tiled yet. So, as discussed above, its data need to be stored and compressed in tiles to increase performances. - -The following gdal_translate command should be appropriate for that purpose: - -.. code-block:: shell - - gdal_translate -co TILED=YES -co COMPRESS=DEFLATE chiangMai_dtm.tif chiangMai_dtm_optimized.tif - -When the data to compress consists of imagery (es. aerial photographs, true-color satellite images, or colored maps) you can use lossy algorithms such as JPEG. We are now compressing data where the precision is important, the band data type is Float32 and elevation values should not be altered, so a lossy algorithm such as JPEG is not suitable. JPEG should generally only be used with Byte data (8 bit per channel) so we have choosen the lossless DEFLATE compression through the COMPRESS=DEFLATE creation option. - -Calling the gdalinfo command again: - -.. code-block:: shell - - gdalinfo chiangMai_dtm_optimized.tif - -We can observe the following results: - -.. code-block:: shell - - Driver: GTiff/GeoTIFF - Files: chiangMai_dtm_optimized.tif - Size is 12638, 13240 - Coordinate System is: - PROJCS["WGS 84 / UTM zone 47N", - GEOGCS["WGS 84", - DATUM["WGS_1984", - SPHEROID["WGS 84",6378137,298.257223563, - AUTHORITY["EPSG","7030"]], - AUTHORITY["EPSG","6326"]], - PRIMEM["Greenwich",0, - AUTHORITY["EPSG","8901"]], - UNIT["degree",0.0174532925199433, - AUTHORITY["EPSG","9122"]], - AUTHORITY["EPSG","4326"]], - PROJECTION["Transverse_Mercator"], - PARAMETER["latitude_of_origin",0], - PARAMETER["central_meridian",99], - PARAMETER["scale_factor",0.9996], - PARAMETER["false_easting",500000], - PARAMETER["false_northing",0], - UNIT["metre",1, - AUTHORITY["EPSG","9001"]], - AXIS["Easting",EAST], - AXIS["Northing",NORTH], - AUTHORITY["EPSG","32647"]] - Origin = (487068.774750000040513,2057413.889810000080615) - Pixel Size = (0.144270000000000,-0.144270000000000) - Metadata: - AREA_OR_POINT=Area - TIFFTAG_SOFTWARE=pix4dmapper - Image Structure Metadata: - COMPRESSION=DEFLATE - INTERLEAVE=BAND - Corner Coordinates: - Upper Left ( 487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N) - Lower Left ( 487068.775, 2055503.755) ( 98d52'38.77"E, 18d35'25.19"N) - Upper Right ( 488892.059, 2057413.890) ( 98d53'40.94"E, 18d36'27.37"N) - Lower Right ( 488892.059, 2055503.755) ( 98d53'40.98"E, 18d35'25.22"N) - Center ( 487980.417, 2056458.822) ( 98d53' 9.85"E, 18d35'56.28"N) - Band 1 Block=256x256 Type=Float32, ColorInterp=Gray - NoData Value=-10000 - -We need also to create overviews through the gdaladdo command: - -.. code-block:: shell - - gdaladdo -r nearest chiangMai_dtm_optimized.tif 2 4 8 16 32 64 - -Unlike the previous example, overviews will be created with the **nearest resampling algorithm**. That is due to the nature of the data we are representing: we should not consider the average between two elevation values but simply the closer one, it is more reliable regarding the conservation of the original data. - -Calling the gdalinfo command again: - -.. code-block:: shell - - gdalinfo chiangMai_dtm_optimized.tif - -We can see the following information: - -.. code-block:: shell - - Driver: GTiff/GeoTIFF - Files: chiangMai_dtm_optimized.tif - Size is 12638, 13240 - Coordinate System is: - PROJCS["WGS 84 / UTM zone 47N", - GEOGCS["WGS 84", - DATUM["WGS_1984", - SPHEROID["WGS 84",6378137,298.257223563, - AUTHORITY["EPSG","7030"]], - AUTHORITY["EPSG","6326"]], - PRIMEM["Greenwich",0, - AUTHORITY["EPSG","8901"]], - UNIT["degree",0.0174532925199433, - AUTHORITY["EPSG","9122"]], - AUTHORITY["EPSG","4326"]], - PROJECTION["Transverse_Mercator"], - PARAMETER["latitude_of_origin",0], - PARAMETER["central_meridian",99], - PARAMETER["scale_factor",0.9996], - PARAMETER["false_easting",500000], - PARAMETER["false_northing",0], - UNIT["metre",1, - AUTHORITY["EPSG","9001"]], - AXIS["Easting",EAST], - AXIS["Northing",NORTH], - AUTHORITY["EPSG","32647"]] - Origin = (487068.774750000040513,2057413.889810000080615) - Pixel Size = (0.144270000000000,-0.144270000000000) - Metadata: - AREA_OR_POINT=Area - TIFFTAG_SOFTWARE=pix4dmapper - Image Structure Metadata: - COMPRESSION=DEFLATE - INTERLEAVE=BAND - Corner Coordinates: - Upper Left ( 487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N) - Lower Left ( 487068.775, 2055503.755) ( 98d52'38.77"E, 18d35'25.19"N) - Upper Right ( 488892.059, 2057413.890) ( 98d53'40.94"E, 18d36'27.37"N) - Lower Right ( 488892.059, 2055503.755) ( 98d53'40.98"E, 18d35'25.22"N) - Center ( 487980.417, 2056458.822) ( 98d53' 9.85"E, 18d35'56.28"N) - Band 1 Block=256x256 Type=Float32, ColorInterp=Gray - NoData Value=-10000 - Overviews: 6319x6620, 3160x3310, 1580x1655, 790x828, 395x414, 198x207 - -Overviews have been created. By default, they inherit the same compression type of the original dataset (there is no evidence of it in the gdalinfo output). - -Other Raster Data Use Cases -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -* `Serving a large number of GrayScale GeoTiff with Palette `_ -* `Serving a large number of DTM ASCII Grid Files `_ -* `Serving a large number of Cartographic Black/White GeoTiff with Palette `_ -* `Serving a large number of satellite/aerial RGB GeoTiff with compression `_ -* `Optimizing and serving UAV data `_ -* `Optimizing and serving 16-bits satellite/aerial RGB GeoTiff `_ - -Process Raster Datasets Programmatically -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In this section we will provide a set of :guilabel:`shell` scripts which might be very useful to batch process a lot of raster datasets programmatically. - -1. ``process_gray.sh`` - - .. code-block:: shell - - for filename in *.tif*; do echo gdal_translate -co TILED=YES -co COMPRESS=DEFLATE $filename ${filename//.tif/.optimized.tif}; done > gdal_translate.sh - chmod +x gdal_translate.sh - ./gdal_translate.sh - - .. code-block:: shell - - for filename in *.optimized.tif*; do echo gdaladdo -r nearest $filename 2 4 8 16 32 64 128 256 512; done > gdaladdo.sh - for filename in *.optimized.tif*; do echo mv \"$filename\" \"${filename//.optimized.tif/.tif}\"; done > rename.sh - chmod +x *.sh - ./gdaladdo.sh - ./rename.sh - -2. ``process_rgb.sh`` - - .. code-block:: shell - - for filename in *.tif*; do echo gdal_translate -co TILED=YES -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR -b 1 -b 2 -b 3 $filename ${filename//.tif/.optimized.tif}; done > gdal_translate.sh - chmod +x gdal_translate.sh - ./gdal_translate.sh - - .. code-block:: shell - - for filename in *.optimized.tif*; do echo gdaladdo -r average $filename 2 4 8 16 32 64 128 256 512; done > gdaladdo.sh - for filename in *.optimized.tif*; do echo mv \"$filename\" \"${filename//.optimized.tif/.tif}\"; done > rename.sh - chmod +x *.sh - ./gdaladdo.sh - ./rename.sh - -3. ``process_rgb_alpha.sh`` - - .. code-block:: shell - - for filename in *.tif*; do echo gdal_translate -co TILED=YES -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR --config GDAL_TIFF_INTERNAL_MASK YES -b 1 -b 2 -b 3 -mask 4 $filename ${filename//.tif/.optimized.tif}; done > gdal_translate.sh - chmod +x gdal_translate.sh - ./gdal_translate.sh - - .. code-block:: shell - - for filename in *.optimized.tif*; do echo gdaladdo -r average $filename 2 4 8 16 32 64 128 256 512; done > gdaladdo.sh - for filename in *.optimized.tif*; do echo mv \"$filename\" \"${filename//.optimized.tif/.tif}\"; done > rename.sh - chmod +x *.sh - ./gdaladdo.sh - ./rename.sh - -4. ``process_rgb_palette.sh`` - - .. code-block:: shell - - for filename in *.tif*; do echo gdal_translate -co TILED=YES -co COMPRESS=DEFLATE $filename ${filename//.tif/.optimized.tif}; done > gdal_translate.sh - chmod +x gdal_translate.sh - ./gdal_translate.sh - - .. code-block:: shell - - for filename in *.optimized.tif*; do echo gdaladdo -r average $filename 2 4 8 16 32 64 128 256 512; done > gdaladdo.sh - for filename in *.optimized.tif*; do echo mv \"$filename\" \"${filename//.optimized.tif/.tif}\"; done > rename.sh - chmod +x *.sh - ./gdaladdo.sh - ./rename.sh - -.. _createsuperuser: - -Thesaurus Import and Export -=========================== - -See :ref:`load_thesaurus` and :ref:`dump_thesaurus`. - - -Create Users and Super Users -============================ - -Your first step will be to create a user. There are three options to do so, depending on which kind of user you want to create you may -choose a different option. We will start with creating a *superuser*, because this user is the most important. A superuser -has all the permissions without explicitly assigning them. - -The easiest way to create a superuser (in linux) is to open your terminal and type: - - .. code-block:: shell - - $ DJANGO_SETTINGS_MODULE=geonode.settings python manage.py createsuperuser - - .. note:: If you enabled ``local_settings.py`` the command will change as following: - - .. code-block:: shell - - $ DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py createsuperuser - -You will be asked a username (in this tutorial we will call the superuser you now create *your_superuser*), an email address and a password. - -Now you've created a superuser you should become familiar with the *Django Admin Interface*. As a superuser you are having -access to this interface, where you can manage users, Datasets, permission and more. To learn more detailed about this interface -check this LINK. For now it will be enough to just follow the steps. To attend the *Django Admin Interface*, go to your geonode website and *sign in* with *your_superuser*. Once you've logged in, the name of your user will appear on the top right. Click on it and the following menu -will show up: - - .. figure:: img/menu_admin.png - :align: center - - -Clicking on *Admin* causes the interface to show up. - - .. figure:: img/admin_interface.png - :align: center - -Go to *Auth* -> *Users* and you will see all the users that exist at the moment. In your case it will only be *your_superuser*. Click on it, and you will see a section on *Personal Info*, one on *Permissions* and one on *Important dates*. For the moment, the section on *Permissions* is the most important. - - .. figure:: img/permissions_django_admin.png - :align: center - -As you can see, there are three boxes that can be checked and unchecked. Because you've created a superuser, all three boxes -are checked as default. If only the box *active* would have been checked, the user would not be a superuser and would not be able to -access the *Django Admin Interface* (which is only available for users with the *staff* status). Therefore keep the following -two things in mind: - -* a superuser is able to access the *Django Admin Interface* and he has all permissions on the data uploaded to GeoNode. -* an ordinary user (created from the GeoNode interface) only has *active* permissions by default. The user will not have the ability to access the *Django Admin Interface* and certain permissions have to be added for him. - -Until now we've only created superusers. So how do you create an ordinary user? You have two options: - -#. Django Admin Interface - - First we will create a user via the *Django Admin Interface* because we've still got it open. Therefore go back to *Auth* -> *Users* and you should find a button on the right that says *Add user*. - - .. figure:: img/add_user.png - :align: center - - Click on it and a form to fill out will appear. Name the new user test_user, choose a password and click *save* at the right bottom of the site. - - .. figure:: img/add_test_user.png - :align: center - - Now you should be directed to the site where you could - change the permissions on the user *test_user*. As default only *active* is checked. If you want this user also to be able to attend this admin interface - you could also check *staff status*. But for now we leave the settings as they are! - - .. todo:: groups and permissions! - - To test whether the new user was successfully created, go back to the GeoNode web page and try to sign in. - -#. GeoNode website - - To create an ordinary user you could also just use the GeoNode website. If you installed GeoNode using a release, you should - see a *Register* button on the top, beside the *Sign in* button (you might have to log out before). - - .. figure:: img/register.png - :align: center - - Hit the button and again a form will appear for you to fill out. This user will be named *geonode_user* - - .. figure:: img/sign_up_test_user.png - :align: center - - .. todo:: NEW IMAGE WITH GEONODE USER! - - By hitting *Sign up* the user will be signed up, as default only with the status *active*. - -.. _batch_sync_permissions: - -Batch Sync Permissions -====================== - -GeoNode provides a very useful management command ``set_layers_permisions`` allowing an administrator to easily add / remove permissions to groups and users on one or more Datasets. - -The ``set_layers_permisions`` command arguments are: - -- **permissions** to set/unset --> read, download, edit, manage - - .. code-block:: python - - READ_PERMISSIONS = [ - 'view_resourcebase' - ] - DOWNLOAD_PERMISSIONS = [ - 'view_resourcebase', - 'download_resourcebase' - ] - EDIT_PERMISSIONS = [ - 'view_resourcebase', - 'change_dataset_style', - 'download_resourcebase', - 'change_resourcebase_metadata', - 'change_dataset_data', - 'change_resourcebase' - ] - MANAGE_PERMISSIONS = [ - 'delete_resourcebase', - 'change_resourcebase', - 'view_resourcebase', - 'change_resourcebase_permissions', - 'change_dataset_style', - 'change_resourcebase_metadata', - 'publish_resourcebase', - 'change_dataset_data', - 'download_resourcebase' - ] - -NB: the above permissions list may change with the ADVANCED_WORKFLOW enabled. For additional info: https://docs.geonode.org/en/master/admin/admin_panel/index.html#how-to-enable-the-advanced-workflow - -- **resources** (Datasets) which permissions will be assigned on --> type the Dataset id, multiple choices can be typed with comma separator, if no ids are provided all the Datasets will be considered -- **users** who permissions will be assigned to, multiple choices can be typed with a comma separator -- **groups** who permissions will be assigned to, multiple choices can be typed with a comma separator -- **delete** flag (optional) which means the permissions will be unset - -Usage examples: ---------------- - -1. Assign **edit** permissions on the Datasets with id **1** and **2** to the users **username1** and **username2** and to the group **group_name1**. - - .. code-block:: shell - - python manage.py set_layers-permissions -p edit -u username1,username2 -g group_name1 -r 1,2 - -2. Assign **manage** permissions on all the Datasets to the group **group_name1**. - - .. code-block:: shell - - python manage.py set_layers-permissions -p manage -g group_C - -3. Unset **download** permissions on the Dataset with id **1** for the user **username1**. - - .. code-block:: shell - - python manage.py set_layers-permissions -p download -u username1 -r 1 -d - -The same functionalities, with some limitations, are available also from the :guilabel:`Admin Dashboard >> Users` or :guilabel:`Admin Dashboard >> Groups >> Group profiles`. - -.. figure:: img/layer_batch_perms_admin.png - :align: center - -An action named :guilabel:`Set layer permissions` is available from the list, redirecting the administrator to a form to set / unset read, edit, download permissions on the selected Users/group profile. - -.. figure:: img/layer_batch_perms_form.png - :align: center - -Is enough to select the dataset and press "Submit". If the async mode is activated, the permission assign is asyncronous - -.. _delete_resources: - -Delete Certain GeoNode Resources -================================ - -The ``delete_resources`` :guilabel:`Management Command` allows to remove resources meeting a certain condition, -specified in a form of a serialized django Q() expression. - -First of all let's take a look at the ``--help`` option of the ``delete_resources`` -management command in order to inspect all the command options and features. - -Run - -.. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources --help - -.. note:: If you enabled ``local_settings.py`` the command will change as following: - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py delete_resources --help - -This will produce output the following output: - -.. code-block:: shell - - usage: manage.py delete_resources [-h] [-c CONFIG_PATH] - [-l LAYER_FILTERS [LAYER_FILTERS ...]] - [-m MAP_FILTERS [MAP_FILTERS ...]] - [-d DOCUMENT_FILTERS [DOCUMENT_FILTERS ...]] - [--version] [-v {0,1,2,3}] - [--settings SETTINGS] - [--pythonpath PYTHONPATH] [--traceback] - [--no-color] [--force-color] - - Delete resources meeting a certain condition - - optional arguments: - -h, --help show this help message and exit - -c CONFIG_PATH, --config CONFIG_PATH - Configuration file path. Default is: - delete_resources.json - -l LAYER_FILTERS [LAYER_FILTERS ...], --layer_filters LAYER_FILTERS [LAYER_FILTERS ...] - -m MAP_FILTERS [MAP_FILTERS ...], --map_filters MAP_FILTERS [MAP_FILTERS ...] - -d DOCUMENT_FILTERS [DOCUMENT_FILTERS ...], --document_filters DOCUMENT_FILTERS [DOCUMENT_FILTERS ...] - --version show program's version number and exit - -v {0,1,2,3}, --verbosity {0,1,2,3} - Verbosity level; 0=minimal output, 1=normal output, - 2=verbose output, 3=very verbose output - --settings SETTINGS The Python path to a settings module, e.g. - "myproject.settings.main". If this isn't provided, the - DJANGO_SETTINGS_MODULE environment variable will be - used. - --pythonpath PYTHONPATH - A directory to add to the Python path, e.g. - "/home/djangoprojects/myproject". - --traceback Raise on CommandError exceptions - --no-color Don't colorize the command output. - --force-color Force colorization of the command output. - - -There are two ways to declare Q() expressions filtering which resources should be deleted: - -1. With a JSON configuration file: passing ``-c`` argument specifying the path to the JSON configuration file. - -* **Example 1**: Relative path to the config file (to ``manage.py``) - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources -c geonode/base/management/commands/delete_resources.json - - -* **Example 2**: Absolute path to the config file - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources -c /home/User/Geonode/configs/delete_resources.json - - -2. With CLI: passing ``-l`` ``-d`` ``-m`` list arguments for each of resources (Datasets, documents, maps) - -* **Example 3**: Delete resources without configuration file - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources -l 'Q(pk__in: [1, 2]) | Q(title__icontains:"italy")' 'Q(owner__name=admin)' -d '*' -m "Q(pk__in=[1, 2])" - - - -Configuration File ------------------- - -The JSON configuration file should contain a single `filters` object, which consists of `Dataset`, `map` and `document` lists. -Each list specifies the filter conditions applied to a corresponding queryset, defining which items will be deleted. -The filters are evaluated and directly inserted into Django .filter() method, which means the filters occurring as -separated list items are treated as AND condition. To create OR query ``|`` operator should be used. For more info please check Django -[documentation](https://docs.djangoproject.com/en/3.2/topics/db/queries/#complex-lookups-with-q-objects)). -The only exception is passing a list with ``'*'`` which will cause deleting all the queryset of the resource. - -* **Example 4**: Example content of the configuration file, which will delete Datasets with ID's 1, 2, and 3, those owned by `admin` user, along with all defined maps. - - .. code-block:: shell - - { - "filters": { - "Dataset": [ - "Q(pk__in=[1, 2, 3]) | Q(title__icontains='italy')", - "Q(user__name=admin)" - ], - "map": ["*"], - "document": [] - } - } - - -CLI ---- - -The CLI configuration can be specified with ``-l`` ``-d`` ``-m`` list arguments, which in fact are a translation -of the configuration JSON file. ``-l`` ``-d`` ``-m`` arguments are evaluated in the same manner as filters.Dataset, -filters.map and filter.document accordingly from the Example 4. -The following example's result will be equivalent to Example 4: - -* **Example 5**: Example CLI configuration, which will delete Datasets with ID's 1, 2, and 3, along with all maps. - - .. code-block:: shell - - DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources -l 'Q(pk__in: [1, 2, 3]) | Q(title__icontains:"italy")' 'Q(owner__name=admin)' -m '*' - - -Async execution over http -========================= - -It is possible to expose and run management commands over http. - -To run `custom django management commands `_ usually we make make use of the command line: - -.. code-block:: shell - - python manage.py ping_mngmt_commands_http - $> pong - -The ``management_commands_http`` app allows us to run commands when we have no access to the command line. -It's possible to run a command using the API or the django admin GUI. - -For security reasons, only admin users can access the feature and the desired command needs to be explicitly exposed. -By default the following commands are exposed: *ping_mngmt_commands_http*, *updatelayers*, *sync_geonode_datasets*, *sync_geonode_maps*, *importlayers* and *set_all_datasets_metadata*. - -To expose more command you can change the environment variable ``MANAGEMENT_COMMANDS_EXPOSED_OVER_HTTP`` and the added commands will be exposed in your application. - -The list of exposed commands is available by the endpoint ``list_management_commands`` and also presented by the form in the admin page ``create management command job``. - -.. note:: To use the commands in an asyncronous approach ``ASYNC_SIGNALS`` needs to be set to ``True`` and celery should be running. - -Manage using django admin interface ------------------------------------ - -Creating a job -^^^^^^^^^^^^^^ - -Access the admin panel: ``http:///admin`` and go to "Management command jobs". - -.. figure:: img/management_commands_over_http_admin.png - :align: center - :alt: Management commands over http admin menu. - - *Management command admin section* - -You will arrive at http:///en/admin/management_commands_http/managementcommandjob/, -then click on the buttton ``+ Add management command job`` (``http:///en/admin/management_commands_http/managementcommandjob/add/``). - -.. figure:: img/management_commands_over_http_btn_add_job.png - :align: center - :alt: Button: Add management command job. - - *Add management command job* - -Select the command and fill the form, with the arguments and/or key-arguments if needed. -Save you job and in the list select the ``start`` action, alterantively you can mark the ``autostart`` option and the command will be automatic started when created. - -.. figure:: img/management_commands_over_http_add_job.png - :align: center - :alt: Form: Add management command job. - - *Creating a management command job form* - - -Starting a job -^^^^^^^^^^^^^^ - -To start a job: - -.. figure:: img/management_commands_over_http_start_job.png - :align: center - :alt: Starting a management command job. - - *Starting a job* - -1. Select the job to be started. -2. Select the ``start`` action. -3. Click in ``Go``. -4. The page will refresh and the job status will have changed. If it takes a long to run, refresh the page to see the updated the status. -5. A ``stop`` option is also available. - -.. note:: - If it takes too long to load the page, ``ASYNC_SIGNALS`` may not be activated. - If its status gets stuck at ``QUEUED``, verify if celery is running and properly configured. - - -Job status -^^^^^^^^^^ - -Clicking at the link in the ID of a job, we can see the details of this job. -For the job we just created, we can verify the output message and celery job status. - -.. figure:: img/management_commands_over_http_job_status.png - :align: center - :alt: Management command job status. - - *Example job status* - - -When we have an error during execution the traceback message will be available in the ``Celery traceback``. -In the next image a ``ping_mngmt_commands_http`` job was created with the arguments ``["--force_exception", true]``. -Checking the text in this field can be useful when troubleshooting errors. - -.. figure:: img/management_commands_over_http_job_status_with_traceback.png - :align: center - :alt: Management command job status with a traceback message. - - *Example job traceback message* - - -Manage using API endpoints --------------------------- - -The execution of the management commands can be handled by http requests to an API: ``http:///api/v2/management/``. - -All the requests need to be authenticated with administrative permissions (*superuser*). - -You can find here a postman collection with all the exemples listed here and other available endpoints: - -:download:`geonode_mngmt_commands.postman_collection.json ` - - -List exposed commands -^^^^^^^^^^^^^^^^^^^^^ - -Getting a list of the exposed commands: - -.. code-block:: shell - - curl --location --request GET 'http:///api/v2/management/commands/' --header 'Authorization: Basic YWRtaW46YWRtaW4=' - -Response: - -.. code-block:: json - - { - "success": true, - "error": null, - "data": [ - "ping_mngmt_commands_http", - "updatelayers", - "set_all_datasets_metadata", - "sync_geonode_maps", - "importlayers", - "sync_geonode_datasets" - ] - } - -.. note:: You should change the header ``Authorization`` (``Basic YWRtaW46YWRtaW4=``) to your Auth token, in this example I am using a token for ``admin`` as username and ``admin`` as password. - -Creating a job -^^^^^^^^^^^^^^ - -Optionally, before creating the job you can get its *help message* with the following call: - -.. code-block:: shell - - curl --location --request GET 'http:///api/v2/management/commands/ping_mngmt_commands_http/' --header 'Authorization: Basic YWRtaW46YWRtaW4=' - - -Creating a job for running ``ping_mngmt_commands_http`` with 30 seconds of sleep time: - -.. code-block:: shell - - curl --location --request POST 'http:///api/v2/management/commands/ping_mngmt_commands_http/jobs/' \ - --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ - --header 'Content-Type: application/json' \ - --data-raw '{ - "args": ["--sleep", 30], - "kwargs": {}, - "autostart": false - }' - -Response: - -.. code-block:: json - - { - "success": true, - "error": null, - "data": { - "id": 8, - "command": "ping_mngmt_commands_http", - "app_name": "geonode.management_commands_http", - "user": 1000, - "status": "CREATED", - "created_at": "2021-10-08T18:17:25.045752Z", - "start_time": null, - "end_time": null, - "args": [ - "--sleep", - 30 - ], - "kwargs": {}, - "celery_result_id": null, - "output_message": null - } - } - - -.. note:: Alterantively you can omit the ``jobs`` part of the url to create a job. (Using ``http:///api/v2/management/commands/ping_mngmt_commands_http/`` as url) - - -Start/Stop actions -^^^^^^^^^^^^^^^^^^ - -To start the created job: - -.. code-block:: shell - - curl --location --request PATCH 'http:///api/v2/management/jobs/8/start/' --header 'Authorization: Basic YWRtaW46YWRtaW4=' - -Response: - -.. code-block:: json - - { - "success": true, - "error": null, - "data": { - "id": 8, - "command": "ping_mngmt_commands_http", - "app_name": "geonode.management_commands_http", - "user": 1000, - "status": "QUEUED", - "created_at": "2021-10-08T18:17:25.045752Z", - "start_time": null, - "end_time": null, - "args": [ - "--sleep", - 30 - ], - "kwargs": {}, - "celery_result_id": null, - "output_message": null - } - } - -.. note:: During execution the job can be interrupted using the following call: - - .. code-block:: shell - - curl --location --request PATCH 'http:///api/v2/management/jobs/8/stop/' --header 'Authorization: Basic YWRtaW46YWRtaW4=' - -Note that the status changed from **CREATED** to **QUEUED**, during execution it will be **STARTED** and at the end **FINISHED**. - -Jobs list and status -^^^^^^^^^^^^^^^^^^^^ - -You can verify your job status and details with the following call: - -.. code-block:: shell - - curl --location --request GET 'http:///api/v2/management/jobs/8/status/' --header 'Authorization: Basic YWRtaW46YWRtaW4=' - -Response: - -.. code-block:: json - - { - "id": 8, - "command": "ping_mngmt_commands_http", - "app_name": "geonode.management_commands_http", - "user": 1000, - "status": "FINISHED", - "created_at": "2021-10-08T18:17:25.045752Z", - "start_time": "2021-10-08T18:20:02.761475Z", - "end_time": "2021-10-08T18:20:32.802007Z", - "args": [ - "--sleep", - 30 - ], - "kwargs": {}, - "celery_result_id": "fe7359a6-5f8c-47bf-859a-84351b5ed80c", - "output_message": "Sleeping for 30.0 seconds...\npong\n", - "celery_task_meta": { - "date_done": "2021-10-08T18:20:32.810649Z", - "status": "SUCCESS", - "traceback": null, - "worker": "worker1@4f641ffa9c0b" - } - } - -When runing multiple jobs and to audit already ran jobs. A list of jobs can be retrieved using the following call: - -.. code-block:: shell - - curl --location --request GET 'http:///api/v2/management/jobs/' --header 'Authorization: Basic YWRtaW46YWRtaW4=' - -Response: - -.. code-block:: json - - { - "links": { - "next": null, - "previous": null - }, - "total": 1, - "page": 1, - "page_size": 10, - "data": [ - { - "id": 1, - "command": "ping_mngmt_commands_http", - "app_name": "geonode.management_commands_http", - "user": 1000, - "status": "FINISHED", - "created_at": "2021-10-08T18:17:25.045752Z" - } - ] - } - - -.. note:: This list can be filtered by the fields "celery_result_id", "command", "app_name", "status", "user" and "user__username". - - diff --git a/admin/thesaurus/index.rst b/admin/thesaurus/index.rst deleted file mode 100644 index 8efa8f6a..00000000 --- a/admin/thesaurus/index.rst +++ /dev/null @@ -1,330 +0,0 @@ -.. _thesauri: - -Thesauri -======== - -Introduction ------------- - -A **thesaurus** is a structured vocabulary used to manage and standardize keywords (also known as tags) that describe resources. It helps improve metadata quality, searchability, and interoperability by enforcing controlled vocabularies. - -Key Functions -------------- - -Controlled Vocabulary - Instead of free-text keywords, thesauri offer predefined and **standardized terms** organized thematically (e.g., ISO 19115 topics, GEMET, INSPIRE themes). - -Semantic Consistency - Users tagging datasets can choose from a consistent list of terms, **reducing redundancy** and ambiguity (e.g., avoiding both "roads" and "road" as separate tags). - -Improved search and filtering - Thesauri enable structured tagging of datasets, allowing more accurate searches and the use of **faceted filters** to easily narrow down results. - -Localization - Each concept in a thesaurus can have **translations for different languages**, allowing localized display based on the user’s interface language. - -Metadata Standards Integration - Thesauri can align with **international standards** (like ISO, INSPIRE, GEMET), which is especially important when GeoNode is used in institutional or governmental contexts. - - -Data model ----------- - -The *GeoNode thesaurus model* is designed to support multilingual, structured vocabularies. It consists of the following key components: - -Thesaurus: - * Represents a full controlled vocabulary (e.g., GEMET, INSPIRE themes). - * In SKOS terms, it's a ``skos:ConceptScheme``. - -ThesaurusLabel: - * Stores the localized names (titles/descriptions) of a thesaurus for different languages. - * In SKOS terms, it's a ``skos:preflabel`` within the ``skos:ConceptScheme``. - -ThesaurusKeyword: - * Represents a single concept or term within a thesaurus (e.g., "Land Cover", "Transport"), also storing the default label (used where the translation for a given requested language is not defined) and its identifying URI. - * In SKOS terms, it's a ``skos:Concept``. - -ThesaurusKeywordLabel: - * Stores the multilingual labels for each keyword. - * In SKOS terms, it's a ``skos:preflabel`` within the ``skos:Concept``. - -.. _thesaurus_add: - -Adding a Thesaurus -================== - -A thesaurus can be added in Geonode by: - -* creating a new thesaurus instance within the GeoNode admin pages. - As a minumum, you need to: - - * add a thesaurus in admin / base / Thesaurus - * add one or more instances of Keywords in admin / base / ThesaurusKeywords -* uploading a RDF file (either xml, ttl, jsonld or any other format recognized by `RDFlib `__). - When uploading a file, the behaviour is the same as running the command ``thesaurus load --action update`` (see :ref:`load_thesaurus`) -* loading a RDF file using the `thesaurus load` management command (see :ref:`load_thesaurus`). - - -Upload an RDF file via the thesaurus admin page ------------------------------------------------ - -Navigate to the thesaurus page in the admin panel ``http:///admin/base/thesaurus``. -On the top-right of the page a button named :guilabel:`Upload thesaurus` will be available: - -.. figure:: img/thesaurus_admin_1.png - :align: center - -After clicking on it, a simple form for the upload will be shown which will allow you to select your desired RDF file: - -.. figure:: img/thesaurus_admin_2.png - :align: center - -By clicking on `Upload RDF`, the system will load the thesaurus and assign it a "slugified" name based on the file name. -The name can be easily changed later in the edit page. - -If everything goes fine, a success message will be shown: - -.. figure:: img/thesaurus_admin_success.png - :align: center - -Otherwise the UI will show the error message: - -.. figure:: img/thesaurus_admin_fail.png - :align: center - - - -Management commands -=================== - -GeoNode provides a single command (``thesaurus``) with multiple actions: - -* ``list``: list existing thesauri -* ``load``: load a RDF file -* ``dump``: dump a thesaurus into a file - -.. code-block:: - - python manage.py thesaurus --help - - usage: manage.py thesaurus [-h] [-i [IDENTIFIER]] [-f [FILE]] [--action {create,update,append,parse}] [-o [OUT]] - [--include INCLUDE] [--exclude EXCLUDE] - [--format {json-ld,n3,nt,pretty-xml,sorted-xml,trig,ttl,xml}] [--default-lang LANG] [--version] - [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] - [--force-color] [--skip-checks] - [{list,load,dump}] - - Handles thesaurus commands ['list', 'load', 'dump'] - - positional arguments: - {list,load,dump} thesaurus operation to run - - options: - -h, --help show this help message and exit - --version Show program's version number and exit. - -v {0,1,2,3}, --verbosity {0,1,2,3} - Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output - --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the - DJANGO_SETTINGS_MODULE environment variable will be used. - --pythonpath PYTHONPATH - A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". - --traceback Raise on CommandError exceptions. - --no-color Don't colorize the command output. - --force-color Force colorization of the command output. - --skip-checks Skip system checks. - - Common params: - -i [IDENTIFIER], --identifier [IDENTIFIER] - Thesaurus identifier. Dump: required. Load: optional - if omitted will be created out of the filename - - Params for "load" subcommand: - -f [FILE], --file [FILE] - Full path to a thesaurus in RDF format - --action {create,update,append,parse} - Actions to run upon data loading (default: create) - - Params for "dump" subcommand: - -o [OUT], --out [OUT] - Full path to the output file to be created - --include INCLUDE Inclusion filter (wildcard is * as suffix or prefix); can be repeated - --exclude EXCLUDE Exclusion filter (wildcard is * as suffix or prefix); can be repeated - --format {json-ld,n3,nt,pretty-xml,sorted-xml,trig,ttl,xml} - Format string supported by rdflib, or sorted-xml (default: sorted-xml) - --default-lang LANG Default language code for untagged string literals (default: None) - - -List thesauri: ``thesaurus list`` ---------------------------------- - -Get a list of the thesauri in GeoNode. - -Useful to find out the id of the thesauri when you need to export one of them. - - -.. _load_thesaurus: - -Importing a thesaurus: ``thesaurus load`` ------------------------------------------ - -The ``load`` command may create an entire Thesaurus, or just update part of it. - -Allowed params: - -* ``file``: file to load; required -* ``action``: ``create``, ``update``, ``append``, ``parse``; optional, default ``create``; -* ``identifier``: the id of the thesaurus; optional, defaults to a name created using the filename. - -The **automatic identifier creation** skips all the chars after the first dot in order to allow a thesaurus partitioning. - -For instance we may have different rdf files containing the labels for multiple projects, e.g.: ``labels-i18n.proj1.rdf``, ``labels-i18n.proj2.rdf``... We may simply loop on the filenames and run the ``load`` subcommand on each of them, and all the keywords will be added to the same Thesaurus having id ``labels-i18n``. - -The ``load`` command has different behaviours according to the ``action`` parameter: - -Actions: - -* ``parse``: - parse the file and loop on all the concepts without writng anything in the db. Is equivalent to the classic ``dryrun`` option; -* ``create`` (default action): - tries and create a thesaurus. If the thesaurus already exists, raises an exception. -* ``append``: - creates entries if they do not exist; pk are the ones listed in update action. - If the entry already exists, it is not changed in any way. -* ``update``: - creates and updates entries: - - * *Thesaurus*: creates it if it doesn't exist, pk is "identifier". - If it exists updates "date", "description", "title", "about" - * *ThesaurusLabel*: creates it if it doesn't exist, pk is "thesaurus", "lang" - If it exists updates "value" - * *ThesaurusKeyword*: creates it if it doesn't exist, pk is "thesaurus", "about" - If it exists updates "alt_label" - * *ThesaurusKeywordlabel*: creates it if it doesn't exist, pk is "thesauruskeyword", "lang" - If it exists updates "label" - - -.. _dump_thesaurus: - -Exporting a thesaurus: ``thesaurus dump`` ------------------------------------------ - -The ``dump`` command may export an entire Thesaurus or just a part of it. - - -Allowed params: - -- ``identifier``: the id of the thesaurus; required. -- ``include``: Optional; filter ThesaurusKeywords to be dumped. Can be repeated. Filtering is applied on the `about` field. Filters are in the format either `*string` or `string*` -- ``exclude``: Optional; like `include` but filter out ThesaurusKeywords from being dumped. -- ``format``: optional, RDF format for the output (``json-ld``, ``n3``, ``nt``, ``pretty-xml``, ``sorted-xml``, ``trig``, ``ttl``, ``xml``). Default ``sorted-xml`` -- ``default-lang``: Default language code for untagged string literals; default is from ``settings.THESAURUS_DEFAULT_LANG`` -- ``out``: Full path to the output file to be created. Optional; if omitted the RDF content is sent to stderr. - - -Format -^^^^^^ - -All the formats, except for ``sorted-xml``, use the *RDFlib* library to serialize the thesaurus data. Since RDFlib handles the concepts as a graph, there is no ordering in the output data. This means that two consecutive ``dump`` of the same thesaurus may create two different files. - -When importing and exporting thesauri as a file, it may be useful to perform diff on them to find out what has changed. -The format ``sorted-xml`` creates a predictable output, where the ConceptScheme is at the start of the file, and the Concepts are sorted by their ``about`` field. Furthermore, the ``prefLabel``'s are sorted by their ``lang`` attribute. - - -Partial export -^^^^^^^^^^^^^^ - -The ``dump`` command also allows to export a subset of the keywords (concepts) in a Thesaurus. - -As an example, let's say we have the ``labels-i18n`` thesaurus, which contains some GeoNode official labels. - -In our project we added some keywords prefixed with "proj1_", since they belong to project1. - -Also in our GeoNode instance, we added some labels which override the standard ones, and are postfixed with ``_ovr``. - -In order to only export the entries we edited, we'll issue the command:: - - python manage.py thesaurus dump -i labels-i18n --include "proj1_*" --include "*_ovr" -f labels-i18n.proj1.rdf - - -Configuring a Thesaurus -======================= - -After a thesaurus is loaded o created in GeoNode, it should be configured in the :guilabel:`Admin` panel. - -The panel can be reached from :guilabel:`Admin` link of the *User Menu* in the navigation bar or through this URL: ``http:///admin/base/thesaurus``. - -Once you are on the Thesaurus lists, select one thesaurus to open the Edit page - -.. figure:: img/thesaurus_edit_page.png - :align: center - - *The GeoNode Thesaurus edit Interface* - -These are the thesaurus main attributes: - -- ``identifier``: (mandatory) the thesaurus identifier (set by the ``--identifier`` parameter in the ``thesaurus load`` command, or automatically generated using the file name). -- ``title``: (mandatory) The default title of the thesaurus (may be set from the loaded RDF file). -- ``date``: (mandatory) The Date of the thesaurus (may be set from the loaded RDF file). -- ``description``: (mandatory) The description of the thesaurus (may be set from the loaded RDF file). -- ``slug``: (deprecated, use ``identifier`` instead) The slug of the thesaurus. -- ``about``: (optional) The ``rdf:about`` URI of the thesaurus (may be set from the loaded RDF file). - -Next attributes define **how the thesaurus shall be used** within GeoNode. - -- ``card min``: (optional) The minimum cardinality, default = 0 -- ``card max``: (optional) The maximum cardinality, default = -1 (no limit) -- ``facet``: (boolean) Decide if the thesaurus will be shown in the facet list, default: True -- To be set to `true` only when ``card_max != 0`` -- ``order``: (integer) Set the listing order of the thesaurus in the facet list and in the metadata editor, default: 0, asc order from 0 to N - -If ``card max`` is not zero, the metadata editor will automatically display the Thesaurus in the list of the controlled terms. -More precisely these are the cases according to the two cardinality fields: - -- `card_max=0` --> Disabled, The Thesaurus will not appear in the GUI -- `card_max=1` & `card_min = 0` --> Single choice, optional. -- `card_max=1` & `card_min = 1` --> Single choice, required -- `card_max=-1` & `card_min = 0` --> [0..N] Multiple choices, optional -- `card_max=-1` & `card_min = 1` --> [1..N] Multiple choices, required - -The metadata editor will show all the thesauri with ``card_max != 0``, each one with its own title, like in the following image: - - -.. figure:: img/thesaurus_choices.png - :align: center - - *The metadata interface with the Thesaurus enabled* - - -The Thesauri having ``card_max == 0`` are used as **codelists**: it means that they will be referred within GeoNode via their identifier for specific purposes. There will be ad-hoc documentation for each of such codelists. - -For instance, the thesaurus with identifier ``labels-i18n`` is used for the metadata labels translations. - - -Using keywords from a thesaurus -=============================== - -After you've finished the setup you should find a new input widget in the metadata editor allowing you to choose keywords from the thesaurus for your resource. - -Also, if you set the ``facet`` attribute to ``true``, the thesaurus should be listed in the filter section in GeoNode's resource list views. - -For instance, if we have these thesauri: - -.. figure:: img/thesaurus_admin_list.png - :align: center - :width: 450px - - *List of configured sample thesauri* - - -both set with ``card max != 0`` and ``facet = true``, we'll have in the editor: - -.. figure:: img/thesaurus_edit_sample.png - :align: center - :width: 450px - - *Keyword selectors for sample thesauri* - -and we'll also have them in the filtering panel as facets: - -.. figure:: img/thesaurus_facet_sample.png - :align: center - - *Facets selectors for sample thesauri* diff --git a/admin/upgrade/3.2-to-3.3/index.rst b/admin/upgrade/3.2-to-3.3/index.rst deleted file mode 100644 index c72b5617..00000000 --- a/admin/upgrade/3.2-to-3.3/index.rst +++ /dev/null @@ -1,113 +0,0 @@ -.. _geonode-upgrade-3.2.x-3.3.x: - -Upgrade from 3.2.x / 3.3.x -========================== - -1. Upgrade the dependencies -2. Perform the ``migrations`` management command; in case some attribute is conflicting, remove it manually from the DB -3. Proform the ``collectstatic`` management command - -Upgrade the instance dependencies -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Check the :ref:`install_dep` and :ref:`install_venv` sections in order to upgrade your Python environment. - -Also, make sure the code is ``Python 3.8`` compatible and that you switched and aligned the **source code** and the **requirements.txt** to the ``master`` branch. - -This must be done manually and with particular attention. - -.. code-block:: shell - - workon - cd - pip install -r requirements.txt - - cd / - - pip install pip --upgrade - pip install -r requirements.txt --upgrade - pip install -e . --upgrade - pip install pygdal=="`gdal-config --version`.*" - - ./manage.sh collectstatic --noinput - - -Run GeoNode migrations -^^^^^^^^^^^^^^^^^^^^^^ - -Activate your GeoNode virtualenv and set the env vars: - -.. code-block:: sql - - . env/bin/Activate - export vars_210 - -Here are the variables to export - update them to your environment settings: - -.. code-block:: shell - - export DATABASE_URL=postgis://user:***@localhost:5432/dbname - export DEFAULT_BACKEND_DATASTORE=data - export GEODATABASE_URL=postgis://user:***@localhost:5432/geonode_data - export ALLOWED_HOSTS="['localhost', '192.168.100.10']" - export STATIC_ROOT=~/www/geonode/static/ - export GEOSERVER_LOCATION=http://localhost:8080/geoserver/ - export GEOSERVER_PUBLIC_LOCATION=http://localhost:8080/geoserver/ - export GEOSERVER_ADMIN_PASSWORD=geoserver - export SESSION_EXPIRED_CONTROL_ENABLED=False - - -Apply migrations and apply basic fixtures: - -.. code-block:: shell - - ./manage.py migrate --fake-initial - paver sync - - -.. note:: Incase of an error of :guilabel:`django.db.utils.ProgrammingError: column "colum-name" of relation "table-name" already exists` on running migrations, you can backup the field data with the following steps. - -.. code-block:: shell - - ./manage.sh dbshell - -.. code-block:: sql - - ALTER TABLE ADD COLUMN _bkp varchar; - UPDATE
SET _bkp = colum-name; - ALTER TABLE
DROP COLUMN ; - - \q - -Run migration then: - -.. code-block:: shell - - ./manage.sh dbshell - -.. code-block:: sql - - UPDATE
SET = _bkp; - ALTER TABLE
DROP COLUMN _bkp; - - \q - - -Create superuser -^^^^^^^^^^^^^^^^ - -To create a superuser you should drop the following constraints (they can be re-enabled if needed): - -.. code-block:: sql - - alter table people_profile alter column last_login drop not null; - -.. code-block:: shell - - ./manage createsuperuser - - -Update Templates -^^^^^^^^^^^^^^^^ - -Update available templates to use {% load static %} instead of {% load staticfiles %} diff --git a/admin/upgrade/index.rst b/admin/upgrade/index.rst deleted file mode 100644 index fb45b0c9..00000000 --- a/admin/upgrade/index.rst +++ /dev/null @@ -1,13 +0,0 @@ -=============== -Upgrade GeoNode -=============== - -.. toctree:: - :maxdepth: 3 - - 3.2-to-3.3/index - -.. toctree:: - :maxdepth: 3 - - 3.x-to-4.x/index diff --git a/admin/upload-parallelism-limit/index.rst b/admin/upload-parallelism-limit/index.rst deleted file mode 100644 index d4ebc1a3..00000000 --- a/admin/upload-parallelism-limit/index.rst +++ /dev/null @@ -1,31 +0,0 @@ - -.. _upload-parallelism-limits: - -Upload Parallelism Limit -======================== - -Default: ``5`` - -When `uploading datasets <../../usage/managing_datasets/uploading_datasets.html#datasets-uploading>`__, this value limits the number os parallel uploads. - -You will get the following message in case of error: ``The number of active parallel uploads exceeds 5. Wait for the pending ones to finish.`` - -This means that you to finish current active uploads before creating a new one. -The default maximum number is set during installation by `DEFAULT_MAX_PARALLEL_UPLOADS_PER_USER <../../basic/settings/index.html#default-max-parallel-uploads-per-user>`__, usually 5 parallel uploads. - -After installation, only an user with administrative rights can change it by the admin painel or `accessing by api <../../devel/api/V2/index.html#get--api-v2-upload-parallelism-limits->`__. - - -Accessing the limits in the admin panel -======================================= -Access the admin panel, scroll until you find the "Upload Parallelism limits" option. - - .. figure:: img/admin_panel_upload_parallelism_limit.png - :align: center - -Click at it, and you will be redirected to the limits list. - - .. figure:: img/admin_panel_parallel_limit.png - :align: center - -To change it, click at the slug of the limit, then you can change its description and the max_number value. Changing the slug will not produce any effect. diff --git a/admin/upload-size-limits/index.rst b/admin/upload-size-limits/index.rst deleted file mode 100644 index 3a8d64e3..00000000 --- a/admin/upload-size-limits/index.rst +++ /dev/null @@ -1,65 +0,0 @@ - -.. _upload-size-limits: - -Upload Size Limits -================== - -Default: ``104857600`` (100 MB in total bytes) - -When `uploading datasets <../../usage/managing_datasets/uploading_datasets.html#datasets-uploading>`__ -or `uploading documents <../../usage/managing_documents/uploading_documents.html#uploading-documents>`__, -the total size of the uploaded files is verified. - -If it passes a limit value, you will get something similar to the following error: - .. figure:: img/size-limit-error.png - :align: center - - -With the following message: ``Total upload size exceeds 100.0 MB. Please try again with smaller files.`` - -This means that the file you tried to upload is too large. It consider the sum of sizes in the case of datasets/layers with multiple files (ESRI Shapefile). -The default maximum value is set during installation by `DEFAULT_MAX_UPLOAD_SIZE <../../basic/settings/index.html#default-max-upload-size>`__, usually 100 MB. - -After installation, only an user with administrative rights can change it by the admin painel or `accessing by api <../../devel/api/V2/index.html#get--api-v2-upload-size-limits->`__. - - -Accessing the limits in the admin panel -======================================= -Access the admin panel, scroll until you find the "Upload Size limits" option. - - .. figure:: img/admin_panel_upload_size_limits.png - :align: center - -Click at it, and you will be redirected to the limits list. - - .. figure:: img/admin_panel_size_limits_list.png - :align: center - -We have two limits here, one for datasets/layers and other for documents. - -Changing a Limit -================ - -To change it, click at the slug of one of the limits. - - .. figure:: img/changing_limit_to_200mb.jpg - :align: center - -You can change its description and the max_size value. Changing the slug will not produce any effect. - -Max size value should be entered in bytes, this means that 200 MB is around 200000000 bytes, or 200 * 1024 * 1024 = 209715200 if you want to use exact values. -After editing, remember to save and verify in the list if the value is the expected one. - -If you try to upload a dataset/layer larger than 100 MB again (but smaller than the new limit) it won't raise any errors. - -Advanced notes for developers -============================= - -When uploading a file there is an additional validation step, to avoid the creation of big temporary files. -This happens with the use of a custom `File Upload Handler `__. - -During this step we verify the total size of the request, if it's considerably bigger than the defined limit, an empty file with a fake large number set as its size is created when processing the request. -Later, during the form validation, the related error will be raised. - -The threshould for this to happen is set to ``2 * MAX_UPLOAD_SIZE + 2 MB`` where MAX_UPLOAD_SIZE is the Upload Size Limit defined by the admin user. -When changing the processes related to the upload size limiting, the upload handlers should also be taken into account. Otherwise, this can lead to the creating of empty datasets and documents. diff --git a/advanced/components/index.rst b/advanced/components/index.rst deleted file mode 100644 index bb9c04c5..00000000 --- a/advanced/components/index.rst +++ /dev/null @@ -1,1076 +0,0 @@ -.. _geoserver_geonode_aa: - -OAuth2 Security: Authentication and Authorization -================================================= -GeoNode interacts with GeoServer through an advanced security mechanism based on OAuth2 Protocol and GeoFence. This section is a walk through of the configuration and setup of GeoNode and GeoServer Advanced Security. - -What we will see in this section is: - -* **Introduction** - -* **GeoNode** (Security Backend): - - #. Django Authentication - - #. Django OAuth Toolkit Setup and Configuration - - #. Details on ``settings.py`` Security Settings - -* **GeoServer** (Security Backend): - - #. GeoServer Security Subsystem - - #. Introduction to the GeoServer OAuth2 Security Plugin - - #. Configuration of the ``GeoNode REST Role Service`` - - #. Configuration of the ``GeoNode OAuth2 Authentication Filter`` - - #. The GeoServer Authentication Filter Chains - - #. Introduction to GeoFence Plugin, the Advanced Security Framework for GeoServer - -* **Troubleshooting and Advanced Features**: - - #. Common Issues and Fixes - - #. How to setup ``HTTPS`` secured endpoints - - #. GeoFence Advanced Features - -Introduction ------------- - -GeoServer, i.e. the geospatial backend server of GeoNode, is a spatial server which needs authenticated users in order to access protected resources or administration functions. - -GeoServer supports several kind of Authentication and Authorization mechanisms. Those systems are pluggable and GeoServer can use them at the same time by the use of a ``Filter Chain``. -Briefly this mechanism allows GeoServer to check for different A&A protocols one by one. The first one matching is used by GeoServer to authorize the users. - -GeoNode Authentication is based by default on Django Security Subsystem. Django authentication allows GeoNode to manage its internal users, groups, roles and sessions. - -GeoNode has some external components, like GeoServer or QGis Server, which are pluggable and stand-alone services, devoted to the management of geospatial data. -Those external services have their own authentication and authorization mechanisms which must be synchronized somehow with the GeoNode one. Also, those external services -maintain, in most of the cases and unless specific configuration does not disable this, alternative security access which for instance allow GeoNode to modify the geospatial catalog -under the hood, or a system administrator to have independent and privileged access to the servers. - -Before going deeply on how GeoServer/GeoNode A&A works and how it can be configured in order to work correctly with GeoNode, let's quickly clarify the difference between the ``Authentication`` and ``Authorization`` concepts. - -Authentication -^^^^^^^^^^^^^^ - -Authentication is the process of verifying the identity of someone through the use of some sort of credentials and a handshake protocol. -If the credentials are valid, the authorization process starts. Authentication process always proceeds to Authorization process (although they may often seem to be combined). -The two terms are often used synonymously but they are two different processes. - -For more details and explanation about the authentication concepts, take a look `here `_. - -Authorization -^^^^^^^^^^^^^ - -Authorization is the process of allowing authenticated users to access protected resources by checking its roles and rights against some sort of security rules mechanism or protocol. -In other words it allows to control access rights by granting or denying specific permissions to specific authorized users. - -GeoNode Security Backend ------------------------- - -DJango Authentication -^^^^^^^^^^^^^^^^^^^^^ - -The Django authentication system handles both authentication and authorization. - -The auth system consists of: - -#. Users -#. Permissions: Binary (yes/no) flags designating whether a user may perform a certain task. -#. Groups: A generic way of applying labels and permissions to more than one user. -#. A configurable password hashing system -#. Forms and view tools for logging in users, or restricting content -#. A pluggable backend system - -The authentication system in Django aims to be very generic and doesn’t provide some features commonly found in web authentication systems. Solutions for some of these common problems have been implemented in third-party packages: - -#. Password strength checking -#. Throttling of login attempts -#. Authentication against third-parties (OAuth, for example) - -.. note:: For more details on installation and configuration of Django authentication system, please refer to the official guide https://docs.djangoproject.com/en/3.2/topics/auth/. - -GeoNode communicates with GeoServer through Basic Authentication under the hood, in order to configure the data and the GeoServer catalog. - -In order to do this, you must be sure that GeoNode knows the **internal** admin user and password of GeoServer. - -.. warning:: This must be an internal GeoServer user with admin rights, not a GeoNode one. - -Make sure the credentials are correctly configured into the file ``settings.py`` - -OGC_SERVER -^^^^^^^^^^ - -Ensure that the ``OGC_SERVER`` settings are correctly configured. - -Notice that the two properties ``LOGIN_ENDPOINT`` and ``LOGOUT_ENDPOINT`` must specify the GeoServer OAuth2 Endpoints (see details below). -The default values ``'j_spring_oauth2_geonode_login'`` and ``'j_spring_oauth2_geonode_logout'`` work in most of the cases, unless you need some specific endpoints different from the later. -In any case those values **must** be coherent with the GeoServer OAuth2 Plugin configuration. - -If in doubt, please use the default values here below. - -Default values are: - -.. code-block:: python - - ... - # OGC (WMS/WFS/WCS) Server Settings - # OGC (WMS/WFS/WCS) Server Settings - OGC_SERVER = { - 'default': { - 'BACKEND': 'geonode.geoserver', - 'LOCATION': GEOSERVER_LOCATION, - 'LOGIN_ENDPOINT': 'j_spring_oauth2_geonode_login', - 'LOGOUT_ENDPOINT': 'j_spring_oauth2_geonode_logout', - # PUBLIC_LOCATION needs to be kept like this because in dev mode - # the proxy won't work and the integration tests will fail - # the entire block has to be overridden in the local_settings - 'PUBLIC_LOCATION': GEOSERVER_PUBLIC_LOCATION, - 'USER': 'admin', - 'PASSWORD': 'geoserver', - 'MAPFISH_PRINT_ENABLED': True, - 'PRINT_NG_ENABLED': True, - 'GEONODE_SECURITY_ENABLED': True, - 'WMST_ENABLED': False, - 'BACKEND_WRITE_ENABLED': True, - 'WPS_ENABLED': False, - 'LOG_FILE': '%s/geoserver/data/logs/geoserver.log' % os.path.abspath(os.path.join(PROJECT_ROOT, os.pardir)), - # Set to name of database in DATABASES dictionary to enable - 'DATASTORE': '', # 'datastore', - 'TIMEOUT': 10 # number of seconds to allow for HTTP requests - } - } - ... - -GeoNode and GeoServer A&A Interaction -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The GeoServer instance used by GeoNode, has a particular setup that allows the two frameworks to correctly interact and exchange informations on users credentials and permissions. - -In particular GeoServer is configured with a ``Filter Chain`` for Authorization that makes use of the two following protocols: - - #. **Basic Authentication**; this is the default GeoServer Authentication mechanism. This makes use of `rfc2617 - Basic and Digest Access Authentication `_ in order to check for user's credentials. - In other words, GeoServer takes a ``username`` and a ``password`` encoded `Base64 `_ on the HTTP Request Headers and compare them against its internal database (which by default is an encrypted XML file on the GeoServer Data Dir). - If the user's credentials match, then GeoServer checks for Authorization through its ``Role Services`` (we will see those services in details on the *GeoServer (Security Backend)* section below). - - .. note:: GeoServer ships by default with ``admin`` and ``geoserver`` as the default administrator user name and password. Before putting the GeoServer on-line it is imperative to change at least the administrator password. - - #. **OAuth2 Authentication**; this module allows GeoServer to authenticate against the `OAuth2 Protocol `_. If the Basic Authentication fails, GeoServer falls back to this by using GeoNode as OAuth2 Provider by default. - -.. note:: Further details can be found directly on the official GeoServer documentation at section "`Authentication Chain `_" - -From the **GeoNode backend (server) side**, the server will make use of **Basic Authentication** with administrator credentials to configure the GeoServer catalog. GeoServer must be reachable by GeoNode of course, and GeoNode must know the internal GeoServer admin credentials. - -From the **GeoNode frontend (browser and GUI) side**, the *Authentication* goal is to allow GeoServer to recognize as valid a user which has been already logged into GeoNode, providing kind of an `SSO `_ mechanism between the two applications. - -GeoServer must know and must be able to access GeoNode via HTTP/HTTPS. In other words, an external user connected to GeoNode must be authenticated to GeoServer with same permissions. This is possible through the **OAuth2 Authentication** Protocol. - -**GeoNode / GeoServer Authentication Mechanism** - - *GeoNode as OAuth2 Provider (OP)* - - OpenID Connect is an identity framework built on OAuth 2.0 protocol which extends the authorization of OAuth 2.0 processes to implement its authentication mechanism. OpenID Connect adds a discovery mechanism allowing users to use an external trusted authority as an identity provider. From another point of view, this can be seen as a single sign on (SSO) system. - - OAuth 2.0 is an authorization framework which is capable of providing a way for clients to access a resource with restricted access on behalf of the resource owner. OpenID Connect allows clients to verify the users with an authorization server based authentication. - - As an OP, GeoNode will be able to act as trusted identity provider, thus allowing the system working on an isolated environment and/or allow GeoNode to authenticate private users managed by the local DJango auth subsystem. - - *GeoServer as OAuth2 Relying Party (RP)* - - Thanks to the **OAuth2 Authentication** GeoServer is able to retrieve an end user's identity directly from the OAuth2 Provider (OP). - - With GeoNode acting as an OP, the mechanism will avoid the use of cookies relying, instead, on the OAuth2 secure protocol. - - How the OAuth2 Protocol works: - - .. figure:: img/oauth001.png - - 1. The relying party sends the request to the OAuth2 provider to authenticate the end user - - 2. The OAuth2 provider authenticates the user - - 3. The OAuth2 provider sends the ID token and access token to the relying party - - 4. The relying party sends a request to the user info endpoint with the access token received from OAuth2 provider - - 5. The user info endpoint returns the claims. - -**GeoNode / GeoServer Authorization Mechanism** - - Allowing GeoServer to make use of a OAuth2 in order to act as an OAuth2 RP, is not sufficient to map a user identity to its roles though. - - On GeoServer side we will still need to a ``RoleService`` which would be able to talk to GeoNode and transform the tokens into a User Principal to be used within the GeoServer Security subsystem itself. - - In other words after a successfull Authentication, GeoServer needs to Authorize the user in order to understand which resources he is enable to access or not. - A ``REST based RoleService`` on GeoNode side, allows GeoServer to talk to GeoNode via `REST `_ to get the current user along with the list of its Roles. - - Nevertheless knowing the Roles associated to a user is not sufficient. The complete GeoServer Authorization needs to catch a set of ``Access Rules``, associated to the Roles, in order to establish which resources and data are accessible by the user. - - The GeoServer Authorization is based on Roles only, therefore for each authenticated user we need also to know: - - 1. The Roles associated to a valid user session - - 2. The access permissions associated to a GeoServer Resource - - The Authentication mechanism above allows GeoServer to get information about the user and his Roles, which addresses point 1. - - About point 2, GeoServer makes use of the `GeoFence Embedded Server `_ plugin. - GeoFence is a java web application that provides an advanced authentication / authorization engine for GeoServer using the interface described in `here `_. - GeoFence has its own rules database for the management of Authorization rules, and overrides the standard GeoServer security management system by implementing a sophisticated Resource Access Manager. - Least but not last, GeoFence implements and exposes a `REST API `_ allowing remote authorized clients to read / write / modify security rules. - - The advantages using such plugin are multiple: - - 1. The Authorizations rules have a fine granularity. The security rules are handled by GeoFence in a way similar to the iptables ones, and allow to define security constraints even on sub-regions and attributes of layers. - - 2. GeoFence exposes a REST interface to its internal rule database, allowing external managers to update the security constraints programmatically - - 3. GeoFence implements an internal caching mechanism which improves considerably the performances under load. - - *GeoNode interaction with GeoFence* - - GeoNode itself is able to push/manage Authorization rules to GeoServer through the GeoFence `REST API `_, acting as an administrator for GeoServer. - GeoNode properly configures the GeoFence rules anytime it is needed, i.e. the permissions of a Resource / Layer are updated. - -GeoServer must know and must be able to access GeoNode via HTTP/HTTPS. In other words, an external user connected to GeoNode must be authenticated to GeoServer with same permissions. This is possible through the **GeoNodeCoockieProcessingFiler**. - -Summarizing we will have different ways to access GeoNode Layers: - -#. Through GeoNode via DJango Authentication and **GeoNodeCoockieProcessingFiler**; basically the users available in GeoNode are also valid for GeoServer or any other backend. - - .. warning:: If a GeoNode user has "administrator" rights, he will be able to administer GeoServer too. - -#. Through GeoServer Security Subsystem; it will be always possible to access to GeoServer using its internal security system and users, unless explictly disabled (**warning** this is dangerous, you must know what you are doing). - -Let's now see in details how the single pieces are configured and how they can be configured. - -DJango OAuth Toolkit Setup and Configuration -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -As stated above, GeoNode makes use of the OAuth2 protocol for all the frontend interactions with GeoServer. GeoNode must be configured as an OAuth2 Provider and provide a ``Client ID`` and a ``Client Sercret`` key to GeoServer. -This is possible by enabling and configuring the `Django OAuth Toolkit Plugin `_. - -.. warning:: GeoNode and GeoServer won't work at all if the following steps are not executed at the first installation. - -Default ``settings.py`` Security Settings for OAuth2 -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Double check that the OAuth2 Provider and Security Plugin is enabled and that the settings below are correctly configured. - -AUTH_IP_WHITELIST -^^^^^^^^^^^^^^^^^ - -``AUTH_IP_WHITELIST`` property limits access to users/groups REST Role Service endpoints to the only whitelisted IP addresses. Empty list means 'allow all'. -If you need to limit 'api' REST calls to only some specific IPs fill the list like this: ``AUTH_IP_WHITELIST = ['192.168.1.158', '192.168.1.159']`` - -Default values are: - -.. code-block:: python - - ... - AUTH_IP_WHITELIST = [] - ... - -INSTALLED_APPS -^^^^^^^^^^^^^^ - -In order to allow GeoNode to act as an OAuth2 Provider, we need to enable the ``oauth2_provider`` DJango application provided by the "Django OAuth Toolkit". - -Default values are: - -.. code-block:: python - - ... - INSTALLED_APPS = ( - - 'modeltranslation', - - ... - 'guardian', - 'oauth2_provider', - ... - - ) + GEONODE_APPS - ... - -MIDDLEWARE_CLASSES -^^^^^^^^^^^^^^^^^^ - -Installing the `oauth2_provider`` DJango application is not sufficient to enable the full functionality. We need also GeoNode to include additional entities to its internal model. - -Default values are: - -.. code-block:: python - - ... - MIDDLEWARE_CLASSES = ( - 'django.middleware.common.CommonMiddleware', - 'django.contrib.sessions.middleware.SessionMiddleware', - 'django.contrib.messages.middleware.MessageMiddleware', - - # The setting below makes it possible to serve different languages per - # user depending on things like headers in HTTP requests. - 'django.middleware.locale.LocaleMiddleware', - 'pagination.middleware.PaginationMiddleware', - 'django.middleware.csrf.CsrfViewMiddleware', - 'django.contrib.auth.middleware.AuthenticationMiddleware', - 'django.middleware.clickjacking.XFrameOptionsMiddleware', - - # If you use SessionAuthenticationMiddleware, be sure it appears before OAuth2TokenMiddleware. - # SessionAuthenticationMiddleware is NOT required for using django-oauth-toolkit. - 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', - 'oauth2_provider.middleware.OAuth2TokenMiddleware', - ) - ... - -AUTHENTICATION_BACKENDS -^^^^^^^^^^^^^^^^^^^^^^^ - -In order to allow GeoNode to act as an OAuth2 Provider, we need to enable the ``oauth2_provider.backends.OAuth2Backend`` DJango backend provided by the "Django OAuth Toolkit". -Also notice that we need to specify the OAuth2 Provider scopes and declare which ``generator`` to use in order to create ``OAuth2 Client IDs``. - -Default values are: - -.. code-block:: python - - ... - # Replacement of default authentication backend in order to support - # permissions per object. - AUTHENTICATION_BACKENDS = ( - 'oauth2_provider.backends.OAuth2Backend', - 'django.contrib.auth.backends.ModelBackend', - 'guardian.backends.ObjectPermissionBackend', - ) - - OAUTH2_PROVIDER = { - 'SCOPES': { - 'read': 'Read scope', - 'write': 'Write scope', - 'groups': 'Access to your groups' - }, - - 'CLIENT_ID_GENERATOR_CLASS': 'oauth2_provider.generators.ClientIdGenerator', - } - ... - -Django OAuth Toolkit Admin Setup -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Once the ``settings.py`` and ``local_settings.py`` have been correctly configured for your system: - -#. Complete the GeoNode setup steps - - * Prepare the model - - .. code-block:: python - - python manage.py makemigrations - python manage.py migrate - python manage.py syncdb - - * Prepare the static data - - .. code-block:: python - - python manage.py collectstatic - - * Make sure the database has been populated with initial default data - - .. warning:: *Deprecated* this command will be replaced by migrations in the future, so be careful. - - .. code-block:: python - - python manage.py loaddata initial_data.json - - * Make sure there exists a superuser for your environment - - .. warning:: *Deprecated* this command will be replaced by migrations in the future, so be careful. - - .. code-block:: python - - python manage.py createsuperuser - - .. note:: Read the base tutorials on GeoNode Developer documentation for details on the specific commands and how to use them. - -#. Start the application - - Start GeoNode accordingly on how the setup has been done; run debug mode through ``paver``, or proxied by an HTTP Server like Apache2 HTTPD, Nginx or others. - -#. Finalize the setup of the OAuth2 Provider - - First of all you need to configure and create a new OAuth2 Application called ``GeoServer`` through the GeoNode Admin Dashboard - - * Access the GeoNode Admin Dashboard - - .. figure:: img/oauth002.png - - * Go to ``Django OAuth Toolkit`` > ``Applications`` - - .. figure:: img/oauth003.png - - * Update or create the Application named ``GeoServer`` - - .. warning:: The Application name **must** be ``GeoServer`` - - .. figure:: img/oauth004.png - - - ``Client id``; An alphanumeric code representing the OAuth2 Client Id. GeoServer OAuth2 Plugin **will** use **this** value. - - .. warning:: In a production environment it is **highly** recommended to modify the default value provided with GeoNode installation. - - - ``User``; Search for the ``admin`` user. Its ``ID`` will be automatically updated into the form. - - - ``Redirect uris``; It is possible to specify many URIs here. Those must coincide with the GeoServer instances URIs. - - - ``Client type``; Choose ``Confidential`` - - - ``Authorization grant type``; Choose ``Authorization code`` - - - ``Client secret``; An alphanumeric code representing the OAuth2 Client Secret. GeoServer OAuth2 Plugin **will** use **this** value. - - .. warning:: In a production environment it is **highly** recommended to modify the default value provided with GeoNode installation. - - - ``Name``; **Must** be ``GeoServer`` - -GeoServer Security Backend --------------------------- - -GeoServer Security Subsystem -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -GeoServer has a robust security subsystem, modeled on Spring Security. Most of the security features are available through the Web administration interface. - -For more details on how this works and how to configure and modify it, please refer to the official GeoServer guide http://docs.geoserver.org/stable/en/user/security/webadmin/index.html - -By using the ``GeoServer Data Dir`` provided with GeoNode build, the following configuration are already available. You will need just to update them accordingly to your environment (like IP addresses and Host names, OAuth2 Keys, and similar things). -However it is recommended to read carefully all the following passages in order to understand exactly how the different component are configured and easily identify any possible issue during the deployment. - -The main topics of this section are: - -#. Connection to the GeoNode REST Role Service - -#. Setup of the GeoServer OAuth2 Authentication Filter - -#. Configuration of the GeoServer Filter Chains - -#. Setup and test of the GeoFence Server and Default Rules - -Connection to the GeoNode REST Role Service -------------------------------------------- - -Preliminary checks -^^^^^^^^^^^^^^^^^^ - -* GeoServer is up and running and you have admin rights - -* GeoServer must reach the GeoNode instance via HTTP - -* The GeoServer Host IP Address must be allowed to access the GeoNode Role Service APIs (see the section ``AUTH_IP_WHITELIST`` above) - -Setup of the GeoNode REST Role Service -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -#. Login as ``admin`` to the GeoServer GUI - - .. warning:: In a production system remember to change the default admin credentials ``admin`` \ ``geoserver`` - - .. figure:: img/oauth005.png - -#. Access the ``Security`` > ``Users, Groups, Roles`` section - - .. figure:: img/oauth006.png - -#. **If not yet configured** the service ``geonode REST role service``, click on ``Role Services`` > ``Add new`` - - .. note:: This passage is **not** needed if the ``geonode REST role service`` has been already created. If so it will be displayed among the Role Services list - - .. figure:: img/oauth008.png - - .. figure:: img/oauth007.png - -#. **If not yet configured** the service ``geonode REST role service``, choose ``AuthKEY REST - Role service from REST endpoint`` - - .. figure:: img/oauth009.png - -#. Create / update the ``geonode REST role service`` accordingly - - .. figure:: img/oauth010.png - - - ``Name``; **Must** be ``geonode REST role service`` - - - ``Base Server URL``; Must point to the GeoNode instance base URL (e.g. ``http://``) - - - ``Roles REST Endpoint``; Enter ``/api/roles`` - - - ``Admin Role REST Endpoint``; Enter ``/api/adminRole`` - - - ``Users REST Endpoint``; Enter ``/api/users`` - - - ``Roles JSON Path``; Enter ``$.groups`` - - - ``Admin Role JSON Path``; Enter ``$.adminRole`` - - - ``Users JSON Path``; Enter ``$.users[0].groups`` - - Once everything has been setup and it is working, choose the ``Administrator role`` and ``Group administrator role`` as ``ROLE_ADMIN`` - -Allow GeoFence to validate rules with ``ROLES`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. warning:: The following instruction are different accordingly to the GeoServer version you are currently using. - -GeoServer 2.9.x and 2.10.x --------------------------- - -#. Access the ``Security`` > ``Settings`` section - - .. figure:: img/oauth011.png - -#. Choose the ``geonode REST role service`` as ``Active role service`` - - .. figure:: img/oauth012.png - -GeoServer 2.12.x and above --------------------------- - -With the latest updates to GeoFence Plugin, the latter no more recognizes the Role Service from the default settings but from the ``geofence-server.properties`` file. - -That said, it is important that the ``Security`` > ``Settings`` role service will be set to **default**, in order to allow GeoServer following the standard authorization chain. - -On the other side, you will need to be sure that the ``geofence-server.properties`` file under the ``$GEOSERVER_DATA_DIR/geofence`` folder, contains the two following additional properties: :: - - gwc.context.suffix=gwc - org.geoserver.rest.DefaultUserGroupServiceName=geonode REST role service - - -Setup of the GeoServer OAuth2 Authentication Filter ---------------------------------------------------- - -It is necessary now check that GeoServer can connect to OAuth2 Providers (specifically to GeoNode OP), and being able to Authenticate users through it. - -Preliminary checks -^^^^^^^^^^^^^^^^^^ - -* GeoServer is up and running and you have admin rights - -* GeoServer must reach the GeoNode instance via HTTP - -* OAuth2 ``Client ID`` and ``Client Secret`` have been generated on GeoNode and known - -Setup of the GeoNode OAuth2 Security Filter -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -#. Access the ``Security`` > ``Authentication`` section - - .. figure:: img/oauth013.png - -#. **If not yet configured** the Authentication Filter ``geonode-oauth2 - Authentication using a GeoNode OAuth2``, click on ``Authentication Filters`` > ``Add new`` - - .. note:: This passage is **not** needed if the ``geonode-oauth2 - Authentication using a GeoNode OAuth2`` has been already created. If so it will be displayed among the Authentication Filters list - - .. figure:: img/oauth015.png - - .. figure:: img/oauth014.png - -#. **If not yet configured** the Authentication Filter ``geonode-oauth2 - Authentication using a GeoNode OAuth2``, choose ``GeoNode OAuth2 - Authenticates by looking up for a valid GeoNode OAuth2 access_token key sent as URL parameter`` - - .. figure:: img/oauth016.png - -#. Create / update the ``geonode-oauth2 - Authentication using a GeoNode OAuth2`` accordingly - - .. figure:: img/oauth017.png - - - ``Name``; **Must** be ``geonode-oauth2`` - - - ``Enable Redirect Authentication EntryPoint``; It is recommended to put this to ``False``, otherwise GeoServer won't allow you to connect to its Admin GUI through the ``Form`` but only through GeoNode - - - ``Login Authentication EndPoint``; Unless you have specific needs, keep the default value ``/j_spring_oauth2_geonode_login`` - - - ``Logout Authentication EndPoint``; Unless you have specific needs, keep the default value ``/j_spring_oauth2_geonode_logout`` - - - ``Force Access Token URI HTTPS Secured Protocol``; This must be ``False`` unless you enabled a ``Secured Connection`` on GeoNode. In that case you will need to trust the GeoNode ``Certificate`` on the GeoServer JVM Keystore. Please see details below - - - ``Access Token URI``; Set this to ``http:///o/token/`` - - - ``Force User Authorization URI HTTPS Secured Protocol``; This must be ``False`` unless you enabled a ``Secured Connection`` on GeoNode. In that case you will need to trust the GeoNode ``Certificate`` on the GeoServer JVM Keystore. Please see details below - - - ``User Authorization URI``; Set this to ``http:///o/authorize/`` - - - ``Redirect URI``; Set this to ``http:///geoserver``. This address **must** be present on the ``Redirect uris`` of GeoNode ``OAuth2`` > ``Applications`` > ``GeoServer`` (see above) - - - ``Check Token Endpoint URL``; Set this to ``http:///api/o/v4/tokeninfo/`` - - - ``Logout URI``; Set this to ``http:///account/logout/`` - - - ``Scopes``; Unless you have specific needs, keep the default value ``read,write,groups`` - - - ``Client ID``; The ``Client id`` alphanumeric key generated by the GeoNode ``OAuth2`` > ``Applications`` > ``GeoServer`` (see above) - - - ``Client Secret``; The ``Client secret`` alphanumeric key generated by the GeoNode ``OAuth2`` > ``Applications`` > ``GeoServer`` (see above) - - - ``Role source``; In order to authorize the user against GeoNode, choose ``Role service`` > ``geonode REST role service`` - -Configuration of the GeoServer Filter Chains --------------------------------------------- - -The following steps ensure GeoServer can adopt more Authentication methods. As stated above, it is possible to Authenticate to GeoServer using different protocols. - -GeoServer scans the authentication filters chain associated to the specified path and tries them one by one sequentially. -The first one matching the protocol and able to grant access to the user, breaks the cycle by creating a ``User Principal`` and injecting it into the GeoServer ``SecurityContext``. -The Authentication process, then, ends here and the control goes to the Authorization one, which will try to retrieve the authenticated user's Roles through the available GeoServer Role Services associated to the Authentication Filter that granted the access. - -Preliminary checks -^^^^^^^^^^^^^^^^^^ - -* GeoServer is up and running and you have admin rights - -* GeoServer must reach the GeoNode instance via HTTP - -* The ``geonode-oauth2 - Authentication using a GeoNode OAuth2`` Authentication Filter and the ``geonode REST role service`` have been correctly configured - -Setup of the GeoServer Filter Chains -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -#. Access the ``Security`` > ``Authentication`` section - - .. figure:: img/oauth013.png - -#. Identify the section ``Filter Chains`` - - .. figure:: img/oauth018.png - -#. Make sure the ``web`` Filter Chain is configured as shown below - - .. figure:: img/oauth019.png - - .. warning:: Every time you modify a Filter Chain, **don't forget to save** the ``Authentication`` settings. This **must** be done for **each** change. - - .. figure:: img/oauth020.png - -#. Make sure the ``rest`` Filter Chain is configured as shown below - - .. figure:: img/oauth021.png - - .. warning:: Every time you modify a Filter Chain, **don't forget to save** the ``Authentication`` settings. This **must** be done for **each** change. - - .. figure:: img/oauth020.png - -#. Make sure the ``gwc`` Filter Chain is configured as shown below - - .. figure:: img/oauth022.png - - .. warning:: Every time you modify a Filter Chain, **don't forget to save** the ``Authentication`` settings. This **must** be done for **each** change. - - .. figure:: img/oauth020.png - -#. Make sure the ``default`` Filter Chain is configured as shown below - - .. figure:: img/oauth023.png - - .. warning:: Every time you modify a Filter Chain, **don't forget to save** the ``Authentication`` settings. This **must** be done for **each** change. - - .. figure:: img/oauth020.png - -#. Add the ``GeoNode Login Endpoints`` to the comma-delimited list of the ``webLogin`` Filter Chain - - .. figure:: img/oauth024.png - - .. warning:: Every time you modify a Filter Chain, **don't forget to save** the ``Authentication`` settings. This **must** be done for **each** change. - - .. figure:: img/oauth020.png - -#. Add the ``GeoNode Logout Endpoints`` to the comma-delimited list of the ``webLogout`` Filter Chain - - .. figure:: img/oauth025.png - - .. warning:: Every time you modify a Filter Chain, **don't forget to save** the ``Authentication`` settings. This **must** be done for **each** change. - - .. figure:: img/oauth020.png - - -#. Add the ``GeoNode Logout Endpoints`` to the comma-delimited list of the ``formLogoutChain`` XML node in ``/security/filter/formLogout/config.xml`` - - You will need a text editor to modify the file. - - .. note:: If the ```` XML node does not exist at all, create a **new one** as specified below - - .. code-block:: xml - - - ... - /web/ - /j_spring_security_logout,/j_spring_security_logout/,/j_spring_oauth2_geonode_logout,/j_spring_oauth2_geonode_logout/ - - - .. warning:: The value ``j_spring_oauth2_geonode_logout`` **must** be the same specified as ``Logout Authentication EndPoint`` in the ``geonode-oauth2 - Authentication using a GeoNode OAuth2`` above. - -Setup and test of the GeoFence Server and Default Rules -------------------------------------------------------- - -In order to work correctly, GeoServer needs the `GeoFence Embedded Server `_ plugin to be installed and configured on the system. - -The GeoServer configuration provided for GeoNode, has the plugin already installed with a default configuration. In that case double check that the plugin works correctly and the default rules have been setup by following the next steps. - -Preliminary checks -^^^^^^^^^^^^^^^^^^ - -* GeoServer is up and running and you have admin rights - -* The `GeoFence Embedded Server `_ plugin has been installed on GeoServer - -Setup of the GeoServer Filter Chains -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -#. Access the ``Security`` > ``Authentication`` section - - .. figure:: img/oauth013.png - -#. Identify the section ``Authentication Providers`` and make sure the ``geofence`` Authentication Provider is present - - .. figure:: img/oauth032.png - -#. Make sure the ``Provider Chain`` is configured as shown below - - .. figure:: img/oauth033.png - - .. warning:: Every time you modify an Authentication Providers, **don't forget to save** the ``Authentication`` settings. This **must** be done for **each** change. - - .. figure:: img/oauth020.png - -Setup of the GeoFence Server and Rules -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -#. Make sure GeoFence server works and the default settings are correctly configured - - - Access the ``Security`` > ``GeoFence`` section - - .. figure:: img/oauth026.png - - - Make sure the ``Options`` are configured as follows and the server works well when performing a ``Test Connection`` - - .. figure:: img/oauth027.png - - * ``Allow remote and inline layers in SLD``; Set it to ``True`` - - * ``Allow SLD and SLD_BODY parameters in requests``; Set it to ``True`` - - * ``Authenticated users can write``; Set it to ``True`` - - * ``Use GeoServer roles to get authorizations``; Set it to ``False`` - -#. Check the GeoFence default Rules - - - Access the ``Security`` > ``GeoFence Data Rules`` section - - .. figure:: img/oauth028.png - - - Make sure the ``DENY ALL`` Rule is present by default, otherwise your data will be accessible to everyone - - .. note:: This rule is **always** the last one - - .. figure:: img/oauth029.png - - .. warning:: If that rule does not exists **at the very bottom** (this rule is **always** the last one), add it manually. - - - Access the ``Security`` > ``GeoFence Admin Rules`` section - - .. figure:: img/oauth030.png - - - No Rules needed here - - .. figure:: img/oauth031.png - -Troubleshooting and Advanced Features --------------------------------------- - -Common Issues and Fixes -^^^^^^^^^^^^^^^^^^^^^^^ - -* GeoServer/GeoNode OAuth2 does not authenticate as Administrator even using GeoNode ``admin`` users - - **Symptoms** - - When trying to authenticate with an ``admin`` user using OAuth2, the process correctly redirects to GeoServer page but I'm not a GeoServer Administrator. - - **Cause** - - That means that somehow GeoServer could not successfully complete the Authorization and Authentication process. - - The possible causes of the problem may be the following ones: - - 1. The OAuth2 Authentication fails on GeoServer side - - This is usually due to an exception while trying to complete the Authentication process. - - - A typical cause is that GeoServer tries to use HTTPS connections but the GeoNode certificate is not trusted; - - In that case please refer to the section below. Also take a look at the logs (in particular the GeoServer one) as explained in :ref:`debug_geonode`. - The GeoServer logs should contain a detailed Exception explaining the cause of the problem. - If no exception is listed here (even after raised the log level to *DEBUG*), try to check for the GeoNode Role Service as explained below. - - - Another possible issue is that somehow the OAuth2 handshake cannot complete successfully; - - 1. Login into GeoServer as administrator through its WEB login form. - - 2. Double check that all the ``geonode-oauth2 - Authentication using a GeoNode OAuth2`` parameters are correct. If everything is ok, take a look at the logs (in particular the GeoServer one) as explained in :ref:`debug_geonode`. - The GeoServer logs should contain a detailed Exception explaining the cause of the problem. If no exception is listed here (even after raised the log level to *DEBUG*), try to check for the GeoNode Role Service as explained below. - - 2. GeoServer is not able to retrieve the user Role from a Role Service - - Always double check both HTTP Server and GeoServer log as specified in section :ref:`debug_geonode`. This might directly guide you to the cause of the problem. - - - Check that the GeoServer host is granted to access GeoNode Role Service REST APIs in the ``AUTH_IP_WHITELIST`` of the ``settings.py`` - - - Check that the ``geonode REST role service`` is the default Role service and that the GeoServer OAuth2 Plugin has been configured to use it by default - - - Check that the GeoNode REST Role Service APIs are functional and produce correct JSON. - - This is possible by using simple ``cUrl`` GET calls like - - .. code-block:: shell - - curl http://localhost/api/adminRole - $> {"adminRole": "admin"} - - - curl http://localhost/api/users - $> {"users": [{"username": "AnonymousUser", "groups": ["anonymous"]}, {"username": "afabiani", "groups": ["anonymous", "test"]}, {"username": "admin", "groups": ["anonymous", "test", "admin"]}]} - - curl http://localhost/api/roles - $> {"groups": ["anonymous", "test", "admin"]} - - curl http://localhost/api/users/admin - $> {"users": [{"username": "admin", "groups": ["anonymous", "test", "admin"]}]} - - -How to setup ``HTTPS`` secured endpoints -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In a production system it is a good practice to encrypt the connection between GeoServer and GeoNode. That would be possible by enabling HTTPS Protocol on the GeoNode REST Role Service APIs and OAuth2 Endpoints. - -Most of the times you will rely on a self-signed HTTPS connection using a generated certificate. That makes the connection *untrusted* and you will need to tell to the GeoServer Java Virtual Machine to trust it. - -This can be done by following the steps below. - -For any issue take a look at the logs (in particular the GeoServer one) as explained in :ref:`debug_geonode`. The GeoServer logs should contain a detailed Exception explaining the cause of the problem. - -SSL Trusted Certificates ------------------------- - -When using a custom ``Keystore`` or trying to access a non-trusted or self-signed SSL-protected OAuth2 Provider from a non-SSH connection, you will need to add the certificates to the JVM ``Keystore``. - -In order to do this you can follow the next steps: - - In this example we are going to - - #. Retrieve SSL Certificate from GeoNode domain: - - "Access Token URI" = https:///o/token/ therefore we need to trust ``https://`` or (``:443``) - - .. note:: You will need to get and trust certificates from every different HTTPS URL used on OAuth2 Endpoints. - - #. Store SSL Certificates on local hard-disk - - #. Add SSL Certificates to the Java Keystore - - #. Enable the JVM to check for SSL Certificates from the Keystore - -1. Retrieve the SSL Certificate from GeoNode domain - - Use the ``openssl`` command in order to dump the certificate - - For ``https://`` - - .. code-block:: shell - - openssl s_client -connect :443 - - .. figure:: img/google_ssl_001.png - :align: center - -2. Store SSL Certificate on local hard-disk - - Copy-and-paste the section ``-BEGIN CERTIFICATE-``, ``-END CERTIFICATE-`` and save it into a ``.cert`` file - - .. note:: ``.cert`` file are plain text files containing the ASCII characters included on the ``-BEGIN CERTIFICATE-``, ``-END CERTIFICATE-`` sections - - ``geonode.cert`` (or whatever name you want with ``.cert`` extension) - - .. figure:: img/google_ssl_003.png - :align: center - -3. Add SSL Certificates to the Java Keystore - - You can use the Java command ``keytool`` like this - - ``geonode.cert`` (or whatever name you want with ``.cert`` extension) - - .. code-block:: shell - - keytool -import -noprompt -trustcacerts -alias geonode -file geonode.cert -keystore ${KEYSTOREFILE} -storepass ${KEYSTOREPASS} - - or, alternatively, you can use some graphic tool which helps you managing the SSL Certificates and Keystores, like `Portecle `_ - - .. code-block:: shell - - java -jar c:\apps\portecle-1.9\portecle.jar - - .. figure:: img/google_ssl_005.png - :align: center - - .. figure:: img/google_ssl_006.png - :align: center - - .. figure:: img/google_ssl_007.png - :align: center - - .. figure:: img/google_ssl_008.png - :align: center - - .. figure:: img/google_ssl_009.png - :align: center - - .. figure:: img/google_ssl_010.png - :align: center - - .. figure:: img/google_ssl_011.png - :align: center - - .. figure:: img/google_ssl_012.png - :align: center - - .. figure:: img/google_ssl_013.png - :align: center - -4. Enable the JVM to check for SSL Certificates from the Keystore - - In order to do this, you need to pass a ``JAVA_OPTION`` to your JVM: - - .. code-block:: shell - - -Djavax.net.ssl.trustStore=F:\tmp\keystore.key - -5. Restart your server - -.. note:: Here below you can find a bash script which simplifies the Keystore SSL Certificates importing. Use it at your convenience. - - .. code-block:: shell - - HOST=myhost.example.com - PORT=443 - KEYSTOREFILE=dest_keystore - KEYSTOREPASS=changeme - - # get the SSL certificate - openssl s_client -connect ${HOST}:${PORT} ${HOST}.cert - - # create a keystore and import certificate - keytool -import -noprompt -trustcacerts \ - -alias ${HOST} -file ${HOST}.cert \ - -keystore ${KEYSTOREFILE} -storepass ${KEYSTOREPASS} - - # verify we've got it. - keytool -list -v -keystore ${KEYSTOREFILE} -storepass ${KEYSTOREPASS} -alias ${HOST} - -GeoFence Advanced Features --------------------------- - -GeoFence Rules Management and Tutorials -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -* `This tutorial `_ shows how to install and configure the Geofence Internal Server plug-in. It shows how to create rules in two ways: using the GUI and REST methods. - -* GeoFence Rules can be created / updated / deleted through a REST API, accessible only by a GeoServer Admin user. You can find more details on how the GeoFence REST API works `here `_. - -GeoFence Rules Storage Configuration -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -By default GeoFence is configured to use a filesystem based DB stored on the GeoServer Data Dir ```_. - -1. Add ``Java Libraries`` to ``GeoServer`` - - .. code-block:: shell - - wget --no-check-certificate "https://maven.geo-solutions.it/org/hibernatespatial/hibernate-spatial-postgis/1.1.3.2/hibernate-spatial-postgis-1.1.3.2.jar" -O hibernate-spatial-postgis-1.1.3.2.jar - wget --no-check-certificate "https://repo1.maven.org/maven2/org/postgis/postgis-jdbc/1.3.3/postgis-jdbc-1.3.3.jar" -O postgis-jdbc-1.3.3.jar - - cp hibernate-spatial-postgis-1.1.3.2.jar /WEB-INF/lib - cp postgis-jdbc-1.3.3.jar /WEB-INF/lib - - restart geoserver - -2. Either create a DB with the updated schema here https://github.com/geoserver/geofence/blob/master/doc/setup/sql/002_create_schema_postgres.sql or enable the hbm2ddl auto creation through the configuration file (see step ``3``) - - .. note:: Notice that "update" also creates the tables if they do not exist. In production, however, I would suggest to change it to "validate" - - .. code-block:: shell - - # If you want to create a new DB for GeoFence - sudo -u postgres createdb -O geonode geofence; \ - sudo -u postgres psql -d geofence -c 'CREATE EXTENSION postgis;'; \ - sudo -u postgres psql -d geofence -c 'GRANT ALL ON geometry_columns TO PUBLIC;'; \ - sudo -u postgres psql -d geofence -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;'; \ - sudo -u postgres psql -d geofence -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO geonode;' - -3. Add configuration similar to ``geofence-datasource-ovr.properties`` sample below (if loaded as GeoServer extension) - - **/geofence/geofence-datasource-ovr.properties** - - .. code-block:: shell - - # /* (c) 2019 Open Source Geospatial Foundation - all rights reserved - # * This code is licensed under the GPL 2.0 license, available at the root - # * application directory. - # */ - # - geofenceVendorAdapter.databasePlatform=org.hibernatespatial.postgis.PostgisDialect - geofenceDataSource.driverClassName=org.postgresql.Driver - geofenceDataSource.url=jdbc:postgresql://localhost:5432/geofence - geofenceDataSource.username=postgres - geofenceDataSource.password=postgres - geofenceEntityManagerFactory.jpaPropertyMap[hibernate.default_schema]=public - - ################################################################################ - ## Other setup entries - ################################################################################ - ## hbm2ddl.auto may assume one of these values: - ## - validate: validates the DB schema at startup against the internal model. May fail on oracle spatial. - ## - update: updates the schema, according to the internal model. Updating automatically the production DB is dangerous. - ## - create-drop: drop the existing schema and recreates it according to the internal model. REALLY DANGEROUS, YOU WILL LOSE YOUR DATA. - ## You may want not to redefine the property entirely, in order to leave the default value (no action). - - geofenceEntityManagerFactory.jpaPropertyMap[hibernate.hbm2ddl.auto]=update - geofenceEntityManagerFactory.jpaPropertyMap[javax.persistence.validation.mode]=none - geofenceEntityManagerFactory.jpaPropertyMap[hibernate.validator.apply_to_ddl]=false - geofenceEntityManagerFactory.jpaPropertyMap[hibernate.validator.autoregister_listeners]=false - - ## - ## ShowSQL is set to true in the configuration file; putting showsql=false in - ## this file, you can easily check that this override file has been properly applied. - - # geofenceVendorAdapter.generateDdl=false - # geofenceVendorAdapter.showSql=false - - ## Set to "true" in specific use cases - # workspaceConfigOpts.showDefaultGroups=false - - - - ################################################################################ - ## Disable second level cache. - ## This is needed in a geofence-clustered environment. - - #geofenceEntityManagerFactory.jpaPropertyMap[hibernate.cache.use_second_level_cache]=false - - ################################################################################ - ## Use external ehcache configuration file. - ## Useful to change cache settings, for example diskStore path. - #geofenceEntityManagerFactory.jpaPropertyMap[hibernate.cache.provider_configuration_file_resource_path]=file:/path/to/geofence-ehcache-override.xml diff --git a/advanced/contrib/index.rst b/advanced/contrib/index.rst deleted file mode 100644 index 9f270c1a..00000000 --- a/advanced/contrib/index.rst +++ /dev/null @@ -1,307 +0,0 @@ -Geonode auth via LDAP -===================== - -This package provides utilities for using LDAP as an authentication and -authorization backend for geonode. - -The `django_auth_ldap `_ package is a very capable way to add LDAP integration -with django projects. It provides a lot of flexibility in mapping LDAP users to -geonode users and is able to manage user authentication. - -However, in order to provide full support for mapping LDAP groups with -geonode's and enforce group permissions on resources, a custom geonode -authentication backend is required. This contrib package provides such a -backend, based on `django_auth_ldap `_. - - -Installation ------------- - -Installing this contrib package is a matter of: - -1. Installing geonode -2. Installing system LDAP libraries (development packages needed) -3. Cloning this repository locally -4. Change to the `ldap` directory and install this contrib package - -.. code-block:: sh - - # 1. install geonode (not shown here for brevity) - # 2. install systemwide LDAP libraries - sudo apt install \ - libldap2-dev \ - libsasl2-dev - - # 3. get geonode/contribs code - git clone https://github.com/GeoNode/geonode-contribs.git - - # 4. install geonode ldap contrib package - cd geonode-contribs/ldap - pip install . - -Configuration -------------- - -1. Add ``geonode_ldap.backend.GeonodeLdapBackend`` as an additional auth - backend. - - .. code-block:: python - - # e.g. by updating your settings.py or local_settings.py - AUTHENTICATION_BACKENDS += ( - "geonode_ldap.backend.GeonodeLdapBackend", - ) - - You may use additional auth backends, the django authentication framework - tries them all according to the order listed in the settings. This means that - geonode can be setup in such a way as to permit internal organization users - to login with their LDAP credentials, while at the same time allowing for - casual users to use their facebook login (as long as you enable facebook - social auth provider). - - .. note:: The django's ``django.contrib.auth.backends.ModelBackend`` must also be used in order to provide full geonode integration with LDAP. - However this is included by default on GeoNode ``settings`` - - .. code-block:: python - - # The GeoNode default settings are the following - AUTHENTICATION_BACKENDS = ( - 'oauth2_provider.backends.OAuth2Backend', - 'django.contrib.auth.backends.ModelBackend', - 'guardian.backends.ObjectPermissionBackend', - 'allauth.account.auth_backends.AuthenticationBackend', - ) - -2. Set some additional configuration values. Some of these variables are - prefixed with ``AUTH_LDAP`` (these are used directly by `django_auth_ldap `_) - while others are prefixed with ``GEONODE_LDAP`` (these are used by - ``geonode_ldap``). The geonode custom variables are: - - * ``GEONODE_LDAP_GROUP_PROFILE_FILTERSTR`` - This is an LDAP search fragment - with the filter that allows querying for existing groups. See example below - - * ``GEONODE_LDAP_GROUP_NAME_ATTRIBUTE`` - This is the name of the LDAP - attribute that will be used for deriving the geonode group name. If not - specified it will default to `cn`, which means that the LDAP object's - `common name` will be used for generating the name of the geonode group - - * ``GEONODE_LDAP_GROUP_PROFILE_MEMBER_ATTR`` - This is the name of the LDAP - attribute that will be used for deriving the geonode membership. If not - specified it will default to ``member`` - - -Example configuration: - -.. code-block:: python - - # add these import lines to the top of your geonode settings file - from django_auth_ldap import config as ldap_config - from geonode_ldap.config import GeonodeNestedGroupOfNamesType - import ldap - - # enable logging - import logging - logger = logging.getLogger('django_auth_ldap') - logger.addHandler(logging.StreamHandler()) - logger.setLevel(logging.DEBUG) - - # add both standard ModelBackend auth and geonode.contrib.ldap auth - AUTHENTICATION_BACKENDS += ( - 'geonode_ldap.backend.GeonodeLdapBackend', - ) - - # django_auth_ldap configuration - AUTH_LDAP_SERVER_URI = os.getenv("LDAP_SERVER_URL") - AUTH_LDAP_BIND_DN = os.getenv("LDAP_BIND_DN") - AUTH_LDAP_BIND_PASSWORD = os.getenv("LDAP_BIND_PASSWORD") - AUTH_LDAP_USER_SEARCH = ldap_config.LDAPSearch( - os.getenv("LDAP_USER_SEARCH_DN"), - ldap.SCOPE_SUBTREE, - os.getenv("LDAP_USER_SEARCH_FILTERSTR") - ) - # should LDAP groups be used to spawn groups in GeoNode? - AUTH_LDAP_MIRROR_GROUPS = strtobool(os.getenv("LDAP_MIRROR_GROUPS", 'True')) - AUTH_LDAP_GROUP_SEARCH = ldap_config.LDAPSearch( - os.getenv("LDAP_GROUP_SEARCH_DN"), - ldap.SCOPE_SUBTREE, - os.getenv("LDAP_GROUP_SEARCH_FILTERSTR") - ) - AUTH_LDAP_GROUP_TYPE = GeonodeNestedGroupOfNamesType() - AUTH_LDAP_USER_ATTR_MAP = { - "first_name": "givenName", - "last_name": "sn", - "email": "mailPrimaryAddress" - } - AUTH_LDAP_FIND_GROUP_PERMS = True - AUTH_LDAP_MIRROR_GROUPS_EXCEPT = [ - "test_group" - ] - - # these are not needed by django_auth_ldap - we use them to find and match - # GroupProfiles and GroupCategories - GEONODE_LDAP_GROUP_NAME_ATTRIBUTE = os.getenv("LDAP_GROUP_NAME_ATTRIBUTE", default="cn") - GEONODE_LDAP_GROUP_PROFILE_FILTERSTR = os.getenv("LDAP_GROUP_SEARCH_FILTERSTR", default='(ou=research group)') - GEONODE_LDAP_GROUP_PROFILE_MEMBER_ATTR = os.getenv("LDAP_GROUP_PROFILE_MEMBER_ATTR", default='member') - -Example environment variables: - -.. code-block:: shell - - LDAP_SERVER_URL=ldap:// - LDAP_BIND_DN=uid=ldapinfo,cn=users,dc=ad,dc=example,dc=org - LDAP_BIND_PASSWORD= - LDAP_USER_SEARCH_DN=dc=ad,dc=example,dc=org - LDAP_USER_SEARCH_FILTERSTR=(&(uid=%(user)s)(objectClass=person)) - LDAP_MIRROR_GROUPS=True - LDAP_GROUP_SEARCH_DN=cn=groups,dc=ad,dc=example,dc=org - LDAP_GROUP_SEARCH_FILTERSTR=(|(cn=abt1)(cn=abt2)(cn=abt3)(cn=abt4)(cn=abt5)(cn=abt6)) - LDAP_GROUP_PROFILE_MEMBER_ATTR=uniqueMember - -The configuration seen in the example above will allow LDAP users to login to -geonode with their LDAP credentials. - -On first login, a geonode user is created from the LDAP user and its LDAP -attributes ``cn`` and ``sn`` are used to populate the geonode user's -``first_name`` and ``last_name`` profile fields. - -Any groups that the user is a member of in LDAP (under the -``cn=groups,dc=ad,dc=example,dc=org`` search base and belonging to one of -``(|(cn=abt1)(cn=abt2)(cn=abt3)(cn=abt4)(cn=abt5)(cn=abt6))`` groups) will be mapped to the corresponding -geonode groups, even creating these groups in geonode in case they do not -exist yet. The geonode user is also made a member of these geonode groups. - -Upon each login, the user's geonode group memberships are re-evaluated -according to the information extracted from LDAP. The -``AUTH_LDAP_MIRROR_GROUPS_EXCEPT`` setting can be used to specify groups -whose memberships will not be re-evaluated. - -If no LDAP groups shall be mirrored ``LDAP_MIRROR_GROUPS`` and ``LDAP_MIRROR_GROUPS_EXCEPT`` must be set to ``False``. - -.. note:: Users mapped from LDAP will be marked with an ``ldap`` tag. This will be used to keep them in sync. - -.. warning:: If you remove the ``ldap`` tag, the users will be threaten as pure internal GeoNode ones. - -You may also manually generate the geonode groups in advance, before users -login. In this case, when a user logs in and the mapped LDAP group already -exists, the user is merely added to the geonode group - -Be sure to check out `django_auth_ldap `_ for more information on the various -configuration options. - -Keep Users and Groups Synchronized ----------------------------------- - -In order to constantly keep the remote LDAP Users and Groups **synchronized** with GeoNode, -you will need to run periodically some specific management commands. - -.. code-block:: shell - - */10 * * * * /opt/geonode/my-geonode/manage.sh updateldapgroups >> /var/log/cron.log 2>&1 - */10 * * * * /opt/geonode/my-geonode/manage.sh updateldapusers >> /var/log/cron.log 2>&1 - -Where the ``manage.sh`` is a bash script similar to the following one: - -**manage.sh** - -.. code-block:: shell - - export $(grep -v '^#' /opt/geonode/my-geonode/.env | xargs -d '\n'); /home//.virtualenvs/geonode/bin/python /opt/geonode/my-geonode/manage.py $@ - -and the ``/opt/geonode/my-geonode/.env`` is something similar to the following one: - -**/opt/geonode/my-geonode/.env** - -.. code-block:: shell - - DEBUG=False - DJANGO_ALLOWED_HOSTS=,localhost,127.0.0.1 - DJANGO_DATABASE_URL=postgis://my_geonode:**********@localhost:5432/my_geonode_db - DEFAULT_BACKEND_UPLOADER=geonode.importer - DEFAULT_FROM_EMAIL=geonode@example.org - DJANGO_EMAIL_HOST=smtp.example.org - DJANGO_EMAIL_HOST_PASSWORD=********** - DJANGO_EMAIL_HOST_USER=geonode - DJANGO_EMAIL_PORT=465 - DJANGO_EMAIL_USE_SSL=True - DJANGO_SETTINGS_MODULE=my_geonode.settings - DJANGO_SECRET_KEY=********** - OAUTH2_API_KEY=********** - PROXY_URL=/proxy/?url= - EXIF_ENABLED=True - EMAIL_ENABLE=True - TIME_ENABLED=True - ACCOUNT_OPEN_SIGNUP=True - ACCOUNT_APPROVAL_REQUIRED=True - ACCOUNT_EMAIL_REQUIRED=True - ACCOUNT_EMAIL_VERIFICATION=optional - AVATAR_GRAVATAR_SSL=True - GEONODE_DB_URL=postgis://my_geonode:**********@localhost:5432/my_geonode_data - GEOSERVER_ADMIN_PASSWORD=********** - GEOSERVER_LOCATION=https:///geoserver/ - GEOSERVER_PUBLIC_HOST= - GEOSERVER_PUBLIC_LOCATION=https:///geoserver/ - GEOSERVER_WEB_UI_LOCATION=https:///geoserver/ - LDAP_SERVER_URL=ldap:// - LDAP_BIND_DN=uid=ldapinfo,cn=users,dc=ad,dc=example,dc=org - LDAP_BIND_PASSWORD= - LDAP_USER_SEARCH_DN=dc=ad,dc=example,dc=org - LDAP_USER_SEARCH_FILTERSTR=(&(uid=%(user)s)(objectClass=person)) - LDAP_MIRROR_GROUPS=True - LDAP_GROUP_SEARCH_DN=cn=groups,dc=ad,dc=example,dc=org - LDAP_GROUP_SEARCH_FILTERSTR=(|(cn=abt1)(cn=abt2)(cn=abt3)(cn=abt4)(cn=abt5)(cn=abt6)) - LDAP_GROUP_PROFILE_MEMBER_ATTR=uniqueMember - OGC_REQUEST_MAX_RETRIES=3 - OGC_REQUEST_POOL_CONNECTIONS=100 - OGC_REQUEST_POOL_MAXSIZE=100 - OGC_REQUEST_TIMEOUT=60 - SITEURL=https:/// - SITE_HOST_NAME= - FREETEXT_KEYWORDS_READONLY=False - # Advanced Workflow Settings - ADMIN_MODERATE_UPLOADS=False - GROUP_MANDATORY_RESOURCES=False - GROUP_PRIVATE_RESOURCES=False - RESOURCE_PUBLISHING=False - -.. note:: You might want to use the same ``/opt/geonode/my-geonode/.env`` for your ``UWSGI`` configuration too: - - .. code-block:: shell - - [uwsgi] - socket = 0.0.0.0:8000 - uid = - gid = www-data - - plugins = python3 - virtualenv = /home//.virtualenvs/geonode - - # set environment variables from .env file - env LANG=en_US.utf8 - env LC_ALL=en_US.UTF-8 - env LC_LANG=en_US.UTF-8 - - for-readline = /opt/geonode/my-geonode/.env - env = %(_) - endfor = - - chdir = /opt/geonode/my-geonode - module = my_geonode.wsgi:application - - processes = 12 - threads = 2 - enable-threads = true - master = true - - # logging - # path to where uwsgi logs will be saved - logto = /storage/my_geonode/logs/geonode.log - daemonize = /storage/my_geonode/logs/geonode.log - touch-reload = /opt/geonode/my-geonode/my_geonode/wsgi.py - buffer-size = 32768 - max-requests = 500 - harakiri = 300 # respawn processes taking more than 5 minutes (300 seconds) - # limit-as = 1024 # avoid Errno 12 cannot allocate memory - harakiri-verbose = true - vacuum = true - thunder-lock = true - diff --git a/advanced/hardening/index.rst b/advanced/hardening/index.rst deleted file mode 100644 index 18ee4c22..00000000 --- a/advanced/hardening/index.rst +++ /dev/null @@ -1,38 +0,0 @@ -.. _geonode_on_https: - -Publish on other than HTTP port (for e.g. 8082) -=============================================== - -By default geonode will be installed in the port 80 (i.e. HTTP) port. But what if you want to change the port of the geonode to other than HTTP port (For this example, I am taking 8082 port)? We need to edit couple of things in the web configuration. First things is, we need to update the `/etc/uwsgi/apps-enabled/geonode.ini` file, - -.. code-block:: shell - - sudo vi /etc/uwsgi/apps-enabled/geonode.ini - -Edit the following lines, - -.. code-block:: shell - - env = SITE_HOST_NAME=localhost:8082 - env = SITEURL=http://localhost:8082 - - SITE_HOST_NAME=localhost - SITE_HOST_PORT=8082 - GEOSERVER_WEB_UI_LOCATION=http://localhost:8082/geoserver/ - GEOSERVER_PUBLIC_LOCATION=http://localhost:8082/geoserver/ - - - -After that we need to update the `/etc/nginx/sites-enabled/geonode` file, - -.. code-block:: shell - - sudo vi /etc/nginx/sites-enabled/geonode - -Edit the following lines, - -.. code-block:: shell - - server { - listen 8082 default_server; - listen [::]:8082 default_server; diff --git a/advanced/index.rst b/advanced/index.rst deleted file mode 100644 index 98807e69..00000000 --- a/advanced/index.rst +++ /dev/null @@ -1,36 +0,0 @@ -GeoNode Components and Architecture -=================================== - -.. toctree:: - :caption: GeoNode Components and Architecture - :maxdepth: 3 - - components/index - -Hardening GeoNode -================= - -.. toctree:: - :caption: Hardening GeoNode - :maxdepth: 3 - - hardening/index - -Social Login -============ - -.. toctree:: - :caption: Social Login - :maxdepth: 3 - - social/index - -GeoNode Django Contrib Apps -=========================== - -.. toctree:: - :caption: GeoNode Django Contrib Apps - :maxdepth: 3 - - contrib/index - diff --git a/advanced/social/index.rst b/advanced/social/index.rst deleted file mode 100644 index 7c1f244a..00000000 --- a/advanced/social/index.rst +++ /dev/null @@ -1,112 +0,0 @@ -GeoNode Social Accounts -======================= - -.. contents:: - :depth: 4 - -Allow GeoNode to Login throguh Social Accounts (OIDC) ------------------------------------------------------ - -Introduction -~~~~~~~~~~~~ - - Through the so-called "social accounts," GeoNode allows authentication through external providers that support the OIDC (OpenID Connect) protocol, such as Google and Microsoft Azure. - - How does it work? Once the authentication provider is configured, GeoNode's sign-on will display a new login button that redirects the user to the external authentication page. After successfully authenticating with the external provider and accepting privacy consents, the browser will redirect the user back to the GeoNode page, prompting them to enter any missing information the first time and automatically authenticating them on subsequent occasions. - - -Usage -~~~~~ - - Once the provider has been correctly configured (see below), GeoNode will allow the user to login through it. - - The first time you login you will probably need to confirm you `e-mail` and other fields of the `profile`. - - .. note:: - - If you want a user to be automatically signed as a member of a group, you will need to: - - 1. Create the `GroupProfile` in GeoNode - 2. Provide the `groups` or `roles` the user belongs to throguh the `id_token` or `user_info` metadata from the `OIDC` provider itself. - - Notice that, in the case you would like to benefit from this functionality: - - 1. Every time the user sing-in again, the groups will be automatically re-assigned by GeoNode, and therefore it won't be possible to assign them manually anymore. - 2. If you need a user to be recognized as a `manager` of the `groups` declared from the provider, you will need to send a claim `is_manager: True` on the user info metadata. - -Quick Configuration -~~~~~~~~~~~~~~~~~~~ - - Currently GeoNode comes with two predefined configurations that you can use to enable either Google or Microsoft Azure. - - **Google** - - 1. Add to your `.env` the following settings - - .. code-block:: shell - - SOCIALACCOUNT_OIDC_PROVIDER_ENABLED=True - SOCIALACCOUNT_PROVIDER=google - - - 2. Login into GeoNode as an `admin`, go to the `Social Account` settings, create a new `geonode_openid_connect` provider and insert the - - .. code-block:: shell - - Client ID - Client Secret - - **Microsoft Azure** - - 1. Add to your `.env` the following settings - - .. code-block:: shell - - MICROSOFT_TENANT_ID= - SOCIALACCOUNT_OIDC_PROVIDER_ENABLED=True - SOCIALACCOUNT_PROVIDER=azure - - 2. Login into GeoNode as an `admin`, go to the `Social Account` settings, create a new `geonode_openid_connect` provider and insert the - - .. code-block:: shell - - Client ID - Client Secret - -Advanced Configuration -~~~~~~~~~~~~~~~~~~~~~~ - - In the case you need to change the default behavior of GeoNode or add a new/custom OIDC provider, you will need to update the `settings` manually as follows. - - .. code-block:: python - - SOCIALACCOUNT_PROVIDERS = { - SOCIALACCOUNT_OIDC_PROVIDER: { - "NAME": "Your Custom Provider", - "SCOPE": [ - # Custom scopes comma-separated - ], - "AUTH_PARAMS": { - # Custom AUTH PARAMS - }, - "COMMON_FIELDS": {"email": "email", "last_name": "family_name", "first_name": "given_name"}, # Custom common fields mappings - "IS_MANAGER_FIELD": "the_custom_manager_claim", # This is optional - "ACCOUNT_CLASS": "the_custom_account_class", - "ACCESS_TOKEN_URL": "the_custom_token_uri", - "AUTHORIZE_URL": "the_custom_auth_uri", - "ID_TOKEN_ISSUER": "the_custom_uri", # or "PROFILE_URL": "the_custom_user_info_uri"; if you specify the "ID_TOKEN_ISSUER" this will take precedence - "OAUTH_PKCE_ENABLED": True, - } - } - - .. note:: - - If you specify the "ID_TOKEN_ISSUER" this will take precedence trying to fetch the user info metadata from the `id_token`. - - If the `id_token` won't be available, it will try to fallback to the "PROFILE_URL" uri. - - In the case you will need to customzie how the `Adapter` works and manages the `Groups` registration, you can inject a new class throguh the settings: - - .. code:: shell - - SOCIALACCOUNT_ADAPTER="geonode.people.adapters.GenericOpenIDConnectAdapter" # This is the default value diff --git a/basic/docker_env_vars/index.rst b/basic/docker_env_vars/index.rst deleted file mode 100644 index fbadbda3..00000000 --- a/basic/docker_env_vars/index.rst +++ /dev/null @@ -1,179 +0,0 @@ -.. _dockerenvvars: - -============================== -Docker Environmental variables -============================== - -This list reports the specific environmental variables requried for the configuration and execution of GeoNode Docker services - -.. comment: - :local: - :depth: 1 - -********* -Geoserver -********* -DATABASE_HOST - | Default: ``db`` - - Internal IP or hostname for the PosgreSQL (``db``) Docker container. - -DATABASE_PORT - | Default: ``django`` - - Internal port for the PosgreSQL (``db``) Docker container. - -GEONODE_LB_HOST_IP - | Default: ``django`` - - Internal IP or hostname for the GeoNode (``django``) Docker container. - -GEONODE_LB_PORT - | Default: ``8000`` - - Internal port for the GeoNode (``django``) Docker container. - -GEONODE_GEODATABSE - | Default: ``geonode_data`` - - GeoNode geodatabase name. - -GEONODE_GEODATABASE_USER - | Default: ``geonode_data`` - - GeoNode geodatabase user. - -GEONODE_GEODATABASE_PASSWORD - | Default: ``geonode_data`` - - GeoNode geodatabase password. - -GEONODE_GEODATABASE_SCHEMA - | Default: ``publix`` - - GeoNode geodatabase schema. - -GEOSERVER_JAVA_OPTS - | Default: ``'-Djava.awt.headless=true -Xms4G -Xmx4G -Dgwc.context.suffix=gwc -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:LogFile=/var/log/jvm.log -XX:PerfDataSamplingInterval=500 -XX:SoftRefLRUPolicyMSPerMB=36000 -XX:-UseGCOverheadLimit -XX:ParallelGCThreads=4 -Dfile.encoding=UTF8 -Djavax.servlet.request.encoding=UTF-8 -Djavax.servlet.response.encoding=UTF-8 -Duser.timezone=GMT -Dorg.geotools.shapefile.datetime=false -DGS-SHAPEFILE-CHARSET=UTF-8 -DGEOSERVER_CSRF_DISABLED=true -DPRINT_BASE_URL={geoserver_ui}/geoserver/pdf -DALLOW_ENV_PARAMETRIZATION=true -Xbootclasspath/a:/usr/local/tomcat/webapps/geoserver/WEB-INF/lib/marlin-0.9.3-Unsafe.jar -Dsun.java2d.renderer=org.marlin.pisces.MarlinRenderingEngine'`` - Geoserver JAVA OPTS. - -GEOSERVER_LB_HOST_IP - | Default: ``geoserver`` - - Internal IP or hostname for the GeoNode (``geoserver``) Docker container. - -GEOSERVER_LB_PORT - | Default: ``8080`` - - Internal port for the GeoNode (``geoserver``) Docker container. - -NGINX_BASE_URL - | Default: ``http(s):/localhost`` - - Public URL of the ``nginx`` service. This value must match GeoNodes's ``SITEURL`` setting - -OAUTH2_API_KEY - | Default: ``(empty)`` - - Optionsl API Key for the communication between Geoserver and GeoNode Oauth2 services. - -OAUTH2_CLIENT_ID - | Default: ``(empty)`` - - Geoserer Client ID for GeoNode Oauth2 server. Cannot be empty. - -OAUTH2_CLIENT_SECRET - | Default: ``(empty)`` - - Geoserer Client secret for GeoNode Oauth2 server. Cannot be empty. - -***** -Nginx -***** - -GEONODE_LB_HOST_IP - | Default: ``django`` - - Internal IP or hostname for the GeoNode (``django``) Docker container. - -GEONODE_LB_PORT - | Default: ``8000`` - - Internal port for the GeoNode (``django``) Docker container. - -GEOSERVER_LB_HOST_IP - | Default: ``geoserver`` - - Internal IP or hostname for the GeoNode (``geoserver``) Docker container. - -GEOSERVER_LB_PORT - | Default: ``8080`` - - Internal port for the GeoNode (``geoserver``) Docker container. - -HTTPS_HOST - | Default: ``localhost`` - - The publis hostname for HTTPS (wihtout ``http:/`` scheme prefix). - -HTTPS_HOST - | Default: ``localhost`` - - The publis hostname for HTTPS (wihtout ``https:/`` scheme prefix). - -HTTP_PORT - | Default: ``80`` - - Host port on which to expose the Nginx service for HTTP. - -HTTPS_PORT - | Default: ``443`` - - Host port on which to expose the Nginx service for HTTPS. - - -LETSENCRYPT_MODE - | Default: ``disabled`` - - Possible values: ``disabled`` (SSL disabled), ``staging`` (SSL local cert), ``production`` (SSL released by an ACME server) - -******************** -PostgreSQL / PostGIS -******************** - -GEONODE_DATABASE - | Default: ``geonode`` - - GeoNode database name. - -GEONODE_DATABASE_USER - | Default: ``geonode_data`` - - GeoNode database user. - -GEONODE_DATABASE_PASSWORD - | Default: ``geonode_data`` - - GeoNode database password. - -GEONODE_GEODATABSE - | Default: ``geonode_data`` - - GeoNode geodatabase name. - -GEONODE_GEODATABASE_USER - | Default: ``geonode_data`` - - GeoNode geodatabase user. - -GEONODE_GEODATABASE_PASSWORD - | Default: ``geonode_data`` - GeoNode geodatabase password. - -POSTGRES_USER - | Default: ``geonode_data`` - PostgreSQL admin user. - -POSTGRES_USER - | Default: ``geonode_data`` - PostgreSQL admin password. diff --git a/basic/index.rst b/basic/index.rst deleted file mode 100644 index 1f8e960a..00000000 --- a/basic/index.rst +++ /dev/null @@ -1,37 +0,0 @@ -GeoNode Settings -================ - - :ref:`settings` - -.. toctree:: - :hidden: - :maxdepth: 3 - - settings/index - -Docker Environmental variables -=============================== - - :ref:`dockerenvvars` - -.. toctree:: - :hidden: - :maxdepth: 3 - - docker_env_vars/index - -Customize the Look and Feel -=========================== - -.. toctree:: - :maxdepth: 3 - - theme/index - -GeoNode permissions -=================== - -.. toctree:: - :maxdepth: 3 - - permissions/index diff --git a/basic/permissions/index.rst b/basic/permissions/index.rst deleted file mode 100644 index 76bc22d5..00000000 --- a/basic/permissions/index.rst +++ /dev/null @@ -1,112 +0,0 @@ -.. _permissions: - -=========== -Permissions -=========== - -| Permissions in GeoNode are set per resource, where a resource can be a dataset, a map, a document, a service or a geoapp. The way the permissions are set is the same for all of them. - -.. warning:: - - GeoNode has a set of default permissions that are applied on resource creation **when** you don't explicitly declare them. - This is particularly relevant when creating and saving a map, where you won't have the possibility to set the its permissions during the creation phase. - GeoNode can be tuned to make sure that by default the new created resource are not public, this can be done by changing two settings, see `Default view permissions <../settings/index.html#default-anonymous-view-permission>`_ and `Default download permissions <../settings/index.html#default-anonymous-download-permission>`_ - -Single Resource permissions ---------------------------- - -| Resource permissions can be generally set from the *resource detail* page. The detail page has a menu item *Share* which is visible to people who are permitted to set permissions on a resource. -The share link opens a page on the right with a provision to edit user and group permissions on the resource. see picture below - -.. figure:: img/change_dataset_permissions.png - :align: center - :width: 500 - - *Change Dataset Permissions* - -| The page for setting the permissions, allows addition of users/groups and selection of a permission to assign each of them. - -.. figure:: img/resource_permissions_dialogue.png - :align: center - :width: 500 - - *Resource Permission Dialogue* - -You can set the following types of permissions: - -* *View:* allows to view the resource; -* *Download* allows to download the resource; -* *Edit:* allows to change attributes, properties of the datasets features, styles and metadata for the specified resource; -* *Manage:* allows to update, delete, change permissions, publish and unpublish the resource. - -.. warning:: When assigning permissions to a group, all the group members will have those permissions. Be careful in case of editing permissions. - -Geo Limits permissions ----------------------- - -.. note:: This feature is available **only** when enabling ```GeoServer``` as geospatial backend. Also make sure that the properties ```GEONODE_SECURITY_ENABLED```, ```GEOFENCE_SECURITY_ENABLED``` and ```GEOFENCE_URL``` are correctly set for the ```OGC_SERVER```. - -:guilabel:`Geo Limits` are an extension of the GeoNode standard permissions. :guilabel:`Geo Limits` allows the owner of the resource, or the administrator, to restrict users or groups to a specific geographical area, in order to limit the access to the dataset to only the portions contained within that geographic restriction, excluding data outside of it. - -In order to be able to set :guilabel:`Geo Limits` you must be an ```administrator``` of the system or the ```owner``` of the resource or you must have ```Manage Permissions``` rights to the resource. - -If you have the permissions to set the :guilabel:`Geo Limits`, you should be able to see the permissions section and the globe icon on each user or group. - -.. figure:: img/geo_limits_001.png - :align: center - :width: 500 - - *Geo Limits Icon* - -You should be able to see an interactive preview of the resource along with few small drawing tools, that allows you to start creating limits on the map manually if you want. - -This opens a map dialog, with 3 options at the top: - -.. figure:: img/geo_limits_002.png - :align: center - :width: 500 - - *Geo Limits: Preview Window with Drawing Tools* - - -The |edit_icon| icon allows you to draw limits on a map for which a user will be able to see. Click on it to start drawing on the map. Once you are done drawing, click on it again to deactivate drawing mode. - -.. |edit_icon| image:: img/edit_icon.png - :width: 30px - :height: 30px - :align: middle - -The |delete_icon| icon enables you to remove the limits you have drawn. Click on the limit drawn, and then click the delete icon. - -.. |delete_icon| image:: img/delete_icon.png - :width: 30px - :height: 30px - :align: middle - -The |refresh_icon| icon removes all changes that are not saved. - -.. |refresh_icon| image:: img/refresh_icon.png - :width: 30px - :height: 30px - :align: middle - - -.. figure:: img/geo_limits_003.png - :align: center - :width: 500 - - *Geo Limits: Editing the Geometries* - -Once you finished editing your geometries, save them into the DB using the *Save* link in the resource menu. - -The user with the specified geometries won't be able from now on to access the whole dataset data. - -.. figure:: img/geo_limits_004.png - :align: center - :width: 500 - - *Geo Limits: Geospatial restrictions applies for the user* - -.. warning :: The :guilabel:`Geo Limits` will be persisted on GeoNode DB for that resource. That means that everytime you will update the general permissions, also the geospatial restrictions will be applied. - - In order to remove the :guilabel:`Geo Limits` for a certain user or group, you can just :guilabel:`Save` an **empty geometry**. This will **delete** the entry from the DB also. diff --git a/basic/settings/index.rst b/basic/settings/index.rst deleted file mode 100644 index 295b2bae..00000000 --- a/basic/settings/index.rst +++ /dev/null @@ -1,2646 +0,0 @@ -.. _settings: - -======== -Settings -======== - -Here’s a list of settings available in GeoNode and their default values. This includes settings for some external applications that -GeoNode depends on. - -For most of them, default values are good. Those should be changed only for advanced configurations in production or heavily hardened systems. - -The most common ones can be set through ``environment`` variables to avoid touching the ``settings.py`` file at all. -This is a good practice and also the preferred one to configure GeoNode (and Django apps in general). -Whenever you need to change them, set the environment variable accordingly (where it is available) instead of overriding it through the ``local_settings``. - -.. comment: - :local: - :depth: 1 - -A -= - -ACCESS_TOKEN_EXPIRE_SECONDS ---------------------------- - - | Default: ``86400`` - | Env: ``ACCESS_TOKEN_EXPIRE_SECONDS`` - - When a user logs into GeoNode, if no ``ACCESS_TOKEN`` exists, a new one will be created with a default expiration time of ``ACCESS_TOKEN_EXPIRE_SECONDS`` seconds (1 day by default). - -ACCOUNT_ADAPTER ---------------- - - | Default: ``geonode.people.adapters.LocalAccountAdapter`` - - Custom GeoNode People (Users) Account Adapter. - -ACCOUNT_APPROVAL_REQUIRED -------------------------- - - | Default: ``False`` - | Env: ``ACCOUNT_APPROVAL_REQUIRED`` - - If ``ACCOUNT_APPROVAL_REQUIRED`` equals ``True``, newly registered users must be activated by a superuser through the Admin gui, before they can access GeoNode. - -ACCOUNT_CONFIRM_EMAIL_ON_GET ----------------------------- - - | Default: ``True`` - - This is a `django-allauth setting `__ - It allows specifying the HTTP method used when confirming e-mail addresses. - -ACCOUNT_EMAIL_REQUIRED ----------------------- - - | Default: ``True`` - - This is a `django-allauth setting `__ - which controls whether the user is required to provide an e-mail address upon registration. - -ACCOUNT_EMAIL_VERIFICATION --------------------------- - - | Default: ``optional`` - - This is a `django-allauth setting `__ - -ACCOUNT_LOGIN_REDIRECT_URL --------------------------- - - | Default: ``SITEURL`` - | Env: ``LOGIN_REDIRECT_URL`` - - This is a `django-user-accounts setting `__ - It allows specifying the default redirect URL after a successful login. - -ACCOUNT_LOGOUT_REDIRECT_URL ---------------------------- - - | Default: ``SITEURL`` - | Env: ``LOGOUT_REDIRECT_URL`` - - This is a `django-user-accounts setting `__ - It allows specifying the default redirect URL after a successful logout. - -ACCOUNT_NOTIFY_ON_PASSWORD_CHANGE ---------------------------------- - - | Default: ``True`` - | Env: ``ACCOUNT_NOTIFY_ON_PASSWORD_CHANGE`` - - This is a `django-user-accounts setting `__ - -ACCOUNT_OPEN_SIGNUP -------------------- - - | Default: ``True`` - | Env: ``ACCOUNT_OPEN_SIGNUP`` - - This is a `django-user-accounts setting `__ - Whether or not people are allowed to self-register to GeoNode or not. - -ACCOUNT_SIGNUP_FORM_CLASS -------------------------- - - | Default: ``geonode.people.forms.AllauthReCaptchaSignupForm`` - | Env: ``ACCOUNT_SIGNUP_FORM_CLASS`` - - Enabled only when the :ref:`recaptcha_enabled` option is ``True``. - - Ref. to :ref:`recaptcha_enabled` - -ACTSTREAM_SETTINGS ------------------- - - Default:: - - { - 'FETCH_RELATIONS': True, - 'USE_PREFETCH': False, - 'USE_JSONFIELD': True, - 'GFK_FETCH_DEPTH': 1, - } - - Actstream Settings. - - -ADDITIONAL_DATASET_FILE_TYPES ------------------------------ - -External application can define additional supported file type other than the default one declared in the `SUPPORTED_DATASET_FILE_TYPES` . - -The variable should be declared in this way in `settings.py` (or via application hook): - - - .. code:: - ADDITIONAL_DATASET_FILE_TYPES=[ - { - "id": "dummy_type", - "label": "Dummy Type", - "format": "dummy", - "ext": ["dummy"] - }, - ] - -Please rely on geonode.tests.test_utils.TestSupportedTypes for an example - -ADMIN_IP_WHITELIST ----------------------- - - | Default: ``[]`` - - When this list is popuplated with a list of IPs or IP ranges (e.g. 192.168.1.0/24) requests from and admin user will be allowe only from IPs matching with the list. - -ADMIN_MODERATE_UPLOADS ----------------------- - - | Default: ``False`` - - When this variable is set to ``True``, every uploaded resource must be approved before becoming visible to the public users. - - Until a resource is in ``PENDING APPROVAL`` state, only the superusers, owner and group members can access it, unless specific edit permissions have been set for other users or groups. - - A ``Group Manager`` *can* approve the resource, but he cannot publish it whenever the setting ``RESOURCE_PUBLISHING`` is set to ``True``. - Otherwise, if ``RESOURCE_PUBLISHING`` is set to ``False``, the resource becomes accessible as soon as it is approved. - -ADMINS_ONLY_NOTICE_TYPES ------------------------- - - | Default: ``['monitoring_alert',]`` - - A list of notification labels that standard users should not either see or set. - - Such notifications will be hidden from the notify settings page and automatically set to false for non-superusers. - - -ADVANCED_EDIT_EXCLUDE_FIELD ---------------------------- - | Default: ``[]`` - - A list of element (item name) to exclude from the Advanced Edit page. - - Example: - - ``ADVANCED_EDIT_EXCLUDE_FIELD=['title', 'keywords', 'tkeywords']`` - - -AGON_RATINGS_CATEGORY_CHOICES ------------------------------ - - Default:: - - { - "maps.Map": { - "map": "How good is this map?" - }, - "layers.Layer": { - "layer": "How good is this layer?" - }, - "documents.Document": { - "document": "How good is this document?" - } - } - - -ALLOWED_DOCUMENT_TYPES ----------------------- - - Default:: - - ['doc', 'docx', 'gif', 'jpg', 'jpeg', 'ods', 'odt', 'odp', 'pdf', 'png', - 'ppt', 'pptx', 'rar', 'sld', 'tif', 'tiff', 'txt', 'xls', 'xlsx', 'xml', - 'zip', 'gz', 'qml'] - - A list of acceptable file extensions that can be uploaded to the Documents app. - -ANONYMOUS_USER_ID ------------------ - - | Default: ``-1`` - | Env: ``ANONYMOUS_USER_ID`` - - The id of an anonymous user. This is an django-guardian setting. - -API_INCLUDE_REGIONS_COUNT -------------------------- - - | Default: ``False`` - | Env: ``API_INCLUDE_REGIONS_COUNT`` - - If set to ``True``, a counter with the total number of available regions will be added to the API JSON Serializer. - -API_LIMIT_PER_PAGE ------------------- - - | Default: ``200`` - | Env: ``API_LIMIT_PER_PAGE`` - - The Number of items returned by the APIs 0 equals no limit. Different from ``CLIENT_RESULTS_LIMIT``, affecting the number of items per page in the resource list. - -API_LOCKDOWN ------------- - - | Default: ``True`` - | Env: ``API_LOCKDOWN`` - - If this is set to ``True`` users must be authenticated to get search results when search for for users, groups, categories, regions, tags etc. - Filtering search results of Resourcebase-objects like Layers, Maps or Documents by one of the above types does not work. - Attention: If API_LOCKDOWN is set to ``False`` all details can be accessed by anonymous users. - -ASYNC_SIGNALS -------------- - - | Default: ``False`` - | Env: ``ACCOUNT_NOTIFY_ON_PASSWORD_CHANGE`` - -AUTH_EXEMPT_URLS ----------------- - - Default:: - - (r'^/?$', - '/gs/*', - '/static/*', - '/o/*', - '/api/o/*', - '/api/roles', - '/api/adminRole', - '/api/users', - '/api/layers',) - - A tuple of URL patterns that the user can visit without being authenticated. - This setting has no effect if ``LOCKDOWN_GEONODE`` is not True. For example, - ``AUTH_EXEMPT_URLS = ('/maps',)`` will allow unauthenticated users to - browse maps. - -AUTO_ASSIGN_REGISTERED_MEMBERS_TO_CONTRIBUTORS ---------------------------------------------------------------- - - | Default: ``True`` - | Env: ``AUTO_ASSIGN_REGISTERED_MEMBERS_TO_CONTRIBUTORS`` - - Assign **new** registered users to the contributors group. If set to `False` new registered members will not obtain automatic permissions to create and edit resources. - -AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_NAME ---------------------------------------------------------------- - - | Default: ``True`` - | Env: ``AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_NAME`` - - Auto assign users to a default ``REGISTERED_MEMBERS_GROUP_NAME`` private group after ``AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_AT``. - -AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_AT -------------------------------------------------------------- - - | Default: ``activation`` - | Env: ``AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_AT`` - | Options: ``"registration" | "activation" | "login"`` - - Auto assign users to a default ``REGISTERED_MEMBERS_GROUP_NAME`` private group after {"registration" | "activation" | "login"}. - - Notice that whenever ``ACCOUNT_EMAIL_VERIFICATION == True`` and ``ACCOUNT_APPROVAL_REQUIRED == False``, users will be able to register and they became ``active`` already, even if they won't be able to login until the email has been verified. - -AUTO_GENERATE_AVATAR_SIZES --------------------------- - - | Default: ``20, 30, 32, 40, 50, 65, 70, 80, 100, 140, 200, 240`` - - An iterable of integers representing the sizes of avatars to generate on upload. This can save rendering time later on if you pre-generate the resized versions. - -AVATAR_GRAVATAR_SSL -------------------- - - | Default: ``False`` - | Env: ``AVATAR_GRAVATAR_SSL`` - | Options: ``True | False`` - - Force SSL when loading fallback image from gravatar.com. - -AVATAR_DEFAULT_URL ------------------- - - | Default: ``/geonode/img/avatar.png`` - | Env: ``AVATAR_GRAVATAR_SSL`` - | Options: ``"filepath to image"`` - - Allows to set a custom fallback image in case a User has not uploaded a profile image. - Needs ``AVATAR_PROVIDERS`` to be set correctly. - -AVATAR_PROVIDERS ----------------- - - | Default: - .. code-block:: - - 'avatar.providers.PrimaryAvatarProvider','avatar.providers.GravatarAvatarProvider','avatar.providers.DefaultAvatarProvider' - - - | Env: ``AVATAR_PROVIDERS`` - | Options: ``Avatar provider object`` - - - This setting configures in which order gravatar images are loaded. A common use case is the use of a local image over a fallback image loaded from gravatar.com. - To do so you would change the order like: - - .. code-block:: - - 'avatar.providers.PrimaryAvatarProvider','avatar.providers.DefaultAvatarProvider','avatar.providers.GravatarAvatarProvider' - - (DefaultAvatarProvider before GravatarAvatarProvider) - - -B -= - -BING_API_KEY ------------- - - | Default: ``None`` - | Env: ``BING_API_KEY`` - - This property allows to enable a Bing Aerial background. - - If using ``mapstore`` client library, make sure the ``MAPSTORE_BASELAYERS`` include the following: - - .. code-block:: python - - if BING_API_KEY: - BASEMAP = { - "type": "bing", - "title": "Bing Aerial", - "name": "AerialWithLabels", - "source": "bing", - "group": "background", - "apiKey": "{{apiKey}}", - "visibility": False - } - DEFAULT_MS2_BACKGROUNDS = [BASEMAP,] + DEFAULT_MS2_BACKGROUNDS - - -BROKER_HEARTBEAT ----------------- - - | Default: ``0`` - - Heartbeats are used both by the client and the broker to detect if a connection was closed. - This is a `Celery setting `__. - - -BROKER_TRANSPORT_OPTIONS ------------------------- - - Default:: - - { - 'fanout_prefix': True, - 'fanout_patterns': True, - 'socket_timeout': 60, - 'visibility_timeout': 86400 - } - - This is a `Celery setting `__. - - -C -= - -CACHES ------- - - Default:: - - CACHES = { - 'default': { - 'BACKEND': 'django.core.cache.backends.dummy.DummyCache', - }, - 'resources': { - 'BACKEND': 'django.core.cache.backends.locmem.LocMemCache', - 'TIMEOUT': 600, - 'OPTIONS': { - 'MAX_ENTRIES': 10000 - } - } - } - - A dictionary containing the settings for all caches to be used with Django. - This is a `Django setting `__ - - The ``'default'`` cache is disabled because we don't have a mechanism to discriminate between client sessions right now, and we don't want all users fetch the same api results. - - The ``'resources'`` is not currently used. It might be helpful for `caching Django template fragments `__ and/or `Tastypie API Caching `__. - - -CACHE_BUSTING_STATIC_ENABLED ----------------------------- - - | Default: ``False`` - | Env: ``CACHE_BUSTING_STATIC_ENABLED`` - - This is a `Django Compressed Manifet storage provided by WhiteNoise `__ - A boolean allowing you to enable the ``WhiteNoise CompressedManifestStaticFilesStorage storage``. This works only on a production system. - - .. warning:: This works only if ``DEBUG = False`` - - -CASCADE_WORKSPACE ------------------ - - | Default: ``geonode`` - | Env: ``CASCADE_WORKSPACE`` - - -CATALOGUE ---------- - - A dict with the following keys: - - ENGINE: The CSW backend (default is ``geonode.catalogue.backends.pycsw_local``) - URL: The FULLY QUALIFIED base URL to the CSW instance for this GeoNode - USERNAME: login credentials (if required) - PASSWORD: login credentials (if required) - - pycsw is the default CSW enabled in GeoNode. pycsw configuration directives - are managed in the PYCSW entry. - -CATALOGUE_METADATA_TEMPLATE ---------------------------- - - Default : ``catalogue/full_metadata.xml`` - - A string with the catalogue xml file needed for the metadata. - -CATALOGUE_METADATA_XSL ----------------------- - - Default : ``'/static/metadataxsl/metadata.xsl`` - - A string pointing to the XSL used to transform the metadata XML into human readable HTML. - -CELERYD_POOL_RESTARTS ---------------------- - - Default: ``True`` - - This is a `Celery setting `__. - -CELERY_ACCEPT_CONTENT ---------------------- - - Defaul: ``['json']`` - - This is a `Celery setting `__. - -CELERY_ACKS_LATE ----------------- - - Default: ``True`` - - This is a `Celery setting `__ - -CELERY_BEAT_SCHEDULE --------------------- - - Here you can define your scheduled task. - -CELERY_DISABLE_RATE_LIMITS --------------------------- - - Default: ``False`` - - This is a `Celery setting `__. - -CELERY_ENABLE_UTC ------------------ - - Default: ``True`` - - This is a `Celery setting `__. - -CELERY_MAX_CACHED_RESULTS -------------------------- - - Default: ``32768`` - - This is a `Celery setting `__. - -CELERY_MESSAGE_COMPRESSION --------------------------- - - Default: ``gzip`` - - This is a `Celery setting `__. - -CELERY_RESULT_PERSISTENT ------------------------- - - Default: ``False`` - - This is a `Celery setting `__. - -CELERY_RESULT_SERIALIZER ------------------------- - - Default: ``json`` - - This is a `Celery setting `__. - -CELERY_SEND_TASK_SENT_EVENT ---------------------------- - - Default: ``True`` - - If enabled, a task-sent event will be sent for every task so tasks can be tracked before they are consumed by a worker. This is a `Celery setting `__. - - -CELERY_TASK_ALWAYS_EAGER ------------------------- - - Default: ``False if ASYNC_SIGNALS else True`` - - This is a `Celery setting `__. - -CELERY_TASK_CREATE_MISSING_QUEUES ---------------------------------- - - Default: ``True`` - - This is a `Celery setting `__. - -CELERY_TASK_IGNORE_RESULT -------------------------- - - Default: ``True`` - - This is a `Celery setting `__. - -CELERY_TASK_QUEUES ------------------- - - Default:: - - Queue('default', GEONODE_EXCHANGE, routing_key='default'), - Queue('geonode', GEONODE_EXCHANGE, routing_key='geonode'), - Queue('update', GEONODE_EXCHANGE, routing_key='update'), - Queue('cleanup', GEONODE_EXCHANGE, routing_key='cleanup'), - Queue('email', GEONODE_EXCHANGE, routing_key='email'), - - A tuple with registered Queues. - -CELERY_TASK_RESULT_EXPIRES --------------------------- - - | Default: ``43200`` - | Env: ``CELERY_TASK_RESULT_EXPIRES`` - - This is a `Celery setting `__. - -CELERY_TASK_SERIALIZER ----------------------- - - Default: ``json`` - Env: ``CELERY_TASK_SERIALIZER`` - - This is a `Celery setting `__. - -CELERY_TIMEZONE ---------------- - - | Default: ``UTC`` - | Env: ``TIME_ZONE`` - - This is a `Celery setting `__. - -CELERY_TRACK_STARTED --------------------- - - Default: ``True`` - - This is a `Celery setting `__. - -CELERY_WORKER_DISABLE_RATE_LIMITS ---------------------------------- - - Default: ``False`` - - Disable the worker rate limits (number of tasks that can be run in a given time frame). - -CELERY_WORKER_SEND_TASK_EVENTS ------------------------------- - - Default: ``False`` - - Send events so the worker can be monitored by other tools. - -CLIENT_RESULTS_LIMIT --------------------- - - | Default: ``5`` - | Env: ``CLIENT_RESULTS_LIMIT`` - - The Number of results per page listed in the GeoNode search pages. Different from ``API_LIMIT_PER_PAGE``, affecting the number of items returned by the APIs. - -CORS_ALLOW_ALL_ORIGINS ----------------------- - - | Default: ``False`` - | Env: ``CORS_ALLOW_ALL_ORIGINS`` - - If set to true `Access-Control-Allow-Origin: *` header is set for any response. A safer option (not managed through env vars at the moment) is `CORS_ALLOWED_ORIGINS`, where a list of hosts can be configured, o `CORS_ALLOWED_ORIGIN_REGEXES`, where the list can contain regexes. - Notice that the Nginx in front of GeoNode always includes `Access-Control-Allow-Credentials true`. This must also taken into account when CORS is enabled. - -CREATE_LAYER ------------- - - | Default: ``False`` - | Env: ``CREATE_LAYER`` - - Enable the create layer plugin. - -CKAN_ORIGINS ------------- - - Default:: - - CKAN_ORIGINS = [{ - "label":"Humanitarian Data Exchange (HDX)", - "url":"https://data.hdx.rwlabs.org/dataset/new?title={name}¬es={abstract}", - "css_class":"hdx" - }] - - A list of dictionaries that are used to generate the links to CKAN instances displayed in the Share tab. For each origin, the name and abstract format parameters are replaced by the actual values of the ResourceBase object (layer, map, document). This is not enabled by default. To enable, uncomment the following line: SOCIAL_ORIGINS.extend(CKAN_ORIGINS). - -CSRF_COOKIE_HTTPONLY --------------------- - - | Default: ``False`` - | Env: ``CSRF_COOKIE_HTTPONLY`` - - Whether to use HttpOnly flag on the CSRF cookie. If this is set to True, client-side JavaScript will not be able to access the CSRF cookie. This is a `Django Setting `__ - -CSRF_COOKIE_SECURE ------------------- - - | Default: ``False`` - | Env: ``CSRF_COOKIE_SECURE`` - - Whether to use a secure cookie for the CSRF cookie. If this is set to True, the cookie will be marked as “secure,” which means browsers may ensure that the cookie is only sent with an HTTPS connection. This is a `Django Setting `__ - -CUSTOM_METADATA_SCHEMA ----------------------- - - | Default: ``{}`` - - If present, will extend the available metadata schema used for store - new value for each resource. By default override the existing one. - The expected schema is the same as the default - -D -= - -DATA_UPLOAD_MAX_NUMBER_FIELDS ------------------------------ - - Default: ``100000`` - - Maximum value of parsed attributes. - - -DATASET_DOWNLOAD_HANDLERS -------------------------- - - Default: ``[]`` - - Additional download handlers that provides a link to download the resource - -DEBUG ------ - - | Default: ``False`` - | Env: ``DEBUG`` - - One of the main features of debug mode is the display of detailed error pages. If your app raises an exception when DEBUG is True, Django will display a detailed traceback, including a lot of metadata about your environment, such as all the currently defined Django settings (from settings.py). - This is a `Django Setting `__ - -DEFAULT_ANONYMOUS_DOWNLOAD_PERMISSION -------------------------------------- - - Default: ``True`` - - Whether the uploaded resources should downloadable by default. - -DEFAULT_ANONYMOUS_VIEW_PERMISSION ---------------------------------- - - Default: ``True`` - - Whether the uploaded resources should be public by default. - -DEFAULT_AUTO_FIELD ------------------- - - Default: ``django.db.models.AutoField`` - - Default primary key field type to use for models that don’t have a field with primary_key=True. - Django documentation https://docs.djangoproject.com/it/3.2/ref/settings/#std:setting-DEFAULT_AUTO_FIELD - -DEFAULT_DATASET_DOWNLOAD_HANDLER --------------------------------- - - Default: ``geonode.layers.download_handler.DatasetDownloadHandler`` - - from GeoNode 4.2.x has bee introduced with this issue #11296 and later improved with this issue #11421 - the concept of Download Handler and ofc GeoNode provides a default implementation of it - which process the download via WPS - -DEFAULT_EXTRA_METADATA_SCHEMA ------------------------------ - - Default - - .. code-block:: json - - { - Optional("id"): int, - "filter_header": object, - "field_name": object, - "field_label": object, - "field_value": object, - } - - Define the default metadata schema used for add to the resource extra metadata without modify the actual model. - This schema is used as validation for the input metadata provided by the user - - - `id`: (optional int): the identifier of the metadata. Optional for creation, required in Upgrade phase - - `filter_header`: (required object): Can be any type, is used to generate the facet filter header. Is also an identifier. - - `field_name`: (required object): name of the metadata field - - `field_label`: (required object): verbose string of the name. Is used as a label in the facet filters. - - `field_value`: (required object): metadata values - - An example of metadata that can be ingested is the follow: - - .. code-block:: json - - [ - { - "filter_header": "Bike Brand", - "field_name": "name", - "field_label": "Bike Name", - "field_value": "KTM", - }, - { - "filter_header": "Bike Brand", - "field_name": "name", - "field_label": "Bike Name", - "field_value": "Bianchi", - } - ] - - -DEFAULT_LAYER_FORMAT --------------------- - - | Default: ``image/png`` - | Env: ``DEFAULT_LAYER_FORMAT`` - - The default format for requested tile images. - - -DEFAULT_MAP_CENTER ------------------- - - | Default: ``(0, 0)`` - | Env: ``DEFAULT_MAP_CENTER_X`` ``DEFAULT_MAP_CENTER_Y`` - - A 2-tuple with the latitude/longitude coordinates of the center-point to use - in newly created maps. - -DEFAULT_MAP_CRS ---------------- - - | Default: ``EPSG:3857`` - | Env: ``DEFAULT_MAP_CRS`` - - The default map projection. Default: EPSG:3857 - -DEFAULT_MAP_ZOOM ----------------- - - | Default: ``0`` - | Env: ``DEFAULT_MAP_ZOOM`` - - The zoom-level to use in newly created maps. This works like the OpenLayers - zoom level setting; 0 is at the world extent and each additional level cuts - the viewport in half in each direction. - -DEFAULT_MAX_PARALLEL_UPLOADS_PER_USER ------------------------ - -Default: ``5`` - -When `uploading datasets <../../usage/managing_datasets/uploading_datasets.html#datasets-uploading>`__, -this value limits the number os parallel uploads. - -The parallelism limit is set during installation using the value of this variable. -After installation, only an user with administrative rights can change it. -These limits can be changed in the `admin panel <../../admin/upload-parallelism-limit/index.html#upload-parallelism-limit>`__ -or `accessing by api <../../devel/api/V2/index.html#get--api-v2-upload-parallelism-limits->`__. - - - -DEFAULT_MAX_UPLOAD_SIZE ------------------------ - -Default: ``104857600`` (100 MB in bytes) - -When `uploading datasets <../../usage/managing_datasets/uploading_datasets.html#datasets-uploading>`__ -or `uploading documents <../../usage/managing_documents/uploading_documents.html#uploading-documents>`__, -the total size of the uploaded files is verified. - -The size limits are set during installation using the value of this variable. -After installation, only an user with administrative rights can change it. -These limits can be changed in the `admin panel <../../admin/upload-size-limits/index.html#upload-size-limits>`__ -or `accessing by api <../../devel/api/V2/index.html#get--api-v2-upload-size-limits->`__. - - -DEFAULT_SEARCH_SIZE -------------------- - - | Default: ``10`` - | Env: ``DEFAULT_SEARCH_SIZE`` - - An integer that specifies the default search size when using ``geonode.search`` for querying data. - -DEFAULT_WORKSPACE ------------------ - - | Default: ``geonode`` - | Env: ``DEFAULT_WORKSPACE`` - - The standard GeoServer workspace. - -DELAYED_SECURITY_SIGNALS ------------------------- - - | Default: ``False`` - | Env: ``DELAYED_SECURITY_SIGNALS`` - - This setting only works when ``GEOFENCE_SECURITY_ENABLED`` has been set to ``True`` and GeoNode is making use of the ``GeoServer BACKEND``. - - By setting this to ``True``, every time the permissions will be updated/changed for a Layer, they won't be applied immediately but only and only if - either: - - a. A Celery Worker is running and it is able to execute the ``geonode.security.tasks.synch_guardian`` periodic task; - notice that the task will be executed at regular intervals, based on the interval value defined in the corresponding PeriodicTask model. - - b. A periodic ``cron`` job runs the ``sync_security_rules`` management command, or either it is manually executed from the Django shell. - - c. The user, owner of the Layer or with rights to change its permissions, clicks on the GeoNode UI button ``Sync permissions immediately`` - - .. warning:: Layers won't be accessible to public users anymore until the Security Rules are not synchronized! - -DISPLAY_COMMENTS ----------------- - - | Default: ``True`` - | Env: ``DISPLAY_COMMENTS`` - - If set to False comments are hidden. - - -DISPLAY_RATINGS ---------------- - - | Default: ``True`` - | Env: ``DISPLAY_RATINGS`` - - If set to False ratings are hidden. - -DISPLAY_SOCIAL --------------- - - | Default: ``True`` - | Env: ``DISPLAY_SOCIAL`` - - If set to False social sharing is hidden. - -DISPLAY_WMS_LINKS ------------------ - - | Default: ``True`` - | Env: ``DISPLAY_WMS_LINKS`` - - If set to False direct WMS link to GeoServer is hidden. - -DISPLAY_ORIGINAL_DATASET_LINK ------------------------------ - - | Default: ``True`` - | Env: ``DISPLAY_ORIGINAL_DATASET_LINK`` - - If set to False original dataset download is hidden. - -DOWNLOAD_FORMATS_METADATA -------------------------- - - Specifies which metadata formats are available for users to download. - - Default:: - - DOWNLOAD_FORMATS_METADATA = [ - 'Atom', 'DIF', 'Dublin Core', 'ebRIM', 'FGDC', 'ISO', - ] - -DOWNLOAD_FORMATS_VECTOR ------------------------ - - Specifies which formats for vector data are available for users to download. - - Default:: - - DOWNLOAD_FORMATS_VECTOR = [ - 'JPEG', 'PDF', 'PNG', 'Zipped Shapefile', 'GML 2.0', 'GML 3.1.1', 'CSV', - 'Excel', 'GeoJSON', 'KML', 'View in Google Earth', 'Tiles', - ] - -DOWNLOAD_FORMATS_RASTER ------------------------ - - Specifies which formats for raster data are available for users to download. - - Default:: - - DOWNLOAD_FORMATS_RASTER = [ - 'JPEG', 'PDF', 'PNG' 'Tiles', - ] - -E -= - -EMAIL_ENABLE ------------- - - | Default: ``False`` - - Options: - - * EMAIL_BACKEND - - Default: ``django.core.mail.backends.smtp.EmailBackend`` - - Env: ``DJANGO_EMAIL_BACKEND`` - - * EMAIL_HOST - - Default: ``localhost`` - - * EMAIL_PORT - - Default: ``25`` - - * EMAIL_HOST_USER - - Default: ``''`` - - * EMAIL_HOST_PASSWORD - - Default: ``''`` - - * EMAIL_USE_TLS - - Default: ``False`` - - * EMAIL_USE_SSL - - Default: ``False`` - - * DEFAULT_FROM_EMAIL - - Default: ``GeoNode `` - -EPSG_CODE_MATCHES ------------------ - - | Default: - - .. code-block:: python - - { - 'EPSG:4326': '(4326) WGS 84', - 'EPSG:900913': '(900913) Google Maps Global Mercator', - 'EPSG:3857': '(3857) WGS 84 / Pseudo-Mercator', - 'EPSG:3785': '(3785 DEPRECATED) Popular Visualization CRS / Mercator', - 'EPSG:32647': '(32647) WGS 84 / UTM zone 47N', - 'EPSG:32736': '(32736) WGS 84 / UTM zone 36S' - } - - Supported projections human readable descriptions associated to their EPSG Codes. - This list will be presented to the user during the upload process whenever GeoNode won't be able to recognize a suitable projection. - Those codes should be aligned to the `UPLOADER` ones and available in GeoServer also. - -EXTRA_METADATA_SCHEMA ---------------------- - - | Default: - - .. code-block:: python - - EXTRA_METADATA_SCHEMA = {**{ - "map": os.getenv('MAP_EXTRA_METADATA_SCHEMA', DEFAULT_EXTRA_METADATA_SCHEMA), - "layer": os.getenv('DATASET_EXTRA_METADATA_SCHEMA', DEFAULT_EXTRA_METADATA_SCHEMA), - "document": os.getenv('DOCUMENT_EXTRA_METADATA_SCHEMA', DEFAULT_EXTRA_METADATA_SCHEMA), - "geoapp": os.getenv('GEOAPP_EXTRA_METADATA_SCHEMA', DEFAULT_EXTRA_METADATA_SCHEMA) - }, **CUSTOM_METADATA_SCHEMA} - - Variable used to actually get the expected metadata schema for each resource_type. - In this way, each resource type can have a different metadata schema - -F -= - -FREETEXT_KEYWORDS_READONLY --------------------------- - - | Default: ``False`` - | Env: ``FREETEXT_KEYWORDS_READONLY`` - - Make Free-Text Keywords writable from users. Or read-only when set to False. - - -FACET_PROVIDERS ---------------- - - | Default: ``pre filled list of providers`` - | Env: ``FACET_PROVIDERS`` - - Contains the list of the providers available to perform an serve the facets. - In case the user wants remove a facets, is enough to remove the path - of the proider from the list - - -G -= - -GEOFENCE_SECURITY_ENABLED -------------------------- - - | Default: ``True`` (False is Test is true) - | Env: ``GEOFENCE_SECURITY_ENABLED`` - - Whether the geofence security system is used. - -GEOIP_PATH ----------- - - | Default: ``Path to project`` - | Env: ``PROJECT_ROOT`` - - The local path where GeoIPCities.dat is written to. Make sure your user has to have write permissions. - -GEONODE_APPS_ENABLED --------------------- - - | Default: ``True`` - - If enabled contrib apps are used. - If disabled: - - the geoapps URLs are not included in the routing paths - - the geoapps resources are excluded from the search - - the resource detail are forwarded to the homepage - - ``ENABLE -> DISABLE`` transition: - - This should be done if the geoapps were enabled in an environment where they are not needed. - - ``DISABLE -> ENABLE`` transition: - - It should be done only once to enable geoapps in an environment where are needed - -GEONODE_CLIENT_LAYER_PREVIEW_LIBRARY ------------------------------------- - - Default: ``"mapstore"`` - - The library to use for display preview images of layers. The library choices are: - - ``"mapstore"`` - ``"leaflet"`` - ``"react"`` - -GEONODE_EXCHANGE ----------------- - - | Default:: ``Exchange("default", type="direct", durable=True)`` - - The definition of Exchanges published by geonode. Find more about Exchanges at `celery docs `__. - -GEOSERVER_ADMIN_USER ---------------------------- - - | Default: ``admin`` - | Env: ``GEOSERVER_ADMIN_PASSWORD`` - - The geoserver admin username. - -GEOSERVER_ADMIN_PASSWORD ------------------------ - - | Default: ``geoserver`` - | Env: ``GEOSERVER_ADMIN_USER`` - - The GeoServer admin password. - -GEOSERVER_FACTORY_PASSWORD ------------------------ - - | Default: ``geoserver`` - | Env: ``GEOSERVER_FACTORY_PASSWORD`` - - The GeoServer admin factory password, required for the first time configuration fo Geoserver (Docker setup). - -GEOSERVER_EXCHANGE ------------------- - - | Default:: ``Exchange("geonode", type="topic", durable=False)`` - - The definition of Exchanges published by GeoServer. Find more about Exchanges at `celery docs `__. - -GEOSERVER_LOCATION ------------------- - - | Default: ``http://localhost:8080/geoserver/`` - | Env: ``GEOSERVER_LOCATION`` - - Url under which GeoServer is available. - -GEOSERVER_PUBLIC_HOST ---------------------- - - | Default: ``SITE_HOST_NAME`` (Variable) - | Env: ``GEOSERVER_PUBLIC_HOST`` - - Public hostname under which GeoServer is available. - -GEOSERVER_PUBLIC_LOCATION -------------------------- - - | Default: ``SITE_HOST_NAME`` (Variable) - | Env: ``GEOSERVER_PUBLIC_LOCATION`` - - Public location under which GeoServer is available. - -GEOSERVER_PUBLIC_PORT ---------------------- - - | Default: ``8080 (Variable)`` - | Env: ``GEOSERVER_PUBLIC_PORT`` - - - Public Port under which GeoServer is available. - -GEOSERVER_WEB_UI_LOCATION -------------------------- - - | Default: ``GEOSERVER_PUBLIC_LOCATION (Variable)`` - | Env: ``GEOSERVER_WEB_UI_LOCATION`` - - Public location under which GeoServer is available. - -GROUP_PRIVATE_RESOURCES ------------------------ - - | Default: ``False`` - | Env: ``GROUP_PRIVATE_RESOURCES`` - - If this option is enabled, Resources belonging to a Group won't be visible by others - -I -= - -IMPORTER HANDLERS ------------------ - - | Default: ``pre filled list of handlers`` - | Env: ``IMPORTER_HANDLERS`` - - Contains the list of the handlers available to perform an import of a resource. - In case the user wants to drop the support during the import phase, is enough to - remove the path of the Handler from the list - -L -= - -LEAFLET_CONFIG --------------- - - A dictionary used for Leaflet configuration. - -LICENSES --------- - - | Default: - - .. code-block:: python - - { - 'ENABLED': True, - 'DETAIL': 'above', - 'METADATA': 'verbose', - } - - Enable Licenses User Interface - -LOCAL_SIGNALS_BROKER_URL ------------------------- - - | Default: ``memory://`` - -LOCKDOWN_GEONODE ----------------- - - | Default: ``False`` - | Env: ``LOCKDOWN_GEONODE`` - - By default, the GeoNode application allows visitors to view most pages without being authenticated. If this is set to ``True`` - users must be authenticated before accessing URL routes not included in ``AUTH_EXEMPT_URLS``. - -LOGIN_URL ---------- - - | Default: ``{}account/login/'.format(SITEURL)`` - | Env: ``LOGIN_URL`` - - The URL where requests are redirected for login. - - -LOGOUT_URL ----------- - - | Default: ``{}account/login/'.format(SITEURL)`` - | Env: ``LOGOUT_URL`` - - The URL where requests are redirected for logout. - -M -= - -MAP_CLIENT_USE_CROSS_ORIGIN_CREDENTIALS ---------------------------------------- - - | Default: ``False`` - | Env: ``MAP_CLIENT_USE_CROSS_ORIGIN_CREDENTIALS`` - - Enables cross origin requests for geonode-client. - -MAPSTORE_BASELAYERS -------------------- - - | Default: - - .. code-block:: python - - [ - { - "type": "osm", - "title": "Open Street Map", - "name": "mapnik", - "source": "osm", - "group": "background", - "visibility": True - }, { - "type": "tileprovider", - "title": "OpenTopoMap", - "provider": "OpenTopoMap", - "name": "OpenTopoMap", - "source": "OpenTopoMap", - "group": "background", - "visibility": False - }, { - "type": "wms", - "title": "Sentinel-2 cloudless - https://s2maps.eu", - "format": "image/jpeg", - "id": "s2cloudless", - "name": "s2cloudless:s2cloudless", - "url": "https://maps.geo-solutions.it/geoserver/wms", - "group": "background", - "thumbURL": "%sstatic/mapstorestyle/img/s2cloudless-s2cloudless.png" % SITEURL, - "visibility": False - }, { - "source": "ol", - "group": "background", - "id": "none", - "name": "empty", - "title": "Empty Background", - "type": "empty", - "visibility": False, - "args": ["Empty Background", {"visibility": False}] - } - ] - - | Env: ``MAPSTORE_BASELAYERS`` - - Allows to specify which backgrounds MapStore should use. The parameter ``visibility`` for a layer, specifies which one is the default one. - - A sample configuration using the Bing background without OpenStreetMap, could be the following one: - - .. code-block:: python - - [ - { - "type": "bing", - "title": "Bing Aerial", - "name": "AerialWithLabels", - "source": "bing", - "group": "background", - "apiKey": "{{apiKey}}", - "visibility": True - }, { - "type": "tileprovider", - "title": "OpenTopoMap", - "provider": "OpenTopoMap", - "name": "OpenTopoMap", - "source": "OpenTopoMap", - "group": "background", - "visibility": False - }, { - "type": "wms", - "title": "Sentinel-2 cloudless - https://s2maps.eu", - "format": "image/jpeg", - "id": "s2cloudless", - "name": "s2cloudless:s2cloudless", - "url": "https://maps.geo-solutions.it/geoserver/wms", - "group": "background", - "thumbURL": "%sstatic/mapstorestyle/img/s2cloudless-s2cloudless.png" % SITEURL, - "visibility": False - }, { - "source": "ol", - "group": "background", - "id": "none", - "name": "empty", - "title": "Empty Background", - "type": "empty", - "visibility": False, - "args": ["Empty Background", {"visibility": False}] - } - ] - - .. warning:: To use a Bing background, you need to correctly set and provide a valid ``BING_API_KEY`` - -MAX_DOCUMENT_SIZE ------------------ - - | Default:``2`` - | Env: ``MAX_DOCUMENT_SIZE`` - - Allowed size for documents in MB. - -METADATA_PARSERS ----------------- - -Is possible to define multiple XML parsers for ingest XML during the layer upload. - -The variable should be declared in this way in `settings.py`: - -`METADATA_PARSERS = ['list', 'of', 'parsing', 'functions']` - -If you want to always use the default metadata parser and after use your own, the variable must be set with first value as `__DEFAULT__` -Example: - -`METADATA_PARSERS = ['__DEFAULT__', 'custom_parsing_function]` - -If not set, the system will use the `__DEFAULT__` parser. - -The custom parsing function must be accept in input 6 parameter that are: - - | - exml (xmlfile) - | - uuid (str) - | - vals (dict) - | - regions (list) - | - keywords (list) - | - custom (dict) - -If you want to use your parser after the default one, here is how the variable are populated: - - | - exml: the XML file to parse - | - uuid: the UUID of the layer - | - vals: Dictionary of information that belong to ResourceBase - | - regions: List of regions extracted from the XML - | - keywords: List of dict of keywords already divided between free-text and thesarus - | - custom: Custom varible - -NOTE: the keywords must be in a specific format, since later this dict, will be ingested by the `KeywordHandler` which will assign the keywords/thesaurus to the layer. - - .. code:: - { - "keywords": [list_of_keyword_extracted], - "thesaurus": {"date": None, "datetype": None, "title": None}, # thesaurus informations - "type": theme, #extracted theme if present - } - -Here is an example of expected parser function - - .. code:: - def custom_parsing_function(exml, uuid, vals, regions, keywords, custom): - # Place here your code - return uuid, vals, regions, keywords, custom - -For more information, please rely to `TestCustomMetadataParser` which contain a smoke test to explain the functionality - - -METADATA_STORERS ----------------- - -Is possible to define multiple Layer storer during the layer upload. - -The variable should be declared in this way: - -`METADATA_STORERS = ['custom_storer_function']` - -NOTE: By default the Layer is always saved with the default behaviour. - -The custom storer function must be accept in input 2 parameter that are: - - | - Layer (layer model instance) - | - custom (dict) - -Here is how the variable are populated by default: - - | - layer (layer model instance) that we wanto to change - | - custom: custom dict populated by the parser - -Here is an example of expected storer function - - .. code:: - def custom_storer_function(layer, custom): - # do something here - pass - -For more information, please rely to `TestMetadataStorers` which contain a smoke test to explain the functionality - - -MISSING_THUMBNAIL ------------------ - - Default: ``geonode/img/missing_thumb.png`` - - The path to an image used as thumbnail placeholder. - - -MEMCACHED_BACKEND ------------------ - Default: ``django.core.cache.backends.memcached.PyMemcacheCache`` - - Define which backend of memcached will be used - - -MEMCACHED_ENABLED ------------------ - Default: ``False`` - - If True, will use MEMCACHED_BACKEND as default backend in CACHES - - -MODIFY_TOPICCATEGORY --------------------- - - Default: ``False`` - - Metadata Topic Categories list should not be modified, as it is strictly defined - by ISO (See: http://www.isotc211.org/2005/resources/Codelist/gmxCodelists.xml - and check the element). - - Some customization is still possible changing the is_choice and the GeoNode - description fields. - - In case it is necessary to add/delete/update categories, it is - possible to set the MODIFY_TOPICCATEGORY setting to True. - -MONITORING_ENABLED ------------------- - - Default: ``False`` - - Enable internal monitoring application (`geonode.monitoring`). If set to `True`, add following code to your local settings: - - .. code:: - - MONITORING_ENABLED = True - # add following lines to your local settings to enable monitoring - if MONITORING_ENABLED: - INSTALLED_APPS + ('geonode.monitoring',) - MIDDLEWARE_CLASSES + ('geonode.monitoring.middleware.MonitoringMiddleware',) - - See :ref:`geonode_monitoring` for details. - -.. _monitoring-data-aggregation: - -MONITORING_DATA_AGGREGATION ---------------------------- - - Default: - - .. code:: - - ( - (timedelta(seconds=0), timedelta(minutes=1),), - (timedelta(days=1), timedelta(minutes=60),), - (timedelta(days=14), timedelta(days=1),), - ) - - Configure aggregation of past data to control data resolution. It lists data age and aggregation in reverse order, by default: - - | - for current data, 1 minute resolution - | - for data older than 1 day, 1-hour resolution - | - for data older than 2 weeks, 1 day resolution - - See :ref:`geonode_monitoring` for further details. - - This setting takes effects only if :ref:`user-analytics` is true. - -MONITORING_DATA_TTL -------------------- - - | Default: ``365`` - | Env: ``MONITORING_DATA_TTL`` - - How long monitoring data should be stored in days. - -MONITORING_DISABLE_CSRF ------------------------ - - | Default: ``False`` - | Env: ``MONITORING_DISABLE_CSRF`` - - Set this to true to disable csrf check for notification config views, use with caution - for dev purpose only. - -.. _monitoring-skip-paths: - -MONITORING_SKIP_PATHS ------------------------ - - Default: - - .. code:: - - ( - '/api/o/', - '/monitoring/', - '/admin', - '/jsi18n', - STATIC_URL, - MEDIA_URL, - re.compile('^/[a-z]{2}/admin/'), - ) - - Skip certain useless paths to not to mud analytics stats too much. - See :ref:`geonode_monitoring` to learn more about it. - - This setting takes effects only if :ref:`user-analytics` is true. - -N -= - -NOTIFICATIONS_MODULE --------------------- - - Default: ``pinax.notifications`` - - App used for notifications. (pinax.notifications or notification) - -NOTIFICATION_ENABLED --------------------- - - | Default: ``True`` - | Env: ``NOTIFICATION_ENABLED`` - - Enable or disable the notification system. - -O -= - -OAUTH2_API_KEY --------------- - - | Default: ``None`` - | Env: ``OAUTH2_API_KEY`` - - In order to protect oauth2 REST endpoints, used by GeoServer to fetch user roles and infos, you should set this key and configure the ``geonode REST role service`` accordingly. Keep it secret! - - .. warning:: If not set, the endpoint can be accessed by users without authorization. - -OAUTH2_PROVIDER ---------------- - - Ref.: `OAuth Toolkit settings `__ - -OAUTH2_PROVIDER_APPLICATION_MODEL ---------------------------------- - - | Default: ``oauth2_provider.Application`` - - Ref.: `OAuth Toolkit settings `__ - -OAUTH2_PROVIDER_ACCESS_TOKEN_MODEL ----------------------------------- - - | Default: ``oauth2_provider.AccessToken`` - - Ref.: `OAuth Toolkit settings `__ - -OAUTH2_PROVIDER_ID_TOKEN_MODEL ------------------------------- - - | Default: ``oauth2_provider.IDToken`` - - Ref.: `OAuth Toolkit settings `__ - -OAUTH2_PROVIDER_GRANT_MODEL ---------------------------- - - | Default: ``oauth2_provider.Grant`` - - Ref.: `OAuth Toolkit settings `__ - -OAUTH2_PROVIDER_REFRESH_TOKEN_MODEL ------------------------------------ - - | Default: ``oauth2_provider.RefreshToken`` - - Ref.: `OAuth Toolkit settings `__ - -OGC_SERVER ----------- - - Default: ``{}`` (Empty dictionary) - - A dictionary of OGC servers and their options. The main - server should be listed in the 'default' key. If there is no 'default' - key or if the ``OGC_SERVER`` setting does not exist, Geonode will raise - an Improperly Configured exception. Below is an example of the ``OGC_SERVER`` - setting:: - - OGC_SERVER = { - 'default' : { - 'LOCATION' : 'http://localhost:8080/geoserver/', - 'USER' : 'admin', - 'PASSWORD' : 'geoserver', - } - } - - * BACKEND - - Default: ``"geonode.geoserver"`` - - The OGC server backend to use. The backend choices are: - - ``'geonode.geoserver'`` - - * BACKEND_WRITE_ENABLED - - Default: ``True`` - - Specifies whether the OGC server can be written to. If False, actions that modify - data on the OGC server will not execute. - - * DATASTORE - - Default: ``''`` (Empty string) - - An optional string that represents the name of a vector datastore, where Geonode uploads are imported into. To support vector datastore imports there also needs to be an - entry for the datastore in the ``DATABASES`` dictionary with the same name. Example:: - - OGC_SERVER = { - 'default' : { - 'LOCATION' : 'http://localhost:8080/geoserver/', - 'USER' : 'admin', - 'PASSWORD' : 'geoserver', - 'DATASTORE': 'geonode_imports' - } - } - - DATABASES = { - 'default': { - 'ENGINE': 'django.db.backends.sqlite3', - 'NAME': 'development.db', - }, - 'geonode_imports' : { - 'ENGINE': 'django.contrib.gis.db.backends.postgis', - 'NAME': 'geonode_imports', - 'USER' : 'geonode_user', - 'PASSWORD' : 'a_password', - 'HOST' : 'localhost', - 'PORT' : '5432', - } - } - - * GEONODE_SECURITY_ENABLED - - Default: ``True`` - - A boolean that represents whether GeoNode's security application is enabled. - - * LOCATION - - Default: ``"http://localhost:8080/geoserver/"`` - - A base URL from which GeoNode can construct OGC service URLs. - If using GeoServer you can determine this by - visiting the GeoServer administration home page without the - /web/ at the end. For example, if your GeoServer administration app is at - http://example.com/geoserver/web/, your server's location is http://example.com/geoserver. - - * MAPFISH_PRINT_ENABLED - - Default: ``True`` - - A boolean that represents whether the MapFish printing extension is enabled on the server. - - * PASSWORD - - Default: ``'geoserver'`` - - The administrative password for the OGC server as a string. - - * PRINT_NG_ENABLED - - Default: ``True`` - - A boolean that represents whether printing of maps and layers is enabled. - - * PUBLIC_LOCATION - - Default: ``"http://localhost:8080/geoserver/"`` - - The URL used to in most public requests from Geonode. This setting allows a user to write to one OGC server (the LOCATION setting) - and read from a separate server or the PUBLIC_LOCATION. - - * USER - - Default: ``'admin'`` - - The administrative username for the OGC server as a string. - - * WMST_ENABLED - - Default: ``False`` - - Not implemented. - - * WPS_ENABLED - - Default: ``False`` - - Not implemented. - - * TIMEOUT - - Default: ``10`` - - The maximum time, in seconds, to wait for the server to respond. - -OGP_URL -------- - - | Default: ``http://geodata.tufts.edu/solr/select`` - | Env: ``OGP_URL`` - - Endpoint of geodata.tufts.edu getCapabilities. - -OPENGRAPH_ENABLED ------------------ - - Default:: ``True`` - - A boolean that specifies whether Open Graph is enabled. Open Graph is used by Facebook and Slack. - -P -= - -PINAX_NOTIFICATIONS_BACKENDS ----------------------------- - - Default: ``("email", _EMAIL_BACKEND, 0),`` - - Used notification backend. This is a `pinax notification setting: `__ - -PINAX_NOTIFICATIONS_LOCK_WAIT_TIMEOUT -------------------------------------- - - | Default: ``-1`` - | Env: ``NOTIFICATIONS_LOCK_WAIT_TIMEOUT`` - - It defines how long to wait for the lock to become available. Default of -1 means to never wait for the lock to become available. - This is a `pinax notification setting: `__ - -PINAX_NOTIFICATIONS_QUEUE_ALL ------------------------------ - - | Default: ``-1`` - | Env: ``NOTIFICATIONS_LOCK_WAIT_TIMEOUT`` - - By default, calling notification.send will send the notification immediately, however, if you set this setting to True, then the default behavior of the send method will be to queue messages in the database for sending via the emit_notices command. - This is a `pinax notification setting: `__ - -PINAX_RATINGS_CATEGORY_CHOICES ------------------------------- - - Default:: - - { - "maps.Map": { - "map": "How good is this map?" - }, - "layers.Layer": { - "layer": "How good is this layer?" - }, - "documents.Document": { - "document": "How good is this document?" - } - } - - -PROFILE_EDIT_EXCLUDE_FIELD ---------------------------- - | Default: ``[]`` - - A list of element (item name) to exclude from the Profile Edit page. - - Example: - - ``PROFILE_EDIT_EXCLUDE_FIELD=['organization', 'language']`` - - -PROXY_ALLOWED_HOSTS -------------------- - - Default: ``()`` (Empty tuple) - - A tuple of strings representing the host/domain names that GeoNode can proxy requests to. This is a security measure - to prevent an attacker from using the GeoNode proxy to render malicious code or access internal sites. - - Values in this tuple can be fully qualified names (e.g. 'www.geonode.org'), in which case they will be matched against - the request’s Host header exactly (case-insensitive, not including port). A value beginning with a period can be used - as a subdomain wildcard: ``.geonode.org`` will match geonode.org, www.geonode.org, and any other subdomain of - geonode.org. A value of '*' will match anything and is not recommended for production deployments. - - -PROXY_URL ---------- - - Default ``/proxy/?url=`` - - The URL to a proxy that will be used when making client-side requests in GeoNode. By default, the - internal GeoNode proxy is used but administrators may favor using their own, less restrictive proxies. - - -PYCSW ------ - - A dict with pycsw's configuration with two possible keys CONFIGURATION and FILTER. - - CONFIGURATION - Of note are the sections ``metadata:main`` to set CSW server metadata and ``metadata:inspire`` - to set INSPIRE options. Setting ``metadata:inspire['enabled']`` to ``true`` - will enable INSPIRE support. Server level configurations can be overridden - in the ``server`` section. See http://docs.pycsw.org/en/latest/configuration.html - for full pycsw configuration details. - - FILTER - Optional settings in order to add a filter to the CSW filtering. - The filter follow the django orm structure and must be a `ResourceBase` field/related field. - By default CSW will filter only for `layer` resource_type - - Example of PYCSW configuration. - PYCSW: { - 'CONFIGURATION': {...}, - 'FILTER': {'resource_type__in':['layer'] } - } - -R -= - -RABBITMQ_SIGNALS_BROKER_URL ---------------------------- - - Default: ``amqp://localhost:5672`` - - The Rabbitmq endpoint - -.. _recaptcha_enabled: - -RECAPTCHA_ENABLED ------------------ - - | Default: ``False`` - | Env: ``RECAPTCHA_ENABLED`` - - Allows enabling reCaptcha field on signup form. - Valid Captcha Public and Private keys will be needed as specified here https://pypi.org/project/django-recaptcha/#installation - - You will need to generate a keys pair for ``reCaptcha v2`` for your domain from https://www.google.com/recaptcha/admin/create - - More options will be available by enabling this setting: - - * **ACCOUNT_SIGNUP_FORM_CLASS** - - | Default: ``geonode.people.forms.AllauthReCaptchaSignupForm`` - | Env: ``ACCOUNT_SIGNUP_FORM_CLASS`` - - Enabled only when the :ref:`recaptcha_enabled` option is ``True``. - - * **INSTALLED_APPS** - - The ``captcha`` must be present on ``INSTALLED_APPS``, otherwise you'll get an error. - - When enabling the :ref:`recaptcha_enabled` option through the ``environment``, this setting will be automatically added by GeoNode as follows: - - .. code:: python - - if 'captcha' not in INSTALLED_APPS: - INSTALLED_APPS += ('captcha',) - - * **RECAPTCHA_PUBLIC_KEY** - - | Default: ``geonode_RECAPTCHA_PUBLIC_KEY`` - | Env: ``RECAPTCHA_PUBLIC_KEY`` - - You will need to generate a keys pair for ``reCaptcha v2`` for your domain from https://www.google.com/recaptcha/admin/create - - For mode details on the reCaptcha package, please see: - - #. https://pypi.org/project/django-recaptcha/#installation - #. https://pypi.org/project/django-recaptcha/#local-development-and-functional-testing - - * **RECAPTCHA_PRIVATE_KEY** - - | Default: ``geonode_RECAPTCHA_PRIVATE_KEY`` - | Env: ``RECAPTCHA_PRIVATE_KEY`` - - You will need to generate a keys pair for ``reCaptcha v2`` for your domain from https://www.google.com/recaptcha/admin/create - - For mode details on the reCaptcha package, please see: - - #. https://pypi.org/project/django-recaptcha/#installation - #. https://pypi.org/project/django-recaptcha/#local-development-and-functional-testing - -RECAPTCHA_PUBLIC_KEY --------------------- - - | Default: ``geonode_RECAPTCHA_PUBLIC_KEY`` - | Env: ``RECAPTCHA_PUBLIC_KEY`` - - You will need to generate a keys pair for ``reCaptcha v2`` for your domain from https://www.google.com/recaptcha/admin/create - - Ref. to :ref:`recaptcha_enabled` - -RECAPTCHA_PRIVATE_KEY ---------------------- - - | Default: ``geonode_RECAPTCHA_PRIVATE_KEY`` - | Env: ``RECAPTCHA_PRIVATE_KEY`` - - You will need to generate a keys pair for ``reCaptcha v2`` for your domain from https://www.google.com/recaptcha/admin/create - - Ref. to :ref:`recaptcha_enabled` - -REDIS_SIGNALS_BROKER_URL ------------------------- - - Default: ``redis://localhost:6379/0`` - - The Redis endpoint. - -REGISTERED_MEMBERS_GROUP_NAME ------------------------------ - - | Default: ``registered-members`` - | Env: ``REGISTERED_MEMBERS_GROUP_NAME`` - - Used by ``AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_NAME`` settings. - -REGISTERED_MEMBERS_GROUP_TITLE ------------------------------- - - | Default: ``Registered Members`` - | Env: ``REGISTERED_MEMBERS_GROUP_TITLE`` - - Used by ``AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_NAME`` settings. - -REGISTRATION_OPEN ------------------ - - Default: ``False`` - - A boolean that specifies whether users can self-register for an account on your site. - -RESOURCE_PUBLISHING -------------------- - - Default: ``False`` - - By default, the GeoNode application allows GeoNode staff members to - publish/unpublish resources. - By default, resources are published when created. When this setting is set to - True the staff members will be able to unpublish a resource (and eventually - publish it back). - -S -= - -SEARCH_FILTERS --------------- - - Default:: - - 'TEXT_ENABLED': True, - 'TYPE_ENABLED': True, - 'CATEGORIES_ENABLED': True, - 'OWNERS_ENABLED': True, - 'KEYWORDS_ENABLED': True, - 'H_KEYWORDS_ENABLED': True, - 'T_KEYWORDS_ENABLED': True, - 'DATE_ENABLED': True, - 'REGION_ENABLED': True, - 'EXTENT_ENABLED': True, - - Enabled Search Filters for filtering resources. - -SECURE_BROWSER_XSS_FILTER -------------------------- - - | Default: ``True`` - | Env: ``SECURE_BROWSER_XSS_FILTER`` - - If True, the SecurityMiddleware sets the X-XSS-Protection: 1; mode=block header on all responses that do not already have it. - This is ``__ - -SECURE_CONTENT_TYPE_NOSNIFF ---------------------------- - - | Default: ``True`` - | Env: ``SECURE_CONTENT_TYPE_NOSNIFF`` - - If True, the SecurityMiddleware sets the X-Content-Type-Options: nosniff header on all responses that do not already have it. - This is `Django settings: `__ - - -SECURE_HSTS_INCLUDE_SUBDOMAINS ------------------------------- - - | Default: ``True`` - | Env: ``SECURE_HSTS_INCLUDE_SUBDOMAINS`` - - This is Django settings: https://docs.djangoproject.com/en/3.2/ref/settings/#secure-hsts-include-subdomains - -SECURE_HSTS_SECONDS -------------------- - - | Default: ``3600`` - | Env: ``SECURE_HSTS_SECONDS`` - - This is `Django settings: `__ - If set to a non-zero integer value, the SecurityMiddleware sets the HTTP Strict Transport Security header on all responses that do not already have it. - -SECURE_SSL_REDIRECT -------------------- - - If True, the SecurityMiddleware redirects all non-HTTPS requests to HTTPS (except for those URLs matching a regular expression listed in SECURE_REDIRECT_EXEMPT). - This is `Django settings: `__ - -SERVICES_TYPE_MODULES ---------------------- - -It's possible to define multiple Service Types Modules for custom service type with it's own Handler. - -The variable should be declared in this way in `settings.py`: - -`SERVICES_TYPE_MODULES = [ 'path.to.module1','path.to.module2', ... ]` - -Default service types are already included - -Inside each module in the list we need to define a variable: - -`services_type = { - "": { - "OWS": True/False, - "handler": "", - "label": "
' - }, - { - "title": 'Starting my story', - "description": 'A cure for writers block', - "content": 'Once upon a time...' - }, - { - "title": 'New list with dates', - "description": 'New List with dates', - "content": '
cdate
mdate

My List

' - } - ], - "template_cdate_format": '[Date Created (CDATE): %m/%d/%Y : %H:%M:%S]', - "template_mdate_format": '[Date Modified (MDATE): %m/%d/%Y : %H:%M:%S]', - } - - HTML WYSIWYG Editor (TINYMCE) Menu Bar Settings. For more info see: - - - https://django-tinymce.readthedocs.io/en/latest/installation.html#configuration - - :ref:`getfetureinfo-templates` -U -= - -UI_REQUIRED_FIELDS ------------------- -If this option is enabled, the input selected (we are referring to the one present in the optional Metadata-Tab on the Metadata-Wizard) will become mandatory. - -The fields that can be mandatory are: - - | id_resource-edition => Label: Edition - | id_resource-purpose => Label: Purpose - | id_resource-supplemental_information => Label: Supplemental information - | id_resource-temporal_extent_start_pickers => Label: temporal extent start - | id_resource-temporal_extent_end => Label: temporal extent end - | id_resource-maintenance_frequency => Label: Maintenance frequency - | id_resource-spatial_representation_type => Label: Spatial representation type - -If at least one on the above ids is set in this configuration, the panel header will change from `Optional` to `Mandatory` - - | Confiugration Example: - | UI_REQUIRED_FIELDS = ['id_resource-edition'] - - -UNOCONV_ENABLE --------------- - - | Default: ``False`` - | Env: ``UNOCONV_ENABLE`` - -UPLOADER --------- - - Default:: - - { - 'BACKEND' : 'geonode.importer', - 'OPTIONS' : { - 'TIME_ENABLED': False, - } - } - - A dictionary of Uploader settings and their values. - - * BACKEND - - Default: ``'geonode.importer'`` - - The importer backend requires the GeoServer importer extension to be enabled. - - * OPTIONS - - Default:: - - 'OPTIONS' : { - 'TIME_ENABLED': False, - } - - * TIME_ENABLED - - Default: ``False`` - - A boolean that specifies whether the upload should allow the user to enable time support when uploading data. - -USER_MESSAGES_ALLOW_MULTIPLE_RECIPIENTS ---------------------------------------- - - | Default: ``True`` - | Env: ``USER_MESSAGES_ALLOW_MULTIPLE_RECIPIENTS`` - - Set to true to have multiple recipients in /message/create/ - -.. _user-deletion-rules: - -USER_DELETION_RULES ----------------------- - | Default: ``["geonode.people.utils.user_has_resources"]`` - | Env: ``USER_DELETION_RULES`` - - List of callables that will be called the deletion of a user account is requested. - The deletion will fail if any of the callables return ``False``. - New rules can be added, as a string path to the callable, as long as they take as parameter - the user object and return a boolean. - -.. _user-analytics: - - -UUID HANDLER ------------- - -Is possible to define an own uuidhandler for the Layer. - -To start using your own handler, is needed to add the following configuration: - -`LAYER_UUID_HANDLER = "mymodule.myfile.MyObject"` - -The Object must accept as `init` the `instance` of the layer and have a method named `create_uuid()` - -here is an example: - - | class MyObject(): - | def __init__(self, instance): - | self.instance = instance - | - | def create_uuid(self): - | # here your code - | pass - - -X -= - -X_FRAME_OPTIONS ---------------- - -Default: ``'ALLOW-FROM %s' % SITEURL`` - -This is a `Django setting `__ diff --git a/basic/theme/index.rst b/basic/theme/index.rst deleted file mode 100644 index 16002433..00000000 --- a/basic/theme/index.rst +++ /dev/null @@ -1,278 +0,0 @@ -GeoNode Themes -============== - -| We have already explained in :ref:`simple-theming` how to change the GeoNode theme directly from the *Admin Interface*. - This is an easy way for customizing GeoNode appearance but, in some cases, you might want to have more control on it. -| In those cases, you have to venture into the code and it is highly recommended to use a GeoNode Project and customize it instead of the GeoNode default HTML/CSS code. - See the following sections to learn more about that. - -Theming your GeoNode Project -============================ - -There are a range of options available to you if you want to change the default look and feel of your :ref:`geonode-project`. - -Logos and graphics -^^^^^^^^^^^^^^^^^^ - -GeoNode intentionally does not include a large number of graphics files in its interface. -This keeps page loading time to a minimum and makes for a more responsive interface. -That said, you are free to customize your GeoNode’s interface by simply changing the default logo, or by adding your own images and graphics to deliver a GeoNode experience the way you envision it. - -Your GeoNode project has a directory already set up for storing your own images at ``/static/img``. -You should place any image files that you intend to use for your project in this directory. - -Let’s walk through an example of the steps necessary to change the default logo. - -#. Change to the :file:`img` directory: - - .. code-block:: console - - $ cd /static/img - -#. If you haven't already, obtain your logo image. The URL below is just an example, so you will need to change this URL to match the location of your file or copy it to this location: - - .. code-block:: console - - $ wget https://upload.wikimedia.org/wikipedia/commons/thumb/a/ac/Service_mark.svg/500px-Service_mark.svg.png - $ wget https://upload.wikimedia.org/wikipedia/commons/thumb/c/c8/Wikimapia_logo_without_label.svg/426px-Wikimapia_logo_without_label.svg.png -O logo.png - -#. Create snippets directory : - - .. code-block:: console - - $ cd ../../.. - $ mkdir /templates/geonode-mapstore-client/snippets - $ cd /templates/geonode-mapstore-client/snippets - -#. Create a new HTML file named :file:`brand_navbar.html` - - .. code-block:: console - - $ sudo vi brand_navbar.html - - .. code-block:: css - - {% extends "geonode-mapstore-client/snippets/brand_navbar.html" %} - {% load static %} - {% block extra_style %} - - {% endblock %} - {% block logo_src %} - {% static 'img/logo.png' %} - {% endblock %} - -#. Restart your GeoNode project and look at the page in your browser: - - .. code-block:: console - - $ cd /home/geonode - $ sudo rm -Rf geonode/geonode/static_root/* - $ cd my_geonode - $ python manage.py collectstatic - $ sudo service apache2 restart - - .. note:: It is a good practice to cleanup the **static_folder** and the Browser Cache before reloading in order to be sure that the changes have been correctly taken and displayed on the screen. - -Visit your site at http://localhost/ or the remote URL for your site. - - .. figure:: img/logo_override.png - :align: center - - *Custom logo* - -In the following sections you will learn how to customize this header to make it as you want. - -.. note:: You should commit these changes to your repository as you progress through this section, and get in the habit of committing early and often so that you and others can track your project on GitHub. - Making many atomic commits and staying in sync with a remote repository makes it easier to collaborate with others on your project. - -Cascading Style Sheets -^^^^^^^^^^^^^^^^^^^^^^ - -In the last section you already learned how to override GeoNode’s default CSS rules to include your own logo. -You are able to customize any aspect of GeoNode’s appearance this way. -In the last screenshot, you saw that the main area in the homepage is covered up by the expanded header. - -First, we’ll walk through the steps necessary to displace it downward so it is no longer hidden, then change the background color of the header to match the color in our logo graphic. - -#. Reopen :file:`/static/css/brand_navbar.html` in your editor: - - .. code-block:: console - - $ cd /templates/geonode-mapstore-client/snippets - $ sudo vi brand_navbar.html - - -#. Append a rule to change the background color of the header to match the logo graphic: - - .. code-block:: css - - #gn-brand-navbar { - .... - background-color: #ff0000 !important; - } - -#. Create new file to manipulate *hero* section: - - .. code-block:: console - - $ cd /templates/geonode-mapstore-client/snippets - $ sudo vi hero.html - -#. Add the following code to change the background image and font for the *hero* section: - - .. code-block:: css - - {% extends "geonode-mapstore-client/snippets/hero.html" %} - {% block extra_style %} - - {% endblock %} - -#. Collect the static files into STATIC_ROOT, restart the development server and reload the page: - - .. code-block:: console - - $ python manage.py collectstatic - $ sudo service apache2 restart - - .. figure:: img/css_override.png - :align: center - - *CSS override* - -You can continue adding rules to this file to override the styles that are in the GeoNode base CSS file which is built from `base.less `_. - -.. note:: You may find it helpful to use your browser's development tools to inspect elements of your site that you want to override to determine which rules are already applied. See the screenshot below. - - .. figure:: img/inspect_element.png - :align: center - - *Screenshot of using browser debugger to inspect the CSS overrides* - -Modify GeoNode Homepage -^^^^^^^^^^^^^^^^^^^^^^^ - -So far we learned how to modify some template sections of your GeoNode main page. -You can do it individually per section template, adding a new page under -:file:`/templates/geonode-mapstore-client/snippets` folder with the -section name (ex: :file:`brand_navbar.html`) or by extending the base template file -:file:`custom_theme.html` where you can add different theme settings in one place. - -#. Remove the previous `hero` section :file:`hero.html` file: - - .. code-block:: console - - $ rm /templates/geonode-mapstore-client/snippets/hero.html - -#. Create a new :file:`custom_theme.html` file: - - .. code-block:: console - - $ cd /templates/geonode-mapstore-client/snippets - $ sudo vi custom_theme.html - - -#. Add the following content to this page: - - .. code-block:: css - - {% load static %} - {% block content %} - - {% endblock %} - -#. Restart httpd server - - .. code-block:: console - - $ python manage.py collectstatic - $ sudo service apache2 restart - -#. Your customized layout should be similar to the next picture: - - .. figure:: img/customized_geonode_project_home.png - :align: center - -#. Edit title and intro message - -Login as administrator on GeoNode and go to ``Admin`` page: - - .. figure:: img/admin_menu.png - :align: center - -Create a new theme under `GeoNode Themes Library` and ``Themes``: - - .. figure:: img/themes_admin_section.png - :align: center - -Add a ``Name``, ``Description`` and turn on ``Is enabled`` option. -At the bottom, add a ``Jumbotron title`` and ``Jumbotron content``. -This will override the default GeoNode welcome title and message. -Click ``Save`` at the bottom in the end. - - .. figure:: img/theme_admin_1.png - :align: center - - .. figure:: img/theme_admin_2.png - :align: center - -After this, reload your GeoNode homepage. The output should be similar to this: - - .. figure:: img/custom_home.png - :align: center diff --git a/contribute/index.rst b/contribute/index.rst deleted file mode 100644 index 0ffc4037..00000000 --- a/contribute/index.rst +++ /dev/null @@ -1,27 +0,0 @@ -Participate in the Discussion -============================= - -.. toctree:: - :maxdepth: 3 - - participate_discussion/index - -Write Documentation -=================== - -.. toctree:: - :maxdepth: 3 - - documentation/index - -Provide Translations -==================== - -.. toctree:: - :maxdepth: 3 - - translation/index - -Write Code -========== - diff --git a/contribute/participate_discussion/index.rst b/contribute/participate_discussion/index.rst deleted file mode 100644 index 4d1e9612..00000000 --- a/contribute/participate_discussion/index.rst +++ /dev/null @@ -1,15 +0,0 @@ -.. _contrib_docu: - -=============================================== -Join the community, ask for help or report bugs -=============================================== - -In case of general questions the GeoNode Community is present at following *channels* - -- User Mailing List: https://lists.osgeo.org/cgi-bin/mailman/listinfo/geonode-users -- Developer Mailing List: https://lists.osgeo.org/cgi-bin/mailman/listinfo/geonode-devel -- Gitter Chat: https://gitter.im/GeoNode/general - -For reporting bugs please open a ticket at Github issues: - -- https://github.com/GeoNode/geonode/issues diff --git a/contribute/translation/index.rst b/contribute/translation/index.rst deleted file mode 100644 index 8ef7884d..00000000 --- a/contribute/translation/index.rst +++ /dev/null @@ -1,134 +0,0 @@ -Contribute to Translations -========================== - -Behind the scenes, GeoNode is using a software called GNU gettext further text-based translation files (django.po and djangojs.po) for translating content. If you'd like to know more about how all of this works you'll find a full description at the `Django Docs -`_. -Following will concentrate on what is needed for edit existing or contribute a new translation. - -Download the translation File -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -All language files live in a specific subfolder called after their `iso code `_ within the `locale folder `_. -For example, for French, the main translation file called django.po can be downloaded from `here `_. - -Next, to download the language file, we need to install an OpenSource Editor called "poedit" for editing from: https://poedit.net/download - -Translation process -^^^^^^^^^^^^^^^^^^^ -Make a copy of the file before starting the translation so that you can revert in case of errors. - -After installing 'poedit', you should be able to double click on the '.po' file to open it. Poedit’s interface should look similar to the one shown in the picture below: - -.. figure:: img/poedit_interface.png - :align: center - :width: 680px - :alt: Poedit interface - -Identifying translation issues -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -From the 'poedit' menu 'View', make sure that 'Entries with Errors first' is checked: - - -.. figure:: img/poedit_sort.png - :align: center - :width: 280px - :alt: Validate translation - -Next click on 'Validate Translations' from the 'Catalogue' menu: - -.. figure:: img/poedit_validate.png - :align: center - :width: 280px - :alt: Validate translation - -'Poedit' will place translations which may require additional consideration on top of the list. -A warning mark means that the interpretation might be not entirely consistent with the original phrase. This is not necessarily an error, just a warning asking the user to double check. - -.. figure:: img/poedit_warning.png - :align: center - :width: 680px - :alt: Poedit warning - -Following to marked phrases, 'Poedit' will show untranslated sentences. When clicking on one, it can be translated through the bottom panel. - -During translation pay special attention to the button saying 'needs work'. In case this button is checked, the phrase will be marked as 'fuzzy' and ignored in GeoNode. - -.. figure:: img/poedit_needs_work.png - :align: center - :alt: phrase needs work - -Saving translations -^^^^^^^^^^^^^^^^^^^ -As soon as the translation is complete, it must be saved and compiled. -Saving is straightforward. All you have to do is clicking the 'Save' button from the top menu. - -As a last step we compile the file. Compiling the translation means to create a binary ".mo" file out of the edited ".po" file. To do so, click on "Compile to MO" - -.. figure:: img/poedit_compile.png - :align: center - :width: 280px - :alt: save and compile - -Poedit will ask where to write the ".mo" file to, by default, this is the same folder as the edited '.po' resides in. The '.mo' file can be overwritten if necessary. - -Push translations to the repository -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -For sharing our updates, we must upload the files to GeoNode’s GitHub repository. -Go to the correct file position which, in case for French is: -https://github.com/GeoNode/geonode/tree/master/geonode/locale/fr/LC_MESSAGES - -Click on "Upload Files" - -.. figure:: img/github_upload.png - :align: center - :width: 680px - :alt: upload files - -Drag the updated files into the Upload form, and write a title/description of the changes - -.. figure:: img/github_drag_files.png - :align: center - :width: 680px - :alt: drag files - -Click on "Create a new branch for this commit…" and then click on the green button. - -.. figure:: img/github_commit.png - :align: center - :width: 680px - :alt: drag files - -The last step will create a `PULL REQUEST` which can be reviewed and then approved by a developer. - - -Activate updated translation at your server -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Once the files have been pushed to GitHub, it will be necessary to update your server to respect changed files. - -At this time, this can be done only by an administrator. From the server 'shell' following commands need to be executed: - -.. code-block:: shell - - workon geonode - cd /opt/geonode - DJANGO_SETTINGS_MODULE=geonode.settings python -W ignore manage.py collectstatic --noinput - sudo service uwsgi restart - - - -Texts not listed in .po files -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -In case you find a template output without corresponding translation you can add it as follows: - -Identify the corresponding template file which is responsible for outputting the text. Add a ``{% trans "TEXT" %}`` tag. Save the template file and run the following: - -.. code-block:: shell - - django-admin makemessages --no-location -l en -d django -e "html,txt,py" -i docs - django-admin makemessages --no-location -l en -d djangojs -e "js" -i docs -i node_modules -i lib - - -This will update the english .po file. also to update the language which should be edited by settings the ``-l fr`` parameter. Continue with updating the .po file as described above. diff --git a/convert_rst_to_md.py b/convert_rst_to_md.py new file mode 100644 index 00000000..7c2d9183 --- /dev/null +++ b/convert_rst_to_md.py @@ -0,0 +1,81 @@ +import os, re, subprocess, sys + +labels = {} +rst_files = [] +for root, dirs, files in os.walk('.'): + if root.startswith('./locale') or root.startswith('./i18n'): + continue + for f in files: + if f.endswith('.rst'): + path = os.path.join(root, f) + rst_files.append(path) + with open(path, encoding='utf-8') as fr: + for line in fr: + m = re.match(r'\s*\.\.\s*_([A-Za-z0-9_.-]+):', line) + if m: + labels[m.group(1)] = path + +for rst in rst_files: + md = rst[:-4] + '.md' + subprocess.run(['pandoc','-f','rst','-t','markdown','--markdown-headings=atx','--wrap','preserve',rst,'-o',md], check=True) + + with open(md, encoding='utf-8') as f: + md_text = f.read() + + def resolve(label): + target = labels.get(label) + if not target: + return None + target_md = target[:-4] + '.md' + return os.path.relpath(target_md, os.path.dirname(md)) + + def repl_ref(match): + text = match.group(1) + label = match.group(2) or text + href = resolve(label) + if href: + if href == os.path.basename(md): + url = f"#{label}" + else: + url = f"{href}#{label}" + return f"[{text}]({url})" + return text + + md_text = re.sub(r':ref:`([^`<]+?)\s*<([^`]+)>`', repl_ref, md_text) + md_text = re.sub(r':ref:`([^`]+?)`', repl_ref, md_text) + md_text = re.sub(r':doc:`([^`<]+?)\s*<([^`]+)>`', lambda m: f"[{m.group(1)}]({m.group(2)}.md)", md_text) + md_text = re.sub(r':doc:`([^`]+?)`', lambda m: f"[{m.group(1)}]({m.group(1)}.md)", md_text) + + # append toctree links from rst + sections = [] + lines = open(rst, encoding='utf-8').read().splitlines() + i = 0 + while i < len(lines): + if lines[i].startswith('.. toctree::'): + caption = None + i += 1 + while i < len(lines) and re.match(r'\s+:', lines[i]): + if re.match(r'\s+:caption:', lines[i]): + caption = lines[i].split(':caption:')[1].strip() + i += 1 + while i < len(lines) and not lines[i].strip(): + i += 1 + entries = [] + while i < len(lines) and lines[i].strip(): + entries.append(lines[i].strip()) + i += 1 + sections.append((caption, entries)) + else: + i += 1 + + if sections: + md_text += '\n\n' + for caption, entries in sections: + if caption: + md_text += f"### {caption}\n" + for e in entries: + md_text += f"- [{os.path.basename(e)}]({e}.md)\n" + md_text += '\n' + + with open(md, 'w', encoding='utf-8') as f: + f.write(md_text) diff --git a/devel/api/usage/index.rst b/devel/api/usage/index.rst deleted file mode 100644 index 1428518a..00000000 --- a/devel/api/usage/index.rst +++ /dev/null @@ -1,750 +0,0 @@ -=========== -API usage examples -=========== - -| In this section, we are going to demostrate how GeoNode API can be utilized/integrated with other applications using Python. - -Resource Listing and Details ----------------------------- - -As mentioned in previous chapters, GeoNode resources are categorized in different types e.g. datasets, maps, documents. Etc. -All available resources can be listed with API ``GET /api/v2/resources``. - -To obtain a single resource, a primary key is provided in the url. Eg ``GET /api/v2/resources/{resource.pk}``. - -Example Requests: -^^^^^^^^^^^^^^^^^ - -1. Listing - -.. code-block:: shell - - import requests - - url = "https://master.demo.geonode.org/api/v2/resources" - response = requests.request("GET", url) - -2. Detail - -.. code-block:: shell - - import requests - - url = "https://master.demo.geonode.org/api/v2/resources/1797" - response = requests.request("GET", url) - -.. note:: The above requests work for publicly visible resources. If a resource is private either the Basic Auth or the Bearer token must be included inside the headers. - -3. Listing with basic auth: - -.. code-block:: shell - - import requests - - url = "https://master.demo.geonode.org/api/v2/resources" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("GET", url, headers=headers) - -A token can be used in place of Basic Auth by setting ``'Authorization': 'Bearer '``. - -Resource Download ------------------ - -The download URL for a resource can be obtained from ``resource.download_url``. This URL executes the synchronous download of a resource in its default download format (ESRI Shapefile for vector, Geotiff for rasters and the original format for documents). -Additional export formats for datasets are available through the UI. At the moment the API doesn’t expose them. - -Resource Links ---------------- - -From the resource detail response, URls and links to services can be obtained from the ``resource.links[]`` array value. -The purpose of each link is defined by its ``link_type``. The “name” also can specify additional information about the linked resource. - -Metadata -^^^^^^^^ - -Links to each metadata format can be obtained from links with ``link_type = "metadata"`` - -OGC services -^^^^^^^^ - -OGC requests can be built by taking: -the OGC base url from links from ``resource.links[]`` with ``"link_type"= ("OGC:WMS | OGC:WFS | OGC:WCS)`` -the OGC service layername obtained from the ``resource.alternate`` property - -Embedding -^^^^^^^^ -A resource can be embedded inside a third party website. The “embed view” of a resource is suitable to be placed inside an iframe. -The URL for the embedded view can be obtained from the ``resource.embed_url`` property. - -Resource Searching and Filtering --------------------------------- - -GeoNode resources can be filtered with the following query parameters: - -.. list-table:: - :widths: 25 100 - :header-rows: 1 - - * - Parameters - - URL - * - title and abstract ``(paginated free text search)`` - - /api/v2/resources?page=1&search=text-to-search&search_fields=title&search_fields=abstract - * - resource_type ``(dataset, map, document, service, geostory, dashboard)`` - - /api/v2/resources?filter{resource_type}=map - * - subtype ``(raster,vector, vector_time, remote)`` - - /api/v2/resources?filter{resource_type}=vector - * - favorite ``(Boolean True)`` - - /api/v2/resources?favorite=true - * - featured ``(Boolean True or False)`` - - /api/v2/resources?filter{featured}=true - * - published ``(Boolean True or False)`` - - /api/v2/resources?filter{is_published}=true - * - aprroved ``(Boolean True or False)`` - - /api/v2/resources?filter{is_approved}=true - * - category - - api/v2/resources?filter{category.identifier}=example - * - keywords - - /api/v2/resources?filter{keywords.name}=example - * - regions - - /api/v2/resources?filter{regions.name}=global - * - owner - - /api/v2/resources?filter{owner.username}=test_user - * - extent ``(Four comer separated coordinates)`` - - /api/v2/resources?extent=-180,-90,180,90 - -Examples: - -1. Filter with a single value - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/resources/?filter{resource_type}=map" - response = requests.request("GET", url, headers=headers, data=payload - -2. Filter with multiple values - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/resources/?filter{resource_type.in}=map&filter{resource_type.in}=dataset" - response = requests.request("GET", url, headers=headers, data=payload) - - -.. note:: - With filter APIs of format ``/api/v2/resources?filter{filter_key}=value``, additional methods(in and icontains) can be used on them to provide extensively filtered results. - Eg - ``/api/v2/resources?filter{regions.name.icontains}=global`` - ``/api/v2/resources?filter{regions.name.in}=global``. - - It's important to note that other methods are case sensitive except the icontains. - - -Dataset specific resources ------------------------------ -Get the metadata of uploaded datasets with: - - API: ``GET /api/v2/datasets/{id}`` - - Status Code: ``200`` - .. note:: - This is very similar to `GET /api/v2/resources` but provides additional metadata specifically for datasets like `featureinfo_custom_template` or `attribute_set` - - Example: - - .. code-block:: python - - import requests - - DATASET_ID = "the dataset id" - url = f"https://master.demo.geonode.org/api/v2/datasets/{DATASET_ID}" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("GET", url, headers=headers) - -Resource Upload ---------------- - -The API supports the upload of datasets and documents. - -Datasets -"""""""""" -The dataset upload form accepts file formats of ESRI Shapefile, GeoTIFF, Comma Separated Value (CSV), Zip Archive, XML Metadata File, and Styled Layer Descriptor (SLD). -For a successful upload, the form requires base_file, dbf_file, shx_file, and prj_file. The xml_file, and Sld_file are optional. - -- API: ``POST /api/v2/uploads/upload`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/uploads/upload" - files= [ - ('sld_file',('BoulderCityLimits.sld',open('/home/myuser/BoulderCityLimits.sld','rb'),'application/octet-stream')), ('base_file',('BoulderCityLimits.shp',open('/home/BoulderCityLimits.shp','rb'),'application/octet-stream')), ('dbf_file',('BoulderCityLimits.dbf',open('/home/BoulderCityLimits.dbf','rb'),'application/octet-stream')), ('shx_file',('BoulderCityLimits.shx',open('/home/BoulderCityLimits.shx','rb'),'application/octet-stream')), - ('prj_file',('BoulderCityLimits.prj',open('/home/myuser/BoulderCityLimits.prj','rb'),'application/octet-stream)) - ] - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("POST", url, headers=headers, files=files) - -Documents -"""""""""" -Documents can be uploaded as form data. - -- API: ``POST /api/v2/documents`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "http://localhost:8000/api/v2/documents" - payload={ - 'title': 'An example image' - } - files=[ - ('doc_file',('image.jpg',open('/home/myuser/image.jpg','rb'),'image/jpeg')) - ] - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("POST", url, headers=headers, data=payload, files=files) - -Documents can also be created to reference remote resources. In this case the ``doc_url`` parameter must be used to set the URL of the remote document. - -- API: ``POST /api/v2/documents`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "http://localhost:8000/api/v2/documents" - payload={ - 'title': 'An example image', - 'doc_url' : 'http://examples.com/image.jpg' - } - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("POST", url, headers=headers, data=payload, files=files) - -Notice that if the URL doesn't end with a valid doc extension, the ``extension`` parameter must be used (e.g. ``extension: 'jpeg'``). - -Tracking dataset upload progress -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -When an upload request is executed, GeoNode creates an "Execution request" and keeps updating its state and progress (it’s a property attribute, calculated on getting the response) attributes as the resource is being created and configured in Geoserver. -An execution can be in one of the following status: - - ``ready`` - - ``running`` - - ``failed`` - - ``finished`` - -When the dataset is successfully uploaded, the final state of the upload is set to ``finished``. - -In order to view status of the execution, the API method ``GET /api/v2/executionrequest/{execution_id}`` where ``{execution_id}`` is the value returned by the initial call to the upload API. - -The returned object contains, beyond all the information related to the execution, the inputs that were passed to the execution request, and output params specific to the type of execution. -In the case of a dataset upload, the output params contain the URL of the catalog page for the new datast. - -.. code-block:: json - - "output_params": { - "detail_url": [ - "/catalogue/#/dataset/9881" - ] - }, - -You can also filter executions by status. -Eg ``GET /api/v2/executionrequest?filter{action}=import&filter{source}=upload&filter{status}=finished`` - - -Example: - -.. code-block:: python - - import requests - - url = "https://stable.demo.geonode.org/api/v2/executionrequest/5f640b6b-8c51-4514-a054-995133fee107" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("GET", url, headers=headers) - -Overwriting a dataset -^^^^^^^^^^^^^^^^^ -Uploading a resource will create by default a new dataset. This behaviour can be changed by setting the ``overwrite_existing_layer`` parameter to ``True``. -In this case the upload procedure will overwrite a resource whose name matches with the new one. - -Skip existing dataset -^^^^^^^^^^^^ -If the parameter ``skip_existing_layers`` is set to true ``True`` the uplad procedure will ignore files whose name matched with already existing resources. - -Upload of a metadata file -^^^^^^^^^^^^ -A complete metadata file conforming to ISO-19115 can be uploaded for a dataset. - -- API: ``PUT /api/v2/datasets/{dataset_id}/metadata`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "http://localhost:8000/api/v2/datasets/1/metadata" - files=[ - ('metadata_file',('metadata.xml',open('/home/user/metadata.xml','rb'),'text/xml')) - ] - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("PUT", url, headers=headers, data={}, files=files) - -Resource Delete ---------------- - -- API: ``DELETE /api/v2/resources/{pk}/delete`` -- Status Code: ``204`` - -Example: - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/resources/1778" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("DELETE", url, headers=headers) - -eee - -Resource Download ------------------ - -GeoNode offers a download option to resources of resource_type dataset and document. -For datasets, the download option is available for only datasets with uploaded files. - -Datasets -^^^^^^^^ -- API: ``GET /datasets/{resource.alternate}/dataset_download`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/datasets/geonode:BoulderCityLimits3/dataset_download" - response = requests.request("GET", url) - -Documents -^^^^^^^^ -- API: ``GET /documents/{resource.pk}/download`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/documents/1781/download" - response = requests.request("GET", url) - -Dataset Update Metadata ------------------------------ - -- API: ``PATCH /api/v2/datasets/{id}`` -- Status Code: ``200`` - -The following example changes the title and the license of a dataset. - -.. code-block:: python - - import requests - - url = ROOT + "api/v2/datasets/" + DATASET_ID - auth = (LOGIN_NAME, LOGIN_PASSWORD) - - data = { - "title": "a new title", - "license": 4, - } - response = requests.patch(url, auth=auth, json=data) -.. note:: - `bbox_polygon` and `ll_bbox_polygon` are derived values which cannot be changed. - -Users, Groups and Permissions ------------------------------ - -Users -^^^^^ - -Listing -"""""""""" -- API: ``POST /api/v2/users`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/users" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("GET", url, headers=headers) - - -User detail -"""""""""" -- API: ``POST /api/v2/users/{pk}`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/users/1000" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("GET", url, headers=headers) - -Create a new user -"""""""""" -- API: ``POST /api/v2/users`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/users" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - payload={"username": "username", - "password": "password", - "email": "email@email.com", - "first_name":"first_name", - "last_name":"last_name", - "avatar": "https://www.gravatar.com/avatar/7a68c67c8d409ff07e42aa5d5ab7b765/?s=240"} - response = requests.request("POST", url, headers=headers, data=payload) - - -Edit a User -"""""""""" -- API: ``PATCH /api/v2/users/{pk}`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/users/1000" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - payload={"password": "new_password"} - response = requests.request("PATCH", url, headers=headers, data=payload) - - -Delete a User -"""""""""" -- API: ``DELETE /api/v2/users/{pk}`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/users/1000" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - payload={"password": "new_password"} - response = requests.request("DELETE", url, headers=headers, data=payload) - - -In this case the list of validation rules configured in :ref:`user-deletion-rules` are checked before the deletion is executed. - - -List user groups -"""""""""" -- API: ``POST /api/v2/users/{pk}/groups`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/users/1000/groups" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("GET", url, headers=headers) - - -Transfer resources owned by a user to another -"""""""""" -- API: ``POST /api/v2/users/{pk}/transfer_resources`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - payload={"owner": 1001} - url = "https://master.demo.geonode.org/api/v2/users/1000/transfer_resources" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("POST", url, headers=headers, data=payload) - -In this case the resources will be transfered to the user with id 1001, -instead using the payload={"owner": "DEFAULT"} the resources will be transfered to the principal user - -Remove user as a group manager -"""""""""" -- API: ``POST /api/v2/users/{pk}/remove_from_group_manager`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - payload={"groups": [1,2,3]} - url = "https://master.demo.geonode.org/api/v2/users/1000/remove_from_group_manager" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("POST", url, headers=headers, data=payload) - -In this case the user shall be removed as a group manager from the following group ids, -if the payload would be payload={"groups": "ALL"} the user will be removed as a group manager from all the groups its part of - - - - - -Groups -^^^^^^ - -In GeoNode, On listing groups, the api returns groups which have group profiles. Therefore for django groups which are not related to a group profile are not included in the response. However these can be accessed in the Django Administration panel. - -- API: ``POST /api/v2/groups`` -- Status Code: ``200`` - -Example: - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/groups" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("GET", url, headers=headers) - - - - -Permissions -^^^^^^^^^^^ -Permissions in GeoNode are set per resource and per user or group. The following are general permissions that can be assigned: - -- *View:* allows to view the resource. ``[view_resourcebase]`` -- *Download:* allows to download the resource specifically datasets and documents. ``[ view_resourcebase, download_resourcebase]`` -- *Edit:* allows to change attributes, properties of the datasets features, styles and metadata for the specified resource. ``[view_resourcebase, download_resourcebase, change_resourcebase, change_dataset_style, change_dataset_data, change_resourcebase_metadata]`` -- *Manage:* allows to update, delete, change permissions, publish and unpublish the resource. ``[view_resourcebase, download_resourcebase, change_resourcebase, change_dataset_style, change_dataset_data, publish_resourcebase, delete_resourcebase, change_resourcebase_metadata, change_resourcebase_permissions]`` - -Obtaining permissions on a resource -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -On listing the resources or on resource detail API, GeoNode includes perms attribute to each resource with a list of permissions a user making the request has on the respective resource. - -GeoNode also provides an API to get an overview of permissions set on a resource. The response contains users and groups with permissions set on them. However this API returns ``200`` if a requesting user has ``manage`` permissions on the resource otherwise it will return ``403 (Forbidden)``. - -- API: ``GET /api/v2/resources/1791/permissions`` - -Example: - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/resources/1791/permissions" - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' - } - response = requests.request("GET", url, headers=headers) - - -Changing permissions on a resource -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Permissions are configured with a so called ``perms spec``, which is a JSON structured where permissions for single users and groups can be specified. - -The example below shows a perm specification for following rules: - -- user1 can ``edit`` -- user2 can ``manage`` -- group1 can ``edit`` -- anonymous users (public) can ``view`` -- registered members can ``download`` - -NOTE: The id of the “anonymous” and “registered members” groups can be obtained from the permissions of the resource. They are not listed inside the groups API, since these are spceial internal groups - -.. code-block:: python - - { - "users": [ - { - "id": , - "permissions": "edit" - }, - { - "id": , - "permissions": "manage" - } - ], - "organizations": [ - { - "id": , - "permissions": "edit" - }, - ], - "groups": [ - { - "id": , - "permissions": "view" - }, - { - "id": , - "permissions": "download" - } - ] - } - -The perm spec is sent as JSON, with ``application/json Content-Type``, inside a ``PUT`` request. - -.. code-block:: python - - import requests - import json - - url = "https://master.demo.geonode.org/api/v2/resources/1791/permissions" - payload = json.dumps({ - "users": [ - { - "id": 1001, - "permissions": "edit" - }, - { - "id": 1002, - "permissions": "manage" - } - ], - "organizations": [ - { - "id": 1, - "permissions": "edit" - } - ], - "groups": [ - { - "id": 2, - "permissions": "view" - }, - { - "id": 3, - "permissions": "download" - } - ] - }) - headers = { - 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==', - 'Content-Type': 'application/json', - } - - response = requests.request("PUT", url, headers=headers, data=payload) - -This is an asynchrnous operation which returns a response similar to the following: - -.. code-block:: python - - { - "status": "ready", - "execution_id": "7ed578c2-7db8-47fe-a3f5-6ed3ca545b67", - "status_url": "https://master.demo.geonode.org/api/v2/resource-service/execution-status/7ed578c2-7db8-47fe-a3f5-6ed3ca545b67" - } - - -The ``status_url`` property returns the URL to track kthe progress of the request. Querying the URL a result similar to the following will be returned: - -.. code-block:: python - - { - "user": "admin", - "status": "running", - "func_name": "set_permissions", - "created": "2022-07-08T11:16:32.240453Z", - "finished": null, - "last_updated": "2022-07-08T11:16:32.240485Z", - "input_params": { - … - } - } - - -The operation will be completed once the ``status`` property is updated with the value ``finished``. - -Linked Resources Listing and Details ------------------------------------- - -All available linked_resources can be listed with API ``GET /api/v2/resources/{pk}/linked_resources``. -where pk Resource base id - -Example Requests: -^^^^^^^^^^^^^^^^^ - -1. List all resource links - -.. code-block:: python - - import requests - - url = "https://master.demo.geonode.org/api/v2/resources/{pk}/linked_resources" - response = requests.request("GET", url) diff --git a/devel/docker/index.rst b/devel/docker/index.rst deleted file mode 100644 index 377291ef..00000000 --- a/devel/docker/index.rst +++ /dev/null @@ -1,85 +0,0 @@ -Start to develop with Docker ----------------------------- - -How to run the instance for development -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -There are two options to develop using Docker containers: - -- **Alternative A**: Running by command line and editing the code using your preferred editor (usually harder). -- **Alternative B**: Using the vscode `remote containers `_ extension (easier). - -Alternative A: Building and running Docker for development -.......................................................... - -Build (first time only): - -.. code-block:: shell - - docker-compose --project-name geonode -f docker-compose-dev.yml -f .devcontainer/docker-compose.yml build - -Running: - -.. code-block:: shell - - docker-compose --project-name geonode -f docker-compose-dev.yml -f .devcontainer/docker-compose.yml up - -.. note:: - If you are running ``postgresql`` and ``tomcat9`` services, you need to stop them, - ``docker-compose`` will take care of running the database and geonode service. - - Otherwise, you will get the following error: - - .. code-block:: text - - ERROR: for db Cannot start service db: driver failed programming external connectivity on endpoint db4geonode: Error starting userland proxy: listen tcp4 0.0.0.0:5432: bind: address already in use - ERROR: Encountered errors while bringing up the project. - - -Running the geonode application in debug mode: - -.. code-block:: shell - - docker exec -it django4geonode bash -c "python manage.py runserver 0.0.0.0:8000" - - -When running, you can debug the application using your preferred method. -For example, you can edit a file, save it and see your modifications. -You can also use `ipdb `_ to add breakpoints and inspect your code -(Writing ``import ipdb; ipdb.set_trace()`` in the line you want your breakpoint). - -Another option is to use *debugpy* alongside with *vscode*, for this you have to enable *debugpy* inside your *django4geonode* container: - -.. code-block:: shell - - docker exec -it django4geonode bash -c "pip install debugpy -t /tmp && python /tmp/debugpy --wait-for-client --listen 0.0.0.0:5678 manage.py runserver 0.0.0.0:8000 --nothreading --noreload" - -Select "**Run and Debug**" in vscode and use the following launch instruction in your ``.vscode/launch.json`` file: - -:download:`launch.json ` - - -Alternative B: Using vscode extension -..................................... - -Alternatively, you can develop using the vscode `remote containers `_ extension. -In this approach you need to: - -- Install the extension in your vscode: `ms-vscode-remote.remote-containers `_ - -- On your command pallet, select: "**Remote-Containers: Reopen in Container**" - -- If it's the first time, vscode will take care of building the images. This might take some time. - -- Then a new vscode window will open, and it'll be connected to your docker container. - -- The message "**Dev Container: Debug Docker Compose**" will appear in the bottom-left corner of that window. - -- In the vscode terminal, you're going to see something similar to ``root@77e80acc89b8:/usr/src/geonode#``. - -- To run your application, you can use the integrated terminal (``./manage.py runserver 0.0.0.0:8000``) or the vscode "**Run and Debug**" option. - For launching with "Run and Debug", use the following instruction file: - -:download:`launch.json ` - -For more information, take a read at vscode remote containers `help page `_. diff --git a/devel/index.rst b/devel/index.rst deleted file mode 100644 index 9bc740e1..00000000 --- a/devel/index.rst +++ /dev/null @@ -1,49 +0,0 @@ -GeoNode API -=========== - -.. toctree:: - :maxdepth: 3 - - api/usage/index - api/linked_resources/index - - -Metadata -======== - -.. toctree:: - :maxdepth: 4 - - metadata/index - - -How to Develop -============== - -.. toctree:: - :maxdepth: 3 - - docker/index - -.. toctree:: - :maxdepth: 2 - - installation/index - -.. toctree:: - :maxdepth: 2 - - running/index - -.. toctree:: - :maxdepth: 2 - - mapstore/index - -.. toctree:: - :maxdepth: 2 - - workshops/index - - -This documentation helps developers to install GeoNode-Core and GeoNode-Project from different scenarios. GeoNode-Project can be installed on top of GeoNode-Core if already installed. Also GeoNode-Project can be installed from scratch as it has GeoNode-Core as a prerequisit. diff --git a/devel/installation/index.rst b/devel/installation/index.rst deleted file mode 100644 index cf795158..00000000 --- a/devel/installation/index.rst +++ /dev/null @@ -1,422 +0,0 @@ -How to Install GeoNode-Core for development -=========================================== - -Summary of installation -....................... - -This section demonstrates a summarization of the steps to be followed in order to install GeoNode-Core for development using Ubuntu 18.04. The following steps will be customized to fit both GeoNode-Project and GeoNode-Core for development purpose. - -The steps to be followed are: -............................. - -1- Install build tools and libraries - -2- Install dependencies and supporting tools - -3- Setup Python virtual environment - -4- Clone and install GeoNode from Github - -5- Install and start Geoserver - -6- Start GeoNode - -.. note:: The following commands/steps will be executed on your terminal - -.. warning:: If you have a running GeoNode service, you will need to stop it before starting the following steps. To stop GeoNode you will need to run: - -.. code-block:: shell - - service apache2 stop # or your installed server - service tomcat7 stop # or your version of tomcat - -Install GeoNode-Core for development -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -GeoNode-Core installation is considered the most basic form of GeoNode. It doesn't require any external server to be installed and it can run locally against a file-system based Spatialite database. - -Installation steps -.................. - -1- Install build tools and libraries - -.. warning:: Those instructions might be outdated. Please refer to :ref:`install_dep` - -.. code-block:: shell - - $ sudo apt-get install -y build-essential libxml2-dev libxslt1-dev libpq-dev zlib1g-dev - -2- Install dependencies and supporting tools - - Install python native libraries and tools - -.. warning:: Those instructions might be outdated. Please refer to :ref:`install_dep` - -.. code-block:: shell - - $ sudo apt-get install -y python3-dev python3-pil python3-lxml python3-pyproj python3-shapely python3-nose python3-httplib2 python3-pip software-properties-common - -Install python virtual environment - -.. warning:: Those instructions might be outdated. Please refer to :ref:`install_venv` - -.. code-block:: shell - - $ sudo pip install virtualenvwrapper - -Install postgresql and postgis - -.. warning:: Those instructions might be outdated. Please refer to :ref:`configure_dbs_core` - -.. code-block:: shell - - $ sudo apt-get install postgresql-10 postgresql-10-postgis-2.4 - -Change postgres password expiry and set a password - -.. code-block:: shell - - $ sudo passwd -u postgres # change password expiry infromation - $ sudo passwd postgres # change unix password for postgres - -Create geonode role and database - -.. code-block:: shell - - $ su postgres - $ createdb geonode_dev - $ createdb geonode_dev-imports - $ psql - $ postgres=# - $ postgres=# CREATE USER geonode_dev WITH PASSWORD 'geonode_dev'; # should be same as password in setting.py - $ postgres=# GRANT ALL PRIVILEGES ON DATABASE "geonode_dev" to geonode_dev; - $ postgres=# GRANT ALL PRIVILEGES ON DATABASE "geonode_dev-imports" to geonode_dev; - $ postgres=# \q - $ psql -d geonode_dev-imports -c 'CREATE EXTENSION postgis;' - $ psql -d geonode_dev-imports -c 'GRANT ALL ON geometry_columns TO PUBLIC;' - $ psql -d geonode_dev-imports -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' - $ exit - -Edit PostgreSQL configuration file - -.. code-block:: shell - - sudo gedit /etc/postgresql/10/main/pg_hba.conf - -Scroll to the bottom of the file and edit this line - -.. code-block:: shell - - # "local" is for Unix domain socket connections only - local all all peer - -To be as follows - -.. code-block:: shell - - # "local" is for Unix domain socket connections only - local all all trust - -Then restart PostgreSQL to make the changes effective - -.. code-block:: shell - - sudo service postgresql restart - -Java dependencies - -.. code-block:: shell - - $ sudo apt-get install -y openjdk-11-jdk --no-install-recommends - -Install supporting tools - -.. code-block:: shell - - $ sudo apt-get install -y ant maven git gettext - -3- Setup Python virtual environment (Here is where Geonode will be running) - -Add the virtualenvwrapper to your new environment. - -Since we are using Ubuntu, you can add the following settings to your .bashrc file. Please note that the Ubuntu account here is called "geonode". So you will need to change it according to the name you picked. - -.. code-block:: shell - - $ echo export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python >> ~/.bashrc - $ echo export WORKON_HOME=/home/geonode/dev/.venvs >> ~/.bashrc - $ echo source /usr/local/bin/virtualenvwrapper.sh >> ~/.bashrc - $ echo export PIP_DOWNLOAD_CACHE=$HOME/.pip-downloads >> ~/.bashrc - - And reload the settings by running - $ source ~/.bashrc - -Set up the local virtual environment for Geonode - -.. code-block:: shell - - $ vim ~/.bashrc - # add the following line to the bottom - $ source /usr/share/virtualenvwrapper/virtualenvwrapper.sh - -.. code-block:: shell - - $ source /usr/share/virtualenvwrapper/virtualenvwrapper.sh - $ mkvirtualenv --python=/usr/bin/python3 geonode - $ workon geonode # or $ source /home/geonode/dev/.venvs/geonode/bin/activate - This creates a new directory where you want your project to be and creates a new virtualenvironment - - Alterantively you can also create the virtual env like below - $ python3.8 -m venv /home/geonode/dev/.venvs/geonode - $ source /home/geonode/dev/.venvs/geonode/bin/activate - -4- Download/Clone GeoNode from Github - -To download the latest geonode version from github, the command "git clone" is used - -.. Note:: If you are following the GeoNode training, skip the following command. You can find the cloned repository in /home/geonode/dev - -.. code-block:: shell - - $ git clone https://github.com/GeoNode/geonode.git -b 4.1.x - -Install Nodejs PPA and other tools required for static development - -This is required for static development - -.. Note:: If you are following GeoNode’s training, nodejs is already installed in the Virtual Machine skip the first three command and jump to cd geonode/geonode/static - -.. code-block:: shell - - $ sudo apt-get install nodejs npm - $ cd geonode/geonode/static - $ npm install --save-dev - -.. Note:: Every time you want to update the static files after making changes to the sources, go to geonode/static and run ‘grunt production’. - -.. warning:: Starting from the following step, you have to make sure that you installed GDAL correctly according to the documentation page "Install GDAL for Development" - -Install GeoNode in the new active local virtualenv - -.. code-block:: shell - - $ cd /home/geonode/dev # or to the directory containing your cloned GeoNode - $ pip install -e geonode - $ cd geonode/geonode - -Create local_settings.py - -Copy the sample file /home/geonode/dev/geonode/geonode/local_settings.py.geoserver.sample and rename it to be local_settings.py - -.. code-block:: shell - - $ cd /home/geonode/dev/geonode - $ cp geonode/local_settings.py.geoserver.sample geonode/local_settings.py - $ gedit geonode/local_settings.py - -In the local_settings.py file, add the following line after the import statements: - -.. code-block:: python - - SITEURL = "http://localhost:8000/" - -In the DATABASES dictionary under the 'default' key, change only the values for the keys NAME, USER and PASSWORD to be as follows: - -.. code-block:: python - - DATABASES = { - 'default': { - 'ENGINE': 'django.db.backends.postgresql_psycopg2', - 'NAME': 'geonode_dev', - 'USER': 'geonode_dev', - 'PASSWORD': 'geonode_dev', - ....... - ...... - ..... - .... - ... - }...} - -In the DATABASES dictionary under the 'datastore' key, change only the values for the keys NAME, USER and PASSWORD to be as follows: - -.. code-block:: python - - # vector datastore for uploads - 'datastore' : { - 'ENGINE': 'django.contrib.gis.db.backends.postgis', - #'ENGINE': '', # Empty ENGINE name disables - 'NAME': 'geonode_dev-imports', - 'USER' : 'geonode_dev', - 'PASSWORD' : 'geonode_dev', - ....... - ...... - ..... - .... - ... - } - - -In the CATALOGUE dictionary under the 'default' key, uncomment the USER and PASSWORD keys to activate the credentials for GeoNetwork as follows: - -.. code-block:: python - - CATALOGUE = { - 'default': { - # The underlying CSW implementation - # default is pycsw in local mode (tied directly to GeoNode Django DB) - 'ENGINE': 'geonode.catalogue.backends.pycsw_local', - # pycsw in non-local mode - # 'ENGINE': 'geonode.catalogue.backends.pycsw_http', - # GeoNetwork opensource - # 'ENGINE': 'geonode.catalogue.backends.geonetwork', - # deegree and others - # 'ENGINE': 'geonode.catalogue.backends.generic', - # The FULLY QUALIFIED base url to the CSW instance for this GeoNode - 'URL': urljoin(SITEURL, '/catalogue/csw'), - # 'URL': 'http://localhost:8080/geonetwork/srv/en/csw', - # 'URL': 'http://localhost:8080/deegree-csw-demo-3.0.4/services', - # login credentials (for GeoNetwork) - 'USER': 'admin', - 'PASSWORD': 'admin', - # 'ALTERNATES_ONLY': True, - }} - -5- Install and Start Geoserver - -From the virtual environment, first you need to align the database structure using the following command : - -.. code-block:: shell - - $ cd /home/geonode/dev/geonode - $ python manage.py migrate - -.. warning:: If the start fails because of an import error related to osgeo or libgeos, then please consult the `Install GDAL for Development `_ - - -then setup GeoServer using the following command: - -.. code-block:: shell - - $ paver setup - - $ paver sync - - -6- Now we can start our geonode instance - -.. warning:: Don’t forget to stop the GeoNode Production services if enabled - -.. code-block:: shell - - service apache2 stop - service tomcat7 stop - -.. code-block:: shell - - $ paver start - -The starting process will take around 20 seconds (depends on your machine) and at the end it shows the following message: - -.. image:: ./img/server-is-ready.png - -Now you can visit the geonode site by typing http://0.0.0.0:8000 into your browser window - -.. image:: ./img/geonode-gui.png - - - -Install GeoNode-Project for development after installing GeoNode-Core -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Geonode-Project gives the user flexibility to customize the installation of the GeoNode. Geonode itself will be installed as a requirement of your project. Inside the project structure it is possible to extend, replace or modify all geonode components (e.g. css and other static files, templates, models..) and even register new django apps without touching the original Geonode code. -In order to install GeoNode-Project, the following steps need to be executed alongside the previous GeoNode-Core installation steps. - - -1- Use django-admin.py to create a project "my_geonode" from a GeoNode-Project template as follows: - -.. note:: Before running the following command, make sure that you are currently working on the virtual environment and just outside geonode directory. The command will create a new project called "my_geonode" which should be located at the level of geonode-core installation directory "inside /home/geonode/dev" - -.. code-block:: shell - - $ django-admin.py startproject my_geonode --template=https://github.com/GeoNode/geonode-project/archive/master.zip -e py,rst,json,yml,ini,env,sample -n Dockerfile - - $ ls /home/geonode/dev # should output: geonode my_geonode - -.. note:: Although the following command might show that the majority of requirements are already satisfied "because GeoNode-Core was already installed", it is recommended to still execute it as it might update or install any missing package. - -2- Install all the required packages/tools for GeoNode-Project as follows: - -.. code-block:: shell - - $ pip install -e my_geonode - -.. note:: As mentioned earlier, GeoNode will be installed as requirement for the GeoNode-Project in order to be able to extend it - - -Install GeoNode-Project directly from scratch -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If you didn't install GeoNode-Core earlier and you wanted to install GeoNode-Project directly, please follow these steps - -1- Create a virtual environment as follows: - -.. code-block:: shell - - $ vim ~/.bashrc - # add the following line to the bottom - $ source /usr/share/virtualenvwrapper/virtualenvwrapper.sh - -.. code-block:: shell - - $ source /usr/share/virtualenvwrapper/virtualenvwrapper.sh - $ mkvirtualenv --python=/usr/bin/python3 my_geonode - - Alterantively you can also create the virtual env like below - $ python3.8 -m venv /home/geonode/dev/.venvs/my_geonode - $ source /home/geonode/dev/.venvs/my_geonode/bin/activate - -2- Clone the geonode-project repo from Github - -.. code-block:: shell - - $ git clone https://github.com/GeoNode/geonode-project.git -b 4.1.x - - -3- Install Django framework as follows - -.. code-block:: shell - - $ pip install Django==3.2.13 - -4- Use django-admin.py to create a project "my_geonode" from a GeoNode-Project template as follows: - -.. code-block:: shell - - $ django-admin startproject --template=./geonode-project -e py,sh,md,rst,json,yml,ini,env,sample,properties -n monitoring-cron -n Dockerfile my_geonode - -5- Install all the requirements for the GeoNode-Project and install the GeoNode-Project using pip - -.. code-block:: shell - - $ cd my_geonode - $ pip install -r requirements.txt --upgrade - $ pip install -e . --upgrade - -6- Install GDAL Utilities for Python - -.. code-block:: shell - - $ pip install pygdal=="`gdal-config --version`.*" # or refer to the link - -7- Install GeoServer and Tomcat using paver - -.. code-block:: shell - - $ paver setup - - $ paver sync - - $ paver start - -8- Visit http://localhost:8000/ - diff --git a/devel/installation/install_gdal_for_development.rst b/devel/installation/install_gdal_for_development.rst deleted file mode 100644 index cfd6a818..00000000 --- a/devel/installation/install_gdal_for_development.rst +++ /dev/null @@ -1,78 +0,0 @@ -How to Install GDAL for development -====================================== - -Summary of installation -....................... - -This section demonstrates a summarization of the steps to be followed in order to install GDAL 2.4.2 for development using python2.7 virtualenv with Ubuntu 18.04. The idea here is to install GDAL on the host machine and then refer to this installed version from the virtual environment to have identical versions. - -.. note:: The following commands/steps will be executed on your terminal - -.. warning:: The installation here is meant to serve GeoNode installation for development - -The steps to be followed are: -............................. - -1- Install GDAL on your host environment - -2- Validate your installation on the host machine - -3- From your python virtual environment, Run: pip install GDAL==$(gdal-config --version) --global-option=build_ext --global-option="-I/usr/include/gdal" - -4- Validate installation from host and virtual environment - - -Installation steps - -On a fresh Ubuntu 22.04 installation, we will be installing gdal version 3.3.2 - -1- Using the terminal in the host machine, run the following commands: - -.. code-block:: shell - - $ sudo apt-get update - $ sudo apt-get install python3-gdal=3.3.2+dfsg-2~focal2 gdal-bin=3.3.2+dfsg-2~focal2 libgdal-dev=3.3.2+dfsg-2~focal2 - -.. note:: Make sure you don't have a newer version on the software list under /et/apt/sources.list.d/ - -2- Validate the installed version by running the following commands: - - -.. code-block:: shell - - $ sudo gdal-config --version #it should output 3.3.2 at this step - $ python -c "from osgeo import gdal; print gdal.__version__" #it should output 3.3.2 at this step - -3- Install GDAL on your virtual environment by referring to the same version which we just installed on the host machine. To do that, you will need to run the following command during activated virtual environment session as follows: - - -.. code-block:: shell - - $ virtualenv -p python2.7 my_env # To create a python2.7 virtual environment called my_env - - $ # inside my_env/bin, run the following command to activate the virtual environment we just created - - $ source activate - - $ cd /home/geonode/my_env - - $ export CPLUS_INCLUDE_PATH=/usr/include/gdal - - $ export C_INCLUDE_PATH=/usr/include/gdal - - $ pip install GDAL==2.4.2 - -4- Confirm the installation - -In order to validate the installation between the host and the virtual environment, run the following command on both the host machine and on the activated virtual environment. If the results were identical, then GDAL for development is installed correctly for GeoNode - -.. code-block:: shell - - $ python -c "from osgeo import gdal; print gdal.__version__" # outputs "2.4.2" - -And from the activated environment: - -.. code-block:: shell - - $ (my_env) python -c "from osgeo import gdal; print gdal.__version__" # outputs "2.4.2" - diff --git a/devel/mapstore/index.rst b/devel/mapstore/index.rst deleted file mode 100644 index 0b5664b8..00000000 --- a/devel/mapstore/index.rst +++ /dev/null @@ -1,100 +0,0 @@ -Start MapStore2 client in development mode -========================================== - -Pre-requisites --------------- - -#. You need a running instance of GeoNode somewhere; in this specific example we assume GeoNode is running on ::guilabel:`http://localhost:8000` - -Install needed packages -....................... - -.. code-block:: shell - - sudo apt install nodejs npm - - -Prepare the source code -....................... - -.. code-block:: shell - - git clone --recursive https://github.com/GeoNode/geonode-mapstore-client.git geonode-mapstore-client-dev - -Compile MapStore2 Client -........................ - -.. code-block:: shell - - cd geonode-mapstore-client/geonode_mapstore_client/client/ - npm update - npm install - npm run compile - -Edit the file ``env.json`` -................................................... - -.. code-block:: shell - - vim env.json - -.. code-block:: json - - { - "DEV_SERVER_HOST": "localhost:8000", - "DEV_SERVER_HOST_PROTOCOL": "http" - } - -Run MapStore2 in Development mode -................................. - -.. code-block:: shell - - npm run start - -Connect to :::guilabel:`http://localhost:8081` - -This is a ``proxied`` version of GeoNode form MapStore2 client. **To upload new layers user the original GeoNode**. - -Everytime you render a map, from GeoNode layers details page or map creation, you will access to the MapStore2 dev mode runnig code. - -You can now update the code on the fly. - -Example 1: Disable the PrintPlugin from the Layer Details small map -................................................................... - -.. code-block:: shell - - vim js/previewPlugins.js - -.. code-block:: javascript - - ... - BurgerMenuPlugin: require('../MapStore2/web/client/plugins/BurgerMenu'), - ScaleBoxPlugin: require('../MapStore2/web/client/plugins/ScaleBox'), - MapFooterPlugin: require('../MapStore2/web/client/plugins/MapFooter'), - // PrintPlugin: require('../MapStore2/web/client/plugins/Print'), - TimelinePlugin: require('../MapStore2/web/client/plugins/Timeline'), - PlaybackPlugin: require('../MapStore2/web/client/plugins/Playback'), - ... - -Example 2: Disable the MousePositionPlugin from the big maps -............................................................ - -.. code-block:: shell - - vim js/plugins.js - -.. code-block:: javascript - - ... - SaveAsPlugin: require('../MapStore2/web/client/plugins/SaveAs').default, - MetadataExplorerPlugin: require('../MapStore2/web/client/plugins/MetadataExplorer'), - GridContainerPlugin: require('../MapStore2/web/client/plugins/GridContainer'), - StyleEditorPlugin: require('../MapStore2/web/client/plugins/StyleEditor'), - TimelinePlugin: require('../MapStore2/web/client/plugins/Timeline'), - PlaybackPlugin: require('../MapStore2/web/client/plugins/Playback'), - // MousePositionPlugin: require('../MapStore2/web/client/plugins/MousePosition'), - SearchPlugin: require('../MapStore2/web/client/plugins/Search'), - SearchServicesConfigPlugin: require('../MapStore2/web/client/plugins/SearchServicesConfig'), - ... diff --git a/devel/metadata/client.rst b/devel/metadata/client.rst deleted file mode 100644 index 83eedcfc..00000000 --- a/devel/metadata/client.rst +++ /dev/null @@ -1,168 +0,0 @@ -Metadata editor client -====================== - -The metadata editor client is basically implemented using the `react-jsonschema-form `__ library. - -Some improvements have been made, in order to better integrate with GeoNode, such as handling autocomplete, error reporting etc. - -In order to do, some custom annotation have been added. - - - -``geonode:after`` ------------------ - -This annotation is used server side, to place a field just after another. - -In the metadata editor the fields are presented in the same order as they are declared in the schema; this annotation allows to add a new declared field after an existing one: - -.. code-block:: json - - "short_name": { - "type": ["string", "null"], - "geonode:after": "title", - "geonode:handler": "sparse" - }, - - -Text widget ------------ - -Be default `react-jsonschema-form` presents the string elements in text fields. -If your text is multiline, you may want to edit it in a text area. - -This is how it is declared: - -.. code-block:: json - - - "ui:options": { - "widget": "textarea", - "rows": 5 - }, - -.. _metadata_dropdown: - -Dropdown menus --------------- - -In case you have a field that should be populated with the content of a service, you may use the autocomplete feature: - -.. code-block:: json - - "ui:options": { - "geonode-ui:autocomplete": "/api/v2/metadata/autocomplete/users" - } - -This is an improvement to `react-jsonschema-form` implemented within GeoNode. - -The client will present a dropdown menu with the content got from the autocomplete service. - -Please note that in order to handle such kind of dropdown, the service shall provide a list of entries having ``id`` and ``label`` as fields. - -According to the field type, different widgets will be shown. - - -Single choice -_____________ - -Here an example of an object: the widget will be a single choice dropdown menu: - -.. code-block:: json - - "owner": { - "type": "object", - "title": "Proprietario", - "properties": { - "id": { - "type": "string", - "ui:widget": "hidden" - }, - "label": { - "type": "string", - } - }, - "ui:options": { - "geonode-ui:autocomplete": "/api/v2/metadata/autocomplete/users" - }, - }, - -And this is how it is rendered: - -.. figure:: img/metadata_dropdown_single.png - :align: center - - *Single choice dropdown* - -Multi choice -____________ - -In case the field is an array, the dropdown will be multichoice: - -.. code-block:: json - - - "EOVsReference": { - "type": "array", - "items": { - "type": "object", - "properties": { - "id": { - "type": "string" - }, - "label": { - "type": "string" - } - } - }, - "ui:options": { - "geonode-ui:autocomplete": "/api/v2/metadata/autocomplete/thesaurus/eov/keywords" - } - }, - - -And this is how it is rendered: - -.. figure:: img/metadata_dropdown_multi.png - :align: center - - *Multi choice dropdown* - -.. _metadata_dropdown_codelist: - -Codelists ---------- - -In case a dropdown is fed with a codelist stored in a Thesaurus, you may use the ``geonode:thesaurus`` annotation, that will create an ``autocomplete`` entry pointing to the defined thesaurus. The dropdown can be either *single* or *multi* choice, as documented in the sections above. - -For instance, feeding the ``SparseHandler`` this subschema: - -.. code-block:: json - - "p_data_level": { - "type": "object", - "properties": { - "id": {"type": "string"}, - "label": {"type": "string"} - }, - "geonode:handler": "sparse", - "geonode:thesaurus": "prj1_data_level" - }, - -it will give back toward the client: - -.. code-block:: json - - "p_data_level": { - "type": "object", - "properties": { - "id": {"type": "string"}, - "label": {"type": "string"} - }, - "geonode:handler": "sparse", - "geonode:thesaurus": "prj1_data_level", - "title": "Data Level", - "ui:options": { - "geonode-ui:autocomplete": "/api/v2/metadata/autocomplete/thesaurus/prj1_data_level/keywords" - } - }, diff --git a/devel/metadata/design.rst b/devel/metadata/design.rst deleted file mode 100644 index ace8587a..00000000 --- a/devel/metadata/design.rst +++ /dev/null @@ -1,162 +0,0 @@ -Metadata app design -=================== - -The idea is to use a well-estabilished standard to document the metadata schema and to return the metadata info. - -The metadata schema is presented using the `JSON schema `__ conventions. - -The GeoNode API has been extended to provide: - -- ``/api/v2/metadata/schema`` metadata schema -- ``/api/v2/metadata/instance/`` resource metadata instances handling (GET/PUT) -- ``/api/v2/metadata/<...>`` other utility calls for the metadata handling (autocomplete endpoints referred into the schema, etc) - -One of the basic driving features has been the simplification of **customization of the schema**. The implementation relies on a set of **metadata handlers** that can be easily extended. - -The optional `geonode-inspire `__ module is a working showcase that documents how the GeoNode base model can be customized. - - -Implementation -============== - -The package ``geonode.metadata`` contains all the code related to metadata handling. - -The ``manager`` module contains the base engine that calls methods on the registered field handlers. - - -Handlers overview ------------------ - -**Handlers** are the objects that manage the field lifecycle, i.e.: - -* create the the json **sub-schema** of the field -* take care of **loading data** from the database and convert it into json schema instance -* parse the json schema instance value sent from the client and store it in the database format - -A single handler may handle one or more fields. - - -``MetadataManager`` -------------------- - -The ``geonode.metadata.manager.MetadataManager`` class handles requests from the view, and delegates the work on the fields to the declared handlers. - -The MetadataManager provides these main methods: - - -.. code-block:: - - def get_schema(self, lang=None) - -**Builds and returns the metadata json schema.** - -This method is called by the ``/api/v2/metadata/schema`` endpoint. - -The schema is created by looping on the sorted set of metadata handlers, asking them to update the json schema incrementally via ``def update_schema(self, jsonschema: dict, context, lang=None)``. - -Handlers may add their own field, remove or modify fields added by previously called handlers. - -The schema localization is only related to the `title` and `description` annotations. Localization of dropdown lists is an handler-specific task, and usually carried out via thesauri codelists (see :ref:`thesauri`). - -The default language used is the one set in the cookies, or it can be overriden by adding the query param `lang`. - - -.. code-block:: - - def build_schema_instance(self, resource: ResourceBase, lang=None) - -**Builds and returns the json schema instance related to a resource.** - -The instance is created by examining the json schema and looping on each field. -For each field, the related handler is called to fill in the serialized value for such field. - -Called handlers' methods: - -* ``def load_serialization_context(resource: ResourceBase, jsonschema: dict, context: dict)`` - Allow an handler to pre-load some common info for all of its handled fields -* ``def get_jsonschema_instance(resource: ResourceBase, field_name, context, errors, lang=None)`` - Returns the instance of the sub-schema associated with the field `field_name`. - -.. code-block:: - - def update_schema_instance(self, resource, request_obj, lang=None) -> dict: - -**Parses the incoming json schema instance and stores the value into the DB.** - -Persistence is done by examining the json schema and looping on each field. -For each field, the related handler is called to parse and store the deserialized value(s) for such field. -If the field is a complex object, it's up to the handler to parse and handle it. - -Called handlers' methods: - -* ``def load_deserialization_context(self, resource: ResourceBase, jsonschema: dict, context: dict)`` - Allow handlers to pre-load some common info for all the handled fields -* ``def update_resource(resource: ResourceBase, field_name: str, json_instance: dict, context: dict, errors: dict, **kwargs)`` - Logic to parse and persist the field - - -Handlers --------- - -The list of handlers is declared in ``geonode.metadata.settings.METADATA_HANDLERS``. - -**Order is important** since later declared handlers may want to customize previously defined fields. - -Within each field subschema there are some geonode-specific annotations (custom annotation are allowed by the json schema standard), some are needed for backend logic (for instance ``geonode:handler``), other for the client UI (e.g. ``ui:options``) - -``BaseHandler`` -_______________ - -``BaseHandler`` handles most of the fields declared inside the ``ResourceBase`` class. - -The ``BaseHandler`` will read the main schema from a file (`metadata/schemas/base.json `__), augmenting programmatically some info that can not be statically set (such as autocomplete URLs). - -``SparseHandler`` -_________________ - -This handler is not actively used by the core GeoNode schema, since it's more aimed at customizations. - -You may see an example of its use in the `geonode-inspire` package. - -The basic idea for the SparseField (the single model change in metadata) is to have a "vertical" table with free form fields: - -.. figure:: img/metadata_sparse_field.png - :align: center - - *ResourceBase and its Sparse fields* - - -In the default case, its use is as simple as declaring the field: - -.. code-block:: json - - "prj1_distance": { - "type": "numeric" - } - -or even a complex structure as - -.. code-block:: json - - "prj1_instrumentations": { - "type": "array", - "title": "instrumentations", - "description": "Provides information about any instruments used in the data collection. The description should include vendor, model number, optional equipment, etc.", - "items": { - "type": "string", - "minLength": 1, - "default": "" - }, - "minItems": 1, - "geonode:handler": "sparse" - } - -When the type is ``array`` or ``object`` the SparseHandler will encode the whole content as a json object - - -Some custom annotations ------------------------ - -* ``geonode:handler``: key to the handler defined in ``METADATA_HANDLERS``; tells which is the handler to be called to handle the field's content. Please note that this annotation may be replaced by subsequent handlers if they want to handle the field differently. - -* ``geonode:required``: the jsonschema specs wants the ``required`` array outside the subschema definition. By defining this annotation as ``true``, the metadata manager will add the current field in the ``required`` list. diff --git a/devel/metadata/example.rst b/devel/metadata/example.rst deleted file mode 100644 index 44eacc6b..00000000 --- a/devel/metadata/example.rst +++ /dev/null @@ -1,121 +0,0 @@ -Examples: Adding a metadata field -================================= - -To wrap up the metadata doumentation, here we will present some examples about how to add a brand new field to the metedat editor. - - -Example 1: minimal field ------------------------- - -Some points related to this field: - -* We do not need any special behaviour (i.e. logic for checking internal consistency, updating other fields according to the content of this field, etc) -* The field is declared as a simple type (numeric) - -In this case, where we don't need any specific logic, we can rely on the existing `SparseHandler `__. - -The SparseHandler has its own register of handled field, where custom apps can add their own, so in order to add a numerical field it will be enough to add it in the register, by specifying its name and the JSON sub-schema, e.g.: - -.. code-block:: python - - from geonode.metadata.handlers.sparse import sparse_field_registry - - sparse_field_registry.register("accuracy", {"type": "number"}) - - -This simple code will add the sparse field "accuracy", typed as "numeric", in the schema. - -Please note that the SparseHandler will return to the manager a possibly modified subschema, and not the registered one. -For instance: - -* the ``"geonode:handler". "sparse"`` will be added, in order to mark the handler for the field -* annotations ``title`` and ``description`` may be added or modified (see :ref:`metadata_fields_localization`). - -e.g. - -.. code-block:: json - - "accuracy": { - "type": "number", - "geonode:handler": "sparse", - "title": "Positional accuracy" - } - - -Example 2: complex json field ------------------------------ - -* No special behaviour -* The field may be as much complex as JSON schema specification allows. - -For instance, we want a list of persons defined by 2 fields, one containing the full name, and the other a URI. - -.. code-block:: json - - "prj1_data_creator": { - "type": "array", - "title": "Resource creator", - "minItems": 1, - "items": { - "type": "object", - "properties": { - "fullname": { - "type": "string", - "maxLength": 255, - "title": "Name surname", - "description": "Insert the full name of the person who created the resource" - }, - "orcid": { - "type": "string", - "format": "uri", - "title": "ORCID", - "description": "Insert the ORCID of the person who created the resource" - } - } - }, - "geonode:after": "abstract", - "geonode:handler": "sparse" - }, - -Since the type is a complex one (in this case it's ``array``, but also ``object`` is handled the same way), the SparseHandler will treat the content as a json object. Hera a sample for the content of the above structure: - -.. code-block:: json - - "prj1_data_creator": [ - { - "fullname": "John Doe", - "orcid": "http://sampleid" - }, - ], - - -Example 3: codelist ------------------------- - -You can add a field with a dropdown menu by using thesauri (see :ref:`metadata_dropdown`) - -* creating a brand new thesaurus with the items you needs (see :ref:`thesaurus_add`); - - * for instance, create a thesaurus with identifier ``project1_codelist_foobar`` and add some ThesaurusKeywords to it. - -* create a field (either single or multiple entry) and referencing the thesaurus (see :ref:`metadata_dropdown_codelist`) - - * es: - - .. code-block:: json - - "prj1_foobar": { - "type": "object", - "properties": { - "id": {"type": "string"}, - "label": {"type": "string"} - }, - "geonode:handler": "sparse", - "geonode:thesaurus": "project1_codelist_foobar" - }, - -* you may also add the label for the new field by adding a ThesaurusKeyword to the Thesaurus ``labels_i18n``, - - .. figure:: img/metadata_thesaurus_label.png - :width: 250px - diff --git a/devel/metadata/example2.rst b/devel/metadata/example2.rst deleted file mode 100644 index 7ba869aa..00000000 --- a/devel/metadata/example2.rst +++ /dev/null @@ -1,172 +0,0 @@ -Example: Adding an handler -========================== - -Create the handler ------------------- - -.. code-block:: python - - from geonode.metadata.handlers.abstract import MetadataHandler - - class MySchemaHandler(MetadataHandler): - pass - - -Register your handler, so that that metadata manager is aware of it: - - -.. code-block:: python - - from geonode.metadata.manager import metadata_manager - - metadata_manager.add_handler("myhandler", MySchemaHandler) - - -Create skeleton schema ----------------------- - -In order to define multiple fields, it may be useful to create a skeleton schema and add the fields one by one. - -This is a sample skeleton file ``myproject.json``: - -.. code-block:: json - - { - "p1_short_name": { - "type": ["string", "null"], - "minLength": 1, - "maxLength": 255, - "geonode:after": "title", - "geonode:handler": "sparse" - }, - - "edition": { - "type": "object", - "properties": { - "id": {"type": "string"}, - "label": {"type": "string"} - }, - "geonode:handler": "myhandler" - "geonode:thesaurus": "p1_edition_codelist" - } - } - -p1_short_name: - This is a brand new field, handled by the SparseHandler, so we don't want to handle it in YourSchemaHandler. - It will need to be registered as a SparseField. - -edition: - This field is declared into GeoNode as a field handled by the ``BaseHandler``. We are telling the manager that we want to handle it via the ``YourSchemaHandler``. - Also, we are defining it to be a codelist populated via the "p1_edition_codelist" thesaurus, that shall be created. - Since we are declaring a new Handler for this field, we'll need the logic to handle it. - - -Registering sparse fields -------------------------- - -You may want to register your stuff when initting your app: - -.. code-block:: python - - def load_schema_file(): - with open(os.path.join(os.path.dirname(__file__), "schemas", "myproject.json")) as f: - schema_file = json.load(f) - - return schema_file - - - def init(): - - # register sparse fields: - for property_name, subschema in load_schema_file().items(): - if subschema.get("geonode:handler", None) == "sparse": - sparse_field_registry.register(property_name, subschema) - - metadata_manager.add_handler("myhandler", YourSchemaHandler) - - -Generate schema ---------------- - -Then you need to implement the Handler method that initializes its part of schema (``update_schema``) - -.. code-block:: python - - def update_schema(self, jsonschema, context, lang=None): - - schema_file = load_schema_file() - - # building the full schema using the external file - for property_name, subschema in schema_file.items(): - - # tries to set up "title" and "description" - self._localize_subschema_labels(context, subschema, lang, property_name) - - if "geonode:handler" not in subschema: - # set the default handler if not specified - subschema.update({"geonode:handler": "myhandler"}) - else: - # skip fields that have already been added to the sparsefield register - if subschema.get("geonode:handler", None) == "sparse": - continue - - # add this field's subschema to the full jsonschema - self._add_subschema(jsonschema, property_name, subschema) - - -Now, since we took control of the ``edition`` field, we need to implement the methods that implement - -- the model-to-jsonschema transformation (``get_jsonschema_instance``) -- the jsonschema-to-model transformation (``update_resource``) - - -Handle content --------------- -We need to decide what will be stored in the ``edition`` field. - -The fields bound to thesauri are dict with keys ``id`` and ``label``, as also documented in the schema defined above. - -We want to save into the ``edition`` field ( which is defined in ResourceBase as a text field) some readable info; let's use the default label from the ThesaurusKeyword. In this thesaurus, **do not** define any localized labels, in order to be sure to get the default label as label. - - -When the jsonschema instance is returned, we have to provide the pair (id, label) anyway. We may retrieve the KeywordLabel by filtering by the default label, assuming that there are no duplicates. There may be other caveats, but just assume it is ok for the example. - - -model-to-jsonschema -___________________ - -Let's search for the ThesaurusKeyword with the given default label, and create the return object: - -.. code-block:: python - - def get_jsonschema_instance( - self, resource: ResourceBase, field_name: str, context: dict, errors: dict, lang: str = None - ): - match field_name: - case "edition": - if resource.edition: - if tl := ThesaurusLabel.objects.filter(alt_label=resource.edition).first() - return {"id": tl.pk, "label": resource.edition} - return {} - case _: - raise Exception(f"Unhandled field {field_name}") - - -jsonschema-to-model -___________________ - -The jsonschema instance, if existing, is still a dict with keys "id" and "label"; we'll only return the label: - - -.. code-block:: python - - def update_resource( - self, resource, field_name, json_instance, context, errors, **kwargs - ): - field_value = json_instance.get(field_name, {}) - match field_name: - case "edition": - resource.edition = field_value.get("label", None) - case _: - raise Exception(f"Unhandled field {field_name}") - diff --git a/devel/metadata/i18n.rst b/devel/metadata/i18n.rst deleted file mode 100644 index 3e52007e..00000000 --- a/devel/metadata/i18n.rst +++ /dev/null @@ -1,68 +0,0 @@ -.. _metadata_fields_localization: - -Metadata localization -===================== - -Before GeoNode 5, the labels for metadata editor have been statically localized using ``gettext()`` and i18n ``.po`` files. - -Since the jsonschema is much more dynamic in its behaviour, allowing custom schema, other ways for localizing the schema fields labels were needed. - -The proposed/implemented idea is to use a codelist, implemented as a thesaurus with a well known identifier ``labels_i18n`` where: - -* ``Thesaurus``: - * ``identifier`` = "`labels_i18n`" (fixed) - * ``date`` = latest update - -* ``ThesaurusKeyword``: - * ``about`` = explicit "title" annotation, otherwise field name - * ``alt_label`` = Default label -* ``ThesaurusKeywordLabel``: - * localized label - -In order to avoid clashes, the schema fields keys should be prefixed with the metadata handler id ("``inspire_``", "``myproject_``", ...) - -The implementation will look for translations in the thesaurus, and then will the gettext as fallback. - - -Procedure -_________ - -JSON schema provides the annotations ``title`` and ``description`` to document the purpose of a field. - -The metadata editor will show the "title" as the label for the field and the "description", if present, as a tooltip. -If the ``title`` field does not exist in the schema, the editor will usually show the field name as the label. - - -The SparseHandler will look for the ``title`` and ``description`` annotation in the subschema, and will try to localize them, using the content as the key for the localization procedures. - -So, if ``title`` is defined with the content ``title_key``, the procedures will look, in order, for: - -* a ThesaurusKeyword with about = ``title_key__ovr`` -* a ThesaurusKeyword with about = ``title_key`` -* a ``getText`` entry with key ``title_key`` - -and the first returned result is usaed as a replacemente for the "title" annotation. The `getText` value is used as a fallback anyway, since it returns the searched key if no entry is found. - -.. note:: - - The ``getText`` is used as a fallback for base fields for which localization was already provided. In this way the core GeoNode installation does not require the ``labels_i18n`` thesaurus, allowing for a smooth transition toward the new localization procedure. - -.. note:: - - The ``__ovr`` mechanism has been added as a way to customize locally standard fields' labels, avoiding conflicts when official thesauri content are changed - - Please note that the thesauri can be partially dumped or imported, in order to ease the handling of customized labels. - - - -Even if the ``title`` annotation is not present in the subschema, the GeoNode metadata engine will search for a localized string having key equal to the property name. If found, the ``title`` annotation will be automatically added. - -Similar to the above procedure, these keys are searched for: - -* a ThesaurusKeyword with about = ``__ovr`` -* a ThesaurusKeyword with about = ```` - -If anything is found, the ``title`` annotation will be automatically added with the found content. - - -In the same way, if the ``description`` annotation is not declared, the key ``__descr`` will be searched, and, if found, ``description`` will be added. diff --git a/devel/metadata/index.rst b/devel/metadata/index.rst deleted file mode 100644 index cee5de8c..00000000 --- a/devel/metadata/index.rst +++ /dev/null @@ -1,13 +0,0 @@ -Metadata -======== - -.. toctree:: - :maxdepth: 3 - - intro - design - i18n - caching - client - example - example2 \ No newline at end of file diff --git a/devel/metadata/intro.rst b/devel/metadata/intro.rst deleted file mode 100644 index 872f3da6..00000000 --- a/devel/metadata/intro.rst +++ /dev/null @@ -1,40 +0,0 @@ -Intro to GeoNode metadata management -==================================== - -Introduction ------------- - -Geographic metadata describes the content, quality, location, and structure of geospatial datasets. In GeoNode, it enables dataset discovery, interoperability, and standardized data management through structured, searchable information. - -GeoNode has its own internal metadata model, originally developed to populate the most relevant elements of the ISO 19115 standard. This model supported core geospatial metadata fields and was tightly coupled with the Django backend. - -With the release of GeoNode 5.x, a new metadata engine was introduced. It is based on JSON Schema, making it significantly more flexible and easier to customize. Unlike previous versions, where adding new fields required modifying the Django data model, developers can now define and extend metadata structures through configurable schemas without touching the core database structure. - - -Design overview ---------------- - -The metadata engine - implemented since GeoNode 5 - is based on a JSON Schema–driven model, offering a flexible and extensible way to manage resource metadata. - -Core concepts: - -* JSON Schema Model - - A single JSON Schema defines the structure, validation, and UI behavior of metadata fields across all resources (e.g., datasets, maps). - - The full set of metadata is passed as input and output, fully respecting the JSON Schema instance specifications, ensuring strict compliance with the defined structure. - -* Dynamic UI Generation - - The frontend generates metadata forms dynamically based on the schema, including field types, labels, grouping, and constraints. - -* Data Storage: - - * Core metadata fields (e.g., title, abstract, category) are still stored in the traditional ResourceBase model. - - * Custom or extended fields are stored using a new *Sparse field* mechanism, allowing flexible, schema-driven metadata - extension without altering the database schema. - -* Validation & Localization - - Input is validated against the JSON Schema, and schemas support multilingual labels and descriptions, enabling internationalized forms. diff --git a/devel/running/index.rst b/devel/running/index.rst deleted file mode 100644 index b07f7ce4..00000000 --- a/devel/running/index.rst +++ /dev/null @@ -1,49 +0,0 @@ -How to run GeoNode Core for development -======================================= - - -In order to start Geonode Core for development, you need to make sure that no Geonode instance is running first. This can be done by running the following commands: - -.. code-block:: shell - - $ cd /home/user/geonode - - $ paver stop_geoserver - - $ paver stop_django - -Then you need to start both geoserver and django services as follows: - -.. code-block:: shell - - $ paver start_geoserver - - $ paver start_django - -Now you can visit your Geonode GUI by typing http://localhost:8000 into your browser window - - - -How to run GeoNode Project for development -=========================================== - - -In order to run a project for development, the following steps have to be followed: - -1- Make sure there is no running instance of Geonode first by running the following command: - -.. code-block:: shell - - $ cd /home/user/my_geonode - - $ paver stop - -The above command will stop all services related to Geonode if running - -2- Start the servers by running paver start as follows: - -.. code-block:: shell - - $ paver start - -Now you can visit your geonode project site by typing http://localhost:8000 into your browser window diff --git a/devel/workshops/index.rst b/devel/workshops/index.rst deleted file mode 100644 index 6b5a74a9..00000000 --- a/devel/workshops/index.rst +++ /dev/null @@ -1,898 +0,0 @@ -Workshops -============================================== - -The workshops documentation demonstrates few examples on how to utilize GeoNode-Project in order to extend/customize GeoNode's functionalities according to your business. The covered topics include the following: - -1- Customize your GeoNode with the geonode-project - -2- Customize the look and feel - -3- Create your ResourceBase Metadata - -4- Create your own django app - -5- Add a custom model - -6- Permissions and APIs - -7- Deploy your GeoNode - - -1- Customize your GeoNode with the geonode-project --------------------------------------------------- - -In this example, GeoNode-Project is cloned to create a template instance in which the rest of the examples will be building on top of it. - -1- Assuming you already installed GeoNode-Core, firstly we need to create a GeoNode-Project template and this can be achieved from the following command: - -.. code-block:: shell - - $ django-admin.py startproject my_geonode --template=https://github.com/GeoNode/geonode-project/archive/master.zip -e py,rst,json,yml,ini,env,sample -n Dockerfile - -Here, django-admin is used with startproject option to create my_geonode project copying the template which is passed as GeoNode-project Github repo. It also includes "py,rst,json,yml,ini,env,sample" extensions - -2- Once the cloning finished, the next step is to install the GeoNode-Project we just downloaded as follows: - -.. code-block:: shell - - $ pip install -e my_geonode - - -3- Install geoserver using paver as follows - -.. code-block:: shell - - $ cd /home/geonode/my_geonode/src - $ paver setup - -4- Note the GeoNode database connection parameters mentioned in the .env.sample.py file. Rename it to .env then use psql to create the required user and grant the required privileges as follows: - -.. code-block:: shell - - $ su postgres - $ createdb geonode - $ psql - postgres=# CREATE USER geonode WITH PASSWORD 'geonode'; - CREATE ROLE - postgres=# GRANT ALL PRIVILEGES ON DATABASE "geonode" to geonode; - GRANT - postgres=# \q - -.. warning:: Don't forget to exit from postgres user before executing the following commands - -5- Run GeoNode using paver - -.. code-block:: shell - - $ cd /home/geonode/my_geonode/src - $ paver start - -.. note:: You may find this warning message: You have 132 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): account, actstream, admin, agon_ratings, announcements, auth, avatar, base, contenttypes, dialogos, documents, favorite, geonode_client, geonode_themes, groups, guardian, invitations, layers, maps, mapstore2_adapter, monitoring, oauth2_provider, people, pinax_notifications, services, sessions, sites, socialaccount, taggit, tastypie, upload, user_messages. Which means you have some sql statements not executed yet and you need to run the "migrate" to sync your database first then "paver start" again as follows: - -.. code-block:: shell - - $ python manage.py migrate - $ paver start - -.. warning:: If encountered this message: ``(Invalid HTTP_HOST header: '0.0.0.0:8000'. You may need to add u'0.0.0.0' to ALLOWED_HOSTS)`` It can be fixed in the settings.py file. You will need to add: ``ALLOWED_HOSTS = ['0.0.0.0']`` in settings.py - -6- Once the previous step is done, you can visit 0.0.0.0:8000 to view the GUI of GeoNode. However, we still don't have an account in order to login from the GUI. This can be done using "paver sync". The command will create sync with latest fixtures and also creates a superuser "admin" with default password "admin" - - -7- Use the created account to login from the GUI through localhost:8000 or 0.0.0.0:8000 - -.. image:: ./img/logged-in-geonode.png - - -2- Customize the look and feel ------------------------------- - -In this section we will change the look and feel of GeoNode, in particular we will do some customization to help understanding how the template inheritance works and how to add new stuff to your GeoNode. The changes will include the home page, the top menu, the footer and a generic GeoNode page. - -**Homepage:** - -The geonode-project provides some predefined templates to change the home page and the general site content. - -In the "my_geonode/src/my_geonode/templates/geonode-mapstore-client/snippets" directory we can files with similar names as the geonode-mapstore-client. This way we can override the different parts of the site eg the header, menu, body content and the footer. - -Create a file name hero.html and add the following. - -.. code:: html -
-
-
-

My GeoNode

-

Example customization of geonode project

-
-

-

-
-
- -.. image:: ./img/geonode-is-awesome.png - -**The theme:** - -To change the theme of our geonode-project we can act on the site_base.css file available in the "my_geonode/my_geonode/static/css" folder. - -The file is empty so we can inspect elements of the home page with the browser's developer tools and define css rules in there. - -For example, if we want to change the background of the jumbotron, in this file we can add - -.. code:: css - - .msgapi .gn-hero .jumbotron { background: red } - -Then once we refreshed the browser, we should see the change as follows: - -.. image:: ./img/red-background.png - -**The top menu:** - -Now we can make some changes that will apply to the whole site. We can add an item to both the left and right side of the top menu bar. - -This can be done by creating a get_menu_json.py under templatetags folder to override GeoNodes default menu. - - -.. code-block:: python - - @register.simple_tag(takes_context=True) - def get_base_right_topbar_menu(context): - - is_mobile = _is_mobile_device(context) - - if is_mobile: - return [] - - return [ - { - "type": "link", - "href": "/", - "label": "Custom 3" - }, - { - "type": "link", - "href": "/", - "label": "Custom 4" - }, - ] - @register.simple_tag(takes_context=True) - def get_base_left_topbar_menu(context): - - is_mobile = _is_mobile_device(context) - - return [ - { - "type": "link", - "href": "/", - "label": "Custom 1" - }, - { - "type": "link", - "href": "/", - "label": "Custom 2" - }, - ] - -On browser refresh you will see a new entry in the nav bar which is persistent to the whole site. - -.. image:: ./img/geocollection-menu.png - -**Modify functionality** - - -In this section, we will patch the ResourceBase of GeoNode and update the Templates in order to add one more field to the Metadata Schema. - -We will add a DOI field to the ResourceBase model and modify the Templates in order to show the new field both into the Metadata Wizard page. - -.. Note:: Make sure to be inside "my_geonode" directory to execute the following commands - - -Customizing metadata can be achieved from the model which is defined in the core at "geonode/geonode/base/models.py" as follows: - - -.. code-block:: python - - # internal fields - uuid = models.CharField(max_length=36) - owner = models.ForeignKey( - settings.AUTH_USER_MODEL, - blank=True, - null=True, - related_name='owned_resource', - verbose_name=_("Owner")) - contacts = models.ManyToManyField( - settings.AUTH_USER_MODEL, - through='ContactRole') - title = models.CharField(_('title'), max_length=255, help_text=_( - 'name by which the cited resource is known')) - alternate = models.CharField(max_length=128, null=True, blank=True) - date = models.DateTimeField( - _('date'), - default=now, - help_text=date_help_text) - date_type = models.CharField( - _('date type'), - max_length=255, - choices=VALID_DATE_TYPES, - default='publication', - help_text=date_type_help_text) - edition = models.CharField( - _('edition'), - max_length=255, - blank=True, - null=True, - help_text=edition_help_text) - abstract = models.TextField( - _('abstract'), - max_length=2000, - blank=True, - help_text=abstract_help_text) - purpose = models.TextField( - _('purpose'), - max_length=500, - null=True, - blank=True, - help_text=purpose_help_text) - maintenance_frequency = models.CharField( - _('maintenance frequency'), - max_length=255, - choices=UPDATE_FREQUENCIES, - blank=True, - null=True, - help_text=maintenance_frequency_help_text) - - -To add fields directly to the ResourceBase Class without actually modifying it, this can be done from "my_geonode/src/my_geonode/apps.py" file - -The "ready" method is invoked at initialization time and can be currently used to tweak your app in several ways - -.. code-block:: python - - - class AppConfig(BaseAppConfig): - - name = "my_geonode" - label = "my_geonode" - - def ready(self): - super(AppConfig, self).ready() - run_setup_hooks() - - -Now we will add the "patch_resource_base" method to the AppConfig and execute it from the ready method as follows: - -.. code-block:: python - - from django.db import models - from django.utils.translation import ugettext_lazy as _ - - - class AppConfig(BaseAppConfig): - - name = "my_geonode" - label = "my_geonode" - - def _get_logger(self): - import logging - return logging.getLogger(self.__class__.__module__) - - def patch_resource_base(self, cls): - self._get_logger().info("Patching Resource Base") - doi_help_text = _('a DOI will be added by Admin before publication.') - doi = models.TextField( - _('DOI'), - blank=True, - null=True, - help_text=doi_help_text) - cls.add_to_class('doi', doi) - - def ready(self): - super(AppConfig, self).ready() - run_setup_hooks() - - from geonode.base.models import ResourceBase - self.patch_resource_base(ResourceBase) - -.. note:: you will need to perform migrations as follows: - Add field doi to resourcebase - - -Once you run ``python manage.py migrate``: - -.. code-block:: shell - - Running migrations: - Applying announcements.0002_auto_20200119_1257... OK - Applying base.0031_resourcebase_doi... OK - Applying people.0027_auto_20200119_1257... OK - - -Till now, we have patched the DB. however, it is not yet sufficient as we still need to display the added field. - -Let's extend the default templates so that we can show the newly added field - -3- Create your own django app ------------------------------ - -In this section, we will demonstrate how to create and setup the skeleton of a custom app using the django facilities. The app will add a geocollections functionality to our GeoNode. - -The Geocollections app allows to present in a single page, resources and users grouped by a GeoNode Group. We can assign arbitrary resources to a Geocollection, a Group and a name that will be also used to build a dedicated URL. - -.. Note:: Make sure to be inside "my_geonode" directory to execute the following commands - -Create the django app - -Django gives us an handy command to create apps. We already used startproject to create our geonode-project, now we can use startapp to create the app. - -.. code:: shell - - python manage.py startapp geocollections - -This will create a folder named geocollections that contains empty models and views. - -We need to add the new app to the INSTALLED_APPS of our project. inside "my_geonode/src/my_geonode/settings.py": - -.. code:: python - - INSTALLED_APPS += (PROJECT_NAME,) to be: INSTALLED_APPS += (PROJECT_NAME, 'geocollections',) - - -**Add a custom model** - -In this section, we will add a custom model and the related logic as follows: - -- Add a new model -- Add urls and views -- Add admin panel -- Add the template - -.. code-block:: shell - - vim geocollections/models.py - -.. code-block:: python - - from django.db import models - - from geonode.base.models import ResourceBase - from geonode.groups.models import GroupProfile - - - class Geocollection(models.Model): - """ - A collection is a set of resources linked to a GeoNode group - """ - group = models.ForeignKey(GroupProfile, related_name='group_collections') - resources = models.ManyToManyField(ResourceBase, related_name='resource_collections') - name = models.CharField(max_length=128, unique=True) - slug = models.SlugField(max_length=128, unique=True) - - def __unicode__(self): - return self.name - - -At this point we need to ask django to create the database table. Django since version 1.8 has embedded migrations mechanism and we need to use them in order to change the state of the db. - -.. Note:: Make sure to be inside "my_geonode" directory to execute the following commands - -.. code-block:: shell - - python manage.py makemigrations - - # the above command informs you with the migrations to be executed on the database - - python manage.py migrate - -Next we will use django generic view to show the collections detail. Add the following code in the views.py file: - - -.. code-block:: shell - - vim geocollections/views.py - -.. code-block:: python - - from django.views.generic import DetailView - - from .models import Geocollection - - class GeocollectionDetail(DetailView): - model = Geocollection - - -Add url configuration - -In order to access the created view we also need url mapping. We can create a urls.py file containing a url mapping to our generic view: - - -.. code-block:: shell - - vim geocollections/urls.py - -.. code-block:: python - - from django.conf.urls import url - - from .views import GeocollectionDetail - - urlpatterns = [ - url(r'^(?P[-\w]+)/$', - GeocollectionDetail.as_view(), - name='geocollection-detail'), - ] - -We also need to register the app urls in the project urls. So let's modify the "my_geonode" urls.py file adding the following: - -.. code-block:: shell - - vim my_geonode/src/my_geonode/urls.py - -.. code-block:: python - - ... - urlpatterns += [ - ## include your urls here - url(r'^geocollections/', include('geocollections.urls')), - ] - ... - - -**Enable the admin panel** - -We need a user interface where we can create geocollections. Django makes this very easy, we just need the admin.py file as follows: - -.. code-block:: shell - - vim geocollections/admin.py - -.. code-block:: python - - from django.contrib import admin - - from .models import Geocollection - - - class GeocollectionAdmin(admin.ModelAdmin): - prepopulated_fields = {"slug": ("name",)} - filter_horizontal = ('resources',) - - admin.site.register(Geocollection, GeocollectionAdmin) - -Now we can visit the admin page and create a geocollection from there as follows: - -.. image:: ./img/geocollections-admin.png - -**Adding the template** - -Now we need the template where the geocollection detail will be rendered. Let's create a geocollections directory inside the "my_geonode/templates" directory with a file named geocollection_detail.html: - - -.. code-block:: shell - - mkdir -p my_geonode/templates/geocollections/ - - vim my_geonode/templates/geocollections/geocollection_detail.html - -.. code-block:: python - - {% extends "geonode_base.html" %} - {% block body %} -

Geocollection {{ object.name }}

-

Group: {{ object.group.title }}

-

Resources:

-
    - {% for resource in object.resources.all %} -
  • {{ resource.title }}
    • - {% endfor %} -
- {% endblock %} - -To check the results, create a group in the geonode ui interface and load one or more layers/documents - -login into the admin panel -> geocollections and create a geocollections - -Visit http://localhost:8000/geocollections/ and view the results. - -Now that you know how to customize an html template, you can tune the page as you prefer. - - - -**Permissions and APIs** - -In this section we will add some more advanced logic like permissions and APIs. The permissions in GeoNode are managed with django-guardian, a library which allow to set object level permissions (django has table level authorization). - -The APIs are implemented through django-tastypie. - -The topics to be covered include: - -- Permissions on who can view the geocollection -- How to add templated and js to embed a permission ui in our geocollection detail page -- API to serve json serialized searchable endpoint - - -Permissions logic (permissions objects) - -We need to add the permissions object to the database. We can do this by adding the following meta class to our Geocollection model, guardian will take care of creating the objects for us. - -.. code-block:: shell - - vim geocollections/models.py - -.. code-block:: python - - class Meta: - permissions = ( - ('view_geocollection', 'Can view geocollection'), - ) - -Then run "python manage.py makemigrations" and "python manage.py migrate" to install them - -**Permissions logic (set_default)** - -Let's add a method that will be used to set the default permissions on the Geocollections. We can add this logic to the Geocollection model but could also be a generic Mix-in similar to how it is implemented in GeoNode. - - -.. code-block:: shell - - vim geocollections/models.py - -.. code-block:: python - - - from django.contrib.auth.models import Group - from django.contrib.auth import get_user_model - from django.contrib.contenttypes.models import ContentType - from django.conf import settings - from guardian.shortcuts import assign_perm - - def set_default_permissions(self): - """ - Set default permissions. - """ - - self.remove_object_permissions() - - # default permissions for anonymous users - anonymous_group, created = Group.objects.get_or_create(name='anonymous') - - if settings.DEFAULT_ANONYMOUS_VIEW_PERMISSION: - assign_perm('view_geocollection', anonymous_group, self) - - # default permissions for group members - assign_perm('view_geocollection', self.group, self) - -**Permissions logic (methods)** - -Now we need a method to add generic permissions, we want to be able to assign view permissions to groups and single users. We can add this to our Geocollection model - - -.. code-block:: shell - - vim geocollections/models.py - -.. code-block:: python - - def set_permissions(self, perm_spec): - anonymous_group = Group.objects.get(name='anonymous') - self.remove_object_permissions() - if 'users' in perm_spec and "AnonymousUser" in perm_spec['users']: - assign_perm('view_geocollection', anonymous_group, self) - if 'users' in perm_spec: - for user, perms in perm_spec['users'].items(): - user = get_user_model().objects.get(username=user) - assign_perm('view_geocollection', user, self) - if 'groups' in perm_spec: - for group, perms in perm_spec['groups'].items(): - group = Group.objects.get(name=group) - assign_perm('view_geocollection', group, self) - def remove_object_permissions(self): - from guardian.models import UserObjectPermission, GroupObjectPermission - UserObjectPermission.objects.filter(content_type=ContentType.objects.get_for_model(self), - object_pk=self.id).delete() - GroupObjectPermission.objects.filter(content_type=ContentType.objects.get_for_model(self), - object_pk=self.id).delete() - -Permissions logic (views.py) - -We can add now a view to receive and set our permissions, in views.py: - -.. code-block:: shell - - vim geocollections/views.py - -.. code-block:: python - - import json - from django.core.exceptions import PermissionDenied - from django.http import HttpResponse - from django.contrib.auth import get_user_model - - - User = get_user_model() - - def geocollection_permissions(request, collection_id): - - collection = Geocollection.objects.get(id=collection_id) - user = User.objects.get(id=request.user.id) - - if user.has_perm('view_geocollection', collection): - return HttpResponse( - 'You have the permission to view. please customize a template for this view'', - content_type='text/plain') - - if request.method == 'POST': - success = True - message = "Permissions successfully updated!" - try: - permission_spec = json.loads(request.body) - collection.set_permissions(permission_spec) - - return HttpResponse( - json.dumps({'success': success, 'message': message}), - status=200, - content_type='text/plain' - ) - except: - success = False - message = "Error updating permissions :(" - return HttpResponse( - json.dumps({'success': success, 'message': message}), - status=500, - content_type='text/plain' - ) - -**Permissions logic (url)** - -Lastly we need a url to map our client to our view, in urls.py - -.. code-block:: shell - - vim geocollections/urls.py - -.. code-block:: python - - from django.conf.urls import url - - from .views import GeocollectionDetail, geocollection_permissions - - urlpatterns = [ - url(r'^(?P[-\w]+)/$', - GeocollectionDetail.as_view(), - name='geocollection-detail'), - - url(r'^permissions/(?P\d+)$', - geocollection_permissions, - name='geocollection_permissions'), - ] - -This url will be called with the id of the geocollection, the id will be passed to the view in order to get the permissions. - -.. image:: ./img/view-permission.png - -.. warning:: - - A note on the client part, the server side logic is just one part necessary to implement permissions. - - A checklist of what is necessary: - - - A template snippet that can be embedded in the geocollection_detail.html, you can copy and simplify: _permissions_form.html and _permissions.html (in geonode/templates) - - - A javascript file that will collect permissions settings and send them to the server, you can copy and simplify: _permissions_form_js.html (in geonode/templates) - -**API** - -The GeoNode API system easily allows to plug in new APIs. This section demonstrates the required steps: - -We need first to create an api.py file in our geocollection app. - -.. code-block:: shell - - vim geocollections/api.py - -.. code-block:: python - - import json - from tastypie.resources import ModelResource - from tastypie import fields - from tastypie.constants import ALL_WITH_RELATIONS, ALL - - from geonode.api.api import ProfileResource, GroupResource - from geonode.api.resourcebase_api import ResourceBaseResource - - from .models import Geocollection - class GeocollectionResource(ModelResource): - - users = fields.ToManyField(ProfileResource, attribute=lambda bundle: bundle.obj.group.group.user_set.all(), full=True) - group = fields.ToOneField(GroupResource, 'group', full=True) - resources = fields.ToManyField(ResourceBaseResource, 'resources', full=True) - - class Meta: - queryset = Geocollection.objects.all().order_by('-group') - ordering = ['group'] - allowed_methods = ['get'] - resource_name = 'geocollections' - filtering = { - 'group': ALL_WITH_RELATIONS, - 'id': ALL - } - -**API authorization** - -We want the API to respect our custom permissions, we can easily achieve this by adding the following to the beginning of api.py: - -.. code-block:: shell - - vim geocollections/api.py - -.. code-block:: python - - from tastypie.authorization import DjangoAuthorization - from guardian.shortcuts import get_objects_for_user - - class GeocollectionAuth(DjangoAuthorization): - - def read_list(self, object_list, bundle): - permitted_ids = get_objects_for_user( - bundle.request.user, - 'geocollections.view_geocollection').values('id') - - return object_list.filter(id__in=permitted_ids) - - def read_detail(self, object_list, bundle): - return bundle.request.user.has_perm( - 'view_geocollection', - bundle.obj) - -And this to the GeocollectionResource Meta class: - -.. code-block:: python - - authorization = GeocollectionAuth() - -**Add a url for our API** - -In order to publish our API we need a url and we want that url to appear under the GeoNode's /api domain. - -The final url for our API has to be /api/geocollections. - -We can inject the url into the GeoNode API by adding the following lines to "my_geonode/urls.py" file: - -.. code-block:: shell - - vim my_geonode/urls.py - -.. code-block:: python - - from geonode.api.urls import api - - from geocollections.api import GeocollectionResource - - api.register(GeocollectionResource()) - -And add the following in the urlpatterns: - -.. code-block:: python - - url(r'', include(api.urls)), - -The final result will be: - -.. code-block:: python - - from django.conf.urls import url, include - from django.views.generic import TemplateView - - from geonode.urls import urlpatterns - - from geonode.api.urls import api - from geocollections.api import GeocollectionResource - - api.register(GeocollectionResource()) - - urlpatterns += [ - ## include your urls here - url(r'', include(api.urls)), - url(r'^geocollections/', include('geocollections.urls')), - ] - -Let's test permissions on API - -We can test the permissions on API by manually set a permission from the command line and check that the API respects it. - -With running ``python manage.py shell`` from inside our "my_geonode" folder, it opens a geonode shell. - -A perm spec could look like this: - -.. code:: python - - perms = { - 'users': { - 'AnonymousUser': ['view_geocollection'], - 'alessio': ['view_geocollection']} - } - -and we can assign the permissions with: - -.. code-block:: python - - from geocollections.models import Geocollection - - Geocollection.objects.first().set_permissions(perms) - -our http://localhost:8000/api/geocollections should now list the geocollection. - -If you remove the 'AnonymousUser' line from perms and assign again the permissions it will disappear. - -.. code:: python - - perms = { - 'users': { - 'alessio': ['view_geocollection'] - } - -**Deploy your GeoNode** - - -So far we demonstrated how to modify, extend and style our GeoNode in dev mode but now it's time to go on production. In this section we will clarify how to: - -- commit your work on GitHub - -- setup your server - -- setup your GeoNode for production - - - -Push to GitHub -It is always a good practice to keep your code in a remote repository, GithHub is one of the options and is indeed the most used. - -It is assumed that you already have a GitHub account and that you have git installed and configured with your name and email. - -We will push only the my_geonode folder to GitHub and as we knew earlier, GeoNode for us is a dependency and we'll just reinstall it as it is on the server. - -Steps to push your code to GitHub: - -- Create an empty repository in GitHub and copy it's address - -- In my_geonode, run git init to initialize an empty repository - -- Add your remote repository address with ``git remote add yourname yourremoteaddress`` - -- edit .gitignore adding all *.pyc files - -- ``git add *`` to add all content of my_geonode - -- ``git commit -m 'initial import'`` to make the initial commit - -- ``git push yourname master`` to push the code to the GitHub repository - - -**Setup the server** - -There are several options for deploying GeoNode projects on servers. In this section, we explain how to deploy it on Ubuntu server 18.04 using system-wide installation - -.. note:: For quick installation, follow the INSTALLING documentation at http://docs.geonode.org/en/master/install/core/index.html - - -**Setup our my_geonode** - -We need now to install the developed "my_geonode" project following these steps: - -- git clone from your repository (in the folder of your preference) - -- ``sudo pip install -e my_geonode`` - -- edit the settings where needed - -- edit ``/etc/apache2/sites-enabled/geonode.conf`` replacing the wsgi path to the ``my_geonode/my_geonode/wsgi.py`` file - -- add the apache rights to the "my_geonode" folder with a directory like - -.. code:: apache - - - Order allow,deny - Require all granted - - -- Test your server. diff --git a/about/img/geonode-logo_for_readme.gif b/docs/about/img/geonode-logo_for_readme.gif similarity index 100% rename from about/img/geonode-logo_for_readme.gif rename to docs/about/img/geonode-logo_for_readme.gif diff --git a/about/img/logo.png b/docs/about/img/logo.png similarity index 100% rename from about/img/logo.png rename to docs/about/img/logo.png diff --git a/docs/about/index.md b/docs/about/index.md new file mode 100644 index 00000000..6fd085fc --- /dev/null +++ b/docs/about/index.md @@ -0,0 +1,84 @@ +# What is GeoNode + +![image](img/logo.png) + +GeoNode is a geospatial content management system, a platform for the management and publication of geospatial data. It brings together mature and stable open-source software projects under a consistent and easy-to-use interface allowing non-specialized users to share data and create interactive maps. + +Data management tools built into GeoNode allow for integrated creation of data, metadata, and map visualization. Each dataset in the system can be shared publicly or restricted to allow access to only specific users. Social features like user profiles and commenting and rating systems allow for the development of communities around each platform to facilitate the use, management, and quality control of the data the GeoNode instance contains. + +It is also designed to be a flexible platform that software developers can extend, modify or integrate against to meet requirements in their own applications. + +## Showcase + +A handful of other Open Source projects extend GeoNode's functionality +by tapping into the re-usability of Django applications. +Visit our gallery to see how the community uses GeoNode: [GeoNode Showcase](http://geonode.org/gallery/). + +The development community is very supportive of new projects and contributes ideas and guidance for newcomers. + +For a live demo see also `online_demo`{.interpreted-text role="ref"} + +## Most useful links + +**General** + +- Project homepage: + +- Repository: + +- Official Demo: + +- GeoNode Wiki: + +- Issue tracker: + + > In case of sensitive bugs like security vulnerabilities, please + > contact a GeoNode Core Developer directly instead of using issue + > tracker. We value your effort to improve the security and privacy of + > this project! + +**Related projects** + +- GeoNode Project: +- GeoNode at Docker: +- GeoNode OSGeo-Live: + +# Licensing + +GeoNode is Copyright 2018 Open Source Geospatial Foundation (OSGeo). + +GeoNode is free software: you can redistribute it and/or modify it under +the terms of the GNU General Public License as published by the Free +Software Foundation, either version 3 of the License, or (at your +option) any later version. GeoNode is distributed in the hope that it +will be useful, but WITHOUT ANY WARRANTY; without even the implied +warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License along +with GeoNode. If not, see . + +# Current Version and Features + +GeoNode current version: [3.2.0](https://github.com/GeoNode/geonode/releases/tag/3.2.0) + +Main Features: [State of GeoNode](https://www.geosolutionsgroup.com/blog/state-geonode-3-2-webinar/) + +# Get in touch with the community {#get_in_touch} + +GeoNode is an open source project and contributors are needed to keep this project moving forward. Learn more on how to contribute on our +[Community Bylaws](https://github.com/GeoNode/geonode/wiki/Community-Bylaws). + +- User Mailing List: +- Developer Mailing List: +- Gitter Chat: + +# Roadmap + +GeoNode\'s development roadmap is documented in a series of GeoNode Improvement Projects (GNIPS). +They are documented at [GeoNode Wiki](https://github.com/GeoNode/geonode/wiki/GeoNode-Improvement-Proposals). + +GNIPS are considered to be large undertakings which will add a large amount of features to the project. +As such they are the topic of community discussion and guidance. + +The community discusses these on the developer mailing list: diff --git a/admin/admin_panel/img/add_new_member.png b/docs/admin/admin_panel/img/add_new_member.png similarity index 100% rename from admin/admin_panel/img/add_new_member.png rename to docs/admin/admin_panel/img/add_new_member.png diff --git a/admin/admin_panel/img/add_user_button.png b/docs/admin/admin_panel/img/add_user_button.png similarity index 100% rename from admin/admin_panel/img/add_user_button.png rename to docs/admin/admin_panel/img/add_user_button.png diff --git a/admin/admin_panel/img/add_user_form.png b/docs/admin/admin_panel/img/add_user_form.png similarity index 100% rename from admin/admin_panel/img/add_user_form.png rename to docs/admin/admin_panel/img/add_user_form.png diff --git a/admin/admin_panel/img/add_user_link.png b/docs/admin/admin_panel/img/add_user_link.png similarity index 100% rename from admin/admin_panel/img/add_user_link.png rename to docs/admin/admin_panel/img/add_user_link.png diff --git a/admin/admin_panel/img/admin-datasets-batch-form.png b/docs/admin/admin_panel/img/admin-datasets-batch-form.png similarity index 100% rename from admin/admin_panel/img/admin-datasets-batch-form.png rename to docs/admin/admin_panel/img/admin-datasets-batch-form.png diff --git a/admin/admin_panel/img/admin-datasets-batch.png b/docs/admin/admin_panel/img/admin-datasets-batch.png similarity index 100% rename from admin/admin_panel/img/admin-datasets-batch.png rename to docs/admin/admin_panel/img/admin-datasets-batch.png diff --git a/admin/admin_panel/img/admin-datasets.png b/docs/admin/admin_panel/img/admin-datasets.png similarity index 100% rename from admin/admin_panel/img/admin-datasets.png rename to docs/admin/admin_panel/img/admin-datasets.png diff --git a/admin/admin_panel/img/admin-documents.png b/docs/admin/admin_panel/img/admin-documents.png similarity index 100% rename from admin/admin_panel/img/admin-documents.png rename to docs/admin/admin_panel/img/admin-documents.png diff --git a/admin/admin_panel/img/admin-layers.png b/docs/admin/admin_panel/img/admin-layers.png similarity index 100% rename from admin/admin_panel/img/admin-layers.png rename to docs/admin/admin_panel/img/admin-layers.png diff --git a/admin/admin_panel/img/admin-maps-featured-001.png b/docs/admin/admin_panel/img/admin-maps-featured-001.png similarity index 100% rename from admin/admin_panel/img/admin-maps-featured-001.png rename to docs/admin/admin_panel/img/admin-maps-featured-001.png diff --git a/admin/admin_panel/img/admin-maps-featured-002.png b/docs/admin/admin_panel/img/admin-maps-featured-002.png similarity index 100% rename from admin/admin_panel/img/admin-maps-featured-002.png rename to docs/admin/admin_panel/img/admin-maps-featured-002.png diff --git a/admin/admin_panel/img/admin-maps.png b/docs/admin/admin_panel/img/admin-maps.png similarity index 100% rename from admin/admin_panel/img/admin-maps.png rename to docs/admin/admin_panel/img/admin-maps.png diff --git a/admin/admin_panel/img/admin-panel-menus-0000.png b/docs/admin/admin_panel/img/admin-panel-menus-0000.png similarity index 100% rename from admin/admin_panel/img/admin-panel-menus-0000.png rename to docs/admin/admin_panel/img/admin-panel-menus-0000.png diff --git a/admin/admin_panel/img/admin-panel-menus-0001.png b/docs/admin/admin_panel/img/admin-panel-menus-0001.png similarity index 100% rename from admin/admin_panel/img/admin-panel-menus-0001.png rename to docs/admin/admin_panel/img/admin-panel-menus-0001.png diff --git a/admin/admin_panel/img/admin-panel-menus-0002.png b/docs/admin/admin_panel/img/admin-panel-menus-0002.png similarity index 100% rename from admin/admin_panel/img/admin-panel-menus-0002.png rename to docs/admin/admin_panel/img/admin-panel-menus-0002.png diff --git a/admin/admin_panel/img/admin-panel-menus-0003.png b/docs/admin/admin_panel/img/admin-panel-menus-0003.png similarity index 100% rename from admin/admin_panel/img/admin-panel-menus-0003.png rename to docs/admin/admin_panel/img/admin-panel-menus-0003.png diff --git a/admin/admin_panel/img/admin-panel-menus-0004.png b/docs/admin/admin_panel/img/admin-panel-menus-0004.png similarity index 100% rename from admin/admin_panel/img/admin-panel-menus-0004.png rename to docs/admin/admin_panel/img/admin-panel-menus-0004.png diff --git a/admin/admin_panel/img/admin-panel-metadata-contents-0001.png b/docs/admin/admin_panel/img/admin-panel-metadata-contents-0001.png similarity index 100% rename from admin/admin_panel/img/admin-panel-metadata-contents-0001.png rename to docs/admin/admin_panel/img/admin-panel-metadata-contents-0001.png diff --git a/admin/admin_panel/img/admin-people.png b/docs/admin/admin_panel/img/admin-people.png similarity index 100% rename from admin/admin_panel/img/admin-people.png rename to docs/admin/admin_panel/img/admin-people.png diff --git a/admin/admin_panel/img/admin-profiles-contactroles.png b/docs/admin/admin_panel/img/admin-profiles-contactroles.png similarity index 100% rename from admin/admin_panel/img/admin-profiles-contactroles.png rename to docs/admin/admin_panel/img/admin-profiles-contactroles.png diff --git a/admin/admin_panel/img/admin-roles-add.png b/docs/admin/admin_panel/img/admin-roles-add.png similarity index 100% rename from admin/admin_panel/img/admin-roles-add.png rename to docs/admin/admin_panel/img/admin-roles-add.png diff --git a/admin/admin_panel/img/admin_link.png b/docs/admin/admin_panel/img/admin_link.png similarity index 100% rename from admin/admin_panel/img/admin_link.png rename to docs/admin/admin_panel/img/admin_link.png diff --git a/admin/admin_panel/img/adv_data_workflow/adv_data_workflow_001.jpg b/docs/admin/admin_panel/img/adv_data_workflow/adv_data_workflow_001.jpg similarity index 100% rename from admin/admin_panel/img/adv_data_workflow/adv_data_workflow_001.jpg rename to docs/admin/admin_panel/img/adv_data_workflow/adv_data_workflow_001.jpg diff --git a/admin/admin_panel/img/adv_data_workflow/approbation_manager.gif b/docs/admin/admin_panel/img/adv_data_workflow/approbation_manager.gif similarity index 100% rename from admin/admin_panel/img/adv_data_workflow/approbation_manager.gif rename to docs/admin/admin_panel/img/adv_data_workflow/approbation_manager.gif diff --git a/admin/admin_panel/img/adv_data_workflow/unpublished.png b/docs/admin/admin_panel/img/adv_data_workflow/unpublished.png similarity index 100% rename from admin/admin_panel/img/adv_data_workflow/unpublished.png rename to docs/admin/admin_panel/img/adv_data_workflow/unpublished.png diff --git a/admin/admin_panel/img/announcments/admin-announcments-001.png b/docs/admin/admin_panel/img/announcments/admin-announcments-001.png similarity index 100% rename from admin/admin_panel/img/announcments/admin-announcments-001.png rename to docs/admin/admin_panel/img/announcments/admin-announcments-001.png diff --git a/admin/admin_panel/img/announcments/admin-announcments-002.png b/docs/admin/admin_panel/img/announcments/admin-announcments-002.png similarity index 100% rename from admin/admin_panel/img/announcments/admin-announcments-002.png rename to docs/admin/admin_panel/img/announcments/admin-announcments-002.png diff --git a/admin/admin_panel/img/announcments/admin-announcments-003.png b/docs/admin/admin_panel/img/announcments/admin-announcments-003.png similarity index 100% rename from admin/admin_panel/img/announcments/admin-announcments-003.png rename to docs/admin/admin_panel/img/announcments/admin-announcments-003.png diff --git a/admin/admin_panel/img/announcments/admin-announcments-004.png b/docs/admin/admin_panel/img/announcments/admin-announcments-004.png similarity index 100% rename from admin/admin_panel/img/announcments/admin-announcments-004.png rename to docs/admin/admin_panel/img/announcments/admin-announcments-004.png diff --git a/admin/admin_panel/img/announcments/admin-announcments-005.png b/docs/admin/admin_panel/img/announcments/admin-announcments-005.png similarity index 100% rename from admin/admin_panel/img/announcments/admin-announcments-005.png rename to docs/admin/admin_panel/img/announcments/admin-announcments-005.png diff --git a/admin/admin_panel/img/announcments/admin-announcments-006.png b/docs/admin/admin_panel/img/announcments/admin-announcments-006.png similarity index 100% rename from admin/admin_panel/img/announcments/admin-announcments-006.png rename to docs/admin/admin_panel/img/announcments/admin-announcments-006.png diff --git a/admin/admin_panel/img/announcments/admin-announcments-007.png b/docs/admin/admin_panel/img/announcments/admin-announcments-007.png similarity index 100% rename from admin/admin_panel/img/announcments/admin-announcments-007.png rename to docs/admin/admin_panel/img/announcments/admin-announcments-007.png diff --git a/admin/admin_panel/img/announcments/admin-announcments-008.png b/docs/admin/admin_panel/img/announcments/admin-announcments-008.png similarity index 100% rename from admin/admin_panel/img/announcments/admin-announcments-008.png rename to docs/admin/admin_panel/img/announcments/admin-announcments-008.png diff --git a/admin/admin_panel/img/announcments/admin-announcments-009.png b/docs/admin/admin_panel/img/announcments/admin-announcments-009.png similarity index 100% rename from admin/admin_panel/img/announcments/admin-announcments-009.png rename to docs/admin/admin_panel/img/announcments/admin-announcments-009.png diff --git a/admin/admin_panel/img/announcments/admin-announcments-010.png b/docs/admin/admin_panel/img/announcments/admin-announcments-010.png similarity index 100% rename from admin/admin_panel/img/announcments/admin-announcments-010.png rename to docs/admin/admin_panel/img/announcments/admin-announcments-010.png diff --git a/admin/admin_panel/img/change_password_form.png b/docs/admin/admin_panel/img/change_password_form.png similarity index 100% rename from admin/admin_panel/img/change_password_form.png rename to docs/admin/admin_panel/img/change_password_form.png diff --git a/admin/admin_panel/img/change_password_link.png b/docs/admin/admin_panel/img/change_password_link.png similarity index 100% rename from admin/admin_panel/img/change_password_link.png rename to docs/admin/admin_panel/img/change_password_link.png diff --git a/admin/admin_panel/img/change_user_password_form.png b/docs/admin/admin_panel/img/change_user_password_form.png similarity index 100% rename from admin/admin_panel/img/change_user_password_form.png rename to docs/admin/admin_panel/img/change_user_password_form.png diff --git a/admin/admin_panel/img/change_user_password_link.png b/docs/admin/admin_panel/img/change_user_password_link.png similarity index 100% rename from admin/admin_panel/img/change_user_password_link.png rename to docs/admin/admin_panel/img/change_user_password_link.png diff --git a/admin/admin_panel/img/confirm_group_deletion.png b/docs/admin/admin_panel/img/confirm_group_deletion.png similarity index 100% rename from admin/admin_panel/img/confirm_group_deletion.png rename to docs/admin/admin_panel/img/confirm_group_deletion.png diff --git a/admin/admin_panel/img/create_group_page_link.png b/docs/admin/admin_panel/img/create_group_page_link.png similarity index 100% rename from admin/admin_panel/img/create_group_page_link.png rename to docs/admin/admin_panel/img/create_group_page_link.png diff --git a/admin/admin_panel/img/django_geonode_admin_interface.png b/docs/admin/admin_panel/img/django_geonode_admin_interface.png similarity index 100% rename from admin/admin_panel/img/django_geonode_admin_interface.png rename to docs/admin/admin_panel/img/django_geonode_admin_interface.png diff --git a/admin/admin_panel/img/editing_group_logo.png b/docs/admin/admin_panel/img/editing_group_logo.png similarity index 100% rename from admin/admin_panel/img/editing_group_logo.png rename to docs/admin/admin_panel/img/editing_group_logo.png diff --git a/admin/admin_panel/img/group_activities.png b/docs/admin/admin_panel/img/group_activities.png similarity index 100% rename from admin/admin_panel/img/group_activities.png rename to docs/admin/admin_panel/img/group_activities.png diff --git a/admin/admin_panel/img/group_categories_list.png b/docs/admin/admin_panel/img/group_categories_list.png similarity index 100% rename from admin/admin_panel/img/group_categories_list.png rename to docs/admin/admin_panel/img/group_categories_list.png diff --git a/admin/admin_panel/img/group_creation_form.png b/docs/admin/admin_panel/img/group_creation_form.png similarity index 100% rename from admin/admin_panel/img/group_creation_form.png rename to docs/admin/admin_panel/img/group_creation_form.png diff --git a/admin/admin_panel/img/group_details_page.png b/docs/admin/admin_panel/img/group_details_page.png similarity index 100% rename from admin/admin_panel/img/group_details_page.png rename to docs/admin/admin_panel/img/group_details_page.png diff --git a/admin/admin_panel/img/group_logo.png b/docs/admin/admin_panel/img/group_logo.png similarity index 100% rename from admin/admin_panel/img/group_logo.png rename to docs/admin/admin_panel/img/group_logo.png diff --git a/admin/admin_panel/img/group_profile_details_page.png b/docs/admin/admin_panel/img/group_profile_details_page.png similarity index 100% rename from admin/admin_panel/img/group_profile_details_page.png rename to docs/admin/admin_panel/img/group_profile_details_page.png diff --git a/admin/admin_panel/img/group_profiles_list.png b/docs/admin/admin_panel/img/group_profiles_list.png similarity index 100% rename from admin/admin_panel/img/group_profiles_list.png rename to docs/admin/admin_panel/img/group_profiles_list.png diff --git a/admin/admin_panel/img/group_profiles_list_page.png b/docs/admin/admin_panel/img/group_profiles_list_page.png similarity index 100% rename from admin/admin_panel/img/group_profiles_list_page.png rename to docs/admin/admin_panel/img/group_profiles_list_page.png diff --git a/admin/admin_panel/img/groups_admin_section.png b/docs/admin/admin_panel/img/groups_admin_section.png similarity index 100% rename from admin/admin_panel/img/groups_admin_section.png rename to docs/admin/admin_panel/img/groups_admin_section.png diff --git a/admin/admin_panel/img/groups_link.png b/docs/admin/admin_panel/img/groups_link.png similarity index 100% rename from admin/admin_panel/img/groups_link.png rename to docs/admin/admin_panel/img/groups_link.png diff --git a/admin/admin_panel/img/groups_list_page.png b/docs/admin/admin_panel/img/groups_list_page.png similarity index 100% rename from admin/admin_panel/img/groups_list_page.png rename to docs/admin/admin_panel/img/groups_list_page.png diff --git a/admin/admin_panel/img/groups_page_link.png b/docs/admin/admin_panel/img/groups_page_link.png similarity index 100% rename from admin/admin_panel/img/groups_page_link.png rename to docs/admin/admin_panel/img/groups_page_link.png diff --git a/admin/admin_panel/img/layers_group_category.png b/docs/admin/admin_panel/img/layers_group_category.png similarity index 100% rename from admin/admin_panel/img/layers_group_category.png rename to docs/admin/admin_panel/img/layers_group_category.png diff --git a/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0001.png b/docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0001.png similarity index 100% rename from admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0001.png rename to docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0001.png diff --git a/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0002.png b/docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0002.png similarity index 100% rename from admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0002.png rename to docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0002.png diff --git a/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0003.png b/docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0003.png similarity index 100% rename from admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0003.png rename to docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0003.png diff --git a/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0003a.png b/docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0003a.png similarity index 100% rename from admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0003a.png rename to docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0003a.png diff --git a/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0004.png b/docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0004.png similarity index 100% rename from admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0004.png rename to docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0004.png diff --git a/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0005.png b/docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0005.png similarity index 100% rename from admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0005.png rename to docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0005.png diff --git a/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0006.png b/docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0006.png similarity index 100% rename from admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0006.png rename to docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0006.png diff --git a/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0007.png b/docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0007.png similarity index 100% rename from admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0007.png rename to docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0007.png diff --git a/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0008.png b/docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0008.png similarity index 100% rename from admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0008.png rename to docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0008.png diff --git a/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0009.png b/docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0009.png similarity index 100% rename from admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0009.png rename to docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0009.png diff --git a/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0010.png b/docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0010.png similarity index 100% rename from admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0010.png rename to docs/admin/admin_panel/img/metadata-base/admin-panel-metadata-contents-0010.png diff --git a/admin/admin_panel/img/new_group_category_form.png b/docs/admin/admin_panel/img/new_group_category_form.png similarity index 100% rename from admin/admin_panel/img/new_group_category_form.png rename to docs/admin/admin_panel/img/new_group_category_form.png diff --git a/admin/admin_panel/img/new_group_profile_form.png b/docs/admin/admin_panel/img/new_group_profile_form.png similarity index 100% rename from admin/admin_panel/img/new_group_profile_form.png rename to docs/admin/admin_panel/img/new_group_profile_form.png diff --git a/admin/admin_panel/img/new_members.png b/docs/admin/admin_panel/img/new_members.png similarity index 100% rename from admin/admin_panel/img/new_members.png rename to docs/admin/admin_panel/img/new_members.png diff --git a/admin/admin_panel/img/new_user_active.png b/docs/admin/admin_panel/img/new_user_active.png similarity index 100% rename from admin/admin_panel/img/new_user_active.png rename to docs/admin/admin_panel/img/new_user_active.png diff --git a/admin/admin_panel/img/new_user_disabled.png b/docs/admin/admin_panel/img/new_user_disabled.png similarity index 100% rename from admin/admin_panel/img/new_user_disabled.png rename to docs/admin/admin_panel/img/new_user_disabled.png diff --git a/admin/admin_panel/img/new_user_in_people.png b/docs/admin/admin_panel/img/new_user_in_people.png similarity index 100% rename from admin/admin_panel/img/new_user_in_people.png rename to docs/admin/admin_panel/img/new_user_in_people.png diff --git a/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0001.png b/docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0001.png similarity index 100% rename from admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0001.png rename to docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0001.png diff --git a/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0002.png b/docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0002.png similarity index 100% rename from admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0002.png rename to docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0002.png diff --git a/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003.png b/docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003.png similarity index 100% rename from admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003.png rename to docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003.png diff --git a/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003a.png b/docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003a.png similarity index 100% rename from admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003a.png rename to docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003a.png diff --git a/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003b.png b/docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003b.png similarity index 100% rename from admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003b.png rename to docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003b.png diff --git a/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003c.png b/docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003c.png similarity index 100% rename from admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003c.png rename to docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003c.png diff --git a/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003d.png b/docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003d.png similarity index 100% rename from admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003d.png rename to docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003d.png diff --git a/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003e.png b/docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003e.png similarity index 100% rename from admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003e.png rename to docs/admin/admin_panel/img/oauth2-tokens/admin-panel-tokens-0003e.png diff --git a/admin/admin_panel/img/set_datasets_permissions_action.png b/docs/admin/admin_panel/img/set_datasets_permissions_action.png similarity index 100% rename from admin/admin_panel/img/set_datasets_permissions_action.png rename to docs/admin/admin_panel/img/set_datasets_permissions_action.png diff --git a/admin/admin_panel/img/set_layers_permissions_action.png b/docs/admin/admin_panel/img/set_layers_permissions_action.png similarity index 100% rename from admin/admin_panel/img/set_layers_permissions_action.png rename to docs/admin/admin_panel/img/set_layers_permissions_action.png diff --git a/admin/admin_panel/img/set_layers_permissions_form.png b/docs/admin/admin_panel/img/set_layers_permissions_form.png similarity index 100% rename from admin/admin_panel/img/set_layers_permissions_form.png rename to docs/admin/admin_panel/img/set_layers_permissions_form.png diff --git a/admin/admin_panel/img/staff_and_superuser_permissions.png b/docs/admin/admin_panel/img/staff_and_superuser_permissions.png similarity index 100% rename from admin/admin_panel/img/staff_and_superuser_permissions.png rename to docs/admin/admin_panel/img/staff_and_superuser_permissions.png diff --git a/admin/admin_panel/img/theme-def-0003c.png b/docs/admin/admin_panel/img/theme-def-0003c.png similarity index 100% rename from admin/admin_panel/img/theme-def-0003c.png rename to docs/admin/admin_panel/img/theme-def-0003c.png diff --git a/admin/admin_panel/img/theming/default_theme.png b/docs/admin/admin_panel/img/theming/default_theme.png similarity index 100% rename from admin/admin_panel/img/theming/default_theme.png rename to docs/admin/admin_panel/img/theming/default_theme.png diff --git a/admin/admin_panel/img/theming/first_customization.png b/docs/admin/admin_panel/img/theming/first_customization.png similarity index 100% rename from admin/admin_panel/img/theming/first_customization.png rename to docs/admin/admin_panel/img/theming/first_customization.png diff --git a/admin/admin_panel/img/theming/hide_jumbotron.png b/docs/admin/admin_panel/img/theming/hide_jumbotron.png similarity index 100% rename from admin/admin_panel/img/theming/hide_jumbotron.png rename to docs/admin/admin_panel/img/theming/hide_jumbotron.png diff --git a/admin/admin_panel/img/theming/hide_jumbotron_view.png b/docs/admin/admin_panel/img/theming/hide_jumbotron_view.png similarity index 100% rename from admin/admin_panel/img/theming/hide_jumbotron_view.png rename to docs/admin/admin_panel/img/theming/hide_jumbotron_view.png diff --git a/admin/admin_panel/img/theming/jumbotron_slide_add.png b/docs/admin/admin_panel/img/theming/jumbotron_slide_add.png similarity index 100% rename from admin/admin_panel/img/theming/jumbotron_slide_add.png rename to docs/admin/admin_panel/img/theming/jumbotron_slide_add.png diff --git a/admin/admin_panel/img/theming/theme-def-0001.png b/docs/admin/admin_panel/img/theming/theme-def-0001.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0001.png rename to docs/admin/admin_panel/img/theming/theme-def-0001.png diff --git a/admin/admin_panel/img/theming/theme-def-0002.png b/docs/admin/admin_panel/img/theming/theme-def-0002.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0002.png rename to docs/admin/admin_panel/img/theming/theme-def-0002.png diff --git a/admin/admin_panel/img/theming/theme-def-0003a.png b/docs/admin/admin_panel/img/theming/theme-def-0003a.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0003a.png rename to docs/admin/admin_panel/img/theming/theme-def-0003a.png diff --git a/admin/admin_panel/img/theming/theme-def-0003b.png b/docs/admin/admin_panel/img/theming/theme-def-0003b.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0003b.png rename to docs/admin/admin_panel/img/theming/theme-def-0003b.png diff --git a/admin/admin_panel/img/theming/theme-def-0003d.png b/docs/admin/admin_panel/img/theming/theme-def-0003d.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0003d.png rename to docs/admin/admin_panel/img/theming/theme-def-0003d.png diff --git a/admin/admin_panel/img/theming/theme-def-0003e.png b/docs/admin/admin_panel/img/theming/theme-def-0003e.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0003e.png rename to docs/admin/admin_panel/img/theming/theme-def-0003e.png diff --git a/admin/admin_panel/img/theming/theme-def-0004.png b/docs/admin/admin_panel/img/theming/theme-def-0004.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0004.png rename to docs/admin/admin_panel/img/theming/theme-def-0004.png diff --git a/admin/admin_panel/img/theming/theme-def-0004a.png b/docs/admin/admin_panel/img/theming/theme-def-0004a.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0004a.png rename to docs/admin/admin_panel/img/theming/theme-def-0004a.png diff --git a/admin/admin_panel/img/theming/theme-def-0004b.png b/docs/admin/admin_panel/img/theming/theme-def-0004b.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0004b.png rename to docs/admin/admin_panel/img/theming/theme-def-0004b.png diff --git a/admin/admin_panel/img/theming/theme-def-0004c.png b/docs/admin/admin_panel/img/theming/theme-def-0004c.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0004c.png rename to docs/admin/admin_panel/img/theming/theme-def-0004c.png diff --git a/admin/admin_panel/img/theming/theme-def-0004d.png b/docs/admin/admin_panel/img/theming/theme-def-0004d.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0004d.png rename to docs/admin/admin_panel/img/theming/theme-def-0004d.png diff --git a/admin/admin_panel/img/theming/theme-def-0005.png b/docs/admin/admin_panel/img/theming/theme-def-0005.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0005.png rename to docs/admin/admin_panel/img/theming/theme-def-0005.png diff --git a/admin/admin_panel/img/theming/theme-def-0005a.png b/docs/admin/admin_panel/img/theming/theme-def-0005a.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0005a.png rename to docs/admin/admin_panel/img/theming/theme-def-0005a.png diff --git a/admin/admin_panel/img/theming/theme-def-0005b.png b/docs/admin/admin_panel/img/theming/theme-def-0005b.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0005b.png rename to docs/admin/admin_panel/img/theming/theme-def-0005b.png diff --git a/admin/admin_panel/img/theming/theme-def-0005c.png b/docs/admin/admin_panel/img/theming/theme-def-0005c.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0005c.png rename to docs/admin/admin_panel/img/theming/theme-def-0005c.png diff --git a/admin/admin_panel/img/theming/theme-def-0006.png b/docs/admin/admin_panel/img/theming/theme-def-0006.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0006.png rename to docs/admin/admin_panel/img/theming/theme-def-0006.png diff --git a/admin/admin_panel/img/theming/theme-def-0006a.png b/docs/admin/admin_panel/img/theming/theme-def-0006a.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0006a.png rename to docs/admin/admin_panel/img/theming/theme-def-0006a.png diff --git a/admin/admin_panel/img/theming/theme-def-0006b.png b/docs/admin/admin_panel/img/theming/theme-def-0006b.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0006b.png rename to docs/admin/admin_panel/img/theming/theme-def-0006b.png diff --git a/admin/admin_panel/img/theming/theme-def-0006c.png b/docs/admin/admin_panel/img/theming/theme-def-0006c.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0006c.png rename to docs/admin/admin_panel/img/theming/theme-def-0006c.png diff --git a/admin/admin_panel/img/theming/theme-def-0007a.png b/docs/admin/admin_panel/img/theming/theme-def-0007a.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0007a.png rename to docs/admin/admin_panel/img/theming/theme-def-0007a.png diff --git a/admin/admin_panel/img/theming/theme-def-0007b.png b/docs/admin/admin_panel/img/theming/theme-def-0007b.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0007b.png rename to docs/admin/admin_panel/img/theming/theme-def-0007b.png diff --git a/admin/admin_panel/img/theming/theme-def-0007c.png b/docs/admin/admin_panel/img/theming/theme-def-0007c.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0007c.png rename to docs/admin/admin_panel/img/theming/theme-def-0007c.png diff --git a/admin/admin_panel/img/theming/theme-def-0007d.png b/docs/admin/admin_panel/img/theming/theme-def-0007d.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0007d.png rename to docs/admin/admin_panel/img/theming/theme-def-0007d.png diff --git a/admin/admin_panel/img/theming/theme-def-0007e.png b/docs/admin/admin_panel/img/theming/theme-def-0007e.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0007e.png rename to docs/admin/admin_panel/img/theming/theme-def-0007e.png diff --git a/admin/admin_panel/img/theming/theme-def-0007f.png b/docs/admin/admin_panel/img/theming/theme-def-0007f.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0007f.png rename to docs/admin/admin_panel/img/theming/theme-def-0007f.png diff --git a/admin/admin_panel/img/theming/theme-def-0007g.png b/docs/admin/admin_panel/img/theming/theme-def-0007g.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0007g.png rename to docs/admin/admin_panel/img/theming/theme-def-0007g.png diff --git a/admin/admin_panel/img/theming/theme-def-0007h.png b/docs/admin/admin_panel/img/theming/theme-def-0007h.png similarity index 100% rename from admin/admin_panel/img/theming/theme-def-0007h.png rename to docs/admin/admin_panel/img/theming/theme-def-0007h.png diff --git a/admin/admin_panel/img/theming/theme_properties.png b/docs/admin/admin_panel/img/theming/theme_properties.png similarity index 100% rename from admin/admin_panel/img/theming/theme_properties.png rename to docs/admin/admin_panel/img/theming/theme_properties.png diff --git a/admin/admin_panel/img/theming/themes.png b/docs/admin/admin_panel/img/theming/themes.png similarity index 100% rename from admin/admin_panel/img/theming/themes.png rename to docs/admin/admin_panel/img/theming/themes.png diff --git a/admin/admin_panel/img/theming/view_site.png b/docs/admin/admin_panel/img/theming/view_site.png similarity index 100% rename from admin/admin_panel/img/theming/view_site.png rename to docs/admin/admin_panel/img/theming/view_site.png diff --git a/admin/admin_panel/img/user_details_admin_page.png b/docs/admin/admin_panel/img/user_details_admin_page.png similarity index 100% rename from admin/admin_panel/img/user_details_admin_page.png rename to docs/admin/admin_panel/img/user_details_admin_page.png diff --git a/docs/admin/admin_panel/index.md b/docs/admin/admin_panel/index.md new file mode 100644 index 00000000..3bf369e1 --- /dev/null +++ b/docs/admin/admin_panel/index.md @@ -0,0 +1,1060 @@ +# Accessing the panel + +| The *Admin Panel* is a model-centric interface where trusted users can manage content on GeoNode. +| Only the staff users can access the admin interface. + +::: note +::: title +Note +::: + +The "staff" flag, which controls whether the user is allowed to log in to the admin interface, can be set by the admin panel itself. +::: + +The panel can be reached from `Admin`{.interpreted-text role="guilabel"} link of the *User Menu* in the navigation bar (see the picture below) or through this URL: `http:///admin`. + +
+img/admin_link.png +
The Admin Link of the User Menu
+
+ +When clicking on that link the Django-based *Admin Interface* page opens and shows you all the Django models registered in GeoNode. + +
+img/django_geonode_admin_interface.png +
The GeoNode Admin Interface
+
+ +# Reset or Change the admin password + +From the *Admin Interface* you can access the `Change password`{.interpreted-text role="guilabel"} link by clicking on the username on the right side of the navigation bar. which will open a dropdown. + +
+img/change_password_link.png +
The Change Password Link
+
+ +It allows you to access the *Change Password Form* through which you can change your password. + +
+img/change_password_form.png +
The Change Password Form
+
+ +Once the fields have been filled out, click on `Change my password`{.interpreted-text role="guilabel"} to perform the change. + +# Simple Theming + +GeoNode provides by default some theming options manageable directly from the Administration panel. +Most of the times those options allows you to easily change the GeoNode look and feel without touching a single line of [HTML]{.title-ref} or [CSS]{.title-ref}. + +As an [administrator]{.title-ref} go to `http:///admin/geonode_themes/geonodethemecustomization/`. + +
+img/theming/themes.png +
List of available Themes
+
+ +The panel shows all the available GeoNode themes, if any, and allows you to create new ones. + +::: warning +::: title +Warning +::: + +Only one theme at a time can be **activated** (aka *enabled*). By disabling or deleting all the available themes, GeoNode will turn the gui back to the default one. +::: + +Editing or creating a new Theme, will actually allow you to customize several properties. + +At least you\'ll need to provide a `Name` for the Theme. Optionally you can specify also a `Description`, which will allow you to better +identify the type of Theme you created. + +
+img/theming/theme-def-0001.png +
Theme Name and Description
+
+ +Just below the `Description` field, you will find the `Enabled` checkbox, allowing you to toggle the Theme. + +
+img/theming/theme-def-0002.png +
Theme Name and Description
+
+ +## Jumbotron and Get Started link + +::: note +::: title +Note +::: + +Remember, everytime you want to apply some changes to the Theme, you **must** save the Theme and reload the GeoNode browser tab. +In order to quickly switch back to the Home page, you can just click the `VIEW SITE` link on the top-right corner of the Admin dashboard. + +![](img/theming/view_site.png){.align-center} +::: + +The next section, allows you to define the first important Theme properties. This part involves the GeoNode main page sections. + +
+img/theming/theme_properties.png +
Jumbotron and Logo options
+
+ +By changing those properties as shown above, you will easily change your default home page from this + +
+img/theming/default_theme.png +
GeoNode Default Home
+
+ +to this + +
+img/theming/first_customization.png +
Updating Jumbotron and Logo
+
+ +It is possible to optionally **hide** the `Jumbotron text`. + +![](img/theming/hide_jumbotron.png){.align-center} + +
+img/theming/hide_jumbotron_view.png +
Hide Jumbotron text
+
+ +## Slide show + +To switch between a slide show and a jumbotron, flip the value of the welcome theme from \"slide show\" to \"jumbotron\" and vice versa to either display a jumbotron with content or a slide show in the home page + +For example, to display a slide show, change the welcome theme from jumbotron background + +![](img/theming/theme-def-0007a.png){.align-center} + +to slide show + +![](img/theming/theme-def-0007b.png){.align-center} + +Before creating a slide show, make sure you have slides to select from (in the multi-select widget) to make up the slide show. + +![](img/theming/theme-def-0007c.png){.align-center} + +If no slides exist, click the plus (+) button beside the slide show multi-select widget to add a new slide. + +![](img/theming/theme-def-0007d.png){.align-center} + +Fill in the slide name, slide content using markdown formatting, and upload a slide image (the image that will be displayed when the slide is in view). + +![](img/theming/theme-def-0007e.png){.align-center} + +For slide images that already contain text, hide slide content by checking the checkbox labeled \"Hide text in the jumbotron slide\" as shown below, then save the slide. + +![](img/theming/theme-def-0007f.png){.align-center} + +It is also possible to hide a slide from all slide show themes that use it by unchecking the checkbox labeled \"Is enabled\" as shown below. + +![](img/theming/theme-def-0007g.png){.align-center} + +Selecting the above slide in a slide show and enabling slide show (using the \"welcome theme\" configuration) will create a slide show with a slide as shown below: + +![](img/theming/theme-def-0007h.png){.align-center} + +## Switching between different themes + +In the case you have defined more Themes, switching between them is as easy as `enabling` one and `disabling` the others. + +Remember to save the Themes everytime and refresh the GeoNode home page on the browser to see the changes. + +It is also important that there is **only one** Theme enabled **at a time**. + +In order to go back to the standard GeoNode behavior, just disable or delete all the available Themes. + +# Add a new user + +In GeoNode, administrators can manage other users. For example, they can *Add New Users* through the following form. + +
+img/add_user_form.png +
Adding New Users
+
+ +The form above can be reached from the *Admin Panel* at the following path: *Home \> People \> Users*. Click on `+ Add user`{.interpreted-text role="guilabel"} to open the form page. + +
+img/add_user_button.png +
The Add User button in the Users List page
+
+ +It is also available, in the GeoNode UI, the `Add User`{.interpreted-text role="guilabel"} link of the *About* menu in the navigation bar. + +
+img/add_user_link.png +
Add User Link
+
+ +To perform the user creation fill out the required fields (*username* and *password*) and click on `Save`{.interpreted-text role="guilabel"}. +You will be redirected to the *User Details Page* which allows to insert further information about the user. + +
+img/user_details_admin_page.png +
The User Details Page
+
+ +The user will be visible into the *Users List Page* of the *Admin Panel* and in the *People Page* (see `user-info`{.interpreted-text role="ref"}). + +
+img/new_user_in_people.png +
The User in the People page
+
+ +# Activate/Disable a User + +When created, new users are *active* by default. +You can check that in the *User Details Page* from the *Admin Panel* (see the picture below). + +
+img/new_user_active.png +
New Users Active by default
+
+ +| *Active* users can interact with other users and groups, can manage resources and, more in general, can take actions on the GeoNode platform. +| Untick the *Active* checkbox to disable the user. It will be not considered as user by the GeoNode system. + +
+img/new_user_disabled.png +
Disabled Users
+
+ +# Change a User password + +GeoNode administrators can also change/reset the password for those users who forget it. +As shown in the picture below, click on `this form` link from the *User Details Page* to access the *Change Password Form*. + +
+img/change_user_password_link.png +
Changing Users Passwords
+
+ +The *Change User Password Form* should looks like the following one. +Insert the new password two times and click on `CHANGE PASSWORD`{.interpreted-text role="guilabel"}. + +
+img/change_user_password_form.png +
Changing Users Passwords
+
+ +# Promoting a User to Staff member or superuser + +Active users have not access to admin tools. +GeoNode makes available those tools only to *Staff Members* who have the needed permissions. +*Superusers* are staff members with full access to admin tools (all permissions are assigned to them). + +Administrators can promote a user to *Staff Member* by ticking the **Staff status** checkbox in the *User Details Page*. +To make some user a *Superuser*, the **Superuser status** checkbox should be ticked. See the picture below. + +
+img/staff_and_superuser_permissions.png +
Staff and Superuser permissions
+
+ +# Creating a Group + +| In GeoNode is possible to create new groups with set of permissions which will be inherited by all the group members. +| The creation of a Group can be done both on the GeoNode UI and on the *Admin Panel*, we will explain how in this paragraph. + +The `Create Groups`{.interpreted-text role="guilabel"} link of *About* menu in the navigation bar allows administrators to reach the *Group Creation Page*. + +
+img/create_group_page_link.png +
The Create Group Link
+
+ +The following form will open. + +
+img/group_creation_form.png +
The Group Creation Form
+
+ +Fill out all the required fields and click `Create`{.interpreted-text role="guilabel"} to create the group. +The *Group Details Page* will open. + +
+img/group_details_page.png +
The Group Details Page
+
+ +The new created group will be searchable in the *Groups List Page*. + +
+img/groups_list_page.png +
The Groups List Page
+
+ +::: note +::: title +Note +::: + +The `Create a New Group`{.interpreted-text role="guilabel"} button on the *Groups List Page* allows to reach the *Group Creation Form*. +::: + +| As already mentioned above, groups can also be created from the Django-based *Admin Interface* of GeoNode. +| The *Groups* link of the *AUTHENTICATION AND AUTHORIZATION* section allows to manage basic Django groups which only care about permissions. +| To create a GeoNode group you should take a look at the *GROUPS* section. + +
+img/groups_admin_section.png +
The Groups Section on the Admin Panel
+
+ +As you can see, GeoNode provides two types of groups. You will learn more about that in the next paragraph. + +## Types of Groups + +In GeoNode users can be grouped through a *Group Profile*, an enhanced Django group which can be enriched with some further information such as a description, a logo, an email address, some keywords, etc. +It also possible to define some *Group Categories* based on which those group profiles can be divided and filtered. + +A new **Group Profile** can be created as follow: + +- click on the *Group Profile* `+ Add`{.interpreted-text role="guilabel"} button + +- fill out all the required fields (see the picture below), *Group Profiles* can be explicitly related to group categories + +
+ img/new_group_profile_form.png +
A new Group Profile
+
+ +- click on `Save`{.interpreted-text role="guilabel"} to perform the creation, the new created group profile will be visible in the *Group Profiles List* + +
+ img/group_profiles_list.png +
The Group Profiles List
+
+ +## Group Categories + +*Group Profiles* can also be related to *Group Categories* which represents common topics between groups. +In order to add a new **Group Category** follow these steps: + +- click on the *Group Categories* `+ Add group category`{.interpreted-text role="guilabel"} button + +- fill out the creation form (type *name* and *description*) + +
+ img/new_group_category_form.png +
A new Group Category
+
+ +- click on `Save`{.interpreted-text role="guilabel"} to perform the creation, the new created category will be visible in the *Group Categories List* + +
+ img/group_categories_list.png +
The Group Categories List
+
+ +# Managing a Group + +Through the `Groups`{.interpreted-text role="guilabel"} link of *About* menu in the navigation bar, administrators can reach the *Groups List Page*. + +
+img/groups_link.png +
The Groups Link in the navigation bar
+
+ +In that page all the GeoNode *Group Profiles* are listed. + +
+img/group_profiles_list_page.png +
Group Profiles List Page
+
+ +For each group some summary information (such as the *title*, the *description*, the number of *members* and *managers*) are displayed near the *Group Logo*. + +Administrators can manage a group from the *Group Profile Details Page* which is reachable by clicking on the *title* of the group. + +
+img/group_profile_details_page.png +
Group Profile Details Page
+
+ +As shown in the picture above, all information about the group are available on that page: + +- the group *Title*; +- the *Last Editing Date* which shows a timestamp corresponding to the last editing of the group properties; +- the *Keywords* associated with the group; +- *Permissions* on the group (Public, Public(invite-only), Private); +- *Members* who join the group; +- *Managers* who manage the group. + +There are also four links: + +- The `Edit Group Details`{.interpreted-text role="guilabel"} link opens the *Group Profile Form* through which the following properties can be changed: + + - *Title*. + - *Logo* (see next paragraphs). + - *Description*. + - *Email*, to contact one or all group members. + - *Keywords*, a comma-separated list of keywords. + - *Access*, which regulates permissions: + - *Public*: any registered user can view and join a public group. + - *Public (invite-only)*: only invited users can join, any registered user can view the group. + - *Private*: only invited users can join the group, registered users cannot see any details about the group, including membership. + - *Categories*, the group categories the group belongs to. + +- `Manage Group Members`{.interpreted-text role="guilabel"} (see next paragraphs). + +- the `Delete this Group`{.interpreted-text role="guilabel"}, click on it to delete the Group Profile. GeoNode requires you to confirm this action. + +
+ img/confirm_group_deletion.png +
Confirm Group Deletion
+
+ +- the `Group Activities`{.interpreted-text role="guilabel"} drives you to the *Group Activities Page* where you can see all datasets, maps and documents associated with the group. There is also a *Comments* tab which shows comments on those resources. + +
+ img/group_activities.png +
Group Activities
+
+ +## Group Logo + +Each group represents something in common between its members. +So each group should have a *Logo* which graphically represents the idea that identify the group. + +On the *Group Profile Form* page you can insert a logo from your disk by click on `Browse...`{.interpreted-text role="guilabel"}. + +
+img/editing_group_logo.png +
Editing the Group Logo
+
+ +| Click on `Update`{.interpreted-text role="guilabel"} to apply the changes. +| Take a look at your group now, you should be able to see that logo. + +
+img/group_logo.png +
The Group Logo
+
+ +## Managing Group members + +The `Manage Group Members`{.interpreted-text role="guilabel"} link opens the *Group Members Page* which shows *Group Members* and *Group Managers*. +**Managers** can edit group details, can delete the group, can see the group activities and can manage memberships. +Other **Members** can only see the group activities. + +| In Public Groups, users can join the group without any approval. Other types of groups require the user to be invited by the group managers. +| Only group managers can *Add new members*. In the picture below, you can see the manager can search for users by typing their names into the *User Identifiers* search bar. Once found, he can add them to the group by clicking the `Add Group Members`{.interpreted-text role="guilabel"} button. The *Assign manager role* flag implies that all the users found will become managers of the group. + +
+img/add_new_member.png +
Adding a new Member to the Group
+
+ +The following picture shows you the results. + +
+img/new_members.png +
New Members of the Group
+
+ +If you want to change the role of group members after adding them, you can use the \"promote\" button to make a member into a manager, and the \"demote\" button to make a manager into a regular member. + +# Group based advanced data workflow + +By default GeoNode is configured to make every resource suddenly available to everyone, i.e. publicly accessible +even from anonymous/non-logged in users. + +It is actually possible to change few configuration settings in order to allow GeoNode to enable an advanced publication workflow. + +With the advanced workflow enabled, your resources won\'t be automatically published (i.e. made visible and accessible for all, contributors or simple users). + +For now, your item is only visible by yourself, the manager of the group to which the resource is linked (this information is filled in the metadata), the members of this group, and the GeoNode Administrators. + +Before being published, the resource will follow a two-stage review process, which is described below: + +
+img/adv_data_workflow/adv_data_workflow_001.jpg +
From upload to publication: the review process on GeoNode
+
+ +## How to enable the advanced workflow + +You have to tweak the GeoNode settings accordingly. + +Please see the details of the following GeoNode `Settings`: + +- [RESOURCE_PUBLISHING](../../basic/settings/index.html#resource-publishing) +- [ADMIN_MODERATE_UPLOADS](../../basic/settings/index.html#admin-moderate-uploads) +- [GROUP_PRIVATE_RESOURCES](../../basic/settings/index.html#group-private-resources) + +Summarizing, when all the options above of the Advanced Workflow are enabled, upon a new upload we will have: + +> - The **\"unpublished\"** resources will be **hidden** to **anonymous users only**. The **registered users** will be still able to access the resources (if they have the rights to do that, of course). +> - The **\"unpublished\"** resources will remain hidden to users if the permission (see *Admin Guide section: \'Manage Permissions\'*) will be explicitly removed +> - During the upload, whenever the advanced workflow is enabled, the **owner\'s Groups** are automatically allowed to access the resource, even if the **\"anonymous\"** flag has been disabled. Those permissions can be removed later on +> - During the upload, **\"managers\"** of the owner\'s Groups associated to the resource, are always allowed to edit the resource, the same as they are admin for that resource +> - **\"managers\"** of the owner\'s Groups associated to the resource are allowed to **\"publish\"** also the resources, not only to **\"approve\"** them + +## Change the owner rights in case of advanced workflow is on + +After switching `ADMIN_MODERATE_UPLOADS` to True and resource is approved owner is no longer able +to modify it. He will see new button on the resource detail page: Request change. After clicking this, view with short +form is shown. +On this view user can write short message why he want to modify the resource. + +This message will be sent through messaging and email system to administrators: + +After administrator unapprove the resource owner is again able to modify it. + +## The group Manager approval + +Here, the role of the Manager of the group to which your dataset, document or map is linked is to check that the uploaded item is correct. +Particularly, in the case of a dataset or a map, it consists of checking that the chosen cartographic representation and the style are +fitting but also that the discretization is appropriate. + +The Manager must also check that the metadata are properly completed and that the mandatory information +(Title, Abstract, Edition, Keywords, Category, Group, Region) are filled. + +If needed, the Manager can contact the contributor responsible of the dataset, document or map in order to report potential comments or +request clarifications. + +Members of the group can also take part in the reviewing process and give some potential inputs to the responsible of the +dataset, document or map. + +When the Manager considers that the resource is ready to be published, he should approve it. +To do so, the Manager goes to the resource detail page, then opens the `Edit Metadata`{.interpreted-text role="guilabel"}. +In the `Settings`{.interpreted-text role="guilabel"} tab, the manager checks the `Approved`{.interpreted-text role="guilabel"} box, and then updates the metadata and saves the changes: + +Following this approval, the GeoNode Administrators receive a notification informing them that an item is now waiting for publication + +## The publication by the GeoNode Administrator + +Prior to the public release of an approved resource, the Administrator of the platform performs a final validation of +the item and its metadata, notably to check that it is in line with license policies. + +If needed, the GeoNode Administrator can contact the Manager who has approved the resource, as well as its responsible. + +Once the resource is validated, the item is made public by the Administrator. +It can now be viewed, accessed, and downloaded in accordance with the `Permissions` set by the responsible contributor. + +## Promotion, Demotion and Removal of Group Members + +If the owner is a group Manager, They have permissions to edit, approve, and publish the resource. + +When a group member is promoted to a manager role, they gain permissions to edit, approve and publish the resource. + +When a group manager is demoted to a member role, they lose edit permissions of the resource and only remain with view and download permissions. + +When a member is removed from the group, they can nolonger access the unpublished resource anymore. + +# Manage profiles using the admin panel + +So far GeoNode implements two distinct roles, that can be assigned to resources such as datasets, maps or documents: + +- party who authored the resource +- party who can be contacted for acquiring knowledge about or acquisition of the resource + +These two profiles can be set in the GeoNode interface by accessing the metadata page and setting the `Point of Contact` and `Metadata Author` fields respectively. + +Is possible for an administrator to add new roles if needed, by clicking on the `Add contact role`{.interpreted-text role="guilabel"} button in the `Base -> Contact Roles`{.interpreted-text role="guilabel"} section: + +![](img/admin-roles-add.png){.align-center} + +Clicking on the `People`{.interpreted-text role="guilabel"} section (see figure) will open a web for with some personal information plus a section called `Users`{.interpreted-text role="guilabel"}. + +![](img/admin-people.png) + +Is important that this last section is not modified here unless the administrator is very confident in that operation. + +![](img/admin-profiles-contactroles.png){.align-center} + +Manage datasets using the admin panel +=================================== + +Some of the datasets information can be edited directly through the admin interface although the best place is in the `Dataset -> Metadata Edit`{.interpreted-text role="guilabel"} in GeoNode. + +Clicking on the `Admin > Dataset > Datasets`{.interpreted-text role="guilabel"} link will show the list of available datasets. + +![](img/admin-datasets.png){.align-center} + +::: warning +::: title +Warning +::: + +It is not recommended to modify the Datasets\' `Attributes` or `Styles` directly from the Admin dashboard unless you are aware of your actions. +::: + +The `Metadata` information can be changed for multiple datasets at once through the `Metadata batch edit`{.interpreted-text role="guilabel"} action. Select the datasets you want to edit in the batch and at the bottom, enter the [Metadata batch edit]{.title-ref} action then click `Go`{.interpreted-text role="guilabel"}. + +![](img/admin-datasets-batch.png){.align-center} + +This will open a form with the information you can edit in a batch. see picture below. + +![](img/admin-datasets-batch-form.png){.align-center} + +By clicking over one Dataset link, it will show a detail page allowing you to modify some of the resource info like the metadata, the keywords, the title, etc. + +::: note +::: title +Note +::: + +It is strongly recommended to always use the GeoNode resource `Edit Metadata`{.interpreted-text role="guilabel"} or `Advanced Metadata`{.interpreted-text role="guilabel"} tools in order to edit the metadata info. +::: + +The `Permissions` can be changed also for multiple Datasets at once through the `Set permissions`{.interpreted-text role="guilabel"} action. + +![](img/set_datasets_permissions_action.png){.align-center} + +By clicking over one Dataset link, it will show a detail page allowing you to modify the permissions for the selected resources. + +# Manage the maps using the admin panel + +Similarly to the Datasets, it is possible to manage the available GeoNode Maps through the Admin panel also. + +Move to `Admin > Maps`{.interpreted-text role="guilabel"} to access the Maps list. + +![](img/admin-maps.png){.align-center} + +Notice that by enabling the `Featured` option here, will allow GeoNode to show the Map thumbnail and the Map detail link at the top under featured resources on the `Home Page`{.interpreted-text role="guilabel"} + +![](img/admin-maps-featured-001.png){.align-center} + +![](img/admin-maps-featured-002.png){.align-center} + +# Manage the documents using the admin panel + +Similarly to the Datasets and Maps, it is possible to manage the available GeoNode Documents through the Admin panel also. + +Move to `Admin > Documents`{.interpreted-text role="guilabel"} to access the Documents list. + +![](img/admin-documents.png){.align-center} + +By clicking over one Document link, it will show a detail page allowing you to modify some of the resource info like the metadata, the keywords, the title, etc. + +# Manage the base metadata choices using the admin panel + +`Admin > Base`{.interpreted-text role="guilabel"} contains almost all the objects you need to populate the resources metadata choices. + +
+img/metadata-base/admin-panel-metadata-contents-0001.png +
Admin dashboard Base Panel
+
+ +In other words the options available from the `select-boxes`{.interpreted-text role="guilabel"} of the resource `Edit Metadata`{.interpreted-text role="guilabel"} and `Advanced Metadata`{.interpreted-text role="guilabel"} forms. + +
+img/metadata-base/admin-panel-metadata-contents-0002.png +
Metadata Form
+
+ +
+img/metadata-base/admin-panel-metadata-contents-0003.png +
Advanced Metadata Form
+
+ +::: note +::: title +Note +::: + +When editing the resource metadata through the `Edit Metadata`{.interpreted-text role="guilabel"}, some fields are marked as `mandatory` and by filling those information +the `Completeness` progress will advance accordingly. + +
+img/metadata-base/admin-panel-metadata-contents-0003a.png +
Metadata Completeness
+
+ +Even if not all the fields have been filled, the system won\'t prevent you to update the metadata; this is why the `Mandatory` fields are +mandatory to be fully compliant with an `ISO 19115` metadata schema, but are only recommended to be compliant with GeoNode. + +Also the `Completeness` indicates how far the metadata is to be compliant with an `ISO 19115` metadata schema. + +Of course, it is **highly** recommended to always fill as much as possible at least all the metadata fields marked as `Mandatory`. + +This will improve not only the quality of the data stored into the system, but will help the users to easily search for them on GeoNode. + +All the `Search & Filter` panels and options of GeoNode are, in fact, based on the resources metadata fields. Too much generic descriptions and +too empty metadata fields, will give highly un-precise and very wide search results to the users. +::: + +## Hierarchical keywords + +Through the `Admin > Base > Hierarchical keywords`{.interpreted-text role="guilabel"} panel it will be possible to manage all the keywords associated to the resources. + +
+img/metadata-base/admin-panel-metadata-contents-0004.png +
Hierarchical keywords list
+
+ +
+img/metadata-base/admin-panel-metadata-contents-0005.png +
Hierarchical keywords edit
+
+ +- The `Name`{.interpreted-text role="guilabel"} is the human readable text of the keyword, what users will see. +- The `Slug`{.interpreted-text role="guilabel"} is a unique label used by the system to identify the keyword; most of the times it is equal to the name. + +Notice that through the `Position`{.interpreted-text role="guilabel"} and `Relative to`{.interpreted-text role="guilabel"} selectors, it is possible to establish a hierarchy between the available keywords. +The hierarchy will be reflected in the form of a tree from the metadata panels. + +By default each user with editing metadata rights on any resource, will be able to insert new keywords into the system by simply typing a free text on the keywords metadata field. + +It is possible to force the user to select from a fixed list of keywords through the [FREETEXT_KEYWORDS_READONLY](../../basic/settings/index.html#freetext-keywords-readonly) setting. + +When set to [True]{.title-ref} keywords won\'t be writable from users anymore. Only admins can will be able to manage them through the `Admin > Base > Hierarchical keywords`{.interpreted-text role="guilabel"} panel. + +## Licenses + +Through the `Admin > Base > Licenses`{.interpreted-text role="guilabel"} panel it will be possible to manage all the licenses associated to the resources. + +
+img/metadata-base/admin-panel-metadata-contents-0006.png +
Metadata editor Licenses
+
+ +::: warning +::: title +Warning +::: + +It is **strongly** recommended to not publish resources without an appropriate license. Always make sure the data provider specifies the correct license and that all the restrictions have been honored. +::: + +## Metadata Regions + +Through the `Admin > Base > Metadata Regions`{.interpreted-text role="guilabel"} panel it will be possible to manage all the admin areas associated to the resources. + +
+img/metadata-base/admin-panel-metadata-contents-0009.png +
Resource Metadata Regions
+
+ +Notice that those regions are used by GeoNode to filter search results also through the resource list view. + +::: note +::: title +Note +::: + +GeoNode tries to guess the `Regions` intersecting the data bounding boxes when uploading a new dataset. Those should be refined by the user dataset on anyway. +::: + +## Metadata Restriction Code Types and Spatial Representation Types + +Through the `Admin > Base > Metadata Restriction Code Types`{.interpreted-text role="guilabel"} and `Admin > Base > Metadata Spatial Representation Types`{.interpreted-text role="guilabel"} panels, it will +be possible to **update only** the metadata descriptions for restrictions and spatial representation types. + +Such lists are *read-only* by default since they have been associated to the specific codes of the `ISO 19115` metadata schema. +Changing them would require the system to provide a custom dictionary through the metadata catalog too. Such functionality is not supported actually by GeoNode. + +## Metadata Topic Categories + +Through the `Admin > Base > Metadata Topic Categories`{.interpreted-text role="guilabel"} panel it will be possible to manage all the resource metadata categories avaialble into the system. + +Notice that by default, GeoNode provides the standard topic categories available with the `ISO 19115` metadata schema. Changing them means that the system won\'t be +compliant with the standard `ISO 19115` metadata schema anymore. `ISO 19115` metadata schema extensions are not currently supported natively by GeoNode. + +It is worth notice that GeoNode allows you to associate [Font Awesome Icons](https://fontawesome.com/icons?d=gallery) to each topic category through their `fa-icon` code. +Those icons will be used by GeoNode to represent the topic category on both the `Search & Filter` menus and `Metadata`{.interpreted-text role="guilabel"} panels. + +::: warning +::: title +Warning +::: + +The list of the `Metadata Topic Categories` on the home page is currently fixed. To change it you will need to update or override the GeoNode `index.html` HTML template. +::: + +By default the `Metadata Topic Categories` are *writable*. Meaning that they can be removed or created by the `Admin`{.interpreted-text role="guilabel"} panel. + +It is possible to make them fixed (it will be possible to update their descriptions and icons only) through the [MODIFY_TOPICCATEGORY](../../basic/settings/index.html#modify-topiccategory) setting. + +# Announcements + +As an Administrator you might need to broadcast announcements to the world about your portal or simply to the internal contributors. + +GeoNode `Announcements` allow actually to do that; an admin has the possibility to create three types of messages, accordingly to their severity, +decide their validity in terms of time period (start date and expiring date of the announcement), who can view them or not (everyone or just the +registerd members) and whenever a user can hide the message or not and how long. + +There are three types of announcements accordingly to their severity level: `General`, `Warning` and `Critical` +The difference is mainly the color of the announcement box. + +Only administrators and staff members can create and manage announcements. + +Currently there two ways to access and manage the announcements list: + +1. Via the GeoNode interface, from the `Profile`{.interpreted-text role="guilabel"} panel + + > ::: note + > ::: title + > Note + > ::: + > + > Those are accessible by both admins and staff members. + > ::: + > + >
+ > img/announcments/admin-announcments-005.png + >
Announcements from the Profile panel
+ >
+ +2. Via the GeoNode `Admin`{.interpreted-text role="guilabel"} panel + + > ::: note + > ::: title + > Note + > ::: + > + > Those are accessible by admins only. + > ::: + > + >
+ > img/announcments/admin-announcments-006.png + >
Announcements from the Admin panel
+ >
+ +The functionalities are almost the same for both the interfaces, except that from the `Admin`{.interpreted-text role="guilabel"} panel it is possible to manage the +dismissals too. + +`Dismissals` are basically records of members that have read the announcement and closed the message box. An announcement can have one +`dismissal type` among the three below: + +1. `No Dismissal Allowed`{.interpreted-text role="guilabel"} it won\'t be possible to close the announcement\'s message box at all. +2. `Session Only Dismissal`{.interpreted-text role="guilabel"} (\*) the default one, it will be possible to close the announcement\'s message box for the current browser session. It will show up again at next access. +3. `Permanent Dismissal Allowed`{.interpreted-text role="guilabel"} once the announcement\'s message box is closed, it won\'t appear again for the current member. + +## How to create and manage Announcements + +From the `Profile`{.interpreted-text role="guilabel"} panel, click on `Announcements` link + +
+img/announcments/admin-announcments-007.png +
Announcements List from the Profile panel
+
+ +Click either on `New Announcement`{.interpreted-text role="guilabel"} to create a new one or over a title of an existing one to manage its contents. + +Create a new announcement is quite straight; you have to fill the fields provided by the form. + +::: warning +::: title +Warning +::: + +In order to be visible, you will need to check the `Site wide`{.interpreted-text role="guilabel"} option **in any case**. You might want to hide the message to *anonymous* users by enabling the `Members only`{.interpreted-text role="guilabel"} option too. +::: + +
+img/announcments/admin-announcments-008.png +
Create Announcement from the Profile panel
+
+ +Managing announcements form the `Admin`{.interpreted-text role="guilabel"} panel, is basically the same; the fields for the form will be exactly the same. + +
+img/announcments/admin-announcments-009.png +
Create Announcement from the Admin panel
+
+ +Accessing announcements options from the `Admin`{.interpreted-text role="guilabel"} panel, allows you to manage dismissals also. +Through this interface you will be able to selectively decide members which can or cannot view a specific announcement, or force them to visualize the messages again by deleting the dismissals accordingly. + +
+img/announcments/admin-announcments-010.png +
Create Dismissal from the Admin panel
+
+ +# Menus, Items and Placeholders + +GeoNode provides some integrated functionalities allowing you to quickly and easily customize the top-bar menu (see the example below). + +
+img/admin-panel-menus-0000.png +
GeoNode Top-Bar Menu customization
+
+ +With minor changes of the `basic.html` template, potentially, it could be possible to use the same approach for a more complex customization. +Let\'s start with the simple one. + +By default GeoNode provides custom `placeholders` already defined into the `basic.html` template, called `CARDS_MENU`, `TOPBAR_MENU_RIGHT`, `TOPBAR_MENU_LEFT`, `TOPBAR_MENU`. + +From the `Admin > Base`{.interpreted-text role="guilabel"} panel, it is possible to access to the `Menu`, `Menu Items` and `Menu Placeholder` options. + +
+img/admin-panel-menus-0001.png +
Menu, Menu Items and Menu Placeholder options on the Admin panel
+
+ +The hierarchical structure of a custom `Menu` is the following one: + +1. + + `Menu Placeholder`; first of all you need to define a *placeholder* both into the `Admin > Base`{.interpreted-text role="guilabel"} panel and the `basic.html` template, using the same **keyword**. + + : By default GeoNode provides already defined menus. + +
+ img/admin-panel-menus-0002.png +
The default ``TOPBAR_MENU`` Menu Placeholder on the Admin panel
+
+ +2. + + `Menu`; second thing to do is to create a new *menu* associated to the corresponding *placeholder*. + + : This is still possible from the `Admin > Base`{.interpreted-text role="guilabel"} panel + +
+ img/admin-panel-menus-0003.png +
Create a new Menu from the Admin panel
+
+ + You will need to provide: + + - A `Title`, representing the name of the `Menu` visible by the users + + ::: warning + ::: title + Warning + ::: + + By using this approach, internationalization won\'t be supported. For the time being GeoNode does not support this for menus created from the `Admin > Base`{.interpreted-text role="guilabel"} panel. + ::: + + - A `Menu Placeholder` from the existing ones. + + - A `Order` in the case you\'ll create more menus associated to the same placeholder. + +3. `Menu Item`; finally you will need to create voices belonging to the *menu*. For the time being, GeoNode allows you to create only `href` links. + + >
+ > img/admin-panel-menus-0004.png + >
Create a new Menu Item from the Admin panel
+ >
+ > + > ::: warning + > ::: title + > Warning + > ::: + > + > The `Menu` won\'t be visible until you add more than one `Menu Item`, If you have 1, item, the item will be showed(but not under the menu). + > ::: + +# OAuth2 Access Tokens {#oauth2_admin_panel_access_tokens} + +This small section won\'t cover entirely the GeoNode OAuth2 security integration, this is explained in detail in other sections of the documentation +(refer to `oauth2_fixtures_and_migration`{.interpreted-text role="ref"} and `oauth2_tokens_and_sessions`{.interpreted-text role="ref"}). + +Here we will focus mainly on the `Admin > DJANGO/GEONODE OAUTH TOOLKIT`{.interpreted-text role="guilabel"} panel items with a specific attention to the `Access tokens` management. + +The `Admin > DJANGO/GEONODE OAUTH TOOLKIT`{.interpreted-text role="guilabel"} panel (as shown in the figure below) allows an admin to manage everything related to +GeoNode OAuth2 grants and permissions. + +As better explained in other sections of the documentation, this is needed to correctly handle the communication between GeoNode and GeoServer. + +
+img/oauth2-tokens/admin-panel-tokens-0001.png +
DJANGO/GEONODE OAUTH TOOLKIT Admin panel
+
+ +Specifically from this panel an admin can create, delete or extend OAuth2 `Access tokens`. + +The section `oauth2_tokens_and_sessions`{.interpreted-text role="ref"} better explains the concepts behind OAuth2 sessions; we want just to refresh the mind here +about the basic concepts: + +- If the [SESSION_EXPIRED_CONTROL_ENABLED](../../basic/settings/index.html#session-expired-control-enabled) setting is set to [True]{.title-ref} (by default it is set to [True]{.title-ref}) + a registered user cannot login to neither GeoNode nor GeoServer without a valid `Access token`. +- When logging-in into GeoNode through the sign-up form, GeoNode checks if a valid `Access token` exists and it creates a new one if not, or extends + the existing one if expired. +- New `Access tokens` expire automatically after [ACCESS_TOKEN_EXPIRE_SECONDS](../../basic/settings/index.html#access-token-expire-seconds) setting (by default 86400) +- When an `Access token` expires, the user will be kicked out from the session and forced to login again + +## Create a new token or extend an existing one + +It is possible from the `Admin > DJANGO/GEONODE OAUTH TOOLKIT`{.interpreted-text role="guilabel"} panel to create a new `Access token` for a user. + +In order to do that, just click on the `Add`{.interpreted-text role="guilabel"} button beside `Access tokens` topic + +
+img/oauth2-tokens/admin-panel-tokens-0002.png +
Add a new ``Access token``
+
+ +On the new form + +
+img/oauth2-tokens/admin-panel-tokens-0003.png +
Create an ``Access token``
+
+ +select the followings: + +1. `User`; use the search tool in order to select the correct user. The form want the user PK, which is a number, and **not** the username. + The search tool will do everything for you. + + >
+ > img/oauth2-tokens/admin-panel-tokens-0003a.png + >
Select a User
+ >
+ +2. `Source refresh token`; this is not mandatory, leave it blank. + +3. `Token`; write here any alphanumeric string. This will be the `access_token` that the member can use to access the OWS services. + We suggest to use a service like in order to generate a strong token string. + + >
+ > img/oauth2-tokens/admin-panel-tokens-0003b.png + >
Select a Token
+ >
+ +4. `Application`; select **GeoServer**, this is mandatory + + >
+ > img/oauth2-tokens/admin-panel-tokens-0003c.png + >
Select the GeoServer Application
+ >
+ +5. `Expires`; select an expiration date by using the `date-time`{.interpreted-text role="guilabel"} widgets. + + >
+ > img/oauth2-tokens/admin-panel-tokens-0003d.png + >
Select the Token Expiration
+ >
+ +6. `Scope`; select **write**, this is mandatory. + + >
+ > img/oauth2-tokens/admin-panel-tokens-0003e.png + >
Select the Application Scope
+ >
+ +Do not forget to `Save`{.interpreted-text role="guilabel"}. + +From now on, GeoNode will use this `Access Token` to control the user session (notice that the user need to login again if closing the browser session), +and the user will be able to access the OWS Services by using the new `Access Token`, e.g.: + +``` shell +https://dev.geonode.geo-solutions.it/geoserver/ows?service=wms&version=1.3.0&request=GetCapabilities&access_token=123456 +``` + +Notice the `...quest=GetCapabilities&access_token=123456` (**access_token**) parameter at the end of the URL. + +## Force a User Session to expire + +Everything said about the creation of a new `Access Token`, applies to the deletion of the latter. + +From the same interface an admin can either select an expiration date or delete all the `Access Tokens` associated to a user, in order to +force its session to expire. + +Remember that the user could activate another session by logging-in again on GeoNode with its credentials. + +In order to be sure the user won\'t force GeoNode to refresh the token, reset first its password or de-activate it. diff --git a/docs/admin/async/index.md b/docs/admin/async/index.md new file mode 100644 index 00000000..b061a35f --- /dev/null +++ b/docs/admin/async/index.md @@ -0,0 +1,75 @@ +# Supervisord and Systemd + +# Celery + +# Rabbitmq and Redis + +# How to: Async Upload via API + +In geonode is possible to upload resources via API in async/sync way. + +Here is available a full example of upload via API + + +## Step 1 + +Create a common client session, this is foundamental due the fact that geonode will check the request session. +For example with requests we will do something like: + +``` python +import requests +client = requests.session() +``` + +Note: in Django this part is already managed + +## Step 2 + +Call the [api/v2/uploads/upload]{.title-ref} endpoint in PUT (is a form-data endpoint) by specifing in files a dictionary with the names and the files that we want to uploads and a data payload with the required informations. +For example: + +``` python +params = { + "permissions": '{ "users": {"AnonymousUser": ["view_resourcebase"]} , "groups":{}}', # layer permissions + "time": "false", + "layer_title": "layer_title", + "time": "false", + "charset": "UTF-8", +} + +files = { + "filename": <_io.BufferedReader name="filename"> +} + +client.put( + "http://localhost:8000/api/v2/uploads/upload/", + auth=HTTPBasicAuth(username, password), + data=params, + files=files, +) + +Returns: +- dict with import id of the resource +``` + +## Step 3 + +Call in the final upload page in order to trigger the actual import. +If correclty set, Geoserver will manage the upload asyncronously. + +``` python +client.get("http://localhost:8000/upload/final?id={import_id}") + +The `import_id` is returned from the previous step +``` + +## Step 4 + +The upload as been completed on GeoNode, we should check utill Geoserver has complete his part. +To do so, is enougth to call the detailed information about the upload that we are performing + +``` python +client.get(f"http://localhost:8000/api/v2/uploads/{upload_id}") +``` + +When the status is [PROCESSED]{.title-ref} and the completion is [100%]{.title-ref} we are able to see the resource in geonode and geoserver diff --git a/docs/admin/default_lang/index.md b/docs/admin/default_lang/index.md new file mode 100644 index 00000000..1c601b7b --- /dev/null +++ b/docs/admin/default_lang/index.md @@ -0,0 +1,78 @@ +# Changing the Default Language {#default_language} + +GeoNode\'s default language is English, but GeoNode users can change the interface language with the pulldown menu at the top-right of most GeoNode pages. Once a user selects a language GeoNode remembers that language for subsequent pages. + +# GeoNode Configuration + +As root edit the geonode config file `/home/geonode/geonode/geonode/settings.py`{.interpreted-text role="file"} (or `/etc/geonode/settings.py`{.interpreted-text role="file"} if GeoNode has been installed using **apt-get**) and change `LANGUAGE_CODE` to the desired default language. + +::: note +::: title +Note +::: + +A list of language codes can be found in the global django config file `/usr/local/lib/python2.7/dist-packages/django/conf/global_settings.py`{.interpreted-text role="file"} (or `/var/lib/geonode/lib/python2.7/site-packages/django/conf/global_settings.py`{.interpreted-text role="file"} if GeoNode has been installed using **apt-get**). +::: + +For example, to make French the default language use: + + LANGUAGE_CODE = 'fr' + +Unfortunately Django overrides this setting, giving the language setting of a user\'s browser priority. For example, if `LANGUAGE_CODE` is set to French, but the user has configured their operating system for Spanish they may see the Spanish version when they first visit GeoNode. + +# Additional Steps + +If this is not the desired behaviour, and all users should initially see the default `LANGUAGE_CODE`, regardless of their browser\'s settings, do the following steps to ensure Django ignores the browser language settings. (Users can always use the pulldown language menu to change the language at any time.) + +As **root** create a new directory within GeoNode\'s site packages + +``` shell +mkdir /usr/lib/python2.7/dist-packages/setmydefaultlanguage +``` + +or + +``` shell +mkdir /var/lib/geonode/lib/python2.7/site-packages/setmydefaultlanguage +``` + +if GeoNode has been installed using **apt-get**. + +As root create and edit a new file `/usr/lib/python2.7/dist-packages/setmydefaultlanguage/__init__.py`{.interpreted-text role="file"} and add the following lines + +``` python +class ForceDefaultLanguageMiddleware(object): + """ + Ignore Accept-Language HTTP headers + + This will force the I18N machinery to always choose settings.LANGUAGE_CODE + as the default initial language, unless another one is set via sessions or cookies + + Should be installed *before* any middleware that checks request.META['HTTP_ACCEPT_LANGUAGE'], + namely django.middleware.locale.LocaleMiddleware + """ + def process_request(self, request): + if request.META.has_key('HTTP_ACCEPT_LANGUAGE'): + del request.META['HTTP_ACCEPT_LANGUAGE'] +``` + +At the end of the GeoNode configuration file `/home/geonode/geonode/geonode/settings.py`{.interpreted-text role="file"} (or `/etc/geonode/settings.py`{.interpreted-text role="file"} if GeoNode has been installed using **apt-get**) add the following lines to ensure the above class is executed + +``` python +MIDDLEWARE_CLASSES += ( + 'setmydefaultlanguage.ForceDefaultLanguageMiddleware', +) +``` + +# Restart + +You will need to restart GeoNode accordingly to the installation method you have choosen. + +As an instance in case you are using [NGINX]{.title-ref} with [UWSGI]{.title-ref}, as root you will need to run the following commands + +``` shell +service uwsgi restart +service nginx restart +``` + +Please refer to Translating GeoNode for information on editing GeoNode pages in different languages and create new GeoNode Translations. diff --git a/docs/admin/index.md b/docs/admin/index.md new file mode 100644 index 00000000..695f61a0 --- /dev/null +++ b/docs/admin/index.md @@ -0,0 +1,79 @@ +# GeoNode Admins Guide + +GeoNode has an administration panel, based on the Django admin, which can be used to do some database operations. +Although most of the operations can and should be done through the normal GeoNode interface, +the admin panel provides a quick overview and management tool over the database. + +The following sections will explain more in depth what functionalities the admin panel makes available to you. +It should be highlighted that the sections not covered in this guide are meant to be managed through GeoNode UI. + +::: {.toctree maxdepth="3"} +admin_panel/index +::: + +# GeoNode Management Commands + +Management commands are utility functions for GeoNode maintenance tasks. They are usually run from an SSH/bash shell on the server running GeoNode. +Any call to python is prepended with a configuration parameter to indicate the GeoNode settings module to be used. + +``` shell +DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --help +``` + +::: note +::: title +Note +::: + +If you have enabled `local_settings.py` the command will change as follows: + +``` shell +DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py migrate_baseurl --help +``` +::: + +::: {.toctree caption="GeoNode Management Commands" maxdepth="3"} +mgmt_commands/index +::: + +# Changing the default Languages + +::: {.toctree caption="Changing the default Languages" maxdepth="3"} +default_lang/index +::: + +# GeoNode Upgrade from older versions + +::: {.toctree caption="GeoNode Upgrade from older versions" maxdepth="3"} +upgrade/index +::: + +# GeoNode Async Signals + +::: {.toctree caption="GeoNode Async Signals" maxdepth="3"} +async/index +::: + +# Managing thesauri + +::: {.toctree maxdepth="3"} +thesaurus/index +::: + + +- [index](admin_panel/index.md) + +### GeoNode Management Commands +- [index](mgmt_commands/index.md) + +### Changing the default Languages +- [index](default_lang/index.md) + +### GeoNode Upgrade from older versions +- [index](upgrade/index.md) + +### GeoNode Async Signals +- [index](async/index.md) + +- [index](thesaurus/index.md) + diff --git a/admin/mgmt_commands/data/geonode_mngmt_commands.postman_collection.json b/docs/admin/mgmt_commands/data/geonode_mngmt_commands.postman_collection.json similarity index 100% rename from admin/mgmt_commands/data/geonode_mngmt_commands.postman_collection.json rename to docs/admin/mgmt_commands/data/geonode_mngmt_commands.postman_collection.json diff --git a/admin/mgmt_commands/img/add_test_user.png b/docs/admin/mgmt_commands/img/add_test_user.png similarity index 100% rename from admin/mgmt_commands/img/add_test_user.png rename to docs/admin/mgmt_commands/img/add_test_user.png diff --git a/admin/mgmt_commands/img/add_user.png b/docs/admin/mgmt_commands/img/add_user.png similarity index 100% rename from admin/mgmt_commands/img/add_user.png rename to docs/admin/mgmt_commands/img/add_user.png diff --git a/admin/mgmt_commands/img/admin_interface.png b/docs/admin/mgmt_commands/img/admin_interface.png similarity index 100% rename from admin/mgmt_commands/img/admin_interface.png rename to docs/admin/mgmt_commands/img/admin_interface.png diff --git a/admin/mgmt_commands/img/geoserver_admin.png b/docs/admin/mgmt_commands/img/geoserver_admin.png similarity index 100% rename from admin/mgmt_commands/img/geoserver_admin.png rename to docs/admin/mgmt_commands/img/geoserver_admin.png diff --git a/admin/mgmt_commands/img/geoserver_layer_params.png b/docs/admin/mgmt_commands/img/geoserver_layer_params.png similarity index 100% rename from admin/mgmt_commands/img/geoserver_layer_params.png rename to docs/admin/mgmt_commands/img/geoserver_layer_params.png diff --git a/admin/mgmt_commands/img/geoserver_layers.png b/docs/admin/mgmt_commands/img/geoserver_layers.png similarity index 100% rename from admin/mgmt_commands/img/geoserver_layers.png rename to docs/admin/mgmt_commands/img/geoserver_layers.png diff --git a/admin/mgmt_commands/img/geoserver_new_store.png b/docs/admin/mgmt_commands/img/geoserver_new_store.png similarity index 100% rename from admin/mgmt_commands/img/geoserver_new_store.png rename to docs/admin/mgmt_commands/img/geoserver_new_store.png diff --git a/admin/mgmt_commands/img/geoserver_postgis_params.png b/docs/admin/mgmt_commands/img/geoserver_postgis_params.png similarity index 100% rename from admin/mgmt_commands/img/geoserver_postgis_params.png rename to docs/admin/mgmt_commands/img/geoserver_postgis_params.png diff --git a/admin/mgmt_commands/img/geoserver_publish_layers.png b/docs/admin/mgmt_commands/img/geoserver_publish_layers.png similarity index 100% rename from admin/mgmt_commands/img/geoserver_publish_layers.png rename to docs/admin/mgmt_commands/img/geoserver_publish_layers.png diff --git a/admin/mgmt_commands/img/geoserver_srs.png b/docs/admin/mgmt_commands/img/geoserver_srs.png similarity index 100% rename from admin/mgmt_commands/img/geoserver_srs.png rename to docs/admin/mgmt_commands/img/geoserver_srs.png diff --git a/admin/mgmt_commands/img/geoserver_srs_2.png b/docs/admin/mgmt_commands/img/geoserver_srs_2.png similarity index 100% rename from admin/mgmt_commands/img/geoserver_srs_2.png rename to docs/admin/mgmt_commands/img/geoserver_srs_2.png diff --git a/admin/mgmt_commands/img/layer_batch_perms_admin.png b/docs/admin/mgmt_commands/img/layer_batch_perms_admin.png similarity index 100% rename from admin/mgmt_commands/img/layer_batch_perms_admin.png rename to docs/admin/mgmt_commands/img/layer_batch_perms_admin.png diff --git a/admin/mgmt_commands/img/layer_batch_perms_form.png b/docs/admin/mgmt_commands/img/layer_batch_perms_form.png similarity index 100% rename from admin/mgmt_commands/img/layer_batch_perms_form.png rename to docs/admin/mgmt_commands/img/layer_batch_perms_form.png diff --git a/admin/mgmt_commands/img/layer_info_vector.png b/docs/admin/mgmt_commands/img/layer_info_vector.png similarity index 100% rename from admin/mgmt_commands/img/layer_info_vector.png rename to docs/admin/mgmt_commands/img/layer_info_vector.png diff --git a/admin/mgmt_commands/img/management_commands_over_http_add_job.png b/docs/admin/mgmt_commands/img/management_commands_over_http_add_job.png similarity index 100% rename from admin/mgmt_commands/img/management_commands_over_http_add_job.png rename to docs/admin/mgmt_commands/img/management_commands_over_http_add_job.png diff --git a/admin/mgmt_commands/img/management_commands_over_http_admin.png b/docs/admin/mgmt_commands/img/management_commands_over_http_admin.png similarity index 100% rename from admin/mgmt_commands/img/management_commands_over_http_admin.png rename to docs/admin/mgmt_commands/img/management_commands_over_http_admin.png diff --git a/admin/mgmt_commands/img/management_commands_over_http_btn_add_job.png b/docs/admin/mgmt_commands/img/management_commands_over_http_btn_add_job.png similarity index 100% rename from admin/mgmt_commands/img/management_commands_over_http_btn_add_job.png rename to docs/admin/mgmt_commands/img/management_commands_over_http_btn_add_job.png diff --git a/admin/mgmt_commands/img/management_commands_over_http_job_status.png b/docs/admin/mgmt_commands/img/management_commands_over_http_job_status.png similarity index 100% rename from admin/mgmt_commands/img/management_commands_over_http_job_status.png rename to docs/admin/mgmt_commands/img/management_commands_over_http_job_status.png diff --git a/admin/mgmt_commands/img/management_commands_over_http_job_status_with_traceback.png b/docs/admin/mgmt_commands/img/management_commands_over_http_job_status_with_traceback.png similarity index 100% rename from admin/mgmt_commands/img/management_commands_over_http_job_status_with_traceback.png rename to docs/admin/mgmt_commands/img/management_commands_over_http_job_status_with_traceback.png diff --git a/admin/mgmt_commands/img/management_commands_over_http_start_job.png b/docs/admin/mgmt_commands/img/management_commands_over_http_start_job.png similarity index 100% rename from admin/mgmt_commands/img/management_commands_over_http_start_job.png rename to docs/admin/mgmt_commands/img/management_commands_over_http_start_job.png diff --git a/admin/mgmt_commands/img/menu_admin.png b/docs/admin/mgmt_commands/img/menu_admin.png similarity index 100% rename from admin/mgmt_commands/img/menu_admin.png rename to docs/admin/mgmt_commands/img/menu_admin.png diff --git a/admin/mgmt_commands/img/permissions_django_admin.png b/docs/admin/mgmt_commands/img/permissions_django_admin.png similarity index 100% rename from admin/mgmt_commands/img/permissions_django_admin.png rename to docs/admin/mgmt_commands/img/permissions_django_admin.png diff --git a/admin/mgmt_commands/img/register.png b/docs/admin/mgmt_commands/img/register.png similarity index 100% rename from admin/mgmt_commands/img/register.png rename to docs/admin/mgmt_commands/img/register.png diff --git a/admin/mgmt_commands/img/sign_up_test_user.png b/docs/admin/mgmt_commands/img/sign_up_test_user.png similarity index 100% rename from admin/mgmt_commands/img/sign_up_test_user.png rename to docs/admin/mgmt_commands/img/sign_up_test_user.png diff --git a/admin/mgmt_commands/img/tiled_vs_stripped.png b/docs/admin/mgmt_commands/img/tiled_vs_stripped.png similarity index 100% rename from admin/mgmt_commands/img/tiled_vs_stripped.png rename to docs/admin/mgmt_commands/img/tiled_vs_stripped.png diff --git a/admin/mgmt_commands/img/upload_shapefile.png b/docs/admin/mgmt_commands/img/upload_shapefile.png similarity index 100% rename from admin/mgmt_commands/img/upload_shapefile.png rename to docs/admin/mgmt_commands/img/upload_shapefile.png diff --git a/docs/admin/mgmt_commands/index.md b/docs/admin/mgmt_commands/index.md new file mode 100644 index 00000000..07e28f5f --- /dev/null +++ b/docs/admin/mgmt_commands/index.md @@ -0,0 +1,2414 @@ +# Migrate GeoNode Base URL {#migrate_baseurl} + +The `migrate_baseurl` `Management Command`{.interpreted-text role="guilabel"} allows you to fix all the GeoNode Links whenever, for some reason, +you need to change the `Domain Name`{.interpreted-text role="guilabel"} of `IP Address`{.interpreted-text role="guilabel"} of GeoNode. + +This **must** be used also in the cases you\'ll need to change the network schema from `HTTP` to `HTTPS`, as an instance. + +First of all let\'s take a look at the `--help`{.interpreted-text role="guilabel"} option of the `migrate_baseurl` +management command in order to inspect all the command options and features. + +Run + +``` shell +DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --help +``` + +This will produce output that looks like the following + +``` shell +usage: manage.py migrate_baseurl [-h] [--version] [-v {0,1,2,3}] + [--settings SETTINGS] + [--pythonpath PYTHONPATH] [--traceback] + [--no-color] [-f] + [--source-address SOURCE_ADDRESS] + [--target-address TARGET_ADDRESS] + +Migrate GeoNode VM Base URL + +optional arguments: +-h, --help show this help message and exit +--version show program's version number and exit +-v {0,1,2,3}, --verbosity {0,1,2,3} + Verbosity level; 0=minimal output, 1=normal output, + 2=verbose output, 3=very verbose output +--settings SETTINGS The Python path to a settings module, e.g. + "myproject.settings.main". If this isn't provided, the + DJANGO_SETTINGS_MODULE environment variable will be + used. +--pythonpath PYTHONPATH + A directory to add to the Python path, e.g. + "/home/djangoprojects/myproject". +--traceback Raise on CommandError exceptions +--no-color Don't colorize the command output. +-f, --force Forces the execution without asking for confirmation. +--source-address SOURCE_ADDRESS + Source Address (the one currently on DB e.g. + http://192.168.1.23) +--target-address TARGET_ADDRESS + Target Address (the one to be changed e.g. http://my- + public.geonode.org) +``` + +- **Example 1**: I want to move my GeoNode instance from `http:\\127.0.0.1` to `http:\\example.org` + + > ::: warning + > ::: title + > Warning + > ::: + > + > Make always sure you are using the **correct** settings + > ::: + > + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --source-address=127.0.0.1 --target-address=example.org + > ``` + +- **Example 2**: I want to move my GeoNode instance from `http:\\example.org` to `https:\\example.org` + + > ::: warning + > ::: title + > Warning + > ::: + > + > Make always sure you are using the **correct** settings + > ::: + > + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --source-address=http:\\example.org --target-address=https:\\example.org + > ``` + +- **Example 3**: I want to move my GeoNode instance from `https:\\example.org` to `https:\\geonode.example.org` + + > ::: warning + > ::: title + > Warning + > ::: + > + > Make always sure you are using the **correct** settings + > ::: + > + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl --source-address=example.org --target-address=geonode.example.org + > ``` + +::: note +::: title +Note +::: + +After migrating the base URL, make sure to sanitize the links and catalog metadata also (`sync_layers_and_metadata`{.interpreted-text role="ref"}). +::: + +# Update Permissions, Metadata, Legends and Download Links {#sync_layers_and_metadata} + +The following three utility `Management Commands`{.interpreted-text role="guilabel"}, allow to fixup: + +1. `Users/Groups Permissions`{.interpreted-text role="guilabel"} on `Datasets`{.interpreted-text role="guilabel"}; those will be refreshed and synchronized with the `GIS Server`{.interpreted-text role="guilabel"} ones also +2. `Metadata`{.interpreted-text role="guilabel"}, `Legend`{.interpreted-text role="guilabel"} and `Download`{.interpreted-text role="guilabel"} links on `Datasets`{.interpreted-text role="guilabel"} and `Maps`{.interpreted-text role="guilabel"} +3. Cleanup `Duplicated Links`{.interpreted-text role="guilabel"} and `Outdated Thumbnails`{.interpreted-text role="guilabel"} + +## Management Command `sync_geonode_datasets` + +This command allows to sync already existing permissions on Datasets. In order to change/set Datasets\' permissions refer to the section `batch_sync_permissions`{.interpreted-text role="ref"} + +The options are: + +- **filter**; Only update data the Dataset names that match the given filter. +- **username**; Only update data owned by the specified username. +- **updatepermissions**; Update the Dataset permissions; synchronize it back to the GeoSpatial Server. + This option is also available from the `Layer Details`{.interpreted-text role="guilabel"} page. +- **updateattributes**; Update the Dataset attributes; synchronize it back to the GeoSpatial Server. + This option is also available from the `Layer Details`{.interpreted-text role="guilabel"} page. +- **updatethumbnails**; Update the Dataset thumbnail. + This option is also available from the `Layer Details`{.interpreted-text role="guilabel"} page. +- **updatebbox**; Update the Dataset BBOX and LotLan BBOX. + This option is also available from the `Layer Details`{.interpreted-text role="guilabel"} page. +- **remove-duplicates**; Removes duplicated Links. + +First of all let\'s take a look at the `--help`{.interpreted-text role="guilabel"} option of the `sync_geonode_datasets` +management command in order to inspect all the command options and features. + +Run + +``` shell +DJANGO_SETTINGS_MODULE=geonode.settings python manage.py sync_geonode_datasets --help +``` + +::: note +::: title +Note +::: + +If you enabled `local_settings.py` the command will change as following: + +``` shell +DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py sync_geonode_datasets --help +``` +::: + +This will produce output that looks like the following + +``` shell +usage: manage.py sync_geonode_datasets [-h] [--version] [-v {0,1,2,3}] + [--settings SETTINGS] + [--pythonpath PYTHONPATH] [--traceback] + [--no-color] [-i] [-d] [-f FILTER] + [-u USERNAME] [--updatepermissions] + [--updatethumbnails] [--updateattributes][--updatebbox] + +Update the GeoNode Datasets: permissions (including GeoFence database), +statistics, thumbnails + +optional arguments: +-h, --help show this help message and exit +--version show program's version number and exit +-v {0,1,2,3}, --verbosity {0,1,2,3} + Verbosity level; 0=minimal output, 1=normal output, + 2=verbose output, 3=very verbose output +--settings SETTINGS The Python path to a settings module, e.g. + "myproject.settings.main". If this isn't provided, the + DJANGO_SETTINGS_MODULE environment variable will be + used. +--pythonpath PYTHONPATH + A directory to add to the Python path, e.g. + "/home/djangoprojects/myproject". +--traceback Raise on CommandError exceptions +--no-color Don't colorize the command output. +-i, --ignore-errors Stop after any errors are encountered. +-d, --remove-duplicates + Remove duplicates first. +-f FILTER, --filter FILTER + Only update data the Datasets that match the given + filter. +-u USERNAME, --username USERNAME + Only update data owned by the specified username. +--updatepermissions Update the Dataset permissions. +--updatethumbnails Update the Dataset styles and thumbnails. +--updateattributes Update the Dataset attributes. +--updatebbox Update the Dataset BBOX. +``` + +- **Example 1**: I want to update/sync all Datasets permissions and attributes with the GeoSpatial Server + + > ::: warning + > ::: title + > Warning + > ::: + > + > Make always sure you are using the **correct** settings + > ::: + > + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py sync_geonode_datasets --updatepermissions --updateattributes + > ``` + +- **Example 2**: I want to regenerate the Thumbnails of all the Datasets belonging to `afabiani` + + > ::: warning + > ::: title + > Warning + > ::: + > + > Make always sure you are using the **correct** settings + > ::: + > + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py sync_geonode_datasets -u afabiani --updatethumbnails + > ``` + +## Management Command `sync_geonode_maps` + +This command is basically similar to the previous one, but affects the `Maps`{.interpreted-text role="guilabel"}; with some limitations. + +The options are: + +- **filter**; Only update data the maps titles that match the given filter. +- **username**; Only update data owned by the specified username. +- **updatethumbnails**; Update the map styles and thumbnails. + This option is also available from the `Map Details`{.interpreted-text role="guilabel"} page. +- **remove-duplicates**; Removes duplicated Links. + +First of all let\'s take a look at the `--help`{.interpreted-text role="guilabel"} option of the `sync_geonode_maps` +management command in order to inspect all the command options and features. + +Run + +``` shell +DJANGO_SETTINGS_MODULE=geonode.settings python manage.py sync_geonode_maps --help +``` + +::: note +::: title +Note +::: + +If you enabled `local_settings.py` the command will change as following: + +``` shell +DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py sync_geonode_maps --help +``` +::: + +This will produce output that looks like the following + +``` shell +usage: manage.py sync_geonode_maps [-h] [--version] [-v {0,1,2,3}] + [--settings SETTINGS] + [--pythonpath PYTHONPATH] [--traceback] + [--no-color] [-i] [-d] [-f FILTER] + [-u USERNAME] [--updatethumbnails] + +Update the GeoNode maps: permissions, thumbnails + +optional arguments: +-h, --help show this help message and exit +--version show program's version number and exit +-v {0,1,2,3}, --verbosity {0,1,2,3} + Verbosity level; 0=minimal output, 1=normal output, + 2=verbose output, 3=very verbose output +--settings SETTINGS The Python path to a settings module, e.g. + "myproject.settings.main". If this isn't provided, the + DJANGO_SETTINGS_MODULE environment variable will be + used. +--pythonpath PYTHONPATH + A directory to add to the Python path, e.g. + "/home/djangoprojects/myproject". +--traceback Raise on CommandError exceptions +--no-color Don't colorize the command output. +-i, --ignore-errors Stop after any errors are encountered. +-d, --remove-duplicates + Remove duplicates first. +-f FILTER, --filter FILTER + Only update data the maps that match the given filter. +-u USERNAME, --username USERNAME + Only update data owned by the specified username. +--updatethumbnails Update the map styles and thumbnails. +``` + +- **Example 1**: I want to regenerate the Thumbnail of the Map `This is a test Map` + + > ::: warning + > ::: title + > Warning + > ::: + > + > Make always sure you are using the **correct** settings + > ::: + > + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py sync_geonode_maps --updatethumbnails -f 'This is a test Map' + > ``` + +Management Command `set_all_datasets_metadata` +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-- + +This command allows to reset **Metadata Attributes** and **Catalogue Schema** on Datasets. The command will also update the `CSW Catalogue`{.interpreted-text role="guilabel"} XML and Links of GeoNode. + +The options are: + +- **filter**; Only update data the Datasets that match the given filter. +- **username**; Only update data owned by the specified username. +- **remove-duplicates**; Update the map styles and thumbnails. +- **delete-orphaned-thumbs**; Removes duplicated Links. +- +- **set-uuid**; will refresh the UUID based on the UUID_HANDLER if configured (Default False). +- +- **set_attrib**; If set will refresh the attributes of the resource taken from Geoserver. (Default True). +- +- **set_links**; If set will refresh the links of the resource. (Default True). + +First of all let\'s take a look at the `--help`{.interpreted-text role="guilabel"} option of the `set_all_datasets_metadata` +management command in order to inspect all the command options and features. + +Run + +``` shell +DJANGO_SETTINGS_MODULE=geonode.settings python manage.py set_all_datasets_metadata --help +``` + +::: note +::: title +Note +::: + +If you enabled `local_settings.py` the command will change as following: + +``` shell +DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py set_all_datasets_metadata --help +``` +::: + +This will produce output that looks like the following + +``` shell +usage: manage.py set_all_datasets_metadata [-h] [--version] [-v {0,1,2,3}] + [--settings SETTINGS] + [--pythonpath PYTHONPATH] + [--traceback] [--no-color] [-i] [-d] + [-t] [-f FILTER] [-u USERNAME] + +Resets Metadata Attributes and Schema to All Datasets + +optional arguments: +-h, --help show this help message and exit +--version show program's version number and exit +-v {0,1,2,3}, --verbosity {0,1,2,3} + Verbosity level; 0=minimal output, 1=normal output, + 2=verbose output, 3=very verbose output +--settings SETTINGS The Python path to a settings module, e.g. + "myproject.settings.main". If this isn't provided, the + DJANGO_SETTINGS_MODULE environment variable will be + used. +--pythonpath PYTHONPATH + A directory to add to the Python path, e.g. + "/home/djangoprojects/myproject". +--traceback Raise on CommandError exceptions +--no-color Don't colorize the command output. +-i, --ignore-errors Stop after any errors are encountered. +-d, --remove-duplicates + Remove duplicates first. +-t, --delete-orphaned-thumbs + Delete Orphaned Thumbnails. +-f FILTER, --filter FILTER + Only update data the Datasets that match the given + filter +-u USERNAME, --username USERNAME + Only update data owned by the specified username +``` + +- **Example 1**: After having changed the Base URL, I want to regenerate all the Catalogue Schema and eventually remove all duplicates. + + > ::: warning + > ::: title + > Warning + > ::: + > + > Make always sure you are using the **correct** settings + > ::: + > + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py set_all_datasets_metadata -d + > ``` + +## Management Command `regenerate_xml` + +The command will regenerate the `CSW Catalogue`{.interpreted-text role="guilabel"} XML metadata files. + +The main options are: + +- **layer**; Only process specified layers. +- **dry-run**; Do not actually perform any change. + +When run with the `--help`{.interpreted-text role="guilabel"} option the full list of available options will be presented. + +``` shell +DJANGO_SETTINGS_MODULE=geonode.settings python manage.py regenerate_xml --help +``` + +This will produce the following output: + +``` shell +usage: manage.py regenerate_xml [-h] [-l LAYERS] [--skip-logger-setup] [-d] [--version] [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] [--force-color] + [--skip-checks] + +Re-create XML metadata documents + +options: +-h, --help show this help message and exit +-l LAYERS, --layer LAYERS + Only process specified layers +--skip-logger-setup Skips setup of the "geonode.br" logger, "br" handler and "br" format if not present in settings +-d, --dry-run Do not actually perform any change +--version Show program's version number and exit. +-v {0,1,2,3}, --verbosity {0,1,2,3} + Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output +--settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the DJANGO_SETTINGS_MODULE environment variable will be used. +--pythonpath PYTHONPATH + A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". +--traceback Raise on CommandError exceptions. +--no-color Don't colorize the command output. +--force-color Force colorization of the command output. +--skip-checks Skip system checks. +``` + +- **Example**: + + > ::: warning + > ::: title + > Warning + > ::: + > + > Make sure you are using the **correct** settings + > ::: + > + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py regenerate_xml -d + > ``` + +# Loading Data into GeoNode {#load_data_into_geonode} + +There are situations where it is not possible or not convenient to use the +`Upload Form`{.interpreted-text role="guilabel"} to add new Datasets to GeoNode via the web interface. +For instance: + +- The dataset is too big to be uploaded through a web interface. +- Import data from a mass storage programmatically. +- Import tables from a database. + +This section will walk you through the various options available to load data into your +GeoNode from GeoServer, from the command-line or programmatically. + +::: warning +::: title +Warning +::: + +Some parts of this section have been taken from the [GeoServer](https://geoserver.geo-solutions.it/edu/en) project and training documentation. +::: + +## Management Command `importlayers` {#importlayers} + +The `geonode.geoserver` Django app includes 2 management commands that you can use to +load data in your GeoNode. + +Both of them can be invoked by using the `manage.py` script. + +First of all let\'s take a look at the `--help`{.interpreted-text role="guilabel"} option of the `importlayers` +management command in order to inspect all the command options and features. + +Run + +``` shell +DJANGO_SETTINGS_MODULE=geonode.settings python manage.py importlayers --help +``` + +::: note +::: title +Note +::: + +If you enabled `local_settings.py` the command will change as following: + +``` shell +DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py importlayers --help +``` +::: + +This will produce output that looks like the following + +``` shell +usage: manage.py importlayers [-h] [-hh HOST] [-u USERNAME] [-p PASSWORD] + [--version] [-v {0,1,2,3}] [--settings SETTINGS] + [--pythonpath PYTHONPATH] [--traceback] [--no-color] + [--force-color] [--skip-checks] + [path [path ...]] + +Brings files from a local directory, including subfolders, into a GeoNode site. +The datasets are added to the Django database, the GeoServer configuration, and the +pycsw metadata index. At this moment only files of type Esri Shapefile (.shp) and GeoTiff (.tif) are supported. +In order to perform the import, GeoNode must be up and running. + +positional arguments: +path path [path...] + +optional arguments: +-h, --help show this help message and exit +--version show program's version number and exit +-v {0,1,2,3}, --verbosity {0,1,2,3} + Verbosity level; 0=minimal output, 1=normal output, + 2=verbose output, 3=very verbose output +--settings SETTINGS The Python path to a settings module, e.g. + "myproject.settings.main". If this isn't provided, the + DJANGO_SETTINGS_MODULE environment variable will be + used. +--pythonpath PYTHONPATH + A directory to add to the Python path, e.g. + "/home/djangoprojects/myproject". +-hh HOST, --host HOST + Geonode host url +-u USERNAME, --username USERNAME + Geonode username +-p PASSWORD, --password PASSWORD + Geonode password +``` + +While the description of most of the options should be self explanatory, its worth +reviewing some of the key options a bit more in details. + +- The `-hh`{.interpreted-text role="guilabel"} Identifies the GeoNode server where we want to upload our Datasets. The default value is `http://localhost:8000`{.interpreted-text role="guilabel"}. +- The `-u`{.interpreted-text role="guilabel"} Identifies the username for the login. The default value is `admin`{.interpreted-text role="guilabel"}. +- The `-p`{.interpreted-text role="guilabel"} Identifies the password for the login. The default value is `admin`{.interpreted-text role="guilabel"}. + +The import Datasets management command is invoked by specifying options as described +above and specifying the path to a directory that contains multiple files. For purposes of this exercise, let\'s use the default set of testing Datasets that ship with geonode. +You can replace this path with a directory to your own shapefiles. + +``` shell +First let's run the GeoNode server: +DJANGO_SETTINGS_MODULE=geonode.settings python manage.py runserver +Then let's import the files: +DJANGO_SETTINGS_MODULE=geonode.settings python manage.py importlayers /home/user/.virtualenvs/geonode/lib/python3.8/site-packages/gisdata/data/good/vector/ +``` + +This command will produce the following output to your terminal + +``` shell +san_andres_y_providencia_poi.shp: 201 +san_andres_y_providencia_location.shp: 201 +san_andres_y_providencia_administrative.shp: 201 +san_andres_y_providencia_coastline.shp: 201 +san_andres_y_providencia_highway.shp: 201 +single_point.shp: 201 +san_andres_y_providencia_water.shp: 201 +san_andres_y_providencia_natural.shp: 201 + +1.7456605294117646 seconds per Dataset + +Output data: { + "success": [ + "san_andres_y_providencia_poi.shp", + "san_andres_y_providencia_location.shp", + "san_andres_y_providencia_administrative.shp", + "san_andres_y_providencia_coastline.shp", + "san_andres_y_providencia_highway.shp", + "single_point.shp", + "san_andres_y_providencia_water.shp", + "san_andres_y_providencia_natural.shp" + ], + "errors": [] +} +``` + +As output the command will print: + +``` shell +layer_name: status code for each Layer +upload_time spent of each Dataset + +A json with the representation of the Datasets uploaded or with some errors. +``` + +The status code, is the response coming from GeoNode. For example 201 means that the Dataset has been correctly uploaded + +If you encounter errors while running this command, please check the GeoNode logs for more information. + +## Management Command `updatelayers` {#updatelayers} + +While it is possible to import Datasets directly from your servers filesystem into your +GeoNode, you may have an existing GeoServer that already has data in it, or you may +want to configure data from a GeoServer which is not directly supported by uploading data. + +GeoServer supports a wide range of data formats and connections to database, some of them +may not be supported as GeoNode upload formats. You can add them to your GeoNode by following the procedure described below. + +GeoServer supports 4 types of data: `Raster`{.interpreted-text role="guilabel"}, `Vector`{.interpreted-text role="guilabel"}, `Databases`{.interpreted-text role="guilabel"} and `Cascaded`{.interpreted-text role="guilabel"}. + +For a list of the supported formats for each type of data, consult the following pages: + +- +- +- +- + +::: note +::: title +Note +::: + +Some of these raster or vector formats or database types require that you install specific plugins in your GeoServer in order to use the. Please consult the GeoServer documentation for more information. +::: + +### Data from a PostGIS database + +Lets walk through an example of configuring a new PostGIS database in GeoServer and then +configuring those Datasets in your GeoNode. + +First visit the GeoServer administration interface on your server. This is usually on port 8080 and is available at + +1. You should login with the superuser credentials you setup when you first configured your GeoNode instance. + + > Once you are logged in to the GeoServer Admin interface, you should see the following. + > + > ![](img/geoserver_admin.png){.align-center} + > + > ::: note + > ::: title + > Note + > ::: + > + > The number of stores, Datasets and workspaces may be different depending on what you already have configured in your GeoServer. + > ::: + +2. Next you want to select the \"Stores\" option in the left hand menu, and then the \"Add new Store\" option. The following screen will be displayed. + + > ![](img/geoserver_new_store.png){.align-center} + +3. In this case, we want to select the PostGIS store type to create a connection to our existing database. On the next screen you will need to enter the parameters to connect to your PostGIS database (alter as necessary for your own database). + + > ![](img/geoserver_postgis_params.png){.align-center} + > + > ::: note + > ::: title + > Note + > ::: + > + > If you are unsure about any of the settings, leave them as the default. + > ::: + +4. The next screen lets you configure the Datasets in your database. This will of course be different depending on the Datasets in your database. + + > ![](img/geoserver_publish_layers.png){.align-center} + +5. Select the \"Publish\" button for one of the Datasets and the next screen will be displayed where you can enter metadata for this Dataset. Since we will be managing this metadata in GeoNode, we can leave these alone for now. + + > ![](img/geoserver_layer_params.png){.align-center} + +6. The things that *must* be specified are the Declared SRS and you must select the \"Compute from Data\" and \"Compute from native bounds\" links after the SRS is specified. + + > ![](img/geoserver_srs.png){.align-center} + > + > ![](img/geoserver_srs_2.png){.align-center} + +7. Click save and this Dataset will now be configured for use in your GeoServer. + + > ![](img/geoserver_layers.png){.align-center} + +8. The next step is to configure these Datasets in GeoNode. The `updatelayers` management command can be used for this purpose. As with `importlayers`, it\'s useful to look at the command line options for this command by passing the `--help`{.interpreted-text role="guilabel"} option + + > Run + > + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py updatelayers --help + > ``` + > + > ::: note + > ::: title + > Note + > ::: + > + > If you enabled `local_settings.py` the command will change as following: + > + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py updatelayers --help + > ``` + > ::: + > + > This will produce output that looks like the following + > + > ``` shell + > usage: manage.py updatelayers [-h] [--version] [-v {0,1,2,3}] + > [--settings SETTINGS] [--pythonpath PYTHONPATH] + > [--traceback] [--no-color] [-i] + > [--skip-unadvertised] + > [--skip-geonode-registered] [--remove-deleted] + > [-u USER] [-f FILTER] [-s STORE] [-w WORKSPACE] + > [-p PERMISSIONS] + > + > Update the GeoNode application with data from GeoServer + > + > optional arguments: + > -h, --help show this help message and exit + > --version show program's version number and exit + > -v {0,1,2,3}, --verbosity {0,1,2,3} + > Verbosity level; 0=minimal output, 1=normal output, + > 2=verbose output, 3=very verbose output + > --settings SETTINGS The Python path to a settings module, e.g. + > "myproject.settings.main". If this isn't provided, the + > DJANGO_SETTINGS_MODULE environment variable will be + > used. + > --pythonpath PYTHONPATH + > A directory to add to the Python path, e.g. + > "/home/djangoprojects/myproject". + > --traceback Raise on CommandError exceptions + > --no-color Don't colorize the command output. + > -i, --ignore-errors Stop after any errors are encountered. + > --skip-unadvertised Skip processing unadvertised Datasets from GeoSever. + > --skip-geonode-registered + > Just processing GeoServer Datasets still not registered + > in GeoNode. + > --remove-deleted Remove GeoNode Datasets that have been deleted from + > GeoSever. + > -u USER, --user USER Name of the user account which should own the imported + > Datasets + > -f FILTER, --filter FILTER + > Only update data the Datasets that match the given + > filter + > -s STORE, --store STORE + > Only update data the Datasets for the given geoserver + > store name + > -w WORKSPACE, --workspace WORKSPACE + > Only update data on specified workspace + > -p PERMISSIONS, --permissions PERMISSIONS + > Permissions to apply to each Dataset + > ``` + +The update procedure includes the following steps: + +> - The process fetches from GeoServer the relevant WMS layers (all, by store or by workspace) +> - If a filter is defined, the GeoServer layers are filtered +> - For each of the layers, a GeoNode dataset is created based on the metadata registered on GeoServer (title, abstract, bounds) +> - New layers are added, existing layers are replaced, unless the `--skip-geonode-registered`{.interpreted-text role="guilabel"} option is used +> - The GeoNode layers, added in previous runs of the update process, which are no longer available in GeoServer are removed, if the `--remove-delete`{.interpreted-text role="guilabel"} option is set + +::: warning +::: title +Warning +::: + +One of the `--workspace`{.interpreted-text role="guilabel"} or `--store`{.interpreted-text role="guilabel"} must be always specified if you want to ingest Datasets belonging to a specific `Workspace`. As an instance, in order to ingest the Datasets present into the `geonode` workspace, you will need to specify the option `-w geonode`. +::: + +9. Let\'s ingest the Dataset `geonode:_1_SARMIENTO_ENERO_2018` from the `geonode` workspace. + + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py updatelayers -w geonode -f _1_SARMIENTO_ENERO_2018 + > ``` + > + > ``` shell + > Inspecting the available Datasets in GeoServer ... + > Found 1 Datasets, starting processing + > /usr/local/lib/python2.7/site-packages/owslib/iso.py:117: FutureWarning: the .identification and .serviceidentification properties will merge into .identification being a list of properties. This is currently implemented in .identificationinfo. Please see https://github.com/geopython/OWSLib/issues/38 for more information + > FutureWarning) + > /usr/local/lib/python2.7/site-packages/owslib/iso.py:495: FutureWarning: The .keywords and .keywords2 properties will merge into the .keywords property in the future, with .keywords becoming a list of MD_Keywords instances. This is currently implemented in .keywords2. Please see https://github.com/geopython/OWSLib/issues/301 for more information + > FutureWarning) + > Content-Type: text/html; charset="utf-8" + > MIME-Version: 1.0 + > Content-Transfer-Encoding: 7bit + > Subject: [master.demo.geonode.org] A new Dataset has been uploaded + > From: webmaster@localhost + > To: mapadeldelito@chubut.gov.ar + > Reply-To: webmaster@localhost + > Date: Tue, 08 Oct 2019 12:26:17 -0000 + > Message-ID: <20191008122617.28801.94967@d3cf85425231> + > + > + > + > You have received the following notice from master.demo.geonode.org: + >

+ > + > The user admin uploaded the following Dataset:
+ > _1_SARMIENTO_ENERO_2018
+ > You can visit the Dataset's detail page here: http://master.demo.geonode.org/Datasets/geonode:_1_SARMIENTO_ENERO_2018 + > + >

+ >

+ > To change how you receive notifications, please go to http://master.demo.geonode.org + >

+ > + > + > ------------------------------------------------------------------------------- + > Content-Type: text/html; charset="utf-8" + > MIME-Version: 1.0 + > Content-Transfer-Encoding: 7bit + > Subject: [master.demo.geonode.org] A new Dataset has been uploaded + > From: webmaster@localhost + > To: giacomo8vinci@gmail.com + > Reply-To: webmaster@localhost + > Date: Tue, 08 Oct 2019 12:26:17 -0000 + > Message-ID: <20191008122617.28801.53784@d3cf85425231> + > + > + > + > You have received the following notice from master.demo.geonode.org: + >

+ > + > The user admin uploaded the following Dataset:
+ > _1_SARMIENTO_ENERO_2018
+ > You can visit the Dataset's detail page here: http://master.demo.geonode.org/Datasets/geonode:_1_SARMIENTO_ENERO_2018 + > + >

+ >

+ > To change how you receive notifications, please go to http://master.demo.geonode.org + >

+ > + > + > ------------------------------------------------------------------------------- + > Content-Type: text/html; charset="utf-8" + > MIME-Version: 1.0 + > Content-Transfer-Encoding: 7bit + > Subject: [master.demo.geonode.org] A new Dataset has been uploaded + > From: webmaster@localhost + > To: fmgagliano@gmail.com + > Reply-To: webmaster@localhost + > Date: Tue, 08 Oct 2019 12:26:17 -0000 + > Message-ID: <20191008122617.28801.26265@d3cf85425231> + > + > + > + > You have received the following notice from master.demo.geonode.org: + >

+ > + > The user admin uploaded the following Dataset:
+ > _1_SARMIENTO_ENERO_2018
+ > You can visit the Dataset's detail page here: http://master.demo.geonode.org/Datasets/geonode:_1_SARMIENTO_ENERO_2018 + > + >

+ >

+ > To change how you receive notifications, please go to http://master.demo.geonode.org + >

+ > + > + > ------------------------------------------------------------------------------- + > Found geoserver resource for this Dataset: _1_SARMIENTO_ENERO_2018 + > ... Creating Default Resource Links for Layer [geonode:_1_SARMIENTO_ENERO_2018] + > -- Resource Links[Prune old links]... + > -- Resource Links[Prune old links]...done! + > -- Resource Links[Compute parameters for the new links]... + > -- Resource Links[Create Raw Data download link]... + > -- Resource Links[Create Raw Data download link]...done! + > -- Resource Links[Set download links for WMS, WCS or WFS and KML]... + > -- Resource Links[Set download links for WMS, WCS or WFS and KML]...done! + > -- Resource Links[Legend link]... + > -- Resource Links[Legend link]...done! + > -- Resource Links[Thumbnail link]... + > -- Resource Links[Thumbnail link]...done! + > -- Resource Links[OWS Links]... + > -- Resource Links[OWS Links]...done! + > Content-Type: text/html; charset="utf-8" + > MIME-Version: 1.0 + > Content-Transfer-Encoding: 7bit + > Subject: [master.demo.geonode.org] A Dataset has been updated + > From: webmaster@localhost + > To: mapadeldelito@chubut.gov.ar + > Reply-To: webmaster@localhost + > Date: Tue, 08 Oct 2019 12:26:20 -0000 + > Message-ID: <20191008122620.28801.81598@d3cf85425231> + > + > + > + > You have received the following notice from master.demo.geonode.org: + >

+ > + > The following Dataset was updated:
+ > _1_SARMIENTO_ENERO_2018, owned by admin
+ > You can visit the Dataset's detail page here: http://master.demo.geonode.org/Datasets/geonode:_1_SARMIENTO_ENERO_2018 + > + >

+ >

+ > To change how you receive notifications, please go to http://master.demo.geonode.org + >

+ > + > + > ------------------------------------------------------------------------------- + > Content-Type: text/html; charset="utf-8" + > MIME-Version: 1.0 + > Content-Transfer-Encoding: 7bit + > Subject: [master.demo.geonode.org] A Dataset has been updated + > From: webmaster@localhost + > To: giacomo8vinci@gmail.com + > Reply-To: webmaster@localhost + > Date: Tue, 08 Oct 2019 12:26:20 -0000 + > Message-ID: <20191008122620.28801.93778@d3cf85425231> + > + > + > + > You have received the following notice from master.demo.geonode.org: + >

+ > + > The following Dataset was updated:
+ > _1_SARMIENTO_ENERO_2018, owned by admin
+ > You can visit the Dataset's detail page here: http://master.demo.geonode.org/Datasets/geonode:_1_SARMIENTO_ENERO_2018 + > + >

+ >

+ > To change how you receive notifications, please go to http://master.demo.geonode.org + >

+ > + > + > ------------------------------------------------------------------------------- + > Content-Type: text/html; charset="utf-8" + > MIME-Version: 1.0 + > Content-Transfer-Encoding: 7bit + > Subject: [master.demo.geonode.org] A Dataset has been updated + > From: webmaster@localhost + > To: fmgagliano@gmail.com + > Reply-To: webmaster@localhost + > Date: Tue, 08 Oct 2019 12:26:20 -0000 + > Message-ID: <20191008122620.28801.58585@d3cf85425231> + > + > + > + > You have received the following notice from master.demo.geonode.org: + >

+ > + > The following Dataset was updated:
+ > _1_SARMIENTO_ENERO_2018, owned by admin
+ > You can visit the Dataset's detail page here: http://master.demo.geonode.org/Datasets/geonode:_1_SARMIENTO_ENERO_2018 + > + >

+ >

+ > To change how you receive notifications, please go to http://master.demo.geonode.org + >

+ > + > + > ------------------------------------------------------------------------------- + > Found geoserver resource for this Dataset: _1_SARMIENTO_ENERO_2018 + > /usr/local/lib/python2.7/site-packages/geoserver/style.py:80: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead. + > if not user_style: + > /usr/local/lib/python2.7/site-packages/geoserver/style.py:84: FutureWarning: The behavior of this method will change in future versions. Use specific 'len(elem)' or 'elem is not None' test instead. + > if user_style: + > ... Creating Default Resource Links for Layer [geonode:_1_SARMIENTO_ENERO_2018] + > -- Resource Links[Prune old links]... + > -- Resource Links[Prune old links]...done! + > -- Resource Links[Compute parameters for the new links]... + > -- Resource Links[Create Raw Data download link]... + > -- Resource Links[Create Raw Data download link]...done! + > -- Resource Links[Set download links for WMS, WCS or WFS and KML]... + > -- Resource Links[Set download links for WMS, WCS or WFS and KML]...done! + > -- Resource Links[Legend link]... + > -- Resource Links[Legend link]...done! + > -- Resource Links[Thumbnail link]... + > -- Resource Links[Thumbnail link]...done! + > -- Resource Links[OWS Links]... + > -- Resource Links[OWS Links]...done! + > [created] Layer _1_SARMIENTO_ENERO_2018 (1/1) + > + > + > Finished processing 1 Datasets in 5.0 seconds. + > + > 1 Created Datasets + > 0 Updated Datasets + > 0 Failed Datasets + > 5.000000 seconds per Dataset + > ``` + +::: note +::: title +Note +::: + +In case you don\'t specify the `-f`{.interpreted-text role="guilabel"} option, the Datasets that already exists in your GeoNode will be just updated and the configuration synchronized between GeoServer and GeoNode. +::: + +::: warning +::: title +Warning +::: + +When updating **from** GeoServer, the configuration on GeoNode will be changed! +::: + +## Using `GDAL` and `OGR` to convert your Data for use in GeoNode {#gdal_ogr_data_processing} + +GeoNode supports uploading data in `ESRI shapefiles`{.interpreted-text role="guilabel"}, `GeoTIFF`{.interpreted-text role="guilabel"}, `CSV`{.interpreted-text role="guilabel"}, `GeoJSON`{.interpreted-text role="guilabel"}, `ASCII-GRID`{.interpreted-text role="guilabel"} and `KML / KMZ`{.interpreted-text role="guilabel"} formats (for the last three formats only if you are using the `geonode.importer` backend). + +- If your data is in other formats, you will need to convert it into one of these formats for use in GeoNode. +- If your `Raster`{.interpreted-text role="guilabel"} data is not correctly processed, it might be almost unusable with GeoServer and GeoNode. You will need to process it using [GDAL]{.title-ref}. + +You need to make sure that you have the GDAL library installed on your system. +On Ubuntu you can install this package with the following command: + +``` shell +sudo apt-get install gdal-bin +``` + +### OGR (Vector Data) + +OGR is used to manipulate vector data. In this example, we will use MapInfo .tab files and convert them to shapefiles with the ogr2ogr command. We will use sample MapInfo files from the website linked below. + + + +You can download the Admin;(Postcode) Dataset by issuing the following command: + + $ wget http://services.land.vic.gov.au/sampledata/shape/admin_postcode_vm.zip + +You will need to unzip this dataset by issuing the following command: + + $ unzip admin_postcode_vm.zip + +This will leave you with the following files in the directory where you executed the above commands: + + |-- ANZVI0803003025.htm + |-- DSE_Data_Access_Licence.pdf + |-- VMADMIN.POSTCODE_POLYGON.xml + |-- admin_postcode_vm.zip + --- vicgrid94 + --- mif + --- lga_polygon + --- macedon\ ranges + |-- EXTRACT_POLYGON.mid + |-- EXTRACT_POLYGON.mif + --- VMADMIN + |-- POSTCODE_POLYGON.mid + --- POSTCODE_POLYGON.mif + +First, lets inspect this file set using the following command: + + $ ogrinfo -so vicgrid94/mif/lga_polygon/macedon\ ranges/VMADMIN/POSTCODE_POLYGON.mid POSTCODE_POLYGON + +The output will look like the following: + + Had to open data source read-only. + INFO: Open of `vicgrid94/mif/lga_polygon/macedon ranges/VMADMIN/POSTCODE_POLYGON.mid' + using driver `MapInfo File' successful. + + Layer name: POSTCODE_POLYGON + Geometry: 3D Unknown (any) + Feature Count: 26 + Extent: (2413931.249367, 2400162.366186) - (2508952.174431, 2512183.046927) + Layer SRS WKT: + PROJCS["unnamed", + GEOGCS["unnamed", + DATUM["GDA94", + SPHEROID["GRS 80",6378137,298.257222101], + TOWGS84[0,0,0,-0,-0,-0,0]], + PRIMEM["Greenwich",0], + UNIT["degree",0.0174532925199433]], + PROJECTION["Lambert_Conformal_Conic_2SP"], + PARAMETER["standard_parallel_1",-36], + PARAMETER["standard_parallel_2",-38], + PARAMETER["latitude_of_origin",-37], + PARAMETER["central_meridian",145], + PARAMETER["false_easting",2500000], + PARAMETER["false_northing",2500000], + UNIT["Meter",1]] + PFI: String (10.0) + POSTCODE: String (4.0) + FEATURE_TYPE: String (6.0) + FEATURE_QUALITY_ID: String (20.0) + PFI_CREATED: Date (10.0) + UFI: Real (12.0) + UFI_CREATED: Date (10.0) + UFI_OLD: Real (12.0) + +This gives you information about the number of features, the extent, the projection and the attributes of this Dataset. + +Next, lets go ahead and convert this Dataset into a shapefile by issuing the following command: + + $ ogr2ogr -t_srs EPSG:4326 postcode_polygon.shp vicgrid94/mif/lga_polygon/macedon\ ranges/VMADMIN/POSTCODE_POLYGON.mid POSTCODE_POLYGON + +Note that we have also reprojected the Dataset to the WGS84 spatial reference system with the -t_srs ogr2ogr option. + +The output of this command will look like the following: + + Warning 6: Normalized/laundered field name: 'FEATURE_TYPE' to 'FEATURE_TY' + Warning 6: Normalized/laundered field name: 'FEATURE_QUALITY_ID' to 'FEATURE_QU' + Warning 6: Normalized/laundered field name: 'PFI_CREATED' to 'PFI_CREATE' + Warning 6: Normalized/laundered field name: 'UFI_CREATED' to 'UFI_CREATE' + +This output indicates that some of the field names were truncated to fit into the constraint that attributes in shapefiles are only 10 characters long. + +You will now have a set of files that make up the postcode_polygon.shp shapefile set. We can inspect them by issuing the following command: + + $ ogrinfo -so postcode_polygon.shp postcode_polygon + +The output will look similar to the output we saw above when we inspected the MapInfo file we converted from: + + INFO: Open of `postcode_polygon.shp' + using driver `ESRI Shapefile' successful. + + Layer name: postcode_polygon + Geometry: Polygon + Feature Count: 26 + Extent: (144.030296, -37.898156) - (145.101137, -36.888878) + Layer SRS WKT: + GEOGCS["GCS_WGS_1984", + DATUM["WGS_1984", + SPHEROID["WGS_84",6378137,298.257223563]], + PRIMEM["Greenwich",0], + UNIT["Degree",0.017453292519943295]] + PFI: String (10.0) + POSTCODE: String (4.0) + FEATURE_TY: String (6.0) + FEATURE_QU: String (20.0) + PFI_CREATE: Date (10.0) + UFI: Real (12.0) + UFI_CREATE: Date (10.0) + UFI_OLD: Real (12.0) + +These files can now be loaded into your GeoNode instance via the normal uploader. + +Visit the upload page in your GeoNode, drag and drop the files that composes the shapefile that you have generated using the GDAL ogr2ogr command (postcode_polygon.dbf, postcode_polygon.prj, postcode_polygon.shp, postcode_polygon.shx). Give the permissions as needed and then click the \"Upload files\" button. + +![](img/upload_shapefile.png){.align-center} + +As soon as the import process completes, you will have the possibility to go straight to the Dataset info page (\"Layer Info\" button), or to edit the metadata for that Dataset (\"Edit Metadata\" button), or to manage the styles for that Dataset (\"Manage Styles\"). + +![](img/layer_info_vector.png){.align-center} + +### GDAL (Raster Data) + +Let\'s see several examples on how to either convert raster data into different formats and/or process it to get the best performances. + +References: + +a) +b) + +#### Raster Data Conversion: Arc/Info Binary and ASCII Grid data into GeoTIFF format. + +Let\'s assume we have a sample ASCII Grid file compressed as an archive. + +``` shell +# Un-tar the files +tar -xvf sample_asc.tar +``` + +You will be left with the following files on your filesystem: + +``` shell +|-- batemans_ele +|   |-- dblbnd.adf +|   |-- hdr.adf +|   |-- metadata.xml +|   |-- prj.adf +|   |-- sta.adf +|   |-- w001001.adf +|   |-- w001001x.adf +|-- batemans_elevation.asc +``` + +The file `batemans_elevation.asc` is an Arc/Info ASCII Grid file and the files in +the batemans_ele directory are an Arc/Info Binary Grid file. + +You can use the `gdalinfo` command to inspect both of these files by executing the +following command: + +``` shell +gdalinfo batemans_elevation.asc +``` + +The output should look like the following: + +``` shell +Driver: AAIGrid/Arc/Info ASCII Grid +Files: batemans_elevation.asc +Size is 155, 142 +Coordinate System is `' +Origin = (239681.000000000000000,6050551.000000000000000) +Pixel Size = (100.000000000000000,-100.000000000000000) +Corner Coordinates: +Upper Left ( 239681.000, 6050551.000) +Lower Left ( 239681.000, 6036351.000) +Upper Right ( 255181.000, 6050551.000) +Lower Right ( 255181.000, 6036351.000) +Center ( 247431.000, 6043451.000) +Band 1 Block=155x1 Type=Float32, ColorInterp=Undefined + NoData Value=-9999 +``` + +You can then inspect the batemans_ele files by executing the following command: + +``` shell +gdalinfo batemans_ele +``` + +And this should be the corresponding output: + +``` shell +Driver: AIG/Arc/Info Binary Grid +Files: batemans_ele + batemans_ele/dblbnd.adf + batemans_ele/hdr.adf + batemans_ele/metadata.xml + batemans_ele/prj.adf + batemans_ele/sta.adf + batemans_ele/w001001.adf + batemans_ele/w001001x.adf +Size is 155, 142 +Coordinate System is: +PROJCS["unnamed", + GEOGCS["GDA94", + DATUM["Geocentric_Datum_of_Australia_1994", + SPHEROID["GRS 1980",6378137,298.257222101, + AUTHORITY["EPSG","7019"]], + TOWGS84[0,0,0,0,0,0,0], + AUTHORITY["EPSG","6283"]], + PRIMEM["Greenwich",0, + AUTHORITY["EPSG","8901"]], + UNIT["degree",0.0174532925199433, + AUTHORITY["EPSG","9122"]], + AUTHORITY["EPSG","4283"]], + PROJECTION["Transverse_Mercator"], + PARAMETER["latitude_of_origin",0], + PARAMETER["central_meridian",153], + PARAMETER["scale_factor",0.9996], + PARAMETER["false_easting",500000], + PARAMETER["false_northing",10000000], + UNIT["METERS",1]] +Origin = (239681.000000000000000,6050551.000000000000000) +Pixel Size = (100.000000000000000,-100.000000000000000) +Corner Coordinates: +Upper Left ( 239681.000, 6050551.000) (150d 7'28.35"E, 35d39'16.56"S) +Lower Left ( 239681.000, 6036351.000) (150d 7'11.78"E, 35d46'56.89"S) +Upper Right ( 255181.000, 6050551.000) (150d17'44.07"E, 35d39'30.83"S) +Lower Right ( 255181.000, 6036351.000) (150d17'28.49"E, 35d47'11.23"S) +Center ( 247431.000, 6043451.000) (150d12'28.17"E, 35d43'13.99"S) +Band 1 Block=256x4 Type=Float32, ColorInterp=Undefined + Min=-62.102 Max=142.917 +NoData Value=-3.4028234663852886e+38 +``` + +You will notice that the `batemans_elevation.asc` file does *not* contain projection information while the `batemans_ele` file does. +Because of this, let\'s use the `batemans_ele` files for this exercise and convert them to a GeoTiff for use in GeoNode. +We will also reproject this file into WGS84 in the process. This can be accomplished with the following command. + +``` shell +gdalwarp -t_srs EPSG:4326 batemans_ele batemans_ele.tif +``` + +The output will show you the progress of the conversion and when it is complete, +you will be left with a `batemans_ele.tif` file that you can upload to your GeoNode. + +You can inspect this file with the gdalinfo command: + +``` shell +gdalinfo batemans_ele.tif +``` + +Which will produce the following output: + +``` shell +Driver: GTiff/GeoTIFF +Files: batemans_ele.tif +Size is 174, 130 +Coordinate System is: +GEOGCS["WGS 84", + DATUM["WGS_1984", + SPHEROID["WGS 84",6378137,298.257223563, + AUTHORITY["EPSG","7030"]], + AUTHORITY["EPSG","6326"]], + PRIMEM["Greenwich",0], + UNIT["degree",0.0174532925199433], + AUTHORITY["EPSG","4326"]] +Origin = (150.119938943722502,-35.654598806259330) +Pixel Size = (0.001011114155919,-0.001011114155919) +Metadata: + AREA_OR_POINT=Area +Image Structure Metadata: + INTERLEAVE=BAND +Corner Coordinates: +Upper Left ( 150.1199389, -35.6545988) (150d 7'11.78"E, 35d39'16.56"S) +Lower Left ( 150.1199389, -35.7860436) (150d 7'11.78"E, 35d47' 9.76"S) +Upper Right ( 150.2958728, -35.6545988) (150d17'45.14"E, 35d39'16.56"S) +Lower Right ( 150.2958728, -35.7860436) (150d17'45.14"E, 35d47' 9.76"S) +Center ( 150.2079059, -35.7203212) (150d12'28.46"E, 35d43'13.16"S) +Band 1 Block=174x11 Type=Float32, ColorInterp=Gray +``` + +#### Raster Data Optimization: Optimizing and serving big raster data + +(ref: ) + +When dealing with big raster datasets it could be very useful to use tiles. + +Tiling allows large raster datasets to be broken-up into manageable pieces and are fundamental +in defining and implementing a higher level raster I/O interface. + +In this example we will use the original dataset of the `chiangMai_ortho_optimized` public raster Dataset which +is currently available on the Thai [CHIANG MAI Urban Flooding GeoNode platform](https://urbanflooding.geo-solutions.it/). + +This dataset contains an orthorectified image stored as RGBa GeoTiff with 4 bands, +three bands for the RGB and one for transparency (the alpha channel). + +Calling the gdalinfo command to see detailed information: + +``` shell +gdalinfo chiangMai_ortho.tif +``` + +It will produce the following results: + +``` shell +Driver: GTiff/GeoTIFF +Files: chiangMai_ortho.tif +Size is 63203, 66211 +Coordinate System is: +PROJCS["WGS 84 / UTM zone 47N", + GEOGCS["WGS 84", + DATUM["WGS_1984", + SPHEROID["WGS 84",6378137,298.257223563, + AUTHORITY["EPSG","7030"]], + AUTHORITY["EPSG","6326"]], + PRIMEM["Greenwich",0, + AUTHORITY["EPSG","8901"]], + UNIT["degree",0.0174532925199433, + AUTHORITY["EPSG","9122"]], + AUTHORITY["EPSG","4326"]], + PROJECTION["Transverse_Mercator"], + PARAMETER["latitude_of_origin",0], + PARAMETER["central_meridian",99], + PARAMETER["scale_factor",0.9996], + PARAMETER["false_easting",500000], + PARAMETER["false_northing",0], + UNIT["metre",1, + AUTHORITY["EPSG","9001"]], + AXIS["Easting",EAST], + AXIS["Northing",NORTH], + AUTHORITY["EPSG","32647"]] +Origin = (487068.774750000040513,2057413.889810000080615) +Pixel Size = (0.028850000000000,-0.028850000000000) +Metadata: +AREA_OR_POINT=Area +TIFFTAG_SOFTWARE=pix4dmapper +Image Structure Metadata: +COMPRESSION=LZW +INTERLEAVE=PIXEL +Corner Coordinates: +Upper Left ( 487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N) +Lower Left ( 487068.775, 2055503.702) ( 98d52'38.77"E, 18d35'25.19"N) +Upper Right ( 488892.181, 2057413.890) ( 98d53'40.94"E, 18d36'27.38"N) +Lower Right ( 488892.181, 2055503.702) ( 98d53'40.98"E, 18d35'25.22"N) +Center ( 487980.478, 2056458.796) ( 98d53' 9.85"E, 18d35'56.28"N) +Band 1 Block=63203x1 Type=Byte, ColorInterp=Red +NoData Value=-10000 +Mask Flags: PER_DATASET ALPHA +Band 2 Block=63203x1 Type=Byte, ColorInterp=Green +NoData Value=-10000 +Mask Flags: PER_DATASET ALPHA +Band 3 Block=63203x1 Type=Byte, ColorInterp=Blue +NoData Value=-10000 +Mask Flags: PER_DATASET ALPHA +Band 4 Block=63203x1 Type=Byte, ColorInterp=Alpha +NoData Value=-10000 +``` + +As you can see, this GeoTiff has not been tiled. For accessing subsets though, tiling can make a difference. With tiling, data are stored and compressed in blocks (tiled) rather than line by line (stripped). + +In the command output above it is visible that each band has blocks with the same width of the image (63203) and a unit length. The grids in the picture below show an image with equally sized tiles (left) and the same number of strips (right). To read data from the red subset, the intersected area will have to be decompressed. + +> ![](img/tiled_vs_stripped.png){.align-center} + +In the tiled image we will have to decompress only 16 tiles, whereas in the stripped image on the right we'll have to decompress many more strips. + +Drone images data usually have a stripped structure so, in most cases, they need to be optimized to increase performances. + +Let's take a look at the gdal_translate command used to optimize our GeoTiff: + +``` shell +gdal_translate -co TILED=YES -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR + --config GDAL_TIFF_INTERNAL_MASK YES -b 1 -b 2 -b 3 -mask 4 + chiangMai_ortho.tif + chiangMai_ortho_optimized.tif +``` + +::: note +::: title +Note +::: + +For the details about the command parameters see +::: + +Once the process ended, call the gdalinfo command on the resulting tif file: + +``` shell +gdalinfo chiangMai_ortho_optimized.tif +``` + +The following should be the results: + +``` shell +Driver: GTiff/GeoTIFF +Files: chiangMai_ortho_optimized.tif +Size is 63203, 66211 +Coordinate System is: +PROJCS["WGS 84 / UTM zone 47N", + GEOGCS["WGS 84", + DATUM["WGS_1984", + SPHEROID["WGS 84",6378137,298.257223563, + AUTHORITY["EPSG","7030"]], + AUTHORITY["EPSG","6326"]], + PRIMEM["Greenwich",0, + AUTHORITY["EPSG","8901"]], + UNIT["degree",0.0174532925199433, + AUTHORITY["EPSG","9122"]], + AUTHORITY["EPSG","4326"]], + PROJECTION["Transverse_Mercator"], + PARAMETER["latitude_of_origin",0], + PARAMETER["central_meridian",99], + PARAMETER["scale_factor",0.9996], + PARAMETER["false_easting",500000], + PARAMETER["false_northing",0], + UNIT["metre",1, + AUTHORITY["EPSG","9001"]], + AXIS["Easting",EAST], + AXIS["Northing",NORTH], + AUTHORITY["EPSG","32647"]] +Origin = (487068.774750000040513,2057413.889810000080615) +Pixel Size = (0.028850000000000,-0.028850000000000) +Metadata: +AREA_OR_POINT=Area +TIFFTAG_SOFTWARE=pix4dmapper +Image Structure Metadata: +COMPRESSION=YCbCr JPEG +INTERLEAVE=PIXEL +SOURCE_COLOR_SPACE=YCbCr +Corner Coordinates: +Upper Left ( 487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N) +Lower Left ( 487068.775, 2055503.702) ( 98d52'38.77"E, 18d35'25.19"N) +Upper Right ( 488892.181, 2057413.890) ( 98d53'40.94"E, 18d36'27.38"N) +Lower Right ( 488892.181, 2055503.702) ( 98d53'40.98"E, 18d35'25.22"N) +Center ( 487980.478, 2056458.796) ( 98d53' 9.85"E, 18d35'56.28"N) +Band 1 Block=256x256 Type=Byte, ColorInterp=Red +NoData Value=-10000 +Mask Flags: PER_DATASET +Band 2 Block=256x256 Type=Byte, ColorInterp=Green +NoData Value=-10000 +Mask Flags: PER_DATASET +Band 3 Block=256x256 Type=Byte, ColorInterp=Blue +NoData Value=-10000 +Mask Flags: PER_DATASET +``` + +Our GeoTiff is now tiled with 256x256 tiles, has 3 bands and a 1-bit mask for nodata. + +We can also add internal overviews to the file using the gdaladdo command: + +``` shell +gdaladdo -r average chiangMai_ortho_optimized.tif 2 4 8 16 32 64 128 256 512 +``` + +Overviews are duplicate versions of your original data, but resampled to a lower resolution, they can also be compressed with various algorithms, much in the same way as the original dataset. + +By default, overviews take the same compression type and transparency masks of the input dataset (applied through the gdal_translate command), so the parameters to be specified are: + +> - `-r average`{.interpreted-text role="guilabel"}: computes the average of all non-NODATA contributing pixels +> - `2 4 8 16 32 64 128 256 512`{.interpreted-text role="guilabel"}: the list of integral overview levels to build (from gdal version 2.3 levels are no longer required to build overviews) + +Calling the gdalinfo command again: + +``` shell +gdalinfo chiangMai_ortho_optimized.tif +``` + +It results in: + +``` shell +Driver: GTiff/GeoTIFF +Files: chiangMai_ortho_optimized.tif +Size is 63203, 66211 +Coordinate System is: +PROJCS["WGS 84 / UTM zone 47N", + GEOGCS["WGS 84", + DATUM["WGS_1984", + SPHEROID["WGS 84",6378137,298.257223563, + AUTHORITY["EPSG","7030"]], + AUTHORITY["EPSG","6326"]], + PRIMEM["Greenwich",0, + AUTHORITY["EPSG","8901"]], + UNIT["degree",0.0174532925199433, + AUTHORITY["EPSG","9122"]], + AUTHORITY["EPSG","4326"]], + PROJECTION["Transverse_Mercator"], + PARAMETER["latitude_of_origin",0], + PARAMETER["central_meridian",99], + PARAMETER["scale_factor",0.9996], + PARAMETER["false_easting",500000], + PARAMETER["false_northing",0], + UNIT["metre",1, + AUTHORITY["EPSG","9001"]], + AXIS["Easting",EAST], + AXIS["Northing",NORTH], + AUTHORITY["EPSG","32647"]] +Origin = (487068.774750000040513,2057413.889810000080615) +Pixel Size = (0.028850000000000,-0.028850000000000) +Metadata: +AREA_OR_POINT=Area +TIFFTAG_SOFTWARE=pix4dmapper +Image Structure Metadata: +COMPRESSION=YCbCr JPEG +INTERLEAVE=PIXEL +SOURCE_COLOR_SPACE=YCbCr +Corner Coordinates: +Upper Left ( 487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N) +Lower Left ( 487068.775, 2055503.702) ( 98d52'38.77"E, 18d35'25.19"N) +Upper Right ( 488892.181, 2057413.890) ( 98d53'40.94"E, 18d36'27.38"N) +Lower Right ( 488892.181, 2055503.702) ( 98d53'40.98"E, 18d35'25.22"N) +Center ( 487980.478, 2056458.796) ( 98d53' 9.85"E, 18d35'56.28"N) +Band 1 Block=256x256 Type=Byte, ColorInterp=Red +NoData Value=-10000 +Overviews: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130 +Mask Flags: PER_DATASET +Overviews of mask band: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130 +Band 2 Block=256x256 Type=Byte, ColorInterp=Green +NoData Value=-10000 +Overviews: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130 +Mask Flags: PER_DATASET +Overviews of mask band: 31602x3Results in:3106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130 +Band 3 Block=256x256 Type=Byte, ColorInterp=Blue +NoData Value=-10000 +Overviews: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130 +Mask Flags: PER_DATASET +Overviews of mask band: 31602x33106, 15801x16553, 7901x8277, 3951x4139, 1976x2070, 988x1035, 494x518, 247x259, 124x130 +``` + +Notice that the transparency masks of internal overviews have been applied (their compression does not show up in the file metadata). + +UAVs usually provide also two other types of data: `DTM (Digital Terrain Model)` and `DSM (Digital Surface Model)`. + +Those data require different processes to be optimized. Let's look at some examples to better understand how to use gdal to accomplish that task. + +From the [CHIANG MAI Urban Flooding GeoNode platform](https://urbanflooding.geo-solutions.it/) platform it is currently available the `chiangMai_dtm_optimized` Dataset, +let's download its original dataset. + +This dataset should contain the DTM file `chiangMai_dtm.tif`. + +Calling the gdalinfo command on it: + +``` shell +gdalinfo chiangMai_dtm.tif +``` + +The following information will be displayed: + +``` shell +Driver: GTiff/GeoTIFF +Files: chiangMai_dtm.tif +Size is 12638, 13240 +Coordinate System is: +PROJCS["WGS 84 / UTM zone 47N", + GEOGCS["WGS 84", + DATUM["WGS_1984", + SPHEROID["WGS 84",6378137,298.257223563, + AUTHORITY["EPSG","7030"]], + AUTHORITY["EPSG","6326"]], + PRIMEM["Greenwich",0, + AUTHORITY["EPSG","8901"]], + UNIT["degree",0.0174532925199433, + AUTHORITY["EPSG","9122"]], + AUTHORITY["EPSG","4326"]], + PROJECTION["Transverse_Mercator"], + PARAMETER["latitude_of_origin",0], + PARAMETER["central_meridian",99], + PARAMETER["scale_factor",0.9996], + PARAMETER["false_easting",500000], + PARAMETER["false_northing",0], + UNIT["metre",1, + AUTHORITY["EPSG","9001"]], + AXIS["Easting",EAST], + AXIS["Northing",NORTH], + AUTHORITY["EPSG","32647"]] +Origin = (487068.774750000040513,2057413.889810000080615) +Pixel Size = (0.144270000000000,-0.144270000000000) +Metadata: +AREA_OR_POINT=Area +TIFFTAG_SOFTWARE=pix4dmapper +Image Structure Metadata: +COMPRESSION=LZW +INTERLEAVE=BAND +Corner Coordinates: +Upper Left ( 487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N) +Lower Left ( 487068.775, 2055503.755) ( 98d52'38.77"E, 18d35'25.19"N) +Upper Right ( 488892.059, 2057413.890) ( 98d53'40.94"E, 18d36'27.37"N) +Lower Right ( 488892.059, 2055503.755) ( 98d53'40.98"E, 18d35'25.22"N) +Center ( 487980.417, 2056458.822) ( 98d53' 9.85"E, 18d35'56.28"N) +Band 1 Block=12638x1 Type=Float32, ColorInterp=Gray +NoData Value=-10000 +``` + +Reading this image could be very slow because it has not been tiled yet. So, as discussed above, its data need to be stored and compressed in tiles to increase performances. + +The following gdal_translate command should be appropriate for that purpose: + +``` shell +gdal_translate -co TILED=YES -co COMPRESS=DEFLATE chiangMai_dtm.tif chiangMai_dtm_optimized.tif +``` + +When the data to compress consists of imagery (es. aerial photographs, true-color satellite images, or colored maps) you can use lossy algorithms such as JPEG. We are now compressing data where the precision is important, the band data type is Float32 and elevation values should not be altered, so a lossy algorithm such as JPEG is not suitable. JPEG should generally only be used with Byte data (8 bit per channel) so we have choosen the lossless DEFLATE compression through the COMPRESS=DEFLATE creation option. + +Calling the gdalinfo command again: + +``` shell +gdalinfo chiangMai_dtm_optimized.tif +``` + +We can observe the following results: + +``` shell +Driver: GTiff/GeoTIFF +Files: chiangMai_dtm_optimized.tif +Size is 12638, 13240 +Coordinate System is: +PROJCS["WGS 84 / UTM zone 47N", + GEOGCS["WGS 84", + DATUM["WGS_1984", + SPHEROID["WGS 84",6378137,298.257223563, + AUTHORITY["EPSG","7030"]], + AUTHORITY["EPSG","6326"]], + PRIMEM["Greenwich",0, + AUTHORITY["EPSG","8901"]], + UNIT["degree",0.0174532925199433, + AUTHORITY["EPSG","9122"]], + AUTHORITY["EPSG","4326"]], + PROJECTION["Transverse_Mercator"], + PARAMETER["latitude_of_origin",0], + PARAMETER["central_meridian",99], + PARAMETER["scale_factor",0.9996], + PARAMETER["false_easting",500000], + PARAMETER["false_northing",0], + UNIT["metre",1, + AUTHORITY["EPSG","9001"]], + AXIS["Easting",EAST], + AXIS["Northing",NORTH], + AUTHORITY["EPSG","32647"]] +Origin = (487068.774750000040513,2057413.889810000080615) +Pixel Size = (0.144270000000000,-0.144270000000000) +Metadata: +AREA_OR_POINT=Area +TIFFTAG_SOFTWARE=pix4dmapper +Image Structure Metadata: +COMPRESSION=DEFLATE +INTERLEAVE=BAND +Corner Coordinates: +Upper Left ( 487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N) +Lower Left ( 487068.775, 2055503.755) ( 98d52'38.77"E, 18d35'25.19"N) +Upper Right ( 488892.059, 2057413.890) ( 98d53'40.94"E, 18d36'27.37"N) +Lower Right ( 488892.059, 2055503.755) ( 98d53'40.98"E, 18d35'25.22"N) +Center ( 487980.417, 2056458.822) ( 98d53' 9.85"E, 18d35'56.28"N) +Band 1 Block=256x256 Type=Float32, ColorInterp=Gray +NoData Value=-10000 +``` + +We need also to create overviews through the gdaladdo command: + +``` shell +gdaladdo -r nearest chiangMai_dtm_optimized.tif 2 4 8 16 32 64 +``` + +Unlike the previous example, overviews will be created with the **nearest resampling algorithm**. That is due to the nature of the data we are representing: we should not consider the average between two elevation values but simply the closer one, it is more reliable regarding the conservation of the original data. + +Calling the gdalinfo command again: + +``` shell +gdalinfo chiangMai_dtm_optimized.tif +``` + +We can see the following information: + +``` shell +Driver: GTiff/GeoTIFF +Files: chiangMai_dtm_optimized.tif +Size is 12638, 13240 +Coordinate System is: +PROJCS["WGS 84 / UTM zone 47N", + GEOGCS["WGS 84", + DATUM["WGS_1984", + SPHEROID["WGS 84",6378137,298.257223563, + AUTHORITY["EPSG","7030"]], + AUTHORITY["EPSG","6326"]], + PRIMEM["Greenwich",0, + AUTHORITY["EPSG","8901"]], + UNIT["degree",0.0174532925199433, + AUTHORITY["EPSG","9122"]], + AUTHORITY["EPSG","4326"]], + PROJECTION["Transverse_Mercator"], + PARAMETER["latitude_of_origin",0], + PARAMETER["central_meridian",99], + PARAMETER["scale_factor",0.9996], + PARAMETER["false_easting",500000], + PARAMETER["false_northing",0], + UNIT["metre",1, + AUTHORITY["EPSG","9001"]], + AXIS["Easting",EAST], + AXIS["Northing",NORTH], + AUTHORITY["EPSG","32647"]] +Origin = (487068.774750000040513,2057413.889810000080615) +Pixel Size = (0.144270000000000,-0.144270000000000) +Metadata: +AREA_OR_POINT=Area +TIFFTAG_SOFTWARE=pix4dmapper +Image Structure Metadata: +COMPRESSION=DEFLATE +INTERLEAVE=BAND +Corner Coordinates: +Upper Left ( 487068.775, 2057413.890) ( 98d52'38.72"E, 18d36'27.34"N) +Lower Left ( 487068.775, 2055503.755) ( 98d52'38.77"E, 18d35'25.19"N) +Upper Right ( 488892.059, 2057413.890) ( 98d53'40.94"E, 18d36'27.37"N) +Lower Right ( 488892.059, 2055503.755) ( 98d53'40.98"E, 18d35'25.22"N) +Center ( 487980.417, 2056458.822) ( 98d53' 9.85"E, 18d35'56.28"N) +Band 1 Block=256x256 Type=Float32, ColorInterp=Gray +NoData Value=-10000 +Overviews: 6319x6620, 3160x3310, 1580x1655, 790x828, 395x414, 198x207 +``` + +Overviews have been created. By default, they inherit the same compression type of the original dataset (there is no evidence of it in the gdalinfo output). + +### Other Raster Data Use Cases + +- [Serving a large number of GrayScale GeoTiff with Palette](https://geoserver.geo-solutions.it/edu/en/raster_data/advanced_gdal/example1.html) +- [Serving a large number of DTM ASCII Grid Files](https://geoserver.geo-solutions.it/edu/en/raster_data/advanced_gdal/example2.html) +- [Serving a large number of Cartographic Black/White GeoTiff with Palette](https://geoserver.geo-solutions.it/edu/en/raster_data/advanced_gdal/example3.html) +- [Serving a large number of satellite/aerial RGB GeoTiff with compression](https://geoserver.geo-solutions.it/edu/en/raster_data/advanced_gdal/example4.html) +- [Optimizing and serving UAV data](https://geoserver.geo-solutions.it/edu/en/raster_data/advanced_gdal/example5.html) +- [Optimizing and serving 16-bits satellite/aerial RGB GeoTiff](https://geoserver.geo-solutions.it/edu/en/raster_data/advanced_gdal/example6.html) + +### Process Raster Datasets Programmatically + +In this section we will provide a set of `shell`{.interpreted-text role="guilabel"} scripts which might be very useful to batch process a lot of raster datasets programmatically. + +1. `process_gray.sh` + + > ``` shell + > for filename in *.tif*; do echo gdal_translate -co TILED=YES -co COMPRESS=DEFLATE $filename ${filename//.tif/.optimized.tif}; done > gdal_translate.sh + > chmod +x gdal_translate.sh + > ./gdal_translate.sh + > ``` + > + > ``` shell + > for filename in *.optimized.tif*; do echo gdaladdo -r nearest $filename 2 4 8 16 32 64 128 256 512; done > gdaladdo.sh + > for filename in *.optimized.tif*; do echo mv \"$filename\" \"${filename//.optimized.tif/.tif}\"; done > rename.sh + > chmod +x *.sh + > ./gdaladdo.sh + > ./rename.sh + > ``` + +2. `process_rgb.sh` + + > ``` shell + > for filename in *.tif*; do echo gdal_translate -co TILED=YES -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR -b 1 -b 2 -b 3 $filename ${filename//.tif/.optimized.tif}; done > gdal_translate.sh + > chmod +x gdal_translate.sh + > ./gdal_translate.sh + > ``` + > + > ``` shell + > for filename in *.optimized.tif*; do echo gdaladdo -r average $filename 2 4 8 16 32 64 128 256 512; done > gdaladdo.sh + > for filename in *.optimized.tif*; do echo mv \"$filename\" \"${filename//.optimized.tif/.tif}\"; done > rename.sh + > chmod +x *.sh + > ./gdaladdo.sh + > ./rename.sh + > ``` + +3. `process_rgb_alpha.sh` + + > ``` shell + > for filename in *.tif*; do echo gdal_translate -co TILED=YES -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR --config GDAL_TIFF_INTERNAL_MASK YES -b 1 -b 2 -b 3 -mask 4 $filename ${filename//.tif/.optimized.tif}; done > gdal_translate.sh + > chmod +x gdal_translate.sh + > ./gdal_translate.sh + > ``` + > + > ``` shell + > for filename in *.optimized.tif*; do echo gdaladdo -r average $filename 2 4 8 16 32 64 128 256 512; done > gdaladdo.sh + > for filename in *.optimized.tif*; do echo mv \"$filename\" \"${filename//.optimized.tif/.tif}\"; done > rename.sh + > chmod +x *.sh + > ./gdaladdo.sh + > ./rename.sh + > ``` + +4. `process_rgb_palette.sh` + + > ``` shell + > for filename in *.tif*; do echo gdal_translate -co TILED=YES -co COMPRESS=DEFLATE $filename ${filename//.tif/.optimized.tif}; done > gdal_translate.sh + > chmod +x gdal_translate.sh + > ./gdal_translate.sh + > ``` + > + > ``` shell + > for filename in *.optimized.tif*; do echo gdaladdo -r average $filename 2 4 8 16 32 64 128 256 512; done > gdaladdo.sh + > for filename in *.optimized.tif*; do echo mv \"$filename\" \"${filename//.optimized.tif/.tif}\"; done > rename.sh + > chmod +x *.sh + > ./gdaladdo.sh + > ./rename.sh + > ``` + +# Thesaurus Import and Export {#createsuperuser} + +See `load_thesaurus`{.interpreted-text role="ref"} and `dump_thesaurus`{.interpreted-text role="ref"}. + +# Create Users and Super Users + +Your first step will be to create a user. There are three options to do so, depending on which kind of user you want to create you may +choose a different option. We will start with creating a *superuser*, because this user is the most important. A superuser +has all the permissions without explicitly assigning them. + +The easiest way to create a superuser (in linux) is to open your terminal and type: + +> ``` shell +> $ DJANGO_SETTINGS_MODULE=geonode.settings python manage.py createsuperuser +> ``` +> +> ::: note +> ::: title +> Note +> ::: +> +> If you enabled `local_settings.py` the command will change as following: +> +> ``` shell +> $ DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py createsuperuser +> ``` +> ::: + +You will be asked a username (in this tutorial we will call the superuser you now create *your_superuser*), an email address and a password. + +Now you\'ve created a superuser you should become familiar with the *Django Admin Interface*. As a superuser you are having +access to this interface, where you can manage users, Datasets, permission and more. To learn more detailed about this interface +check this LINK. For now it will be enough to just follow the steps. To attend the *Django Admin Interface*, go to your geonode website and *sign in* with *your_superuser*. Once you\'ve logged in, the name of your user will appear on the top right. Click on it and the following menu +will show up: + +> ![](img/menu_admin.png){.align-center} + +Clicking on *Admin* causes the interface to show up. + +> ![](img/admin_interface.png){.align-center} + +Go to *Auth* -\> *Users* and you will see all the users that exist at the moment. In your case it will only be *your_superuser*. Click on it, and you will see a section on *Personal Info*, one on *Permissions* and one on *Important dates*. For the moment, the section on *Permissions* is the most important. + +> ![](img/permissions_django_admin.png){.align-center} + +As you can see, there are three boxes that can be checked and unchecked. Because you\'ve created a superuser, all three boxes +are checked as default. If only the box *active* would have been checked, the user would not be a superuser and would not be able to +access the *Django Admin Interface* (which is only available for users with the *staff* status). Therefore keep the following +two things in mind: + +- a superuser is able to access the *Django Admin Interface* and he has all permissions on the data uploaded to GeoNode. +- an ordinary user (created from the GeoNode interface) only has *active* permissions by default. The user will not have the ability to access the *Django Admin Interface* and certain permissions have to be added for him. + +Until now we\'ve only created superusers. So how do you create an ordinary user? You have two options: + +1. Django Admin Interface + + First we will create a user via the *Django Admin Interface* because we\'ve still got it open. Therefore go back to *Auth* -\> *Users* and you should find a button on the right that says *Add user*. + + ![](img/add_user.png){.align-center} + + Click on it and a form to fill out will appear. Name the new user test_user, choose a password and click *save* at the right bottom of the site. + + ![](img/add_test_user.png){.align-center} + + Now you should be directed to the site where you could + change the permissions on the user *test_user*. As default only *active* is checked. If you want this user also to be able to attend this admin interface + you could also check *staff status*. But for now we leave the settings as they are! + + ::: todo + groups and permissions! + ::: + + To test whether the new user was successfully created, go back to the GeoNode web page and try to sign in. + +2. GeoNode website + + To create an ordinary user you could also just use the GeoNode website. If you installed GeoNode using a release, you should + + : see a *Register* button on the top, beside the *Sign in* button (you might have to log out before). + + ![](img/register.png){.align-center} + + Hit the button and again a form will appear for you to fill out. This user will be named *geonode_user* + + ![](img/sign_up_test_user.png){.align-center} + + ::: todo + NEW IMAGE WITH GEONODE USER! + ::: + + By hitting *Sign up* the user will be signed up, as default only with the status *active*. + +# Batch Sync Permissions {#batch_sync_permissions} + +GeoNode provides a very useful management command `set_layers_permisions` allowing an administrator to easily add / remove permissions to groups and users on one or more Datasets. + +The `set_layers_permisions` command arguments are: + +- **permissions** to set/unset \--\> read, download, edit, manage + + > ``` python + > READ_PERMISSIONS = [ + > 'view_resourcebase' + > ] + > DOWNLOAD_PERMISSIONS = [ + > 'view_resourcebase', + > 'download_resourcebase' + > ] + > EDIT_PERMISSIONS = [ + > 'view_resourcebase', + > 'change_dataset_style', + > 'download_resourcebase', + > 'change_resourcebase_metadata', + > 'change_dataset_data', + > 'change_resourcebase' + > ] + > MANAGE_PERMISSIONS = [ + > 'delete_resourcebase', + > 'change_resourcebase', + > 'view_resourcebase', + > 'change_resourcebase_permissions', + > 'change_dataset_style', + > 'change_resourcebase_metadata', + > 'publish_resourcebase', + > 'change_dataset_data', + > 'download_resourcebase' + > ] + > ``` + +NB: the above permissions list may change with the ADVANCED_WORKFLOW enabled. For additional info: + +- **resources** (Datasets) which permissions will be assigned on \--\> type the Dataset id, multiple choices can be typed with comma separator, if no ids are provided all the Datasets will be considered +- **users** who permissions will be assigned to, multiple choices can be typed with a comma separator +- **groups** who permissions will be assigned to, multiple choices can be typed with a comma separator +- **delete** flag (optional) which means the permissions will be unset + +## Usage examples: + +1. Assign **edit** permissions on the Datasets with id **1** and **2** to the users **username1** and **username2** and to the group **group_name1**. + + > ``` shell + > python manage.py set_layers-permissions -p edit -u username1,username2 -g group_name1 -r 1,2 + > ``` + +2. Assign **manage** permissions on all the Datasets to the group **group_name1**. + + > ``` shell + > python manage.py set_layers-permissions -p manage -g group_C + > ``` + +3. Unset **download** permissions on the Dataset with id **1** for the user **username1**. + + > ``` shell + > python manage.py set_layers-permissions -p download -u username1 -r 1 -d + > ``` + +The same functionalities, with some limitations, are available also from the `Admin Dashboard >> Users`{.interpreted-text role="guilabel"} or `Admin Dashboard >> Groups >> Group profiles`{.interpreted-text role="guilabel"}. + +![](img/layer_batch_perms_admin.png){.align-center} + +An action named `Set layer permissions`{.interpreted-text role="guilabel"} is available from the list, redirecting the administrator to a form to set / unset read, edit, download permissions on the selected Users/group profile. + +![](img/layer_batch_perms_form.png){.align-center} + +Is enough to select the dataset and press \"Submit\". If the async mode is activated, the permission assign is asyncronous + +# Delete Certain GeoNode Resources {#delete_resources} + +The `delete_resources` `Management Command`{.interpreted-text role="guilabel"} allows to remove resources meeting a certain condition, +specified in a form of a serialized django Q() expression. + +First of all let\'s take a look at the `--help` option of the `delete_resources` +management command in order to inspect all the command options and features. + +Run + +``` shell +DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources --help +``` + +::: note +::: title +Note +::: + +If you enabled `local_settings.py` the command will change as following: + +``` shell +DJANGO_SETTINGS_MODULE=geonode.local_settings python manage.py delete_resources --help +``` +::: + +This will produce output the following output: + +``` shell +usage: manage.py delete_resources [-h] [-c CONFIG_PATH] + [-l LAYER_FILTERS [LAYER_FILTERS ...]] + [-m MAP_FILTERS [MAP_FILTERS ...]] + [-d DOCUMENT_FILTERS [DOCUMENT_FILTERS ...]] + [--version] [-v {0,1,2,3}] + [--settings SETTINGS] + [--pythonpath PYTHONPATH] [--traceback] + [--no-color] [--force-color] + +Delete resources meeting a certain condition + +optional arguments: + -h, --help show this help message and exit + -c CONFIG_PATH, --config CONFIG_PATH + Configuration file path. Default is: + delete_resources.json + -l LAYER_FILTERS [LAYER_FILTERS ...], --layer_filters LAYER_FILTERS [LAYER_FILTERS ...] + -m MAP_FILTERS [MAP_FILTERS ...], --map_filters MAP_FILTERS [MAP_FILTERS ...] + -d DOCUMENT_FILTERS [DOCUMENT_FILTERS ...], --document_filters DOCUMENT_FILTERS [DOCUMENT_FILTERS ...] + --version show program's version number and exit + -v {0,1,2,3}, --verbosity {0,1,2,3} + Verbosity level; 0=minimal output, 1=normal output, + 2=verbose output, 3=very verbose output + --settings SETTINGS The Python path to a settings module, e.g. + "myproject.settings.main". If this isn't provided, the + DJANGO_SETTINGS_MODULE environment variable will be + used. + --pythonpath PYTHONPATH + A directory to add to the Python path, e.g. + "/home/djangoprojects/myproject". + --traceback Raise on CommandError exceptions + --no-color Don't colorize the command output. + --force-color Force colorization of the command output. +``` + +There are two ways to declare Q() expressions filtering which resources should be deleted: + +1. With a JSON configuration file: passing `-c` argument specifying the path to the JSON configuration file. + +- **Example 1**: Relative path to the config file (to `manage.py`) + + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources -c geonode/base/management/commands/delete_resources.json + > ``` + +- **Example 2**: Absolute path to the config file + + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources -c /home/User/Geonode/configs/delete_resources.json + > ``` + +2. With CLI: passing `-l` `-d` `-m` list arguments for each of resources (Datasets, documents, maps) + +- **Example 3**: Delete resources without configuration file + + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources -l 'Q(pk__in: [1, 2]) | Q(title__icontains:"italy")' 'Q(owner__name=admin)' -d '*' -m "Q(pk__in=[1, 2])" + > ``` + +## Configuration File + +The JSON configuration file should contain a single [filters]{.title-ref} object, which consists of [Dataset]{.title-ref}, [map]{.title-ref} and [document]{.title-ref} lists. +Each list specifies the filter conditions applied to a corresponding queryset, defining which items will be deleted. +The filters are evaluated and directly inserted into Django .filter() method, which means the filters occurring as +separated list items are treated as AND condition. To create OR query `|` operator should be used. For more info please check Django +\[documentation\]()). +The only exception is passing a list with `'*'` which will cause deleting all the queryset of the resource. + +- **Example 4**: Example content of the configuration file, which will delete Datasets with ID\'s 1, 2, and 3, those owned by [admin]{.title-ref} user, along with all defined maps. + + > ``` shell + > { + > "filters": { + > "Dataset": [ + > "Q(pk__in=[1, 2, 3]) | Q(title__icontains='italy')", + > "Q(user__name=admin)" + > ], + > "map": ["*"], + > "document": [] + > } + > } + > ``` + +## CLI + +The CLI configuration can be specified with `-l` `-d` `-m` list arguments, which in fact are a translation +of the configuration JSON file. `-l` `-d` `-m` arguments are evaluated in the same manner as filters.Dataset, +filters.map and filter.document accordingly from the Example 4. +The following example\'s result will be equivalent to Example 4: + +- **Example 5**: Example CLI configuration, which will delete Datasets with ID\'s 1, 2, and 3, along with all maps. + + > ``` shell + > DJANGO_SETTINGS_MODULE=geonode.settings python manage.py delete_resources -l 'Q(pk__in: [1, 2, 3]) | Q(title__icontains:"italy")' 'Q(owner__name=admin)' -m '*' + > ``` + +# Async execution over http + +It is possible to expose and run management commands over http. + +To run [custom django management commands](https://docs.djangoproject.com/en/3.2/howto/custom-management-commands/) usually we make make use of the command line: + +``` shell +python manage.py ping_mngmt_commands_http +$> pong +``` + +The `management_commands_http` app allows us to run commands when we have no access to the command line. +It\'s possible to run a command using the API or the django admin GUI. + +For security reasons, only admin users can access the feature and the desired command needs to be explicitly exposed. +By default the following commands are exposed: *ping_mngmt_commands_http*, *updatelayers*, *sync_geonode_datasets*, *sync_geonode_maps*, *importlayers* and *set_all_datasets_metadata*. + +To expose more command you can change the environment variable `MANAGEMENT_COMMANDS_EXPOSED_OVER_HTTP` and the added commands will be exposed in your application. + +The list of exposed commands is available by the endpoint `list_management_commands` and also presented by the form in the admin page `create management command job`. + +::: note +::: title +Note +::: + +To use the commands in an asyncronous approach `ASYNC_SIGNALS` needs to be set to `True` and celery should be running. +::: + +## Manage using django admin interface + +### Creating a job + +Access the admin panel: `http:///admin` and go to \"Management command jobs\". + +
+img/management_commands_over_http_admin.png +
Management command admin section
+
+ +You will arrive at \/en/admin/management_commands_http/managementcommandjob/, +then click on the buttton `+ Add management command job` (`http:///en/admin/management_commands_http/managementcommandjob/add/`). + +
+img/management_commands_over_http_btn_add_job.png +
Add management command job
+
+ +Select the command and fill the form, with the arguments and/or key-arguments if needed. +Save you job and in the list select the `start` action, alterantively you can mark the `autostart` option and the command will be automatic started when created. + +
+img/management_commands_over_http_add_job.png +
Creating a management command job form
+
+ +### Starting a job + +To start a job: + +
+img/management_commands_over_http_start_job.png +
Starting a job
+
+ +1. Select the job to be started. +2. Select the `start` action. +3. Click in `Go`. +4. The page will refresh and the job status will have changed. If it takes a long to run, refresh the page to see the updated the status. +5. A `stop` option is also available. + +::: note +::: title +Note +::: + +If it takes too long to load the page, `ASYNC_SIGNALS` may not be activated. +If its status gets stuck at `QUEUED`, verify if celery is running and properly configured. +::: + +### Job status + +Clicking at the link in the ID of a job, we can see the details of this job. +For the job we just created, we can verify the output message and celery job status. + +
+img/management_commands_over_http_job_status.png +
Example job status
+
+ +When we have an error during execution the traceback message will be available in the `Celery traceback`. +In the next image a `ping_mngmt_commands_http` job was created with the arguments `["--force_exception", true]`. +Checking the text in this field can be useful when troubleshooting errors. + +
+img/management_commands_over_http_job_status_with_traceback.png +
Example job traceback message
+
+ +## Manage using API endpoints + +The execution of the management commands can be handled by http requests to an API: `http:///api/v2/management/`. + +All the requests need to be authenticated with administrative permissions (*superuser*). + +You can find here a postman collection with all the exemples listed here and other available endpoints: + +`geonode_mngmt_commands.postman_collection.json `{.interpreted-text role="download"} + +### List exposed commands + +Getting a list of the exposed commands: + +``` shell +curl --location --request GET 'http:///api/v2/management/commands/' --header 'Authorization: Basic YWRtaW46YWRtaW4=' +``` + +Response: + +``` json +{ + "success": true, + "error": null, + "data": [ + "ping_mngmt_commands_http", + "updatelayers", + "set_all_datasets_metadata", + "sync_geonode_maps", + "importlayers", + "sync_geonode_datasets" + ] +} +``` + +::: note +::: title +Note +::: + +You should change the header `Authorization` (`Basic YWRtaW46YWRtaW4=`) to your Auth token, in this example I am using a token for `admin` as username and `admin` as password. +::: + +### Creating a job + +Optionally, before creating the job you can get its *help message* with the following call: + +``` shell +curl --location --request GET 'http:///api/v2/management/commands/ping_mngmt_commands_http/' --header 'Authorization: Basic YWRtaW46YWRtaW4=' +``` + +Creating a job for running `ping_mngmt_commands_http` with 30 seconds of sleep time: + +``` shell +curl --location --request POST 'http:///api/v2/management/commands/ping_mngmt_commands_http/jobs/' \ +--header 'Authorization: Basic YWRtaW46YWRtaW4=' \ +--header 'Content-Type: application/json' \ +--data-raw '{ + "args": ["--sleep", 30], + "kwargs": {}, + "autostart": false +}' +``` + +Response: + +``` json +{ + "success": true, + "error": null, + "data": { + "id": 8, + "command": "ping_mngmt_commands_http", + "app_name": "geonode.management_commands_http", + "user": 1000, + "status": "CREATED", + "created_at": "2021-10-08T18:17:25.045752Z", + "start_time": null, + "end_time": null, + "args": [ + "--sleep", + 30 + ], + "kwargs": {}, + "celery_result_id": null, + "output_message": null + } +} +``` + +::: note +::: title +Note +::: + +Alterantively you can omit the `jobs` part of the url to create a job. (Using `http:///api/v2/management/commands/ping_mngmt_commands_http/` as url) +::: + +### Start/Stop actions + +To start the created job: + +``` shell +curl --location --request PATCH 'http:///api/v2/management/jobs/8/start/' --header 'Authorization: Basic YWRtaW46YWRtaW4=' +``` + +Response: + +``` json +{ + "success": true, + "error": null, + "data": { + "id": 8, + "command": "ping_mngmt_commands_http", + "app_name": "geonode.management_commands_http", + "user": 1000, + "status": "QUEUED", + "created_at": "2021-10-08T18:17:25.045752Z", + "start_time": null, + "end_time": null, + "args": [ + "--sleep", + 30 + ], + "kwargs": {}, + "celery_result_id": null, + "output_message": null + } +} +``` + +::: note +::: title +Note +::: + +During execution the job can be interrupted using the following call: + +``` shell +curl --location --request PATCH 'http:///api/v2/management/jobs/8/stop/' --header 'Authorization: Basic YWRtaW46YWRtaW4=' +``` +::: + +Note that the status changed from **CREATED** to **QUEUED**, during execution it will be **STARTED** and at the end **FINISHED**. + +### Jobs list and status + +You can verify your job status and details with the following call: + +``` shell +curl --location --request GET 'http:///api/v2/management/jobs/8/status/' --header 'Authorization: Basic YWRtaW46YWRtaW4=' +``` + +Response: + +``` json +{ + "id": 8, + "command": "ping_mngmt_commands_http", + "app_name": "geonode.management_commands_http", + "user": 1000, + "status": "FINISHED", + "created_at": "2021-10-08T18:17:25.045752Z", + "start_time": "2021-10-08T18:20:02.761475Z", + "end_time": "2021-10-08T18:20:32.802007Z", + "args": [ + "--sleep", + 30 + ], + "kwargs": {}, + "celery_result_id": "fe7359a6-5f8c-47bf-859a-84351b5ed80c", + "output_message": "Sleeping for 30.0 seconds...\npong\n", + "celery_task_meta": { + "date_done": "2021-10-08T18:20:32.810649Z", + "status": "SUCCESS", + "traceback": null, + "worker": "worker1@4f641ffa9c0b" + } +} +``` + +When runing multiple jobs and to audit already ran jobs. A list of jobs can be retrieved using the following call: + +``` shell +curl --location --request GET 'http:///api/v2/management/jobs/' --header 'Authorization: Basic YWRtaW46YWRtaW4=' +``` + +Response: + +``` json +{ + "links": { + "next": null, + "previous": null + }, + "total": 1, + "page": 1, + "page_size": 10, + "data": [ + { + "id": 1, + "command": "ping_mngmt_commands_http", + "app_name": "geonode.management_commands_http", + "user": 1000, + "status": "FINISHED", + "created_at": "2021-10-08T18:17:25.045752Z" + } + ] +} +``` + +::: note +::: title +Note +::: + +This list can be filtered by the fields \"celery_result_id\", \"command\", \"app_name\", \"status\", \"user\" and \"user\_\_username\". +::: diff --git a/admin/thesaurus/img/thesaurus_admin_1.png b/docs/admin/thesaurus/img/thesaurus_admin_1.png old mode 100755 new mode 100644 similarity index 100% rename from admin/thesaurus/img/thesaurus_admin_1.png rename to docs/admin/thesaurus/img/thesaurus_admin_1.png diff --git a/admin/thesaurus/img/thesaurus_admin_2.png b/docs/admin/thesaurus/img/thesaurus_admin_2.png old mode 100755 new mode 100644 similarity index 100% rename from admin/thesaurus/img/thesaurus_admin_2.png rename to docs/admin/thesaurus/img/thesaurus_admin_2.png diff --git a/admin/thesaurus/img/thesaurus_admin_fail.png b/docs/admin/thesaurus/img/thesaurus_admin_fail.png old mode 100755 new mode 100644 similarity index 100% rename from admin/thesaurus/img/thesaurus_admin_fail.png rename to docs/admin/thesaurus/img/thesaurus_admin_fail.png diff --git a/admin/thesaurus/img/thesaurus_admin_list.png b/docs/admin/thesaurus/img/thesaurus_admin_list.png similarity index 100% rename from admin/thesaurus/img/thesaurus_admin_list.png rename to docs/admin/thesaurus/img/thesaurus_admin_list.png diff --git a/admin/thesaurus/img/thesaurus_admin_success.png b/docs/admin/thesaurus/img/thesaurus_admin_success.png old mode 100755 new mode 100644 similarity index 100% rename from admin/thesaurus/img/thesaurus_admin_success.png rename to docs/admin/thesaurus/img/thesaurus_admin_success.png diff --git a/admin/thesaurus/img/thesaurus_choices.png b/docs/admin/thesaurus/img/thesaurus_choices.png old mode 100755 new mode 100644 similarity index 100% rename from admin/thesaurus/img/thesaurus_choices.png rename to docs/admin/thesaurus/img/thesaurus_choices.png diff --git a/admin/thesaurus/img/thesaurus_edit_page.png b/docs/admin/thesaurus/img/thesaurus_edit_page.png old mode 100755 new mode 100644 similarity index 100% rename from admin/thesaurus/img/thesaurus_edit_page.png rename to docs/admin/thesaurus/img/thesaurus_edit_page.png diff --git a/admin/thesaurus/img/thesaurus_edit_sample.png b/docs/admin/thesaurus/img/thesaurus_edit_sample.png similarity index 100% rename from admin/thesaurus/img/thesaurus_edit_sample.png rename to docs/admin/thesaurus/img/thesaurus_edit_sample.png diff --git a/admin/thesaurus/img/thesaurus_facet_sample.png b/docs/admin/thesaurus/img/thesaurus_facet_sample.png similarity index 100% rename from admin/thesaurus/img/thesaurus_facet_sample.png rename to docs/admin/thesaurus/img/thesaurus_facet_sample.png diff --git a/docs/admin/thesaurus/index.md b/docs/admin/thesaurus/index.md new file mode 100644 index 00000000..064477bc --- /dev/null +++ b/docs/admin/thesaurus/index.md @@ -0,0 +1,310 @@ +# Thesauri + +## Introduction + +A **thesaurus** is a structured vocabulary used to manage and standardize keywords (also known as tags) that describe resources. It helps improve metadata quality, searchability, and interoperability by enforcing controlled vocabularies. + +## Key Functions + +Controlled Vocabulary + +: Instead of free-text keywords, thesauri offer predefined and **standardized terms** organized thematically (e.g., ISO 19115 topics, GEMET, INSPIRE themes). + +Semantic Consistency + +: Users tagging datasets can choose from a consistent list of terms, **reducing redundancy** and ambiguity (e.g., avoiding both \"roads\" and \"road\" as separate tags). + +Improved search and filtering + +: Thesauri enable structured tagging of datasets, allowing more accurate searches and the use of **faceted filters** to easily narrow down results. + +Localization + +: Each concept in a thesaurus can have **translations for different languages**, allowing localized display based on the user's interface language. + +Metadata Standards Integration + +: Thesauri can align with **international standards** (like ISO, INSPIRE, GEMET), which is especially important when GeoNode is used in institutional or governmental contexts. + +## Data model + +The *GeoNode thesaurus model* is designed to support multilingual, structured vocabularies. It consists of the following key components: + +Thesaurus: + +: - Represents a full controlled vocabulary (e.g., GEMET, INSPIRE themes). + - In SKOS terms, it\'s a `skos:ConceptScheme`. + +ThesaurusLabel: + +: - Stores the localized names (titles/descriptions) of a thesaurus for different languages. + - In SKOS terms, it\'s a `skos:preflabel` within the `skos:ConceptScheme`. + +ThesaurusKeyword: + +: - Represents a single concept or term within a thesaurus (e.g., \"Land Cover\", \"Transport\"), also storing the default label (used where the translation for a given requested language is not defined) and its identifying URI. + - In SKOS terms, it\'s a `skos:Concept`. + +ThesaurusKeywordLabel: + +: - Stores the multilingual labels for each keyword. + - In SKOS terms, it\'s a `skos:preflabel` within the `skos:Concept`. + +# Adding a Thesaurus {#thesaurus_add} + +A thesaurus can be added in Geonode by: + +- creating a new thesaurus instance within the GeoNode admin pages. + As a minumum, you need to: + - add a thesaurus in admin / base / Thesaurus + - add one or more instances of Keywords in admin / base / ThesaurusKeywords +- uploading a RDF file (either xml, ttl, jsonld or any other format recognized by [RDFlib](https://rdflib.dev/)). + When uploading a file, the behaviour is the same as running the command `thesaurus load --action update` (see `load_thesaurus`{.interpreted-text role="ref"}) +- loading a RDF file using the [thesaurus load]{.title-ref} management command (see `load_thesaurus`{.interpreted-text role="ref"}). + +## Upload an RDF file via the thesaurus admin page + +Navigate to the thesaurus page in the admin panel `http:///admin/base/thesaurus`. +On the top-right of the page a button named `Upload thesaurus`{.interpreted-text role="guilabel"} will be available: + +![](img/thesaurus_admin_1.png){.align-center} + +After clicking on it, a simple form for the upload will be shown which will allow you to select your desired RDF file: + +![](img/thesaurus_admin_2.png){.align-center} + +By clicking on [Upload RDF]{.title-ref}, the system will load the thesaurus and assign it a \"slugified\" name based on the file name. +The name can be easily changed later in the edit page. + +If everything goes fine, a success message will be shown: + +![](img/thesaurus_admin_success.png){.align-center} + +Otherwise the UI will show the error message: + +![](img/thesaurus_admin_fail.png){.align-center} + +# Management commands + +GeoNode provides a single command (`thesaurus`) with multiple actions: + +- `list`: list existing thesauri +- `load`: load a RDF file +- `dump`: dump a thesaurus into a file + +``` +python manage.py thesaurus --help + +usage: manage.py thesaurus [-h] [-i [IDENTIFIER]] [-f [FILE]] [--action {create,update,append,parse}] [-o [OUT]] + [--include INCLUDE] [--exclude EXCLUDE] + [--format {json-ld,n3,nt,pretty-xml,sorted-xml,trig,ttl,xml}] [--default-lang LANG] [--version] + [-v {0,1,2,3}] [--settings SETTINGS] [--pythonpath PYTHONPATH] [--traceback] [--no-color] + [--force-color] [--skip-checks] + [{list,load,dump}] + +Handles thesaurus commands ['list', 'load', 'dump'] + +positional arguments: + {list,load,dump} thesaurus operation to run + +options: + -h, --help show this help message and exit + --version Show program's version number and exit. + -v {0,1,2,3}, --verbosity {0,1,2,3} + Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output + --settings SETTINGS The Python path to a settings module, e.g. "myproject.settings.main". If this isn't provided, the + DJANGO_SETTINGS_MODULE environment variable will be used. + --pythonpath PYTHONPATH + A directory to add to the Python path, e.g. "/home/djangoprojects/myproject". + --traceback Raise on CommandError exceptions. + --no-color Don't colorize the command output. + --force-color Force colorization of the command output. + --skip-checks Skip system checks. + +Common params: + -i [IDENTIFIER], --identifier [IDENTIFIER] + Thesaurus identifier. Dump: required. Load: optional - if omitted will be created out of the filename + +Params for "load" subcommand: + -f [FILE], --file [FILE] + Full path to a thesaurus in RDF format + --action {create,update,append,parse} + Actions to run upon data loading (default: create) + +Params for "dump" subcommand: + -o [OUT], --out [OUT] + Full path to the output file to be created + --include INCLUDE Inclusion filter (wildcard is * as suffix or prefix); can be repeated + --exclude EXCLUDE Exclusion filter (wildcard is * as suffix or prefix); can be repeated + --format {json-ld,n3,nt,pretty-xml,sorted-xml,trig,ttl,xml} + Format string supported by rdflib, or sorted-xml (default: sorted-xml) + --default-lang LANG Default language code for untagged string literals (default: None) +``` + +## List thesauri: `thesaurus list` + +Get a list of the thesauri in GeoNode. + +Useful to find out the id of the thesauri when you need to export one of them. + +## Importing a thesaurus: `thesaurus load` {#load_thesaurus} + +The `load` command may create an entire Thesaurus, or just update part of it. + +Allowed params: + +- `file`: file to load; required +- `action`: `create`, `update`, `append`, `parse`; optional, default `create`; +- `identifier`: the id of the thesaurus; optional, defaults to a name created using the filename. + +The **automatic identifier creation** skips all the chars after the first dot in order to allow a thesaurus partitioning. + +For instance we may have different rdf files containing the labels for multiple projects, e.g.: `labels-i18n.proj1.rdf`, `labels-i18n.proj2.rdf`\... We may simply loop on the filenames and run the `load` subcommand on each of them, and all the keywords will be added to the same Thesaurus having id `labels-i18n`. + +The `load` command has different behaviours according to the `action` parameter: + +Actions: + +- + + `parse`: + + : parse the file and loop on all the concepts without writng anything in the db. Is equivalent to the classic `dryrun` option; + +- + + `create` (default action): + + : tries and create a thesaurus. If the thesaurus already exists, raises an exception. + +- + + `append`: + + : creates entries if they do not exist; pk are the ones listed in update action. + If the entry already exists, it is not changed in any way. + +- + + `update`: + + : creates and updates entries: + + - *Thesaurus*: creates it if it doesn\'t exist, pk is \"identifier\". + If it exists updates \"date\", \"description\", \"title\", \"about\" + - *ThesaurusLabel*: creates it if it doesn\'t exist, pk is \"thesaurus\", \"lang\" + If it exists updates \"value\" + - *ThesaurusKeyword*: creates it if it doesn\'t exist, pk is \"thesaurus\", \"about\" + If it exists updates \"alt_label\" + - *ThesaurusKeywordlabel*: creates it if it doesn\'t exist, pk is \"thesauruskeyword\", \"lang\" + If it exists updates \"label\" + +## Exporting a thesaurus: `thesaurus dump` {#dump_thesaurus} + +The `dump` command may export an entire Thesaurus or just a part of it. + +Allowed params: + +- `identifier`: the id of the thesaurus; required. +- `include`: Optional; filter ThesaurusKeywords to be dumped. Can be repeated. Filtering is applied on the [about]{.title-ref} field. Filters are in the format either [\*string]{.title-ref} or [string\*]{.title-ref} +- `exclude`: Optional; like [include]{.title-ref} but filter out ThesaurusKeywords from being dumped. +- `format`: optional, RDF format for the output (`json-ld`, `n3`, `nt`, `pretty-xml`, `sorted-xml`, `trig`, `ttl`, `xml`). Default `sorted-xml` +- `default-lang`: Default language code for untagged string literals; default is from `settings.THESAURUS_DEFAULT_LANG` +- `out`: Full path to the output file to be created. Optional; if omitted the RDF content is sent to stderr. + +### Format + +All the formats, except for `sorted-xml`, use the *RDFlib* library to serialize the thesaurus data. Since RDFlib handles the concepts as a graph, there is no ordering in the output data. This means that two consecutive `dump` of the same thesaurus may create two different files. + +When importing and exporting thesauri as a file, it may be useful to perform diff on them to find out what has changed. +The format `sorted-xml` creates a predictable output, where the ConceptScheme is at the start of the file, and the Concepts are sorted by their `about` field. Furthermore, the `prefLabel`\'s are sorted by their `lang` attribute. + +### Partial export + +The `dump` command also allows to export a subset of the keywords (concepts) in a Thesaurus. + +As an example, let\'s say we have the `labels-i18n` thesaurus, which contains some GeoNode official labels. + +In our project we added some keywords prefixed with \"[proj1]()\", since they belong to project1. + +Also in our GeoNode instance, we added some labels which override the standard ones, and are postfixed with `_ovr`. + +In order to only export the entries we edited, we\'ll issue the command: + + python manage.py thesaurus dump -i labels-i18n --include "proj1_*" --include "*_ovr" -f labels-i18n.proj1.rdf + +# Configuring a Thesaurus + +After a thesaurus is loaded o created in GeoNode, it should be configured in the `Admin`{.interpreted-text role="guilabel"} panel. + +The panel can be reached from `Admin`{.interpreted-text role="guilabel"} link of the *User Menu* in the navigation bar or through this URL: `http:///admin/base/thesaurus`. + +Once you are on the Thesaurus lists, select one thesaurus to open the Edit page + +
+img/thesaurus_edit_page.png +
The GeoNode Thesaurus edit Interface
+
+ +These are the thesaurus main attributes: + +- `identifier`: (mandatory) the thesaurus identifier (set by the `--identifier` parameter in the `thesaurus load` command, or automatically generated using the file name). +- `title`: (mandatory) The default title of the thesaurus (may be set from the loaded RDF file). +- `date`: (mandatory) The Date of the thesaurus (may be set from the loaded RDF file). +- `description`: (mandatory) The description of the thesaurus (may be set from the loaded RDF file). +- `slug`: (deprecated, use `identifier` instead) The slug of the thesaurus. +- `about`: (optional) The `rdf:about` URI of the thesaurus (may be set from the loaded RDF file). + +Next attributes define **how the thesaurus shall be used** within GeoNode. + +- `card min`: (optional) The minimum cardinality, default = 0 +- `card max`: (optional) The maximum cardinality, default = -1 (no limit) +- `facet`: (boolean) Decide if the thesaurus will be shown in the facet list, default: True \-- To be set to [true]{.title-ref} only when `card_max != 0` +- `order`: (integer) Set the listing order of the thesaurus in the facet list and in the metadata editor, default: 0, asc order from 0 to N + +If `card max` is not zero, the metadata editor will automatically display the Thesaurus in the list of the controlled terms. +More precisely these are the cases according to the two cardinality fields: + +- [card_max=0]{.title-ref} \--\> Disabled, The Thesaurus will not appear in the GUI +- [card_max=1]{.title-ref} & [card_min = 0]{.title-ref} \--\> Single choice, optional. +- [card_max=1]{.title-ref} & [card_min = 1]{.title-ref} \--\> Single choice, required +- [card_max=-1]{.title-ref} & [card_min = 0]{.title-ref} \--\> \[0..N\] Multiple choices, optional +- [card_max=-1]{.title-ref} & [card_min = 1]{.title-ref} \--\> \[1..N\] Multiple choices, required + +The metadata editor will show all the thesauri with `card_max != 0`, each one with its own title, like in the following image: + +
+img/thesaurus_choices.png +
The metadata interface with the Thesaurus enabled
+
+ +The Thesauri having `card_max == 0` are used as **codelists**: it means that they will be referred within GeoNode via their identifier for specific purposes. There will be ad-hoc documentation for each of such codelists. + +For instance, the thesaurus with identifier `labels-i18n` is used for the metadata labels translations. + +# Using keywords from a thesaurus + +After you\'ve finished the setup you should find a new input widget in the metadata editor allowing you to choose keywords from the thesaurus for your resource. + +Also, if you set the `facet` attribute to `true`, the thesaurus should be listed in the filter section in GeoNode\'s resource list views. + +For instance, if we have these thesauri: + +
+img/thesaurus_admin_list.png +
List of configured sample thesauri
+
+ +both set with `card max != 0` and `facet = true`, we\'ll have in the editor: + +
+img/thesaurus_edit_sample.png +
Keyword selectors for sample thesauri
+
+ +and we\'ll also have them in the filtering panel as facets: + +
+img/thesaurus_facet_sample.png +
Facets selectors for sample thesauri
+
diff --git a/docs/admin/upgrade/3.2-to-3.3/index.md b/docs/admin/upgrade/3.2-to-3.3/index.md new file mode 100644 index 00000000..e9a33995 --- /dev/null +++ b/docs/admin/upgrade/3.2-to-3.3/index.md @@ -0,0 +1,107 @@ +# Upgrade from 3.2.x / 3.3.x {#geonode-upgrade-3.2.x-3.3.x} + +1. Upgrade the dependencies +2. Perform the `migrations` management command; in case some attribute is conflicting, remove it manually from the DB +3. Proform the `collectstatic` management command + +## Upgrade the instance dependencies + +Check the `install_dep`{.interpreted-text role="ref"} and `install_venv`{.interpreted-text role="ref"} sections in order to upgrade your Python environment. + +Also, make sure the code is `Python 3.8` compatible and that you switched and aligned the **source code** and the **requirements.txt** to the `master` branch. + +This must be done manually and with particular attention. + +``` shell +workon +cd +pip install -r requirements.txt + +cd / + +pip install pip --upgrade +pip install -r requirements.txt --upgrade +pip install -e . --upgrade +pip install pygdal=="`gdal-config --version`.*" + +./manage.sh collectstatic --noinput +``` + +## Run GeoNode migrations + +Activate your GeoNode virtualenv and set the env vars: + +``` sql +. env/bin/Activate +export vars_210 +``` + +Here are the variables to export - update them to your environment settings: + +``` shell +export DATABASE_URL=postgis://user:***@localhost:5432/dbname +export DEFAULT_BACKEND_DATASTORE=data +export GEODATABASE_URL=postgis://user:***@localhost:5432/geonode_data +export ALLOWED_HOSTS="['localhost', '192.168.100.10']" +export STATIC_ROOT=~/www/geonode/static/ +export GEOSERVER_LOCATION=http://localhost:8080/geoserver/ +export GEOSERVER_PUBLIC_LOCATION=http://localhost:8080/geoserver/ +export GEOSERVER_ADMIN_PASSWORD=geoserver +export SESSION_EXPIRED_CONTROL_ENABLED=False +``` + +Apply migrations and apply basic fixtures: + +``` shell +./manage.py migrate --fake-initial +paver sync +``` + +::: note +::: title +Note +::: + +Incase of an error of `django.db.utils.ProgrammingError: column "colum-name" of relation "table-name" already exists`{.interpreted-text role="guilabel"} on running migrations, you can backup the field data with the following steps. +::: + +``` shell +./manage.sh dbshell +``` + +``` sql +ALTER TABLE ADD COLUMN _bkp varchar; +UPDATE
SET _bkp = colum-name; +ALTER TABLE
DROP COLUMN ; + +\q +``` + +Run migration then: + +``` shell +./manage.sh dbshell +``` + +``` sql +UPDATE
SET = _bkp; +ALTER TABLE
DROP COLUMN _bkp; + +\q +``` + +## Create superuser + +To create a superuser you should drop the following constraints (they can be re-enabled if needed): + +``` sql +alter table people_profile alter column last_login drop not null; +``` + +``` shell +./manage createsuperuser +``` + +## Update Templates + +Update available templates to use {% load static %} instead of {% load staticfiles %} diff --git a/admin/upgrade/3.x-to-4.x/index.rst b/docs/admin/upgrade/3.x-to-4.x/index.md similarity index 61% rename from admin/upgrade/3.x-to-4.x/index.rst rename to docs/admin/upgrade/3.x-to-4.x/index.md index 9b7fb3f9..bc24e72d 100644 --- a/admin/upgrade/3.x-to-4.x/index.rst +++ b/docs/admin/upgrade/3.x-to-4.x/index.md @@ -1,8 +1,4 @@ -.. _geonode-upgrade-3.x-4.x: - -======================================== -Upgrade Process from GeoNode 3.x to 4.x -======================================== +# Upgrade Process from GeoNode 3.x to 4.x {#geonode-upgrade-3.x-4.x} The purpose of this document is to detail the process that we will implement to upgrade GeoNode from version 3.x to version 4.x. This document is proposed to ensure that the upgrade process is efficient and understandable for everyone. It is important to note that this guide is specifically for a GeoNode installation running on Docker. @@ -10,97 +6,79 @@ The objective of this document is for everyone to understand the processes and s Table of Contents -* Migration Procedure -* Possible Issues +- Migration Procedure +- Possible Issues -Version Migration Procedure -=========================== +## Version Migration Procedure -1. Download the New Version ---------------------------- +### 1. Download the New Version With the following command, we will download the new version of GeoNode from the official source or the corresponding repository: -:: - https://github.com/GeoNode/geonode.git -b -**Note:** Replace with the appropriate branch or tag for the version you wish to install. +**Note:** Replace \ with the appropriate branch or tag for the version you wish to install. It is recommended to do this in a separate directory to keep the previous version intact in case of need. -* Directory of the previous version: - -:: +- Directory of the previous version: +```{=html} + +``` my/work/dir/geonode3 -* Directory of the new version: - -:: +- Directory of the new version: +```{=html} + +``` my/work/dir/geonode4 -2. Stop the Current GeoNode ---------------------------- +### 2. Stop the Current GeoNode Run the following command in the directory where the current GeoNode is deployed to stop the associated Docker containers: -:: - docker compose down -3. Modify the GeoNode .env File -------------------------------- +### 3. Modify the GeoNode .env File -Open the ``.env`` file of the new GeoNode in a text editor: - -:: +Open the `.env` file of the new GeoNode in a text editor: nano .env Adjust the environment variables to match the configuration of the previous GeoNode version. This may include variables related to the database, URLs, ports, etc. -4. Start the GeoNode Containers -------------------------------- - -Once you have made the necessary changes to the ``.env`` file, run the following command to start the GeoNode containers in the new version: +### 4. Start the GeoNode Containers -:: +Once you have made the necessary changes to the `.env` file, run the following command to start the GeoNode containers in the new version: docker compose up -d -Possible Issues -=============== +## Possible Issues The version upgrade of GeoNode also involves changes in some of its components, which can generate various issues. Below, we list some common problems and their possible solutions: -PostgreSQL Container in Constant Restart ----------------------------------------- +### PostgreSQL Container in Constant Restart **Logs:** -:: - 2024-02-21 18:06:14.056 UTC [1] FATAL: database files are incompatible with server 2024-02-21 18:06:14.056 UTC [1] DETAIL: The data directory was initialized by PostgreSQL version 13, which is not compatible with this version 15.4. **Cause:** The PostgreSQL version of the original GeoNode was lower than the current one. -**Solution:** Downgrade the PostgreSQL version to match the previously installed version in the GeoNode ``docker-compose.yml`` file. +**Solution:** Downgrade the PostgreSQL version to match the previously installed version in the GeoNode `docker-compose.yml` file. -Nginx Container in Constant Restart ------------------------------------ +### Nginx Container in Constant Restart **Logs:** -:: - 2024/02/21 18:11:49 [emerg] 1#1: host not found in "$HTTP_PORT" of the "listen" directive in /etc/nginx/nginx.conf:25 nginx: [emerg] host not found in "$HTTP_PORT" of the "listen" directive in /etc/nginx/nginx.conf:25 -**Cause:** Nginx is having issues replacing the port variables from the ``nginx.conf.envsubst`` file. +**Cause:** Nginx is having issues replacing the port variables from the `nginx.conf.envsubst` file. -**Solution:** Manually replace those variables with the corresponding ports once in the ``nginx.conf.envsubst`` file. After this, Nginx should work correctly and replace the variables normally in the future. +**Solution:** Manually replace those variables with the corresponding ports once in the `nginx.conf.envsubst` file. After this, Nginx should work correctly and replace the variables normally in the future. diff --git a/docs/admin/upgrade/index.md b/docs/admin/upgrade/index.md new file mode 100644 index 00000000..f036da6b --- /dev/null +++ b/docs/admin/upgrade/index.md @@ -0,0 +1,15 @@ +# Upgrade GeoNode + +::: {.toctree maxdepth="3"} +3.2-to-3.3/index +::: + +::: {.toctree maxdepth="3"} +3.x-to-4.x/index +::: + + +- [index](3.2-to-3.3/index.md) + +- [index](3.x-to-4.x/index.md) + diff --git a/admin/upload-parallelism-limit/img/admin_panel_parallel_limit.png b/docs/admin/upload-parallelism-limit/img/admin_panel_parallel_limit.png similarity index 100% rename from admin/upload-parallelism-limit/img/admin_panel_parallel_limit.png rename to docs/admin/upload-parallelism-limit/img/admin_panel_parallel_limit.png diff --git a/admin/upload-parallelism-limit/img/admin_panel_upload_parallelism_limit.png b/docs/admin/upload-parallelism-limit/img/admin_panel_upload_parallelism_limit.png similarity index 100% rename from admin/upload-parallelism-limit/img/admin_panel_upload_parallelism_limit.png rename to docs/admin/upload-parallelism-limit/img/admin_panel_upload_parallelism_limit.png diff --git a/docs/admin/upload-parallelism-limit/index.md b/docs/admin/upload-parallelism-limit/index.md new file mode 100644 index 00000000..c4baa55e --- /dev/null +++ b/docs/admin/upload-parallelism-limit/index.md @@ -0,0 +1,24 @@ +# Upload Parallelism Limit {#upload-parallelism-limits} + +Default: `5` + +When [uploading datasets](../../usage/managing_datasets/uploading_datasets.html#datasets-uploading), this value limits the number os parallel uploads. + +You will get the following message in case of error: `The number of active parallel uploads exceeds 5. Wait for the pending ones to finish.` + +This means that you to finish current active uploads before creating a new one. +The default maximum number is set during installation by [DEFAULT_MAX_PARALLEL_UPLOADS_PER_USER](../../basic/settings/index.html#default-max-parallel-uploads-per-user), usually 5 parallel uploads. + +After installation, only an user with administrative rights can change it by the admin painel or [accessing by api](../../devel/api/V2/index.html#get--api-v2-upload-parallelism-limits-). + +# Accessing the limits in the admin panel + +Access the admin panel, scroll until you find the \"Upload Parallelism limits\" option. + +> ![](img/admin_panel_upload_parallelism_limit.png){.align-center} + +Click at it, and you will be redirected to the limits list. + +> ![](img/admin_panel_parallel_limit.png){.align-center} + +To change it, click at the slug of the limit, then you can change its description and the max_number value. Changing the slug will not produce any effect. diff --git a/admin/upload-size-limits/img/admin_panel_size_limits_list.png b/docs/admin/upload-size-limits/img/admin_panel_size_limits_list.png similarity index 100% rename from admin/upload-size-limits/img/admin_panel_size_limits_list.png rename to docs/admin/upload-size-limits/img/admin_panel_size_limits_list.png diff --git a/admin/upload-size-limits/img/admin_panel_upload_size_limits.png b/docs/admin/upload-size-limits/img/admin_panel_upload_size_limits.png similarity index 100% rename from admin/upload-size-limits/img/admin_panel_upload_size_limits.png rename to docs/admin/upload-size-limits/img/admin_panel_upload_size_limits.png diff --git a/admin/upload-size-limits/img/changing_limit_to_200mb.jpg b/docs/admin/upload-size-limits/img/changing_limit_to_200mb.jpg similarity index 100% rename from admin/upload-size-limits/img/changing_limit_to_200mb.jpg rename to docs/admin/upload-size-limits/img/changing_limit_to_200mb.jpg diff --git a/admin/upload-size-limits/img/size-limit-error.png b/docs/admin/upload-size-limits/img/size-limit-error.png similarity index 100% rename from admin/upload-size-limits/img/size-limit-error.png rename to docs/admin/upload-size-limits/img/size-limit-error.png diff --git a/docs/admin/upload-size-limits/index.md b/docs/admin/upload-size-limits/index.md new file mode 100644 index 00000000..eee3ff9c --- /dev/null +++ b/docs/admin/upload-size-limits/index.md @@ -0,0 +1,54 @@ +# Upload Size Limits + +Default: `104857600` (100 MB in total bytes) + +When [uploading datasets](../../usage/managing_datasets/uploading_datasets.html#datasets-uploading) +or [uploading documents](../../usage/managing_documents/uploading_documents.html#uploading-documents), +the total size of the uploaded files is verified. + +If it passes a limit value, you will get something similar to the following error: + +: ![](img/size-limit-error.png){.align-center} + +With the following message: `Total upload size exceeds 100.0 MB. Please try again with smaller files.` + +This means that the file you tried to upload is too large. It consider the sum of sizes in the case of datasets/layers with multiple files (ESRI Shapefile). +The default maximum value is set during installation by [DEFAULT_MAX_UPLOAD_SIZE](../../basic/settings/index.html#default-max-upload-size), usually 100 MB. + +After installation, only an user with administrative rights can change it by the admin painel or [accessing by api](../../devel/api/V2/index.html#get--api-v2-upload-size-limits-). + +# Accessing the limits in the admin panel + +Access the admin panel, scroll until you find the \"Upload Size limits\" option. + +> ![](img/admin_panel_upload_size_limits.png){.align-center} + +Click at it, and you will be redirected to the limits list. + +> ![](img/admin_panel_size_limits_list.png){.align-center} + +We have two limits here, one for datasets/layers and other for documents. + +# Changing a Limit + +To change it, click at the slug of one of the limits. + +> ![](img/changing_limit_to_200mb.jpg){.align-center} + +You can change its description and the max_size value. Changing the slug will not produce any effect. + +Max size value should be entered in bytes, this means that 200 MB is around 200000000 bytes, or 200 \* 1024 \* 1024 = 209715200 if you want to use exact values. +After editing, remember to save and verify in the list if the value is the expected one. + +If you try to upload a dataset/layer larger than 100 MB again (but smaller than the new limit) it won\'t raise any errors. + +# Advanced notes for developers + +When uploading a file there is an additional validation step, to avoid the creation of big temporary files. +This happens with the use of a custom [File Upload Handler](https://docs.djangoproject.com/en/4.0/ref/settings/#std:setting-FILE_UPLOAD_HANDLERS). + +During this step we verify the total size of the request, if it\'s considerably bigger than the defined limit, an empty file with a fake large number set as its size is created when processing the request. +Later, during the form validation, the related error will be raised. + +The threshould for this to happen is set to `2 * MAX_UPLOAD_SIZE + 2 MB` where MAX_UPLOAD_SIZE is the Upload Size Limit defined by the admin user. +When changing the processes related to the upload size limiting, the upload handlers should also be taken into account. Otherwise, this can lead to the creating of empty datasets and documents. diff --git a/advanced/components/img/google_ssl_001.png b/docs/advanced/components/img/google_ssl_001.png similarity index 100% rename from advanced/components/img/google_ssl_001.png rename to docs/advanced/components/img/google_ssl_001.png diff --git a/advanced/components/img/google_ssl_002.png b/docs/advanced/components/img/google_ssl_002.png similarity index 100% rename from advanced/components/img/google_ssl_002.png rename to docs/advanced/components/img/google_ssl_002.png diff --git a/advanced/components/img/google_ssl_003.png b/docs/advanced/components/img/google_ssl_003.png similarity index 100% rename from advanced/components/img/google_ssl_003.png rename to docs/advanced/components/img/google_ssl_003.png diff --git a/advanced/components/img/google_ssl_004.png b/docs/advanced/components/img/google_ssl_004.png similarity index 100% rename from advanced/components/img/google_ssl_004.png rename to docs/advanced/components/img/google_ssl_004.png diff --git a/advanced/components/img/google_ssl_005.png b/docs/advanced/components/img/google_ssl_005.png similarity index 100% rename from advanced/components/img/google_ssl_005.png rename to docs/advanced/components/img/google_ssl_005.png diff --git a/advanced/components/img/google_ssl_006.png b/docs/advanced/components/img/google_ssl_006.png similarity index 100% rename from advanced/components/img/google_ssl_006.png rename to docs/advanced/components/img/google_ssl_006.png diff --git a/advanced/components/img/google_ssl_007.png b/docs/advanced/components/img/google_ssl_007.png similarity index 100% rename from advanced/components/img/google_ssl_007.png rename to docs/advanced/components/img/google_ssl_007.png diff --git a/advanced/components/img/google_ssl_008.png b/docs/advanced/components/img/google_ssl_008.png similarity index 100% rename from advanced/components/img/google_ssl_008.png rename to docs/advanced/components/img/google_ssl_008.png diff --git a/advanced/components/img/google_ssl_009.png b/docs/advanced/components/img/google_ssl_009.png similarity index 100% rename from advanced/components/img/google_ssl_009.png rename to docs/advanced/components/img/google_ssl_009.png diff --git a/advanced/components/img/google_ssl_010.png b/docs/advanced/components/img/google_ssl_010.png similarity index 100% rename from advanced/components/img/google_ssl_010.png rename to docs/advanced/components/img/google_ssl_010.png diff --git a/advanced/components/img/google_ssl_011.png b/docs/advanced/components/img/google_ssl_011.png similarity index 100% rename from advanced/components/img/google_ssl_011.png rename to docs/advanced/components/img/google_ssl_011.png diff --git a/advanced/components/img/google_ssl_012.png b/docs/advanced/components/img/google_ssl_012.png similarity index 100% rename from advanced/components/img/google_ssl_012.png rename to docs/advanced/components/img/google_ssl_012.png diff --git a/advanced/components/img/google_ssl_013.png b/docs/advanced/components/img/google_ssl_013.png similarity index 100% rename from advanced/components/img/google_ssl_013.png rename to docs/advanced/components/img/google_ssl_013.png diff --git a/advanced/components/img/oauth001.png b/docs/advanced/components/img/oauth001.png similarity index 100% rename from advanced/components/img/oauth001.png rename to docs/advanced/components/img/oauth001.png diff --git a/advanced/components/img/oauth002.png b/docs/advanced/components/img/oauth002.png similarity index 100% rename from advanced/components/img/oauth002.png rename to docs/advanced/components/img/oauth002.png diff --git a/advanced/components/img/oauth003.png b/docs/advanced/components/img/oauth003.png similarity index 100% rename from advanced/components/img/oauth003.png rename to docs/advanced/components/img/oauth003.png diff --git a/advanced/components/img/oauth004.png b/docs/advanced/components/img/oauth004.png similarity index 100% rename from advanced/components/img/oauth004.png rename to docs/advanced/components/img/oauth004.png diff --git a/advanced/components/img/oauth005.png b/docs/advanced/components/img/oauth005.png similarity index 100% rename from advanced/components/img/oauth005.png rename to docs/advanced/components/img/oauth005.png diff --git a/advanced/components/img/oauth006.png b/docs/advanced/components/img/oauth006.png similarity index 100% rename from advanced/components/img/oauth006.png rename to docs/advanced/components/img/oauth006.png diff --git a/advanced/components/img/oauth007.png b/docs/advanced/components/img/oauth007.png similarity index 100% rename from advanced/components/img/oauth007.png rename to docs/advanced/components/img/oauth007.png diff --git a/advanced/components/img/oauth008.png b/docs/advanced/components/img/oauth008.png similarity index 100% rename from advanced/components/img/oauth008.png rename to docs/advanced/components/img/oauth008.png diff --git a/advanced/components/img/oauth009.png b/docs/advanced/components/img/oauth009.png similarity index 100% rename from advanced/components/img/oauth009.png rename to docs/advanced/components/img/oauth009.png diff --git a/advanced/components/img/oauth010.png b/docs/advanced/components/img/oauth010.png similarity index 100% rename from advanced/components/img/oauth010.png rename to docs/advanced/components/img/oauth010.png diff --git a/advanced/components/img/oauth011.png b/docs/advanced/components/img/oauth011.png similarity index 100% rename from advanced/components/img/oauth011.png rename to docs/advanced/components/img/oauth011.png diff --git a/advanced/components/img/oauth012.png b/docs/advanced/components/img/oauth012.png similarity index 100% rename from advanced/components/img/oauth012.png rename to docs/advanced/components/img/oauth012.png diff --git a/advanced/components/img/oauth013.png b/docs/advanced/components/img/oauth013.png similarity index 100% rename from advanced/components/img/oauth013.png rename to docs/advanced/components/img/oauth013.png diff --git a/advanced/components/img/oauth014.png b/docs/advanced/components/img/oauth014.png similarity index 100% rename from advanced/components/img/oauth014.png rename to docs/advanced/components/img/oauth014.png diff --git a/advanced/components/img/oauth015.png b/docs/advanced/components/img/oauth015.png similarity index 100% rename from advanced/components/img/oauth015.png rename to docs/advanced/components/img/oauth015.png diff --git a/advanced/components/img/oauth016.png b/docs/advanced/components/img/oauth016.png similarity index 100% rename from advanced/components/img/oauth016.png rename to docs/advanced/components/img/oauth016.png diff --git a/advanced/components/img/oauth017.png b/docs/advanced/components/img/oauth017.png similarity index 100% rename from advanced/components/img/oauth017.png rename to docs/advanced/components/img/oauth017.png diff --git a/advanced/components/img/oauth018.png b/docs/advanced/components/img/oauth018.png similarity index 100% rename from advanced/components/img/oauth018.png rename to docs/advanced/components/img/oauth018.png diff --git a/advanced/components/img/oauth019.png b/docs/advanced/components/img/oauth019.png similarity index 100% rename from advanced/components/img/oauth019.png rename to docs/advanced/components/img/oauth019.png diff --git a/advanced/components/img/oauth020.png b/docs/advanced/components/img/oauth020.png similarity index 100% rename from advanced/components/img/oauth020.png rename to docs/advanced/components/img/oauth020.png diff --git a/advanced/components/img/oauth021.png b/docs/advanced/components/img/oauth021.png similarity index 100% rename from advanced/components/img/oauth021.png rename to docs/advanced/components/img/oauth021.png diff --git a/advanced/components/img/oauth022.png b/docs/advanced/components/img/oauth022.png similarity index 100% rename from advanced/components/img/oauth022.png rename to docs/advanced/components/img/oauth022.png diff --git a/advanced/components/img/oauth023.png b/docs/advanced/components/img/oauth023.png similarity index 100% rename from advanced/components/img/oauth023.png rename to docs/advanced/components/img/oauth023.png diff --git a/advanced/components/img/oauth024.png b/docs/advanced/components/img/oauth024.png similarity index 100% rename from advanced/components/img/oauth024.png rename to docs/advanced/components/img/oauth024.png diff --git a/advanced/components/img/oauth025.png b/docs/advanced/components/img/oauth025.png similarity index 100% rename from advanced/components/img/oauth025.png rename to docs/advanced/components/img/oauth025.png diff --git a/advanced/components/img/oauth026.png b/docs/advanced/components/img/oauth026.png similarity index 100% rename from advanced/components/img/oauth026.png rename to docs/advanced/components/img/oauth026.png diff --git a/advanced/components/img/oauth027.png b/docs/advanced/components/img/oauth027.png similarity index 100% rename from advanced/components/img/oauth027.png rename to docs/advanced/components/img/oauth027.png diff --git a/advanced/components/img/oauth028.png b/docs/advanced/components/img/oauth028.png similarity index 100% rename from advanced/components/img/oauth028.png rename to docs/advanced/components/img/oauth028.png diff --git a/advanced/components/img/oauth029.png b/docs/advanced/components/img/oauth029.png similarity index 100% rename from advanced/components/img/oauth029.png rename to docs/advanced/components/img/oauth029.png diff --git a/advanced/components/img/oauth030.png b/docs/advanced/components/img/oauth030.png similarity index 100% rename from advanced/components/img/oauth030.png rename to docs/advanced/components/img/oauth030.png diff --git a/advanced/components/img/oauth031.png b/docs/advanced/components/img/oauth031.png similarity index 100% rename from advanced/components/img/oauth031.png rename to docs/advanced/components/img/oauth031.png diff --git a/advanced/components/img/oauth032.png b/docs/advanced/components/img/oauth032.png similarity index 100% rename from advanced/components/img/oauth032.png rename to docs/advanced/components/img/oauth032.png diff --git a/advanced/components/img/oauth033.png b/docs/advanced/components/img/oauth033.png similarity index 100% rename from advanced/components/img/oauth033.png rename to docs/advanced/components/img/oauth033.png diff --git a/docs/advanced/components/index.md b/docs/advanced/components/index.md new file mode 100644 index 00000000..31a6982b --- /dev/null +++ b/docs/advanced/components/index.md @@ -0,0 +1,1158 @@ +# OAuth2 Security: Authentication and Authorization {#geoserver_geonode_aa} + +GeoNode interacts with GeoServer through an advanced security mechanism based on OAuth2 Protocol and GeoFence. This section is a walk through of the configuration and setup of GeoNode and GeoServer Advanced Security. + +What we will see in this section is: + +- **Introduction** + +- **GeoNode** (Security Backend): + + > 1. Django Authentication + > 2. Django OAuth Toolkit Setup and Configuration + > 3. Details on `settings.py` Security Settings + +- **GeoServer** (Security Backend): + + > 1. GeoServer Security Subsystem + > 2. Introduction to the GeoServer OAuth2 Security Plugin + > 3. Configuration of the `GeoNode REST Role Service` + > 4. Configuration of the `GeoNode OAuth2 Authentication Filter` + > 5. The GeoServer Authentication Filter Chains + > 6. Introduction to GeoFence Plugin, the Advanced Security Framework for GeoServer + +- **Troubleshooting and Advanced Features**: + + > 1. Common Issues and Fixes + > 2. How to setup `HTTPS` secured endpoints + > 3. GeoFence Advanced Features + +## Introduction + +GeoServer, i.e. the geospatial backend server of GeoNode, is a spatial server which needs authenticated users in order to access protected resources or administration functions. + +GeoServer supports several kind of Authentication and Authorization mechanisms. Those systems are pluggable and GeoServer can use them at the same time by the use of a `Filter Chain`. +Briefly this mechanism allows GeoServer to check for different A&A protocols one by one. The first one matching is used by GeoServer to authorize the users. + +GeoNode Authentication is based by default on Django Security Subsystem. Django authentication allows GeoNode to manage its internal users, groups, roles and sessions. + +GeoNode has some external components, like GeoServer or QGis Server, which are pluggable and stand-alone services, devoted to the management of geospatial data. +Those external services have their own authentication and authorization mechanisms which must be synchronized somehow with the GeoNode one. Also, those external services +maintain, in most of the cases and unless specific configuration does not disable this, alternative security access which for instance allow GeoNode to modify the geospatial catalog +under the hood, or a system administrator to have independent and privileged access to the servers. + +Before going deeply on how GeoServer/GeoNode A&A works and how it can be configured in order to work correctly with GeoNode, let\'s quickly clarify the difference between the `Authentication` and `Authorization` concepts. + +### Authentication + +Authentication is the process of verifying the identity of someone through the use of some sort of credentials and a handshake protocol. +If the credentials are valid, the authorization process starts. Authentication process always proceeds to Authorization process (although they may often seem to be combined). +The two terms are often used synonymously but they are two different processes. + +For more details and explanation about the authentication concepts, take a look [here](http://searchsecurity.techtarget.com/definition/authentication). + +### Authorization + +Authorization is the process of allowing authenticated users to access protected resources by checking its roles and rights against some sort of security rules mechanism or protocol. +In other words it allows to control access rights by granting or denying specific permissions to specific authorized users. + +## GeoNode Security Backend + +### DJango Authentication + +The Django authentication system handles both authentication and authorization. + +The auth system consists of: + +1. Users +2. Permissions: Binary (yes/no) flags designating whether a user may perform a certain task. +3. Groups: A generic way of applying labels and permissions to more than one user. +4. A configurable password hashing system +5. Forms and view tools for logging in users, or restricting content +6. A pluggable backend system + +The authentication system in Django aims to be very generic and doesn't provide some features commonly found in web authentication systems. Solutions for some of these common problems have been implemented in third-party packages: + +1. Password strength checking +2. Throttling of login attempts +3. Authentication against third-parties (OAuth, for example) + +::: note +::: title +Note +::: + +For more details on installation and configuration of Django authentication system, please refer to the official guide . +::: + +GeoNode communicates with GeoServer through Basic Authentication under the hood, in order to configure the data and the GeoServer catalog. + +In order to do this, you must be sure that GeoNode knows the **internal** admin user and password of GeoServer. + +::: warning +::: title +Warning +::: + +This must be an internal GeoServer user with admin rights, not a GeoNode one. +::: + +Make sure the credentials are correctly configured into the file `settings.py` + +### OGC_SERVER + +Ensure that the `OGC_SERVER` settings are correctly configured. + +Notice that the two properties `LOGIN_ENDPOINT` and `LOGOUT_ENDPOINT` must specify the GeoServer OAuth2 Endpoints (see details below). +The default values `'j_spring_oauth2_geonode_login'` and `'j_spring_oauth2_geonode_logout'` work in most of the cases, unless you need some specific endpoints different from the later. +In any case those values **must** be coherent with the GeoServer OAuth2 Plugin configuration. + +If in doubt, please use the default values here below. + +Default values are: + +``` python +... +# OGC (WMS/WFS/WCS) Server Settings +# OGC (WMS/WFS/WCS) Server Settings +OGC_SERVER = { + 'default': { + 'BACKEND': 'geonode.geoserver', + 'LOCATION': GEOSERVER_LOCATION, + 'LOGIN_ENDPOINT': 'j_spring_oauth2_geonode_login', + 'LOGOUT_ENDPOINT': 'j_spring_oauth2_geonode_logout', + # PUBLIC_LOCATION needs to be kept like this because in dev mode + # the proxy won't work and the integration tests will fail + # the entire block has to be overridden in the local_settings + 'PUBLIC_LOCATION': GEOSERVER_PUBLIC_LOCATION, + 'USER': 'admin', + 'PASSWORD': 'geoserver', + 'MAPFISH_PRINT_ENABLED': True, + 'PRINT_NG_ENABLED': True, + 'GEONODE_SECURITY_ENABLED': True, + 'WMST_ENABLED': False, + 'BACKEND_WRITE_ENABLED': True, + 'WPS_ENABLED': False, + 'LOG_FILE': '%s/geoserver/data/logs/geoserver.log' % os.path.abspath(os.path.join(PROJECT_ROOT, os.pardir)), + # Set to name of database in DATABASES dictionary to enable + 'DATASTORE': '', # 'datastore', + 'TIMEOUT': 10 # number of seconds to allow for HTTP requests + } +} +... +``` + +### GeoNode and GeoServer A&A Interaction + +The GeoServer instance used by GeoNode, has a particular setup that allows the two frameworks to correctly interact and exchange informations on users credentials and permissions. + +In particular GeoServer is configured with a `Filter Chain` for Authorization that makes use of the two following protocols: + +> 1. +> +> **Basic Authentication**; this is the default GeoServer Authentication mechanism. This makes use of [rfc2617 - Basic and Digest Access Authentication](https://tools.ietf.org/html/rfc2617) in order to check for user\'s credentials. +> +> : In other words, GeoServer takes a `username` and a `password` encoded [Base64](https://tools.ietf.org/html/rfc4648) on the HTTP Request Headers and compare them against its internal database (which by default is an encrypted XML file on the GeoServer Data Dir). +> If the user\'s credentials match, then GeoServer checks for Authorization through its `Role Services` (we will see those services in details on the *GeoServer (Security Backend)* section below). +> +> ::: note +> ::: title +> Note +> ::: +> +> GeoServer ships by default with `admin` and `geoserver` as the default administrator user name and password. Before putting the GeoServer on-line it is imperative to change at least the administrator password. +> ::: +> +> 2. **OAuth2 Authentication**; this module allows GeoServer to authenticate against the [OAuth2 Protocol](https://tools.ietf.org/html/rfc6749). If the Basic Authentication fails, GeoServer falls back to this by using GeoNode as OAuth2 Provider by default. + +::: note +::: title +Note +::: + +Further details can be found directly on the official GeoServer documentation at section \"[Authentication Chain](http://docs.geoserver.org/latest/en/user/security/auth/chain.html#security-auth-chain)\" +::: + +From the **GeoNode backend (server) side**, the server will make use of **Basic Authentication** with administrator credentials to configure the GeoServer catalog. GeoServer must be reachable by GeoNode of course, and GeoNode must know the internal GeoServer admin credentials. + +From the **GeoNode frontend (browser and GUI) side**, the *Authentication* goal is to allow GeoServer to recognize as valid a user which has been already logged into GeoNode, providing kind of an [SSO](https://en.wikipedia.org/wiki/Single_sign-on) mechanism between the two applications. + +GeoServer must know and must be able to access GeoNode via HTTP/HTTPS. In other words, an external user connected to GeoNode must be authenticated to GeoServer with same permissions. This is possible through the **OAuth2 Authentication** Protocol. + +**GeoNode / GeoServer Authentication Mechanism** + +> *GeoNode as OAuth2 Provider (OP)* +> +> OpenID Connect is an identity framework built on OAuth 2.0 protocol which extends the authorization of OAuth 2.0 processes to implement its authentication mechanism. OpenID Connect adds a discovery mechanism allowing users to use an external trusted authority as an identity provider. From another point of view, this can be seen as a single sign on (SSO) system. +> +> OAuth 2.0 is an authorization framework which is capable of providing a way for clients to access a resource with restricted access on behalf of the resource owner. OpenID Connect allows clients to verify the users with an authorization server based authentication. +> +> As an OP, GeoNode will be able to act as trusted identity provider, thus allowing the system working on an isolated environment and/or allow GeoNode to authenticate private users managed by the local DJango auth subsystem. +> +> *GeoServer as OAuth2 Relying Party (RP)* +> +> Thanks to the **OAuth2 Authentication** GeoServer is able to retrieve an end user\'s identity directly from the OAuth2 Provider (OP). +> +> With GeoNode acting as an OP, the mechanism will avoid the use of cookies relying, instead, on the OAuth2 secure protocol. +> +> How the OAuth2 Protocol works: +> +> > ![](img/oauth001.png) +> +> 1. The relying party sends the request to the OAuth2 provider to authenticate the end user +> 2. The OAuth2 provider authenticates the user +> 3. The OAuth2 provider sends the ID token and access token to the relying party +> 4. The relying party sends a request to the user info endpoint with the access token received from OAuth2 provider +> 5. The user info endpoint returns the claims. + +**GeoNode / GeoServer Authorization Mechanism** + +> Allowing GeoServer to make use of a OAuth2 in order to act as an OAuth2 RP, is not sufficient to map a user identity to its roles though. +> +> On GeoServer side we will still need to a `RoleService` which would be able to talk to GeoNode and transform the tokens into a User Principal to be used within the GeoServer Security subsystem itself. +> +> In other words after a successfull Authentication, GeoServer needs to Authorize the user in order to understand which resources he is enable to access or not. +> A `REST based RoleService` on GeoNode side, allows GeoServer to talk to GeoNode via [REST](https://en.wikipedia.org/wiki/Representational_state_transfer) to get the current user along with the list of its Roles. +> +> Nevertheless knowing the Roles associated to a user is not sufficient. The complete GeoServer Authorization needs to catch a set of `Access Rules`, associated to the Roles, in order to establish which resources and data are accessible by the user. +> +> The GeoServer Authorization is based on Roles only, therefore for each authenticated user we need also to know: +> +> > 1. The Roles associated to a valid user session +> > 2. The access permissions associated to a GeoServer Resource +> +> The Authentication mechanism above allows GeoServer to get information about the user and his Roles, which addresses point 1. +> +> About point 2, GeoServer makes use of the [GeoFence Embedded Server](http://docs.geoserver.org/latest/en/user/community/geofence-server/index.html) plugin. +> GeoFence is a java web application that provides an advanced authentication / authorization engine for GeoServer using the interface described in [here](https://github.com/geoserver/geofence/wiki/First-steps). +> GeoFence has its own rules database for the management of Authorization rules, and overrides the standard GeoServer security management system by implementing a sophisticated Resource Access Manager. +> Least but not last, GeoFence implements and exposes a [REST API](https://github.com/geoserver/geofence/wiki/REST-API) allowing remote authorized clients to read / write / modify security rules. +> +> The advantages using such plugin are multiple: +> +> > 1. The Authorizations rules have a fine granularity. The security rules are handled by GeoFence in a way similar to the iptables ones, and allow to define security constraints even on sub-regions and attributes of layers. +> > 2. GeoFence exposes a REST interface to its internal rule database, allowing external managers to update the security constraints programmatically +> > 3. GeoFence implements an internal caching mechanism which improves considerably the performances under load. +> +> *GeoNode interaction with GeoFence* +> +> GeoNode itself is able to push/manage Authorization rules to GeoServer through the GeoFence [REST API](https://github.com/geoserver/geofence/wiki/REST-API), acting as an administrator for GeoServer. +> GeoNode properly configures the GeoFence rules anytime it is needed, i.e. the permissions of a Resource / Layer are updated. + +GeoServer must know and must be able to access GeoNode via HTTP/HTTPS. In other words, an external user connected to GeoNode must be authenticated to GeoServer with same permissions. This is possible through the **GeoNodeCoockieProcessingFiler**. + +Summarizing we will have different ways to access GeoNode Layers: + +1. Through GeoNode via DJango Authentication and **GeoNodeCoockieProcessingFiler**; basically the users available in GeoNode are also valid for GeoServer or any other backend. + + > ::: warning + > ::: title + > Warning + > ::: + > + > If a GeoNode user has \"administrator\" rights, he will be able to administer GeoServer too. + > ::: + +2. Through GeoServer Security Subsystem; it will be always possible to access to GeoServer using its internal security system and users, unless explictly disabled (**warning** this is dangerous, you must know what you are doing). + +Let\'s now see in details how the single pieces are configured and how they can be configured. + +### DJango OAuth Toolkit Setup and Configuration + +As stated above, GeoNode makes use of the OAuth2 protocol for all the frontend interactions with GeoServer. GeoNode must be configured as an OAuth2 Provider and provide a `Client ID` and a `Client Sercret` key to GeoServer. +This is possible by enabling and configuring the [Django OAuth Toolkit Plugin](https://django-oauth-toolkit.readthedocs.io/en/latest/). + +::: warning +::: title +Warning +::: + +GeoNode and GeoServer won\'t work at all if the following steps are not executed at the first installation. +::: + +### Default `settings.py` Security Settings for OAuth2 + +Double check that the OAuth2 Provider and Security Plugin is enabled and that the settings below are correctly configured. + +### AUTH_IP_WHITELIST + +`AUTH_IP_WHITELIST` property limits access to users/groups REST Role Service endpoints to the only whitelisted IP addresses. Empty list means \'allow all\'. +If you need to limit \'api\' REST calls to only some specific IPs fill the list like this: `AUTH_IP_WHITELIST = ['192.168.1.158', '192.168.1.159']` + +Default values are: + +``` python +... +AUTH_IP_WHITELIST = [] +... +``` + +### INSTALLED_APPS + +In order to allow GeoNode to act as an OAuth2 Provider, we need to enable the `oauth2_provider` DJango application provided by the \"Django OAuth Toolkit\". + +Default values are: + +``` python +... +INSTALLED_APPS = ( + + 'modeltranslation', + + ... + 'guardian', + 'oauth2_provider', + ... + +) + GEONODE_APPS +... +``` + +### MIDDLEWARE_CLASSES + +Installing the [oauth2_provider]{.title-ref}\` DJango application is not sufficient to enable the full functionality. We need also GeoNode to include additional entities to its internal model. + +Default values are: + +``` python +... +MIDDLEWARE_CLASSES = ( + 'django.middleware.common.CommonMiddleware', + 'django.contrib.sessions.middleware.SessionMiddleware', + 'django.contrib.messages.middleware.MessageMiddleware', + + # The setting below makes it possible to serve different languages per + # user depending on things like headers in HTTP requests. + 'django.middleware.locale.LocaleMiddleware', + 'pagination.middleware.PaginationMiddleware', + 'django.middleware.csrf.CsrfViewMiddleware', + 'django.contrib.auth.middleware.AuthenticationMiddleware', + 'django.middleware.clickjacking.XFrameOptionsMiddleware', + + # If you use SessionAuthenticationMiddleware, be sure it appears before OAuth2TokenMiddleware. + # SessionAuthenticationMiddleware is NOT required for using django-oauth-toolkit. + 'django.contrib.auth.middleware.SessionAuthenticationMiddleware', + 'oauth2_provider.middleware.OAuth2TokenMiddleware', +) +... +``` + +### AUTHENTICATION_BACKENDS + +In order to allow GeoNode to act as an OAuth2 Provider, we need to enable the `oauth2_provider.backends.OAuth2Backend` DJango backend provided by the \"Django OAuth Toolkit\". +Also notice that we need to specify the OAuth2 Provider scopes and declare which `generator` to use in order to create `OAuth2 Client IDs`. + +Default values are: + +``` python +... +# Replacement of default authentication backend in order to support +# permissions per object. +AUTHENTICATION_BACKENDS = ( + 'oauth2_provider.backends.OAuth2Backend', + 'django.contrib.auth.backends.ModelBackend', + 'guardian.backends.ObjectPermissionBackend', +) + +OAUTH2_PROVIDER = { + 'SCOPES': { + 'read': 'Read scope', + 'write': 'Write scope', + 'groups': 'Access to your groups' + }, + + 'CLIENT_ID_GENERATOR_CLASS': 'oauth2_provider.generators.ClientIdGenerator', +} +... +``` + +### Django OAuth Toolkit Admin Setup + +Once the `settings.py` and `local_settings.py` have been correctly configured for your system: + +1. Complete the GeoNode setup steps + + > - Prepare the model + > + > > ``` python + > > python manage.py makemigrations + > > python manage.py migrate + > > python manage.py syncdb + > > ``` + > + > - Prepare the static data + > + > > ``` python + > > python manage.py collectstatic + > > ``` + > + > - Make sure the database has been populated with initial default data + > + > > ::: warning + > > ::: title + > > Warning + > > ::: + > > + > > *Deprecated* this command will be replaced by migrations in the future, so be careful. + > > ::: + > > + > > ``` python + > > python manage.py loaddata initial_data.json + > > ``` + > + > - Make sure there exists a superuser for your environment + > + > > ::: warning + > > ::: title + > > Warning + > > ::: + > > + > > *Deprecated* this command will be replaced by migrations in the future, so be careful. + > > ::: + > > + > > ``` python + > > python manage.py createsuperuser + > > ``` + > + > ::: note + > ::: title + > Note + > ::: + > + > Read the base tutorials on GeoNode Developer documentation for details on the specific commands and how to use them. + > ::: + +2. Start the application + + > Start GeoNode accordingly on how the setup has been done; run debug mode through `paver`, or proxied by an HTTP Server like Apache2 HTTPD, Nginx or others. + +3. Finalize the setup of the OAuth2 Provider + + > First of all you need to configure and create a new OAuth2 Application called `GeoServer` through the GeoNode Admin Dashboard + > + > - Access the GeoNode Admin Dashboard + > + > > ![](img/oauth002.png) + > + > - Go to `Django OAuth Toolkit` \> `Applications` + > + > > ![](img/oauth003.png) + > + > - Update or create the Application named `GeoServer` + > + > > ::: warning + > > ::: title + > > Warning + > > ::: + > > + > > The Application name **must** be `GeoServer` + > > ::: + > > + > > ![](img/oauth004.png) + > > + > > - `Client id`; An alphanumeric code representing the OAuth2 Client Id. GeoServer OAuth2 Plugin **will** use **this** value. + > > + > > > ::: warning + > > > ::: title + > > > Warning + > > > ::: + > > > + > > > In a production environment it is **highly** recommended to modify the default value provided with GeoNode installation. + > > > ::: + > > + > > - `User`; Search for the `admin` user. Its `ID` will be automatically updated into the form. + > > + > > - `Redirect uris`; It is possible to specify many URIs here. Those must coincide with the GeoServer instances URIs. + > > + > > - `Client type`; Choose `Confidential` + > > + > > - `Authorization grant type`; Choose `Authorization code` + > > + > > - `Client secret`; An alphanumeric code representing the OAuth2 Client Secret. GeoServer OAuth2 Plugin **will** use **this** value. + > > + > > > ::: warning + > > > ::: title + > > > Warning + > > > ::: + > > > + > > > In a production environment it is **highly** recommended to modify the default value provided with GeoNode installation. + > > > ::: + > > + > > - `Name`; **Must** be `GeoServer` + +## GeoServer Security Backend + +### GeoServer Security Subsystem + +GeoServer has a robust security subsystem, modeled on Spring Security. Most of the security features are available through the Web administration interface. + +For more details on how this works and how to configure and modify it, please refer to the official GeoServer guide + +By using the `GeoServer Data Dir` provided with GeoNode build, the following configuration are already available. You will need just to update them accordingly to your environment (like IP addresses and Host names, OAuth2 Keys, and similar things). +However it is recommended to read carefully all the following passages in order to understand exactly how the different component are configured and easily identify any possible issue during the deployment. + +The main topics of this section are: + +1. Connection to the GeoNode REST Role Service +2. Setup of the GeoServer OAuth2 Authentication Filter +3. Configuration of the GeoServer Filter Chains +4. Setup and test of the GeoFence Server and Default Rules + +## Connection to the GeoNode REST Role Service + +### Preliminary checks + +- GeoServer is up and running and you have admin rights +- GeoServer must reach the GeoNode instance via HTTP +- The GeoServer Host IP Address must be allowed to access the GeoNode Role Service APIs (see the section `AUTH_IP_WHITELIST` above) + +### Setup of the GeoNode REST Role Service + +1. Login as `admin` to the GeoServer GUI + + > ::: warning + > ::: title + > Warning + > ::: + > + > In a production system remember to change the default admin credentials `admin` `geoserver` + > ::: + > + > ![](img/oauth005.png) + +2. Access the `Security` \> `Users, Groups, Roles` section + + > ![](img/oauth006.png) + +3. **If not yet configured** the service `geonode REST role service`, click on `Role Services` \> `Add new` + + > ::: note + > ::: title + > Note + > ::: + > + > This passage is **not** needed if the `geonode REST role service` has been already created. If so it will be displayed among the Role Services list + > + > ![](img/oauth008.png) + > ::: + > + > ![](img/oauth007.png) + +4. **If not yet configured** the service `geonode REST role service`, choose `AuthKEY REST - Role service from REST endpoint` + + > ![](img/oauth009.png) + +5. Create / update the `geonode REST role service` accordingly + + > ![](img/oauth010.png) + > + > - `Name`; **Must** be `geonode REST role service` + > - `Base Server URL`; Must point to the GeoNode instance base URL (e.g. `http://`) + > - `Roles REST Endpoint`; Enter `/api/roles` + > - `Admin Role REST Endpoint`; Enter `/api/adminRole` + > - `Users REST Endpoint`; Enter `/api/users` + > - `Roles JSON Path`; Enter `$.groups` + > - `Admin Role JSON Path`; Enter `$.adminRole` + > - `Users JSON Path`; Enter `$.users[0].groups` + > + > Once everything has been setup and it is working, choose the `Administrator role` and `Group administrator role` as `ROLE_ADMIN` + +### Allow GeoFence to validate rules with `ROLES` + +::: warning +::: title +Warning +::: + +The following instruction are different accordingly to the GeoServer version you are currently using. +::: + +## GeoServer 2.9.x and 2.10.x + +1. Access the `Security` \> `Settings` section + + > ![](img/oauth011.png) + +2. Choose the `geonode REST role service` as `Active role service` + + > ![](img/oauth012.png) + +## GeoServer 2.12.x and above + +With the latest updates to GeoFence Plugin, the latter no more recognizes the Role Service from the default settings but from the `geofence-server.properties` file. + +That said, it is important that the `Security` \> `Settings` role service will be set to **default**, in order to allow GeoServer following the standard authorization chain. + +On the other side, you will need to be sure that the `geofence-server.properties` file under the `$GEOSERVER_DATA_DIR/geofence` folder, contains the two following additional properties: : + + gwc.context.suffix=gwc + org.geoserver.rest.DefaultUserGroupServiceName=geonode REST role service + +## Setup of the GeoServer OAuth2 Authentication Filter + +It is necessary now check that GeoServer can connect to OAuth2 Providers (specifically to GeoNode OP), and being able to Authenticate users through it. + +### Preliminary checks + +- GeoServer is up and running and you have admin rights +- GeoServer must reach the GeoNode instance via HTTP +- OAuth2 `Client ID` and `Client Secret` have been generated on GeoNode and known + +### Setup of the GeoNode OAuth2 Security Filter + +1. Access the `Security` \> `Authentication` section + + > ![](img/oauth013.png) + +2. **If not yet configured** the Authentication Filter `geonode-oauth2 - Authentication using a GeoNode OAuth2`, click on `Authentication Filters` \> `Add new` + + > ::: note + > ::: title + > Note + > ::: + > + > This passage is **not** needed if the `geonode-oauth2 - Authentication using a GeoNode OAuth2` has been already created. If so it will be displayed among the Authentication Filters list + > + > ![](img/oauth015.png) + > ::: + > + > ![](img/oauth014.png) + +3. **If not yet configured** the Authentication Filter `geonode-oauth2 - Authentication using a GeoNode OAuth2`, choose `GeoNode OAuth2 - Authenticates by looking up for a valid GeoNode OAuth2 access_token key sent as URL parameter` + + > ![](img/oauth016.png) + +4. Create / update the `geonode-oauth2 - Authentication using a GeoNode OAuth2` accordingly + + > ![](img/oauth017.png) + > + > - `Name`; **Must** be `geonode-oauth2` + > - `Enable Redirect Authentication EntryPoint`; It is recommended to put this to `False`, otherwise GeoServer won\'t allow you to connect to its Admin GUI through the `Form` but only through GeoNode + > - `Login Authentication EndPoint`; Unless you have specific needs, keep the default value `/j_spring_oauth2_geonode_login` + > - `Logout Authentication EndPoint`; Unless you have specific needs, keep the default value `/j_spring_oauth2_geonode_logout` + > - `Force Access Token URI HTTPS Secured Protocol`; This must be `False` unless you enabled a `Secured Connection` on GeoNode. In that case you will need to trust the GeoNode `Certificate` on the GeoServer JVM Keystore. Please see details below + > - `Access Token URI`; Set this to `http:///o/token/` + > - `Force User Authorization URI HTTPS Secured Protocol`; This must be `False` unless you enabled a `Secured Connection` on GeoNode. In that case you will need to trust the GeoNode `Certificate` on the GeoServer JVM Keystore. Please see details below + > - `User Authorization URI`; Set this to `http:///o/authorize/` + > - `Redirect URI`; Set this to `http:///geoserver`. This address **must** be present on the `Redirect uris` of GeoNode `OAuth2` \> `Applications` \> `GeoServer` (see above) + > - `Check Token Endpoint URL`; Set this to `http:///api/o/v4/tokeninfo/` + > - `Logout URI`; Set this to `http:///account/logout/` + > - `Scopes`; Unless you have specific needs, keep the default value `read,write,groups` + > - `Client ID`; The `Client id` alphanumeric key generated by the GeoNode `OAuth2` \> `Applications` \> `GeoServer` (see above) + > - `Client Secret`; The `Client secret` alphanumeric key generated by the GeoNode `OAuth2` \> `Applications` \> `GeoServer` (see above) + > - `Role source`; In order to authorize the user against GeoNode, choose `Role service` \> `geonode REST role service` + +## Configuration of the GeoServer Filter Chains + +The following steps ensure GeoServer can adopt more Authentication methods. As stated above, it is possible to Authenticate to GeoServer using different protocols. + +GeoServer scans the authentication filters chain associated to the specified path and tries them one by one sequentially. +The first one matching the protocol and able to grant access to the user, breaks the cycle by creating a `User Principal` and injecting it into the GeoServer `SecurityContext`. +The Authentication process, then, ends here and the control goes to the Authorization one, which will try to retrieve the authenticated user\'s Roles through the available GeoServer Role Services associated to the Authentication Filter that granted the access. + +### Preliminary checks + +- GeoServer is up and running and you have admin rights +- GeoServer must reach the GeoNode instance via HTTP +- The `geonode-oauth2 - Authentication using a GeoNode OAuth2` Authentication Filter and the `geonode REST role service` have been correctly configured + +### Setup of the GeoServer Filter Chains + +1. Access the `Security` \> `Authentication` section + + > ![](img/oauth013.png) + +2. Identify the section `Filter Chains` + + > ![](img/oauth018.png) + +3. Make sure the `web` Filter Chain is configured as shown below + + > ![](img/oauth019.png) + > + > ::: warning + > ::: title + > Warning + > ::: + > + > Every time you modify a Filter Chain, **don\'t forget to save** the `Authentication` settings. This **must** be done for **each** change. + > + > ![](img/oauth020.png) + > ::: + +4. Make sure the `rest` Filter Chain is configured as shown below + + > ![](img/oauth021.png) + > + > ::: warning + > ::: title + > Warning + > ::: + > + > Every time you modify a Filter Chain, **don\'t forget to save** the `Authentication` settings. This **must** be done for **each** change. + > + > ![](img/oauth020.png) + > ::: + +5. Make sure the `gwc` Filter Chain is configured as shown below + + > ![](img/oauth022.png) + > + > ::: warning + > ::: title + > Warning + > ::: + > + > Every time you modify a Filter Chain, **don\'t forget to save** the `Authentication` settings. This **must** be done for **each** change. + > + > ![](img/oauth020.png) + > ::: + +6. Make sure the `default` Filter Chain is configured as shown below + + > ![](img/oauth023.png) + > + > ::: warning + > ::: title + > Warning + > ::: + > + > Every time you modify a Filter Chain, **don\'t forget to save** the `Authentication` settings. This **must** be done for **each** change. + > + > ![](img/oauth020.png) + > ::: + +7. Add the `GeoNode Login Endpoints` to the comma-delimited list of the `webLogin` Filter Chain + + > ![](img/oauth024.png) + > + > ::: warning + > ::: title + > Warning + > ::: + > + > Every time you modify a Filter Chain, **don\'t forget to save** the `Authentication` settings. This **must** be done for **each** change. + > + > ![](img/oauth020.png) + > ::: + +8. Add the `GeoNode Logout Endpoints` to the comma-delimited list of the `webLogout` Filter Chain + + > ![](img/oauth025.png) + > + > ::: warning + > ::: title + > Warning + > ::: + > + > Every time you modify a Filter Chain, **don\'t forget to save** the `Authentication` settings. This **must** be done for **each** change. + > + > ![](img/oauth020.png) + > ::: + +9. Add the `GeoNode Logout Endpoints` to the comma-delimited list of the `formLogoutChain` XML node in `/security/filter/formLogout/config.xml` + + > You will need a text editor to modify the file. + > + > ::: note + > ::: title + > Note + > ::: + > + > If the `` XML node does not exist at all, create a **new one** as specified below + > ::: + > + > ``` xml + > + > ... + > /web/ + > /j_spring_security_logout,/j_spring_security_logout/,/j_spring_oauth2_geonode_logout,/j_spring_oauth2_geonode_logout/ + > + > ``` + > + > ::: warning + > ::: title + > Warning + > ::: + > + > The value `j_spring_oauth2_geonode_logout` **must** be the same specified as `Logout Authentication EndPoint` in the `geonode-oauth2 - Authentication using a GeoNode OAuth2` above. + > ::: + +## Setup and test of the GeoFence Server and Default Rules + +In order to work correctly, GeoServer needs the [GeoFence Embedded Server](http://docs.geoserver.org/latest/en/user/community/geofence-server/index.html) plugin to be installed and configured on the system. + +The GeoServer configuration provided for GeoNode, has the plugin already installed with a default configuration. In that case double check that the plugin works correctly and the default rules have been setup by following the next steps. + +### Preliminary checks + +- GeoServer is up and running and you have admin rights +- The [GeoFence Embedded Server](http://docs.geoserver.org/latest/en/user/community/geofence-server/index.html) plugin has been installed on GeoServer + +### Setup of the GeoServer Filter Chains + +1. Access the `Security` \> `Authentication` section + + > ![](img/oauth013.png) + +2. Identify the section `Authentication Providers` and make sure the `geofence` Authentication Provider is present + + > ![](img/oauth032.png) + +3. Make sure the `Provider Chain` is configured as shown below + + > ![](img/oauth033.png) + > + > ::: warning + > ::: title + > Warning + > ::: + > + > Every time you modify an Authentication Providers, **don\'t forget to save** the `Authentication` settings. This **must** be done for **each** change. + > + > ![](img/oauth020.png) + > ::: + +### Setup of the GeoFence Server and Rules + +1. Make sure GeoFence server works and the default settings are correctly configured + + > - Access the `Security` \> `GeoFence` section + > + > > ![](img/oauth026.png) + > + > - Make sure the `Options` are configured as follows and the server works well when performing a `Test Connection` + > + > > ![](img/oauth027.png) + > > + > > - `Allow remote and inline layers in SLD`; Set it to `True` + > > - `Allow SLD and SLD_BODY parameters in requests`; Set it to `True` + > > - `Authenticated users can write`; Set it to `True` + > > - `Use GeoServer roles to get authorizations`; Set it to `False` + +2. Check the GeoFence default Rules + + > - Access the `Security` \> `GeoFence Data Rules` section + > + > > ![](img/oauth028.png) + > + > - Make sure the `DENY ALL` Rule is present by default, otherwise your data will be accessible to everyone + > + > > ::: note + > > ::: title + > > Note + > > ::: + > > + > > This rule is **always** the last one + > > ::: + > > + > > ![](img/oauth029.png) + > > + > > ::: warning + > > ::: title + > > Warning + > > ::: + > > + > > If that rule does not exists **at the very bottom** (this rule is **always** the last one), add it manually. + > > ::: + > + > - Access the `Security` \> `GeoFence Admin Rules` section + > + > > ![](img/oauth030.png) + > + > - No Rules needed here + > + > > ![](img/oauth031.png) + +## Troubleshooting and Advanced Features + +### Common Issues and Fixes + +- GeoServer/GeoNode OAuth2 does not authenticate as Administrator even using GeoNode `admin` users + + > **Symptoms** + > + > > When trying to authenticate with an `admin` user using OAuth2, the process correctly redirects to GeoServer page but I\'m not a GeoServer Administrator. + > + > **Cause** + > + > > That means that somehow GeoServer could not successfully complete the Authorization and Authentication process. + > > + > > The possible causes of the problem may be the following ones: + > > + > > 1. The OAuth2 Authentication fails on GeoServer side + > > + > > > This is usually due to an exception while trying to complete the Authentication process. + > > > + > > > - A typical cause is that GeoServer tries to use HTTPS connections but the GeoNode certificate is not trusted; + > > > + > > > > In that case please refer to the section below. Also take a look at the logs (in particular the GeoServer one) as explained in `debug_geonode`{.interpreted-text role="ref"}. + > > > > The GeoServer logs should contain a detailed Exception explaining the cause of the problem. + > > > > If no exception is listed here (even after raised the log level to *DEBUG*), try to check for the GeoNode Role Service as explained below. + > > > + > > > - Another possible issue is that somehow the OAuth2 handshake cannot complete successfully; + > > > + > > > > 1. Login into GeoServer as administrator through its WEB login form. + > > > > 2. Double check that all the `geonode-oauth2 - Authentication using a GeoNode OAuth2` parameters are correct. If everything is ok, take a look at the logs (in particular the GeoServer one) as explained in `debug_geonode`{.interpreted-text role="ref"}. + > > > > The GeoServer logs should contain a detailed Exception explaining the cause of the problem. If no exception is listed here (even after raised the log level to *DEBUG*), try to check for the GeoNode Role Service as explained below. + > > + > > 2. GeoServer is not able to retrieve the user Role from a Role Service + > > + > > > Always double check both HTTP Server and GeoServer log as specified in section `debug_geonode`{.interpreted-text role="ref"}. This might directly guide you to the cause of the problem. + > > > + > > > - Check that the GeoServer host is granted to access GeoNode Role Service REST APIs in the `AUTH_IP_WHITELIST` of the `settings.py` + > > > + > > > - Check that the `geonode REST role service` is the default Role service and that the GeoServer OAuth2 Plugin has been configured to use it by default + > > > + > > > - Check that the GeoNode REST Role Service APIs are functional and produce correct JSON. + > > > + > > > > This is possible by using simple `cUrl` GET calls like + > > > > + > > > > ``` shell + > > > > curl http://localhost/api/adminRole + > > > > $> {"adminRole": "admin"} + > > > > + > > > > + > > > > curl http://localhost/api/users + > > > > $> {"users": [{"username": "AnonymousUser", "groups": ["anonymous"]}, {"username": "afabiani", "groups": ["anonymous", "test"]}, {"username": "admin", "groups": ["anonymous", "test", "admin"]}]} + > > > > + > > > > curl http://localhost/api/roles + > > > > $> {"groups": ["anonymous", "test", "admin"]} + > > > > + > > > > curl http://localhost/api/users/admin + > > > > $> {"users": [{"username": "admin", "groups": ["anonymous", "test", "admin"]}]} + > > > > ``` + +### How to setup `HTTPS` secured endpoints + +In a production system it is a good practice to encrypt the connection between GeoServer and GeoNode. That would be possible by enabling HTTPS Protocol on the GeoNode REST Role Service APIs and OAuth2 Endpoints. + +Most of the times you will rely on a self-signed HTTPS connection using a generated certificate. That makes the connection *untrusted* and you will need to tell to the GeoServer Java Virtual Machine to trust it. + +This can be done by following the steps below. + +For any issue take a look at the logs (in particular the GeoServer one) as explained in `debug_geonode`{.interpreted-text role="ref"}. The GeoServer logs should contain a detailed Exception explaining the cause of the problem. + +## SSL Trusted Certificates + +When using a custom `Keystore` or trying to access a non-trusted or self-signed SSL-protected OAuth2 Provider from a non-SSH connection, you will need to add the certificates to the JVM `Keystore`. + +In order to do this you can follow the next steps: + +> In this example we are going to +> +> 1. Retrieve SSL Certificate from GeoNode domain: +> +> > \"Access Token URI\" = \/o/token/ therefore we need to trust `https://` or (`:443`) +> > +> > ::: note +> > ::: title +> > Note +> > ::: +> > +> > You will need to get and trust certificates from every different HTTPS URL used on OAuth2 Endpoints. +> > ::: +> +> 2. Store SSL Certificates on local hard-disk +> +> 3. Add SSL Certificates to the Java Keystore +> +> 4. Enable the JVM to check for SSL Certificates from the Keystore + +1. Retrieve the SSL Certificate from GeoNode domain + + > Use the `openssl` command in order to dump the certificate + > + > For `https://` + > + > > ``` shell + > > openssl s_client -connect :443 + > > ``` + > > + > > ![](img/google_ssl_001.png){.align-center} + +2. Store SSL Certificate on local hard-disk + + > Copy-and-paste the section `-BEGIN CERTIFICATE-`, `-END CERTIFICATE-` and save it into a `.cert` file + > + > ::: note + > ::: title + > Note + > ::: + > + > `.cert` file are plain text files containing the ASCII characters included on the `-BEGIN CERTIFICATE-`, `-END CERTIFICATE-` sections + > ::: + > + > `geonode.cert` (or whatever name you want with `.cert` extension) + > + > > ![](img/google_ssl_003.png){.align-center} + +3. Add SSL Certificates to the Java Keystore + + > You can use the Java command `keytool` like this + > + > `geonode.cert` (or whatever name you want with `.cert` extension) + > + > > ``` shell + > > keytool -import -noprompt -trustcacerts -alias geonode -file geonode.cert -keystore ${KEYSTOREFILE} -storepass ${KEYSTOREPASS} + > > ``` + > + > or, alternatively, you can use some graphic tool which helps you managing the SSL Certificates and Keystores, like [Portecle](http://portecle.sourceforge.net/) + > + > > ``` shell + > > java -jar c:\apps\portecle-1.9\portecle.jar + > > ``` + > > + > > ![](img/google_ssl_005.png){.align-center} + > > + > > ![](img/google_ssl_006.png){.align-center} + > > + > > ![](img/google_ssl_007.png){.align-center} + > > + > > ![](img/google_ssl_008.png){.align-center} + > > + > > ![](img/google_ssl_009.png){.align-center} + > > + > > ![](img/google_ssl_010.png){.align-center} + > > + > > ![](img/google_ssl_011.png){.align-center} + > > + > > ![](img/google_ssl_012.png){.align-center} + > > + > > ![](img/google_ssl_013.png){.align-center} + +4. Enable the JVM to check for SSL Certificates from the Keystore + + > In order to do this, you need to pass a `JAVA_OPTION` to your JVM: + > + > > ``` shell + > > -Djavax.net.ssl.trustStore=F:\tmp\keystore.key + > > ``` + +5. Restart your server + +::: note +::: title +Note +::: + +Here below you can find a bash script which simplifies the Keystore SSL Certificates importing. Use it at your convenience. + +``` shell +HOST=myhost.example.com +PORT=443 +KEYSTOREFILE=dest_keystore +KEYSTOREPASS=changeme + +# get the SSL certificate +openssl s_client -connect ${HOST}:${PORT} ${HOST}.cert + +# create a keystore and import certificate +keytool -import -noprompt -trustcacerts \ + -alias ${HOST} -file ${HOST}.cert \ + -keystore ${KEYSTOREFILE} -storepass ${KEYSTOREPASS} + +# verify we've got it. +keytool -list -v -keystore ${KEYSTOREFILE} -storepass ${KEYSTOREPASS} -alias ${HOST} +``` +::: + +## GeoFence Advanced Features + +### GeoFence Rules Management and Tutorials + +- [This tutorial](http://docs.geoserver.org/latest/en/user/community/geofence-server/tutorial.html) shows how to install and configure the Geofence Internal Server plug-in. It shows how to create rules in two ways: using the GUI and REST methods. +- GeoFence Rules can be created / updated / deleted through a REST API, accessible only by a GeoServer Admin user. You can find more details on how the GeoFence REST API works [here](https://github.com/geoserver/geofence/wiki/REST-API). + +### GeoFence Rules Storage Configuration + +By default GeoFence is configured to use a filesystem based DB stored on the GeoServer Data Dir ` ``` shell + > wget --no-check-certificate "https://maven.geo-solutions.it/org/hibernatespatial/hibernate-spatial-postgis/1.1.3.2/hibernate-spatial-postgis-1.1.3.2.jar" -O hibernate-spatial-postgis-1.1.3.2.jar + > wget --no-check-certificate "https://repo1.maven.org/maven2/org/postgis/postgis-jdbc/1.3.3/postgis-jdbc-1.3.3.jar" -O postgis-jdbc-1.3.3.jar + > + > cp hibernate-spatial-postgis-1.1.3.2.jar /WEB-INF/lib + > cp postgis-jdbc-1.3.3.jar /WEB-INF/lib + > + > restart geoserver + > ``` + +2. Either create a DB with the updated schema here or enable the hbm2ddl auto creation through the configuration file (see step `3`) + + > ::: note + > ::: title + > Note + > ::: + > + > Notice that \"update\" also creates the tables if they do not exist. In production, however, I would suggest to change it to \"validate\" + > ::: + > + > ``` shell + > # If you want to create a new DB for GeoFence + > sudo -u postgres createdb -O geonode geofence; \ + > sudo -u postgres psql -d geofence -c 'CREATE EXTENSION postgis;'; \ + > sudo -u postgres psql -d geofence -c 'GRANT ALL ON geometry_columns TO PUBLIC;'; \ + > sudo -u postgres psql -d geofence -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;'; \ + > sudo -u postgres psql -d geofence -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO geonode;' + > ``` + +3. Add configuration similar to `geofence-datasource-ovr.properties` sample below (if loaded as GeoServer extension) + + > **\/geofence/geofence-datasource-ovr.properties** + > + > ``` shell + > # /* (c) 2019 Open Source Geospatial Foundation - all rights reserved + > # * This code is licensed under the GPL 2.0 license, available at the root + > # * application directory. + > # */ + > # + > geofenceVendorAdapter.databasePlatform=org.hibernatespatial.postgis.PostgisDialect + > geofenceDataSource.driverClassName=org.postgresql.Driver + > geofenceDataSource.url=jdbc:postgresql://localhost:5432/geofence + > geofenceDataSource.username=postgres + > geofenceDataSource.password=postgres + > geofenceEntityManagerFactory.jpaPropertyMap[hibernate.default_schema]=public + > + > ################################################################################ + > ## Other setup entries + > ################################################################################ + > ## hbm2ddl.auto may assume one of these values: + > ## - validate: validates the DB schema at startup against the internal model. May fail on oracle spatial. + > ## - update: updates the schema, according to the internal model. Updating automatically the production DB is dangerous. + > ## - create-drop: drop the existing schema and recreates it according to the internal model. REALLY DANGEROUS, YOU WILL LOSE YOUR DATA. + > ## You may want not to redefine the property entirely, in order to leave the default value (no action). + > + > geofenceEntityManagerFactory.jpaPropertyMap[hibernate.hbm2ddl.auto]=update + > geofenceEntityManagerFactory.jpaPropertyMap[javax.persistence.validation.mode]=none + > geofenceEntityManagerFactory.jpaPropertyMap[hibernate.validator.apply_to_ddl]=false + > geofenceEntityManagerFactory.jpaPropertyMap[hibernate.validator.autoregister_listeners]=false + > + > ## + > ## ShowSQL is set to true in the configuration file; putting showsql=false in + > ## this file, you can easily check that this override file has been properly applied. + > + > # geofenceVendorAdapter.generateDdl=false + > # geofenceVendorAdapter.showSql=false + > + > ## Set to "true" in specific use cases + > # workspaceConfigOpts.showDefaultGroups=false + > + > + > + > ################################################################################ + > ## Disable second level cache. + > ## This is needed in a geofence-clustered environment. + > + > #geofenceEntityManagerFactory.jpaPropertyMap[hibernate.cache.use_second_level_cache]=false + > + > ################################################################################ + > ## Use external ehcache configuration file. + > ## Useful to change cache settings, for example diskStore path. + > #geofenceEntityManagerFactory.jpaPropertyMap[hibernate.cache.provider_configuration_file_resource_path]=file:/path/to/geofence-ehcache-override.xml + > ``` diff --git a/advanced/contrib/img/add_centralized_server.png b/docs/advanced/contrib/img/add_centralized_server.png similarity index 100% rename from advanced/contrib/img/add_centralized_server.png rename to docs/advanced/contrib/img/add_centralized_server.png diff --git a/advanced/contrib/img/centralized_server_admin_ui.png b/docs/advanced/contrib/img/centralized_server_admin_ui.png similarity index 100% rename from advanced/contrib/img/centralized_server_admin_ui.png rename to docs/advanced/contrib/img/centralized_server_admin_ui.png diff --git a/advanced/contrib/img/dispatch_metrics_task.png b/docs/advanced/contrib/img/dispatch_metrics_task.png similarity index 100% rename from advanced/contrib/img/dispatch_metrics_task.png rename to docs/advanced/contrib/img/dispatch_metrics_task.png diff --git a/advanced/contrib/img/dispatch_metrics_task_details.png b/docs/advanced/contrib/img/dispatch_metrics_task_details.png similarity index 100% rename from advanced/contrib/img/dispatch_metrics_task_details.png rename to docs/advanced/contrib/img/dispatch_metrics_task_details.png diff --git a/advanced/contrib/img/periodic_tasks_section.png b/docs/advanced/contrib/img/periodic_tasks_section.png similarity index 100% rename from advanced/contrib/img/periodic_tasks_section.png rename to docs/advanced/contrib/img/periodic_tasks_section.png diff --git a/docs/advanced/contrib/index.md b/docs/advanced/contrib/index.md new file mode 100644 index 00000000..c3e43652 --- /dev/null +++ b/docs/advanced/contrib/index.md @@ -0,0 +1,322 @@ +# Geonode auth via LDAP + +This package provides utilities for using LDAP as an authentication and +authorization backend for geonode. + +The [django_auth_ldap](https://django-auth-ldap.readthedocs.io/en/latest/) package is a very capable way to add LDAP integration +with django projects. It provides a lot of flexibility in mapping LDAP users to +geonode users and is able to manage user authentication. + +However, in order to provide full support for mapping LDAP groups with +geonode\'s and enforce group permissions on resources, a custom geonode +authentication backend is required. This contrib package provides such a +backend, based on [django_auth_ldap](https://django-auth-ldap.readthedocs.io/en/latest/). + +## Installation + +Installing this contrib package is a matter of: + +1. Installing geonode +2. Installing system LDAP libraries (development packages needed) +3. Cloning this repository locally +4. Change to the [ldap]{.title-ref} directory and install this contrib package + +``` sh +# 1. install geonode (not shown here for brevity) +# 2. install systemwide LDAP libraries +sudo apt install \ + libldap2-dev \ + libsasl2-dev + +# 3. get geonode/contribs code +git clone https://github.com/GeoNode/geonode-contribs.git + +# 4. install geonode ldap contrib package +cd geonode-contribs/ldap +pip install . +``` + +## Configuration + +1. Add `geonode_ldap.backend.GeonodeLdapBackend` as an additional auth + backend. + + ``` python + # e.g. by updating your settings.py or local_settings.py + AUTHENTICATION_BACKENDS += ( + "geonode_ldap.backend.GeonodeLdapBackend", + ) + ``` + + You may use additional auth backends, the django authentication framework + tries them all according to the order listed in the settings. This means that + geonode can be setup in such a way as to permit internal organization users + to login with their LDAP credentials, while at the same time allowing for + casual users to use their facebook login (as long as you enable facebook + social auth provider). + + ::: note + ::: title + Note + ::: + + The django\'s `django.contrib.auth.backends.ModelBackend` must also be used in order to provide full geonode integration with LDAP. + However this is included by default on GeoNode `settings` + + ``` python + # The GeoNode default settings are the following + AUTHENTICATION_BACKENDS = ( + 'oauth2_provider.backends.OAuth2Backend', + 'django.contrib.auth.backends.ModelBackend', + 'guardian.backends.ObjectPermissionBackend', + 'allauth.account.auth_backends.AuthenticationBackend', + ) + ``` + ::: + +2. Set some additional configuration values. Some of these variables are + prefixed with `AUTH_LDAP` (these are used directly by [django_auth_ldap](https://django-auth-ldap.readthedocs.io/en/latest/)) + while others are prefixed with `GEONODE_LDAP` (these are used by + `geonode_ldap`). The geonode custom variables are: + + - `GEONODE_LDAP_GROUP_PROFILE_FILTERSTR` - This is an LDAP search fragment + with the filter that allows querying for existing groups. See example below + - `GEONODE_LDAP_GROUP_NAME_ATTRIBUTE` - This is the name of the LDAP + attribute that will be used for deriving the geonode group name. If not + specified it will default to [cn]{.title-ref}, which means that the LDAP object\'s + [common name]{.title-ref} will be used for generating the name of the geonode group + - `GEONODE_LDAP_GROUP_PROFILE_MEMBER_ATTR` - This is the name of the LDAP + attribute that will be used for deriving the geonode membership. If not + specified it will default to `member` + +Example configuration: + +``` python +# add these import lines to the top of your geonode settings file +from django_auth_ldap import config as ldap_config +from geonode_ldap.config import GeonodeNestedGroupOfNamesType +import ldap + +# enable logging +import logging +logger = logging.getLogger('django_auth_ldap') +logger.addHandler(logging.StreamHandler()) +logger.setLevel(logging.DEBUG) + +# add both standard ModelBackend auth and geonode.contrib.ldap auth +AUTHENTICATION_BACKENDS += ( + 'geonode_ldap.backend.GeonodeLdapBackend', +) + +# django_auth_ldap configuration +AUTH_LDAP_SERVER_URI = os.getenv("LDAP_SERVER_URL") +AUTH_LDAP_BIND_DN = os.getenv("LDAP_BIND_DN") +AUTH_LDAP_BIND_PASSWORD = os.getenv("LDAP_BIND_PASSWORD") +AUTH_LDAP_USER_SEARCH = ldap_config.LDAPSearch( + os.getenv("LDAP_USER_SEARCH_DN"), + ldap.SCOPE_SUBTREE, + os.getenv("LDAP_USER_SEARCH_FILTERSTR") +) +# should LDAP groups be used to spawn groups in GeoNode? +AUTH_LDAP_MIRROR_GROUPS = strtobool(os.getenv("LDAP_MIRROR_GROUPS", 'True')) +AUTH_LDAP_GROUP_SEARCH = ldap_config.LDAPSearch( + os.getenv("LDAP_GROUP_SEARCH_DN"), + ldap.SCOPE_SUBTREE, + os.getenv("LDAP_GROUP_SEARCH_FILTERSTR") +) +AUTH_LDAP_GROUP_TYPE = GeonodeNestedGroupOfNamesType() +AUTH_LDAP_USER_ATTR_MAP = { + "first_name": "givenName", + "last_name": "sn", + "email": "mailPrimaryAddress" +} +AUTH_LDAP_FIND_GROUP_PERMS = True +AUTH_LDAP_MIRROR_GROUPS_EXCEPT = [ + "test_group" +] + +# these are not needed by django_auth_ldap - we use them to find and match +# GroupProfiles and GroupCategories +GEONODE_LDAP_GROUP_NAME_ATTRIBUTE = os.getenv("LDAP_GROUP_NAME_ATTRIBUTE", default="cn") +GEONODE_LDAP_GROUP_PROFILE_FILTERSTR = os.getenv("LDAP_GROUP_SEARCH_FILTERSTR", default='(ou=research group)') +GEONODE_LDAP_GROUP_PROFILE_MEMBER_ATTR = os.getenv("LDAP_GROUP_PROFILE_MEMBER_ATTR", default='member') +``` + +Example environment variables: + +``` shell +LDAP_SERVER_URL=ldap:// +LDAP_BIND_DN=uid=ldapinfo,cn=users,dc=ad,dc=example,dc=org +LDAP_BIND_PASSWORD= +LDAP_USER_SEARCH_DN=dc=ad,dc=example,dc=org +LDAP_USER_SEARCH_FILTERSTR=(&(uid=%(user)s)(objectClass=person)) +LDAP_MIRROR_GROUPS=True +LDAP_GROUP_SEARCH_DN=cn=groups,dc=ad,dc=example,dc=org +LDAP_GROUP_SEARCH_FILTERSTR=(|(cn=abt1)(cn=abt2)(cn=abt3)(cn=abt4)(cn=abt5)(cn=abt6)) +LDAP_GROUP_PROFILE_MEMBER_ATTR=uniqueMember +``` + +The configuration seen in the example above will allow LDAP users to login to +geonode with their LDAP credentials. + +On first login, a geonode user is created from the LDAP user and its LDAP +attributes `cn` and `sn` are used to populate the geonode user\'s +`first_name` and `last_name` profile fields. + +Any groups that the user is a member of in LDAP (under the +`cn=groups,dc=ad,dc=example,dc=org` search base and belonging to one of +`(|(cn=abt1)(cn=abt2)(cn=abt3)(cn=abt4)(cn=abt5)(cn=abt6))` groups) will be mapped to the corresponding +geonode groups, even creating these groups in geonode in case they do not +exist yet. The geonode user is also made a member of these geonode groups. + +Upon each login, the user\'s geonode group memberships are re-evaluated +according to the information extracted from LDAP. The +`AUTH_LDAP_MIRROR_GROUPS_EXCEPT` setting can be used to specify groups +whose memberships will not be re-evaluated. + +If no LDAP groups shall be mirrored `LDAP_MIRROR_GROUPS` and `LDAP_MIRROR_GROUPS_EXCEPT` must be set to `False`. + +::: note +::: title +Note +::: + +Users mapped from LDAP will be marked with an `ldap` tag. This will be used to keep them in sync. +::: + +::: warning +::: title +Warning +::: + +If you remove the `ldap` tag, the users will be threaten as pure internal GeoNode ones. +::: + +You may also manually generate the geonode groups in advance, before users +login. In this case, when a user logs in and the mapped LDAP group already +exists, the user is merely added to the geonode group + +Be sure to check out [django_auth_ldap](https://django-auth-ldap.readthedocs.io/en/latest/) for more information on the various +configuration options. + +## Keep Users and Groups Synchronized + +In order to constantly keep the remote LDAP Users and Groups **synchronized** with GeoNode, +you will need to run periodically some specific management commands. + +``` shell +*/10 * * * * /opt/geonode/my-geonode/manage.sh updateldapgroups >> /var/log/cron.log 2>&1 +*/10 * * * * /opt/geonode/my-geonode/manage.sh updateldapusers >> /var/log/cron.log 2>&1 +``` + +Where the `manage.sh` is a bash script similar to the following one: + +**manage.sh** + +``` shell +export $(grep -v '^#' /opt/geonode/my-geonode/.env | xargs -d '\n'); /home//.virtualenvs/geonode/bin/python /opt/geonode/my-geonode/manage.py $@ +``` + +and the `/opt/geonode/my-geonode/.env` is something similar to the following one: + +**/opt/geonode/my-geonode/.env** + +``` shell +DEBUG=False +DJANGO_ALLOWED_HOSTS=,localhost,127.0.0.1 +DJANGO_DATABASE_URL=postgis://my_geonode:**********@localhost:5432/my_geonode_db +DEFAULT_BACKEND_UPLOADER=geonode.importer +DEFAULT_FROM_EMAIL=geonode@example.org +DJANGO_EMAIL_HOST=smtp.example.org +DJANGO_EMAIL_HOST_PASSWORD=********** +DJANGO_EMAIL_HOST_USER=geonode +DJANGO_EMAIL_PORT=465 +DJANGO_EMAIL_USE_SSL=True +DJANGO_SETTINGS_MODULE=my_geonode.settings +DJANGO_SECRET_KEY=********** +OAUTH2_API_KEY=********** +PROXY_URL=/proxy/?url= +EXIF_ENABLED=True +EMAIL_ENABLE=True +TIME_ENABLED=True +ACCOUNT_OPEN_SIGNUP=True +ACCOUNT_APPROVAL_REQUIRED=True +ACCOUNT_EMAIL_REQUIRED=True +ACCOUNT_EMAIL_VERIFICATION=optional +AVATAR_GRAVATAR_SSL=True +GEONODE_DB_URL=postgis://my_geonode:**********@localhost:5432/my_geonode_data +GEOSERVER_ADMIN_PASSWORD=********** +GEOSERVER_LOCATION=https:///geoserver/ +GEOSERVER_PUBLIC_HOST= +GEOSERVER_PUBLIC_LOCATION=https:///geoserver/ +GEOSERVER_WEB_UI_LOCATION=https:///geoserver/ +LDAP_SERVER_URL=ldap:// +LDAP_BIND_DN=uid=ldapinfo,cn=users,dc=ad,dc=example,dc=org +LDAP_BIND_PASSWORD= +LDAP_USER_SEARCH_DN=dc=ad,dc=example,dc=org +LDAP_USER_SEARCH_FILTERSTR=(&(uid=%(user)s)(objectClass=person)) +LDAP_MIRROR_GROUPS=True +LDAP_GROUP_SEARCH_DN=cn=groups,dc=ad,dc=example,dc=org +LDAP_GROUP_SEARCH_FILTERSTR=(|(cn=abt1)(cn=abt2)(cn=abt3)(cn=abt4)(cn=abt5)(cn=abt6)) +LDAP_GROUP_PROFILE_MEMBER_ATTR=uniqueMember +OGC_REQUEST_MAX_RETRIES=3 +OGC_REQUEST_POOL_CONNECTIONS=100 +OGC_REQUEST_POOL_MAXSIZE=100 +OGC_REQUEST_TIMEOUT=60 +SITEURL=https:/// +SITE_HOST_NAME= +FREETEXT_KEYWORDS_READONLY=False +# Advanced Workflow Settings +ADMIN_MODERATE_UPLOADS=False +GROUP_MANDATORY_RESOURCES=False +GROUP_PRIVATE_RESOURCES=False +RESOURCE_PUBLISHING=False +``` + +::: note +::: title +Note +::: + +You might want to use the same `/opt/geonode/my-geonode/.env` for your `UWSGI` configuration too: + +``` shell +[uwsgi] +socket = 0.0.0.0:8000 +uid = +gid = www-data + +plugins = python3 +virtualenv = /home//.virtualenvs/geonode + +# set environment variables from .env file +env LANG=en_US.utf8 +env LC_ALL=en_US.UTF-8 +env LC_LANG=en_US.UTF-8 + +for-readline = /opt/geonode/my-geonode/.env + env = %(_) +endfor = + +chdir = /opt/geonode/my-geonode +module = my_geonode.wsgi:application + +processes = 12 +threads = 2 +enable-threads = true +master = true + +# logging +# path to where uwsgi logs will be saved +logto = /storage/my_geonode/logs/geonode.log +daemonize = /storage/my_geonode/logs/geonode.log +touch-reload = /opt/geonode/my-geonode/my_geonode/wsgi.py +buffer-size = 32768 +max-requests = 500 +harakiri = 300 # respawn processes taking more than 5 minutes (300 seconds) +# limit-as = 1024 # avoid Errno 12 cannot allocate memory +harakiri-verbose = true +vacuum = true +thunder-lock = true +``` +::: diff --git a/docs/advanced/hardening/index.md b/docs/advanced/hardening/index.md new file mode 100644 index 00000000..d68acb79 --- /dev/null +++ b/docs/advanced/hardening/index.md @@ -0,0 +1,33 @@ +# Publish on other than HTTP port (for e.g. 8082) {#geonode_on_https} + +By default geonode will be installed in the port 80 (i.e. HTTP) port. But what if you want to change the port of the geonode to other than HTTP port (For this example, I am taking 8082 port)? We need to edit couple of things in the web configuration. First things is, we need to update the [/etc/uwsgi/apps-enabled/geonode.ini]{.title-ref} file, + +``` shell +sudo vi /etc/uwsgi/apps-enabled/geonode.ini +``` + +Edit the following lines, + +``` shell +env = SITE_HOST_NAME=localhost:8082 +env = SITEURL=http://localhost:8082 + +SITE_HOST_NAME=localhost +SITE_HOST_PORT=8082 +GEOSERVER_WEB_UI_LOCATION=http://localhost:8082/geoserver/ +GEOSERVER_PUBLIC_LOCATION=http://localhost:8082/geoserver/ +``` + +After that we need to update the [/etc/nginx/sites-enabled/geonode]{.title-ref} file, + +``` shell +sudo vi /etc/nginx/sites-enabled/geonode +``` + +Edit the following lines, + +``` shell +server { + listen 8082 default_server; + listen [::]:8082 default_server; +``` diff --git a/docs/advanced/index.md b/docs/advanced/index.md new file mode 100644 index 00000000..6eefe52a --- /dev/null +++ b/docs/advanced/index.md @@ -0,0 +1,37 @@ +# GeoNode Components and Architecture + +::: {.toctree caption="GeoNode Components and Architecture" maxdepth="3"} +components/index +::: + +# Hardening GeoNode + +::: {.toctree caption="Hardening GeoNode" maxdepth="3"} +hardening/index +::: + +# Social Login + +::: {.toctree caption="Social Login" maxdepth="3"} +social/index +::: + +# GeoNode Django Contrib Apps + +::: {.toctree caption="GeoNode Django Contrib Apps" maxdepth="3"} +contrib/index +::: + + +### GeoNode Components and Architecture +- [index](components/index.md) + +### Hardening GeoNode +- [index](hardening/index.md) + +### Social Login +- [index](social/index.md) + +### GeoNode Django Contrib Apps +- [index](contrib/index.md) + diff --git a/advanced/social/img/geonode_logo.png b/docs/advanced/social/img/geonode_logo.png similarity index 100% rename from advanced/social/img/geonode_logo.png rename to docs/advanced/social/img/geonode_logo.png diff --git a/docs/advanced/social/index.md b/docs/advanced/social/index.md new file mode 100644 index 00000000..e63e7c44 --- /dev/null +++ b/docs/advanced/social/index.md @@ -0,0 +1,112 @@ +# GeoNode Social Accounts + +::: {.contents depth="4"} +::: + +## Allow GeoNode to Login throguh Social Accounts (OIDC) + +### Introduction + +> Through the so-called \"social accounts,\" GeoNode allows authentication through external providers that support the OIDC (OpenID Connect) protocol, such as Google and Microsoft Azure. +> +> How does it work? Once the authentication provider is configured, GeoNode\'s sign-on will display a new login button that redirects the user to the external authentication page. After successfully authenticating with the external provider and accepting privacy consents, the browser will redirect the user back to the GeoNode page, prompting them to enter any missing information the first time and automatically authenticating them on subsequent occasions. + +### Usage + +> Once the provider has been correctly configured (see below), GeoNode will allow the user to login through it. +> +> The first time you login you will probably need to confirm you [e-mail]{.title-ref} and other fields of the [profile]{.title-ref}. +> +> ::: note +> ::: title +> Note +> ::: +> +> If you want a user to be automatically signed as a member of a group, you will need to: +> +> > 1. Create the [GroupProfile]{.title-ref} in GeoNode +> > 2. Provide the [groups]{.title-ref} or [roles]{.title-ref} the user belongs to throguh the [id_token]{.title-ref} or [user_info]{.title-ref} metadata from the [OIDC]{.title-ref} provider itself. +> +> Notice that, in the case you would like to benefit from this functionality: +> +> > 1. Every time the user sing-in again, the groups will be automatically re-assigned by GeoNode, and therefore it won\'t be possible to assign them manually anymore. +> > 2. If you need a user to be recognized as a [manager]{.title-ref} of the [groups]{.title-ref} declared from the provider, you will need to send a claim [is_manager: True]{.title-ref} on the user info metadata. +> ::: + +### Quick Configuration + +> Currently GeoNode comes with two predefined configurations that you can use to enable either Google or Microsoft Azure. +> +> **Google** +> +> 1. Add to your [.env]{.title-ref} the following settings +> +> ``` shell +> SOCIALACCOUNT_OIDC_PROVIDER_ENABLED=True +> SOCIALACCOUNT_PROVIDER=google +> ``` +> +> 2. Login into GeoNode as an [admin]{.title-ref}, go to the [Social Account]{.title-ref} settings, create a new [geonode_openid_connect]{.title-ref} provider and insert the +> +> ``` shell +> Client ID +> Client Secret +> ``` +> +> **Microsoft Azure** +> +> 1. Add to your [.env]{.title-ref} the following settings +> +> ``` shell +> MICROSOFT_TENANT_ID= +> SOCIALACCOUNT_OIDC_PROVIDER_ENABLED=True +> SOCIALACCOUNT_PROVIDER=azure +> ``` +> +> 2. Login into GeoNode as an [admin]{.title-ref}, go to the [Social Account]{.title-ref} settings, create a new [geonode_openid_connect]{.title-ref} provider and insert the +> +> ``` shell +> Client ID +> Client Secret +> ``` + +### Advanced Configuration + +> In the case you need to change the default behavior of GeoNode or add a new/custom OIDC provider, you will need to update the [settings]{.title-ref} manually as follows. +> +> ``` python +> SOCIALACCOUNT_PROVIDERS = { +> SOCIALACCOUNT_OIDC_PROVIDER: { +> "NAME": "Your Custom Provider", +> "SCOPE": [ +> # Custom scopes comma-separated +> ], +> "AUTH_PARAMS": { +> # Custom AUTH PARAMS +> }, +> "COMMON_FIELDS": {"email": "email", "last_name": "family_name", "first_name": "given_name"}, # Custom common fields mappings +> "IS_MANAGER_FIELD": "the_custom_manager_claim", # This is optional +> "ACCOUNT_CLASS": "the_custom_account_class", +> "ACCESS_TOKEN_URL": "the_custom_token_uri", +> "AUTHORIZE_URL": "the_custom_auth_uri", +> "ID_TOKEN_ISSUER": "the_custom_uri", # or "PROFILE_URL": "the_custom_user_info_uri"; if you specify the "ID_TOKEN_ISSUER" this will take precedence +> "OAUTH_PKCE_ENABLED": True, +> } +> } +> ``` +> +> ::: note +> ::: title +> Note +> ::: +> +> If you specify the \"ID_TOKEN_ISSUER\" this will take precedence trying to fetch the user info metadata from the [id_token]{.title-ref}. +> +> If the [id_token]{.title-ref} won\'t be available, it will try to fallback to the \"PROFILE_URL\" uri. +> ::: +> +> In the case you will need to customzie how the [Adapter]{.title-ref} works and manages the [Groups]{.title-ref} registration, you can inject a new class throguh the settings: +> +> > ``` shell +> > SOCIALACCOUNT_ADAPTER="geonode.people.adapters.GenericOpenIDConnectAdapter" # This is the default value +> > ``` diff --git a/docs/basic/docker_env_vars/index.md b/docs/basic/docker_env_vars/index.md new file mode 100644 index 00000000..c660c809 --- /dev/null +++ b/docs/basic/docker_env_vars/index.md @@ -0,0 +1,201 @@ +# Docker Environmental variables {#dockerenvvars} + +This list reports the specific environmental variables requried for the configuration and execution of GeoNode Docker services + +## Geoserver + +DATABASE_HOST + +: | Default: `db` + + Internal IP or hostname for the PosgreSQL (`db`) Docker container. + +DATABASE_PORT + +: | Default: `django` + + Internal port for the PosgreSQL (`db`) Docker container. + +GEONODE_LB_HOST_IP + +: | Default: `django` + + Internal IP or hostname for the GeoNode (`django`) Docker container. + +GEONODE_LB_PORT + +: | Default: `8000` + + Internal port for the GeoNode (`django`) Docker container. + +GEONODE_GEODATABSE + +: | Default: `geonode_data` + + GeoNode geodatabase name. + +GEONODE_GEODATABASE_USER + +: | Default: `geonode_data` + + GeoNode geodatabase user. + +GEONODE_GEODATABASE_PASSWORD + +: | Default: `geonode_data` + + GeoNode geodatabase password. + +GEONODE_GEODATABASE_SCHEMA + +: | Default: `publix` + + GeoNode geodatabase schema. + +GEOSERVER_JAVA_OPTS + +: | Default: `'-Djava.awt.headless=true -Xms4G -Xmx4G -Dgwc.context.suffix=gwc -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:LogFile=/var/log/jvm.log -XX:PerfDataSamplingInterval=500 -XX:SoftRefLRUPolicyMSPerMB=36000 -XX:-UseGCOverheadLimit -XX:ParallelGCThreads=4 -Dfile.encoding=UTF8 -Djavax.servlet.request.encoding=UTF-8 -Djavax.servlet.response.encoding=UTF-8 -Duser.timezone=GMT -Dorg.geotools.shapefile.datetime=false -DGS-SHAPEFILE-CHARSET=UTF-8 -DGEOSERVER_CSRF_DISABLED=true -DPRINT_BASE_URL={geoserver_ui}/geoserver/pdf -DALLOW_ENV_PARAMETRIZATION=true -Xbootclasspath/a:/usr/local/tomcat/webapps/geoserver/WEB-INF/lib/marlin-0.9.3-Unsafe.jar -Dsun.java2d.renderer=org.marlin.pisces.MarlinRenderingEngine'` + + Geoserver JAVA OPTS. + +GEOSERVER_LB_HOST_IP + +: | Default: `geoserver` + + Internal IP or hostname for the GeoNode (`geoserver`) Docker container. + +GEOSERVER_LB_PORT + +: | Default: `8080` + + Internal port for the GeoNode (`geoserver`) Docker container. + +NGINX_BASE_URL + +: | Default: `http(s):/localhost` + + Public URL of the `nginx` service. This value must match GeoNodes\'s `SITEURL` setting + +OAUTH2_API_KEY + +: | Default: `(empty)` + + Optionsl API Key for the communication between Geoserver and GeoNode Oauth2 services. + +OAUTH2_CLIENT_ID + +: | Default: `(empty)` + + Geoserer Client ID for GeoNode Oauth2 server. Cannot be empty. + +OAUTH2_CLIENT_SECRET + +: | Default: `(empty)` + + Geoserer Client secret for GeoNode Oauth2 server. Cannot be empty. + +## Nginx + +GEONODE_LB_HOST_IP + +: | Default: `django` + + Internal IP or hostname for the GeoNode (`django`) Docker container. + +GEONODE_LB_PORT + +: | Default: `8000` + + Internal port for the GeoNode (`django`) Docker container. + +GEOSERVER_LB_HOST_IP + +: | Default: `geoserver` + + Internal IP or hostname for the GeoNode (`geoserver`) Docker container. + +GEOSERVER_LB_PORT + +: | Default: `8080` + + Internal port for the GeoNode (`geoserver`) Docker container. + +HTTPS_HOST + +: | Default: `localhost` + + The publis hostname for HTTPS (wihtout `http:/` scheme prefix). + +HTTPS_HOST + +: | Default: `localhost` + + The publis hostname for HTTPS (wihtout `https:/` scheme prefix). + +HTTP_PORT + +: | Default: `80` + + Host port on which to expose the Nginx service for HTTP. + +HTTPS_PORT + +: | Default: `443` + + Host port on which to expose the Nginx service for HTTPS. + +LETSENCRYPT_MODE + +: | Default: `disabled` + + Possible values: `disabled` (SSL disabled), `staging` (SSL local cert), `production` (SSL released by an ACME server) + +## PostgreSQL / PostGIS + +GEONODE_DATABASE + +: | Default: `geonode` + + GeoNode database name. + +GEONODE_DATABASE_USER + +: | Default: `geonode_data` + + GeoNode database user. + +GEONODE_DATABASE_PASSWORD + +: | Default: `geonode_data` + + GeoNode database password. + +GEONODE_GEODATABSE + +: | Default: `geonode_data` + + GeoNode geodatabase name. + +GEONODE_GEODATABASE_USER + +: | Default: `geonode_data` + + GeoNode geodatabase user. + +GEONODE_GEODATABASE_PASSWORD + +: | Default: `geonode_data` + + GeoNode geodatabase password. + +POSTGRES_USER + +: | Default: `geonode_data` + + PostgreSQL admin user. + +POSTGRES_USER + +: | Default: `geonode_data` + + PostgreSQL admin password. diff --git a/docs/basic/index.md b/docs/basic/index.md new file mode 100644 index 00000000..fdedd6fd --- /dev/null +++ b/docs/basic/index.md @@ -0,0 +1,37 @@ +# GeoNode Settings + +> `settings`{.interpreted-text role="ref"} + +::: {.toctree hidden="" maxdepth="3"} +settings/index +::: + +# Docker Environmental variables + +> `dockerenvvars`{.interpreted-text role="ref"} + +::: {.toctree hidden="" maxdepth="3"} +docker_env_vars/index +::: + +# Customize the Look and Feel + +::: {.toctree maxdepth="3"} +theme/index +::: + +# GeoNode permissions + +::: {.toctree maxdepth="3"} +permissions/index +::: + + +- [index](settings/index.md) + +- [index](docker_env_vars/index.md) + +- [index](theme/index.md) + +- [index](permissions/index.md) + diff --git a/basic/permissions/img/add_to_shopping_cart.png b/docs/basic/permissions/img/add_to_shopping_cart.png similarity index 100% rename from basic/permissions/img/add_to_shopping_cart.png rename to docs/basic/permissions/img/add_to_shopping_cart.png diff --git a/basic/permissions/img/change_dataset_permissions.png b/docs/basic/permissions/img/change_dataset_permissions.png similarity index 100% rename from basic/permissions/img/change_dataset_permissions.png rename to docs/basic/permissions/img/change_dataset_permissions.png diff --git a/basic/permissions/img/delete_icon.png b/docs/basic/permissions/img/delete_icon.png similarity index 100% rename from basic/permissions/img/delete_icon.png rename to docs/basic/permissions/img/delete_icon.png diff --git a/basic/permissions/img/edit_icon.png b/docs/basic/permissions/img/edit_icon.png similarity index 100% rename from basic/permissions/img/edit_icon.png rename to docs/basic/permissions/img/edit_icon.png diff --git a/basic/permissions/img/geo_limits_001.png b/docs/basic/permissions/img/geo_limits_001.png similarity index 100% rename from basic/permissions/img/geo_limits_001.png rename to docs/basic/permissions/img/geo_limits_001.png diff --git a/basic/permissions/img/geo_limits_002.png b/docs/basic/permissions/img/geo_limits_002.png similarity index 100% rename from basic/permissions/img/geo_limits_002.png rename to docs/basic/permissions/img/geo_limits_002.png diff --git a/basic/permissions/img/geo_limits_003.png b/docs/basic/permissions/img/geo_limits_003.png similarity index 100% rename from basic/permissions/img/geo_limits_003.png rename to docs/basic/permissions/img/geo_limits_003.png diff --git a/basic/permissions/img/geo_limits_004.png b/docs/basic/permissions/img/geo_limits_004.png similarity index 100% rename from basic/permissions/img/geo_limits_004.png rename to docs/basic/permissions/img/geo_limits_004.png diff --git a/basic/permissions/img/refresh_icon.png b/docs/basic/permissions/img/refresh_icon.png similarity index 100% rename from basic/permissions/img/refresh_icon.png rename to docs/basic/permissions/img/refresh_icon.png diff --git a/basic/permissions/img/resource_permissions_dialogue.png b/docs/basic/permissions/img/resource_permissions_dialogue.png similarity index 100% rename from basic/permissions/img/resource_permissions_dialogue.png rename to docs/basic/permissions/img/resource_permissions_dialogue.png diff --git a/docs/basic/permissions/index.md b/docs/basic/permissions/index.md new file mode 100644 index 00000000..b01b3918 --- /dev/null +++ b/docs/basic/permissions/index.md @@ -0,0 +1,96 @@ +# Permissions + +| Permissions in GeoNode are set per resource, where a resource can be a dataset, a map, a document, a service or a geoapp. The way the permissions are set is the same for all of them. + +::: warning +::: title +Warning +::: + +GeoNode has a set of default permissions that are applied on resource creation **when** you don\'t explicitly declare them. +This is particularly relevant when creating and saving a map, where you won\'t have the possibility to set the its permissions during the creation phase. +GeoNode can be tuned to make sure that by default the new created resource are not public, this can be done by changing two settings, see [Default view permissions](../settings/index.html#default-anonymous-view-permission) and [Default download permissions](../settings/index.html#default-anonymous-download-permission) +::: + +## Single Resource permissions + +| Resource permissions can be generally set from the *resource detail* page. The detail page has a menu item *Share* which is visible to people who are permitted to set permissions on a resource. + +The share link opens a page on the right with a provision to edit user and group permissions on the resource. see picture below + +
+img/change_dataset_permissions.png +
Change Dataset Permissions
+
+ +| The page for setting the permissions, allows addition of users/groups and selection of a permission to assign each of them. + +
+img/resource_permissions_dialogue.png +
Resource Permission Dialogue
+
+ +You can set the following types of permissions: + +- *View:* allows to view the resource; +- *Download* allows to download the resource; +- *Edit:* allows to change attributes, properties of the datasets features, styles and metadata for the specified resource; +- *Manage:* allows to update, delete, change permissions, publish and unpublish the resource. + +::: warning +::: title +Warning +::: + +When assigning permissions to a group, all the group members will have those permissions. Be careful in case of editing permissions. +::: + +## Geo Limits permissions + +::: note +::: title +Note +::: + +This feature is available **only** when enabling `` `GeoServer ``[ as geospatial backend. Also make sure that the properties ]{.title-ref}`GEONODE_SECURITY_ENABLED`[, ]{.title-ref}`GEOFENCE_SECURITY_ENABLED`[ and ]{.title-ref}`GEOFENCE_URL`[ are correctly set for the ]{.title-ref}`OGC_SERVER`\`. +::: + +`Geo Limits`{.interpreted-text role="guilabel"} are an extension of the GeoNode standard permissions. `Geo Limits`{.interpreted-text role="guilabel"} allows the owner of the resource, or the administrator, to restrict users or groups to a specific geographical area, in order to limit the access to the dataset to only the portions contained within that geographic restriction, excluding data outside of it. + +In order to be able to set `Geo Limits`{.interpreted-text role="guilabel"} you must be an `` `administrator ``[ of the system or the ]{.title-ref}`owner`[ of the resource or you must have ]{.title-ref}`Manage Permissions`\` rights to the resource. + +If you have the permissions to set the `Geo Limits`{.interpreted-text role="guilabel"}, you should be able to see the permissions section and the globe icon on each user or group. + +
+img/geo_limits_001.png +
Geo Limits Icon
+
+ +You should be able to see an interactive preview of the resource along with few small drawing tools, that allows you to start creating limits on the map manually if you want. + +This opens a map dialog, with 3 options at the top: + +
+img/geo_limits_002.png +
Geo Limits: Preview Window with Drawing Tools
+
+ +The ![edit_icon](img/edit_icon.png){.align-middle width="30px" height="30px"} icon allows you to draw limits on a map for which a user will be able to see. Click on it to start drawing on the map. Once you are done drawing, click on it again to deactivate drawing mode. + +The ![delete_icon](img/delete_icon.png){.align-middle width="30px" height="30px"} icon enables you to remove the limits you have drawn. Click on the limit drawn, and then click the delete icon. + +The ![refresh_icon](img/refresh_icon.png){.align-middle width="30px" height="30px"} icon removes all changes that are not saved. + +
+img/geo_limits_003.png +
Geo Limits: Editing the Geometries
+
+ +Once you finished editing your geometries, save them into the DB using the *Save* link in the resource menu. + +The user with the specified geometries won\'t be able from now on to access the whole dataset data. + +
+img/geo_limits_004.png +
Geo Limits: Geospatial restrictions applies for the user
+
diff --git a/docs/basic/settings/index.md b/docs/basic/settings/index.md new file mode 100644 index 00000000..8b3375cf --- /dev/null +++ b/docs/basic/settings/index.md @@ -0,0 +1,2409 @@ +# Settings + +Here's a list of settings available in GeoNode and their default values. This includes settings for some external applications that +GeoNode depends on. + +For most of them, default values are good. Those should be changed only for advanced configurations in production or heavily hardened systems. + +The most common ones can be set through `environment` variables to avoid touching the `settings.py` file at all. +This is a good practice and also the preferred one to configure GeoNode (and Django apps in general). +Whenever you need to change them, set the environment variable accordingly (where it is available) instead of overriding it through the `local_settings`. + +## A + +### ACCESS_TOKEN_EXPIRE_SECONDS + +> | Default: `86400` +> | Env: `ACCESS_TOKEN_EXPIRE_SECONDS` +> +> When a user logs into GeoNode, if no `ACCESS_TOKEN` exists, a new one will be created with a default expiration time of `ACCESS_TOKEN_EXPIRE_SECONDS` seconds (1 day by default). + +### ACCOUNT_ADAPTER + +> | Default: `geonode.people.adapters.LocalAccountAdapter` +> +> Custom GeoNode People (Users) Account Adapter. + +### ACCOUNT_APPROVAL_REQUIRED + +> | Default: `False` +> | Env: `ACCOUNT_APPROVAL_REQUIRED` +> +> If `ACCOUNT_APPROVAL_REQUIRED` equals `True`, newly registered users must be activated by a superuser through the Admin gui, before they can access GeoNode. + +### ACCOUNT_CONFIRM_EMAIL_ON_GET + +> | Default: `True` +> +> This is a [django-allauth setting](https://django-allauth.readthedocs.io/en/latest/configuration.html#configuration) +> It allows specifying the HTTP method used when confirming e-mail addresses. + +### ACCOUNT_EMAIL_REQUIRED + +> | Default: `True` +> +> This is a [django-allauth setting](https://django-allauth.readthedocs.io/en/latest/configuration.html#configuration) +> which controls whether the user is required to provide an e-mail address upon registration. + +### ACCOUNT_EMAIL_VERIFICATION + +> | Default: `optional` +> +> This is a [django-allauth setting](https://django-allauth.readthedocs.io/en/latest/configuration.html#configuration) + +### ACCOUNT_LOGIN_REDIRECT_URL + +> | Default: `SITEURL` +> | Env: `LOGIN_REDIRECT_URL` +> +> This is a [django-user-accounts setting](https://django-user-accounts.readthedocs.io/en/latest/settings.html) +> It allows specifying the default redirect URL after a successful login. + +### ACCOUNT_LOGOUT_REDIRECT_URL + +> | Default: `SITEURL` +> | Env: `LOGOUT_REDIRECT_URL` +> +> This is a [django-user-accounts setting](https://django-user-accounts.readthedocs.io/en/latest/settings.html) +> It allows specifying the default redirect URL after a successful logout. + +### ACCOUNT_NOTIFY_ON_PASSWORD_CHANGE + +> | Default: `True` +> | Env: `ACCOUNT_NOTIFY_ON_PASSWORD_CHANGE` +> +> This is a [django-user-accounts setting](https://django-user-accounts.readthedocs.io/en/latest/settings.html) + +### ACCOUNT_OPEN_SIGNUP + +> | Default: `True` +> | Env: `ACCOUNT_OPEN_SIGNUP` +> +> This is a [django-user-accounts setting](https://django-user-accounts.readthedocs.io/en/latest/settings.html) +> Whether or not people are allowed to self-register to GeoNode or not. + +### ACCOUNT_SIGNUP_FORM_CLASS + +> | Default: `geonode.people.forms.AllauthReCaptchaSignupForm` +> | Env: `ACCOUNT_SIGNUP_FORM_CLASS` +> +> Enabled only when the `recaptcha_enabled`{.interpreted-text role="ref"} option is `True`. +> +> Ref. to `recaptcha_enabled`{.interpreted-text role="ref"} + +### ACTSTREAM_SETTINGS + +> Default: +> +> { +> 'FETCH_RELATIONS': True, +> 'USE_PREFETCH': False, +> 'USE_JSONFIELD': True, +> 'GFK_FETCH_DEPTH': 1, +> } +> +> Actstream Settings. + +### ADDITIONAL_DATASET_FILE_TYPES + +External application can define additional supported file type other than the default one declared in the [SUPPORTED_DATASET_FILE_TYPES]{.title-ref} . + +The variable should be declared in this way in [settings.py]{.title-ref} (or via application hook): + +> ``` ADDITIONAL_DATASET_FILE_TYPES=[ +> { +> "id": "dummy_type", +> "label": "Dummy Type", +> "format": "dummy", +> "ext": ["dummy"] +> }, +> ] +> ``` + +Please rely on geonode.tests.test_utils.TestSupportedTypes for an example + +### ADMIN_IP_WHITELIST + +> | Default: `[]` +> +> When this list is popuplated with a list of IPs or IP ranges (e.g. 192.168.1.0/24) requests from and admin user will be allowe only from IPs matching with the list. + +### ADMIN_MODERATE_UPLOADS + +> | Default: `False` +> +> When this variable is set to `True`, every uploaded resource must be approved before becoming visible to the public users. +> +> Until a resource is in `PENDING APPROVAL` state, only the superusers, owner and group members can access it, unless specific edit permissions have been set for other users or groups. +> +> A `Group Manager` *can* approve the resource, but he cannot publish it whenever the setting `RESOURCE_PUBLISHING` is set to `True`. +> Otherwise, if `RESOURCE_PUBLISHING` is set to `False`, the resource becomes accessible as soon as it is approved. + +### ADMINS_ONLY_NOTICE_TYPES + +> | Default: `['monitoring_alert',]` +> +> A list of notification labels that standard users should not either see or set. +> +> Such notifications will be hidden from the notify settings page and automatically set to false for non-superusers. + +### ADVANCED_EDIT_EXCLUDE_FIELD + +> | Default: `[]` +> +> A list of element (item name) to exclude from the Advanced Edit page. +> +> Example: +> +> `ADVANCED_EDIT_EXCLUDE_FIELD=['title', 'keywords', 'tkeywords']` + +### AGON_RATINGS_CATEGORY_CHOICES + +> Default: +> +> { +> "maps.Map": { +> "map": "How good is this map?" +> }, +> "layers.Layer": { +> "layer": "How good is this layer?" +> }, +> "documents.Document": { +> "document": "How good is this document?" +> } +> } + +### ALLOWED_DOCUMENT_TYPES + +> Default: +> +> ['doc', 'docx', 'gif', 'jpg', 'jpeg', 'ods', 'odt', 'odp', 'pdf', 'png', +> 'ppt', 'pptx', 'rar', 'sld', 'tif', 'tiff', 'txt', 'xls', 'xlsx', 'xml', +> 'zip', 'gz', 'qml'] +> +> A list of acceptable file extensions that can be uploaded to the Documents app. + +### ANONYMOUS_USER_ID + +> | Default: `-1` +> | Env: `ANONYMOUS_USER_ID` +> +> The id of an anonymous user. This is an django-guardian setting. + +### API_INCLUDE_REGIONS_COUNT + +> | Default: `False` +> | Env: `API_INCLUDE_REGIONS_COUNT` +> +> If set to `True`, a counter with the total number of available regions will be added to the API JSON Serializer. + +### API_LIMIT_PER_PAGE + +> | Default: `200` +> | Env: `API_LIMIT_PER_PAGE` +> +> The Number of items returned by the APIs 0 equals no limit. Different from `CLIENT_RESULTS_LIMIT`, affecting the number of items per page in the resource list. + +### API_LOCKDOWN + +> | Default: `True` +> | Env: `API_LOCKDOWN` +> +> If this is set to `True` users must be authenticated to get search results when search for for users, groups, categories, regions, tags etc. +> Filtering search results of Resourcebase-objects like Layers, Maps or Documents by one of the above types does not work. +> Attention: If API_LOCKDOWN is set to `False` all details can be accessed by anonymous users. + +### ASYNC_SIGNALS + +> | Default: `False` +> | Env: `ACCOUNT_NOTIFY_ON_PASSWORD_CHANGE` + +### AUTH_EXEMPT_URLS + +> Default: +> +> (r'^/?$', +> '/gs/*', +> '/static/*', +> '/o/*', +> '/api/o/*', +> '/api/roles', +> '/api/adminRole', +> '/api/users', +> '/api/layers',) +> +> A tuple of URL patterns that the user can visit without being authenticated. +> This setting has no effect if `LOCKDOWN_GEONODE` is not True. For example, +> `AUTH_EXEMPT_URLS = ('/maps',)` will allow unauthenticated users to +> browse maps. + +### AUTO_ASSIGN_REGISTERED_MEMBERS_TO_CONTRIBUTORS + +> | Default: `True` +> | Env: `AUTO_ASSIGN_REGISTERED_MEMBERS_TO_CONTRIBUTORS` +> +> Assign **new** registered users to the contributors group. If set to [False]{.title-ref} new registered members will not obtain automatic permissions to create and edit resources. + +### AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_NAME + +> | Default: `True` +> | Env: `AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_NAME` +> +> Auto assign users to a default `REGISTERED_MEMBERS_GROUP_NAME` private group after `AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_AT`. + +### AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_AT + +> | Default: `activation` +> | Env: `AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_AT` +> | Options: `"registration" | "activation" | "login"` +> +> Auto assign users to a default `REGISTERED_MEMBERS_GROUP_NAME` private group after {\"registration\" \| \"activation\" \| \"login\"}. +> +> Notice that whenever `ACCOUNT_EMAIL_VERIFICATION == True` and `ACCOUNT_APPROVAL_REQUIRED == False`, users will be able to register and they became `active` already, even if they won\'t be able to login until the email has been verified. + +### AUTO_GENERATE_AVATAR_SIZES + +> | Default: `20, 30, 32, 40, 50, 65, 70, 80, 100, 140, 200, 240` +> +> An iterable of integers representing the sizes of avatars to generate on upload. This can save rendering time later on if you pre-generate the resized versions. + +### AVATAR_GRAVATAR_SSL + +> | Default: `False` +> | Env: `AVATAR_GRAVATAR_SSL` +> | Options: `True | False` +> +> Force SSL when loading fallback image from gravatar.com. + +### AVATAR_DEFAULT_URL + +> | Default: `/geonode/img/avatar.png` +> | Env: `AVATAR_GRAVATAR_SSL` +> | Options: `"filepath to image"` +> +> Allows to set a custom fallback image in case a User has not uploaded a profile image. +> Needs `AVATAR_PROVIDERS` to be set correctly. + +### AVATAR_PROVIDERS + +> | Default: +> +> ``` +> 'avatar.providers.PrimaryAvatarProvider','avatar.providers.GravatarAvatarProvider','avatar.providers.DefaultAvatarProvider' +> ``` +> +> | Env: `AVATAR_PROVIDERS` +> | Options: `Avatar provider object` +> +> This setting configures in which order gravatar images are loaded. A common use case is the use of a local image over a fallback image loaded from gravatar.com. +> To do so you would change the order like: +> +> ``` +> 'avatar.providers.PrimaryAvatarProvider','avatar.providers.DefaultAvatarProvider','avatar.providers.GravatarAvatarProvider' +> ``` +> +> (DefaultAvatarProvider before GravatarAvatarProvider) + +## B + +### BING_API_KEY + +> | Default: `None` +> | Env: `BING_API_KEY` +> +> This property allows to enable a Bing Aerial background. +> +> If using `mapstore` client library, make sure the `MAPSTORE_BASELAYERS` include the following: +> +> ``` python +> if BING_API_KEY: +> BASEMAP = { +> "type": "bing", +> "title": "Bing Aerial", +> "name": "AerialWithLabels", +> "source": "bing", +> "group": "background", +> "apiKey": "{{apiKey}}", +> "visibility": False +> } +> DEFAULT_MS2_BACKGROUNDS = [BASEMAP,] + DEFAULT_MS2_BACKGROUNDS +> ``` + +### BROKER_HEARTBEAT + +> | Default: `0` +> +> Heartbeats are used both by the client and the broker to detect if a connection was closed. +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#broker-heartbeat). + +### BROKER_TRANSPORT_OPTIONS + +> Default: +> +> { +> 'fanout_prefix': True, +> 'fanout_patterns': True, +> 'socket_timeout': 60, +> 'visibility_timeout': 86400 +> } +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +## C + +### CACHES + +> Default: +> +> CACHES = { +> 'default': { +> 'BACKEND': 'django.core.cache.backends.dummy.DummyCache', +> }, +> 'resources': { +> 'BACKEND': 'django.core.cache.backends.locmem.LocMemCache', +> 'TIMEOUT': 600, +> 'OPTIONS': { +> 'MAX_ENTRIES': 10000 +> } +> } +> } +> +> A dictionary containing the settings for all caches to be used with Django. +> This is a [Django setting](https://docs.djangoproject.com/en/3.2/ref/settings/#std:setting-CACHES) +> +> The `'default'` cache is disabled because we don\'t have a mechanism to discriminate between client sessions right now, and we don\'t want all users fetch the same api results. +> +> The `'resources'` is not currently used. It might be helpful for [caching Django template fragments](https://docs.djangoproject.com/en/3.2/topics/cache/#template-fragment-caching) and/or [Tastypie API Caching](https://django-tastypie.readthedocs.io/en/latest/caching.html). + +### CACHE_BUSTING_STATIC_ENABLED + +> | Default: `False` +> | Env: `CACHE_BUSTING_STATIC_ENABLED` +> +> This is a [Django Compressed Manifet storage provided by WhiteNoise](http://whitenoise.evans.io/en/stable/django.html#add-compression-and-caching-support) +> A boolean allowing you to enable the `WhiteNoise CompressedManifestStaticFilesStorage storage`. This works only on a production system. +> +> ::: warning +> ::: title +> Warning +> ::: +> +> This works only if `DEBUG = False` +> ::: + +### CASCADE_WORKSPACE + +> | Default: `geonode` +> | Env: `CASCADE_WORKSPACE` + +### CATALOGUE + +> A dict with the following keys: +> +> > ENGINE: The CSW backend (default is `geonode.catalogue.backends.pycsw_local`) +> > URL: The FULLY QUALIFIED base URL to the CSW instance for this GeoNode +> > USERNAME: login credentials (if required) +> > PASSWORD: login credentials (if required) +> +> pycsw is the default CSW enabled in GeoNode. pycsw configuration directives +> are managed in the PYCSW entry. + +### CATALOGUE_METADATA_TEMPLATE + +> Default : `catalogue/full_metadata.xml` +> +> A string with the catalogue xml file needed for the metadata. + +### CATALOGUE_METADATA_XSL + +> Default : `'/static/metadataxsl/metadata.xsl` +> +> A string pointing to the XSL used to transform the metadata XML into human readable HTML. + +### CELERYD_POOL_RESTARTS + +> Default: `True` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_ACCEPT_CONTENT + +> Defaul: `['json']` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_ACKS_LATE + +> Default: `True` +> +> This is a [Celery setting](http://docs.celeryproject.org/en/3.1/configuration.html#celery-acks-late) + +### CELERY_BEAT_SCHEDULE + +> Here you can define your scheduled task. + +### CELERY_DISABLE_RATE_LIMITS + +> Default: `False` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_ENABLE_UTC + +> Default: `True` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_MAX_CACHED_RESULTS + +> Default: `32768` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_MESSAGE_COMPRESSION + +> Default: `gzip` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_RESULT_PERSISTENT + +> Default: `False` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_RESULT_SERIALIZER + +> Default: `json` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_SEND_TASK_SENT_EVENT + +> Default: `True` +> +> If enabled, a task-sent event will be sent for every task so tasks can be tracked before they are consumed by a worker. This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_TASK_ALWAYS_EAGER + +> Default: `False if ASYNC_SIGNALS else True` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_TASK_CREATE_MISSING_QUEUES + +> Default: `True` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_TASK_IGNORE_RESULT + +> Default: `True` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_TASK_QUEUES + +> Default: +> +> Queue('default', GEONODE_EXCHANGE, routing_key='default'), +> Queue('geonode', GEONODE_EXCHANGE, routing_key='geonode'), +> Queue('update', GEONODE_EXCHANGE, routing_key='update'), +> Queue('cleanup', GEONODE_EXCHANGE, routing_key='cleanup'), +> Queue('email', GEONODE_EXCHANGE, routing_key='email'), +> +> A tuple with registered Queues. + +### CELERY_TASK_RESULT_EXPIRES + +> | Default: `43200` +> | Env: `CELERY_TASK_RESULT_EXPIRES` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_TASK_SERIALIZER + +> Default: `json` +> Env: `CELERY_TASK_SERIALIZER` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_TIMEZONE + +> | Default: `UTC` +> | Env: `TIME_ZONE` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_TRACK_STARTED + +> Default: `True` +> +> This is a [Celery setting](https://docs.celeryproject.org/en/latest/userguide/configuration.html#new-lowercase-settings). + +### CELERY_WORKER_DISABLE_RATE_LIMITS + +> Default: `False` +> +> Disable the worker rate limits (number of tasks that can be run in a given time frame). + +### CELERY_WORKER_SEND_TASK_EVENTS + +> Default: `False` +> +> Send events so the worker can be monitored by other tools. + +### CLIENT_RESULTS_LIMIT + +> | Default: `5` +> | Env: `CLIENT_RESULTS_LIMIT` +> +> The Number of results per page listed in the GeoNode search pages. Different from `API_LIMIT_PER_PAGE`, affecting the number of items returned by the APIs. + +### CORS_ALLOW_ALL_ORIGINS + +> | Default: `False` +> | Env: `CORS_ALLOW_ALL_ORIGINS` +> +> If set to true [Access-Control-Allow-Origin: \*]{.title-ref} header is set for any response. A safer option (not managed through env vars at the moment) is [CORS_ALLOWED_ORIGINS]{.title-ref}, where a list of hosts can be configured, o [CORS_ALLOWED_ORIGIN_REGEXES]{.title-ref}, where the list can contain regexes. +> Notice that the Nginx in front of GeoNode always includes [Access-Control-Allow-Credentials true]{.title-ref}. This must also taken into account when CORS is enabled. + +### CREATE_LAYER + +> | Default: `False` +> | Env: `CREATE_LAYER` +> +> Enable the create layer plugin. + +### CKAN_ORIGINS + +> Default: +> +> CKAN_ORIGINS = [{ +> "label":"Humanitarian Data Exchange (HDX)", +> "url":"https://data.hdx.rwlabs.org/dataset/new?title={name}¬es={abstract}", +> "css_class":"hdx" +> }] +> +> A list of dictionaries that are used to generate the links to CKAN instances displayed in the Share tab. For each origin, the name and abstract format parameters are replaced by the actual values of the ResourceBase object (layer, map, document). This is not enabled by default. To enable, uncomment the following line: SOCIAL_ORIGINS.extend(CKAN_ORIGINS). + +### CSRF_COOKIE_HTTPONLY + +> | Default: `False` +> | Env: `CSRF_COOKIE_HTTPONLY` +> +> Whether to use HttpOnly flag on the CSRF cookie. If this is set to True, client-side JavaScript will not be able to access the CSRF cookie. This is a [Django Setting](https://docs.djangoproject.com/en/3.2/ref/settings/#csrf-cookie-httponly) + +### CSRF_COOKIE_SECURE + +> | Default: `False` +> | Env: `CSRF_COOKIE_SECURE` +> +> Whether to use a secure cookie for the CSRF cookie. If this is set to True, the cookie will be marked as "secure," which means browsers may ensure that the cookie is only sent with an HTTPS connection. This is a [Django Setting](https://docs.djangoproject.com/en/3.2/ref/settings/#csrf-cookie-secure) + +### CUSTOM_METADATA_SCHEMA + +> | Default: `{}` +> +> If present, will extend the available metadata schema used for store +> new value for each resource. By default override the existing one. +> The expected schema is the same as the default + +## D + +### DATA_UPLOAD_MAX_NUMBER_FIELDS + +> Default: `100000` +> +> Maximum value of parsed attributes. + +### DATASET_DOWNLOAD_HANDLERS + +> Default: `[]` +> +> Additional download handlers that provides a link to download the resource + +### DEBUG + +> | Default: `False` +> | Env: `DEBUG` +> +> One of the main features of debug mode is the display of detailed error pages. If your app raises an exception when DEBUG is True, Django will display a detailed traceback, including a lot of metadata about your environment, such as all the currently defined Django settings (from settings.py). +> This is a [Django Setting](https://docs.djangoproject.com/en/3.2/ref/settings/#debug) + +### DEFAULT_ANONYMOUS_DOWNLOAD_PERMISSION + +> Default: `True` +> +> Whether the uploaded resources should downloadable by default. + +### DEFAULT_ANONYMOUS_VIEW_PERMISSION + +> Default: `True` +> +> Whether the uploaded resources should be public by default. + +### DEFAULT_AUTO_FIELD + +> Default: `django.db.models.AutoField` +> +> Default primary key field type to use for models that don't have a field with primary_key=True. +> Django documentation + +### DEFAULT_DATASET_DOWNLOAD_HANDLER + +> Default: `geonode.layers.download_handler.DatasetDownloadHandler` +> +> from GeoNode 4.2.x has bee introduced with this issue #11296 and later improved with this issue #11421 +> the concept of Download Handler and ofc GeoNode provides a default implementation of it +> which process the download via WPS + +### DEFAULT_EXTRA_METADATA_SCHEMA + +> Default +> +> ``` json +> { +> Optional("id"): int, +> "filter_header": object, +> "field_name": object, +> "field_label": object, +> "field_value": object, +> } +> ``` +> +> Define the default metadata schema used for add to the resource extra metadata without modify the actual model. +> This schema is used as validation for the input metadata provided by the user +> +> - \`id\`: (optional int): the identifier of the metadata. Optional for creation, required in Upgrade phase +> - \`filter_header\`: (required object): Can be any type, is used to generate the facet filter header. Is also an identifier. +> - \`field_name\`: (required object): name of the metadata field +> - \`field_label\`: (required object): verbose string of the name. Is used as a label in the facet filters. +> - \`field_value\`: (required object): metadata values +> +> An example of metadata that can be ingested is the follow: +> +> ``` json +> [ +> { +> "filter_header": "Bike Brand", +> "field_name": "name", +> "field_label": "Bike Name", +> "field_value": "KTM", +> }, +> { +> "filter_header": "Bike Brand", +> "field_name": "name", +> "field_label": "Bike Name", +> "field_value": "Bianchi", +> } +> ] +> ``` + +### DEFAULT_LAYER_FORMAT + +> | Default: `image/png` +> | Env: `DEFAULT_LAYER_FORMAT` +> +> The default format for requested tile images. + +### DEFAULT_MAP_CENTER + +> | Default: `(0, 0)` +> | Env: `DEFAULT_MAP_CENTER_X` `DEFAULT_MAP_CENTER_Y` +> +> A 2-tuple with the latitude/longitude coordinates of the center-point to use +> in newly created maps. + +### DEFAULT_MAP_CRS + +> | Default: `EPSG:3857` +> | Env: `DEFAULT_MAP_CRS` +> +> The default map projection. Default: EPSG:3857 + +### DEFAULT_MAP_ZOOM + +> | Default: `0` +> | Env: `DEFAULT_MAP_ZOOM` +> +> The zoom-level to use in newly created maps. This works like the OpenLayers +> zoom level setting; 0 is at the world extent and each additional level cuts +> the viewport in half in each direction. + +DEFAULT_MAX_PARALLEL_UPLOADS_PER_USER +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-- + +Default: `5` + +When [uploading datasets](../../usage/managing_datasets/uploading_datasets.html#datasets-uploading), +this value limits the number os parallel uploads. + +The parallelism limit is set during installation using the value of this variable. +After installation, only an user with administrative rights can change it. +These limits can be changed in the [admin panel](../../admin/upload-parallelism-limit/index.html#upload-parallelism-limit) +or [accessing by api](../../devel/api/V2/index.html#get--api-v2-upload-parallelism-limits-). + +### DEFAULT_MAX_UPLOAD_SIZE + +Default: `104857600` (100 MB in bytes) + +When [uploading datasets](../../usage/managing_datasets/uploading_datasets.html#datasets-uploading) +or [uploading documents](../../usage/managing_documents/uploading_documents.html#uploading-documents), +the total size of the uploaded files is verified. + +The size limits are set during installation using the value of this variable. +After installation, only an user with administrative rights can change it. +These limits can be changed in the [admin panel](../../admin/upload-size-limits/index.html#upload-size-limits) +or [accessing by api](../../devel/api/V2/index.html#get--api-v2-upload-size-limits-). + +### DEFAULT_SEARCH_SIZE + +> | Default: `10` +> | Env: `DEFAULT_SEARCH_SIZE` +> +> An integer that specifies the default search size when using `geonode.search` for querying data. + +### DEFAULT_WORKSPACE + +> | Default: `geonode` +> | Env: `DEFAULT_WORKSPACE` +> +> The standard GeoServer workspace. + +### DELAYED_SECURITY_SIGNALS + +> | Default: `False` +> | Env: `DELAYED_SECURITY_SIGNALS` +> +> This setting only works when `GEOFENCE_SECURITY_ENABLED` has been set to `True` and GeoNode is making use of the `GeoServer BACKEND`. +> +> By setting this to `True`, every time the permissions will be updated/changed for a Layer, they won\'t be applied immediately but only and only if +> either: +> +> a. A Celery Worker is running and it is able to execute the `geonode.security.tasks.synch_guardian` periodic task; +> notice that the task will be executed at regular intervals, based on the interval value defined in the corresponding PeriodicTask model. +> b. A periodic `cron` job runs the `sync_security_rules` management command, or either it is manually executed from the Django shell. +> c. The user, owner of the Layer or with rights to change its permissions, clicks on the GeoNode UI button `Sync permissions immediately` +> +> ::: warning +> ::: title +> Warning +> ::: +> +> Layers won\'t be accessible to public users anymore until the Security Rules are not synchronized! +> ::: + +### DISPLAY_COMMENTS + +> | Default: `True` +> | Env: `DISPLAY_COMMENTS` +> +> If set to False comments are hidden. + +### DISPLAY_RATINGS + +> | Default: `True` +> | Env: `DISPLAY_RATINGS` +> +> If set to False ratings are hidden. + +### DISPLAY_SOCIAL + +> | Default: `True` +> | Env: `DISPLAY_SOCIAL` +> +> If set to False social sharing is hidden. + +### DISPLAY_WMS_LINKS + +> | Default: `True` +> | Env: `DISPLAY_WMS_LINKS` +> +> If set to False direct WMS link to GeoServer is hidden. + +### DISPLAY_ORIGINAL_DATASET_LINK + +> | Default: `True` +> | Env: `DISPLAY_ORIGINAL_DATASET_LINK` +> +> If set to False original dataset download is hidden. + +### DOWNLOAD_FORMATS_METADATA + +> Specifies which metadata formats are available for users to download. +> +> Default: +> +> DOWNLOAD_FORMATS_METADATA = [ +> 'Atom', 'DIF', 'Dublin Core', 'ebRIM', 'FGDC', 'ISO', +> ] + +### DOWNLOAD_FORMATS_VECTOR + +> Specifies which formats for vector data are available for users to download. +> +> Default: +> +> DOWNLOAD_FORMATS_VECTOR = [ +> 'JPEG', 'PDF', 'PNG', 'Zipped Shapefile', 'GML 2.0', 'GML 3.1.1', 'CSV', +> 'Excel', 'GeoJSON', 'KML', 'View in Google Earth', 'Tiles', +> ] + +### DOWNLOAD_FORMATS_RASTER + +> Specifies which formats for raster data are available for users to download. +> +> Default: +> +> DOWNLOAD_FORMATS_RASTER = [ +> 'JPEG', 'PDF', 'PNG' 'Tiles', +> ] + +## E + +### EMAIL_ENABLE + +> | Default: `False` +> +> Options: +> +> > - EMAIL_BACKEND +> > +> > > Default: `django.core.mail.backends.smtp.EmailBackend` +> > > +> > > Env: `DJANGO_EMAIL_BACKEND` +> > +> > - EMAIL_HOST +> > +> > > Default: `localhost` +> > +> > - EMAIL_PORT +> > +> > > Default: `25` +> > +> > - EMAIL_HOST_USER +> > +> > > Default: `''` +> > +> > - EMAIL_HOST_PASSWORD +> > +> > > Default: `''` +> > +> > - EMAIL_USE_TLS +> > +> > > Default: `False` +> > +> > - EMAIL_USE_SSL +> > +> > > Default: `False` +> > +> > - DEFAULT_FROM_EMAIL +> > +> > > Default: `GeoNode ` + +### EPSG_CODE_MATCHES + +> | Default: +> +> ``` python +> { +> 'EPSG:4326': '(4326) WGS 84', +> 'EPSG:900913': '(900913) Google Maps Global Mercator', +> 'EPSG:3857': '(3857) WGS 84 / Pseudo-Mercator', +> 'EPSG:3785': '(3785 DEPRECATED) Popular Visualization CRS / Mercator', +> 'EPSG:32647': '(32647) WGS 84 / UTM zone 47N', +> 'EPSG:32736': '(32736) WGS 84 / UTM zone 36S' +> } +> ``` +> +> Supported projections human readable descriptions associated to their EPSG Codes. +> This list will be presented to the user during the upload process whenever GeoNode won\'t be able to recognize a suitable projection. +> Those codes should be aligned to the [UPLOADER]{.title-ref} ones and available in GeoServer also. + +### EXTRA_METADATA_SCHEMA + +> | Default: +> +> ``` python +> EXTRA_METADATA_SCHEMA = {**{ +> "map": os.getenv('MAP_EXTRA_METADATA_SCHEMA', DEFAULT_EXTRA_METADATA_SCHEMA), +> "layer": os.getenv('DATASET_EXTRA_METADATA_SCHEMA', DEFAULT_EXTRA_METADATA_SCHEMA), +> "document": os.getenv('DOCUMENT_EXTRA_METADATA_SCHEMA', DEFAULT_EXTRA_METADATA_SCHEMA), +> "geoapp": os.getenv('GEOAPP_EXTRA_METADATA_SCHEMA', DEFAULT_EXTRA_METADATA_SCHEMA) +> }, **CUSTOM_METADATA_SCHEMA} +> ``` +> +> Variable used to actually get the expected metadata schema for each resource_type. +> In this way, each resource type can have a different metadata schema + +## F + +### FREETEXT_KEYWORDS_READONLY + +> | Default: `False` +> | Env: `FREETEXT_KEYWORDS_READONLY` +> +> Make Free-Text Keywords writable from users. Or read-only when set to False. + +### FACET_PROVIDERS + +> | Default: `pre filled list of providers` +> | Env: `FACET_PROVIDERS` +> +> Contains the list of the providers available to perform an serve the facets. +> In case the user wants remove a facets, is enough to remove the path +> of the proider from the list + +## G + +### GEOFENCE_SECURITY_ENABLED + +> | Default: `True` (False is Test is true) +> | Env: `GEOFENCE_SECURITY_ENABLED` +> +> Whether the geofence security system is used. + +### GEOIP_PATH + +> | Default: `Path to project` +> | Env: `PROJECT_ROOT` +> +> The local path where GeoIPCities.dat is written to. Make sure your user has to have write permissions. + +### GEONODE_APPS_ENABLED + +> | Default: `True` +> +> If enabled contrib apps are used. +> If disabled: +> - the geoapps URLs are not included in the routing paths +> - the geoapps resources are excluded from the search +> - the resource detail are forwarded to the homepage +> +> `ENABLE -> DISABLE` transition: +> +> This should be done if the geoapps were enabled in an environment where they are not needed. +> +> `DISABLE -> ENABLE` transition: +> +> It should be done only once to enable geoapps in an environment where are needed + +### GEONODE_CLIENT_LAYER_PREVIEW_LIBRARY + +> Default: `"mapstore"` +> +> The library to use for display preview images of layers. The library choices are: +> +> > `"mapstore"` +> > `"leaflet"` +> > `"react"` + +### GEONODE_EXCHANGE + +> | Default:: `Exchange("default", type="direct", durable=True)` +> +> The definition of Exchanges published by geonode. Find more about Exchanges at [celery docs](https://docs.celeryproject.org/en/latest/userguide/routing.html#exchanges-queues-and-routing-keys). + +### GEOSERVER_ADMIN_USER + +> | Default: `admin` +> | Env: `GEOSERVER_ADMIN_PASSWORD` +> +> The geoserver admin username. + +GEOSERVER_ADMIN_PASSWORD +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-- + +> | Default: `geoserver` +> | Env: `GEOSERVER_ADMIN_USER` +> +> The GeoServer admin password. + +GEOSERVER_FACTORY_PASSWORD +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-- + +> | Default: `geoserver` +> | Env: `GEOSERVER_FACTORY_PASSWORD` +> +> The GeoServer admin factory password, required for the first time configuration fo Geoserver (Docker setup). + +### GEOSERVER_EXCHANGE + +> | Default:: `Exchange("geonode", type="topic", durable=False)` +> +> The definition of Exchanges published by GeoServer. Find more about Exchanges at [celery docs](https://docs.celeryproject.org/en/latest/userguide/routing.html#exchanges-queues-and-routing-keys). + +### GEOSERVER_LOCATION + +> | Default: `http://localhost:8080/geoserver/` +> | Env: `GEOSERVER_LOCATION` +> +> Url under which GeoServer is available. + +### GEOSERVER_PUBLIC_HOST + +> | Default: `SITE_HOST_NAME` (Variable) +> | Env: `GEOSERVER_PUBLIC_HOST` +> +> Public hostname under which GeoServer is available. + +### GEOSERVER_PUBLIC_LOCATION + +> | Default: `SITE_HOST_NAME` (Variable) +> | Env: `GEOSERVER_PUBLIC_LOCATION` +> +> Public location under which GeoServer is available. + +### GEOSERVER_PUBLIC_PORT + +> | Default: `8080 (Variable)` +> | Env: `GEOSERVER_PUBLIC_PORT` +> +> Public Port under which GeoServer is available. + +### GEOSERVER_WEB_UI_LOCATION + +> | Default: `GEOSERVER_PUBLIC_LOCATION (Variable)` +> | Env: `GEOSERVER_WEB_UI_LOCATION` +> +> Public location under which GeoServer is available. + +### GROUP_PRIVATE_RESOURCES + +> | Default: `False` +> | Env: `GROUP_PRIVATE_RESOURCES` +> +> If this option is enabled, Resources belonging to a Group won\'t be visible by others + +## I + +### IMPORTER HANDLERS + +> | Default: `pre filled list of handlers` +> | Env: `IMPORTER_HANDLERS` +> +> Contains the list of the handlers available to perform an import of a resource. +> In case the user wants to drop the support during the import phase, is enough to +> remove the path of the Handler from the list + +## L + +### LEAFLET_CONFIG + +> A dictionary used for Leaflet configuration. + +### LICENSES + +> | Default: +> +> ``` python +> { +> 'ENABLED': True, +> 'DETAIL': 'above', +> 'METADATA': 'verbose', +> } +> ``` +> +> Enable Licenses User Interface + +### LOCAL_SIGNALS_BROKER_URL + +> | Default: `memory://` + +### LOCKDOWN_GEONODE + +> | Default: `False` +> | Env: `LOCKDOWN_GEONODE` +> +> By default, the GeoNode application allows visitors to view most pages without being authenticated. If this is set to `True` +> users must be authenticated before accessing URL routes not included in `AUTH_EXEMPT_URLS`. + +### LOGIN_URL + +> | Default: `{}account/login/'.format(SITEURL)` +> | Env: `LOGIN_URL` +> +> The URL where requests are redirected for login. + +### LOGOUT_URL + +> | Default: `{}account/login/'.format(SITEURL)` +> | Env: `LOGOUT_URL` +> +> The URL where requests are redirected for logout. + +## M + +### MAP_CLIENT_USE_CROSS_ORIGIN_CREDENTIALS + +> | Default: `False` +> | Env: `MAP_CLIENT_USE_CROSS_ORIGIN_CREDENTIALS` +> +> Enables cross origin requests for geonode-client. + +### MAPSTORE_BASELAYERS + +> | Default: +> +> ``` python +> [ +> { +> "type": "osm", +> "title": "Open Street Map", +> "name": "mapnik", +> "source": "osm", +> "group": "background", +> "visibility": True +> }, { +> "type": "tileprovider", +> "title": "OpenTopoMap", +> "provider": "OpenTopoMap", +> "name": "OpenTopoMap", +> "source": "OpenTopoMap", +> "group": "background", +> "visibility": False +> }, { +> "type": "wms", +> "title": "Sentinel-2 cloudless - https://s2maps.eu", +> "format": "image/jpeg", +> "id": "s2cloudless", +> "name": "s2cloudless:s2cloudless", +> "url": "https://maps.geo-solutions.it/geoserver/wms", +> "group": "background", +> "thumbURL": "%sstatic/mapstorestyle/img/s2cloudless-s2cloudless.png" % SITEURL, +> "visibility": False +> }, { +> "source": "ol", +> "group": "background", +> "id": "none", +> "name": "empty", +> "title": "Empty Background", +> "type": "empty", +> "visibility": False, +> "args": ["Empty Background", {"visibility": False}] +> } +> ] +> ``` +> +> | Env: `MAPSTORE_BASELAYERS` +> +> Allows to specify which backgrounds MapStore should use. The parameter `visibility` for a layer, specifies which one is the default one. +> +> A sample configuration using the Bing background without OpenStreetMap, could be the following one: +> +> ``` python +> [ +> { +> "type": "bing", +> "title": "Bing Aerial", +> "name": "AerialWithLabels", +> "source": "bing", +> "group": "background", +> "apiKey": "{{apiKey}}", +> "visibility": True +> }, { +> "type": "tileprovider", +> "title": "OpenTopoMap", +> "provider": "OpenTopoMap", +> "name": "OpenTopoMap", +> "source": "OpenTopoMap", +> "group": "background", +> "visibility": False +> }, { +> "type": "wms", +> "title": "Sentinel-2 cloudless - https://s2maps.eu", +> "format": "image/jpeg", +> "id": "s2cloudless", +> "name": "s2cloudless:s2cloudless", +> "url": "https://maps.geo-solutions.it/geoserver/wms", +> "group": "background", +> "thumbURL": "%sstatic/mapstorestyle/img/s2cloudless-s2cloudless.png" % SITEURL, +> "visibility": False +> }, { +> "source": "ol", +> "group": "background", +> "id": "none", +> "name": "empty", +> "title": "Empty Background", +> "type": "empty", +> "visibility": False, +> "args": ["Empty Background", {"visibility": False}] +> } +> ] +> ``` +> +> ::: warning +> ::: title +> Warning +> ::: +> +> To use a Bing background, you need to correctly set and provide a valid `BING_API_KEY` +> ::: + +### MAX_DOCUMENT_SIZE + +> | Default:`2` +> | Env: `MAX_DOCUMENT_SIZE` +> +> Allowed size for documents in MB. + +### METADATA_PARSERS + +Is possible to define multiple XML parsers for ingest XML during the layer upload. + +The variable should be declared in this way in \`settings.py\`: + +[METADATA_PARSERS = \[\'list\', \'of\', \'parsing\', \'functions\'\]]{.title-ref} + +If you want to always use the default metadata parser and after use your own, the variable must be set with first value as [\_\_DEFAULT\_\_]{.title-ref} +Example: + +[METADATA_PARSERS = \[\'\_\_DEFAULT\_\_\', \'custom_parsing_function\]]{.title-ref} + +If not set, the system will use the [\_\_DEFAULT\_\_]{.title-ref} parser. + +The custom parsing function must be accept in input 6 parameter that are: + +> | - exml (xmlfile) +> | - uuid (str) +> | - vals (dict) +> | - regions (list) +> | - keywords (list) +> | - custom (dict) + +If you want to use your parser after the default one, here is how the variable are populated: + +> | - exml: the XML file to parse +> | - uuid: the UUID of the layer +> | - vals: Dictionary of information that belong to ResourceBase +> | - regions: List of regions extracted from the XML +> | - keywords: List of dict of keywords already divided between free-text and thesarus +> | - custom: Custom varible + +NOTE: the keywords must be in a specific format, since later this dict, will be ingested by the [KeywordHandler]{.title-ref} which will assign the keywords/thesaurus to the layer. + +> ``` { +> "keywords": [list_of_keyword_extracted], +> "thesaurus": {"date": None, "datetype": None, "title": None}, # thesaurus informations +> "type": theme, #extracted theme if present +> } +> ``` + +Here is an example of expected parser function + +> ``` def custom_parsing_function(exml, uuid, vals, regions, keywords, custom): +> # Place here your code +> return uuid, vals, regions, keywords, custom +> ``` + +For more information, please rely to [TestCustomMetadataParser]{.title-ref} which contain a smoke test to explain the functionality + +### METADATA_STORERS + +Is possible to define multiple Layer storer during the layer upload. + +The variable should be declared in this way: + +[METADATA_STORERS = \[\'custom_storer_function\'\]]{.title-ref} + +NOTE: By default the Layer is always saved with the default behaviour. + +The custom storer function must be accept in input 2 parameter that are: + +> | - Layer (layer model instance) +> | - custom (dict) + +Here is how the variable are populated by default: + +> | - layer (layer model instance) that we wanto to change +> | - custom: custom dict populated by the parser + +Here is an example of expected storer function + +> ``` def custom_storer_function(layer, custom): +> # do something here +> pass +> ``` + +For more information, please rely to [TestMetadataStorers]{.title-ref} which contain a smoke test to explain the functionality + +### MISSING_THUMBNAIL + +> Default: `geonode/img/missing_thumb.png` +> +> The path to an image used as thumbnail placeholder. + +### MEMCACHED_BACKEND + +> Default: `django.core.cache.backends.memcached.PyMemcacheCache` +> +> Define which backend of memcached will be used + +### MEMCACHED_ENABLED + +> Default: `False` +> +> If True, will use MEMCACHED_BACKEND as default backend in CACHES + +### MODIFY_TOPICCATEGORY + +> Default: `False` +> +> Metadata Topic Categories list should not be modified, as it is strictly defined +> by ISO (See: +> and check the \ element). +> +> Some customization is still possible changing the is_choice and the GeoNode +> description fields. +> +> In case it is necessary to add/delete/update categories, it is +> possible to set the MODIFY_TOPICCATEGORY setting to True. + +### MONITORING_ENABLED + +> Default: `False` +> +> Enable internal monitoring application ([geonode.monitoring]{.title-ref}). If set to [True]{.title-ref}, add following code to your local settings: +> +> ``` +> MONITORING_ENABLED = True +> # add following lines to your local settings to enable monitoring +> if MONITORING_ENABLED: +> INSTALLED_APPS + ('geonode.monitoring',) +> MIDDLEWARE_CLASSES + ('geonode.monitoring.middleware.MonitoringMiddleware',) +> ``` +> +> See `geonode_monitoring`{.interpreted-text role="ref"} for details. + +### MONITORING_DATA_AGGREGATION {#monitoring-data-aggregation} + +> Default: +> +> ``` +> ( +> (timedelta(seconds=0), timedelta(minutes=1),), +> (timedelta(days=1), timedelta(minutes=60),), +> (timedelta(days=14), timedelta(days=1),), +> ) +> ``` +> +> Configure aggregation of past data to control data resolution. It lists data age and aggregation in reverse order, by default: +> +> | - for current data, 1 minute resolution +> | - for data older than 1 day, 1-hour resolution +> | - for data older than 2 weeks, 1 day resolution +> +> See `geonode_monitoring`{.interpreted-text role="ref"} for further details. +> +> This setting takes effects only if `user-analytics`{.interpreted-text role="ref"} is true. + +### MONITORING_DATA_TTL + +> | Default: `365` +> | Env: `MONITORING_DATA_TTL` +> +> How long monitoring data should be stored in days. + +### MONITORING_DISABLE_CSRF + +> | Default: `False` +> | Env: `MONITORING_DISABLE_CSRF` +> +> Set this to true to disable csrf check for notification config views, use with caution - for dev purpose only. + +### MONITORING_SKIP_PATHS {#monitoring-skip-paths} + +> Default: +> +> ``` +> ( +> '/api/o/', +> '/monitoring/', +> '/admin', +> '/jsi18n', +> STATIC_URL, +> MEDIA_URL, +> re.compile('^/[a-z]{2}/admin/'), +> ) +> ``` +> +> Skip certain useless paths to not to mud analytics stats too much. +> See `geonode_monitoring`{.interpreted-text role="ref"} to learn more about it. +> +> This setting takes effects only if `user-analytics`{.interpreted-text role="ref"} is true. + +## N + +### NOTIFICATIONS_MODULE + +> Default: `pinax.notifications` +> +> App used for notifications. (pinax.notifications or notification) + +### NOTIFICATION_ENABLED + +> | Default: `True` +> | Env: `NOTIFICATION_ENABLED` +> +> Enable or disable the notification system. + +## O + +### OAUTH2_API_KEY + +> | Default: `None` +> | Env: `OAUTH2_API_KEY` +> +> In order to protect oauth2 REST endpoints, used by GeoServer to fetch user roles and infos, you should set this key and configure the `geonode REST role service` accordingly. Keep it secret! +> +> ::: warning +> ::: title +> Warning +> ::: +> +> If not set, the endpoint can be accessed by users without authorization. +> ::: + +### OAUTH2_PROVIDER + +> Ref.: [OAuth Toolkit settings](https://django-oauth-toolkit.readthedocs.io/en/latest/settings.html) + +### OAUTH2_PROVIDER_APPLICATION_MODEL + +> | Default: `oauth2_provider.Application` +> +> Ref.: [OAuth Toolkit settings](https://django-oauth-toolkit.readthedocs.io/en/latest/settings.html) + +### OAUTH2_PROVIDER_ACCESS_TOKEN_MODEL + +> | Default: `oauth2_provider.AccessToken` +> +> Ref.: [OAuth Toolkit settings](https://django-oauth-toolkit.readthedocs.io/en/latest/settings.html) + +### OAUTH2_PROVIDER_ID_TOKEN_MODEL + +> | Default: `oauth2_provider.IDToken` +> +> Ref.: [OAuth Toolkit settings](https://django-oauth-toolkit.readthedocs.io/en/latest/settings.html) + +### OAUTH2_PROVIDER_GRANT_MODEL + +> | Default: `oauth2_provider.Grant` +> +> Ref.: [OAuth Toolkit settings](https://django-oauth-toolkit.readthedocs.io/en/latest/settings.html) + +### OAUTH2_PROVIDER_REFRESH_TOKEN_MODEL + +> | Default: `oauth2_provider.RefreshToken` +> +> Ref.: [OAuth Toolkit settings](https://django-oauth-toolkit.readthedocs.io/en/latest/settings.html) + +### OGC_SERVER + +> Default: `{}` (Empty dictionary) +> +> A dictionary of OGC servers and their options. The main +> server should be listed in the \'default\' key. If there is no \'default\' +> key or if the `OGC_SERVER` setting does not exist, Geonode will raise +> an Improperly Configured exception. Below is an example of the `OGC_SERVER` +> setting: +> +> OGC_SERVER = { +> 'default' : { +> 'LOCATION' : 'http://localhost:8080/geoserver/', +> 'USER' : 'admin', +> 'PASSWORD' : 'geoserver', +> } +> } +> +> - BACKEND +> +> > Default: `"geonode.geoserver"` +> > +> > The OGC server backend to use. The backend choices are: +> > +> > > `'geonode.geoserver'` +> +> - BACKEND_WRITE_ENABLED +> +> > Default: `True` +> > +> > Specifies whether the OGC server can be written to. If False, actions that modify +> > data on the OGC server will not execute. +> +> - DATASTORE +> +> > Default: `''` (Empty string) +> > +> > An optional string that represents the name of a vector datastore, where Geonode uploads are imported into. To support vector datastore imports there also needs to be an +> > entry for the datastore in the `DATABASES` dictionary with the same name. Example: +> > +> > OGC_SERVER = { +> > 'default' : { +> > 'LOCATION' : 'http://localhost:8080/geoserver/', +> > 'USER' : 'admin', +> > 'PASSWORD' : 'geoserver', +> > 'DATASTORE': 'geonode_imports' +> > } +> > } +> > +> > DATABASES = { +> > 'default': { +> > 'ENGINE': 'django.db.backends.sqlite3', +> > 'NAME': 'development.db', +> > }, +> > 'geonode_imports' : { +> > 'ENGINE': 'django.contrib.gis.db.backends.postgis', +> > 'NAME': 'geonode_imports', +> > 'USER' : 'geonode_user', +> > 'PASSWORD' : 'a_password', +> > 'HOST' : 'localhost', +> > 'PORT' : '5432', +> > } +> > } +> +> - GEONODE_SECURITY_ENABLED +> +> > Default: `True` +> > +> > A boolean that represents whether GeoNode\'s security application is enabled. +> +> - LOCATION +> +> > Default: `"http://localhost:8080/geoserver/"` +> > +> > A base URL from which GeoNode can construct OGC service URLs. +> > If using GeoServer you can determine this by +> > visiting the GeoServer administration home page without the +> > /web/ at the end. For example, if your GeoServer administration app is at +> > , your server\'s location is . +> +> - MAPFISH_PRINT_ENABLED +> +> > Default: `True` +> > +> > A boolean that represents whether the MapFish printing extension is enabled on the server. +> +> - PASSWORD +> +> > Default: `'geoserver'` +> > +> > The administrative password for the OGC server as a string. +> +> - PRINT_NG_ENABLED +> +> > Default: `True` +> > +> > A boolean that represents whether printing of maps and layers is enabled. +> +> - PUBLIC_LOCATION +> +> > Default: `"http://localhost:8080/geoserver/"` +> > +> > The URL used to in most public requests from Geonode. This setting allows a user to write to one OGC server (the LOCATION setting) +> > and read from a separate server or the PUBLIC_LOCATION. +> +> - USER +> +> > Default: `'admin'` +> > +> > The administrative username for the OGC server as a string. +> +> - WMST_ENABLED +> +> > Default: `False` +> > +> > Not implemented. +> +> - WPS_ENABLED +> +> > Default: `False` +> > +> > Not implemented. +> +> - TIMEOUT +> +> > Default: `10` +> > +> > The maximum time, in seconds, to wait for the server to respond. + +### OGP_URL + +> | Default: `http://geodata.tufts.edu/solr/select` +> | Env: `OGP_URL` +> +> Endpoint of geodata.tufts.edu getCapabilities. + +### OPENGRAPH_ENABLED + +> Default:: `True` +> +> A boolean that specifies whether Open Graph is enabled. Open Graph is used by Facebook and Slack. + +## P + +### PINAX_NOTIFICATIONS_BACKENDS + +> Default: `("email", _EMAIL_BACKEND, 0),` +> +> Used notification backend. This is a [pinax notification setting:](https://django-notification.readthedocs.io/en/latest/settings.html#pinax-notifications-backends) + +### PINAX_NOTIFICATIONS_LOCK_WAIT_TIMEOUT + +> | Default: `-1` +> | Env: `NOTIFICATIONS_LOCK_WAIT_TIMEOUT` +> +> It defines how long to wait for the lock to become available. Default of -1 means to never wait for the lock to become available. +> This is a [pinax notification setting:](https://django-notification.readthedocs.io/en/latest/settings.html#pinax-notifications-lock-wait-timeout) + +### PINAX_NOTIFICATIONS_QUEUE_ALL + +> | Default: `-1` +> | Env: `NOTIFICATIONS_LOCK_WAIT_TIMEOUT` +> +> By default, calling notification.send will send the notification immediately, however, if you set this setting to True, then the default behavior of the send method will be to queue messages in the database for sending via the emit_notices command. +> This is a [pinax notification setting:](https://django-notification.readthedocs.io/en/latest/settings.html#pinax-notifications-queue-all) + +### PINAX_RATINGS_CATEGORY_CHOICES + +> Default: +> +> { +> "maps.Map": { +> "map": "How good is this map?" +> }, +> "layers.Layer": { +> "layer": "How good is this layer?" +> }, +> "documents.Document": { +> "document": "How good is this document?" +> } +> } + +### PROFILE_EDIT_EXCLUDE_FIELD + +> | Default: `[]` +> +> A list of element (item name) to exclude from the Profile Edit page. +> +> Example: +> +> `PROFILE_EDIT_EXCLUDE_FIELD=['organization', 'language']` + +### PROXY_ALLOWED_HOSTS + +> Default: `()` (Empty tuple) +> +> A tuple of strings representing the host/domain names that GeoNode can proxy requests to. This is a security measure +> to prevent an attacker from using the GeoNode proxy to render malicious code or access internal sites. +> +> Values in this tuple can be fully qualified names (e.g. \'www.geonode.org\'), in which case they will be matched against +> the request's Host header exactly (case-insensitive, not including port). A value beginning with a period can be used +> as a subdomain wildcard: `.geonode.org` will match geonode.org, www.geonode.org, and any other subdomain of +> geonode.org. A value of \'\*\' will match anything and is not recommended for production deployments. + +### PROXY_URL + +> Default `/proxy/?url=` +> +> The URL to a proxy that will be used when making client-side requests in GeoNode. By default, the +> internal GeoNode proxy is used but administrators may favor using their own, less restrictive proxies. + +### PYCSW + +> A dict with pycsw\'s configuration with two possible keys CONFIGURATION and FILTER. +> +> CONFIGURATION +> Of note are the sections `metadata:main` to set CSW server metadata and `metadata:inspire` +> to set INSPIRE options. Setting `metadata:inspire['enabled']` to `true` +> will enable INSPIRE support. Server level configurations can be overridden +> in the `server` section. See +> for full pycsw configuration details. +> +> FILTER +> Optional settings in order to add a filter to the CSW filtering. +> The filter follow the django orm structure and must be a [ResourceBase]{.title-ref} field/related field. +> By default CSW will filter only for [layer]{.title-ref} resource_type +> +> Example of PYCSW configuration. +> PYCSW: { +> \'CONFIGURATION\': {\...}, +> \'FILTER\': {\'resource_type\_\_in\':\[\'layer\'\] } +> } + +## R + +### RABBITMQ_SIGNALS_BROKER_URL + +> Default: `amqp://localhost:5672` +> +> The Rabbitmq endpoint + +### RECAPTCHA_ENABLED + +> | Default: `False` +> | Env: `RECAPTCHA_ENABLED` +> +> Allows enabling reCaptcha field on signup form. +> Valid Captcha Public and Private keys will be needed as specified here +> +> You will need to generate a keys pair for `reCaptcha v2` for your domain from +> +> More options will be available by enabling this setting: +> +> - **ACCOUNT_SIGNUP_FORM_CLASS** +> +> > | Default: `geonode.people.forms.AllauthReCaptchaSignupForm` +> > | Env: `ACCOUNT_SIGNUP_FORM_CLASS` +> > +> > Enabled only when the `recaptcha_enabled`{.interpreted-text role="ref"} option is `True`. +> +> - **INSTALLED_APPS** +> +> > The `captcha` must be present on `INSTALLED_APPS`, otherwise you\'ll get an error. +> > +> > When enabling the `recaptcha_enabled`{.interpreted-text role="ref"} option through the `environment`, this setting will be automatically added by GeoNode as follows: +> > +> > ``` python +> > if 'captcha' not in INSTALLED_APPS: +> > INSTALLED_APPS += ('captcha',) +> > ``` +> +> - **RECAPTCHA_PUBLIC_KEY** +> +> > | Default: `geonode_RECAPTCHA_PUBLIC_KEY` +> > | Env: `RECAPTCHA_PUBLIC_KEY` +> > +> > You will need to generate a keys pair for `reCaptcha v2` for your domain from +> > +> > For mode details on the reCaptcha package, please see: +> > +> > 1. +> > 2. +> +> - **RECAPTCHA_PRIVATE_KEY** +> +> > | Default: `geonode_RECAPTCHA_PRIVATE_KEY` +> > | Env: `RECAPTCHA_PRIVATE_KEY` +> > +> > You will need to generate a keys pair for `reCaptcha v2` for your domain from +> > +> > For mode details on the reCaptcha package, please see: +> > +> > 1. +> > 2. + +### RECAPTCHA_PUBLIC_KEY + +> | Default: `geonode_RECAPTCHA_PUBLIC_KEY` +> | Env: `RECAPTCHA_PUBLIC_KEY` +> +> You will need to generate a keys pair for `reCaptcha v2` for your domain from +> +> Ref. to `recaptcha_enabled`{.interpreted-text role="ref"} + +### RECAPTCHA_PRIVATE_KEY + +> | Default: `geonode_RECAPTCHA_PRIVATE_KEY` +> | Env: `RECAPTCHA_PRIVATE_KEY` +> +> You will need to generate a keys pair for `reCaptcha v2` for your domain from +> +> Ref. to `recaptcha_enabled`{.interpreted-text role="ref"} + +### REDIS_SIGNALS_BROKER_URL + +> Default: `redis://localhost:6379/0` +> +> The Redis endpoint. + +### REGISTERED_MEMBERS_GROUP_NAME + +> | Default: `registered-members` +> | Env: `REGISTERED_MEMBERS_GROUP_NAME` +> +> Used by `AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_NAME` settings. + +### REGISTERED_MEMBERS_GROUP_TITLE + +> | Default: `Registered Members` +> | Env: `REGISTERED_MEMBERS_GROUP_TITLE` +> +> Used by `AUTO_ASSIGN_REGISTERED_MEMBERS_TO_REGISTERED_MEMBERS_GROUP_NAME` settings. + +### REGISTRATION_OPEN + +> Default: `False` +> +> A boolean that specifies whether users can self-register for an account on your site. + +### RESOURCE_PUBLISHING + +> Default: `False` +> +> By default, the GeoNode application allows GeoNode staff members to +> publish/unpublish resources. +> By default, resources are published when created. When this setting is set to +> True the staff members will be able to unpublish a resource (and eventually +> publish it back). + +## S + +### SEARCH_FILTERS + +> Default: +> +> 'TEXT_ENABLED': True, +> 'TYPE_ENABLED': True, +> 'CATEGORIES_ENABLED': True, +> 'OWNERS_ENABLED': True, +> 'KEYWORDS_ENABLED': True, +> 'H_KEYWORDS_ENABLED': True, +> 'T_KEYWORDS_ENABLED': True, +> 'DATE_ENABLED': True, +> 'REGION_ENABLED': True, +> 'EXTENT_ENABLED': True, +> +> Enabled Search Filters for filtering resources. + +### SECURE_BROWSER_XSS_FILTER + +> | Default: `True` +> | Env: `SECURE_BROWSER_XSS_FILTER` +> +> If True, the SecurityMiddleware sets the X-XSS-Protection: 1; mode=block header on all responses that do not already have it. +> This is [Django settings. https://docs.djangoproject.com/en/3.2/ref/settings/#secure-browser-xss-filter](Django%20settings.%20https://docs.djangoproject.com/en/3.2/ref/settings/#secure-browser-xss-filter) + +### SECURE_CONTENT_TYPE_NOSNIFF + +> | Default: `True` +> | Env: `SECURE_CONTENT_TYPE_NOSNIFF` +> +> If True, the SecurityMiddleware sets the X-Content-Type-Options: nosniff header on all responses that do not already have it. +> This is [Django settings:](https://docs.djangoproject.com/en/3.2/ref/settings/#secure-content-type-nosniff) + +### SECURE_HSTS_INCLUDE_SUBDOMAINS + +> | Default: `True` +> | Env: `SECURE_HSTS_INCLUDE_SUBDOMAINS` +> +> This is Django settings: + +### SECURE_HSTS_SECONDS + +> | Default: `3600` +> | Env: `SECURE_HSTS_SECONDS` +> +> This is [Django settings:](https://docs.djangoproject.com/en/3.2/ref/settings/#secure-hsts-seconds) +> If set to a non-zero integer value, the SecurityMiddleware sets the HTTP Strict Transport Security header on all responses that do not already have it. + +### SECURE_SSL_REDIRECT + +> If True, the SecurityMiddleware redirects all non-HTTPS requests to HTTPS (except for those URLs matching a regular expression listed in SECURE_REDIRECT_EXEMPT). +> This is [Django settings:](https://docs.djangoproject.com/en/3.2/ref/settings/#secure-ssl-redirect) + +### SERVICES_TYPE_MODULES + +It\'s possible to define multiple Service Types Modules for custom service type with it\'s own Handler. + +The variable should be declared in this way in \`settings.py\`: + +[SERVICES_TYPE_MODULES = \[ \'path.to.module1\',\'path.to.module2\', \... \]]{.title-ref} + +Default service types are already included + +Inside each module in the list we need to define a variable: + +[services_type = { +\"\\": { +\"OWS\": True/False, +\"handler\": \"\\", +\"label\": \"\
' +> }, +> { +> "title": 'Starting my story', +> "description": 'A cure for writers block', +> "content": 'Once upon a time...' +> }, +> { +> "title": 'New list with dates', +> "description": 'New List with dates', +> "content": '
cdate
mdate

My List

' +> } +> ], +> "template_cdate_format": '[Date Created (CDATE): %m/%d/%Y : %H:%M:%S]', +> "template_mdate_format": '[Date Modified (MDATE): %m/%d/%Y : %H:%M:%S]', +> } +> +> HTML WYSIWYG Editor (TINYMCE) Menu Bar Settings. For more info see: +> +> > - +> > - `getfetureinfo-templates`{.interpreted-text role="ref"} + +## U + +### UI_REQUIRED_FIELDS + +If this option is enabled, the input selected (we are referring to the one present in the optional Metadata-Tab on the Metadata-Wizard) will become mandatory. + +The fields that can be mandatory are: + +> | id_resource-edition =\> Label: Edition +> | id_resource-purpose =\> Label: Purpose +> | id_resource-supplemental_information =\> Label: Supplemental information +> | id_resource-temporal_extent_start_pickers =\> Label: temporal extent start +> | id_resource-temporal_extent_end =\> Label: temporal extent end +> | id_resource-maintenance_frequency =\> Label: Maintenance frequency +> | id_resource-spatial_representation_type =\> Label: Spatial representation type + +If at least one on the above ids is set in this configuration, the panel header will change from [Optional]{.title-ref} to [Mandatory]{.title-ref} + +> | Confiugration Example: +> | UI_REQUIRED_FIELDS = \[\'id_resource-edition\'\] + +### UNOCONV_ENABLE + +> | Default: `False` +> | Env: `UNOCONV_ENABLE` + +### UPLOADER + +> Default: +> +> { +> 'BACKEND' : 'geonode.importer', +> 'OPTIONS' : { +> 'TIME_ENABLED': False, +> } +> } +> +> A dictionary of Uploader settings and their values. +> +> - BACKEND +> +> > Default: `'geonode.importer'` +> > +> > The importer backend requires the GeoServer importer extension to be enabled. +> +> - OPTIONS +> +> > Default: +> > +> > 'OPTIONS' : { +> > 'TIME_ENABLED': False, +> > } +> > +> > - TIME_ENABLED +> > +> > > Default: `False` +> > > +> > > A boolean that specifies whether the upload should allow the user to enable time support when uploading data. + +### USER_MESSAGES_ALLOW_MULTIPLE_RECIPIENTS + +> | Default: `True` +> | Env: `USER_MESSAGES_ALLOW_MULTIPLE_RECIPIENTS` +> +> Set to true to have multiple recipients in /message/create/ + +### USER_DELETION_RULES {#user-deletion-rules} + +> | Default: `["geonode.people.utils.user_has_resources"]` +> | Env: `USER_DELETION_RULES` +> +> List of callables that will be called the deletion of a user account is requested. +> The deletion will fail if any of the callables return `False`. +> New rules can be added, as a string path to the callable, as long as they take as parameter +> the user object and return a boolean. + +### UUID HANDLER {#user-analytics} + +Is possible to define an own uuidhandler for the Layer. + +To start using your own handler, is needed to add the following configuration: + +[LAYER_UUID_HANDLER = \"mymodule.myfile.MyObject\"]{.title-ref} + +The Object must accept as [init]{.title-ref} the [instance]{.title-ref} of the layer and have a method named [create_uuid()]{.title-ref} + +here is an example: + +> | class MyObject(): +> |    def \_\_init\_\_(self, instance): +> |        self.instance = instance +> | +> |    def create_uuid(self): +> |        # here your code +> |        pass + +## X + +### X_FRAME_OPTIONS + +Default: `'ALLOW-FROM %s' % SITEURL` + +This is a [Django setting](https://docs.djangoproject.com/en/3.2/ref/clickjacking/#setting-x-frame-options-for-all-responses) diff --git a/basic/theme/img/admin_menu.png b/docs/basic/theme/img/admin_menu.png similarity index 100% rename from basic/theme/img/admin_menu.png rename to docs/basic/theme/img/admin_menu.png diff --git a/basic/theme/img/bootswatch_theme.png b/docs/basic/theme/img/bootswatch_theme.png similarity index 100% rename from basic/theme/img/bootswatch_theme.png rename to docs/basic/theme/img/bootswatch_theme.png diff --git a/basic/theme/img/css_override.png b/docs/basic/theme/img/css_override.png similarity index 100% rename from basic/theme/img/css_override.png rename to docs/basic/theme/img/css_override.png diff --git a/basic/theme/img/custom_home.png b/docs/basic/theme/img/custom_home.png similarity index 100% rename from basic/theme/img/custom_home.png rename to docs/basic/theme/img/custom_home.png diff --git a/basic/theme/img/customized_geonode_project_home.png b/docs/basic/theme/img/customized_geonode_project_home.png similarity index 100% rename from basic/theme/img/customized_geonode_project_home.png rename to docs/basic/theme/img/customized_geonode_project_home.png diff --git a/basic/theme/img/inspect_element.png b/docs/basic/theme/img/inspect_element.png similarity index 100% rename from basic/theme/img/inspect_element.png rename to docs/basic/theme/img/inspect_element.png diff --git a/basic/theme/img/inspect_element_old.png b/docs/basic/theme/img/inspect_element_old.png similarity index 100% rename from basic/theme/img/inspect_element_old.png rename to docs/basic/theme/img/inspect_element_old.png diff --git a/basic/theme/img/logo_override.png b/docs/basic/theme/img/logo_override.png similarity index 100% rename from basic/theme/img/logo_override.png rename to docs/basic/theme/img/logo_override.png diff --git a/basic/theme/img/theme_admin_1.png b/docs/basic/theme/img/theme_admin_1.png similarity index 100% rename from basic/theme/img/theme_admin_1.png rename to docs/basic/theme/img/theme_admin_1.png diff --git a/basic/theme/img/theme_admin_2.png b/docs/basic/theme/img/theme_admin_2.png similarity index 100% rename from basic/theme/img/theme_admin_2.png rename to docs/basic/theme/img/theme_admin_2.png diff --git a/basic/theme/img/themes_admin_section.png b/docs/basic/theme/img/themes_admin_section.png similarity index 100% rename from basic/theme/img/themes_admin_section.png rename to docs/basic/theme/img/themes_admin_section.png diff --git a/docs/basic/theme/index.md b/docs/basic/theme/index.md new file mode 100644 index 00000000..504f2d83 --- /dev/null +++ b/docs/basic/theme/index.md @@ -0,0 +1,281 @@ +# GeoNode Themes + +| We have already explained in `simple-theming`{.interpreted-text role="ref"} how to change the GeoNode theme directly from the *Admin Interface*. This is an easy way for customizing GeoNode appearance but, in some cases, you might want to have more control on it. +| In those cases, you have to venture into the code and it is highly recommended to use a GeoNode Project and customize it instead of the GeoNode default HTML/CSS code. See the following sections to learn more about that. + +# Theming your GeoNode Project + +There are a range of options available to you if you want to change the default look and feel of your `geonode-project`{.interpreted-text role="ref"}. + +## Logos and graphics + +GeoNode intentionally does not include a large number of graphics files in its interface. +This keeps page loading time to a minimum and makes for a more responsive interface. +That said, you are free to customize your GeoNode's interface by simply changing the default logo, or by adding your own images and graphics to deliver a GeoNode experience the way you envision it. + +Your GeoNode project has a directory already set up for storing your own images at `/static/img`. +You should place any image files that you intend to use for your project in this directory. + +Let's walk through an example of the steps necessary to change the default logo. + +1. Change to the `img`{.interpreted-text role="file"} directory: + + ``` console + $ cd /static/img + ``` + +2. If you haven\'t already, obtain your logo image. The URL below is just an example, so you will need to change this URL to match the location of your file or copy it to this location: + + ``` console + $ wget https://upload.wikimedia.org/wikipedia/commons/thumb/a/ac/Service_mark.svg/500px-Service_mark.svg.png + $ wget https://upload.wikimedia.org/wikipedia/commons/thumb/c/c8/Wikimapia_logo_without_label.svg/426px-Wikimapia_logo_without_label.svg.png -O logo.png + ``` + +3. Create snippets directory : + + ``` console + $ cd ../../.. + $ mkdir /templates/geonode-mapstore-client/snippets + $ cd /templates/geonode-mapstore-client/snippets + ``` + +4. Create a new HTML file named `brand_navbar.html`{.interpreted-text role="file"} + + ``` console + $ sudo vi brand_navbar.html + ``` + + ``` css + {% extends "geonode-mapstore-client/snippets/brand_navbar.html" %} + {% load static %} + {% block extra_style %} + + {% endblock %} + {% block logo_src %} + {% static 'img/logo.png' %} + {% endblock %} + ``` + +5. Restart your GeoNode project and look at the page in your browser: + + ``` console + $ cd /home/geonode + $ sudo rm -Rf geonode/geonode/static_root/* + $ cd my_geonode + $ python manage.py collectstatic + $ sudo service apache2 restart + ``` + + ::: note + ::: title + Note + ::: + + It is a good practice to cleanup the **static_folder** and the Browser Cache before reloading in order to be sure that the changes have been correctly taken and displayed on the screen. + ::: + +Visit your site at or the remote URL for your site. + +>
+> img/logo_override.png +>
Custom logo
+>
+ +In the following sections you will learn how to customize this header to make it as you want. + +::: note +::: title +Note +::: + +You should commit these changes to your repository as you progress through this section, and get in the habit of committing early and often so that you and others can track your project on GitHub. +Making many atomic commits and staying in sync with a remote repository makes it easier to collaborate with others on your project. +::: + +## Cascading Style Sheets + +In the last section you already learned how to override GeoNode's default CSS rules to include your own logo. +You are able to customize any aspect of GeoNode's appearance this way. +In the last screenshot, you saw that the main area in the homepage is covered up by the expanded header. + +First, we'll walk through the steps necessary to displace it downward so it is no longer hidden, then change the background color of the header to match the color in our logo graphic. + +1. Reopen `/static/css/brand_navbar.html`{.interpreted-text role="file"} in your editor: + +> ``` console +> $ cd /templates/geonode-mapstore-client/snippets +> $ sudo vi brand_navbar.html +> ``` + +1. Append a rule to change the background color of the header to match the logo graphic: + +> ``` css +> #gn-brand-navbar { +> .... +> background-color: #ff0000 !important; +> } +> ``` + +1. Create new file to manipulate *hero* section: + +> ``` console +> $ cd /templates/geonode-mapstore-client/snippets +> $ sudo vi hero.html +> ``` + +1. Add the following code to change the background image and font for the *hero* section: + +> ``` css +> {% extends "geonode-mapstore-client/snippets/hero.html" %} +> {% block extra_style %} +> +> {% endblock %} +> ``` + +1. Collect the static files into STATIC_ROOT, restart the development server and reload the page: + + > ``` console + > $ python manage.py collectstatic + > $ sudo service apache2 restart + > ``` + > + >
+ > img/css_override.png + >
CSS override
+ >
+ +You can continue adding rules to this file to override the styles that are in the GeoNode base CSS file which is built from [base.less](https://github.com/GeoNode/geonode/blob/master/geonode/static/geonode/less/base.less). + +::: note +::: title +Note +::: + +You may find it helpful to use your browser\'s development tools to inspect elements of your site that you want to override to determine which rules are already applied. See the screenshot below. + +
+img/inspect_element.png +
Screenshot of using browser debugger to inspect the CSS overrides
+
+::: + +## Modify GeoNode Homepage + +So far we learned how to modify some template sections of your GeoNode main page. +You can do it individually per section template, adding a new page under +`/templates/geonode-mapstore-client/snippets`{.interpreted-text role="file"} folder with the +section name (ex: `brand_navbar.html`{.interpreted-text role="file"}) or by extending the base template file +`custom_theme.html`{.interpreted-text role="file"} where you can add different theme settings in one place. + +1. Remove the previous [hero]{.title-ref} section `hero.html`{.interpreted-text role="file"} file: + +> ``` console +> $ rm /templates/geonode-mapstore-client/snippets/hero.html +> ``` + +1. Create a new `custom_theme.html`{.interpreted-text role="file"} file: + +> ``` console +> $ cd /templates/geonode-mapstore-client/snippets +> $ sudo vi custom_theme.html +> ``` + +1. Add the following content to this page: + +> ``` css +> {% load static %} +> {% block content %} +> +> {% endblock %} +> ``` + +1. Restart httpd server + +> ``` console +> $ python manage.py collectstatic +> $ sudo service apache2 restart +> ``` + +1. Your customized layout should be similar to the next picture: + +> ![](img/customized_geonode_project_home.png){.align-center} + +1. Edit title and intro message + +Login as administrator on GeoNode and go to `Admin` page: + +> ![](img/admin_menu.png){.align-center} + +Create a new theme under [GeoNode Themes Library]{.title-ref} and `Themes`: + +> ![](img/themes_admin_section.png){.align-center} + +Add a `Name`, `Description` and turn on `Is enabled` option. +At the bottom, add a `Jumbotron title` and `Jumbotron content`. +This will override the default GeoNode welcome title and message. +Click `Save` at the bottom in the end. + +> ![](img/theme_admin_1.png){.align-center} +> +> ![](img/theme_admin_2.png){.align-center} + +After this, reload your GeoNode homepage. The output should be similar to this: + +> ![](img/custom_home.png){.align-center} diff --git a/contribute/documentation/index.rst b/docs/contribute/documentation/index.md similarity index 53% rename from contribute/documentation/index.rst rename to docs/contribute/documentation/index.md index 476d2fc2..3094222d 100644 --- a/contribute/documentation/index.rst +++ b/docs/contribute/documentation/index.md @@ -1,61 +1,54 @@ -.. _contrib_docu: - -============================================ -How to contribute to GeoNode's Documentation -============================================ - +# How to contribute to GeoNode\'s Documentation {#contrib_docu} If you feel like adding or changing something in the GeoNode documentation you are very welcome to do so. The documentation always needs improvement as the development of the software is going quite fast. To contribute to the GeoNode documentation you should: -* Read the GeoServer Style Guidelines -* Create an account on GitHub -* Fork the GeoNode repository -* Edit the files -* Submit pull requests +- Read the GeoServer Style Guidelines +- Create an account on GitHub +- Fork the GeoNode repository +- Edit the files +- Submit pull requests -All these things can generally be done within your browser, you won't need to download anything. However, if you need to add images or planning bigger changes working locally is recommended. +All these things can generally be done within your browser, you won\'t need to download anything. However, if you need to add images or planning bigger changes working locally is recommended. -Style Guidelines ----------------- -While we do not have strict rules for writing docs, we encourage you to read GeoServer Style Guidelines before you start writing: https://docs.geoserver.org/latest/en/docguide/style.html +## Style Guidelines -Create an account on GitHub ---------------------------- +While we do not have strict rules for writing docs, we encourage you to read GeoServer Style Guidelines before you start writing: -The first step is to create an account on GitHub. Just go to `Github `_, find a username that suits you, enter your email and a password and hit *Sign up for GitHub*. -After you've signed in, visit the geonode_documentation repository https://github.com/geonode/documentation. +## Create an account on GitHub -Fork the documentation repository ---------------------------------- +The first step is to create an account on GitHub. Just go to [Github](https://github.com), find a username that suits you, enter your email and a password and hit *Sign up for GitHub*. +After you\'ve signed in, visit the geonode_documentation repository . -In order to make changes, you first have to fork the repository. On the top right of the website, you will find a button named "fork" to do so. +## Fork the documentation repository -If you want to read more about forking please visit the official GitHub docs: https://help.github.com/articles/fork-a-repo. +In order to make changes, you first have to fork the repository. On the top right of the website, you will find a button named \"fork\" to do so. +If you want to read more about forking please visit the official GitHub docs: . -Edit files on Github --------------------- +## Edit files on Github For smaller changes you can use the GitHub website. Navigate your Browser to your forked repository. To make changes to files, navigate to the file in question and hit the *edit* button on the right top. -.. note:: - The documentation is written in *reStructeredText*, a lightweight markup language. To learn how to use it see: https://docutils.sourceforge.net/docs/user/rst/quickref.html. +::: note +::: title +Note +::: + +The documentation is written in *reStructeredText*, a lightweight markup language. To learn how to use it see: . +::: By hitting the *preview* button you will be able to see how your changes will look like. To save your changes, click on *Commit Changes* at the bottom of the site. To ask the documentation maintainers to integrate your changes the creation of a *Pull Request* is needed. -Therefore use the *new pull request* button to start the process. Find more about Pull requests at the official GitHub documentation: https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests . - +Therefore use the *new pull request* button to start the process. Find more about Pull requests at the official GitHub documentation: . -Edit files locally ------------------- +## Edit files locally -If you're planning bigger changes on the structure of the documentation, it is advisable to make your changes locally. Further, while you can work on your master branch, it is recommended to create a dedicated branch for your changes. - -Start by navigating to a folder where you like to keep your repository locally and install the needed dependencies :: +If you\'re planning bigger changes on the structure of the documentation, it is advisable to make your changes locally. Further, while you can work on your master branch, it is recommended to create a dedicated branch for your changes. +Start by navigating to a folder where you like to keep your repository locally and install the needed dependencies : $ cd /opt $ git clone https://github.com/your_documentation_repository @@ -73,40 +66,37 @@ Start by navigating to a folder where you like to keep your repository locally a $ git push # update your repository at GitHub (origin) -Your repository should now be up to date! For more information on those commands go to https://git-scm.com/docs. -Let's install the dependencies :: +Your repository should now be up to date! For more information on those commands go to . +Let\'s install the dependencies : $ pip install virtualenv $ virtualenv docs_env $ source docs_env/bin/activate $ pip install sphinx sphinx_rtd_theme sphinx-autobuild -You can now start the sphinx development server which will serve and live-reload your docs at https://localhost:8000 :: +You can now start the sphinx development server which will serve and live-reload your docs at : $ sphinx-autobuild . _build -When finished create a build with following command :: +When finished create a build with following command : $ make html # for a last check you can open the index.html in _build subdirectory +## Create a pull request -Create a pull request ---------------------- - -As with directly editing files in your browser, you will need to create a Pull request to ask for integrating your changes into the main repository. :: - - $ git status - # will list all changed files +As with directly editing files in your browser, you will need to create a Pull request to ask for integrating your changes into the main repository. : - $ git add ... - # add the files of interest + $ git status + # will list all changed files - $ git commit -m 'Fixes #1234 Updated docs for ...' - # choose a meaningful commit message + $ git add ... + # add the files of interest - $ git push + $ git commit -m 'Fixes #1234 Updated docs for ...' + # choose a meaningful commit message + $ git push After running these commands, navigate your browser to your GitHub repository and create a pull request as explained above. diff --git a/docs/contribute/index.md b/docs/contribute/index.md new file mode 100644 index 00000000..d4a8debd --- /dev/null +++ b/docs/contribute/index.md @@ -0,0 +1,27 @@ +# Participate in the Discussion + +::: {.toctree maxdepth="3"} +participate_discussion/index +::: + +# Write Documentation + +::: {.toctree maxdepth="3"} +documentation/index +::: + +# Provide Translations + +::: {.toctree maxdepth="3"} +translation/index +::: + +# Write Code + + +- [index](participate_discussion/index.md) + +- [index](documentation/index.md) + +- [index](translation/index.md) + diff --git a/docs/contribute/participate_discussion/index.md b/docs/contribute/participate_discussion/index.md new file mode 100644 index 00000000..7d32f052 --- /dev/null +++ b/docs/contribute/participate_discussion/index.md @@ -0,0 +1,11 @@ +# Join the community, ask for help or report bugs {#contrib_docu} + +In case of general questions the GeoNode Community is present at following *channels* + +- User Mailing List: +- Developer Mailing List: +- Gitter Chat: + +For reporting bugs please open a ticket at Github issues: + +- diff --git a/contribute/translation/img/github_commit.png b/docs/contribute/translation/img/github_commit.png similarity index 100% rename from contribute/translation/img/github_commit.png rename to docs/contribute/translation/img/github_commit.png diff --git a/contribute/translation/img/github_drag_files.png b/docs/contribute/translation/img/github_drag_files.png similarity index 100% rename from contribute/translation/img/github_drag_files.png rename to docs/contribute/translation/img/github_drag_files.png diff --git a/contribute/translation/img/github_upload.png b/docs/contribute/translation/img/github_upload.png similarity index 100% rename from contribute/translation/img/github_upload.png rename to docs/contribute/translation/img/github_upload.png diff --git a/contribute/translation/img/poedit_compile.png b/docs/contribute/translation/img/poedit_compile.png similarity index 100% rename from contribute/translation/img/poedit_compile.png rename to docs/contribute/translation/img/poedit_compile.png diff --git a/contribute/translation/img/poedit_interface.png b/docs/contribute/translation/img/poedit_interface.png similarity index 100% rename from contribute/translation/img/poedit_interface.png rename to docs/contribute/translation/img/poedit_interface.png diff --git a/contribute/translation/img/poedit_needs_work.png b/docs/contribute/translation/img/poedit_needs_work.png similarity index 100% rename from contribute/translation/img/poedit_needs_work.png rename to docs/contribute/translation/img/poedit_needs_work.png diff --git a/contribute/translation/img/poedit_sort.png b/docs/contribute/translation/img/poedit_sort.png similarity index 100% rename from contribute/translation/img/poedit_sort.png rename to docs/contribute/translation/img/poedit_sort.png diff --git a/contribute/translation/img/poedit_validate.png b/docs/contribute/translation/img/poedit_validate.png similarity index 100% rename from contribute/translation/img/poedit_validate.png rename to docs/contribute/translation/img/poedit_validate.png diff --git a/contribute/translation/img/poedit_warning.png b/docs/contribute/translation/img/poedit_warning.png similarity index 100% rename from contribute/translation/img/poedit_warning.png rename to docs/contribute/translation/img/poedit_warning.png diff --git a/docs/contribute/translation/index.md b/docs/contribute/translation/index.md new file mode 100644 index 00000000..43af8275 --- /dev/null +++ b/docs/contribute/translation/index.md @@ -0,0 +1,97 @@ +# Contribute to Translations + +Behind the scenes, GeoNode is using a software called GNU gettext further text-based translation files (django.po and djangojs.po) for translating content. If you\'d like to know more about how all of this works you\'ll find a full description at the [Django Docs](https://docs.djangoproject.com/en/3.2/topics/i18n/translation/). +Following will concentrate on what is needed for edit existing or contribute a new translation. + +## Download the translation File + +All language files live in a specific subfolder called after their [iso code](https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes/) within the [locale folder](https://github.com/GeoNode/geonode/tree/master/geonode/locale/). +For example, for French, the main translation file called django.po can be downloaded from [here](https://github.com/GeoNode/geonode/blob/master/geonode/locale/fr/LC_MESSAGES/django.po/). + +Next, to download the language file, we need to install an OpenSource Editor called \"poedit\" for editing from: + +## Translation process + +Make a copy of the file before starting the translation so that you can revert in case of errors. + +After installing \'poedit\', you should be able to double click on the \'.po\' file to open it. Poedit's interface should look similar to the one shown in the picture below: + +![](img/poedit_interface.png){.align-center width="680px"} + +## Identifying translation issues + +From the \'poedit\' menu \'View\', make sure that \'Entries with Errors first\' is checked: + +![](img/poedit_sort.png){.align-center width="280px"} + +Next click on \'Validate Translations\' from the \'Catalogue\' menu: + +![](img/poedit_validate.png){.align-center width="280px"} + +\'Poedit\' will place translations which may require additional consideration on top of the list. +A warning mark means that the interpretation might be not entirely consistent with the original phrase. This is not necessarily an error, just a warning asking the user to double check. + +![](img/poedit_warning.png){.align-center width="680px"} + +Following to marked phrases, \'Poedit\' will show untranslated sentences. When clicking on one, it can be translated through the bottom panel. + +During translation pay special attention to the button saying \'needs work\'. In case this button is checked, the phrase will be marked as \'fuzzy\' and ignored in GeoNode. + +![](img/poedit_needs_work.png){.align-center} + +## Saving translations + +As soon as the translation is complete, it must be saved and compiled. +Saving is straightforward. All you have to do is clicking the \'Save\' button from the top menu. + +As a last step we compile the file. Compiling the translation means to create a binary \".mo\" file out of the edited \".po\" file. To do so, click on \"Compile to MO\" + +![](img/poedit_compile.png){.align-center width="280px"} + +Poedit will ask where to write the \".mo\" file to, by default, this is the same folder as the edited \'.po\' resides in. The \'.mo\' file can be overwritten if necessary. + +## Push translations to the repository + +For sharing our updates, we must upload the files to GeoNode's GitHub repository. +Go to the correct file position which, in case for French is: + + +Click on \"Upload Files\" + +![](img/github_upload.png){.align-center width="680px"} + +Drag the updated files into the Upload form, and write a title/description of the changes + +![](img/github_drag_files.png){.align-center width="680px"} + +Click on \"Create a new branch for this commit...\" and then click on the green button. + +![](img/github_commit.png){.align-center width="680px"} + +The last step will create a [PULL REQUEST]{.title-ref} which can be reviewed and then approved by a developer. + +## Activate updated translation at your server + +Once the files have been pushed to GitHub, it will be necessary to update your server to respect changed files. + +At this time, this can be done only by an administrator. From the server \'shell\' following commands need to be executed: + +``` shell +workon geonode +cd /opt/geonode +DJANGO_SETTINGS_MODULE=geonode.settings python -W ignore manage.py collectstatic --noinput +sudo service uwsgi restart +``` + +## Texts not listed in .po files + +In case you find a template output without corresponding translation you can add it as follows: + +Identify the corresponding template file which is responsible for outputting the text. Add a `{% trans "TEXT" %}` tag. Save the template file and run the following: + +``` shell +django-admin makemessages --no-location -l en -d django -e "html,txt,py" -i docs +django-admin makemessages --no-location -l en -d djangojs -e "js" -i docs -i node_modules -i lib +``` + +This will update the english .po file. also to update the language which should be edited by settings the `-l fr` parameter. Continue with updating the .po file as described above. diff --git a/docs/devel/api/usage/index.md b/docs/devel/api/usage/index.md new file mode 100644 index 00000000..8069c64e --- /dev/null +++ b/docs/devel/api/usage/index.md @@ -0,0 +1,730 @@ + -------------------- + API usage examples + -------------------- + +| In this section, we are going to demostrate how GeoNode API can be utilized/integrated with other applications using Python. + +# Resource Listing and Details + +As mentioned in previous chapters, GeoNode resources are categorized in different types e.g. datasets, maps, documents. Etc. +All available resources can be listed with API `GET /api/v2/resources`. + +To obtain a single resource, a primary key is provided in the url. Eg `GET /api/v2/resources/{resource.pk}`. + +## Example Requests: + +1. Listing + +``` shell +import requests + +url = "https://master.demo.geonode.org/api/v2/resources" +response = requests.request("GET", url) +``` + +2. Detail + +``` shell +import requests + +url = "https://master.demo.geonode.org/api/v2/resources/1797" +response = requests.request("GET", url) +``` + +::: note +::: title +Note +::: + +The above requests work for publicly visible resources. If a resource is private either the Basic Auth or the Bearer token must be included inside the headers. +::: + +3. Listing with basic auth: + +``` shell +import requests + +url = "https://master.demo.geonode.org/api/v2/resources" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("GET", url, headers=headers) +``` + +A token can be used in place of Basic Auth by setting `'Authorization': 'Bearer '`. + +# Resource Download + +The download URL for a resource can be obtained from `resource.download_url`. This URL executes the synchronous download of a resource in its default download format (ESRI Shapefile for vector, Geotiff for rasters and the original format for documents). +Additional export formats for datasets are available through the UI. At the moment the API doesn't expose them. + +# Resource Links + +From the resource detail response, URls and links to services can be obtained from the `resource.links[]` array value. +The purpose of each link is defined by its `link_type`. The "name" also can specify additional information about the linked resource. + +## Metadata + +Links to each metadata format can be obtained from links with `link_type = "metadata"` + +OGC services +\^\^\^\^\^\^\^\^ + +OGC requests can be built by taking: +the OGC base url from links from `resource.links[]` with `"link_type"= ("OGC:WMS | OGC:WFS | OGC:WCS)` +the OGC service layername obtained from the `resource.alternate` property + +Embedding +\^\^\^\^\^\^\^\^ +A resource can be embedded inside a third party website. The "embed view" of a resource is suitable to be placed inside an iframe. +The URL for the embedded view can be obtained from the `resource.embed_url` property. + +# Resource Searching and Filtering + +GeoNode resources can be filtered with the following query parameters: + + -------------------------------------------------------------------------------------------------------------------------------------------------------------------- + Parameters URL + ------------------------------------------------------------------------ ------------------------------------------------------------------------------------------- + title and abstract `(paginated free text search)` /api/v2/resources?page=1&search=text-to-search&search_fields=title&search_fields=abstract + + resource_type `(dataset, map, document, service, geostory, dashboard)` /api/v2/resources?filter{resource_type}=map + + subtype `(raster,vector, vector_time, remote)` /api/v2/resources?filter{resource_type}=vector + + favorite `(Boolean True)` /api/v2/resources?favorite=true + + featured `(Boolean True or False)` /api/v2/resources?filter{featured}=true + + published `(Boolean True or False)` /api/v2/resources?filter{is_published}=true + + aprroved `(Boolean True or False)` /api/v2/resources?filter{is_approved}=true + + category api/v2/resources?filter{category.identifier}=example + + keywords /api/v2/resources?filter{keywords.name}=example + + regions /api/v2/resources?filter{regions.name}=global + + owner /api/v2/resources?filter{owner.username}=test_user + + extent `(Four comer separated coordinates)` /api/v2/resources?extent=-180,-90,180,90 + -------------------------------------------------------------------------------------------------------------------------------------------------------------------- + +Examples: + +1. Filter with a single value + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/resources/?filter{resource_type}=map" +response = requests.request("GET", url, headers=headers, data=payload +``` + +2. Filter with multiple values + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/resources/?filter{resource_type.in}=map&filter{resource_type.in}=dataset" +response = requests.request("GET", url, headers=headers, data=payload) +``` + +::: note +::: title +Note +::: + +With filter APIs of format `/api/v2/resources?filter{filter_key}=value`, additional methods(in and icontains) can be used on them to provide extensively filtered results. +Eg +`/api/v2/resources?filter{regions.name.icontains}=global` +`/api/v2/resources?filter{regions.name.in}=global`. + +It\'s important to note that other methods are case sensitive except the icontains. +::: + +# Dataset specific resources + +Get the metadata of uploaded datasets with: + +: - API: `GET /api/v2/datasets/{id}` + + \- Status Code: `200` + .. note:: + This is very similar to [GET /api/v2/resources]{.title-ref} but provides additional metadata specifically for datasets like [featureinfo_custom_template]{.title-ref} or [attribute_set]{.title-ref} + + Example: + + ``` python + import requests + + DATASET_ID = "the dataset id" + url = f"https://master.demo.geonode.org/api/v2/datasets/{DATASET_ID}" + headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' + } + response = requests.request("GET", url, headers=headers) + ``` + +# Resource Upload + +The API supports the upload of datasets and documents. + +### Datasets + +The dataset upload form accepts file formats of ESRI Shapefile, GeoTIFF, Comma Separated Value (CSV), Zip Archive, XML Metadata File, and Styled Layer Descriptor (SLD). +For a successful upload, the form requires base_file, dbf_file, shx_file, and prj_file. The xml_file, and Sld_file are optional. + +- API: `POST /api/v2/uploads/upload` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/uploads/upload" +files= [ +('sld_file',('BoulderCityLimits.sld',open('/home/myuser/BoulderCityLimits.sld','rb'),'application/octet-stream')), ('base_file',('BoulderCityLimits.shp',open('/home/BoulderCityLimits.shp','rb'),'application/octet-stream')), ('dbf_file',('BoulderCityLimits.dbf',open('/home/BoulderCityLimits.dbf','rb'),'application/octet-stream')), ('shx_file',('BoulderCityLimits.shx',open('/home/BoulderCityLimits.shx','rb'),'application/octet-stream')), +('prj_file',('BoulderCityLimits.prj',open('/home/myuser/BoulderCityLimits.prj','rb'),'application/octet-stream)) +] +headers = { +'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("POST", url, headers=headers, files=files) +``` + +### Documents + +Documents can be uploaded as form data. + +- API: `POST /api/v2/documents` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "http://localhost:8000/api/v2/documents" +payload={ + 'title': 'An example image' +} +files=[ + ('doc_file',('image.jpg',open('/home/myuser/image.jpg','rb'),'image/jpeg')) +] +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("POST", url, headers=headers, data=payload, files=files) +``` + +Documents can also be created to reference remote resources. In this case the `doc_url` parameter must be used to set the URL of the remote document. + +- API: `POST /api/v2/documents` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "http://localhost:8000/api/v2/documents" +payload={ + 'title': 'An example image', + 'doc_url' : 'http://examples.com/image.jpg' +} +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("POST", url, headers=headers, data=payload, files=files) +``` + +Notice that if the URL doesn\'t end with a valid doc extension, the `extension` parameter must be used (e.g. `extension: 'jpeg'`). + +## Tracking dataset upload progress + +When an upload request is executed, GeoNode creates an \"Execution request\" and keeps updating its state and progress (it's a property attribute, calculated on getting the response) attributes as the resource is being created and configured in Geoserver. +An execution can be in one of the following status: +- `ready` +- `running` +- `failed` +- `finished` + +When the dataset is successfully uploaded, the final state of the upload is set to `finished`. + +In order to view status of the execution, the API method `GET /api/v2/executionrequest/{execution_id}` where `{execution_id}` is the value returned by the initial call to the upload API. + +The returned object contains, beyond all the information related to the execution, the inputs that were passed to the execution request, and output params specific to the type of execution. +In the case of a dataset upload, the output params contain the URL of the catalog page for the new datast. + +``` json +"output_params": { + "detail_url": [ + "/catalogue/#/dataset/9881" + ] +}, +``` + +You can also filter executions by status. +Eg `GET /api/v2/executionrequest?filter{action}=import&filter{source}=upload&filter{status}=finished` + +Example: + +``` python +import requests + +url = "https://stable.demo.geonode.org/api/v2/executionrequest/5f640b6b-8c51-4514-a054-995133fee107" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("GET", url, headers=headers) +``` + +Overwriting a dataset +\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^ +Uploading a resource will create by default a new dataset. This behaviour can be changed by setting the `overwrite_existing_layer` parameter to `True`. +In this case the upload procedure will overwrite a resource whose name matches with the new one. + +Skip existing dataset +\^\^\^\^\^\^\^\^\^\^\^\^ +If the parameter `skip_existing_layers` is set to true `True` the uplad procedure will ignore files whose name matched with already existing resources. + +Upload of a metadata file +\^\^\^\^\^\^\^\^\^\^\^\^ +A complete metadata file conforming to ISO-19115 can be uploaded for a dataset. + +- API: `PUT /api/v2/datasets/{dataset_id}/metadata` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "http://localhost:8000/api/v2/datasets/1/metadata" +files=[ + ('metadata_file',('metadata.xml',open('/home/user/metadata.xml','rb'),'text/xml')) + ] +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("PUT", url, headers=headers, data={}, files=files) +``` + +# Resource Delete + +- API: `DELETE /api/v2/resources/{pk}/delete` +- Status Code: `204` + +Example: + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/resources/1778" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("DELETE", url, headers=headers) +``` + +eee + +# Resource Download + +GeoNode offers a download option to resources of resource_type dataset and document. +For datasets, the download option is available for only datasets with uploaded files. + +## Datasets + +- API: `GET /datasets/{resource.alternate}/dataset_download` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "https://master.demo.geonode.org/datasets/geonode:BoulderCityLimits3/dataset_download" +response = requests.request("GET", url) +``` + +Documents +\^\^\^\^\^\^\^\^ +- API: `GET /documents/{resource.pk}/download` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "https://master.demo.geonode.org/documents/1781/download" +response = requests.request("GET", url) +``` + +# Dataset Update Metadata + +- API: `PATCH /api/v2/datasets/{id}` +- Status Code: `200` + +The following example changes the title and the license of a dataset. + +``` python +import requests + +url = ROOT + "api/v2/datasets/" + DATASET_ID +auth = (LOGIN_NAME, LOGIN_PASSWORD) + +data = { + "title": "a new title", + "license": 4, +} +response = requests.patch(url, auth=auth, json=data) +``` + +::: note +::: title +Note +::: + +[bbox_polygon]{.title-ref} and [ll_bbox_polygon]{.title-ref} are derived values which cannot be changed. +::: + +# Users, Groups and Permissions + +## Users + +### Listing + +- API: `POST /api/v2/users` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/users" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("GET", url, headers=headers) +``` + +User detail +\"\"\"\"\"\"\"\"\"\" +- API: `POST /api/v2/users/{pk}` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/users/1000" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("GET", url, headers=headers) +``` + +Create a new user +\"\"\"\"\"\"\"\"\"\" +- API: `POST /api/v2/users` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/users" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +payload={"username": "username", + "password": "password", + "email": "email@email.com", + "first_name":"first_name", + "last_name":"last_name", + "avatar": "https://www.gravatar.com/avatar/7a68c67c8d409ff07e42aa5d5ab7b765/?s=240"} +response = requests.request("POST", url, headers=headers, data=payload) +``` + +Edit a User +\"\"\"\"\"\"\"\"\"\" +- API: `PATCH /api/v2/users/{pk}` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/users/1000" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +payload={"password": "new_password"} +response = requests.request("PATCH", url, headers=headers, data=payload) +``` + +Delete a User +\"\"\"\"\"\"\"\"\"\" +- API: `DELETE /api/v2/users/{pk}` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/users/1000" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +payload={"password": "new_password"} +response = requests.request("DELETE", url, headers=headers, data=payload) +``` + +In this case the list of validation rules configured in `user-deletion-rules`{.interpreted-text role="ref"} are checked before the deletion is executed. + +List user groups +\"\"\"\"\"\"\"\"\"\" +- API: `POST /api/v2/users/{pk}/groups` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/users/1000/groups" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("GET", url, headers=headers) +``` + +Transfer resources owned by a user to another +\"\"\"\"\"\"\"\"\"\" +- API: `POST /api/v2/users/{pk}/transfer_resources` +- Status Code: `200` + +Example: + +``` python +import requests +payload={"owner": 1001} +url = "https://master.demo.geonode.org/api/v2/users/1000/transfer_resources" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("POST", url, headers=headers, data=payload) +``` + +In this case the resources will be transfered to the user with id 1001, +instead using the payload={\"owner\": \"DEFAULT\"} the resources will be transfered to the principal user + +Remove user as a group manager +\"\"\"\"\"\"\"\"\"\" +- API: `POST /api/v2/users/{pk}/remove_from_group_manager` +- Status Code: `200` + +Example: + +``` python +import requests +payload={"groups": [1,2,3]} +url = "https://master.demo.geonode.org/api/v2/users/1000/remove_from_group_manager" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("POST", url, headers=headers, data=payload) +``` + +In this case the user shall be removed as a group manager from the following group ids, +if the payload would be payload={\"groups\": \"ALL\"} the user will be removed as a group manager from all the groups its part of + +## Groups + +In GeoNode, On listing groups, the api returns groups which have group profiles. Therefore for django groups which are not related to a group profile are not included in the response. However these can be accessed in the Django Administration panel. + +- API: `POST /api/v2/groups` +- Status Code: `200` + +Example: + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/groups" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("GET", url, headers=headers) +``` + +## Permissions + +Permissions in GeoNode are set per resource and per user or group. The following are general permissions that can be assigned: + +- *View:* allows to view the resource. `[view_resourcebase]` +- *Download:* allows to download the resource specifically datasets and documents. `[ view_resourcebase, download_resourcebase]` +- *Edit:* allows to change attributes, properties of the datasets features, styles and metadata for the specified resource. `[view_resourcebase, download_resourcebase, change_resourcebase, change_dataset_style, change_dataset_data, change_resourcebase_metadata]` +- *Manage:* allows to update, delete, change permissions, publish and unpublish the resource. `[view_resourcebase, download_resourcebase, change_resourcebase, change_dataset_style, change_dataset_data, publish_resourcebase, delete_resourcebase, change_resourcebase_metadata, change_resourcebase_permissions]` + +## Obtaining permissions on a resource + +On listing the resources or on resource detail API, GeoNode includes perms attribute to each resource with a list of permissions a user making the request has on the respective resource. + +GeoNode also provides an API to get an overview of permissions set on a resource. The response contains users and groups with permissions set on them. However this API returns `200` if a requesting user has `manage` permissions on the resource otherwise it will return `403 (Forbidden)`. + +- API: `GET /api/v2/resources/1791/permissions` + +Example: + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/resources/1791/permissions" +headers = { + 'Authorization': 'Basic dXNlcjpwYXNzd29yZA==' +} +response = requests.request("GET", url, headers=headers) +``` + +## Changing permissions on a resource + +Permissions are configured with a so called `perms spec`, which is a JSON structured where permissions for single users and groups can be specified. + +The example below shows a perm specification for following rules: + +- user1 can `edit` +- user2 can `manage` +- group1 can `edit` +- anonymous users (public) can `view` +- registered members can `download` + +NOTE: The id of the "anonymous" and "registered members" groups can be obtained from the permissions of the resource. They are not listed inside the groups API, since these are spceial internal groups + +``` python +{ + "users": [ + { + "id": , + "permissions": "edit" + }, + { + "id": , + "permissions": "manage" + } + ], + "organizations": [ + { + "id": , + "permissions": "edit" + }, + ], + "groups": [ + { + "id": , + "permissions": "view" + }, + { + "id": , + "permissions": "download" + } + ] +} +``` + +The perm spec is sent as JSON, with `application/json Content-Type`, inside a `PUT` request. + +``` python +import requests +import json + +url = "https://master.demo.geonode.org/api/v2/resources/1791/permissions" +payload = json.dumps({ +"users": [ + { + "id": 1001, + "permissions": "edit" + }, + { + "id": 1002, + "permissions": "manage" + } +], +"organizations": [ + { + "id": 1, + "permissions": "edit" + } +], +"groups": [ + { + "id": 2, + "permissions": "view" + }, + { + "id": 3, + "permissions": "download" + } +] +}) +headers = { +'Authorization': 'Basic dXNlcjpwYXNzd29yZA==', +'Content-Type': 'application/json', +} + +response = requests.request("PUT", url, headers=headers, data=payload) +``` + +This is an asynchrnous operation which returns a response similar to the following: + +``` python +{ + "status": "ready", + "execution_id": "7ed578c2-7db8-47fe-a3f5-6ed3ca545b67", + "status_url": "https://master.demo.geonode.org/api/v2/resource-service/execution-status/7ed578c2-7db8-47fe-a3f5-6ed3ca545b67" +} +``` + +The `status_url` property returns the URL to track kthe progress of the request. Querying the URL a result similar to the following will be returned: + +``` python +{ + "user": "admin", + "status": "running", + "func_name": "set_permissions", + "created": "2022-07-08T11:16:32.240453Z", + "finished": null, + "last_updated": "2022-07-08T11:16:32.240485Z", + "input_params": { + … + } +} +``` + +The operation will be completed once the `status` property is updated with the value `finished`. + +# Linked Resources Listing and Details + +All available linked_resources can be listed with API `GET /api/v2/resources/{pk}/linked_resources`. +where pk Resource base id + +## Example Requests: + +1. List all resource links + +``` python +import requests + +url = "https://master.demo.geonode.org/api/v2/resources/{pk}/linked_resources" +response = requests.request("GET", url) +``` diff --git a/devel/docker/data/vscode_debugpy_launch.json b/docs/devel/docker/data/vscode_debugpy_launch.json old mode 100755 new mode 100644 similarity index 100% rename from devel/docker/data/vscode_debugpy_launch.json rename to docs/devel/docker/data/vscode_debugpy_launch.json diff --git a/devel/docker/data/vscode_runserver_launch.json b/docs/devel/docker/data/vscode_runserver_launch.json old mode 100755 new mode 100644 similarity index 100% rename from devel/docker/data/vscode_runserver_launch.json rename to docs/devel/docker/data/vscode_runserver_launch.json diff --git a/docs/devel/docker/index.md b/docs/devel/docker/index.md new file mode 100644 index 00000000..13c2772a --- /dev/null +++ b/docs/devel/docker/index.md @@ -0,0 +1,77 @@ +# Start to develop with Docker + +## How to run the instance for development + +There are two options to develop using Docker containers: + +- **Alternative A**: Running by command line and editing the code using your preferred editor (usually harder). +- **Alternative B**: Using the vscode [remote containers](https://code.visualstudio.com/docs/remote/containers) extension (easier). + +### Alternative A: Building and running Docker for development + +Build (first time only): + +``` shell +docker-compose --project-name geonode -f docker-compose-dev.yml -f .devcontainer/docker-compose.yml build +``` + +Running: + +``` shell +docker-compose --project-name geonode -f docker-compose-dev.yml -f .devcontainer/docker-compose.yml up +``` + +::: note +::: title +Note +::: + +If you are running `postgresql` and `tomcat9` services, you need to stop them, +`docker-compose` will take care of running the database and geonode service. + +Otherwise, you will get the following error: + +``` text +ERROR: for db Cannot start service db: driver failed programming external connectivity on endpoint db4geonode: Error starting userland proxy: listen tcp4 0.0.0.0:5432: bind: address already in use +ERROR: Encountered errors while bringing up the project. +``` +::: + +Running the geonode application in debug mode: + +``` shell +docker exec -it django4geonode bash -c "python manage.py runserver 0.0.0.0:8000" +``` + +When running, you can debug the application using your preferred method. +For example, you can edit a file, save it and see your modifications. +You can also use [ipdb](https://github.com/gotcha/ipdb) to add breakpoints and inspect your code +(Writing `import ipdb; ipdb.set_trace()` in the line you want your breakpoint). + +Another option is to use *debugpy* alongside with *vscode*, for this you have to enable *debugpy* inside your *django4geonode* container: + +``` shell +docker exec -it django4geonode bash -c "pip install debugpy -t /tmp && python /tmp/debugpy --wait-for-client --listen 0.0.0.0:5678 manage.py runserver 0.0.0.0:8000 --nothreading --noreload" +``` + +Select \"**Run and Debug**\" in vscode and use the following launch instruction in your `.vscode/launch.json` file: + +`launch.json `{.interpreted-text role="download"} + +### Alternative B: Using vscode extension + +Alternatively, you can develop using the vscode [remote containers](https://code.visualstudio.com/docs/remote/containers) extension. +In this approach you need to: + +- Install the extension in your vscode: [ms-vscode-remote.remote-containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) +- On your command pallet, select: \"**Remote-Containers: Reopen in Container**\" +- If it\'s the first time, vscode will take care of building the images. This might take some time. +- Then a new vscode window will open, and it\'ll be connected to your docker container. +- The message \"**Dev Container: Debug Docker Compose**\" will appear in the bottom-left corner of that window. +- In the vscode terminal, you\'re going to see something similar to `root@77e80acc89b8:/usr/src/geonode#`. +- To run your application, you can use the integrated terminal (`./manage.py runserver 0.0.0.0:8000`) or the vscode \"**Run and Debug**\" option. + For launching with \"Run and Debug\", use the following instruction file: + +`launch.json `{.interpreted-text role="download"} + +For more information, take a read at vscode remote containers [help page](https://code.visualstudio.com/docs/remote/containers). diff --git a/docs/devel/index.md b/docs/devel/index.md new file mode 100644 index 00000000..0f6fc0df --- /dev/null +++ b/docs/devel/index.md @@ -0,0 +1,53 @@ +# GeoNode API + +::: {.toctree maxdepth="3"} +api/usage/index +api/linked_resources/index +::: + +# Metadata + +::: {.toctree maxdepth="4"} +metadata/index +::: + +# How to Develop + +::: {.toctree maxdepth="3"} +docker/index +::: + +::: {.toctree maxdepth="2"} +installation/index +::: + +::: {.toctree maxdepth="2"} +running/index +::: + +::: {.toctree maxdepth="2"} +mapstore/index +::: + +::: {.toctree maxdepth="2"} +workshops/index +::: + +This documentation helps developers to install GeoNode-Core and GeoNode-Project from different scenarios. GeoNode-Project can be installed on top of GeoNode-Core if already installed. Also GeoNode-Project can be installed from scratch as it has GeoNode-Core as a prerequisit. + + +- [index](api/usage/index.md) +- [index](api/linked_resources/index.md) + +- [index](metadata/index.md) + +- [index](docker/index.md) + +- [index](installation/index.md) + +- [index](running/index.md) + +- [index](mapstore/index.md) + +- [index](workshops/index.md) + diff --git a/devel/installation/img/geonode-gui.png b/docs/devel/installation/img/geonode-gui.png similarity index 100% rename from devel/installation/img/geonode-gui.png rename to docs/devel/installation/img/geonode-gui.png diff --git a/devel/installation/img/server-is-ready.png b/docs/devel/installation/img/server-is-ready.png similarity index 100% rename from devel/installation/img/server-is-ready.png rename to docs/devel/installation/img/server-is-ready.png diff --git a/docs/devel/installation/index.md b/docs/devel/installation/index.md new file mode 100644 index 00000000..cbf7c633 --- /dev/null +++ b/docs/devel/installation/index.md @@ -0,0 +1,496 @@ +# How to Install GeoNode-Core for development + +## Summary of installation + +This section demonstrates a summarization of the steps to be followed in order to install GeoNode-Core for development using Ubuntu 18.04. The following steps will be customized to fit both GeoNode-Project and GeoNode-Core for development purpose. + +## The steps to be followed are: + +1- Install build tools and libraries + +2- Install dependencies and supporting tools + +3- Setup Python virtual environment + +4- Clone and install GeoNode from Github + +5- Install and start Geoserver + +6- Start GeoNode + +::: note +::: title +Note +::: + +The following commands/steps will be executed on your terminal +::: + +::: warning +::: title +Warning +::: + +If you have a running GeoNode service, you will need to stop it before starting the following steps. To stop GeoNode you will need to run: +::: + +``` shell +service apache2 stop # or your installed server +service tomcat7 stop # or your version of tomcat +``` + +### Install GeoNode-Core for development + +GeoNode-Core installation is considered the most basic form of GeoNode. It doesn\'t require any external server to be installed and it can run locally against a file-system based Spatialite database. + +## Installation steps + +1- Install build tools and libraries + +::: warning +::: title +Warning +::: + +Those instructions might be outdated. Please refer to `install_dep`{.interpreted-text role="ref"} +::: + +``` shell +$ sudo apt-get install -y build-essential libxml2-dev libxslt1-dev libpq-dev zlib1g-dev +``` + +2- Install dependencies and supporting tools + +> Install python native libraries and tools + +::: warning +::: title +Warning +::: + +Those instructions might be outdated. Please refer to `install_dep`{.interpreted-text role="ref"} +::: + +``` shell +$ sudo apt-get install -y python3-dev python3-pil python3-lxml python3-pyproj python3-shapely python3-nose python3-httplib2 python3-pip software-properties-common +``` + +Install python virtual environment + +::: warning +::: title +Warning +::: + +Those instructions might be outdated. Please refer to `install_venv`{.interpreted-text role="ref"} +::: + +``` shell +$ sudo pip install virtualenvwrapper +``` + +Install postgresql and postgis + +::: warning +::: title +Warning +::: + +Those instructions might be outdated. Please refer to `configure_dbs_core`{.interpreted-text role="ref"} +::: + +``` shell +$ sudo apt-get install postgresql-10 postgresql-10-postgis-2.4 +``` + +Change postgres password expiry and set a password + +``` shell +$ sudo passwd -u postgres # change password expiry infromation +$ sudo passwd postgres # change unix password for postgres +``` + +Create geonode role and database + +``` shell +$ su postgres +$ createdb geonode_dev +$ createdb geonode_dev-imports +$ psql +$ postgres=# +$ postgres=# CREATE USER geonode_dev WITH PASSWORD 'geonode_dev'; # should be same as password in setting.py +$ postgres=# GRANT ALL PRIVILEGES ON DATABASE "geonode_dev" to geonode_dev; +$ postgres=# GRANT ALL PRIVILEGES ON DATABASE "geonode_dev-imports" to geonode_dev; +$ postgres=# \q +$ psql -d geonode_dev-imports -c 'CREATE EXTENSION postgis;' +$ psql -d geonode_dev-imports -c 'GRANT ALL ON geometry_columns TO PUBLIC;' +$ psql -d geonode_dev-imports -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' +$ exit +``` + +Edit PostgreSQL configuration file + +``` shell +sudo gedit /etc/postgresql/10/main/pg_hba.conf +``` + +Scroll to the bottom of the file and edit this line + +``` shell +# "local" is for Unix domain socket connections only +local all all peer +``` + +To be as follows + +``` shell +# "local" is for Unix domain socket connections only +local all all trust +``` + +Then restart PostgreSQL to make the changes effective + +``` shell +sudo service postgresql restart +``` + +Java dependencies + +``` shell +$ sudo apt-get install -y openjdk-11-jdk --no-install-recommends +``` + +Install supporting tools + +``` shell +$ sudo apt-get install -y ant maven git gettext +``` + +3- Setup Python virtual environment (Here is where Geonode will be running) + +Add the virtualenvwrapper to your new environment. + +Since we are using Ubuntu, you can add the following settings to your .bashrc file. Please note that the Ubuntu account here is called \"geonode\". So you will need to change it according to the name you picked. + +``` shell +$ echo export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python >> ~/.bashrc +$ echo export WORKON_HOME=/home/geonode/dev/.venvs >> ~/.bashrc +$ echo source /usr/local/bin/virtualenvwrapper.sh >> ~/.bashrc +$ echo export PIP_DOWNLOAD_CACHE=$HOME/.pip-downloads >> ~/.bashrc + +And reload the settings by running +$ source ~/.bashrc +``` + +Set up the local virtual environment for Geonode + +``` shell +$ vim ~/.bashrc +# add the following line to the bottom +$ source /usr/share/virtualenvwrapper/virtualenvwrapper.sh +``` + +``` shell +$ source /usr/share/virtualenvwrapper/virtualenvwrapper.sh +$ mkvirtualenv --python=/usr/bin/python3 geonode +$ workon geonode # or $ source /home/geonode/dev/.venvs/geonode/bin/activate +This creates a new directory where you want your project to be and creates a new virtualenvironment + +Alterantively you can also create the virtual env like below +$ python3.8 -m venv /home/geonode/dev/.venvs/geonode +$ source /home/geonode/dev/.venvs/geonode/bin/activate +``` + +4- Download/Clone GeoNode from Github + +To download the latest geonode version from github, the command \"git clone\" is used + +::: note +::: title +Note +::: + +If you are following the GeoNode training, skip the following command. You can find the cloned repository in /home/geonode/dev +::: + +``` shell +$ git clone https://github.com/GeoNode/geonode.git -b 4.1.x +``` + +Install Nodejs PPA and other tools required for static development + +This is required for static development + +::: note +::: title +Note +::: + +If you are following GeoNode's training, nodejs is already installed in the Virtual Machine skip the first three command and jump to cd geonode/geonode/static +::: + +``` shell +$ sudo apt-get install nodejs npm +$ cd geonode/geonode/static +$ npm install --save-dev +``` + +::: note +::: title +Note +::: + +Every time you want to update the static files after making changes to the sources, go to geonode/static and run 'grunt production'. +::: + +::: warning +::: title +Warning +::: + +Starting from the following step, you have to make sure that you installed GDAL correctly according to the documentation page \"Install GDAL for Development\" +::: + +Install GeoNode in the new active local virtualenv + +``` shell +$ cd /home/geonode/dev # or to the directory containing your cloned GeoNode +$ pip install -e geonode +$ cd geonode/geonode +``` + +Create local_settings.py + +Copy the sample file /home/geonode/dev/geonode/geonode/local_settings.py.geoserver.sample and rename it to be local_settings.py + +``` shell +$ cd /home/geonode/dev/geonode +$ cp geonode/local_settings.py.geoserver.sample geonode/local_settings.py +$ gedit geonode/local_settings.py +``` + +In the local_settings.py file, add the following line after the import statements: + +``` python +SITEURL = "http://localhost:8000/" +``` + +In the DATABASES dictionary under the \'default\' key, change only the values for the keys NAME, USER and PASSWORD to be as follows: + +``` python +DATABASES = { +'default': { + 'ENGINE': 'django.db.backends.postgresql_psycopg2', + 'NAME': 'geonode_dev', + 'USER': 'geonode_dev', + 'PASSWORD': 'geonode_dev', + ....... + ...... + ..... + .... + ... + }...} +``` + +In the DATABASES dictionary under the \'datastore\' key, change only the values for the keys NAME, USER and PASSWORD to be as follows: + +``` python +# vector datastore for uploads +'datastore' : { + 'ENGINE': 'django.contrib.gis.db.backends.postgis', + #'ENGINE': '', # Empty ENGINE name disables + 'NAME': 'geonode_dev-imports', + 'USER' : 'geonode_dev', + 'PASSWORD' : 'geonode_dev', + ....... + ...... + ..... + .... + ... +} +``` + +In the CATALOGUE dictionary under the \'default\' key, uncomment the USER and PASSWORD keys to activate the credentials for GeoNetwork as follows: + +``` python +CATALOGUE = { +'default': { + # The underlying CSW implementation + # default is pycsw in local mode (tied directly to GeoNode Django DB) + 'ENGINE': 'geonode.catalogue.backends.pycsw_local', + # pycsw in non-local mode + # 'ENGINE': 'geonode.catalogue.backends.pycsw_http', + # GeoNetwork opensource + # 'ENGINE': 'geonode.catalogue.backends.geonetwork', + # deegree and others + # 'ENGINE': 'geonode.catalogue.backends.generic', + # The FULLY QUALIFIED base url to the CSW instance for this GeoNode + 'URL': urljoin(SITEURL, '/catalogue/csw'), + # 'URL': 'http://localhost:8080/geonetwork/srv/en/csw', + # 'URL': 'http://localhost:8080/deegree-csw-demo-3.0.4/services', + # login credentials (for GeoNetwork) + 'USER': 'admin', + 'PASSWORD': 'admin', + # 'ALTERNATES_ONLY': True, + }} +``` + +5- Install and Start Geoserver + +From the virtual environment, first you need to align the database structure using the following command : + +``` shell +$ cd /home/geonode/dev/geonode +$ python manage.py migrate +``` + +::: warning +::: title +Warning +::: + +If the start fails because of an import error related to osgeo or libgeos, then please consult the [Install GDAL for Development](https://training.geonode.geo-solutions.it/005_dev_workshop/004_devel_env/gdal_install.html) +::: + +then setup GeoServer using the following command: + +``` shell +$ paver setup + +$ paver sync +``` + +6- Now we can start our geonode instance + +::: warning +::: title +Warning +::: + +Don't forget to stop the GeoNode Production services if enabled +::: + +``` shell +service apache2 stop +service tomcat7 stop +``` + +``` shell +$ paver start +``` + +The starting process will take around 20 seconds (depends on your machine) and at the end it shows the following message: + +![image](./img/server-is-ready.png) + +Now you can visit the geonode site by typing into your browser window + +![image](./img/geonode-gui.png) + +### Install GeoNode-Project for development after installing GeoNode-Core + +Geonode-Project gives the user flexibility to customize the installation of the GeoNode. Geonode itself will be installed as a requirement of your project. Inside the project structure it is possible to extend, replace or modify all geonode components (e.g. css and other static files, templates, models..) and even register new django apps without touching the original Geonode code. +In order to install GeoNode-Project, the following steps need to be executed alongside the previous GeoNode-Core installation steps. + +1- Use django-admin.py to create a project \"my_geonode\" from a GeoNode-Project template as follows: + +::: note +::: title +Note +::: + +Before running the following command, make sure that you are currently working on the virtual environment and just outside geonode directory. The command will create a new project called \"my_geonode\" which should be located at the level of geonode-core installation directory \"inside /home/geonode/dev\" +::: + +``` shell +$ django-admin.py startproject my_geonode --template=https://github.com/GeoNode/geonode-project/archive/master.zip -e py,rst,json,yml,ini,env,sample -n Dockerfile + +$ ls /home/geonode/dev # should output: geonode my_geonode +``` + +::: note +::: title +Note +::: + +Although the following command might show that the majority of requirements are already satisfied \"because GeoNode-Core was already installed\", it is recommended to still execute it as it might update or install any missing package. +::: + +2- Install all the required packages/tools for GeoNode-Project as follows: + +``` shell +$ pip install -e my_geonode +``` + +::: note +::: title +Note +::: + +As mentioned earlier, GeoNode will be installed as requirement for the GeoNode-Project in order to be able to extend it +::: + +### Install GeoNode-Project directly from scratch + +If you didn\'t install GeoNode-Core earlier and you wanted to install GeoNode-Project directly, please follow these steps + +1- Create a virtual environment as follows: + +``` shell +$ vim ~/.bashrc +# add the following line to the bottom +$ source /usr/share/virtualenvwrapper/virtualenvwrapper.sh +``` + +``` shell +$ source /usr/share/virtualenvwrapper/virtualenvwrapper.sh +$ mkvirtualenv --python=/usr/bin/python3 my_geonode + +Alterantively you can also create the virtual env like below +$ python3.8 -m venv /home/geonode/dev/.venvs/my_geonode +$ source /home/geonode/dev/.venvs/my_geonode/bin/activate +``` + +2- Clone the geonode-project repo from Github + +``` shell +$ git clone https://github.com/GeoNode/geonode-project.git -b 4.1.x +``` + +3- Install Django framework as follows + +``` shell +$ pip install Django==3.2.13 +``` + +4- Use django-admin.py to create a project \"my_geonode\" from a GeoNode-Project template as follows: + +``` shell +$ django-admin startproject --template=./geonode-project -e py,sh,md,rst,json,yml,ini,env,sample,properties -n monitoring-cron -n Dockerfile my_geonode +``` + +5- Install all the requirements for the GeoNode-Project and install the GeoNode-Project using pip + +``` shell +$ cd my_geonode +$ pip install -r requirements.txt --upgrade +$ pip install -e . --upgrade +``` + +6- Install GDAL Utilities for Python + +``` shell +$ pip install pygdal=="`gdal-config --version`.*" # or refer to the link +``` + +7- Install GeoServer and Tomcat using paver + +``` shell +$ paver setup + +$ paver sync + +$ paver start +``` + +8- Visit diff --git a/docs/devel/installation/install_gdal_for_development.md b/docs/devel/installation/install_gdal_for_development.md new file mode 100644 index 00000000..167db8d7 --- /dev/null +++ b/docs/devel/installation/install_gdal_for_development.md @@ -0,0 +1,89 @@ +# How to Install GDAL for development + +## Summary of installation + +This section demonstrates a summarization of the steps to be followed in order to install GDAL 2.4.2 for development using python2.7 virtualenv with Ubuntu 18.04. The idea here is to install GDAL on the host machine and then refer to this installed version from the virtual environment to have identical versions. + +::: note +::: title +Note +::: + +The following commands/steps will be executed on your terminal +::: + +::: warning +::: title +Warning +::: + +The installation here is meant to serve GeoNode installation for development +::: + +## The steps to be followed are: + +1- Install GDAL on your host environment + +2- Validate your installation on the host machine + +3- From your python virtual environment, Run: pip install GDAL==\$(gdal-config \--version) \--global-option=build_ext \--global-option=\"-I/usr/include/gdal\" + +4- Validate installation from host and virtual environment + +Installation steps + +On a fresh Ubuntu 22.04 installation, we will be installing gdal version 3.3.2 + +1- Using the terminal in the host machine, run the following commands: + +``` shell +$ sudo apt-get update +$ sudo apt-get install python3-gdal=3.3.2+dfsg-2~focal2 gdal-bin=3.3.2+dfsg-2~focal2 libgdal-dev=3.3.2+dfsg-2~focal2 +``` + +::: note +::: title +Note +::: + +Make sure you don\'t have a newer version on the software list under /et/apt/sources.list.d/ +::: + +2- Validate the installed version by running the following commands: + +``` shell +$ sudo gdal-config --version #it should output 3.3.2 at this step +$ python -c "from osgeo import gdal; print gdal.__version__" #it should output 3.3.2 at this step +``` + +3- Install GDAL on your virtual environment by referring to the same version which we just installed on the host machine. To do that, you will need to run the following command during activated virtual environment session as follows: + +``` shell +$ virtualenv -p python2.7 my_env # To create a python2.7 virtual environment called my_env + +$ # inside my_env/bin, run the following command to activate the virtual environment we just created + +$ source activate + +$ cd /home/geonode/my_env + +$ export CPLUS_INCLUDE_PATH=/usr/include/gdal + +$ export C_INCLUDE_PATH=/usr/include/gdal + +$ pip install GDAL==2.4.2 +``` + +4- Confirm the installation + +In order to validate the installation between the host and the virtual environment, run the following command on both the host machine and on the activated virtual environment. If the results were identical, then GDAL for development is installed correctly for GeoNode + +``` shell +$ python -c "from osgeo import gdal; print gdal.__version__" # outputs "2.4.2" +``` + +And from the activated environment: + +``` shell +$ (my_env) python -c "from osgeo import gdal; print gdal.__version__" # outputs "2.4.2" +``` diff --git a/docs/devel/mapstore/index.md b/docs/devel/mapstore/index.md new file mode 100644 index 00000000..a7c75cc4 --- /dev/null +++ b/docs/devel/mapstore/index.md @@ -0,0 +1,90 @@ +# Start MapStore2 client in development mode + +## Pre-requisites + +1. You need a running instance of GeoNode somewhere; in this specific example we assume GeoNode is running on `http://localhost:8000`{.interpreted-text role=":guilabel"} + +### Install needed packages + +``` shell +sudo apt install nodejs npm +``` + +### Prepare the source code + +``` shell +git clone --recursive https://github.com/GeoNode/geonode-mapstore-client.git geonode-mapstore-client-dev +``` + +### Compile MapStore2 Client + +``` shell +cd geonode-mapstore-client/geonode_mapstore_client/client/ +npm update +npm install +npm run compile +``` + +### Edit the file `env.json` + +``` shell +vim env.json +``` + +``` json +{ + "DEV_SERVER_HOST": "localhost:8000", + "DEV_SERVER_HOST_PROTOCOL": "http" +} +``` + +### Run MapStore2 in Development mode + +``` shell +npm run start +``` + +Connect to :`http://localhost:8081`{.interpreted-text role=":guilabel"} + +This is a `proxied` version of GeoNode form MapStore2 client. **To upload new layers user the original GeoNode**. + +Everytime you render a map, from GeoNode layers details page or map creation, you will access to the MapStore2 dev mode runnig code. + +You can now update the code on the fly. + +### Example 1: Disable the PrintPlugin from the Layer Details small map + +``` shell +vim js/previewPlugins.js +``` + +``` javascript +... +BurgerMenuPlugin: require('../MapStore2/web/client/plugins/BurgerMenu'), +ScaleBoxPlugin: require('../MapStore2/web/client/plugins/ScaleBox'), +MapFooterPlugin: require('../MapStore2/web/client/plugins/MapFooter'), +// PrintPlugin: require('../MapStore2/web/client/plugins/Print'), +TimelinePlugin: require('../MapStore2/web/client/plugins/Timeline'), +PlaybackPlugin: require('../MapStore2/web/client/plugins/Playback'), +... +``` + +### Example 2: Disable the MousePositionPlugin from the big maps + +``` shell +vim js/plugins.js +``` + +``` javascript +... +SaveAsPlugin: require('../MapStore2/web/client/plugins/SaveAs').default, +MetadataExplorerPlugin: require('../MapStore2/web/client/plugins/MetadataExplorer'), +GridContainerPlugin: require('../MapStore2/web/client/plugins/GridContainer'), +StyleEditorPlugin: require('../MapStore2/web/client/plugins/StyleEditor'), +TimelinePlugin: require('../MapStore2/web/client/plugins/Timeline'), +PlaybackPlugin: require('../MapStore2/web/client/plugins/Playback'), +// MousePositionPlugin: require('../MapStore2/web/client/plugins/MousePosition'), +SearchPlugin: require('../MapStore2/web/client/plugins/Search'), +SearchServicesConfigPlugin: require('../MapStore2/web/client/plugins/SearchServicesConfig'), +... +``` diff --git a/devel/metadata/caching.rst b/docs/devel/metadata/caching.md similarity index 69% rename from devel/metadata/caching.rst rename to docs/devel/metadata/caching.md index bbd4fdaa..c67b854c 100644 --- a/devel/metadata/caching.rst +++ b/docs/devel/metadata/caching.md @@ -1,8 +1,7 @@ -Schema caching -============== +# Schema caching The JSON Schema is requested and used very often when dealing with metadata: the client requests the schema every time the metadata editor is opened, the metadata manager uses the schema both when creating and receiving an instance. -Generating the JSON Schema may be a bit heavy on the backend, so there's a caching mecahnism that holds a built schema for every language (remember that also the fields' labels and descriptions are inside the JSON schema, and different languages may have different labels). +Generating the JSON Schema may be a bit heavy on the backend, so there\'s a caching mecahnism that holds a built schema for every language (remember that also the fields\' labels and descriptions are inside the JSON schema, and different languages may have different labels). The JSONSchema structure is usually fixed, so the cache should be expunged only when there are changes in the labels. There are mechanisms, based on the label thesaurus date that force the different workers to recreate the schema when labels change. diff --git a/docs/devel/metadata/client.md b/docs/devel/metadata/client.md new file mode 100644 index 00000000..384259da --- /dev/null +++ b/docs/devel/metadata/client.md @@ -0,0 +1,150 @@ +# Metadata editor client + +The metadata editor client is basically implemented using the [react-jsonschema-form](https://github.com/rjsf-team/react-jsonschema-form) library. + +Some improvements have been made, in order to better integrate with GeoNode, such as handling autocomplete, error reporting etc. + +In order to do, some custom annotation have been added. + +## `geonode:after` + +This annotation is used server side, to place a field just after another. + +In the metadata editor the fields are presented in the same order as they are declared in the schema; this annotation allows to add a new declared field after an existing one: + +``` json +"short_name": { + "type": ["string", "null"], + "geonode:after": "title", + "geonode:handler": "sparse" + }, +``` + +## Text widget + +Be default [react-jsonschema-form]{.title-ref} presents the string elements in text fields. +If your text is multiline, you may want to edit it in a text area. + +This is how it is declared: + +``` json +"ui:options": { + "widget": "textarea", + "rows": 5 +}, +``` + +## Dropdown menus {#metadata_dropdown} + +In case you have a field that should be populated with the content of a service, you may use the autocomplete feature: + +``` json +"ui:options": { + "geonode-ui:autocomplete": "/api/v2/metadata/autocomplete/users" +} +``` + +This is an improvement to [react-jsonschema-form]{.title-ref} implemented within GeoNode. + +The client will present a dropdown menu with the content got from the autocomplete service. + +Please note that in order to handle such kind of dropdown, the service shall provide a list of entries having `id` and `label` as fields. + +According to the field type, different widgets will be shown. + +### Single choice + +Here an example of an object: the widget will be a single choice dropdown menu: + +``` json +"owner": { + "type": "object", + "title": "Proprietario", + "properties": { + "id": { + "type": "string", + "ui:widget": "hidden" + }, + "label": { + "type": "string", + } + }, + "ui:options": { + "geonode-ui:autocomplete": "/api/v2/metadata/autocomplete/users" + }, +}, +``` + +And this is how it is rendered: + +
+img/metadata_dropdown_single.png +
Single choice dropdown
+
+ +### Multi choice + +In case the field is an array, the dropdown will be multichoice: + +``` json +"EOVsReference": { + "type": "array", + "items": { + "type": "object", + "properties": { + "id": { + "type": "string" + }, + "label": { + "type": "string" + } + } + }, + "ui:options": { + "geonode-ui:autocomplete": "/api/v2/metadata/autocomplete/thesaurus/eov/keywords" + } +}, +``` + +And this is how it is rendered: + +
+img/metadata_dropdown_multi.png +
Multi choice dropdown
+
+ +## Codelists {#metadata_dropdown_codelist} + +In case a dropdown is fed with a codelist stored in a Thesaurus, you may use the `geonode:thesaurus` annotation, that will create an `autocomplete` entry pointing to the defined thesaurus. The dropdown can be either *single* or *multi* choice, as documented in the sections above. + +For instance, feeding the `SparseHandler` this subschema: + +``` json +"p_data_level": { + "type": "object", + "properties": { + "id": {"type": "string"}, + "label": {"type": "string"} + }, + "geonode:handler": "sparse", + "geonode:thesaurus": "prj1_data_level" +}, +``` + +it will give back toward the client: + +``` json +"p_data_level": { + "type": "object", + "properties": { + "id": {"type": "string"}, + "label": {"type": "string"} + }, + "geonode:handler": "sparse", + "geonode:thesaurus": "prj1_data_level", + "title": "Data Level", + "ui:options": { + "geonode-ui:autocomplete": "/api/v2/metadata/autocomplete/thesaurus/prj1_data_level/keywords" + } + }, +``` diff --git a/docs/devel/metadata/design.md b/docs/devel/metadata/design.md new file mode 100644 index 00000000..28a95598 --- /dev/null +++ b/docs/devel/metadata/design.md @@ -0,0 +1,145 @@ +# Metadata app design + +The idea is to use a well-estabilished standard to document the metadata schema and to return the metadata info. + +The metadata schema is presented using the [JSON schema](https://json-schema.org/) conventions. + +The GeoNode API has been extended to provide: + +- `/api/v2/metadata/schema` metadata schema +- `/api/v2/metadata/instance/` resource metadata instances handling (GET/PUT) +- `/api/v2/metadata/<...>` other utility calls for the metadata handling (autocomplete endpoints referred into the schema, etc) + +One of the basic driving features has been the simplification of **customization of the schema**. The implementation relies on a set of **metadata handlers** that can be easily extended. + +The optional [geonode-inspire](https://github.com/geosolutions-it/geonode-inspire) module is a working showcase that documents how the GeoNode base model can be customized. + +# Implementation + +The package `geonode.metadata` contains all the code related to metadata handling. + +The `manager` module contains the base engine that calls methods on the registered field handlers. + +## Handlers overview + +**Handlers** are the objects that manage the field lifecycle, i.e.: + +- create the the json **sub-schema** of the field +- take care of **loading data** from the database and convert it into json schema instance +- parse the json schema instance value sent from the client and store it in the database format + +A single handler may handle one or more fields. + +## `MetadataManager` + +The `geonode.metadata.manager.MetadataManager` class handles requests from the view, and delegates the work on the fields to the declared handlers. + +The MetadataManager provides these main methods: + +``` +def get_schema(self, lang=None) +``` + +**Builds and returns the metadata json schema.** + +This method is called by the `/api/v2/metadata/schema` endpoint. + +The schema is created by looping on the sorted set of metadata handlers, asking them to update the json schema incrementally via `def update_schema(self, jsonschema: dict, context, lang=None)`. + +Handlers may add their own field, remove or modify fields added by previously called handlers. + +The schema localization is only related to the [title]{.title-ref} and [description]{.title-ref} annotations. Localization of dropdown lists is an handler-specific task, and usually carried out via thesauri codelists (see `thesauri`{.interpreted-text role="ref"}). + +The default language used is the one set in the cookies, or it can be overriden by adding the query param [lang]{.title-ref}. + +``` +def build_schema_instance(self, resource: ResourceBase, lang=None) +``` + +**Builds and returns the json schema instance related to a resource.** + +The instance is created by examining the json schema and looping on each field. +For each field, the related handler is called to fill in the serialized value for such field. + +Called handlers\' methods: + +- `def load_serialization_context(resource: ResourceBase, jsonschema: dict, context: dict)` + Allow an handler to pre-load some common info for all of its handled fields +- `def get_jsonschema_instance(resource: ResourceBase, field_name, context, errors, lang=None)` + Returns the instance of the sub-schema associated with the field [field_name]{.title-ref}. + +``` +def update_schema_instance(self, resource, request_obj, lang=None) -> dict: +``` + +**Parses the incoming json schema instance and stores the value into the DB.** + +Persistence is done by examining the json schema and looping on each field. +For each field, the related handler is called to parse and store the deserialized value(s) for such field. +If the field is a complex object, it\'s up to the handler to parse and handle it. + +Called handlers\' methods: + +- `def load_deserialization_context(self, resource: ResourceBase, jsonschema: dict, context: dict)` + Allow handlers to pre-load some common info for all the handled fields +- `def update_resource(resource: ResourceBase, field_name: str, json_instance: dict, context: dict, errors: dict, **kwargs)` + Logic to parse and persist the field + +## Handlers + +The list of handlers is declared in `geonode.metadata.settings.METADATA_HANDLERS`. + +**Order is important** since later declared handlers may want to customize previously defined fields. + +Within each field subschema there are some geonode-specific annotations (custom annotation are allowed by the json schema standard), some are needed for backend logic (for instance `geonode:handler`), other for the client UI (e.g. `ui:options`) + +### `BaseHandler` + +`BaseHandler` handles most of the fields declared inside the `ResourceBase` class. + +The `BaseHandler` will read the main schema from a file ([metadata/schemas/base.json](https://github.com/GeoNode/geonode/blob/metadata_manager/geonode/metadata/schemas/base.json)), augmenting programmatically some info that can not be statically set (such as autocomplete URLs). + +### `SparseHandler` + +This handler is not actively used by the core GeoNode schema, since it\'s more aimed at customizations. + +You may see an example of its use in the [geonode-inspire]{.title-ref} package. + +The basic idea for the SparseField (the single model change in metadata) is to have a \"vertical\" table with free form fields: + +
+img/metadata_sparse_field.png +
ResourceBase and its Sparse fields
+
+ +In the default case, its use is as simple as declaring the field: + +``` json +"prj1_distance": { + "type": "numeric" +} +``` + +or even a complex structure as + +``` json +"prj1_instrumentations": { + "type": "array", + "title": "instrumentations", + "description": "Provides information about any instruments used in the data collection. The description should include vendor, model number, optional equipment, etc.", + "items": { + "type": "string", + "minLength": 1, + "default": "" + }, + "minItems": 1, + "geonode:handler": "sparse" +} +``` + +When the type is `array` or `object` the SparseHandler will encode the whole content as a json object + +## Some custom annotations + +- `geonode:handler`: key to the handler defined in `METADATA_HANDLERS`; tells which is the handler to be called to handle the field\'s content. Please note that this annotation may be replaced by subsequent handlers if they want to handle the field differently. +- `geonode:required`: the jsonschema specs wants the `required` array outside the subschema definition. By defining this annotation as `true`, the metadata manager will add the current field in the `required` list. diff --git a/docs/devel/metadata/example.md b/docs/devel/metadata/example.md new file mode 100644 index 00000000..0ca7aa2d --- /dev/null +++ b/docs/devel/metadata/example.md @@ -0,0 +1,111 @@ +# Examples: Adding a metadata field + +To wrap up the metadata doumentation, here we will present some examples about how to add a brand new field to the metedat editor. + +## Example 1: minimal field + +Some points related to this field: + +- We do not need any special behaviour (i.e. logic for checking internal consistency, updating other fields according to the content of this field, etc) +- The field is declared as a simple type (numeric) + +In this case, where we don\'t need any specific logic, we can rely on the existing [SparseHandler](https://github.com/GeoNode/geonode/blob/master/geonode/metadata/handlers/sparse.py). + +The SparseHandler has its own register of handled field, where custom apps can add their own, so in order to add a numerical field it will be enough to add it in the register, by specifying its name and the JSON sub-schema, e.g.: + +``` python +from geonode.metadata.handlers.sparse import sparse_field_registry + +sparse_field_registry.register("accuracy", {"type": "number"}) +``` + +This simple code will add the sparse field \"accuracy\", typed as \"numeric\", in the schema. + +Please note that the SparseHandler will return to the manager a possibly modified subschema, and not the registered one. +For instance: + +- the `"geonode:handler". "sparse"` will be added, in order to mark the handler for the field +- annotations `title` and `description` may be added or modified (see `metadata_fields_localization`{.interpreted-text role="ref"}). + +e.g. + +``` json +"accuracy": { + "type": "number", + "geonode:handler": "sparse", + "title": "Positional accuracy" +} +``` + +## Example 2: complex json field + +- No special behaviour +- The field may be as much complex as JSON schema specification allows. + +For instance, we want a list of persons defined by 2 fields, one containing the full name, and the other a URI. + +``` json +"prj1_data_creator": { + "type": "array", + "title": "Resource creator", + "minItems": 1, + "items": { + "type": "object", + "properties": { + "fullname": { + "type": "string", + "maxLength": 255, + "title": "Name surname", + "description": "Insert the full name of the person who created the resource" + }, + "orcid": { + "type": "string", + "format": "uri", + "title": "ORCID", + "description": "Insert the ORCID of the person who created the resource" + } + } + }, + "geonode:after": "abstract", + "geonode:handler": "sparse" + }, +``` + +Since the type is a complex one (in this case it\'s `array`, but also `object` is handled the same way), the SparseHandler will treat the content as a json object. Hera a sample for the content of the above structure: + +``` json +"prj1_data_creator": [ + { + "fullname": "John Doe", + "orcid": "http://sampleid" + }, + ], +``` + +## Example 3: codelist + +You can add a field with a dropdown menu by using thesauri (see `metadata_dropdown`{.interpreted-text role="ref"}) + +- creating a brand new thesaurus with the items you needs (see `thesaurus_add`{.interpreted-text role="ref"}); + + - for instance, create a thesaurus with identifier `project1_codelist_foobar` and add some ThesaurusKeywords to it. + +- create a field (either single or multiple entry) and referencing the thesaurus (see `metadata_dropdown_codelist`{.interpreted-text role="ref"}) + + - es: + + > ``` json + > "prj1_foobar": { + > "type": "object", + > "properties": { + > "id": {"type": "string"}, + > "label": {"type": "string"} + > }, + > "geonode:handler": "sparse", + > "geonode:thesaurus": "project1_codelist_foobar" + > }, + > ``` + +- you may also add the label for the new field by adding a ThesaurusKeyword to the Thesaurus `labels_i18n`, + + > ![](img/metadata_thesaurus_label.png){width="250px"} diff --git a/docs/devel/metadata/example2.md b/docs/devel/metadata/example2.md new file mode 100644 index 00000000..641adeaa --- /dev/null +++ b/docs/devel/metadata/example2.md @@ -0,0 +1,155 @@ +# Example: Adding an handler + +## Create the handler + +``` python +from geonode.metadata.handlers.abstract import MetadataHandler + +class MySchemaHandler(MetadataHandler): + pass +``` + +Register your handler, so that that metadata manager is aware of it: + +``` python +from geonode.metadata.manager import metadata_manager + +metadata_manager.add_handler("myhandler", MySchemaHandler) +``` + +## Create skeleton schema + +In order to define multiple fields, it may be useful to create a skeleton schema and add the fields one by one. + +This is a sample skeleton file `myproject.json`: + +``` json +{ + "p1_short_name": { + "type": ["string", "null"], + "minLength": 1, + "maxLength": 255, + "geonode:after": "title", + "geonode:handler": "sparse" + }, + + "edition": { + "type": "object", + "properties": { + "id": {"type": "string"}, + "label": {"type": "string"} + }, + "geonode:handler": "myhandler" + "geonode:thesaurus": "p1_edition_codelist" + } +} +``` + +p1_short_name: + +: This is a brand new field, handled by the SparseHandler, so we don\'t want to handle it in YourSchemaHandler. + It will need to be registered as a SparseField. + +edition: + +: This field is declared into GeoNode as a field handled by the `BaseHandler`. We are telling the manager that we want to handle it via the `YourSchemaHandler`. + Also, we are defining it to be a codelist populated via the \"p1_edition_codelist\" thesaurus, that shall be created. + Since we are declaring a new Handler for this field, we\'ll need the logic to handle it. + +## Registering sparse fields + +You may want to register your stuff when initting your app: + +``` python +def load_schema_file(): + with open(os.path.join(os.path.dirname(__file__), "schemas", "myproject.json")) as f: + schema_file = json.load(f) + + return schema_file + + +def init(): + + # register sparse fields: + for property_name, subschema in load_schema_file().items(): + if subschema.get("geonode:handler", None) == "sparse": + sparse_field_registry.register(property_name, subschema) + + metadata_manager.add_handler("myhandler", YourSchemaHandler) +``` + +## Generate schema + +Then you need to implement the Handler method that initializes its part of schema (`update_schema`) + +``` python +def update_schema(self, jsonschema, context, lang=None): + + schema_file = load_schema_file() + + # building the full schema using the external file + for property_name, subschema in schema_file.items(): + + # tries to set up "title" and "description" + self._localize_subschema_labels(context, subschema, lang, property_name) + + if "geonode:handler" not in subschema: + # set the default handler if not specified + subschema.update({"geonode:handler": "myhandler"}) + else: + # skip fields that have already been added to the sparsefield register + if subschema.get("geonode:handler", None) == "sparse": + continue + + # add this field's subschema to the full jsonschema + self._add_subschema(jsonschema, property_name, subschema) +``` + +Now, since we took control of the `edition` field, we need to implement the methods that implement + +- the model-to-jsonschema transformation (`get_jsonschema_instance`) +- the jsonschema-to-model transformation (`update_resource`) + +## Handle content + +We need to decide what will be stored in the `edition` field. + +The fields bound to thesauri are dict with keys `id` and `label`, as also documented in the schema defined above. + +We want to save into the `edition` field ( which is defined in ResourceBase as a text field) some readable info; let\'s use the default label from the ThesaurusKeyword. In this thesaurus, **do not** define any localized labels, in order to be sure to get the default label as label. + +When the jsonschema instance is returned, we have to provide the pair (id, label) anyway. We may retrieve the KeywordLabel by filtering by the default label, assuming that there are no duplicates. There may be other caveats, but just assume it is ok for the example. + +### model-to-jsonschema + +Let\'s search for the ThesaurusKeyword with the given default label, and create the return object: + +``` python +def get_jsonschema_instance( + self, resource: ResourceBase, field_name: str, context: dict, errors: dict, lang: str = None +): + match field_name: + case "edition": + if resource.edition: + if tl := ThesaurusLabel.objects.filter(alt_label=resource.edition).first() + return {"id": tl.pk, "label": resource.edition} + return {} + case _: + raise Exception(f"Unhandled field {field_name}") +``` + +### jsonschema-to-model + +The jsonschema instance, if existing, is still a dict with keys \"id\" and \"label\"; we\'ll only return the label: + +``` python +def update_resource( + self, resource, field_name, json_instance, context, errors, **kwargs +): + field_value = json_instance.get(field_name, {}) + match field_name: + case "edition": + resource.edition = field_value.get("label", None) + case _: + raise Exception(f"Unhandled field {field_name}") +``` diff --git a/docs/devel/metadata/i18n.md b/docs/devel/metadata/i18n.md new file mode 100644 index 00000000..4551bfc9 --- /dev/null +++ b/docs/devel/metadata/i18n.md @@ -0,0 +1,77 @@ +# Metadata localization {#metadata_fields_localization} + +Before GeoNode 5, the labels for metadata editor have been statically localized using `gettext()` and i18n `.po` files. + +Since the jsonschema is much more dynamic in its behaviour, allowing custom schema, other ways for localizing the schema fields labels were needed. + +The proposed/implemented idea is to use a codelist, implemented as a thesaurus with a well known identifier `labels_i18n` where: + +- + + `Thesaurus`: + + : - `identifier` = \"[labels_i18n]{.title-ref}\" (fixed) + - `date` = latest update + +- + + `ThesaurusKeyword`: + + : - `about` = explicit \"title\" annotation, otherwise field name + - `alt_label` = Default label + +- + + `ThesaurusKeywordLabel`: + + : - localized label + +In order to avoid clashes, the schema fields keys should be prefixed with the metadata handler id (\"`inspire_`\", \"`myproject_`\", \...) + +The implementation will look for translations in the thesaurus, and then will the gettext as fallback. + +## Procedure + +JSON schema provides the annotations `title` and `description` to document the purpose of a field. + +The metadata editor will show the \"title\" as the label for the field and the \"description\", if present, as a tooltip. +If the `title` field does not exist in the schema, the editor will usually show the field name as the label. + +The SparseHandler will look for the `title` and `description` annotation in the subschema, and will try to localize them, using the content as the key for the localization procedures. + +So, if `title` is defined with the content `title_key`, the procedures will look, in order, for: + +- a ThesaurusKeyword with about = `title_key__ovr` +- a ThesaurusKeyword with about = `title_key` +- a `getText` entry with key `title_key` + +and the first returned result is usaed as a replacemente for the \"title\" annotation. The [getText]{.title-ref} value is used as a fallback anyway, since it returns the searched key if no entry is found. + +::: note +::: title +Note +::: + +The `getText` is used as a fallback for base fields for which localization was already provided. In this way the core GeoNode installation does not require the `labels_i18n` thesaurus, allowing for a smooth transition toward the new localization procedure. +::: + +::: note +::: title +Note +::: + +The `__ovr` mechanism has been added as a way to customize locally standard fields\' labels, avoiding conflicts when official thesauri content are changed + +Please note that the thesauri can be partially dumped or imported, in order to ease the handling of customized labels. +::: + +Even if the `title` annotation is not present in the subschema, the GeoNode metadata engine will search for a localized string having key equal to the property name. If found, the `title` annotation will be automatically added. + +Similar to the above procedure, these keys are searched for: + +- a ThesaurusKeyword with about = `__ovr` +- a ThesaurusKeyword with about = `` + +If anything is found, the `title` annotation will be automatically added with the found content. + +In the same way, if the `description` annotation is not declared, the key `__descr` will be searched, and, if found, `description` will be added. diff --git a/devel/metadata/img/metadata_dropdown_multi.png b/docs/devel/metadata/img/metadata_dropdown_multi.png similarity index 100% rename from devel/metadata/img/metadata_dropdown_multi.png rename to docs/devel/metadata/img/metadata_dropdown_multi.png diff --git a/devel/metadata/img/metadata_dropdown_single.png b/docs/devel/metadata/img/metadata_dropdown_single.png similarity index 100% rename from devel/metadata/img/metadata_dropdown_single.png rename to docs/devel/metadata/img/metadata_dropdown_single.png diff --git a/devel/metadata/img/metadata_sparse_field.png b/docs/devel/metadata/img/metadata_sparse_field.png similarity index 100% rename from devel/metadata/img/metadata_sparse_field.png rename to docs/devel/metadata/img/metadata_sparse_field.png diff --git a/devel/metadata/img/metadata_thesaurus_label.png b/docs/devel/metadata/img/metadata_thesaurus_label.png similarity index 100% rename from devel/metadata/img/metadata_thesaurus_label.png rename to docs/devel/metadata/img/metadata_thesaurus_label.png diff --git a/docs/devel/metadata/index.md b/docs/devel/metadata/index.md new file mode 100644 index 00000000..3448cc21 --- /dev/null +++ b/docs/devel/metadata/index.md @@ -0,0 +1,21 @@ +# Metadata + +::: {.toctree maxdepth="3"} +intro +design +i18n +caching +client +example +example2 +::: + + +- [intro](intro.md) +- [design](design.md) +- [i18n](i18n.md) +- [caching](caching.md) +- [client](client.md) +- [example](example.md) +- [example2](example2.md) + diff --git a/docs/devel/metadata/intro.md b/docs/devel/metadata/intro.md new file mode 100644 index 00000000..428f056f --- /dev/null +++ b/docs/devel/metadata/intro.md @@ -0,0 +1,35 @@ +# Intro to GeoNode metadata management + +## Introduction + +Geographic metadata describes the content, quality, location, and structure of geospatial datasets. In GeoNode, it enables dataset discovery, interoperability, and standardized data management through structured, searchable information. + +GeoNode has its own internal metadata model, originally developed to populate the most relevant elements of the ISO 19115 standard. This model supported core geospatial metadata fields and was tightly coupled with the Django backend. + +With the release of GeoNode 5.x, a new metadata engine was introduced. It is based on JSON Schema, making it significantly more flexible and easier to customize. Unlike previous versions, where adding new fields required modifying the Django data model, developers can now define and extend metadata structures through configurable schemas without touching the core database structure. + +## Design overview + +The metadata engine - implemented since GeoNode 5 - is based on a JSON Schema--driven model, offering a flexible and extensible way to manage resource metadata. + +Core concepts: + +- JSON Schema Model + + A single JSON Schema defines the structure, validation, and UI behavior of metadata fields across all resources (e.g., datasets, maps). + + The full set of metadata is passed as input and output, fully respecting the JSON Schema instance specifications, ensuring strict compliance with the defined structure. + +- Dynamic UI Generation + + The frontend generates metadata forms dynamically based on the schema, including field types, labels, grouping, and constraints. + +- Data Storage: + + - Core metadata fields (e.g., title, abstract, category) are still stored in the traditional ResourceBase model. + - Custom or extended fields are stored using a new *Sparse field* mechanism, allowing flexible, schema-driven metadata + extension without altering the database schema. + +- Validation & Localization + + Input is validated against the JSON Schema, and schemas support multilingual labels and descriptions, enabling internationalized forms. diff --git a/docs/devel/running/index.md b/docs/devel/running/index.md new file mode 100644 index 00000000..f6410f34 --- /dev/null +++ b/docs/devel/running/index.md @@ -0,0 +1,43 @@ +# How to run GeoNode Core for development + +In order to start Geonode Core for development, you need to make sure that no Geonode instance is running first. This can be done by running the following commands: + +``` shell +$ cd /home/user/geonode + +$ paver stop_geoserver + +$ paver stop_django +``` + +Then you need to start both geoserver and django services as follows: + +``` shell +$ paver start_geoserver + +$ paver start_django +``` + +Now you can visit your Geonode GUI by typing into your browser window + +# How to run GeoNode Project for development + +In order to run a project for development, the following steps have to be followed: + +1- Make sure there is no running instance of Geonode first by running the following command: + +``` shell +$ cd /home/user/my_geonode + +$ paver stop +``` + +The above command will stop all services related to Geonode if running + +2- Start the servers by running paver start as follows: + +``` shell +$ paver start +``` + +Now you can visit your geonode project site by typing into your browser window diff --git a/devel/workshops/img/geocollection-menu.png b/docs/devel/workshops/img/geocollection-menu.png similarity index 100% rename from devel/workshops/img/geocollection-menu.png rename to docs/devel/workshops/img/geocollection-menu.png diff --git a/devel/workshops/img/geocollections-admin.png b/docs/devel/workshops/img/geocollections-admin.png similarity index 100% rename from devel/workshops/img/geocollections-admin.png rename to docs/devel/workshops/img/geocollections-admin.png diff --git a/devel/workshops/img/geonode-is-awesome.png b/docs/devel/workshops/img/geonode-is-awesome.png similarity index 100% rename from devel/workshops/img/geonode-is-awesome.png rename to docs/devel/workshops/img/geonode-is-awesome.png diff --git a/devel/workshops/img/logged-in-geonode.png b/docs/devel/workshops/img/logged-in-geonode.png similarity index 100% rename from devel/workshops/img/logged-in-geonode.png rename to docs/devel/workshops/img/logged-in-geonode.png diff --git a/devel/workshops/img/red-background.png b/docs/devel/workshops/img/red-background.png similarity index 100% rename from devel/workshops/img/red-background.png rename to docs/devel/workshops/img/red-background.png diff --git a/devel/workshops/img/view-permission.png b/docs/devel/workshops/img/view-permission.png similarity index 100% rename from devel/workshops/img/view-permission.png rename to docs/devel/workshops/img/view-permission.png diff --git a/docs/devel/workshops/index.md b/docs/devel/workshops/index.md new file mode 100644 index 00000000..8f534328 --- /dev/null +++ b/docs/devel/workshops/index.md @@ -0,0 +1,903 @@ +# Workshops + +The workshops documentation demonstrates few examples on how to utilize GeoNode-Project in order to extend/customize GeoNode\'s functionalities according to your business. The covered topics include the following: + +1- Customize your GeoNode with the geonode-project + +2- Customize the look and feel + +3- Create your ResourceBase Metadata + +4- Create your own django app + +5- Add a custom model + +6- Permissions and APIs + +7- Deploy your GeoNode + +## 1- Customize your GeoNode with the geonode-project + +In this example, GeoNode-Project is cloned to create a template instance in which the rest of the examples will be building on top of it. + +1- Assuming you already installed GeoNode-Core, firstly we need to create a GeoNode-Project template and this can be achieved from the following command: + +``` shell +$ django-admin.py startproject my_geonode --template=https://github.com/GeoNode/geonode-project/archive/master.zip -e py,rst,json,yml,ini,env,sample -n Dockerfile +``` + +Here, django-admin is used with startproject option to create my_geonode project copying the template which is passed as GeoNode-project Github repo. It also includes \"py,rst,json,yml,ini,env,sample\" extensions + +2- Once the cloning finished, the next step is to install the GeoNode-Project we just downloaded as follows: + +``` shell +$ pip install -e my_geonode +``` + +3- Install geoserver using paver as follows + +``` shell +$ cd /home/geonode/my_geonode/src +$ paver setup +``` + +4- Note the GeoNode database connection parameters mentioned in the .env.sample.py file. Rename it to .env then use psql to create the required user and grant the required privileges as follows: + +``` shell +$ su postgres +$ createdb geonode +$ psql +postgres=# CREATE USER geonode WITH PASSWORD 'geonode'; +CREATE ROLE +postgres=# GRANT ALL PRIVILEGES ON DATABASE "geonode" to geonode; +GRANT +postgres=# \q +``` + +::: warning +::: title +Warning +::: + +Don\'t forget to exit from postgres user before executing the following commands +::: + +5- Run GeoNode using paver + +``` shell +$ cd /home/geonode/my_geonode/src +$ paver start +``` + +::: note +::: title +Note +::: + +You may find this warning message: You have 132 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): account, actstream, admin, agon_ratings, announcements, auth, avatar, base, contenttypes, dialogos, documents, favorite, geonode_client, geonode_themes, groups, guardian, invitations, layers, maps, mapstore2_adapter, monitoring, oauth2_provider, people, pinax_notifications, services, sessions, sites, socialaccount, taggit, tastypie, upload, user_messages. Which means you have some sql statements not executed yet and you need to run the \"migrate\" to sync your database first then \"paver start\" again as follows: +::: + +``` shell +$ python manage.py migrate +$ paver start +``` + +::: warning +::: title +Warning +::: + +If encountered this message: `(Invalid HTTP_HOST header: '0.0.0.0:8000'. You may need to add u'0.0.0.0' to ALLOWED_HOSTS)` It can be fixed in the settings.py file. You will need to add: `ALLOWED_HOSTS = ['0.0.0.0']` in settings.py +::: + +6- Once the previous step is done, you can visit 0.0.0.0:8000 to view the GUI of GeoNode. However, we still don\'t have an account in order to login from the GUI. This can be done using \"paver sync\". The command will create sync with latest fixtures and also creates a superuser \"admin\" with default password \"admin\" + +7- Use the created account to login from the GUI through localhost:8000 or 0.0.0.0:8000 + +![image](./img/logged-in-geonode.png) + +## 2- Customize the look and feel + +In this section we will change the look and feel of GeoNode, in particular we will do some customization to help understanding how the template inheritance works and how to add new stuff to your GeoNode. The changes will include the home page, the top menu, the footer and a generic GeoNode page. + +**Homepage:** + +The geonode-project provides some predefined templates to change the home page and the general site content. + +In the \"my_geonode/src/my_geonode/templates/geonode-mapstore-client/snippets\" directory we can files with similar names as the geonode-mapstore-client. This way we can override the different parts of the site eg the header, menu, body content and the footer. + +Create a file name hero.html and add the following. + +``` html +
+
+
+

My GeoNode

+

Example customization of geonode project

+
+

+

+
+
+``` + +![image](./img/geonode-is-awesome.png) + +**The theme:** + +To change the theme of our geonode-project we can act on the site_base.css file available in the \"my_geonode/my_geonode/static/css\" folder. + +The file is empty so we can inspect elements of the home page with the browser\'s developer tools and define css rules in there. + +For example, if we want to change the background of the jumbotron, in this file we can add + +``` css +.msgapi .gn-hero .jumbotron { background: red } +``` + +Then once we refreshed the browser, we should see the change as follows: + +![image](./img/red-background.png) + +**The top menu:** + +Now we can make some changes that will apply to the whole site. We can add an item to both the left and right side of the top menu bar. + +This can be done by creating a get_menu_json.py under templatetags folder to override GeoNodes default menu. + +``` python +@register.simple_tag(takes_context=True) +def get_base_right_topbar_menu(context): + + is_mobile = _is_mobile_device(context) + + if is_mobile: + return [] + + return [ + { + "type": "link", + "href": "/", + "label": "Custom 3" + }, + { + "type": "link", + "href": "/", + "label": "Custom 4" + }, + ] +@register.simple_tag(takes_context=True) +def get_base_left_topbar_menu(context): + + is_mobile = _is_mobile_device(context) + + return [ + { + "type": "link", + "href": "/", + "label": "Custom 1" + }, + { + "type": "link", + "href": "/", + "label": "Custom 2" + }, + ] +``` + +On browser refresh you will see a new entry in the nav bar which is persistent to the whole site. + +![image](./img/geocollection-menu.png) + +**Modify functionality** + +In this section, we will patch the ResourceBase of GeoNode and update the Templates in order to add one more field to the Metadata Schema. + +We will add a DOI field to the ResourceBase model and modify the Templates in order to show the new field both into the Metadata Wizard page. + +::: note +::: title +Note +::: + +Make sure to be inside \"my_geonode\" directory to execute the following commands +::: + +Customizing metadata can be achieved from the model which is defined in the core at \"geonode/geonode/base/models.py\" as follows: + +``` python +# internal fields +uuid = models.CharField(max_length=36) +owner = models.ForeignKey( + settings.AUTH_USER_MODEL, + blank=True, + null=True, + related_name='owned_resource', + verbose_name=_("Owner")) +contacts = models.ManyToManyField( + settings.AUTH_USER_MODEL, + through='ContactRole') +title = models.CharField(_('title'), max_length=255, help_text=_( + 'name by which the cited resource is known')) +alternate = models.CharField(max_length=128, null=True, blank=True) +date = models.DateTimeField( + _('date'), + default=now, + help_text=date_help_text) +date_type = models.CharField( + _('date type'), + max_length=255, + choices=VALID_DATE_TYPES, + default='publication', + help_text=date_type_help_text) +edition = models.CharField( + _('edition'), + max_length=255, + blank=True, + null=True, + help_text=edition_help_text) +abstract = models.TextField( + _('abstract'), + max_length=2000, + blank=True, + help_text=abstract_help_text) +purpose = models.TextField( + _('purpose'), + max_length=500, + null=True, + blank=True, + help_text=purpose_help_text) +maintenance_frequency = models.CharField( + _('maintenance frequency'), + max_length=255, + choices=UPDATE_FREQUENCIES, + blank=True, + null=True, + help_text=maintenance_frequency_help_text) +``` + +To add fields directly to the ResourceBase Class without actually modifying it, this can be done from \"my_geonode/src/my_geonode/apps.py\" file + +The \"ready\" method is invoked at initialization time and can be currently used to tweak your app in several ways + +``` python +class AppConfig(BaseAppConfig): + + name = "my_geonode" + label = "my_geonode" + + def ready(self): + super(AppConfig, self).ready() + run_setup_hooks() +``` + +Now we will add the \"patch_resource_base\" method to the AppConfig and execute it from the ready method as follows: + +``` python +from django.db import models +from django.utils.translation import ugettext_lazy as _ + + +class AppConfig(BaseAppConfig): + + name = "my_geonode" + label = "my_geonode" + + def _get_logger(self): + import logging + return logging.getLogger(self.__class__.__module__) + + def patch_resource_base(self, cls): + self._get_logger().info("Patching Resource Base") + doi_help_text = _('a DOI will be added by Admin before publication.') + doi = models.TextField( + _('DOI'), + blank=True, + null=True, + help_text=doi_help_text) + cls.add_to_class('doi', doi) + + def ready(self): + super(AppConfig, self).ready() + run_setup_hooks() + + from geonode.base.models import ResourceBase + self.patch_resource_base(ResourceBase) +``` + +::: note +::: title +Note +::: + +you will need to perform migrations as follows: - Add field doi to resourcebase +::: + +Once you run `python manage.py migrate`: + +``` shell +Running migrations: +Applying announcements.0002_auto_20200119_1257... OK +Applying base.0031_resourcebase_doi... OK +Applying people.0027_auto_20200119_1257... OK +``` + +Till now, we have patched the DB. however, it is not yet sufficient as we still need to display the added field. + +Let\'s extend the default templates so that we can show the newly added field + +## 3- Create your own django app + +In this section, we will demonstrate how to create and setup the skeleton of a custom app using the django facilities. The app will add a geocollections functionality to our GeoNode. + +The Geocollections app allows to present in a single page, resources and users grouped by a GeoNode Group. We can assign arbitrary resources to a Geocollection, a Group and a name that will be also used to build a dedicated URL. + +::: note +::: title +Note +::: + +Make sure to be inside \"my_geonode\" directory to execute the following commands +::: + +Create the django app + +Django gives us an handy command to create apps. We already used startproject to create our geonode-project, now we can use startapp to create the app. + +``` shell +python manage.py startapp geocollections +``` + +This will create a folder named geocollections that contains empty models and views. + +We need to add the new app to the INSTALLED_APPS of our project. inside \"my_geonode/src/my_geonode/settings.py\": + +``` python +INSTALLED_APPS += (PROJECT_NAME,) to be: INSTALLED_APPS += (PROJECT_NAME, 'geocollections',) +``` + +**Add a custom model** + +In this section, we will add a custom model and the related logic as follows: + +- Add a new model +- Add urls and views +- Add admin panel +- Add the template + +``` shell +vim geocollections/models.py +``` + +``` python +from django.db import models + +from geonode.base.models import ResourceBase +from geonode.groups.models import GroupProfile + + +class Geocollection(models.Model): + """ + A collection is a set of resources linked to a GeoNode group + """ + group = models.ForeignKey(GroupProfile, related_name='group_collections') + resources = models.ManyToManyField(ResourceBase, related_name='resource_collections') + name = models.CharField(max_length=128, unique=True) + slug = models.SlugField(max_length=128, unique=True) + + def __unicode__(self): + return self.name +``` + +At this point we need to ask django to create the database table. Django since version 1.8 has embedded migrations mechanism and we need to use them in order to change the state of the db. + +::: note +::: title +Note +::: + +Make sure to be inside \"my_geonode\" directory to execute the following commands +::: + +``` shell +python manage.py makemigrations + +# the above command informs you with the migrations to be executed on the database + +python manage.py migrate +``` + +Next we will use django generic view to show the collections detail. Add the following code in the views.py file: + +``` shell +vim geocollections/views.py +``` + +``` python +from django.views.generic import DetailView + +from .models import Geocollection + +class GeocollectionDetail(DetailView): + model = Geocollection +``` + +Add url configuration + +In order to access the created view we also need url mapping. We can create a urls.py file containing a url mapping to our generic view: + +``` shell +vim geocollections/urls.py +``` + +``` python +from django.conf.urls import url + +from .views import GeocollectionDetail + +urlpatterns = [ + url(r'^(?P[-\w]+)/$', + GeocollectionDetail.as_view(), + name='geocollection-detail'), +] +``` + +We also need to register the app urls in the project urls. So let\'s modify the \"my_geonode\" urls.py file adding the following: + +``` shell +vim my_geonode/src/my_geonode/urls.py +``` + +``` python +... +urlpatterns += [ +## include your urls here + url(r'^geocollections/', include('geocollections.urls')), +] +... +``` + +**Enable the admin panel** + +We need a user interface where we can create geocollections. Django makes this very easy, we just need the admin.py file as follows: + +``` shell +vim geocollections/admin.py +``` + +``` python +from django.contrib import admin + +from .models import Geocollection + + +class GeocollectionAdmin(admin.ModelAdmin): + prepopulated_fields = {"slug": ("name",)} + filter_horizontal = ('resources',) + +admin.site.register(Geocollection, GeocollectionAdmin) +``` + +Now we can visit the admin page and create a geocollection from there as follows: + +![image](./img/geocollections-admin.png) + +**Adding the template** + +Now we need the template where the geocollection detail will be rendered. Let\'s create a geocollections directory inside the \"my_geonode/templates\" directory with a file named geocollection_detail.html: + +``` shell +mkdir -p my_geonode/templates/geocollections/ + +vim my_geonode/templates/geocollections/geocollection_detail.html +``` + +``` python +{% extends "geonode_base.html" %} +{% block body %} +

Geocollection {{ object.name }}

+

Group: {{ object.group.title }}

+

Resources:

+
    + {% for resource in object.resources.all %} +
  • {{ resource.title }}
    • + {% endfor %} +
+{% endblock %} +``` + +To check the results, create a group in the geonode ui interface and load one or more layers/documents + +login into the admin panel -\> geocollections and create a geocollections + +Visit \ and view the results. + +Now that you know how to customize an html template, you can tune the page as you prefer. + +**Permissions and APIs** + +In this section we will add some more advanced logic like permissions and APIs. The permissions in GeoNode are managed with django-guardian, a library which allow to set object level permissions (django has table level authorization). + +The APIs are implemented through django-tastypie. + +The topics to be covered include: + +- Permissions on who can view the geocollection +- How to add templated and js to embed a permission ui in our geocollection detail page +- API to serve json serialized searchable endpoint + +Permissions logic (permissions objects) + +We need to add the permissions object to the database. We can do this by adding the following meta class to our Geocollection model, guardian will take care of creating the objects for us. + +``` shell +vim geocollections/models.py +``` + +``` python +class Meta: + permissions = ( + ('view_geocollection', 'Can view geocollection'), + ) +``` + +Then run \"python manage.py makemigrations\" and \"python manage.py migrate\" to install them + +**Permissions logic (set_default)** + +Let\'s add a method that will be used to set the default permissions on the Geocollections. We can add this logic to the Geocollection model but could also be a generic Mix-in similar to how it is implemented in GeoNode. + +``` shell +vim geocollections/models.py +``` + +``` python +from django.contrib.auth.models import Group +from django.contrib.auth import get_user_model +from django.contrib.contenttypes.models import ContentType +from django.conf import settings +from guardian.shortcuts import assign_perm + +def set_default_permissions(self): + """ + Set default permissions. + """ + + self.remove_object_permissions() + + # default permissions for anonymous users + anonymous_group, created = Group.objects.get_or_create(name='anonymous') + + if settings.DEFAULT_ANONYMOUS_VIEW_PERMISSION: + assign_perm('view_geocollection', anonymous_group, self) + + # default permissions for group members + assign_perm('view_geocollection', self.group, self) +``` + +**Permissions logic (methods)** + +Now we need a method to add generic permissions, we want to be able to assign view permissions to groups and single users. We can add this to our Geocollection model + +``` shell +vim geocollections/models.py +``` + +``` python +def set_permissions(self, perm_spec): + anonymous_group = Group.objects.get(name='anonymous') + self.remove_object_permissions() + if 'users' in perm_spec and "AnonymousUser" in perm_spec['users']: + assign_perm('view_geocollection', anonymous_group, self) + if 'users' in perm_spec: + for user, perms in perm_spec['users'].items(): + user = get_user_model().objects.get(username=user) + assign_perm('view_geocollection', user, self) + if 'groups' in perm_spec: + for group, perms in perm_spec['groups'].items(): + group = Group.objects.get(name=group) + assign_perm('view_geocollection', group, self) +def remove_object_permissions(self): + from guardian.models import UserObjectPermission, GroupObjectPermission + UserObjectPermission.objects.filter(content_type=ContentType.objects.get_for_model(self), + object_pk=self.id).delete() + GroupObjectPermission.objects.filter(content_type=ContentType.objects.get_for_model(self), + object_pk=self.id).delete() +``` + +Permissions logic (views.py) + +We can add now a view to receive and set our permissions, in views.py: + +``` shell +vim geocollections/views.py +``` + +``` python +import json +from django.core.exceptions import PermissionDenied +from django.http import HttpResponse +from django.contrib.auth import get_user_model + + +User = get_user_model() + +def geocollection_permissions(request, collection_id): + + collection = Geocollection.objects.get(id=collection_id) + user = User.objects.get(id=request.user.id) + + if user.has_perm('view_geocollection', collection): + return HttpResponse( + 'You have the permission to view. please customize a template for this view'', + content_type='text/plain') + + if request.method == 'POST': + success = True + message = "Permissions successfully updated!" + try: + permission_spec = json.loads(request.body) + collection.set_permissions(permission_spec) + + return HttpResponse( + json.dumps({'success': success, 'message': message}), + status=200, + content_type='text/plain' + ) + except: + success = False + message = "Error updating permissions :(" + return HttpResponse( + json.dumps({'success': success, 'message': message}), + status=500, + content_type='text/plain' + ) +``` + +**Permissions logic (url)** + +Lastly we need a url to map our client to our view, in urls.py + +``` shell +vim geocollections/urls.py +``` + +``` python +from django.conf.urls import url + +from .views import GeocollectionDetail, geocollection_permissions + +urlpatterns = [ + url(r'^(?P[-\w]+)/$', + GeocollectionDetail.as_view(), + name='geocollection-detail'), + + url(r'^permissions/(?P\d+)$', + geocollection_permissions, + name='geocollection_permissions'), +] +``` + +This url will be called with the id of the geocollection, the id will be passed to the view in order to get the permissions. + +![image](./img/view-permission.png) + +::: warning +::: title +Warning +::: + +A note on the client part, the server side logic is just one part necessary to implement permissions. + +A checklist of what is necessary: + +- A template snippet that can be embedded in the geocollection_detail.html, you can copy and simplify: \_permissions_form.html and \_permissions.html (in geonode/templates) +- A javascript file that will collect permissions settings and send them to the server, you can copy and simplify: \_permissions_form_js.html (in geonode/templates) +::: + +**API** + +The GeoNode API system easily allows to plug in new APIs. This section demonstrates the required steps: + +We need first to create an api.py file in our geocollection app. + +``` shell +vim geocollections/api.py +``` + +``` python +import json +from tastypie.resources import ModelResource +from tastypie import fields +from tastypie.constants import ALL_WITH_RELATIONS, ALL + +from geonode.api.api import ProfileResource, GroupResource +from geonode.api.resourcebase_api import ResourceBaseResource + +from .models import Geocollection +class GeocollectionResource(ModelResource): + + users = fields.ToManyField(ProfileResource, attribute=lambda bundle: bundle.obj.group.group.user_set.all(), full=True) + group = fields.ToOneField(GroupResource, 'group', full=True) + resources = fields.ToManyField(ResourceBaseResource, 'resources', full=True) + + class Meta: + queryset = Geocollection.objects.all().order_by('-group') + ordering = ['group'] + allowed_methods = ['get'] + resource_name = 'geocollections' + filtering = { + 'group': ALL_WITH_RELATIONS, + 'id': ALL + } +``` + +**API authorization** + +We want the API to respect our custom permissions, we can easily achieve this by adding the following to the beginning of api.py: + +``` shell +vim geocollections/api.py +``` + +``` python +from tastypie.authorization import DjangoAuthorization +from guardian.shortcuts import get_objects_for_user + +class GeocollectionAuth(DjangoAuthorization): + + def read_list(self, object_list, bundle): + permitted_ids = get_objects_for_user( + bundle.request.user, + 'geocollections.view_geocollection').values('id') + + return object_list.filter(id__in=permitted_ids) + + def read_detail(self, object_list, bundle): + return bundle.request.user.has_perm( + 'view_geocollection', + bundle.obj) +``` + +And this to the GeocollectionResource Meta class: + +``` python +authorization = GeocollectionAuth() +``` + +**Add a url for our API** + +In order to publish our API we need a url and we want that url to appear under the GeoNode\'s /api domain. + +The final url for our API has to be /api/geocollections. + +We can inject the url into the GeoNode API by adding the following lines to \"my_geonode/urls.py\" file: + +``` shell +vim my_geonode/urls.py +``` + +``` python +from geonode.api.urls import api + +from geocollections.api import GeocollectionResource + +api.register(GeocollectionResource()) +``` + +And add the following in the urlpatterns: + +``` python +url(r'', include(api.urls)), +``` + +The final result will be: + +``` python +from django.conf.urls import url, include +from django.views.generic import TemplateView + +from geonode.urls import urlpatterns + +from geonode.api.urls import api +from geocollections.api import GeocollectionResource + +api.register(GeocollectionResource()) + +urlpatterns += [ +## include your urls here +url(r'', include(api.urls)), +url(r'^geocollections/', include('geocollections.urls')), +] +``` + +Let\'s test permissions on API + +We can test the permissions on API by manually set a permission from the command line and check that the API respects it. + +With running `python manage.py shell` from inside our \"my_geonode\" folder, it opens a geonode shell. + +A perm spec could look like this: + +``` python +perms = { +'users': { +'AnonymousUser': ['view_geocollection'], +'alessio': ['view_geocollection']} +} +``` + +and we can assign the permissions with: + +``` python +from geocollections.models import Geocollection + +Geocollection.objects.first().set_permissions(perms) +``` + +our should now list the geocollection. + +If you remove the \'AnonymousUser\' line from perms and assign again the permissions it will disappear. + +``` python +perms = { +'users': { +'alessio': ['view_geocollection'] +} +``` + +**Deploy your GeoNode** + +So far we demonstrated how to modify, extend and style our GeoNode in dev mode but now it\'s time to go on production. In this section we will clarify how to: + +- commit your work on GitHub +- setup your server +- setup your GeoNode for production + +Push to GitHub +It is always a good practice to keep your code in a remote repository, GithHub is one of the options and is indeed the most used. + +It is assumed that you already have a GitHub account and that you have git installed and configured with your name and email. + +We will push only the my_geonode folder to GitHub and as we knew earlier, GeoNode for us is a dependency and we\'ll just reinstall it as it is on the server. + +Steps to push your code to GitHub: + +- Create an empty repository in GitHub and copy it\'s address +- In my_geonode, run git init to initialize an empty repository +- Add your remote repository address with `git remote add yourname yourremoteaddress` +- edit .gitignore adding all \*.pyc files +- `git add *` to add all content of my_geonode +- `git commit -m 'initial import'` to make the initial commit +- `git push yourname master` to push the code to the GitHub repository + +**Setup the server** + +There are several options for deploying GeoNode projects on servers. In this section, we explain how to deploy it on Ubuntu server 18.04 using system-wide installation + +::: note +::: title +Note +::: + +For quick installation, follow the INSTALLING documentation at +::: + +**Setup our my_geonode** + +We need now to install the developed \"my_geonode\" project following these steps: + +- git clone from your repository (in the folder of your preference) +- `sudo pip install -e my_geonode` +- edit the settings where needed +- edit `/etc/apache2/sites-enabled/geonode.conf` replacing the wsgi path to the `my_geonode/my_geonode/wsgi.py` file +- add the apache rights to the \"my_geonode\" folder with a directory like + +``` apache + + Order allow,deny + Require all granted + +``` + +- Test your server. diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 00000000..4d349d51 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,79 @@ +# GeoNode\'s Documentation {#mainindex} + +Welcome to GeoNode\'s Documentation. + +GeoNode is an Open Source, Content Management System (CMS) for geospatial data. It is a web-based application and platform for developing geospatial information systems (GIS) and for deploying spatial data infrastructures (SDI). + +## Table of contents + +::: {.toctree caption="About" maxdepth="3"} +about/index +::: + +::: {.toctree caption="Getting Started" maxdepth="3"} +start/index +::: + +::: {.toctree caption="Using" maxdepth="3"} +usage/index +::: + +::: {.toctree caption="Installing" maxdepth="3"} +install/index +::: + +::: {.toctree caption="Configuring (Basic)" maxdepth="3"} +basic/index +::: + +::: {.toctree caption="Configuring (Intermediate)" maxdepth="3"} +intermediate/index +::: + +::: {.toctree caption="Configuring (Advanced)" maxdepth="3"} +advanced/index +::: + +::: {.toctree caption="Administering" maxdepth="3"} +admin/index +::: + +::: {.toctree caption="Contributing" maxdepth="3"} +contribute/index +::: + +::: {.toctree caption="Developing" maxdepth="3"} +devel/index +::: + + +### About +- [index](about/index.md) + +### Getting Started +- [index](start/index.md) + +### Using +- [index](usage/index.md) + +### Installing +- [index](install/index.md) + +### Configuring (Basic) +- [index](basic/index.md) + +### Configuring (Intermediate) +- [index](intermediate/index.md) + +### Configuring (Advanced) +- [index](advanced/index.md) + +### Administering +- [index](admin/index.md) + +### Contributing +- [index](contribute/index.md) + +### Developing +- [index](devel/index.md) + diff --git a/docs/install/advanced/core/docker/connect_to_external_postgres_db.md b/docs/install/advanced/core/docker/connect_to_external_postgres_db.md new file mode 100644 index 00000000..e3375f45 --- /dev/null +++ b/docs/install/advanced/core/docker/connect_to_external_postgres_db.md @@ -0,0 +1,170 @@ +# Connecting GeoNode Docker to an External Database + +## 1. Make sure docker components are down + +``` sh +>_ docker-compose down +``` + +## 2. Disable GeoNode docker database and related commands + +- `@docker-compose.yml` comment out the `db` section + + ``` yml + # # PostGIS database. + # db: + # # use geonode official postgis 11 image + # image: geonode/postgis:latest + # container_name: db4${COMPOSE_PROJECT_NAME} + # env_file: + # - ./scripts/docker/env/${DOCKER_ENV}/db.env + # volumes: + # - dbdata:/var/lib/postgresql/data + # - dbbackups:/pg_backups + # restart: on-failure + # # uncomment to enable remote connections to postgres + # #ports: + # # - "5432:5432" + ``` + +- `@docker-compose.yml` comment out the related `volumes` section + + ``` yml + # dbdata: + # name: ${COMPOSE_PROJECT_NAME}-dbdata + # dbbackups: + # name: ${COMPOSE_PROJECT_NAME}-dbbackups + ``` + +- `@entrypoint.sh` comment out the related `"waitfordbs task done"` + section + + - Note 1: **It is expected to have the external database up running + before running, So no need to wait for the databases** + - Note 2: **Please check how the script for \`\`wait for databases\`\` + could be improved at the end on the file** + + ``` sh + # /usr/local/bin/invoke waitfordbs > /usr/src/geonode/invoke.log 2>&1 + # echo "waitfordbs task done" + ``` + +- Rebuild django docker service + + ``` sh + >_ docker-compose build + ``` + +## 3. As for the external PostgreSQL database it is expected to have the following parameters: + +- database user with password for connection +- database name for geonode django tables +- database name for geonode data tables +- exposed database IP address +- exposed database port +- Docker containers network IP address + +## 4. Prepare The External PostgreSQL Database: + +> Skip this part if it is already prepared + +- Create database user `` with `` + + ``` sh + >_ sudo -u postgres createuser -P + ``` + +- Create databases for `geonode django tables` with name `geonode` + + ``` sh + >_ sudo -u postgres createdb -O geonode + ``` + +- Create databases for `geonode django tables` with name + `geonode_data` + + ``` sh + >_ sudo -u postgres createdb -O geonode_data + ``` + +- Use `PostGIS` extension for the created databases \> Replace + `` with your username + + ``` sh + >_ sudo -u postgres psql -d geonode -c 'CREATE EXTENSION postgis;' + >_ sudo -u postgres psql -d geonode -c 'GRANT ALL ON geometry_columns TO PUBLIC;' + >_ sudo -u postgres psql -d geonode -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' + >_ sudo -u postgres psql -d geonode -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO ;' + + >_ sudo -u postgres psql -d geonode_data -c 'CREATE EXTENSION postgis;' + >_ sudo -u postgres psql -d geonode_data -c 'GRANT ALL ON geometry_columns TO PUBLIC;' + >_ sudo -u postgres psql -d geonode_data -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' + >_ sudo -u postgres psql -d geonode_data -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO ;' + ``` + +- Scenario 1: PostgreSQL service with docker service on the same host + machine + + - Get the geonode docker network IP address: + + ``` sh + >_ docker network inspect geonode_default | grep Subnet + # example output: + # "Subnet": "172.21.0.0/16", + ``` + +- Scenario 2: PostgreSQL is running on separate host machine: + + - Get the host machine IP address: + + ``` sh + >_ ifconfig + ``` + +- Now, Having the IP address, Add two rules for both databases at + `/etc/postgres//pg_hba.conf` + + ``` conf + # Syntax + # host DATABASE USER ADDRESS METHOD + # Ex + # host geonode_data geonode_user 172.21.0.0/16 md5 + host geonode /16 md5 + host geonode_data /16 md5 + ``` + +- Restart postgresql database service + + ``` sh + >_ sudo service postgresql restart + ``` + +- Allow GeoNode docker to connect to the external database: + + - At geonode project dir, edit `.env` file + + - Replace with your `username`, `password`, `ip_address`, + `port` + + - Replace database names if changed than `geonode & geonode_data` + + ``` conf + POSTGRES_USER= + POSTGRES_PASSWORD= + GEONODE_DATABASE=geonode + GEONODE_DATABASE_PASSWORD=geonode + GEONODE_GEODATABASE=geonode_data + GEONODE_GEODATABASE_PASSWORD=geonode_data + DATABASE_URL=postgis://:@:/geonode + GEODATABASE_URL=postgis://:@:/geonode_data + ``` + + - Start the docker containers + + > Expected to see the migrations running again + + ``` sh + >_ docker-compose up -d + ``` + + - **Done!** diff --git a/install/advanced/core/img/ubuntu-https-001.png b/docs/install/advanced/core/img/ubuntu-https-001.png similarity index 100% rename from install/advanced/core/img/ubuntu-https-001.png rename to docs/install/advanced/core/img/ubuntu-https-001.png diff --git a/install/advanced/core/img/ubuntu-https-002.png b/docs/install/advanced/core/img/ubuntu-https-002.png similarity index 100% rename from install/advanced/core/img/ubuntu-https-002.png rename to docs/install/advanced/core/img/ubuntu-https-002.png diff --git a/install/advanced/core/img/ubuntu-https-003.png b/docs/install/advanced/core/img/ubuntu-https-003.png similarity index 100% rename from install/advanced/core/img/ubuntu-https-003.png rename to docs/install/advanced/core/img/ubuntu-https-003.png diff --git a/install/advanced/core/img/ubuntu-https-004.png b/docs/install/advanced/core/img/ubuntu-https-004.png similarity index 100% rename from install/advanced/core/img/ubuntu-https-004.png rename to docs/install/advanced/core/img/ubuntu-https-004.png diff --git a/docs/install/advanced/core/index.md b/docs/install/advanced/core/index.md new file mode 100644 index 00000000..779a2723 --- /dev/null +++ b/docs/install/advanced/core/index.md @@ -0,0 +1,1616 @@ +# GeoNode Vanilla + +## Overview + +The following steps will guide you to a fresh setup of GeoNode. + +All guides will first install and configure the system to run it in `DEBUG` mode (also known as `DEVELOPMENT` mode) +and then by configuring an HTTPD server to serve GeoNode through the standard `HTTP` (`80`) port. + +::: warning +::: title +Warning +::: + +Those guides **are not** meant to be used on a production system. +There will be dedicated chapters that will show you some *hints* to optimize GeoNode for a production-ready machine. +In any case, we strongly suggest to task an experienced *DevOp* or *System Administrator* before exposing your server to the `WEB`. +::: + +## Ubuntu 22.04 LTS + +This part of the documentation describes the complete setup process for GeoNode on an Ubuntu 22.04.1LTS **64-bit** clean environment (Desktop or Server). + +All examples use shell commands that you must enter on a local terminal or a remote shell. + +- If you have a graphical desktop environment you can open the terminal application after login; +- if you are working on a remote server the provider or sysadmin should has given you access through an ssh client. + +### 1. Install the dependencies {#install_dep} + +In this section, we are going to install all the basic packages and tools needed for a complete GeoNode installation. + +::: warning +::: title +Warning +::: + +To follow this guide, a basic knowledge about Ubuntu Server configuration and working with a shell is required. +::: + +::: note +::: title +Note +::: + +This guide uses `vim` as the editor; fill free to use `nano`, `gedit` or others. +::: + +#### Upgrade system packages + +Check that your system is already up-to-date with the repository running the following commands: + +``` shell +sudo apt update -y +``` + +#### Packages Installation + +::: note +::: title +Note +::: + +You don\'t need to install the **system packages** if you want to run the project using Docker +::: + +We will use **example.org** as fictitious Domain Name. + +First, we are going to install all the **system packages** needed for the GeoNode setup. Login to the target machine and execute the following commands: + +``` shell +# Install packages from GeoNode core +sudo apt install -y --allow-downgrades build-essential \ + python3-gdal=3.4.1+dfsg-1build4 gdal-bin=3.4.1+dfsg-1build4 libgdal-dev=3.4.1+dfsg-1build4 \ + python3-all-dev python3.10-dev python3.10-venv virtualenvwrapper \ + libxml2 libxml2-dev gettext libmemcached-dev zlib1g-dev \ + libxslt1-dev libjpeg-dev libpng-dev libpq-dev \ + software-properties-common build-essential \ + git unzip gcc zlib1g-dev libgeos-dev libproj-dev \ + sqlite3 spatialite-bin libsqlite3-mod-spatialite libsqlite3-dev + +# Install Openjdk +sudo apt install openjdk-11-jdk-headless default-jdk-headless -y + +# Verify GDAL version +gdalinfo --version + $> GDAL 3.4.1, released 2021/12/27 + +# Verify Python version +python3.10 --version + $> Python 3.10.4 + +which python3.10 + $> /usr/bin/python3.10 + +# Verify Java version +java -version + $> openjdk version "11.0.16" + +# Install VIM +sudo apt install -y vim + +# Cleanup the packages +sudo apt update -y; sudo apt autoremove --purge +``` + +::: warning +::: title +Warning +::: + +GeoNode 4.4.x is not compatible with Python \< 3.7 +::: + +### 2. GeoNode Installation {#install_venv} + +This is the most basic installation of GeoNode. It won\'t use any external server like `Apache Tomcat`, `PostgreSQL` or `HTTPD`. + +First of all we need to prepare a new Python Virtual Environment + +Since geonode needs a large number of different python libraries and packages, its recommended to use a python virtual environment to avoid conflicts on dependencies with system wide python packages and other installed software. See also documentation of [Virtualenvwrapper](https://virtualenvwrapper.readthedocs.io/en/stable/) package for more information + +::: note +::: title +Note +::: + +The GeoNode Virtual Environment must be created only the first time. You won\'t need to create it again everytime. +::: + +``` shell +which python3.10 # copy the path of python executable + +# Create the GeoNode Virtual Environment (first time only) +export WORKON_HOME=~/.virtualenvs +source /usr/share/virtualenvwrapper/virtualenvwrapper.sh +mkvirtualenv --python=/usr/bin/python3.10 geonode # Use the python path from above + +# Alterantively you can also create the virtual env like below +mkdir -p ~/.virtualenvs +python3.10 -m venv ~/.virtualenvs/geonode +source ~/.virtualenvs/geonode/bin/activate +``` + +At this point your command prompt shows a `(geonode)` prefix, this indicates that your virtualenv is active. + +::: note +::: title +Note +::: + +The next time you need to access the Virtual Environment just run + +``` shell +source /usr/share/virtualenvwrapper/virtualenvwrapper.sh +workon geonode + +# Alterantively you can also create the virtual env like below +source ~/.virtualenvs/geonode/bin/activate +``` +::: + +::: note +::: title +Note +::: + +In order to save permanently the virtualenvwrapper environment + +``` shell +vim ~/.bashrc + +# Write to the bottom of the file the following lines +export WORKON_HOME=~/.virtualenvs +source /usr/share/virtualenvwrapper/virtualenvwrapper.sh +``` +::: + +``` shell +# Let's create the GeoNode core base folder and clone it +sudo mkdir -p /opt/geonode/; sudo usermod -a -G www-data $USER; sudo chown -Rf $USER:www-data /opt/geonode/; sudo chmod -Rf 775 /opt/geonode/ + +# Clone the GeoNode source code on /opt/geonode +cd /opt; git clone https://github.com/GeoNode/geonode.git -b 4.4.x geonode +``` + +``` shell +# Install the Python packages +cd /opt/geonode +pip install -r requirements.txt --upgrade +pip install -e . --upgrade +pip install pygdal=="`gdal-config --version`.*" +``` + +Edit `/opt/geonode/celery-cmd`. + +``` shell +CELERY__STATE_DB=${CELERY__STATE_DB:-"/opt/geonode/worker@%h.state"} +``` + +Edit `/opt/geonode/geonode/settings.py`. + +``` python +FILE_UPLOAD_DIRECTORY_PERMISSIONS = 0o777 +FILE_UPLOAD_PERMISSIONS = 0o777 +``` + +Edit `/opt/geonode/uwsgi.ini`. + +``` ini +chdir = /opt/geonode/ + +touch-reload = /opt/geonode/geonode/wsgi.py +``` + +### 3. Postgis database Setup + +::: warning +::: title +Warning +::: +::: + +Be sure you have successfully completed all the steps of the section `install_dep`{.interpreted-text role="ref"}. + +In this section, we are going to setup users and databases for GeoNode in PostgreSQL. + +#### Install and Configure the PostgreSQL Database System + +In this section we are going to install the `PostgreSQL` packages along with the `PostGIS` extension. Those steps must be done **only** if you don\'t have the DB already installed on your system. + +``` shell +# Ubuntu 22.04.1 (focal) +sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" >> /etc/apt/sources.list.d/pgdg.list' +sudo wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - +sudo apt update -y; sudo apt install -y postgresql-13 postgresql-13-postgis-3 postgresql-13-postgis-3-scripts postgresql-13 postgresql-client-13 +``` + +We now must create two databases, `geonode` and `geonode_data`, belonging to the role `geonode`. + +::: warning +::: title +Warning +::: + +This is our default configuration. +You can use any database or role you need. +The connection parameters must be correctly configured on `settings`, as we will see later in this section. +::: + +#### Databases and Permissions + +First, create the geonode user. GeoNode is going to use this user to access the database + +``` shell +sudo service postgresql start +sudo -u postgres createuser -P geonode + +# Use the password: geonode +``` + +You will be prompted asked to set a password for the user. **Enter geonode as password**. + +::: warning +::: title +Warning +::: + +This is a sample password used for the sake of simplicity. This password is very **weak** and should be changed in a production environment. +::: + +Create database `geonode` and `geonode_data` with owner `geonode` + +``` shell +sudo -u postgres createdb -O geonode geonode +sudo -u postgres createdb -O geonode geonode_data +``` + +Next let\'s create PostGIS extensions + +``` shell +sudo -u postgres psql -d geonode -c 'CREATE EXTENSION postgis;' +sudo -u postgres psql -d geonode -c 'GRANT ALL ON geometry_columns TO PUBLIC;' +sudo -u postgres psql -d geonode -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' +sudo -u postgres psql -d geonode -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO geonode;' +sudo -u postgres psql -d geonode -c 'GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO geonode;' + +sudo -u postgres psql -d geonode_data -c 'CREATE EXTENSION postgis;' +sudo -u postgres psql -d geonode_data -c 'GRANT ALL ON geometry_columns TO PUBLIC;' +sudo -u postgres psql -d geonode_data -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' +sudo -u postgres psql -d geonode_data -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO geonode;' +sudo -u postgres psql -d geonode_data -c 'GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO geonode;' +``` + +Final step is to change user access policies for local connections in the file `pg_hba.conf` + +``` shell +sudo vim /etc/postgresql/13/main/pg_hba.conf +``` + +Scroll down to the bottom of the document. We want to make local connection `trusted` for the default user. + +Make sure your configuration looks like the one below. + +``` shell +... +# DO NOT DISABLE! +# If you change this first entry you will need to make sure that the +# database superuser can access the database using some other method. +# Noninteractive access to all databases is required during automatic +# maintenance (custom daily cronjobs, replication, and similar tasks). +# +# Database administrative login by Unix domain socket +local all postgres trust + +# TYPE DATABASE USER ADDRESS METHOD + +# "local" is for Unix domain socket connections only +local all all md5 +# IPv4 local connections: +host all all 127.0.0.1/32 md5 +# IPv6 local connections: +host all all ::1/128 md5 +# Allow replication connections from localhost, by a user with the +# replication privilege. +local replication all peer +host replication all 127.0.0.1/32 md5 +host replication all ::1/128 md5 +``` + +::: warning +::: title +Warning +::: + +If your `PostgreSQL` database resides on a **separate/remote machine**, you\'ll have to **allow** remote access to the databases in the `/etc/postgresql/13/main/pg_hba.conf` to the `geonode` user and tell PostgreSQL to **accept** non-local connections in your `/etc/postgresql/13/main/postgresql.conf` file +::: + +Restart PostgreSQL to make the change effective. + +``` shell +sudo service postgresql restart +``` + +PostgreSQL is now ready. To test the configuration, try to connect to the `geonode` database as `geonode` role. + +``` shell +psql -U postgres geonode +# This should not ask for any password + +psql -U geonode geonode +# This should ask for the password geonode + +# Repeat the test with geonode_data DB +psql -U postgres geonode_data +psql -U geonode geonode_data +``` + +### 4. Install GeoServer + +In this section, we are going to install the `Apache Tomcat 8` Servlet Java container, which will be started by default on the internal port `8080`. + +We will also perform several optimizations to: + +1. Correctly setup the Java VM Options, like the available heap memory and the garbage collector options. +2. Externalize the `GeoServer` and `GeoWebcache` catalogs in order to allow further updates without the risk of deleting our datasets. + +::: note +::: title +Note +::: + +This is still a basic setup of those components. More details will be provided on sections of the documentation concerning the hardening of the system in a production environment. Nevertheless, you will need to tweak a bit those settings accordingly with your current system. As an instance, if your machine does not have enough memory, you will need to lower down the initial amount of available heap memory. **Warnings** and **notes** will be placed below the statements that will require your attention. +::: + +#### Install Apache Tomcat + +The reference version of Tomcat for the Geoserver for GeoNode is **Tomcat 9**. + +The following steps have been adapted from + +::: warning +::: title +Warning +::: + +Apache Tomcat 9 and Geoserver require Java 11 or newer to be installed on the server. +Check the steps before in order to be sure you have OpenJDK 11 correctly installed on your system. +::: + +First, it is not recommended to run Apache Tomcat as user root, so we will create a new system user which will run the Apache Tomcat server + +``` shell +sudo useradd -m -U -d /opt/tomcat -s /bin/bash tomcat +sudo usermod -a -G www-data tomcat +``` + +::: warning +::: title +Warning +::: + +Now, go to the official Apache Tomcat [website](https://tomcat.apache.org/) and download the most recent version of the software to your server. But don\'t use Tomcat10 because there are still some errors between Geoserver and Tomcat. +::: + +``` shell +VERSION=9.0.65; wget https://archive.apache.org/dist/tomcat/tomcat-9/v${VERSION}/bin/apache-tomcat-${VERSION}.tar.gz +``` + +Once the download is complete, extract the tar file to the /opt/tomcat directory: + +``` shell +sudo mkdir /opt/tomcat +sudo tar -xf apache-tomcat-${VERSION}.tar.gz -C /opt/tomcat/; rm apache-tomcat-${VERSION}.tar.gz +``` + +Apache Tomcat is updated regulary. So, to have more control over versions and updates, we'll create a symbolic link as below: + +``` shell +sudo ln -s /opt/tomcat/apache-tomcat-${VERSION} /opt/tomcat/latest +``` + +Now, let's change the ownership of all Apache Tomcat files as below: + +``` shell +sudo chown -R tomcat:www-data /opt/tomcat/ +``` + +Make the shell scripts inside the bin directory executable: + +``` shell +sudo sh -c 'chmod +x /opt/tomcat/latest/bin/*.sh' +``` + +Create the a systemd file with the following content: + +``` shell +# Check the correct JAVA_HOME location +JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::") +echo $JAVA_HOME + $> /usr/lib/jvm/java-11-openjdk-amd64/ + +# Let's create a symbolic link to the JDK +sudo ln -s /usr/lib/jvm/java-1.11.0-openjdk-amd64 /usr/lib/jvm/jre + +# Let's create the tomcat service +sudo vim /etc/systemd/system/tomcat9.service +``` + +``` bash +[Unit] +Description=Tomcat 9 servlet container +After=network.target + +[Service] +Type=forking + +User=tomcat +Group=tomcat + +Environment="JAVA_HOME=/usr/lib/jvm/jre" +Environment="JAVA_OPTS=-Djava.security.egd=file:///dev/urandom -Djava.awt.headless=true" + +Environment="CATALINA_BASE=/opt/tomcat/latest" +Environment="CATALINA_HOME=/opt/tomcat/latest" +Environment="CATALINA_PID=/opt/tomcat/latest/temp/tomcat.pid" +Environment="CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC" + +ExecStart=/opt/tomcat/latest/bin/startup.sh +ExecStop=/opt/tomcat/latest/bin/shutdown.sh + +[Install] +WantedBy=multi-user.target +``` + +Now you can start the Apache Tomcat 9 server and enable it to start on boot time using the following command: + +``` shell +sudo systemctl daemon-reload +sudo systemctl start tomcat9.service +sudo systemctl status tomcat9.service +sudo systemctl enable tomcat9.service +``` + +For verification, type the following ss command, which will show you the 8080 open port number, the default open port reserved for Apache Tomcat Server. + +``` shell +ss -ltn +``` + +In a clean Ubuntu 22.04.1, the ss command may not be found and the iproute2 library should be installed first. + +``` shell +sudo apt install iproute2 +# Then run the ss command +ss -ltn +``` + +In a clean Ubuntu 22.04.1, the ss command may not be found and the iproute2 library should be installed first. + +``` shell +sudo apt install iproute2 +# Then run the ss command +ss -ltn +``` + +If your server is protected by a firewall and you want to access Tomcat from the outside of your local network, you need to open port 8080. + +Use the following command to open the necessary port: + +``` shell +sudo ufw allow 8080/tcp +``` + +::: warning +::: title +Warning +::: + +Generally, when running Tomcat in a production environment, you should use a load balancer or reverse proxy. + +It's a best practice to allow access to port `8080` only from your internal network. + +We will use `NGINX` in order to provide Apache Tomcat through the standard `HTTP` port. +::: + +::: note +::: title +Note +::: + +Alternatively you can define the Tomcat Service as follow, in case you would like to use `systemctl` + +``` shell +sudo vim /usr/lib/systemd/system/tomcat9.service +``` + +``` ini +[Unit] +Description=Apache Tomcat Server +After=syslog.target network.target + +[Service] +Type=forking +User=tomcat +Group=tomcat + +Environment=JAVA_HOME=/usr/lib/jvm/jre +Environment=JAVA_OPTS=-Djava.security.egd=file:///dev/urandom +Environment=CATALINA_PID=/opt/tomcat/latest/temp/tomcat.pid +Environment=CATALINA_HOME=/opt/tomcat/latest +Environment=CATALINA_BASE=/opt/tomcat/latest + +ExecStart=/opt/tomcat/latest/bin/startup.sh +ExecStop=/opt/tomcat/latest/bin/shutdown.sh + +RestartSec=30 +Restart=always + +[Install] +WantedBy=multi-user.target +``` + +``` shell +sudo systemctl daemon-reload +sudo systemctl enable tomcat9.service +sudo systemctl start tomcat9.service +``` +::: + +#### Install GeoServer on Tomcat + +Let\'s externalize the `GEOSERVER_DATA_DIR` and `logs` + +``` shell +# Create the target folders +sudo mkdir -p /opt/data +sudo chown -Rf $USER:www-data /opt/data +sudo chmod -Rf 775 /opt/data +sudo mkdir -p /opt/data/logs +sudo chown -Rf $USER:www-data /opt/data/logs +sudo chmod -Rf 775 /opt/data/logs + +# Download and extract the default GEOSERVER_DATA_DIR +GS_VERSION=2.24.2 +sudo wget "https://artifacts.geonode.org/geoserver/$GS_VERSION/geonode-geoserver-ext-web-app-data.zip" -O data-$GS_VERSION.zip + +sudo unzip data-$GS_VERSION.zip -d /opt/data/ + +sudo mv /opt/data/data/ /opt/data/geoserver_data +sudo chown -Rf tomcat:www-data /opt/data/geoserver_data +sudo chmod -Rf 775 /opt/data/geoserver_data + +sudo mkdir -p /opt/data/geoserver_logs +sudo chown -Rf tomcat:www-data /opt/data/geoserver_logs +sudo chmod -Rf 775 /opt/data/geoserver_logs + +sudo mkdir -p /opt/data/gwc_cache_dir +sudo chown -Rf tomcat:www-data /opt/data/gwc_cache_dir +sudo chmod -Rf 775 /opt/data/gwc_cache_dir + +# Download and install GeoServer +sudo wget "https://artifacts.geonode.org/geoserver/$GS_VERSION/geoserver.war" -O geoserver-$GS_VERSION.war +sudo mv geoserver-$GS_VERSION.war /opt/tomcat/latest/webapps/geoserver.war +``` + +Let\'s now configure the `JAVA_OPTS`, i.e. the parameters to run the Servlet Container, like heap memory, garbage collector and so on. + +``` shell +sudo sed -i -e 's/xom-\*\.jar/xom-\*\.jar,bcprov\*\.jar/g' /opt/tomcat/latest/conf/catalina.properties + +export JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::") +echo 'JAVA_HOME='$JAVA_HOME | sudo tee --append /opt/tomcat/latest/bin/setenv.sh +sudo sed -i -e "s/JAVA_OPTS=/#JAVA_OPTS=/g" /opt/tomcat/latest/bin/setenv.sh + +echo 'GEOSERVER_DATA_DIR="/opt/data/geoserver_data"' | sudo tee --append /opt/tomcat/latest/bin/setenv.sh +echo 'GEOSERVER_LOG_LOCATION="/opt/data/geoserver_logs/geoserver.log"' | sudo tee --append /opt/tomcat/latest/bin/setenv.sh +echo 'GEOWEBCACHE_CACHE_DIR="/opt/data/gwc_cache_dir"' | sudo tee --append /opt/tomcat/latest/bin/setenv.sh +echo 'GEOFENCE_DIR="$GEOSERVER_DATA_DIR/geofence"' | sudo tee --append /opt/tomcat/latest/bin/setenv.sh +echo 'TIMEZONE="UTC"' | sudo tee --append /opt/tomcat/latest/bin/setenv.sh + +echo 'JAVA_OPTS="-server -Djava.awt.headless=true -Dorg.geotools.shapefile.datetime=false -DGS-SHAPEFILE-CHARSET=UTF-8 -XX:+UseParallelGC -XX:ParallelGCThreads=4 -Dfile.encoding=UTF8 -Duser.timezone=$TIMEZONE -Xms512m -Xmx4096m -Djavax.servlet.request.encoding=UTF-8 -Djavax.servlet.response.encoding=UTF-8 -DGEOSERVER_CSRF_DISABLED=true -DPRINT_BASE_URL=http://localhost:8080/geoserver/pdf -DGEOSERVER_DATA_DIR=$GEOSERVER_DATA_DIR -Dgeofence.dir=$GEOFENCE_DIR -DGEOSERVER_LOG_LOCATION=$GEOSERVER_LOG_LOCATION -DGEOWEBCACHE_CACHE_DIR=$GEOWEBCACHE_CACHE_DIR -Dgwc.context.suffix=gwc"' | sudo tee --append /opt/tomcat/latest/bin/setenv.sh +``` + +::: note +::: title +Note +::: + +After the execution of the above statements, you should be able to see the new options written at the bottom of the file `/opt/tomcat/latest/bin/setenv.sh`. + +``` shell +... +# If you run Tomcat on port numbers that are all higher than 1023, then you +# do not need authbind. It is used for binding Tomcat to lower port numbers. +# (yes/no, default: no) +#AUTHBIND=no +JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64/ +GEOSERVER_DATA_DIR="/opt/data/geoserver_data" +GEOSERVER_LOG_LOCATION="/opt/data/geoserver_logs/geoserver.log" +GEOWEBCACHE_CACHE_DIR="/opt/data/gwc_cache_dir" +GEOFENCE_DIR="$GEOSERVER_DATA_DIR/geofence" +TIMEZONE="UTC" +JAVA_OPTS="-server -Djava.awt.headless=true -Dorg.geotools.shapefile.datetime=false -DGS-SHAPEFILE-CHARSET=UTF-8 -XX:+UseParallelGC -XX:ParallelGCThreads=4 -Dfile.encoding=UTF8 -Duser.timezone=$TIMEZONE -Xms512m -Xmx4096m -Djavax.servlet.request.encoding=UTF-8 -Djavax.servlet.response.encoding=UTF-8 -DGEOSERVER_CSRF_DISABLED=true -DPRINT_BASE_URL=http://localhost:8080/geoserver/pdf -DGEOSERVER_DATA_DIR=$GEOSERVER_DATA_DIR -Dgeofence.dir=$GEOFENCE_DIR -DGEOSERVER_LOG_LOCATION=$GEOSERVER_LOG_LOCATION -DGEOWEBCACHE_CACHE_DIR=$GEOWEBCACHE_CACHE_DIR" +``` + +Those options could be updated or changed manually at any time, accordingly to your needs. +::: + +::: warning +::: title +Warning +::: + +The default options we are going to add to the Servlet Container, assume you can reserve at least `4GB` of `RAM` to `GeoServer` (see the option `-Xmx4096m`). You must be sure your machine has enough memory to run both `GeoServer` and `GeoNode`, which in this case means at least `4GB` for `GeoServer` plus at least `2GB` for `GeoNode`. A total of at least `6GB` of `RAM` available on your machine. If you don\'t have enough `RAM` available, you can lower down the values `-Xms512m -Xmx4096m`. Consider that with less `RAM` available, the performances of your services will be highly impacted. +::: + +``` shell +# Create the Logrotate config +sudo tee /etc/logrotate.d/geoserver < 1. Update `NGINX` configuration in order to serve the new domain name. +> +> ``` shell +> sudo vim /etc/nginx/sites-enabled/geonode +> +> # Update the 'server_name' directive +> server_name example.org www.example.org; +> +> # Restart the service +> sudo service nginx restart +> ``` +> +> 2. Update `.env` with the new domain name. +> +> ``` shell +> vim /opt/geonode/.env +> +> # Change everywhere 'localhost' to the new hostname +> :%s/localhost/www.example.org/g +> :wq +> +> # Restart the service +> sudo systemctl restart geonode-uwsgi +> ``` +> +> 3. Update `OAuth2` configuration in order to hit the new hostname. +> +> ``` shell +> workon geonode +> ``` +> +> sudo su +> +> : cd /opt/geonode +> +> \# Update the GeoNode ip or hostname +> PYTHONWARNINGS=ignore VIRTUAL_ENV=\$VIRTUAL_ENV DJANGO_SETTINGS_MODULE=geonode.settings GEONODE_ETC=/opt/geonode/geonode GEOSERVER_DATA_DIR=/opt/data/geoserver_data TOMCAT_SERVICE=\"service tomcat9\" APACHE_SERVICE=\"service nginx\" geonode_updateip -l localhost -p www.example.org +> +> exit +> +> 4. Update the existing `GeoNode` links in order to hit the new hostname. +> +> ``` shell +> workon geonode +> ``` +> +> \# To avoid spatialite conflict if using postgresql +> vim \$VIRTUAL_ENV/bin/postactivate +> +> \# Add these to make available. Change user, password and server information to yours +> export DATABASE_URL=\'postgresql://\:\@localhost:5432/geonode\' +> +> \# Close virtual environment and aopen it again to update variables +> deactivate +> +> workon geonode +> +> : cd /opt/geonode +> +> \# Update the GeoNode ip or hostname +> DJANGO_SETTINGS_MODULE=geonode.settings python manage.py migrate_baseurl \--source-address=http://localhost \--target-address=http://www.example.org + +::: note +::: title +Note +::: + +If at the end you get a \"bad gateway\" error when accessing your geonode site, check uwsgi log with `sudo tail -f /var/log/geonode.log` and if theres is an error related with port 5432 check the listening configuration from the postgresql server and allow the incoming traffic from geonode. +::: + +### 7. Install and enable HTTPS secured connection through the Let\'s Encrypt provider + +``` shell +# Install Let's Encrypt Certbot +# sudo add-apt-repository ppa:certbot/certbot # for ubuntu 18.04 and lower +sudo apt update -y; sudo apt install python3-certbot-nginx -y + +# Reload NGINX config and make sure the firewall denies access to HTTP +sudo systemctl reload nginx +sudo ufw allow 'Nginx Full' +sudo ufw delete allow 'Nginx HTTP' + +# Create and dump the Let's Encrypt Certificates +sudo certbot --nginx -d example.org -d www.example.org +# ...choose the redirect option when asked for +``` + +Next, the steps to do are: + +> 1. Update the `GeoNode` **OAuth2** `Redirect URIs` accordingly. +> +> From the `GeoNode Admin Dashboard` go to `Home › Django/GeoNode OAuth Toolkit › Applications › GeoServer` +> +>
+> img/ubuntu-https-001.png +>
Redirect URIs
+>
+> +> 2. Update the `GeoServer` `Proxy Base URL` accordingly. +> +> From the `GeoServer Admin GUI` go to `About & Status > Global` +> +>
+> img/ubuntu-https-002.png +>
Proxy Base URL
+>
+> +> 3. Update the `GeoServer` `Role Base URL` accordingly. +> +> From the `GeoServer Admin GUI` go to `Security > Users, Groups, Roles > geonode REST role service` +> +>
+> img/ubuntu-https-003.png +>
Role Base URL
+>
+> +> 4. Update the `GeoServer` `OAuth2 Service Parameters` accordingly. +> +> From the `GeoServer Admin GUI` go to `Security > Authentication > Authentication Filters > geonode-oauth2` +> +>
+> img/ubuntu-https-004.png +>
OAuth2 Service Parameters
+>
+> +> 5. Update the `.env` file +> +> ``` shell +> vim /opt/geonode/.env +> +> # Change everywhere 'http' to 'https' +> %s/http/https/g +> +> # Restart the service +> sudo systemctl restart geonode-uwsgi +> ``` + +### 8. Enabling Fully Asynchronous Tasks + +#### Install and configure [\"rabbitmq-server\"](https://lindevs.com/install-rabbitmq-on-ubuntu/) + +::: seealso +A [March 2021 blog post](https://blog.rabbitmq.com/posts/2021/03/migrate-off-of-bintray/) from RabbitMQ provides alternative installations for other systems. +::: + +**Install rabbitmq-server** + +Reference: [lindevs.com/install-rabbitmq-on-ubuntu/](https://lindevs.com/install-rabbitmq-on-ubuntu/) & [www.rabbitmq.com/install-debian.html/](https://www.rabbitmq.com/install-debian.html#apt-cloudsmith/) + +``` bash +sudo apt install curl -y + +## Import GPG Key +sudo apt update +sudo apt install curl software-properties-common apt-transport-https lsb-release +echo 'deb [signed-by=/etc/apt/keyrings/erlang.gpg] http://binaries2.erlang-solutions.com/ubuntu/ jammy-esl-erlang-26 contrib' | sudo tee /etc/apt/sources.list.d/erlang.list +curl -fsSL https://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc | sudo gpg --dearmor -o /etc/apt/keyrings/erlang.gpg + +## Add Erlang Repository to Ubuntu +sudo apt update +sudo apt install esl-erlang + +## Add RabbitMQ Repository to Ubuntu +curl -s https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/script.deb.sh | sudo bash + +## Install RabbitMQ Server +sudo apt install rabbitmq-server + +# check the status (it should already be running) +sudo systemctl status rabbitmq-server + +# check the service is enabled (it should already be enabled) +sudo systemctl is-enabled rabbitmq-server.service + +# enable the web frontend and allow access through firewall +# view this interface at http://:15672 +sudo rabbitmq-plugins enable rabbitmq_management +sudo ufw allow proto tcp from any to any port 5672,15672 +``` + +**Create admin user** + +This is the user that GeoNode will use to communicate with rabbitmq-server. + +``` +sudo rabbitmqctl delete_user guest +sudo rabbitmqctl add_user admin +sudo rabbitmqctl set_user_tags admin administrator +sudo rabbitmqctl add_vhost /localhost +sudo rabbitmqctl set_permissions -p / admin ".*" ".*" ".*" +sudo rabbitmqctl set_permissions -p /localhost admin ".*" ".*" ".*" +``` + +**Managing RabbitMQ** + +You can manage the rabbitmq-server service like any other service: + + sudo systemctl stop rabbitmq-server + sudo systemctl start rabbitmq-server + sudo systemctl restart rabbitmq-server + +You can manage the rabbitmq-server node with [rabbitmqctl](https://www.rabbitmq.com/rabbitmqctl.8.html). +For example, to fully reset the server, use these commands: + + sudo rabbitmqctl stop_app + sudo rabbitmqctl reset + sudo rabbitmqctl start_app + +After reset, you\'ll need to recreate the `admin` user (see above). + +Daemonize and configure Celery +\...\...\...\...\...\...\...\...\.... + +**Create the Systemd unit** + +``` shell +sudo vim /etc/systemd/system/celery.service +``` + +``` ini +[Unit] +Description=Celery +After=network.target + +[Service] +Type=simple +# the specific user that our service will run as +EnvironmentFile=/opt/geonode/.env +User=geosolutions +Group=geosolutions +# another option for an even more restricted service is +# DynamicUser=yes +# see http://0pointer.net/blog/dynamic-users-with-systemd.html +RuntimeDirectory=celery +WorkingDirectory=/opt/geonode +ExecStart=bash -c 'source /home/geosolutions/.virtualenvs/geonode/bin/activate && /opt/geonode/celery-cmd' +ExecReload=/bin/kill -s HUP $MAINPID +Restart=always +TimeoutSec=900 +TimeoutStopSec=60 +PrivateTmp=true + +[Install] +WantedBy=multi-user.target +``` + +``` shell +# Create the Logrotate config +sudo tee /etc/logrotate.d/celery <`{.interpreted-text role="ref"} to prepare a Ubuntu 22.04 server with Docker and Docker Compose + +### 1. Clone GeoNode + +``` shell +# Let's create the GeoNode core base folder and clone it +sudo mkdir -p /opt/geonode/ +sudo usermod -a -G www-data geonode +sudo chown -Rf geonode:www-data /opt/geonode/ +sudo chmod -Rf 775 /opt/geonode/ + +# Clone the GeoNode source code on /opt/geonode +cd /opt +git clone https://github.com/GeoNode/geonode.git +``` + +### 2. Prepare the .env file + +Follow the instructions at `Docker create env file`{.interpreted-text role="ref"} + +### 3. Build and run + +Follow the instructions at `Docker build and run`{.interpreted-text role="ref"} + +#### Test the instance and follow the logs + +If you run the containers daemonized (with the `-d` option), you can either run specific Docker commands to follow the `startup and initialization logs` or entering the image `shell` and check for the `GeoNode logs`. + +In order to follow the `startup and initialization logs`, you will need to run the following command from the repository folder + +``` shell +cd /opt/geonode +docker logs -f django4geonode +``` + +Alternatively: + +``` shell +cd /opt/geonode +docker-compose logs -f django +``` + +You should be able to see several initialization messages. Once the container is up and running, you will see the following statements + +``` shell +... +789 static files copied to '/mnt/volumes/statics/static'. +static data refreshed +Executing UWSGI server uwsgi --ini /usr/src/app/uwsgi.ini for Production +[uWSGI] getting INI configuration from /usr/src/app/uwsgi.ini +``` + +To exit just hit `CTRL+C`. + +This message means that the GeoNode containers have bee started. Browsing to `http://localhost/` will show the GeoNode home page. You should be able to successfully log with the default admin user (`admin` / `admin`) and start using it right away. + +With Docker it is also possible to run a shell in the container and follow the logs exactly the same as you deployed it on a physical host. To achieve this run + +``` shell +docker exec -it django4geonode /bin/bash + +# Once logged in the GeoNode image, follow the logs by executing +tail -F -n 300 /var/log/geonode.log +``` + +Alternatively: + +``` shell +docker-compose exec django /bin/bash +``` + +To exit just hit `CTRL+C` and `exit` to return to the host. + +#### Override the ENV variables to deploy on a public IP or domain + +If you would like to start the containers on a `public IP` or `domain`, let\'s say `www.example.org`, you can follow the instructions at `Deploy to production`{.interpreted-text role="ref"} + +ariables to customize the GeoNode instance. See the `GeoNode Settings` section in order to get a list of the available options. + +#### Remove all data and bring your running GeoNode deployment to the initial stage + +This procedure allows you to stop all the containers and reset all the data with the deletion of all the volumes. + +``` shell +cd /opt/geonode + +# stop containers and remove volumes +docker-compose down -v +``` + +#### Get rid of old Docker images and volumes (reset the environment completely) + +::: note +::: title +Note +::: + +For more details on Docker commands, please refer to the official Docker documentation. +::: + +It is possible to let docker show which containers are currently running (add `-a` for all containers, also stopped ones) + +``` shell +# Show the currently running containers +docker ps + +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +4729b3dd1de7 geonode/geonode:4.0 "/usr/src/geonode/en…" 29 minutes ago Up 5 minutes 8000/tcp celery4geonode +418da5579b1a geonode/geonode:4.0 "/usr/src/geonode/en…" 29 minutes ago Up 5 minutes (healthy) 8000/tcp django4geonode +d6b043f16526 geonode/letsencrypt:4.0 "./docker-entrypoint…" 29 minutes ago Up 9 seconds 80/tcp, 443/tcp letsencrypt4geonode +c77e1fa3ab2b geonode/geoserver:2.19.6 "/usr/local/tomcat/t…" 29 minutes ago Up 5 minutes (healthy) 8080/tcp geoserver4geonode +a971cedfd788 rabbitmq:3.7-alpine "docker-entrypoint.s…" 29 minutes ago Up 5 minutes 4369/tcp, 5671-5672/tcp, 25672/tcp rabbitmq4geonode +a2e4c69cb80f geonode/nginx:4.0 "/docker-entrypoint.…" 29 minutes ago Up 5 minutes 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443- >443/tcp, :::443->443/tcp nginx4geonode +d355d34cac4b geonode/postgis:13 "docker-entrypoint.s…" 29 minutes ago Up 5 minutes 5432/tcp db4geonode +``` + +Stop all the containers by running + +``` shell +docker-compose stop +``` + +Force kill all containers by running + +``` shell +docker kill $(docker ps -q) +``` + +I you want to clean up all containers and images, without deleting the static volumes (i.e. the `DB` and the `GeoServer catalog`), issue the following commands + +``` shell +# Remove all containers +docker rm $(docker ps -a -q) + +# Remove all docker images +docker rmi $(docker images -q) + +# Prune the old images +docker system prune -a +``` + +If you want to remove a `volume` also + +``` shell +# List of the running volumes +docker volume ls + +# Remove the GeoServer catalog by its name +docker volume rm -f geonode-gsdatadir + +# Remove all dangling docker volumes +docker volume rm $(docker volume ls -qf dangling=true) + +# update all images, should be run regularly to fetch published updates +for i in $(docker images| awk 'NR>1{print $1":"$2}'| grep -v ''); do docker pull "$i" ;done +``` diff --git a/docs/install/advanced/index.md b/docs/install/advanced/index.md new file mode 100644 index 00000000..1297052a --- /dev/null +++ b/docs/install/advanced/index.md @@ -0,0 +1,21 @@ +# GeoNode Advanced Installation + +::: {.toctree maxdepth="3"} +core/index +::: + +::: {.toctree maxdepth="3"} +project/index +::: + +::: {.toctree maxdepth="3"} +production_enhancements/index +::: + + +- [index](core/index.md) + +- [index](project/index.md) + +- [index](production_enhancements/index.md) + diff --git a/install/advanced/production_enhancements/img/geoserver_setup_001.png b/docs/install/advanced/production_enhancements/img/geoserver_setup_001.png similarity index 100% rename from install/advanced/production_enhancements/img/geoserver_setup_001.png rename to docs/install/advanced/production_enhancements/img/geoserver_setup_001.png diff --git a/install/advanced/production_enhancements/img/geoserver_setup_002.png b/docs/install/advanced/production_enhancements/img/geoserver_setup_002.png similarity index 100% rename from install/advanced/production_enhancements/img/geoserver_setup_002.png rename to docs/install/advanced/production_enhancements/img/geoserver_setup_002.png diff --git a/install/advanced/production_enhancements/img/geoserver_setup_003.png b/docs/install/advanced/production_enhancements/img/geoserver_setup_003.png similarity index 100% rename from install/advanced/production_enhancements/img/geoserver_setup_003.png rename to docs/install/advanced/production_enhancements/img/geoserver_setup_003.png diff --git a/install/advanced/production_enhancements/img/geoserver_setup_004.png b/docs/install/advanced/production_enhancements/img/geoserver_setup_004.png similarity index 100% rename from install/advanced/production_enhancements/img/geoserver_setup_004.png rename to docs/install/advanced/production_enhancements/img/geoserver_setup_004.png diff --git a/install/advanced/production_enhancements/img/production_geoserver_001.png b/docs/install/advanced/production_enhancements/img/production_geoserver_001.png similarity index 100% rename from install/advanced/production_enhancements/img/production_geoserver_001.png rename to docs/install/advanced/production_enhancements/img/production_geoserver_001.png diff --git a/install/advanced/production_enhancements/img/production_geoserver_002.png b/docs/install/advanced/production_enhancements/img/production_geoserver_002.png similarity index 100% rename from install/advanced/production_enhancements/img/production_geoserver_002.png rename to docs/install/advanced/production_enhancements/img/production_geoserver_002.png diff --git a/install/advanced/production_enhancements/img/production_geoserver_003.png b/docs/install/advanced/production_enhancements/img/production_geoserver_003.png similarity index 100% rename from install/advanced/production_enhancements/img/production_geoserver_003.png rename to docs/install/advanced/production_enhancements/img/production_geoserver_003.png diff --git a/install/advanced/production_enhancements/img/production_geoserver_004.png b/docs/install/advanced/production_enhancements/img/production_geoserver_004.png similarity index 100% rename from install/advanced/production_enhancements/img/production_geoserver_004.png rename to docs/install/advanced/production_enhancements/img/production_geoserver_004.png diff --git a/install/advanced/production_enhancements/img/production_geoserver_005.png b/docs/install/advanced/production_enhancements/img/production_geoserver_005.png similarity index 100% rename from install/advanced/production_enhancements/img/production_geoserver_005.png rename to docs/install/advanced/production_enhancements/img/production_geoserver_005.png diff --git a/install/advanced/production_enhancements/img/production_geoserver_006.png b/docs/install/advanced/production_enhancements/img/production_geoserver_006.png similarity index 100% rename from install/advanced/production_enhancements/img/production_geoserver_006.png rename to docs/install/advanced/production_enhancements/img/production_geoserver_006.png diff --git a/install/advanced/production_enhancements/img/production_geoserver_007.png b/docs/install/advanced/production_enhancements/img/production_geoserver_007.png similarity index 100% rename from install/advanced/production_enhancements/img/production_geoserver_007.png rename to docs/install/advanced/production_enhancements/img/production_geoserver_007.png diff --git a/install/advanced/production_enhancements/img/production_geoserver_008.png b/docs/install/advanced/production_enhancements/img/production_geoserver_008.png similarity index 100% rename from install/advanced/production_enhancements/img/production_geoserver_008.png rename to docs/install/advanced/production_enhancements/img/production_geoserver_008.png diff --git a/install/advanced/production_enhancements/img/production_geoserver_009.png b/docs/install/advanced/production_enhancements/img/production_geoserver_009.png similarity index 100% rename from install/advanced/production_enhancements/img/production_geoserver_009.png rename to docs/install/advanced/production_enhancements/img/production_geoserver_009.png diff --git a/install/advanced/production_enhancements/img/production_geoserver_010.png b/docs/install/advanced/production_enhancements/img/production_geoserver_010.png similarity index 100% rename from install/advanced/production_enhancements/img/production_geoserver_010.png rename to docs/install/advanced/production_enhancements/img/production_geoserver_010.png diff --git a/install/advanced/production_enhancements/img/throubleshooting_prod_001.png b/docs/install/advanced/production_enhancements/img/throubleshooting_prod_001.png similarity index 100% rename from install/advanced/production_enhancements/img/throubleshooting_prod_001.png rename to docs/install/advanced/production_enhancements/img/throubleshooting_prod_001.png diff --git a/docs/install/advanced/production_enhancements/index.md b/docs/install/advanced/production_enhancements/index.md new file mode 100644 index 00000000..74255200 --- /dev/null +++ b/docs/install/advanced/production_enhancements/index.md @@ -0,0 +1,332 @@ +# Further Production Enhancements {#geonode-production-enhancements} + +## GeoServer Production Settings + +### JVM Settings: Memory And GeoServer Options + +The `.env`{.interpreted-text role="guilabel"} file provides a way to customize GeoServer JVM Options. + +The variable `GEOSERVER_JAVA_OPTS` allows you to tune-up the GeoServer container and to enable specific GeoServer options. + +``` shell +GEOSERVER_JAVA_OPTS= + -Djava.awt.headless=true -Xms4G -Xmx4G -XX:PerfDataSamplingInterval=500 + -XX:SoftRefLRUPolicyMSPerMB=36000 -XX:-UseGCOverheadLimit -XX:+UseConcMarkSweepGC + -XX:+UseParNewGC -XX:ParallelGCThreads=4 -Dfile.encoding=UTF8 -Djavax.servlet.request.encoding=UTF-8 + -Djavax.servlet.response.encoding=UTF-8 -Duser.timezone=GMT + -Dorg.geotools.shapefile.datetime=false -DGS-SHAPEFILE-CHARSET=UTF-8 -DGEOSERVER_CSRF_DISABLED=true -DPRINT_BASE_URL=http://geoserver:8080/geoserver/pdf +``` + +`-Djava.awt.headless (true)` + +Work with graphics-based applications in Java without an actual display, keyboard, or mouse +A typical use case of UI components running in a headless environment could be an image converter app. Though it needs graphics data for image processing, a display is not really necessary. The app could be run on a server and converted files saved or sent over the network to another machine for display. + +`-Xms4G -Xmx4G` + +This means that your JVM will be started with Xms amount of memory and will be able to use a maximum of Xmx amount of memory. Above will start a JVM like with 2 GB of memory and will allow the process to use up to 4 GB of memory. You need to adjust this value depening on your availabnle RAM. + +`-DGEOSERVER_CSRF_DISABLED (True)` + +The GeoServer web admin employs a CSRF (Cross-Site Request Forgery) protection filter that will block any form submissions that didn't appear to originate from GeoServer. This can sometimes cause problems for certain proxy configurations. You can disable the CSRF filter by setting the GEOSERVER_CSRF_DISABLED property to true. + + +Whenever you need to change one or more of the JVM options, you will need to restart the GeoServer Docker container. + +``` shell +# Hard restart of the container: the only way to update the .env variables +docker-compose up -d geoserver +``` + +This command will **preserve** all the GeoServer configuration and data, since the `GEOSERVER_DATA_DIR` is stored on a Docker static volume. + +Nevertheless, any change you have made manually to the container, e.g. added a new plugin to GeoServer or updated some JARs into the `WEB-INF/lib` library folder, will be lost. + +You will need to add the JARs again and restart GeoServer *softly* + +``` shell +# Soft restart of the container: the .env variables won't be updated +docker-compose restart geoserver +``` + +### Global And Services Settings + +> - Check the GeoServer Memory usage and status; ensure the `GEOSERVER_DATA_DIR` path points to the static volume +> +>
+> img/production_geoserver_001.png +>
GeoServer Status
+>
+> +> - GeoServer `Global Settings`{.interpreted-text role="guilabel"}; make sure the `Proxy Base Url` points to the publlc URL and the `LOGGING` levels are set to `Production Mode`{.interpreted-text role="guilabel"} +> +>
+> img/production_geoserver_002.png +>
Global Settings
+>
+> +> - GeoServer `Image Processing Settings`{.interpreted-text role="guilabel"}; unless you are using some specific renderer or GeoServer plugin, use the following recommended options +> +> ::: note +> ::: title +> Note +> ::: +> +> Further details at +> ::: +> +>
+> img/production_geoserver_003.png +>
Image Processing Settings
+>
+> +> - Tune up `GeoServer Services Configuration`{.interpreted-text role="guilabel"}; `WCS`{.interpreted-text role="guilabel"}, `WFS`{.interpreted-text role="guilabel"}, `WMS`{.interpreted-text role="guilabel"} and `WPS`{.interpreted-text role="guilabel"}; +> +> > - **WCS**: Update the limits accordingly to your needs. Do not use very high values, this will set GeoServer prone to DoS Attacks. +> > +> >
+> > img/production_geoserver_004.png +> >
WCS Resource Consuption Limits
+> >
+> > +> > - **WMS**: Specify here the SRS List you are going to use. Empty means all the ones supported by GeoServer, but be carefull since the `GetCapabilities` output will become huge. +> > +> >
+> > img/production_geoserver_005.png +> >
WMS Supported SRS List
+> >
+> > +> > - **WMS**: `Raster Rendering Options`{.interpreted-text role="guilabel"} allows you to tune up the WMS output for better performance or quality. Best Performance: `Nearest Neighbour` - Best Quality: `Bicubic` +> > +> > ::: warning +> > ::: title +> > Warning +> > ::: +> > +> > Raster Images should be always optimized before ingested into GeoNode. The general recommendation is to **never** upload a non-processed GeoTIFF image to GeoNode. +> > +> > Further details at: +> > +> > - +> > - +> > ::: +> > +> >
+> > img/production_geoserver_006.png +> >
WMS Raster Rendering Options
+> >
+> > +> > - **WMS**: Update the limits accordingly to your needs. Do not use very high values, this will set GeoServer prone to DoS Attacks. +> > +> >
+> > img/production_geoserver_007.png +> >
WMS Resource Consuption Limits
+> >
+ +### GeoWebCache DiskQuota On Postgis + +By default GeoWebCache DiskQuota is disabled. That means that the layers cache might potentially grow up indefinitely. + +GeoWebCache DiskQuota should be always enabled on a production system. In the case it is enabled, this **must** be configured to make use of a DB engine like Postgis to store its indexes. + +> - First of all ensure `Tile Caching`{.interpreted-text role="guilabel"} is enabled on all available layers +> +> ::: note +> ::: title +> Note +> ::: +> +> GeoNode tipically does this automatically for you. It is worth to double check anyway. +> ::: +> +>
+> img/production_geoserver_008.png +>
Tile Caching: Tiled Datasets
+>
+> +> - Configure `Disk Quota`{.interpreted-text role="guilabel"} by providing the connection string to the DB Docker Container as specified in the `.env`{.interpreted-text role="guilabel"} file +> +>
+> img/production_geoserver_009.png +>
Tile Caching: Disk Quota Configuration
+>
+ +### GeoFence Security Rules On Postgis + +By default GeoFence stores the security rules on an `H2`{.interpreted-text role="guilabel"} db. + +On a production system, this is not really recommended. You will need to update the GeoServer Docker container in order to enable GeoFence storing the rules into the DB Docker Container instead. + +In order to do that, follow the procedure below: + +``` shell +# Enter the GeoServer Docker Container +docker-compose exec geoserver bash + +# Install a suitable editor +apt update +apt install nano + +# Edit the GeoFence DataStore .properties file +nano /geoserver_data/data/geofence/geofence-datasource-ovr.properties +``` + +::: note +::: title +Note +::: + +Make sure to provide the same connection parameters specified in the `.env`{.interpreted-text role="guilabel"} file +::: + +``` ini +geofenceVendorAdapter.databasePlatform=org.hibernatespatial.postgis.PostgisDialect +geofenceDataSource.driverClassName=org.postgresql.Driver +geofenceDataSource.url=jdbc:postgresql://db:5432/my_geonode_data +geofenceDataSource.username=my_geonode_data +geofenceDataSource.password=******** +geofenceEntityManagerFactory.jpaPropertyMap[hibernate.default_schema]=public +``` + +``` shell +# Update the GeoServer WEB-INF/lib JARs accordingly +wget --no-check-certificate "https://repo1.maven.org/maven2/org/postgis/postgis-jdbc/1.3.3/postgis-jdbc-1.3.3.jar" -O postgis-jdbc-1.3.3.jar && \ +wget --no-check-certificate "https://maven.geo-solutions.it/org/hibernatespatial/hibernate-spatial-postgis/1.1.3.2/hibernate-spatial-postgis-1.1.3.2.jar" -O hibernate-spatial-postgis-1.1.3.2.jar && \ +rm /usr/local/tomcat/webapps/geoserver/WEB-INF/lib/hibernate-spatial-h2-geodb-1.1.3.1.jar && \ +mv hibernate-spatial-postgis-1.1.3.2.jar /usr/local/tomcat/webapps/geoserver/WEB-INF/lib/ && \ +mv postgis-jdbc-1.3.3.jar /usr/local/tomcat/webapps/geoserver/WEB-INF/lib/ +``` + +The container is ready to be restarted now. + +::: warning +::: title +Warning +::: + +Remember to do a **soft restart** otherwise the WEB-INF/lib JARs will be reset to the original state +::: + +``` shell +# Exit the GeoServer container +exit + +# Soft Restart GeoServer Docker Container +docker-compose restart geoserver +``` + +**IMPORTANT**: The first time you perform this procedure, GeoFence won\'t be able to retrieve the old security rules anymore. + +You will need to `fixup_geonode_layers_permissions`{.interpreted-text role="ref"} in order to regenerate the security rules. + +## Fixup GeoNode Datasets Permissions {#fixup_geonode_layers_permissions} + +The list of the GeoFence Security Rules is available from the `GeoFence Data Rules`{.interpreted-text role="guilabel"} section. + +Always double check the list is accessible and the data rules are there. If empty, no layer will be accessible by standard users other than admin. + +
+img/production_geoserver_010.png +
GeoFence Data Rules
+
+ +In order to re-sync the GeoFence security rules, follow the procedure below: + +``` shell +# Enter the GeoNode Docker Container +docker-compose exec django bash + +# Run the `sync_geonode_datasets` management command +./manage.sh sync_geonode_datasets --updatepermissions +``` + +## Regenerate GeoNode Datasets Thumbnails + +The following procedure allows you to *batch* regenerate all Datasets Thumbnails: + +``` shell +# Enter the GeoNode Docker Container +docker-compose exec django bash + +# Run the `sync_geonode_datasets` management command +./manage.sh sync_geonode_datasets --updatethumbnails +``` + +## Regenerate GeoNode Datasets BBOXES + +The following procedure allows you to *batch* regenerate all Datasets BBOXES: + +``` shell +# Enter the GeoNode Docker Container +docker-compose exec django bash + +# Run the `sync_geonode_datasets` management command +./manage.sh sync_geonode_datasets --updatebbox +``` + +## Fixup GeoNode Datasets Metadata And Download Links + +The following procedure allows you to fix-up broken or incorrect Metadata Links: + +``` shell +# Enter the GeoNode Docker Container +docker-compose exec django bash + +# Run the `set_all_datasets_metadata` management command +./manage.sh set_all_datasets_metadata -d +``` + +It is also possible to *force* purging the links before regenerating: + +``` shell +# Enter the GeoNode Docker Container +docker-compose exec django bash + +# Run the `set_all_datasets_metadata` management command +./manage.sh set_all_datasets_metadata -d --prune +``` + +## Migrate GeoNode To A New Hostname + +In the case you will need to move your instance to another domain, as an example from `https://my_geonode.geonode.org/` to `https://prod_geonode.geonode.org/`, follow the procedure below: + +- Update the `.env`{.interpreted-text role="guilabel"} file by specifyig the new name accordingly. + +- Restart the GeoNode Docker Container. + + > ``` shell + > docker-compose up -d geonode + > ``` + +- Run the following management commands from inside the GeoNode Docker Container. + + > ``` shell + > # Enter the GeoNode Docker Container + > docker-compose exec django bash + > + > # Run the `migrate_baseurl` management command + > ./manage.sh migrate_baseurl --source-address=my_geonode.geonode.org --target-address=prod_geonode.geonode.org + > + > # Run the `set_all_datasets_metadata` management command + > ./manage.sh set_all_datasets_metadata -d + > ``` + +## Add Huge Or DB Datasets To Your Instance + +Uploaing huge datasets, or DB tables, to GeoNode from the `Web Upload Interface`{.interpreted-text role="guilabel"} is not really possible sometimes. + +The suggested procedure in such cases is the following one: + +- Add the dataset to `GeoServer`{.interpreted-text role="guilabel"} first directly. + + > You must upload the data into the GeoServer Docker Container Static Volume first and then adding manually the layer throught the `GeoServer Admin GUI`{.interpreted-text role="guilabel"}. + +- Once the dataset is correctly configured on GeoServer, run the following management command from inside the GeoNode Docker Container + + > ``` shell + > # Enter the GeoNode Docker Container + > docker-compose exec django bash + > + > # Run the `updatelayers` management command + > ./manage.sh updatelayers -w -f + > ``` diff --git a/docs/install/advanced/project/index.md b/docs/install/advanced/project/index.md new file mode 100644 index 00000000..0db8f347 --- /dev/null +++ b/docs/install/advanced/project/index.md @@ -0,0 +1,590 @@ +# GeoNode Project + +## Overview + +The following steps will guide you to a new setup of GeoNode Project. All guides will first install and configure the system to run it in `DEBUG` mode (also known as `DEVELOPMENT` mode) and then by configuring an HTTPD server to serve GeoNode through the standard `HTTP` (`80`) port. + +Those guides **are not** meant to be used on a production system. There will be dedicated chapters that will show you some *hints* to optimize GeoNode for a production-ready machine. In any case, we strongly suggest to task an experienced *DevOp* or *System Administrator* before exposing your server to the `WEB`. + +## Ubuntu 22.04 + +This part of the documentation describes the complete setup process for GeoNode on an Ubuntu 22.04 64-bit clean environment (Desktop or Server). All examples use shell commands that you must enter on a local terminal or a remote shell. +- If you have a graphical desktop environment you can open the terminal application after login; +- if you are working on a remote server the provider or sysadmin should has given you access through an ssh client. + +### Install the dependencies {#install_dep_proj} + +In this section, we are going to install all the basic packages and tools needed for a complete GeoNode installation. To follow this guide, a piece of basic knowledge about Ubuntu Server configuration and working with a shell is required. This guide uses `vim` as the editor; fill free to use `nano`, `gedit` or others. + +#### Upgrade system packages + +Check that your system is already up-to-date with the repository running the following commands: + +``` shell +sudo apt update +``` + +#### Create a Dedicated User + +In the following steps a User named `geonode` is used: to run installation commands the user must be in the `sudo` group. + +Create User `geonode` **if not present**: + +``` shell +# Follow the prompts to set the new user's information. +# It is fine to accept the defaults to leave all of this information blank. +sudo adduser geonode + +# The following command adds the user geonode to group sudo +sudo usermod -aG sudo geonode + +# make sure the newly created user is allowed to login by ssh +# (out of the scope of this documentation) and switch to User geonode +su geonode +``` + +#### Packages Installation + +Add the Ubuntu GIS packages prior to installing the other system packages. + +``` shell +sudo apt update +``` + +::: note +::: title +Note +::: + +You don\'t need to install the **system packages** if you want to run the project using Docker +::: + +First, we are going to install all the **system packages** needed for the GeoNode setup. + +``` shell +# Install packages from GeoNode core +sudo apt install -y python3-gdal=3.3.2+dfsg-2~focal2 gdal-bin=3.3.2+dfsg-2~focal2 libgdal-dev=3.3.2+dfsg-2~focal2 +sudo apt install -y python3-pip python3-dev python3-virtualenv python3-venv virtualenvwrapper +sudo apt install -y libxml2 libxml2-dev gettext +sudo apt install -y libxslt1-dev libjpeg-dev libpng-dev libpq-dev libmemcached-dev +sudo apt install -y software-properties-common build-essential +sudo apt install -y git unzip gcc zlib1g-dev libgeos-dev libproj-dev +sudo apt install -y sqlite3 spatialite-bin libsqlite3-mod-spatialite + +# Install Openjdk +sudo -i apt update +sudo apt install openjdk-8-jdk-headless default-jdk-headless -y + +# Remember to select the correct java version /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java +sudo update-alternatives --config java + +sudo apt update -y +sudo apt autoremove -y +sudo apt autoclean -y +sudo apt purge -y +sudo apt clean -y + +# Install Packages for Virtual environment management +sudo apt install -y virtualenv virtualenvwrapper + +# Install text editor +sudo apt install -y vim +``` + +### Geonode Project Installation + +Geonode project is the proper way to run a customized installation of Geonode. The repository of geonode-project contains a minimal set of files following the structure of a django-project. Geonode itself will be installed as a requirement of your project. +Inside the project structure is possible to extend, replace or modify all geonode components (e.g. css and other static files, templates, models..) and even register new django apps **without touching the original Geonode code**. + +::: note +::: title +Note +::: + +You can call your geonode project whatever you like following the naming conventions for python packages (generally lower case with underscores (\_). In the examples below, replace `my_geonode` with whatever you would like to name your project. +::: + +See also the [README](https://github.com/GeoNode/geonode-project/blob/master/README.md) file on geonode-project repository + +First of all we need to prepare a new Python Virtual Environment + +Prepare the environment + +``` shell +sudo mkdir -p /opt/geonode_custom/ +sudo usermod -a -G www-data geonode +sudo chown -Rf geonode:www-data /opt/geonode_custom/ +sudo chmod -Rf 775 /opt/geonode_custom/ +``` + +Clone the source code + +``` shell +cd /opt/geonode_custom/ +git clone https://github.com/GeoNode/geonode-project.git +``` + +This will clone the `master` branch. You will have to checkout the desidered branch or tag. +As an example, if you want to generate a propject for GeoNode 4.2.2 you will docker + +``` shell +git checkout -b 4.2.2 +``` + +Make an instance out of the `Django Template` + +::: note +::: title +Note +::: + +We will call our instance `my_geonode`. You can change the name at your convenience. +::: + +``` shell +vim ~/.bashrc +# add the following line to the bottom +source /usr/share/virtualenvwrapper/virtualenvwrapper.sh +``` + +``` shell +source /usr/share/virtualenvwrapper/virtualenvwrapper.sh +mkvirtualenv --python=/usr/bin/python3 my_geonode + +Alterantively you can also create the virtual env like below +python3.8 -m venv /home/geonode/dev/.venvs/my_geonode +source /home/geonode/dev/.venvs/my_geonode/bin/activate + +pip install Django==3.2.13 + +django-admin startproject --template=./geonode-project -e py,sh,md,rst,json,yml,ini,env,sample,properties -n monitoring-cron -n Dockerfile my_geonode + +# Install the Python packages +cd /opt/geonode_custom/my_geonode +pip install -r src/requirements.txt --upgrade +pip install -e src/ --upgrade + +# Install GDAL Utilities for Python +pip install pygdal=="`gdal-config --version`.*" + +# Dev scripts +mv .override_dev_env.sample src/.override_dev_env +mv src/manage_dev.sh.sample src/manage_dev.sh +mv src/paver_dev.sh.sample src/paver_dev.sh +``` + +#### Install and Configure the PostgreSQL Database System + +In this section we are going to install the `PostgreSQL` packages along with the `PostGIS` extension. Those steps must be done **only** if you don\'t have the DB already installed on your system. + +``` shell +# Ubuntu 22.04 +sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" >> /etc/apt/sources.list.d/pgdg.list' +sudo wget --no-check-certificate --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - +sudo apt update -y; sudo apt install -y postgresql-13 postgresql-13-postgis-3 postgresql-13-postgis-3-scripts postgresql-13 postgresql-client-13 +``` + +We now must create two databases, `my_geonode` and `my_geonode_data`, belonging to the role `my_geonode`. + +::: warning +::: title +Warning +::: + +This is our default configuration. +You can use any database or role you need. +The connection parameters must be correctly configured on `settings`, as we will see later in this section. +::: + +#### Databases and Permissions + +First, create the geonode user. GeoNode is going to use this user to access the database + +``` shell +sudo service postgresql start +sudo -u postgres createuser -P my_geonode + +# Use the password: geonode +``` + +You will be prompted asked to set a password for the user. **Enter geonode as password**. + +::: warning +::: title +Warning +::: + +This is a sample password used for the sake of simplicity. This password is very **weak** and should be changed in a production environment. +::: + +Create database `my_geonode` and `my_geonode_data` with owner `my_geonode` + +``` shell +sudo -u postgres createdb -O my_geonode my_geonode +sudo -u postgres createdb -O my_geonode my_geonode_data +``` + +Next let\'s create PostGIS extensions + +``` shell +sudo -u postgres psql -d my_geonode -c 'CREATE EXTENSION postgis;' +sudo -u postgres psql -d my_geonode -c 'GRANT ALL ON geometry_columns TO PUBLIC;' +sudo -u postgres psql -d my_geonode -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' +sudo -u postgres psql -d my_geonode -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO my_geonode;' + +sudo -u postgres psql -d my_geonode_data -c 'CREATE EXTENSION postgis;' +sudo -u postgres psql -d my_geonode_data -c 'GRANT ALL ON geometry_columns TO PUBLIC;' +sudo -u postgres psql -d my_geonode_data -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' +sudo -u postgres psql -d my_geonode_data -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO my_geonode;' +``` + +Final step is to change user access policies for local connections in the file `pg_hba.conf` + +``` shell +sudo vim /etc/postgresql/13/main/pg_hba.conf +``` + +Scroll down to the bottom of the document. We want to make local connection `trusted` for the default user. + +Make sure your configuration looks like the one below. + +``` shell +... +# DO NOT DISABLE! +# If you change this first entry you will need to make sure that the +# database superuser can access the database using some other method. +# Noninteractive access to all databases is required during automatic +# maintenance (custom daily cronjobs, replication, and similar tasks). +# +# Database administrative login by Unix domain socket +local all postgres trust + +# TYPE DATABASE USER ADDRESS METHOD + +# "local" is for Unix domain socket connections only +local all all md5 +# IPv4 local connections: +host all all 127.0.0.1/32 md5 +# IPv6 local connections: +host all all ::1/128 md5 +# Allow replication connections from localhost, by a user with the +# replication privilege. +local replication all peer +host replication all 127.0.0.1/32 md5 +host replication all ::1/128 md5 +``` + +::: warning +::: title +Warning +::: + +If your `PostgreSQL` database resides on a **separate/remote machine**, you\'ll have to **allow** remote access to the databases in the `/etc/postgresql/13/main/pg_hba.conf` to the `geonode` user and tell PostgreSQL to **accept** non-local connections in your `/etc/postgresql/13/main/postgresql.conf` file +::: + +Restart PostgreSQL to make the change effective. + +``` shell +sudo service postgresql restart +``` + +PostgreSQL is now ready. To test the configuration, try to connect to the `geonode` database as `geonode` role. + +``` shell +psql -U postgres my_geonode +# This should not ask for any password + +psql -U my_geonode my_geonode +# This should ask for the password geonode + +# Repeat the test with geonode_data DB +psql -U postgres my_geonode_data +psql -U my_geonode my_geonode_data +``` + +### Run GeoNode Project for the first time in DEBUG Mode + +::: warning +::: title +Warning +::: + +Be sure you have successfully completed all the steps of the section `install_dep_proj`{.interpreted-text role="ref"}. +::: + +This command will run both GeoNode and GeoServer locally after having prepared the Spatialite database. The server will start in `DEBUG` (or `DEVELOPMENT`) mode, and it will start the following services: + +1. GeoNode on `http://localhost:8000/` +2. GeoServer on `http://localhost:8080/geoserver/` + +This modality is beneficial to debug issues and/or develop new features, but it cannot be used on a production system. + +``` shell +# Prepare the GeoNode Spatialite database (the first time only) +cd src/ +chmod +x paver_dev.sh +./paver_dev.sh setup +./paver_dev.sh sync +``` + +::: note +::: title +Note +::: + +In case you want to start again from a clean situation, just run + +``` shell +./paver_dev.sh reset_hard +``` +::: + +::: warning +::: title +Warning +::: + +This will blow up completely your `local_settings`, delete the SQLlite database and remove the GeoServer data dir. +::: + +``` shell +# Run the server in DEBUG mode +./paver_dev.sh start +``` + +Once the server has finished the initialization and prints on the console the sentence `GeoNode is now available.`, you can open a browser and go to: + + http://localhost:8000/ + +Sign-in with: + + user: admin + password: admin + +From now on, everything already said for GeoNode Core (please refer to the section `configure_dbs_core`{.interpreted-text role="ref"} and following), applies to a +GeoNode Project. + +**Be careful** to use the **new** paths and names everywhere: + +- Everytime you\'ll find the keyword `geonode`, you\'ll need to use your geonode custom name instead (in this example `my_geonode`). +- Everytime you\'ll find paths pointing to `/opt/geonode/`, you\'ll need to update them to point to your custom project instead (in this example `/opt/geonode_custom/my_geonode`). + +## Docker + +::: warning +::: title +Warning +::: + +Before moving with this section, you should have read and clearly understood the `INSTALLATION > GeoNode Core` sections, and in particular the `Docker` one. Everything said for the GeoNode Core Vanilla applies here too, except that the Docker container names will be slightly different. As an instance if you named your project `my_geonode`, your containers will be called: + +``` shell +'django4my_geonode' instead of 'django4geonode' and so on... +``` +::: + +### Deploy an instance of a geonode-project Django template with Docker on localhost + +Prepare the environment + +``` shell +sudo mkdir -p /opt/geonode_custom/ +sudo usermod -a -G www-data geonode +sudo chown -Rf geonode:www-data /opt/geonode_custom/ +sudo chmod -Rf 775 /opt/geonode_custom/ +``` + +Clone the source code + +``` shell +cd /opt/geonode_custom/ +git clone https://github.com/GeoNode/geonode-project.git +``` + +This will clone the `master` branch. You will have to checkout the desidered branch or tag. +As an example, if you want to generate a propject for GeoNode 4.2.2 you will docker + +``` shell +git checkout -b 4.2.2 +``` + +Make an instance out of the `Django Template` + +::: note +::: title +Note +::: + +We will call our instance `my_geonode`. You can change the name at your convenience. +::: + +``` shell +source /usr/share/virtualenvwrapper/virtualenvwrapper.sh +mkvirtualenv --python=/usr/bin/python3 my_geonode + +Alterantively you can also create the virtual env like below +python3.8 -m venv /home/geonode/dev/.venvs/my_geonode +source /home/geonode/dev/.venvs/my_geonode/bin/activate + +pip install Django==3.2.13 + +django-admin startproject --template=./geonode-project -e py,sh,md,rst,json,yml,ini,env,sample,properties -n monitoring-cron -n Dockerfile my_geonode +cd /opt/geonode_custom/my_geonode +``` + +Create the .env file + +An [.env]{.title-ref} file is requird to run the application. It can be created from the [.env.sample]{.title-ref} either manually or with the create-envfile.py script. + +The script accepts several parameters to create the file, in detail: + +> - *hostname*: e.g. master.demo.geonode.org, default localhost +> - *https*: (boolean), default value is False +> - *email*: Admin email (this is required if https is set to True since a valid email is required by Letsencrypt certbot) +> - *emv_type*: [prod]{.title-ref}, [test]{.title-ref} or [dev]{.title-ref}. It will set the [DEBUG]{.title-ref} variable to [False]{.title-ref} ([prod]{.title-ref}, [test]{.title-ref}) or [True]{.title-ref} ([dev]{.title-ref}) +> - *geonodepwd*: GeoNode admin password (required inside the .env) +> - *geoserverpwd*: Geoserver admin password (required inside the .env) +> - *pgpwd*: PostgreSQL password (required inside the .env) +> - *dbpwd*: GeoNode DB user password (required inside the .env) +> - *geodbpwd*: Geodatabase user password (required inside the .env) +> - *clientid*: Oauth2 client id (required inside the .env) +> - *clientsecret*: Oauth2 client secret (required inside the .env) +> - *secret key*: Django secret key (required inside the .env) +> - *sample_file*: absolute path to a env_sample file used to create the env_file. If not provided, the one inside the GeoNode project is used. +> - *file*: absolute path to a json file that contains all the above configuration + +::: note +::: title +Note +::: + +if the same configuration is passed in the json file and as an argument, the CLI one will overwrite the one in the JSON file. If some value is not provided, a random string is used + +Example USAGE +::: + +```` shell +```bash +python create-envfile.py -f /opt/core/geonode-project/file.json \ + --hostname localhost \ + --https \ + --email random@email.com \ + --geonodepwd gn_password \ + --geoserverpwd gs_password \ + --pgpwd pg_password \ + --dbpwd db_password \ + --geodbpwd _db_password \ + --clientid 12345 \ + --clientsecret abc123 +``` +```` + +Example JSON expected: + +```` shell +```JSON +{ + "hostname": "value", + "https": "value", + "email": "value", + "geonodepwd": "value", + "geoserverpwd": "value", + "pgpwd": "value", + "dbpwd": "value", + "geodbpwd": "value", + "clientid": "value", + "clientsecret": "value" +} +``` +```` + +Modify the code and the templates and rebuild the Docker Containers + +``` shell +docker-compose -f docker-compose.yml build --no-cache +``` + +Finally, run the containers + +``` shell +docker-compose -f docker-compose.yml up -d +``` + +### Deploy an instance of a geonode-project Django template with Docker on a domain + +::: note +::: title +Note +::: + +We will use `www.example.org` as an example. You can change the name at your convenience. +::: + +Stop the containers + +``` shell +cd /opt/geonode_custom/my_geonode + +docker-compose -f docker-compose.yml stop +``` + +Edit the `ENV` override file in order to deploy on `www.example.org` + +Replace everywhere `localhost` with `www.example.org` + +``` shell +vim .env +``` + +``` shell +# e.g.: :%s/localhost/www.example.org/g +``` + +::: note +::: title +Note +::: + +It is possible to override here even more variables to customize the GeoNode instance. See the `GeoNode Settings` section in order to get a list of the available options. +::: + +Run the containers in daemon mode + +``` shell +docker-compose -f docker-compose.yml -f docker-compose.override.example-org.yml up --build -d +``` + +### Test geonode-project with vagrant + +::: note +::: title +Note +::: + +Inside geonode-project files you will find one file named [Vagrantfile.compose]{.title-ref} and one named [Vagrantfile.stack]{.title-ref}, copy one of them onto file [Vagrantfile]{.title-ref} to use them with vagrant. +::: + +``` shell +apt-get install -y vagrant +#choose what to test (in this case docker-compose.yml) +cp Vagrantfile.compose Vagrantfile +#this will start a vargant virtual machine, generate and build geonode-project +vagrant up +# check services are up upon reboot +vagrant ssh geonode-compose -c 'docker ps' +vagrant destroy -f +# test docker swarm +cp Vagrantfile.stack Vagrantfile +vagrant up +# check services are up upon reboot +vagrant ssh geonode-vagrant -c 'docker service ls' +vagrant destroy -f +``` + +::: note +::: title +Note +::: + +Vagrant will generate a dummi project named \"antani\" inside vagrant, starting with the geonode-project codebase, this way it is possible to test inside vagrant almost instantly what one modifies into geonode-project +::: diff --git a/install/basic/img/geoserver_setup_001.png b/docs/install/basic/img/geoserver_setup_001.png similarity index 100% rename from install/basic/img/geoserver_setup_001.png rename to docs/install/basic/img/geoserver_setup_001.png diff --git a/install/basic/img/geoserver_setup_002.png b/docs/install/basic/img/geoserver_setup_002.png similarity index 100% rename from install/basic/img/geoserver_setup_002.png rename to docs/install/basic/img/geoserver_setup_002.png diff --git a/install/basic/img/geoserver_setup_003.png b/docs/install/basic/img/geoserver_setup_003.png similarity index 100% rename from install/basic/img/geoserver_setup_003.png rename to docs/install/basic/img/geoserver_setup_003.png diff --git a/install/basic/img/geoserver_setup_004.png b/docs/install/basic/img/geoserver_setup_004.png similarity index 100% rename from install/basic/img/geoserver_setup_004.png rename to docs/install/basic/img/geoserver_setup_004.png diff --git a/install/basic/img/production_geoserver_001.png b/docs/install/basic/img/production_geoserver_001.png similarity index 100% rename from install/basic/img/production_geoserver_001.png rename to docs/install/basic/img/production_geoserver_001.png diff --git a/install/basic/img/production_geoserver_002.png b/docs/install/basic/img/production_geoserver_002.png similarity index 100% rename from install/basic/img/production_geoserver_002.png rename to docs/install/basic/img/production_geoserver_002.png diff --git a/install/basic/img/production_geoserver_003.png b/docs/install/basic/img/production_geoserver_003.png similarity index 100% rename from install/basic/img/production_geoserver_003.png rename to docs/install/basic/img/production_geoserver_003.png diff --git a/install/basic/img/production_geoserver_004.png b/docs/install/basic/img/production_geoserver_004.png similarity index 100% rename from install/basic/img/production_geoserver_004.png rename to docs/install/basic/img/production_geoserver_004.png diff --git a/install/basic/img/production_geoserver_005.png b/docs/install/basic/img/production_geoserver_005.png similarity index 100% rename from install/basic/img/production_geoserver_005.png rename to docs/install/basic/img/production_geoserver_005.png diff --git a/install/basic/img/production_geoserver_006.png b/docs/install/basic/img/production_geoserver_006.png similarity index 100% rename from install/basic/img/production_geoserver_006.png rename to docs/install/basic/img/production_geoserver_006.png diff --git a/install/basic/img/production_geoserver_007.png b/docs/install/basic/img/production_geoserver_007.png similarity index 100% rename from install/basic/img/production_geoserver_007.png rename to docs/install/basic/img/production_geoserver_007.png diff --git a/install/basic/img/production_geoserver_008.png b/docs/install/basic/img/production_geoserver_008.png similarity index 100% rename from install/basic/img/production_geoserver_008.png rename to docs/install/basic/img/production_geoserver_008.png diff --git a/install/basic/img/production_geoserver_009.png b/docs/install/basic/img/production_geoserver_009.png similarity index 100% rename from install/basic/img/production_geoserver_009.png rename to docs/install/basic/img/production_geoserver_009.png diff --git a/install/basic/img/production_geoserver_010.png b/docs/install/basic/img/production_geoserver_010.png similarity index 100% rename from install/basic/img/production_geoserver_010.png rename to docs/install/basic/img/production_geoserver_010.png diff --git a/install/basic/img/throubleshooting_prod_001.png b/docs/install/basic/img/throubleshooting_prod_001.png similarity index 100% rename from install/basic/img/throubleshooting_prod_001.png rename to docs/install/basic/img/throubleshooting_prod_001.png diff --git a/docs/install/basic/index.md b/docs/install/basic/index.md new file mode 100644 index 00000000..0ec6d9a4 --- /dev/null +++ b/docs/install/basic/index.md @@ -0,0 +1,382 @@ +# GeoNode Basic Installation {#geonode-project-basic} + +## Overview + +The followings are the easiest and recommended ways to deploy a full-stack GeoNode server on your host. + +1. **First Step**: Deploy `GeoNode on a local server`{.interpreted-text role="guilabel"}, running as `http://localhost/` service. `GeoServer`{.interpreted-text role="guilabel"} will be also available at `http://localhost/geoserver/` +2. **Second Step**: Deploy `GeoNode on a production server`{.interpreted-text role="guilabel"}, running as `https://my_geonode.geonode.org/` service. `GeoServer`{.interpreted-text role="guilabel"} will be also available at `https://my_geonode.geonode.org/geoserver/` +3. **Third Step**: Customize `.env`{.interpreted-text role="guilabel"} to match your needs +4. **Fourth Step**: Secure your production deployment; change the `admin`{.interpreted-text role="guilabel"} passwords and `OAUth2`{.interpreted-text role="guilabel"} keys +5. **Further Production Enhancements** + +## First Step: Deploy GeoNode on a local server + +### Docker Setup for Ubuntu (22.04) {#Ubuntu (22.04) Basic Setup} + +``` shell +# install OS level packages.. +sudo add-apt-repository universe +sudo apt-get update -y +sudo apt-get install -y git-core git-buildpackage debhelper devscripts python3.10-dev python3.10-venv virtualenvwrapper +sudo apt-get install -y apt-transport-https ca-certificates curl lsb-release gnupg gnupg-agent software-properties-common vim + +# add docker repo and packages... +sudo mkdir -p /etc/apt/keyrings +curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg +sudo chmod a+r /etc/apt/keyrings/docker.gpg +echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null + +sudo apt-get update -y +sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose +sudo apt autoremove --purge + +# add your user to the docker group... +sudo usermod -aG docker ${USER} +su ${USER} +``` + +### Create an instance of your `geonode-project` + +Let\'s say your project is named `my_geonode`{.interpreted-text role="guilabel"} perform the following steps: + +``` shell +git clone https://github.com/GeoNode/geonode-project.git + +# Checkout the desired branch. For example, 4.2.2 to generate a project for GeoNode 4.2.2 + +git checkout -b 4.2.2 + +# Ubuntu +source /usr/share/virtualenvwrapper/virtualenvwrapper.sh +mkvirtualenv --python=/usr/bin/python3 my_geonode + +Alterantively you can also create the virtual env like below +python3.8 -m venv /home/geonode/dev/.venvs/my_geonode +source /home/geonode/dev/.venvs/my_geonode/bin/activate + +pip install Django==3.2.13 + +# CentOS +virtualenv -p python3 my_geonode +source my_geonode/bin/activate + +django-admin startproject --template=./geonode-project -e py,sh,md,rst,json,yml,ini,env,sample,properties -n monitoring-cron -n Dockerfile my_geonode + +# If the previous command does not work for some reason, try the following one +python -m django startproject --template=./geonode-project -e py,sh,md,rst,json,yml,ini,env,sample,properties -n monitoring-cron -n Dockerfile my_geonode +``` + +### Prepare the .env file {#Docker create env file} + +GeoNode uses an `.env` where the environment variables required by Docker and GeoNode itself must be configured. +A simple way to generate a basic `.env` is using the `create-envfile.py` utility command. If run without arguments it will generate a valid `.env` for a local deployment. + +``` shell +cd my_geonode +python create-envfile.py +``` + +The command accepts the following arguments: + +- \`\--https\`: Enable SSL. It\'s disabled by default + +- + + \`\--env_type\`: + + : - When set to [prod]{.title-ref} [DEBUG]{.title-ref} is disabled and the creation of a valid [SSL]{.title-ref} is requested to Letsencrypt\'s ACME server + - When set to [test]{.title-ref} [DEBUG]{.title-ref} is disabled and a test [SSL]{.title-ref} certificate is generated for local testing + - When set to [dev]{.title-ref} [DEBUG]{.title-ref} is enabled and no [SSL]{.title-ref} certificate is generated + +- \`\--hostname\`: The URL that will serve GeoNode ([localhost]{.title-ref} by default) + +- \`\--email\`: The administrator\'s email. Notice that a real email and valid SMPT configurations are required if [\--env_type]{.title-ref} is set to [prod]{.title-ref}. Letsencrypt uses email for issuing the SSL certificate + +- \`\--geonodepwd\`: GeoNode\'s administrator password. A random value is set if left empty + +- \`\--geoserverpwd\`: GeoNode\'s administrator password. A random value is set if left empty + +- \`\--pgpwd\`: PostgreSQL\'s administrator password. A random value is set if left empty + +- \`\--dbpwd\`: GeoNode DB user role\'s password. A random value is set if left empty + +- \`\--geodbpwd\`: GeoNode data DB user role\'s password. A random value is set if left empty + +- \`\--clientid\`: Client id of Geoserver\'s GeoNode Oauth2 client. A random value is set if left empty + +- \`\--clientsecret\`: Client secret of Geoserver\'s GeoNode Oauth2 client. A random value is set if left empty + +### Build and run {#Docker build and run} + +``` shell +docker compose build +docker compose up -d +``` + +- You can follow the containers startup by running the following commands from `my_geonode` root folder: + + > ``` shell + > # GeoNode Container + > docker-compose logs -f django + > + > # GeoServer Container + > docker-compose logs -f geoserver + > + > # DB Container + > docker-compose logs -f db + > + > # NGINX Container + > docker-compose logs -f geonode + > ``` + +- If any error occurs, try to catch the error stacktrace by running the following commands from `my_geonode` root folder: + + > ``` shell + > # GeoNode “entrypoint.sh” Logs + > tail -F -n 300 invoke.log + > ``` + +### Connect to `http://localhost/`{.interpreted-text role="guilabel"} + +The startup typically takes some time, so be patient... + +If everything goes well, you should be able to see from the `geonode startup logs` a line similar to the following one: + +``` shell + [UWSGI] Uwsgi running... +``` + +Connect to `http://localhost/`{.interpreted-text role="guilabel"} + +The default credentials are: + +> - GeoNode (`http://localhost/`{.interpreted-text role="guilabel"}) `admin`{.interpreted-text role="guilabel"}: +> +> > `username: admin` +> > `password: admin` +> +> - GeoServer (`http://localhost/geoserver/`{.interpreted-text role="guilabel"}) `admin`{.interpreted-text role="guilabel"}: +> +> > `username: admin` +> > `password: geoserver` + +## Second Step: Deploy GeoNode on a production server {#Docker deploy to production} + +In the case you would like to deploy to, let\'s say, `https://my_geonode.geonode.org/`{.interpreted-text role="guilabel"}, you will need to change `.env` as follows: + +``` diff +DOCKER_ENV=production +SITEURL=https://my_geonode.geonode.org/ +NGINX_BASE_URL=https://my_geonode.geonode.org/ +ALLOWED_HOSTS=['django',] +GEOSERVER_WEB_UI_LOCATION=https://my_geonode.geonode.org/geoserver/ +GEOSERVER_PUBLIC_LOCATION=https://my_geonode.geonode.org/geoserver/ +HTTP_HOST= +HTTPS_HOST=my_geonode.geonode.org +HTTP_PORT=80 +HTTPS_PORT=443 +LETSENCRYPT_MODE=production # This will use Letsencrypt and the ACME server to generate valid SSL certificates +``` + +These variables are automatically set by the `create-envfile.py` script if the `--https` and `--hostname` variables are used. + +::: warning +::: title +Warning +::: + +When `LETSENCRYPT_MODE` is set to `production` a valid email and email SMPT server are required to make the system generate a valid certificate. +::: + +### Restart the containers + +Whenever you change someting on `.env`{.interpreted-text role="guilabel"} file, you will need to rebuild the container + +::: warning +::: title +Warning +::: + +Notche that the following command drops any change you might have done manually inside the containers, except for the static volumes. +::: + +``` shell +docker-compose up -d +``` + +### Troubleshooting + +If for some reason you are not able to reach the server on the `HTTPS`{.interpreted-text role="guilabel"} channel, please check the `NGINX`{.interpreted-text role="guilabel"} configuration files below: + +1. Enter the `NGINX`{.interpreted-text role="guilabel"} container + + > ``` shell + > docker-compose exec geonode sh + > ``` + +2. Install an editor if not present + + > ``` shell + > apk add nano + > ``` + +3. Double check that the `nginx.https.enabled.conf` link has been correctly created + + > ``` shell + > ls -lah + > ``` + > + > ![](img/throubleshooting_prod_001.png){.align-center} + > + > If the list does not match exactly the figure above, please run the following commands, and check again + > + > ``` shell + > rm nginx.https.enabled.conf + > ln -s nginx.https.available.conf nginx.https.enabled.conf + > ``` + +4. Inspect the `nginx.https.enabled.conf` contents + + > ``` shell + > nano nginx.https.enabled.conf + > ``` + > + > Make sure the contents match the following + > + > ::: warning + > ::: title + > Warning + > ::: + > + > Change the `Hostname`{.interpreted-text role="guilabel"} accordingly. **This is only an example!** + > ::: + > + > ``` shell + > # NOTE : $VARIABLES are env variables replaced by entrypoint.sh using envsubst + > # not to be mistaken for nginx variables (also starting with $, but usually lowercase) + > + > # This file is to be included in the main nginx.conf configuration if HTTPS_HOST is set + > ssl_session_cache shared:SSL:10m; + > ssl_session_timeout 10m; + > + > # this is the actual HTTPS host + > server { + > listen 443 ssl; + > server_name my_geonode.geonode.org; + > keepalive_timeout 70; + > + > ssl_certificate /certificate_symlink/fullchain.pem; + > ssl_certificate_key /certificate_symlink/privkey.pem; + > ssl_protocols TLSv1 TLSv1.1 TLSv1.2; + > ssl_ciphers HIGH:!aNULL:!MD5; + > + > include sites-enabled/*.conf; + > } + > + > # if we try to connect from http, we redirect to https + > server { + > listen 80; + > server_name my_geonode.geonode.org; # TODO : once geoserver supports relative urls, we should allow access though both HTTP and HTTPS at the same time and hence remove HTTP_HOST from this line + > + > # Except for let's encrypt challenge + > location /.well-known { + > alias /geonode-certificates/.well-known; + > include /etc/nginx/mime.types; + > } + > + > # Redirect to https + > location / { + > return 302 https://my_geonode.geonode.org/$request_uri; # TODO : we should use 301 (permanent redirect, but not practical for debug) + > } + > } + > ``` + > + > ::: warning + > ::: title + > Warning + > ::: + > + > **Save the changes, if any, and exit!** + > ::: + +5. Reload the NGINX configuration + + > ``` shell + > nginx -s reload + > 2020/06/24 10:00:11 [notice] 112#112: signal process started + > /etc/nginx# exit + > ``` + +6. It may be helpful to disable https to isolate the source of errors. After reverting the HTTPS-related changes in the [.env]{.title-ref} file, repeat the above steps and ensure that the `nginx.http.enabled.conf` link has been correctly created. + + > ``` shell + > ln -s nginx.conf nginx.http.enabled.conf + > nano nginx.http.enabled.conf + > ``` + +## Customize `.env`{.interpreted-text role="guilabel"} to match your needs + +In the case you would like to modify the GeoNode behavior, always use the `.env`{.interpreted-text role="guilabel"} file in order to update the `settings`{.interpreted-text role="guilabel"}. + +If you need to change a setting which does not exist in `.env`{.interpreted-text role="guilabel"}, you can force the values inside `my_geonode/settings.py`{.interpreted-text role="guilabel"} + +Refer to the section: `settings`{.interpreted-text role="ref"} + +You can add here any property referred as + +> | Env: `PROPERTY_NAME` + +### Restart the containers + +Whenever you change someting on `.env`{.interpreted-text role="guilabel"} file, you will need to rebuild the containers. + +::: warning +::: title +Warning +::: + +**Be careful!** The following command drops any change you might have done manually inside the containers, except for the static volumes. +::: + +``` shell +docker-compose up -d django +``` + +\[Optional\] Configure your SSL Certificates +\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^\^ + +In production deployment mode, GeoNode uses by default `Let's Encrypt`{.interpreted-text role="guilabel"} certificates + +You may want to provide your own certificates to GeoNode + +``` shell +docker exec -it nginx4my_geonode_geonode sh -c 'mkdir /geonode-certificates/my_geonode' + +wget --no-check-certificate 'http://' \ + -O chain.crt + +wget --no-check-certificate 'http://' \ + -O my_geonode.key + +docker cp chain.crt nginx4my_geonode_geonode:/geonode-certificates/my_geonode + +docker cp my_geonode.key nginx4my_geonode_geonode:/geonode-certificates/my_geonode + +docker-compose exec geonode sh +apk add vim + +vim nginx.https.enabled.conf +``` + +``` diff +-ssl_certificate /certificate_symlink/fullchain.pem; +-ssl_certificate_key /certificate_symlink/privkey.pem; ++ssl_certificate /geonode-certificates/my_geonode/chain.crt; ++ssl_certificate_key /geonode-certificates/my_geonode/my_geonode.key; +``` + +``` shell +nginx -s reload +exit +``` diff --git a/docs/install/index.md b/docs/install/index.md new file mode 100644 index 00000000..a02c71d0 --- /dev/null +++ b/docs/install/index.md @@ -0,0 +1,13 @@ +::: {.toctree maxdepth="3"} +basic/index +::: + +::: {.toctree maxdepth="3"} +advanced/index +::: + + +- [index](basic/index.md) + +- [index](advanced/index.md) + diff --git a/intermediate/backup/img/br_1.png b/docs/intermediate/backup/img/br_1.png similarity index 100% rename from intermediate/backup/img/br_1.png rename to docs/intermediate/backup/img/br_1.png diff --git a/intermediate/backup/img/br_2.png b/docs/intermediate/backup/img/br_2.png similarity index 100% rename from intermediate/backup/img/br_2.png rename to docs/intermediate/backup/img/br_2.png diff --git a/intermediate/backup/img/br_3.png b/docs/intermediate/backup/img/br_3.png similarity index 100% rename from intermediate/backup/img/br_3.png rename to docs/intermediate/backup/img/br_3.png diff --git a/intermediate/backup/img/br_jenkins_1.png b/docs/intermediate/backup/img/br_jenkins_1.png similarity index 100% rename from intermediate/backup/img/br_jenkins_1.png rename to docs/intermediate/backup/img/br_jenkins_1.png diff --git a/intermediate/backup/img/br_jenkins_10.png b/docs/intermediate/backup/img/br_jenkins_10.png similarity index 100% rename from intermediate/backup/img/br_jenkins_10.png rename to docs/intermediate/backup/img/br_jenkins_10.png diff --git a/intermediate/backup/img/br_jenkins_11.png b/docs/intermediate/backup/img/br_jenkins_11.png similarity index 100% rename from intermediate/backup/img/br_jenkins_11.png rename to docs/intermediate/backup/img/br_jenkins_11.png diff --git a/intermediate/backup/img/br_jenkins_12.png b/docs/intermediate/backup/img/br_jenkins_12.png similarity index 100% rename from intermediate/backup/img/br_jenkins_12.png rename to docs/intermediate/backup/img/br_jenkins_12.png diff --git a/intermediate/backup/img/br_jenkins_13.png b/docs/intermediate/backup/img/br_jenkins_13.png similarity index 100% rename from intermediate/backup/img/br_jenkins_13.png rename to docs/intermediate/backup/img/br_jenkins_13.png diff --git a/intermediate/backup/img/br_jenkins_14.png b/docs/intermediate/backup/img/br_jenkins_14.png similarity index 100% rename from intermediate/backup/img/br_jenkins_14.png rename to docs/intermediate/backup/img/br_jenkins_14.png diff --git a/intermediate/backup/img/br_jenkins_15.png b/docs/intermediate/backup/img/br_jenkins_15.png similarity index 100% rename from intermediate/backup/img/br_jenkins_15.png rename to docs/intermediate/backup/img/br_jenkins_15.png diff --git a/intermediate/backup/img/br_jenkins_16.png b/docs/intermediate/backup/img/br_jenkins_16.png similarity index 100% rename from intermediate/backup/img/br_jenkins_16.png rename to docs/intermediate/backup/img/br_jenkins_16.png diff --git a/intermediate/backup/img/br_jenkins_17.png b/docs/intermediate/backup/img/br_jenkins_17.png similarity index 100% rename from intermediate/backup/img/br_jenkins_17.png rename to docs/intermediate/backup/img/br_jenkins_17.png diff --git a/intermediate/backup/img/br_jenkins_18.png b/docs/intermediate/backup/img/br_jenkins_18.png similarity index 100% rename from intermediate/backup/img/br_jenkins_18.png rename to docs/intermediate/backup/img/br_jenkins_18.png diff --git a/intermediate/backup/img/br_jenkins_19.png b/docs/intermediate/backup/img/br_jenkins_19.png similarity index 100% rename from intermediate/backup/img/br_jenkins_19.png rename to docs/intermediate/backup/img/br_jenkins_19.png diff --git a/intermediate/backup/img/br_jenkins_2.png b/docs/intermediate/backup/img/br_jenkins_2.png similarity index 100% rename from intermediate/backup/img/br_jenkins_2.png rename to docs/intermediate/backup/img/br_jenkins_2.png diff --git a/intermediate/backup/img/br_jenkins_20.png b/docs/intermediate/backup/img/br_jenkins_20.png similarity index 100% rename from intermediate/backup/img/br_jenkins_20.png rename to docs/intermediate/backup/img/br_jenkins_20.png diff --git a/intermediate/backup/img/br_jenkins_21.png b/docs/intermediate/backup/img/br_jenkins_21.png similarity index 100% rename from intermediate/backup/img/br_jenkins_21.png rename to docs/intermediate/backup/img/br_jenkins_21.png diff --git a/intermediate/backup/img/br_jenkins_22.png b/docs/intermediate/backup/img/br_jenkins_22.png similarity index 100% rename from intermediate/backup/img/br_jenkins_22.png rename to docs/intermediate/backup/img/br_jenkins_22.png diff --git a/intermediate/backup/img/br_jenkins_23.png b/docs/intermediate/backup/img/br_jenkins_23.png similarity index 100% rename from intermediate/backup/img/br_jenkins_23.png rename to docs/intermediate/backup/img/br_jenkins_23.png diff --git a/intermediate/backup/img/br_jenkins_24.png b/docs/intermediate/backup/img/br_jenkins_24.png similarity index 100% rename from intermediate/backup/img/br_jenkins_24.png rename to docs/intermediate/backup/img/br_jenkins_24.png diff --git a/intermediate/backup/img/br_jenkins_25.png b/docs/intermediate/backup/img/br_jenkins_25.png similarity index 100% rename from intermediate/backup/img/br_jenkins_25.png rename to docs/intermediate/backup/img/br_jenkins_25.png diff --git a/intermediate/backup/img/br_jenkins_26.png b/docs/intermediate/backup/img/br_jenkins_26.png similarity index 100% rename from intermediate/backup/img/br_jenkins_26.png rename to docs/intermediate/backup/img/br_jenkins_26.png diff --git a/intermediate/backup/img/br_jenkins_27.png b/docs/intermediate/backup/img/br_jenkins_27.png similarity index 100% rename from intermediate/backup/img/br_jenkins_27.png rename to docs/intermediate/backup/img/br_jenkins_27.png diff --git a/intermediate/backup/img/br_jenkins_28.png b/docs/intermediate/backup/img/br_jenkins_28.png similarity index 100% rename from intermediate/backup/img/br_jenkins_28.png rename to docs/intermediate/backup/img/br_jenkins_28.png diff --git a/intermediate/backup/img/br_jenkins_29.png b/docs/intermediate/backup/img/br_jenkins_29.png similarity index 100% rename from intermediate/backup/img/br_jenkins_29.png rename to docs/intermediate/backup/img/br_jenkins_29.png diff --git a/intermediate/backup/img/br_jenkins_3.png b/docs/intermediate/backup/img/br_jenkins_3.png similarity index 100% rename from intermediate/backup/img/br_jenkins_3.png rename to docs/intermediate/backup/img/br_jenkins_3.png diff --git a/intermediate/backup/img/br_jenkins_30.png b/docs/intermediate/backup/img/br_jenkins_30.png similarity index 100% rename from intermediate/backup/img/br_jenkins_30.png rename to docs/intermediate/backup/img/br_jenkins_30.png diff --git a/intermediate/backup/img/br_jenkins_31.png b/docs/intermediate/backup/img/br_jenkins_31.png similarity index 100% rename from intermediate/backup/img/br_jenkins_31.png rename to docs/intermediate/backup/img/br_jenkins_31.png diff --git a/intermediate/backup/img/br_jenkins_32.png b/docs/intermediate/backup/img/br_jenkins_32.png similarity index 100% rename from intermediate/backup/img/br_jenkins_32.png rename to docs/intermediate/backup/img/br_jenkins_32.png diff --git a/intermediate/backup/img/br_jenkins_33.png b/docs/intermediate/backup/img/br_jenkins_33.png similarity index 100% rename from intermediate/backup/img/br_jenkins_33.png rename to docs/intermediate/backup/img/br_jenkins_33.png diff --git a/intermediate/backup/img/br_jenkins_4.png b/docs/intermediate/backup/img/br_jenkins_4.png similarity index 100% rename from intermediate/backup/img/br_jenkins_4.png rename to docs/intermediate/backup/img/br_jenkins_4.png diff --git a/intermediate/backup/img/br_jenkins_5.png b/docs/intermediate/backup/img/br_jenkins_5.png similarity index 100% rename from intermediate/backup/img/br_jenkins_5.png rename to docs/intermediate/backup/img/br_jenkins_5.png diff --git a/intermediate/backup/img/br_jenkins_6.png b/docs/intermediate/backup/img/br_jenkins_6.png similarity index 100% rename from intermediate/backup/img/br_jenkins_6.png rename to docs/intermediate/backup/img/br_jenkins_6.png diff --git a/intermediate/backup/img/br_jenkins_7.png b/docs/intermediate/backup/img/br_jenkins_7.png similarity index 100% rename from intermediate/backup/img/br_jenkins_7.png rename to docs/intermediate/backup/img/br_jenkins_7.png diff --git a/intermediate/backup/img/br_jenkins_8.png b/docs/intermediate/backup/img/br_jenkins_8.png similarity index 100% rename from intermediate/backup/img/br_jenkins_8.png rename to docs/intermediate/backup/img/br_jenkins_8.png diff --git a/intermediate/backup/img/br_jenkins_9.png b/docs/intermediate/backup/img/br_jenkins_9.png similarity index 100% rename from intermediate/backup/img/br_jenkins_9.png rename to docs/intermediate/backup/img/br_jenkins_9.png diff --git a/docs/intermediate/backup/index.md b/docs/intermediate/backup/index.md new file mode 100644 index 00000000..c062ff46 --- /dev/null +++ b/docs/intermediate/backup/index.md @@ -0,0 +1,621 @@ +# Full GeoNode Backup & Restore {#backup_restore_geonode} + +The admin command to backup and restore GeoNode, allows to extract consistently the GeoNode and GeoServer data models in a serializable +meta-format which is being interpreted later by the restore procedure in order to exactly rebuild the whole structure. + +In particular the tool helps developers and administrators to correctly extract and serialize the following resources: + +- **GeoNode** (Resource Base Model): + + > 1. Layers (both raster and vectors) + > 2. Maps + > 3. Documents + > 4. People with Credentials + > 5. Permissions + > 6. Associated Styles + > 7. Static data and templates + +- **GeoServer** (Catalog): + + > 1. OWS Services configuration and limits + > 2. Security model along with auth filters configuration, users and credentials + > 3. Workspaces + > 4. Stores (both DataStores and CoverageStores) + > 5. Layers + > 6. Styles + +The tool exposes two GeoNode Management Commands, \'backup\' and \'restore\'. + +The commands allow to: + +1. Fully backup GeoNode data and fixtures on a zip archive +2. Fully backup GeoServer configuration (physical datasets - tables, shapefiles, geotiffs) +3. Fully restore GeoNode and GeoServer fixtures and catalog from the zip archive + +The usage of those commands is quite easy and straightforward. + +The first step is to ensure that everything is correctly configured and the requisites respected in order to successfully +perform a backup and restore of GeoNode. + +::: warning +::: title +Warning +::: + +It is worth to notice that this functionality requires the latest [GeoServer Extension](https://build.geo-solutions.it/geonode/geoserver/latest/) (2.9.x or greater) for GeoNode in order to correctly work. +::: + +::: note +::: title +Note +::: + +GeoServer full documentation is also available here [GeoServer Docs](https://docs.geoserver.org/stable/en/user/community/backuprestore/index.html) +::: + +## Requisites and Setup + +**Before** running a GeoNode backup / restore, it is necessary to ensure everything is correctly configured and setup. + +### Settings + +Accordingly to the admin needs, the file `settings.ini` must be ceated before running a backup or restore. + +The default files can be found at `geonode/br/management/commands/settings_sample.ini` and `geonode/br/management/commands/settings_docker_sample.ini` +for the classic and Docker environments accordingly. The content is similar in both of them (an example from `settings_sample.ini`): + +``` ini +[database] +pgdump = pg_dump +pgrestore = pg_restore + +[geoserver] +datadir = geoserver/data +dumpvectordata = yes +dumprasterdata = yes + +[fixtures] +# NOTE: Order is important +apps = contenttypes,auth,people,groups,account,guardian,admin,actstream,announcements,avatar,base,dialogos,documents,geoserver,invitations,pinax_notifications,layers,maps,oauth2_provider,services,sites,socialaccount,taggit,tastypie,upload,user_messages +dumps = contenttypes,auth,people,groups,account,guardian,admin,actstream,announcements,avatar,base,dialogos,documents,geoserver,invitations,pinax_notifications,layers,maps,oauth2_provider,services,sites,socialaccount,taggit,tastypie,upload,user_messages +``` + +The `settings.ini` file can be created in any directory accessible by GeoNode, and it\'s path can be passed to the backup / restore +procedures using [-c]{.title-ref} ([\--config]{.title-ref}) argument. + +There are few different sections of the configuration file, that must be carefully checked before running a backup / restore command. + +#### Settings: \[database\] Section + +``` ini +[database] +pgdump = pg_dump +pgrestore = pg_restore +``` + +This section is quite simple. It contains only two properties: + +- *pgdump*; the path of the `pg_dump` local command. +- *pgrestore*; the path of the `pg_restore` local command. + +::: warning +::: title +Warning +::: + +Those properties are ignored in case GeoNode is not configured to use a DataBase as backend (see `settings.py` and `local_settings.py` sections) +::: + +::: note +::: title +Note +::: + +Database connection settings (both for GeoNode and GeoServer) will be taken from `settings.py` and `local_settings.py` configuration files. Make sure they are correctly configured (on the target GeoNode instance, too) and the DataBase server is accessible while executing a backup / restore command. +::: + +#### Settings: \[geoserver\] Section + +``` ini +[geoserver] +datadir = /opt/gs_data_dir +datadir_exclude_file_path = +dumpvectordata = yes +dumprasterdata = yes +data_dt_filter = +data_layername_filter = +data_layername_exclude_filter = +``` + +This section allows to enable / disable a full data backup / restore of GeoServer. + +- *datadir*: the full path of GeoServer Data Dir, by default `/opt/gs_data_dir`. The path **must** be accessible and **fully writable** by the `geonode` and / or `httpd server` users when executing a backup / restore command. +- *datadir_exclude_file_path*: comma separated list of paths to exclude from `geoserver_catalog.zip`; This list will be sent and managed directly by the GeoServer Backup REST API. +- *dumpvectordata*: a boolean flag enabling or disabling creation of a vector data dump from GeoServer (shapefiles or DB tables). If `false` (or `no`) vector data won\'t be stored / re-stored. +- *dumprasterdata*: a boolean flag enabling or disabling creation of a raster data dump from GeoServer (geotiffs). If `false` (or `no`) raster data won\'t be stored / re-stored. +- *data_dt_filter*: {cmp_operator} {ISO8601} e.g. \> 2019-04-05T24:00 which means \"include on backup archive only the files that have been modified later than 2019-04-05T24:00 +- *data_layername_filter*: comma separated list of `layer names`, optionally with glob syntax e.g.: [tuscany]()*,italy; Only \`\`RASTER\`\` original data and \`\`VECTORIAL\`\` table dumps matching those filters will beincluded*\* into the backup ZIP archive +- *data_layername_exclude_filter*: comma separated list of `layer names`, optionally with glob syntax e.g.: [tuscany]()*,italy; The \`\`RASTER\`\` original data and \`\`VECTORIAL\`\` table dumps matching those filters will beexcluded*\* from the backup ZIP archive + +::: warning +::: title +Warning +::: + +Enabling these options **requires** the GeoServer Data Dir to be accessible and **fully writable** for the `geonode` and / or `httpd server` users when executing a backup / restore command. +::: + +#### Settings: \[fixtures\] Section + +``` ini +[fixtures] +#NOTE: Order is important +apps = people,account,avatar.avatar,base.backup,base.license,base.topiccategory,base.region,base.resourcebase,base.contactrole,base.link,base.restrictioncodetype,base.spatialrepresentationtype,guardian.userobjectpermission,guardian.groupobjectpermission,layers.uploadsession,layers.style,layers.layer,layers.attribute,layers.layerfile,maps.map,maps.maplayer,maps.mapsnapshot,documents.document,taggit + +dumps = people,accounts,avatars,backups,licenses,topiccategories,regions,resourcebases,contactroles,links,restrictioncodetypes,spatialrepresentationtypes,useropermissions,groupopermissions,uploadsessions,styles,layers,attributes,layerfiles,maps,maplayers,mapsnapshots,documents,tags +``` + +This section is the most complex one. Usually you don\'t need to modify it. Only an expert user who knows Python and GeoNode model structure should modify this section. + +What its properties mean: + +- *apps*; an ordered list of GeoNode Django applications. The backup / restore procedure will dump / restore the fixtures in a portable format. +- *dumps*; this is the list of `files` associated to the Django applications. The order **must** be the same as in the *apps* property above. Each name represents the `file name` where to dump to / read from the single app\'s fixtures. + +## Executing from the CLI + +The following sections shows instructions on how to perform backup / restore from the command line by using the Django Admin Management Commands. + +In order to obtain a basic user guide for the management command from the command line, just run + +> ``` shell +> python manage.py backup --help +> +> python manage.py restore --help +> ``` + +`--help` will provide the list of available command line options with a brief description. + +By default both procedures activate [Read Only]{.title-ref} mode, disabling any content modifying requests, which is reverted +to the previous state (from before the execution) after finish, regardless of the command\'s result (success or failure). +To disable activation of this mode, `--skip-read-only` argument can be passed to the command. + +It is worth notice that both commands allows the following option + +> ``` shell +> python manage.py backup --force / -f +> +> python manage.py restore --force / -f +> ``` + +Which enables a non-interactive mode, meaning the user will not be asked for an explicit confirmation. + +### Backup + +In order to perform a backup just run the command: + +> ``` shell +> python manage.py backup --backup-dir= --config= +> ``` + +The management command will automatically generate a `.zip` archive file on the target folder in case of success. In the target directory +`.md5` file with the same name as backup will be created. It contains the MD5 hash of the backup file, which can be used to check archive\'s +integrity before restoration. + +It is worth to mention that `br` (Backup & Restore GeoNode application) will not be dumped, even if specified in the `settings.ini` as +its content is strictly related to the certain GeoNode instance. + +Currently, GeoNode does not support any automatic extraction of the backup file. It should be manually transferred, if needed to the target +instance environment. + +### Restore + +The `restore` command has a number of arguments, modifying its execution: + +\# `-c` / `--config`: path to the `settings.ini` configuration file. If the Backup archive is provided with his settings, the latter will be used by the restore command and this option won\'t be mandatory anymore + +1. `--skip-geoserver`: the GeoServer backup restoration won\'t be performed +2. `--skip-geoserver-info`: {Default: True} Skips GeoServer Global Infos, like the proxy base url and other global GeoServer metadata info +3. `--skip-geoserver-security`: {Default: True} Skips GeoServer all the Security Settings +4. `--backup-file`: (exclusive together with `--backup-files-dir`) path to the backup `.zip` archive +5. `--backup-files-dir`: (exclusive together with `--backup-file`) directory containing backup archives. The directory may contain a number of files, but **only** backup archives are allowed with a `.zip` extension. In case multiple archives are present in the directory, the newest one, created after the last already restored backup creation time, will be restored. This option was implemented with a thought of automated restores. +6. `--recovery-file`: Backup archive containing GeoNode data to restore in case of failure. +7. `-l` / `--with-logs`: the backup file will be checked against the restoration logs (history). In case this backup has already been restored (MD5 based comparision), RuntimeError is raised, preventing restore execution. +8. `-n` / `--notify`: the restore procedure outcome will be send by an e-mail notification to the superusers of the instance (note: notification will be sent to the superusers of the instance before restoration). +9. `--skip-read-only`: the restore procedure will be conducted without setting [Read Only]{.title-ref} mode during execution. +10. `--soft-reset`: the restore procedure will preserve geoserver table / resources during the restore. By default the procedure will drop tables and resources + +In order to perform a default backup restoration just run the command: + +> ``` shell +> python manage.py restore --backup-file= --config= +> ``` + +For restore to run it requires either `--backup-file` or `--backup-files-dir` argument defined. + +::: warning +::: title +Warning +::: + +The Restore will **overwrite** the whole target instances of GeoNode (and by default GeoServer) including users, catalog and database, so be very careful. +::: + +## GeoNode Admin GUI Inspection + +The history of restored backups can be verified in the admin panel. + +Login to the admin panel and select `Restored backups` table from `BACKUP/RESTORE` application. + +![](img/br_1.png){.align-center} + +A list will be displayed with a history of all restored backups. You can select a certain backup to view it\'s data. + +![](img/br_2.png){.align-center} + +The detailed view of the restored backup shows backup archive\'s name, it\'s MD5 hash, it\'s creation/modification date (in the target folder), and the date of the restoration. Please note Restored Backup history cannot be modified. + +![](img/br_3.png){.align-center} + +## B/R in Docker environment + +When executing B/R in the Docker environment, creation backup to / restoration from should be executed in `/backup_restore` directory. +It is a shared volume between Geoserver and Geonode images, created for this purpose only. Pointing at another +location will fail, as one of the images won\'t have an access to the files. + +::: warning +::: title +Warning +::: + +When executing B/R in Docker environment **remember** to create `settings.ini` file basing on `settings_docker_sample.ini` to point at a proper Geoserver data directory! In other case configuration mismatch may cause unexpected errors. +::: + +::: warning +::: title +Warning +::: + +The only other volume shared between images is `/geoserver_data/data`, but backup creation **should not** be performed there, as the recursive Geoserver backups may be created in such case. +::: + +## B/R Jenkins Job in Docker environment + +When installing GeoNode through the `geonode-project`{.interpreted-text role="guilabel"} Docker (see `geonode-project-basic`{.interpreted-text role="ref"}), an instance of [Jenkins CI/CD](https://www.jenkins.io/) is also automatically deployed and available through `http:///jenkins`{.interpreted-text role="guilabel"}. + +![](img/br_jenkins_1.png){.align-center} + +### Configure Jenkins at first startup + +The very first time you try to access Jenkins, you will need to unlock it and generate a new administrator username and password. + +In order to do that, you need to print the contents of the auto-generated file `/var/jenkins_home/secrets/initialAdminPassword` + +1. First of all search for the Jenkins container ID, usually `jenkins4{{project_name}}`{.interpreted-text role="guilabel"} where `{{project_name}}` is the name of your `geonode-project`{.interpreted-text role="guilabel"} instance (e.g. `my_geonode`) + +``` shell +$> docker ps + + CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES + e9fc97a75d1a geonode/nginx:geoserver "/docker-entrypoint.…" 2 hours ago Up 2 hours 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp nginx4my_geonode + c5496400b1b9 my_geonode_django "/bin/sh -c 'service…" 2 hours ago Up 2 hours django4my_geonode + bc899f81fa28 my_geonode_celery "/bin/sh -c 'service…" 2 hours ago Up 2 hours celery4my_geonode + 3b213400d630 geonode/geoserver:2.17.1 "/usr/local/tomcat/t…" 2 hours ago Up 2 hours 8080/tcp geoserver4my_geonode + d2f59d70a0d3 geonode/postgis:11 "docker-entrypoint.s…" 2 hours ago Up 2 hours 5432/tcp db4my_geonode + 3f9ce0be7f88 rabbitmq "docker-entrypoint.s…" 2 hours ago Up 2 hours 4369/tcp, 5671-5672/tcp, 25672/tcp rabbitmq4my_geonode + 02fdbce9ae73 geonode/letsencrypt:latest "./docker-entrypoint…" 2 hours ago Up 14 seconds my_geonode_letsencrypt_1 + c745520fd551 jenkins/jenkins:lts "/sbin/tini -- /usr/…" 2 hours ago Up 2 hours 0.0.0.0:9080->9080/tcp, 8080/tcp, 0.0.0.0:50000->50000/tcp, 0.0.0.0:9443->8443/tcp jenkins4my_geonode +``` + +1. Now just `cat` the file above inside the Jenkins container + +``` shell +$> docker container exec -u 0 -it jenkins4my_geonode sh -c 'cat /var/jenkins_home/secrets/initialAdminPassword' + + b91e9d*****************373834 +``` + +1. Copy the hash code you just got form the print above, and copy-and-paste to the browser window + +![](img/br_jenkins_2.png){.align-center} + +In the next step just install the `Default Plugins`{.interpreted-text role="guilabel"}. You can install more of them later on from the management page. + +![](img/br_jenkins_3.png){.align-center} + +Wait until Jenkins has finished configuring the plugins + +![](img/br_jenkins_4.png){.align-center} + +Provide the administrator credentials as requested + +![](img/br_jenkins_5.png){.align-center} + +Confirm the Jenkins instance URL, this can be changed form the configuration later in case you will need to update the server address + +![](img/br_jenkins_6.png){.align-center} + +Well done, Jenkins is ready now + +![](img/br_jenkins_7.png){.align-center} + +The next step is to configure a Jenkins Job able to interact with the Django Docker container and run a full backup + +![](img/br_jenkins_8.png){.align-center} + +### Configure a Jenkins Job to run a full backup on the Django Container + +Before creating the new Jenkins job, we need to install and configure a new plugin, [Publish over SSH](https://plugins.jenkins.io/publish-over-ssh) + +In order to do that, once logged in as `admin`, go to the Jenkins `Management Page > Manage Plugins`{.interpreted-text role="guilabel"} tab + +![](img/br_jenkins_9.png){.align-center} + +![](img/br_jenkins_10.png){.align-center} + +Click on `Available`{.interpreted-text role="guilabel"} tab and search for `SSH` available plugins + +![](img/br_jenkins_11.png){.align-center} + +Select and check the `Publish over SSH` one + +![](img/br_jenkins_12.png){.align-center} + +Install the plugins and restart Jenkins + +![](img/br_jenkins_13.png){.align-center} + +The next step is to configure the `SSH Server Connection` for the `Publish over SSH`{.interpreted-text role="guilabel"} plugin. + +Move to `Jenkins Configuration`{.interpreted-text role="guilabel"} + +![](img/br_jenkins_14.png){.align-center} + +Scroll down until you find the `Publish over SSH`{.interpreted-text role="guilabel"} plugin section + +![](img/br_jenkins_15.png){.align-center} + +Depending on how your `HOST SSH service` has been configured, you might need several information in order to setup the connection. + +Here below an example using a global host (`master.demo.geonode.org`) accepting `SSH` connections via `RSA keys` + +![](img/br_jenkins_16.png){.align-center} + +::: note +::: title +Note +::: + +Before saving the configuration always ensure the connection is ok by using the `Test Configuration`{.interpreted-text role="guilabel"} button + +![](img/br_jenkins_17.png){.align-center} +::: + +It is possible also to run and configure `Jenkins`{.interpreted-text role="guilabel"} to run locally, as an instance on `localhost`{.interpreted-text role="guilabel"}. +In that case you will need to change few things in order to allow `Jenkins`{.interpreted-text role="guilabel"} to access your local network. + +1. First of all, be sure `OpenSSH Server`{.interpreted-text role="guilabel"} is correctly installed and running on your PC. Eventually check any firewall rules. + + ``` shell + $> sudo apt install openssh-server + + # Test your connection locally + $> ssh -p 22 user@localhost + user@localhost's password: + ``` + +2. You will need to do some changed to your `docker-compose.yml` file in order to enable the `host network`{.interpreted-text role="guilabel"} configuration. + + ::: note + ::: title + Note + ::: + + Enable `network_mode: "host"` on Jenkins container + ::: + + ``` shell + $> vim docker-compose.yml + + ... + jenkins: + image: jenkins/jenkins:lts + # image: istresearch/jenkins:latest + container_name: jenkins4${COMPOSE_PROJECT_NAME} + user: jenkins + ports: + - '${JENKINS_HTTP_PORT}:${JENKINS_HTTP_PORT}' + - '${JENKINS_HTTPS_PORT}:${JENKINS_HTTPS_PORT}' + - '50000:50000' + network_mode: "host" + volumes: + - jenkins_data:/var/jenkins_home + - backup-restore:/backup_restore + # - /var/run/docker.sock:/var/run/docker.sock + environment: + - 'JENKINS_OPTS=--httpPort=${JENKINS_HTTP_PORT} --httpsPort=${JENKINS_HTTPS_PORT} --prefix=/jenkins' + ... + + # Recreate the Jenkins container + $> docker-compose stop jenkins + $> docker-compose rm jenkins + $> docker-compose up -d jenkins + ``` + + ::: warning + ::: title + Warning + ::: + + From now on, your local Jenkins instance will be accessible from `http://localhost:9080/jenkins`{.interpreted-text role="guilabel"} + ::: + +3. Add `localhost` Server to the `Publish over SSH`{.interpreted-text role="guilabel"} plugin configuration + + Mode to `http://localhost:9080/jenkins/configure`{.interpreted-text role="guilabel"} and fill the required information + + ![](img/br_jenkins_18.png){.align-center} + + ::: note + ::: title + Note + ::: + + Before saving the configuration always ensure the connection is ok by using the `Test Configuration`{.interpreted-text role="guilabel"} button + + ![](img/br_jenkins_17.png){.align-center} + ::: + +We are now ready to create the Jenkins Job which will run a full backup & restore of our GeoNode dockerized instance. + +1. Move to the Jenkins Home and click on `Create a Job`{.interpreted-text role="guilabel"} button + + ![](img/br_jenkins_19.png){.align-center} + +2. Provide a name to the Job and select `Freestyle project`{.interpreted-text role="guilabel"} + + ![](img/br_jenkins_20.png){.align-center} + +3. Enable the `Log rotation`{.interpreted-text role="guilabel"} strategy if needed + + ![](img/br_jenkins_21.png){.align-center} + +4. Configure the `Job Parameters`{.interpreted-text role="guilabel"} which will be used by the script later on. + + Add three `String Parameters`{.interpreted-text role="guilabel"} + + ![](img/br_jenkins_22.png){.align-center} + + as shown below + + 1. `BKP_FOLDER_NAME`{.interpreted-text role="guilabel"} + + ![](img/br_jenkins_23.png){.align-center} + + 2. `SOURCE_URL`{.interpreted-text role="guilabel"} + + ::: warning + ::: title + Warning + ::: + + Provide the correct URL of your GeoNode instance + ::: + + ![](img/br_jenkins_24.png){.align-center} + + 3. `TARGET_URL`{.interpreted-text role="guilabel"} + + ::: warning + ::: title + Warning + ::: + + Provide the correct URL of your GeoNode instance + ::: + + ![](img/br_jenkins_25.png){.align-center} + +5. Enable the `Delete workspace before build starts`{.interpreted-text role="guilabel"} and `Add timestamps to the Console Output`{.interpreted-text role="guilabel"} `Build Environment`{.interpreted-text role="guilabel"} options + + ![](img/br_jenkins_26.png){.align-center} + +6. Finally let\'s create the `SSH Build Step`{.interpreted-text role="guilabel"} + + ![](img/br_jenkins_27.png){.align-center} + + Select the correct `SSH Server`{.interpreted-text role="guilabel"} and provide the `Exec Command`{.interpreted-text role="guilabel"} below + + > ::: warning + > ::: title + > Warning + > ::: + > + > Replace `{{project_name}}`{.interpreted-text role="guilabel"} with your `geonode-project instance name`{.interpreted-text role="guilabel"} (e.g. `my_geonode`{.interpreted-text role="guilabel"}) + > ::: + > + > ``` shell + > # Replace {{project_name}} with your geonode-project instance name (e.g. my_geonode) + > # docker exec -u 0 -it django4{{project_name}} sh -c 'SOURCE_URL=$SOURCE_URL TARGET_URL=$TARGET_URL ./{{project_name}}/br/backup.sh $BKP_FOLDER_NAME' + > # e.g.: + > docker exec -u 0 -it django4my_geonode sh -c 'SOURCE_URL=$SOURCE_URL TARGET_URL=$TARGET_URL ./my_geonode/br/backup.sh $BKP_FOLDER_NAME' + > ``` + > + > ![](img/br_jenkins_28.png){.align-center} + +> Click on `Advanced`{.interpreted-text role="guilabel"} and change the parameters as shown below +> +> > ![](img/br_jenkins_29.png){.align-center} + +**Save!** You are ready to run the Job\... + +![](img/br_jenkins_30.png){.align-center} + +![](img/br_jenkins_31.png){.align-center} + +![](img/br_jenkins_32.png){.align-center} + +![](img/br_jenkins_33.png){.align-center} + +### Link the `backup_restore`{.interpreted-text role="guilabel"} folder to a local folder on the `HOST`{.interpreted-text role="guilabel"} + +In the case you need to save the backup archives outside the docker container, there\'s the possibility to directly link the `backup_restore`{.interpreted-text role="guilabel"} folder to a local folder on the `HOST`{.interpreted-text role="guilabel"}. + +In that case you won\'t need to `docker cp`{.interpreted-text role="guilabel"} the files everytime from the containers, they will be directly available on the host filesystem. + +::: warning +::: title +Warning +::: + +Always keep an eye to the disk space. Backups archives may be huge. +::: + +::: note +::: title +Note +::: + +You might want also to consider filtering the files through the backup dt filters on the `settings.ini`{.interpreted-text role="guilabel"} in order to reduce the size of the archive files, including only the new ones. +::: + +Modify the `docker-compose.override.yml` as follows in order to link the backup folders outside. + +::: note +::: title +Note +::: + +`/data/backup_restore` is a folder physically located into the host filesystem. +::: + +``` shell +$> vim docker-compose.override.yml + +version: '2.2' +services: + +django: + build: . + # Loading the app is defined here to allow for + # autoreload on changes it is mounted on top of the + # old copy that docker added when creating the image + volumes: + - '.:/usr/src/my_geonode' + - '/data/backup_restore:/backup_restore' # Link to local volume in the HOST + +celery: + volumes: + - '/data/backup_restore:/backup_restore' # Link to local volume in the HOST + +geoserver: + volumes: + - '/data/backup_restore:/backup_restore' # Link to local volume in the HOST + +jenkins: + volumes: + - '/data/backup_restore:/backup_restore' # Link to local volume in the HOST + +# Restart the containers +$> docker-compose up -d +``` diff --git a/docs/intermediate/harvesting/index.md b/docs/intermediate/harvesting/index.md new file mode 100644 index 00000000..a684f4ba --- /dev/null +++ b/docs/intermediate/harvesting/index.md @@ -0,0 +1,486 @@ +# Harvesting resources from remote services + +GeoNode is able to harvest resource metadata from multiple remote services. + +Harvesting is the process by which a metadata catalogue, *i.e.* GeoNode, is able to connect to other remote catalogues +and retrieve information about their resources. This process is usually performed periodically, in order to keep the +local catalogue in sync with the remote. + +When appropriately configured, GeoNode will contact the remote service, extract a list of relevant resources that can be +harvested and create local resources for each remote resource. It will also keep the resources synchronized with the +remote service by periodically updating them. + +Out of the box, GeoNode ships with support for harvesting from: + +1. `Other remote GeoNode instances `{.interpreted-text role="ref"}; +2. `OGC WMS servers `{.interpreted-text role="ref"}; +3. `ArcGIS REST services `{.interpreted-text role="ref"}. + +Adding support for `additional harvesting sources `{.interpreted-text role="ref"} is also possible. + +## GeoNode harvesting concepts + +When a **harvester** is configured, GeoNode is able to use its corresponding **harvester worker** to contact +the remote service and generate a list of **harvestable resources**. +The user is then able to select which of those resources are of interest. Depending on its configured update +frequency, sometime later, the **harvesting scheduler** will create new **harvesting sessions** in order to create +local GeoNode resources from the remote harvestable resources that had been marked as relevant by the user. + +The above description uses the following key concepts: + +::: {#harvester-label} + +harvester + +: This is the configuration object that is used to parametrize harvesting of a remote service. It is configurable + at runtime and is preserved in the GeoNode database. + + Harvesters and their properties can be managed by visiting the `Harvesting -> Harvesters` section of the GeoNode + admin area, or by visiting the `api/v2/harvesters/` API endpoint with an admin user. + + Among other parameters, a harvester holds: + + remote_url + + : Base URL of the remote service being harvested, *e.g.* `https://stable.demo.geonode.org` + + harvester_type + + : Type of harvester worker that will be used to perform harvesting. See the + `Harvester worker concept `{.interpreted-text role="ref"} and the `standard harvester workers + `{.interpreted-text role="ref"} sections below for more detail. Example: `geonode.harvesting.harvesters.geonodeharvester.GeonodeUnifiedHarvesterWorker`. + + scheduling_enabled + + : Whether harvesting shall be performed periodically by the + `harvesting scheduler `{.interpreted-text role="ref"} or not. + + harvesting_session_update_frequency + + : How often (in minutes) should new `harvesting sessions `{.interpreted-text role="ref"} be + automatically scheduled? + + refresh_harvestable_resources_update_frequency + + : How often (in minutes) should new `refresh sessions `{.interpreted-text role="ref"} be automatically scheduled? + + default_owner + + : Which GeoNode user shall be made the owner of harvested resources + + harvest_new_resources_by_default + + : Should new remote resources be harvested automatically? When this option is selected, the user does not + need to specify which `harvestable resources `{.interpreted-text role="ref"} should be harvested, + as all of them will be automatically marked for harvesting by GeoNode. + + delete_orphan_resources_automatically + + : Orphan resources are those that have previously been created by means of a harvesting operation but that + GeoNode can no longer find on the remote service being harvested. Should these resources be deleted from + GeoNode automatically? This also applies to when a harvester configuration is deleted, in which case all of + the resources that originated from that harvester are now considered to be orphan. +::: + +::: {#harvester-worker-label} + +harvester worker + +: Harvester workers implement retrieval for concrete remote service types. Each harvester uses a specific worker, + depending on the type of remote service that it gets data from. Harvester workers may accept their own additional + configuration parameters. + + Harvester workers are set as the `harvester_type` attribute on a `harvester `{.interpreted-text role="ref"}. Their configuration is set as a JSON + object on the `harvester_type_specific_configuration` attribute of the harvester. + + GeoNode ships with the following harvester workers: + + 1. `GeoNode `{.interpreted-text role="ref"} - Enables harvesting from other GeoNode deployments + 2. `WMS `{.interpreted-text role="ref"} - Enables harvesting from OGC WMS servers + 3. `ArcGIS REST services `{.interpreted-text role="ref"} - Enables harvesting from ArcGIS REST services + + `Adding new harvester workers `{.interpreted-text role="ref"} is also possible. This allows custom GeoNode + deployments to add support for harvesting from other remote sources. +::: + +::: {#harvestable-resource-label} + +harvestable resource + +: A resource that is available on the remote server. Harvestable resources are persisted in the GeoNode DB. They are + created during `refresh operations `{.interpreted-text role="ref"}, when the harvester worker + interacts with the remote service in order to discover which remote resources can be harvested. + + Harvestable resources can be managed by visiting the `Harvesting -> Harvestable resources` section of the + GeoNode admin area, or by visiting the `api/v2/harvesters/{harvester-id}/harvestable-resources` API endpoint + with an admin user. + + In order to be harvested by the `harvesting scheduler `{.interpreted-text role="ref"}, a harvestable resource + must have its `should_be_harvested` attribute set to `True`. This attribute can be set manually by the user + or it can be set automatically by the harvester worker, in case the corresponding harvester is configured with + `harvest_new_resources_by_default = True` +::: + +::: {#asynchronous-session-label} + +harvesting session + +: In GeoNode, discovering remote resources and harvesting them is always done under the scope of a harvesting + session. These sessions are stored in the GeoNode DB and can be inspected by visiting the + `Harvesting -> Asynchronous harvesting sessions` section of the GeoNode admin area. + + Harvesting sessions are used to keep track of the progress of execution of the relevant harvesting operations. They + are updated while each operation is running. There are two types of sessions: + + ::: {#refresh-session-label} + + refresh session + + : This session is created during the `update of harvestable resources operation `{.interpreted-text role="ref"}. + It has `type=discover-harvestable-resources`. During a refresh session, the harvester worker discovers remote + resources and creates their respective `harvestable resources `{.interpreted-text role="ref"} on the GeoNode + DB. After such session is finished, the user can inspect the found harvestable resources and mark those that + are relevant with `should_be_harvester=True`. + ::: + + ::: {#harvesting-session-label} + + harvesting session + + : This session is created during the `perform harvesting operation `{.interpreted-text role="ref"}. It has + `type=harvesting`. During a harvesting session, the harvester worker creates or updates new GeoNode resources + based on the harvestable resources that have been configured with `should_be_harvested=True`. + ::: + + In addition to the aforementioned `type`, harvesting sessions also carry the `status` attribute, which provides + context on the current status of the session (and consequently of the underlying harvesting operation). +::: + +::: {#harvesting-scheduler-label} + +harvesting scheduler + +: The scheduler is responsible for initiating new `harvesting operations `{.interpreted-text role="ref"} + in an automated fashion. Periodically, the scheduler goes through the list of existing harvesters, checking if it + is time to dispatch one of the harvesting operations mentioned in the next section. + + The scheduler\'s operation frequency is configurable by defining a `HARVESTER_SCHEDULER_FREQUENCY_MINUTES` setting + - the default is to trigger the scheduler every 30 seconds. + + ::: note + ::: title + Note + ::: + + Since the harvesting scheduler only checks if there is work to do once every `x` seconds + (defaulting to 30 seconds, as mentioned above), there will usually be a delay between the time a harvesting + operation is supposed to be scheduled and the actual time when it is indeed scheduled. Moreover, the harvesting + scheduler is implemented as a celery task. This means that, if the celery worker is busy, that may also cause a + delay in scheduling harvesting operations, as the scheduler\'s celery task may not be triggered immediately. + ::: +::: + +## Harvesting workflows + +There are two main possible harvesting workflows: + +1. `Continuous harvesting `{.interpreted-text role="ref"} +2. `One-time harvesting `{.interpreted-text role="ref"} + +### Continuous harvesting {#continuous-harvesting-label} + +> This workflow relies on the `harvesting scheduler `{.interpreted-text role="ref"} in order to ensure harvested +> resources are continuously kept up to date with their remote counterparts. +> +> 1. User creates harvester and sets its `scheduling_enabled` attribute to `True`; +> 2. When the time comes, the harvesting scheduler calls the +> `update list of harvestable resources operation `{.interpreted-text role="ref"}. Alternatively, +> the user may call this operation manually the first time. +> 3. When the previous operation is done, user goes through the list of generated +> `harvestable resources `{.interpreted-text role="ref"} and, for each relevant harvestable resource, sets +> it `should_be_harvested` attribute to `True`. Alternatively, if the harvester has its +> `harvest_new_resources_automatically` attribute set to `True`, the harvestable resources will already be +> marked as to be harvested, without requiring manual user intervention; +> 4. When the time comes, the harvesting scheduler calls the +> `perform harvesting operation `{.interpreted-text role="ref"}. This causes +> the remote resources to be harvested. These now show up as resources on the local GeoNode. + +### One-time harvesting {#one-time-harvesting-label} + +> This workflow is mostly executed manually by the user. +> +> 1. User creates harvester and sets its `scheduling_enabled` attribute to `False`; +> 2. User calls the `update list of harvestable resources operation `{.interpreted-text role="ref"}; +> 3. When the previous operation is done, user goes through the list of generated +> `harvestable resources `{.interpreted-text role="ref"} and, for each relevant harvestable resource, sets +> it `should_be_harvested` attribute to `True`; +> 4. User then proceeds to call the `perform harvesting operation `{.interpreted-text role="ref"}. This causes +> the remote resources to be harvested. These now show up as resources on the local GeoNode. + +## Harvester operations {#harvesting-operations-label} + +Each GeoNode harvester is able to perform a finite set of operations. These can be performed either: + +1. In an **automated fashion**, being dispatched by the harvesting scheduler. Automated harvesting is only performed when + the corresponding `harvester `{.interpreted-text role="ref"} has `scheduling_enabled=True`; +2. **On-demand**, by explicit request of the user. On-demand execution can be requested by one of two ways: + 1. By selecting the relevant harvester(s) in the `Harvesting -> Harvesters` section of the GeoNode admin area and + then selecting and running an action from the drop-down menu; + + 2. By interacting with the GeoNode REST API. Harvester actions are requested by issuing `HTTP PATCH` requests to + the `/api/v2/harvesters/{harvester-id}/` endpoint. The payload of such requests must specify the corresponding + status. For example, by issuing a request like: + + curl -X PATCH http:/localhost/api/v2/harvesters/1/ \ + -H "Content-Type: application/json" \ + -u "myuser:mypass" \ + --data '{"status": "updating-harvestable-resources"}' + + We are asking that the harvester\'s status be changed to `updating-harvestable-resources`. If the server accepts + this request, then the `update list of harvestable resources operation `{.interpreted-text role="ref"} + is triggered. + + ::: note + ::: title + Note + ::: + + The server will not accept the API request if the harvester\'s current status is not `ready`. + ::: + +While performing an action, the harvester\'s `status` property transitions from `ready` to whatever action-related +status is appropriate (as indicated below). As the operation finishes execution, the harvester\'s status transitions +back to `ready`. If the harvester has any status other than `ready`, then it is currently busy. When a harvester +is busy it cannot execute other operations, you\'ll need to wait until the current operation finishes. + +### Check if the remote service is available operation {#check-remote-available-action-label} + +This operation causes the harvester to perform a simple health check on the remote service, in order to check whether it +responds successfully. The response is stored in the harvester\'s `remote_available` property. This operation is +performed in the same process of the main GeoNode (*i.e.* it runs synchronously). + +When triggered, this operation causes the harvester\'s status to transition to `checking-availability`. +As the operation finishes, the harvester\'s status transitions back to `ready`. + +Invocation via the GeoNode admin is performed by selecting the +`Check availability of selected harvesters` command. + +Invocation via the GeoNode REST API is performed by issuing an HTTP PATCH request with a payload that sets the +harvester status. + +### Update the list of harvestable resources operation {#update-harvestable-resources-action-label} + +This operation causes the harvester to interact with the remote service in order to discover which resources are +available for being harvested. Existing remote resources are then saved as +`harvestable resources `{.interpreted-text role="ref"}. + +Since this operation can potentially take a long time to complete (as we don\'t know how may resources may exist on the +remote service), it is run using a background process. GeoNode creates a new `refresh session `{.interpreted-text role="ref"} +and uses it to track the progress of this operation. + +When triggered, this operation causes the harvester\'s status to transition to `updating-harvestable-resources`. +As the operation finishes, the harvester\'s status transitions back to `ready`. + +Invocation via the GeoNode admin is performed by selecting the +`Update harvestable resources for selected harvesters` command. + +Invocation via the GeoNode REST API is performed by issuing an HTTP PATCH request with a payload that sets the +harvester status. + +### Perform harvesting operation {#perform-harvesting-action-label} + +This operation causes the harvester to check which harvestable resources are currently marked as being harvestable +and then, for each one, harvest the resource from the remote server. + +Since this operation can potentially take a long time to complete (as we don\'t know how may resources may exist on the +remote service), it is run using a background process. GeoNode creates a new `harvesting session `{.interpreted-text role="ref"} +and uses it to track the progress of this operation. + +When triggered, this operation causes the harvester\'s status to transition to `harvesting-resources`. +As the operation finishes, the harvester\'s status transitions back to `ready`. + +Invocation via the GeoNode admin is performed by selecting the `Perform harvesting on selected harvesters` command. + +Invocation via the GeoNode REST API is performed by issuing an HTTP PATCH request with a payload that sets the +harvester status. + +### Abort update of harvestable resources operation {#abort-refresh-action-label} + +This operation causes the harvester to abort an on-going +`update list of harvestable resources operation `{.interpreted-text role="ref"}. + +When triggered, this operation causes the harvester\'s status to transition to `aborting-update-harvestable-resources`. +As the operation finishes, the harvester\'s status transitions back to `ready`. + +Invocation via the GeoNode admin is performed by selecting the +`Abort on-going update of harvestable resources for selected harvesters` command. + +Invocation via the GeoNode REST API is performed by issuing an HTTP PATCH request with a payload that sets the +harvester status. + +### Abort harvesting operation {#abort-harvesting-action-label} + +This operation causes the harvester to abort an on-going +`perform harvesting operation `{.interpreted-text role="ref"}. + +When triggered, this operation causes the harvester\'s status to transition to `aborting-performing-harvesting`. +As the operation finishes, the harvester\'s status transitions back to `ready`. + +Invocation via the GeoNode admin is performed by selecting the +`Abort on-going harvesting sessions for selected harvesters` command. + +Invocation via the GeoNode REST API is performed by issuing an HTTP PATCH request with a payload that sets the +harvester status. + +### Reset harvester operation {#reset-harvester-action-label} + +This operation causes the harvester\'s status to be reset back to `ready`. It is mainly useful for troubleshooting +potential errors, in order to unlock harvesters that may get stuck in a non-actionable status when some unforeseen +error occurs. + +When triggered, this operation causes the harvester\'s status to transition to `ready` immediately. + +Invocation via the GeoNode admin is performed by selecting the `Reset harvester status` command. + +This operation cannot be called via the GeoNode API. + +## Standard harvester workers {#standard-harvester-workers-label} + +::: note +::: title +Note +::: + +Remember that, as stated above, a harvester worker is configured by means of setting the `harvester_type` and +`harvester_type_specific_configuration` attributes on the `harvester `{.interpreted-text role="ref"}. + +Moreover, the format of the `harvester_type_specific_configuration` attribute must be a JSON object. +::: + +### GeoNode harvester worker {#geonode-harvester-worker-label} + +This worker is able to harvest remote GeoNode deployments. In addition to creating local resources by retrieving +the remote metadata, this harvester is also able to copy remote datasets over to the local GeoNode. This means +that this harvester can even be used in order to generate replicated GeoNode instances. + +This harvester can be used by setting `harvester_type=geonode.harvesting.harvesters.geonodeharvester.GeonodeUnifiedHarvesterWorker` +in the harvester configuration. + +It recognizes the following `harvester_type_specific_configuration` parameters: + +harvest_datasets + +: Whether to harvest remote resources of type `dataset` or not. Acceptable values: `true` (the default) or `false`. + +copy_datasets + +: Whether to copy remote resources of type `dataset` over to the local GeoNode. Acceptable values: `true` or `false` (the default). + +harvest_documents + +: Whether to harvest remote resources of type `document` or not. Acceptable values: `true` (the default) or `false`. + +copy_documents + +: Whether to copy remote resources of type `document` over to the local GeoNode. Acceptable values: `true` or `false` (the default). + +resource_title_filter + +: A string that must be present in the remote resources\' `title` in order for them to be acknowledged as + harvestable resources. This allows filtering out resources that are not relevant. Acceptable values: any + alphanumeric value. + + Example: setting this to a value of `"water"` would mean that the harvester would generate harvestable resources + for remote resources that are titled *water basins*, *Water territories*, etc. The harvester would not generate + harvestable resources for remote resources whose title does not contain the word *water*. + +start_date_filter + +: A string specifying a datetime that is used to filter out resources by their start_date. This is parsed with + `dateutil.parser.parse() `{.interpreted-text role="ref"}, + which means that it accepts many different formats (e.g. [2021-06-31T13:04:05Z]{.title-ref}) + +end_date_filter + +: Similar to `start_date_filter` but uses resources\' [end_date]{.title-ref} as a filter parameter. + +keywords_filter + +: A list of keywords that are used to filter remote resources. + +categories_filter + +: A list of categories that are used to filter remote resources. + +### WMS harvester worker {#wms-harvester-worker-label} + +This worker is able to harvest from remote OGC WMS servers. + +This harvester can be used by setting `harvester_type=geonode.harvesting.harvesters.wms.OgcWmsHarvester` +in the harvester configuration. + +It recognizes the following `harvester_type_specific_configuration` parameters: + +dataset_title_filter + +: A string that is used to filter remote WMS layers by their `title` property. If a remote layer\'s title contains + the string defined by this parameter, then the layer is recognized by the harvester worker. + +### ArcGIS REST Services harvester worker {#arcgis-harvester-worker-label} + +This worker is able to harvest from remote ArcGIS REST Services catalogs. + +This worker is able to recognize two types of `remote_url`: + +1. URL of the ArcGIS REST services catalog. This URL usually ends in `rest/services`. A catalog may expose several + different services. This harvester worker is able to descend into the available ArcGIS Rest services and retrieve + their respective resources. Example: + + https://sampleserver6.arcgisonline.com/arcgis/rest/services + +2. URL of the ArcGIS REST services Service. This URL usually takes the form `{base-url}/rest/services/{service-name}/{service-type}`. Example: + + https://sampleserver6.arcgisonline.com/arcgis/rest/services/CharlotteLAS/ImageServer + +This harvester worker can be used by setting `harvester_type=geonode.harvesting.harvesters.arcgis.ArcgisHarvesterWorker` +in the harvester configuration. + +It recognizes the following `harvester_type_specific_configuration` parameters: + +harvest_map_services + +: Whether services of type [MapServer]{.title-ref} ought to be harvested. Defaults to `True`. + +harvest_image_services + +: Whether services of type [ImageServer]{.title-ref} ought to be harvested. Defaults to `True`. + +resource_name_filter + +: A string that is used to filter remote WMS layers by their `title` property. If a remote layer\'s name contains + the string defined by this parameter, then the layer is recognized by the harvester worker. + +service_names_filter + +: A list of names that are used to filter the remote ArcGIS catalog. + +## Creating new harvesting workers {#creating-new-workers-label} + +New harvesting workers can be created by writing classes derived from +`geonode.harvesting.harvesters.base.BaseGeonodeHarvesterWorker `{.interpreted-text role="ref"}. +This class defines an abstract interface that must be implemented. All methods decorated with `abc.abstractmethod` +must be implemented in the custom harvester worker class. Study the implementation of the standard GeoNode harvester worker +classes in order to gain insight on how to implement custom ones. + +After writing a custom harvester worker class, it can be added to the list of known harvester workers by defining +the `HARVESTER_CLASSES` GeoNode setting. This setting is a list of strings, containing the Python class path to each +harvester worker class. It has a default value of: + + HARVESTER_CLASSES = [ + "geonode.harvesting.harvesters.geonodeharvester.GeonodeUnifiedHarvesterWorker", + "geonode.harvesting.harvesters.wms.OgcWmsHarvester", + "geonode.harvesting.harvesters.arcgis.ArcgisHarvesterWorker", + ] + +These are the standard harvester worker classes shipped by GeoNode. If this setting is defined, its value will simply +extend the default list. This means that it is not possible to disable the standard worker classes, only to add new ones. diff --git a/docs/intermediate/index.md b/docs/intermediate/index.md new file mode 100644 index 00000000..4e5cea5a --- /dev/null +++ b/docs/intermediate/index.md @@ -0,0 +1,26 @@ +# Read-Only and Maintenance Mode {#geonode_monitoring} + +::: {.toctree maxdepth="3"} +modes/index +::: + +# Harvesting resources from remote services + +::: {.toctree maxdepth="3"} +harvesting/index +::: + +# GeoNode Backup and Restore + +::: {.toctree caption="GeoNode Backup and Restore" maxdepth="3"} +backup/index +::: + + +- [index](modes/index.md) + +- [index](harvesting/index.md) + +### GeoNode Backup and Restore +- [index](backup/index.md) + diff --git a/intermediate/modes/img/configuration_admin_panel.png b/docs/intermediate/modes/img/configuration_admin_panel.png similarity index 100% rename from intermediate/modes/img/configuration_admin_panel.png rename to docs/intermediate/modes/img/configuration_admin_panel.png diff --git a/docs/intermediate/modes/index.md b/docs/intermediate/modes/index.md new file mode 100644 index 00000000..2baad2d1 --- /dev/null +++ b/docs/intermediate/modes/index.md @@ -0,0 +1,45 @@ +# Read-Only and Maintenance Modes + +## Overview + +GeoNode gives an option to operate in different modes, according to the needs and demands of the certain application system. + +Changing the currently used mode can be done in the admin panel by the user with super-user privileges, by modifying +`Configuration` singleton model in the `BASE` application: + +>
+> img/configuration_admin_panel.png +>
Configuration change in the admin panel
+>
+ +## Read-Only Mode + +Activating the Read-Only Mode (by setting `Read only` True in the `Configuration`) activates a middleware rejecting all modifying requests +(POST/PUT/DELETE), with an exception for: + +- POST to login view +- POST to logout view +- POST to admin login view +- POST to admin logout view +- all requests to OWS endpoint +- all requests ordered by a super-user + +Additionally, all UI elements allowing modifying GeoNode\'s content are hidden, so e.g. the button \"Upload Layer\" is not rendered in the templates. + +In case a user tries to perform a forbidden request, they will be presented with a static HTML page informing them, the GeoNode is in the Read-Only +mode and this action is currently forbidden. + +## Maintenance Mode + +Activating the Maintenance Mode (by setting `Maintenance` True in the `Configuration`) activates the highest level middleware +(the one executed as the first) rejecting all requests to the GeoNode instance, with an exception for: + +- POST to admin login view +- POST to admin logout view +- all requests ordered by a super-user + +In case a user tries to perform any request against the GeoNode (including GET requests), they will be presented with a static HTML page informing +them, the maintenance actions are taken on the GeoNode instance, and asking them to try again soon. + +The maintenance mode was implemented with a thought of the backup and restore procedures without a necessity to put down the instance, +but at the same time with a restriction of any outer interference. diff --git a/start/img/geonode.png b/docs/start/img/geonode.png similarity index 100% rename from start/img/geonode.png rename to docs/start/img/geonode.png diff --git a/start/img/gn_is_made_for.png b/docs/start/img/gn_is_made_for.png similarity index 100% rename from start/img/gn_is_made_for.png rename to docs/start/img/gn_is_made_for.png diff --git a/start/img/gn_publication_data.png b/docs/start/img/gn_publication_data.png similarity index 100% rename from start/img/gn_publication_data.png rename to docs/start/img/gn_publication_data.png diff --git a/start/img/gn_publication_data_2.png b/docs/start/img/gn_publication_data_2.png similarity index 100% rename from start/img/gn_publication_data_2.png rename to docs/start/img/gn_publication_data_2.png diff --git a/start/img/gn_simplified_architecture.png b/docs/start/img/gn_simplified_architecture.png similarity index 100% rename from start/img/gn_simplified_architecture.png rename to docs/start/img/gn_simplified_architecture.png diff --git a/start/img/online_demo-001.png b/docs/start/img/online_demo-001.png similarity index 100% rename from start/img/online_demo-001.png rename to docs/start/img/online_demo-001.png diff --git a/docs/start/index.md b/docs/start/index.md new file mode 100644 index 00000000..ecf00010 --- /dev/null +++ b/docs/start/index.md @@ -0,0 +1,130 @@ +# GeoNode Basics {#geonode_basics} + +> ![](img/geonode.png){.align-center} + +is a platform for the management and publication of geospatial data. +It brings together mature open-source software projects under an easy to use interface. + +>
+> img/gn_simplified_architecture.png +>
GeoNode simplified architecture
+>
+ +## *With GeoNode, non-specialized users can share data and create interactive maps.* + +> ![](img/gn_is_made_for.png){.align-center} +> +> ![](img/gn_publication_data.png){.align-center} +> +> ![](img/gn_publication_data_2.png){.align-center} + +## Geospatial data storage + +GeoNode allows users to upload vector data (currently shapefiles, json, csv, kml and kmz) and raster data in their original projections +using a web form. + +Vector data is converted into geospatial tables on a DB, satellite imagery and other kinds of raster data are retained as GeoTIFFs. + +Special importance is given to standard metadata formats like ISO 19139:2007 / ISO 19115 metadata standards. + +As soon as the upload is finished, the user can fill the resource metadata in order to make it suddenly available through the [CSW](http://www.opengeospatial.org/standards/cat) (OGC Catalogue Service) +endpoints and APIs. + +Users may also upload a metadata XML document (ISO, FGDC, and Dublin Core format) to fill key GeoNode metadata elements automatically. + +Similarly, GeoNode provides a web based styler that lets the users to change the data portrayals and preview the changes at real time. + +## Data mixing, maps creation + +Once the data has been uploaded, GeoNode lets the user search for it geographically or via keywords in order to create fancy maps. + +All the datasets are automatically re-projected to web Mercator for maps display, making it possible to use different popular base datasets, +like Open Street Map, Google Satellite or Bing datasets. + +Once the maps are saved, it is possible to embed them in any web page or get a PDF version for printing. + +## GeoNode as a building block + +A handful of other Open Source projects extend GeoNode\'s functionality by tapping into the re-usability of Django applications. + +Visit our gallery to see how the community uses GeoNode: [GeoNode Projects](http://geonode.org/gallery/). + +The development community is very supportive of new projects and contributes ideas and guidance for newcomers. + +## Convinced! Where do I sign? + +The next steps are: + +1. Make a ride on the `online_demo`{.interpreted-text role="ref"} +2. Follow the `quick_setup_guide`{.interpreted-text role="ref"} in order to play with your own local instance and access all the admin functionalities +3. Read the documentation starting from the `user guide `{.interpreted-text role="doc"} to the `admin guide `{.interpreted-text role="doc"} +4. Subscribe to the [geonode-users](https://lists.osgeo.org/mailman/listinfo/geonode-users) and/or [geonode-devel](https://lists.osgeo.org/mailman/listinfo/geonode-devel) mailing lists to join the community. + See also the section `get_in_touch`{.interpreted-text role="ref"} for more info. + +Thanks for your interest! + +# Supported Browsers {#supported_browsers} + +GeoNode is known to be working on all modern web browsers. + +This list includes (but is not limited to): + +- [Google Chrome](http://www.google.com/chrome/). +- [Apple Safari](https://www.apple.com/safari/). +- [Mozilla Firefox](https://www.mozilla.org/en-US/firefox/new/). +- [Microsoft Edge](https://developer.microsoft.com/en-us/microsoft-edge/). + +::: note +::: title +Note +::: + +The vast majority of GeoNode developers prefer using Google Chrome. +::: + +# Online Demo {#online_demo} + +::: note +::: title +Note +::: + +**Disclaimer** we do not guarantee for any data published on this Demo Site. Publish the data at your own risk. +Every dataset will be removed automatically every Sunday. If you find some dataset that shouldn\'t be there, please +write suddenly to developers and maintainers. + +See the section `get_in_touch`{.interpreted-text role="ref"} for details. +::: + +A live demo of the latest stable build is available at . + +>
+> img/online_demo-001.png +>
Online Demo @ master.demo.geonode.org
+>
+ +Anyone may sign up for a user account, upload and style data, create and share maps, and change permissions. + +Since it is a demo site, every sunday all the datasets will be wiped out. Users, passwords and groups will be preserved. + +It should hopefully allow you to easily and quickly make a tour of the main capabilities of GeoNode. + +::: warning +::: title +Warning +::: + +This GeoNode instance is configured with standards settings and a very low security level. +This is a demo only not to be considered a really production ready system. +For a complete list of settings, refer to the section: `settings`{.interpreted-text role="ref"} +::: + +# Quick Installation Guide {#quick_setup_guide} + +::: {.toctree maxdepth="3"} +quick/index +::: + + +- [index](quick/index.md) + diff --git a/start/quick/img/docker.png b/docs/start/quick/img/docker.png similarity index 100% rename from start/quick/img/docker.png rename to docs/start/quick/img/docker.png diff --git a/start/quick/img/osgeo_live.png b/docs/start/quick/img/osgeo_live.png similarity index 100% rename from start/quick/img/osgeo_live.png rename to docs/start/quick/img/osgeo_live.png diff --git a/docs/start/quick/index.md b/docs/start/quick/index.md new file mode 100644 index 00000000..b405eaf9 --- /dev/null +++ b/docs/start/quick/index.md @@ -0,0 +1,61 @@ +# Introduction + +The following is a quick guide to get started with GeoNode in most common operating systems. + +::: note +::: title +Note +::: + +For a full setup and deployment, please refer to the `complete installation guides `{.interpreted-text role="doc"} +::: + +This is meant to be run on a fresh machine with no previously installed packages or GeoNode versions. + +::: warning +::: title +Warning +::: + +The methods presented here are meant to be used for a limited internal demo only. +Before exposing your GeoNode instance to a public server, please read carefully the `hardening guide `{.interpreted-text role="doc"} +::: + +# Recommended Minimum System Requirements + +A definite specification of technical requirements is difficult to recommend. Accepted performance is highly subjective. Furthermore, the performance depends on factors such as concurrent users, records in the database or the network connectivity of your infrastructure. + +For deployment of GeoNode on a single server, the following are the *bare minimum* system requirements: + +- 8GB of RAM (16GB or more preferred for a production deployment). +- 2.2GHz processor with 4 cores. (Additional processing power may be required for multiple concurrent styling renderings) +- 30 GB software disk usage (Reserved to OS and source code only). +- Additional disk space for any data hosted with GeoNode, data stored on the DataBase and tiles cached with GeoWebCache. + For db, spatial data, cached tiles, and \"scratch space\" useful for administration, a decent baseline size for GeoNode deployments is between 50GB and 100GB. +- 64-bit hardware **strongly** recommended. + +# Install via Docker + +[Docker](https://www.docker.com/) is a free software platform used for packaging software into standardized units for development, shipment and deployment. + +> ![](img/docker.png) +> +> ::: note +> ::: title +> Note +> ::: +> +> credits to Docker +> ::: + +A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings. + +Docker containers running on a single machine share that machine\'s operating system kernel; they start instantly and use less compute and RAM. + +Containers can share a single kernel, and the only information that needs to be in a container image is the executable and its package dependencies, which never need to be installed on the host system. + +Multiple containers can run on the same machine and share the OS kernel with other containers, each running as isolated processes in user space. + +This documentation will refer to the usage of Docker on a Ubuntu host, but you can of course run Docker also on other Linux distributions, Windows and Mac. + +For the steps to setup Docker on Ubuntu you can jump ahead to `Ubuntu 22.04 `{.interpreted-text role="ref"} diff --git a/usage/accounts_user_profile/index.rst b/docs/usage/accounts_user_profile/index.md similarity index 77% rename from usage/accounts_user_profile/index.rst rename to docs/usage/accounts_user_profile/index.md index d8f8defe..4c4fbae2 100644 --- a/usage/accounts_user_profile/index.rst +++ b/docs/usage/accounts_user_profile/index.md @@ -1,5 +1,4 @@ -Accounts and User Profile -========================= +# Accounts and User Profile In GeoNode many contents are public so unregistered users have read-only access to public maps, datasets,documents, geostory and dashboard. In order to create maps, dashboard or geostory and add datasets or documents, edit the data and share these resources with other users, you need to sign in. @@ -8,8 +7,12 @@ GeoNode is primarily a *social* platform, thus a primary component of any GeoNod This section will guide you through account registration, updating your account information and preferences, connections with social networks and email addresses. -.. toctree:: - :maxdepth: 1 +::: {.toctree maxdepth="1"} +new_account/index +updating_profile/index +::: + + +- [index](new_account/index.md) +- [index](updating_profile/index.md) - new_account/index - updating_profile/index diff --git a/usage/accounts_user_profile/new_account/img/confirm_logout.png b/docs/usage/accounts_user_profile/new_account/img/confirm_logout.png similarity index 100% rename from usage/accounts_user_profile/new_account/img/confirm_logout.png rename to docs/usage/accounts_user_profile/new_account/img/confirm_logout.png diff --git a/usage/accounts_user_profile/new_account/img/logout_link.png b/docs/usage/accounts_user_profile/new_account/img/logout_link.png similarity index 100% rename from usage/accounts_user_profile/new_account/img/logout_link.png rename to docs/usage/accounts_user_profile/new_account/img/logout_link.png diff --git a/usage/accounts_user_profile/new_account/img/register_alerts.png b/docs/usage/accounts_user_profile/new_account/img/register_alerts.png similarity index 100% rename from usage/accounts_user_profile/new_account/img/register_alerts.png rename to docs/usage/accounts_user_profile/new_account/img/register_alerts.png diff --git a/usage/accounts_user_profile/new_account/img/register_button.png b/docs/usage/accounts_user_profile/new_account/img/register_button.png similarity index 100% rename from usage/accounts_user_profile/new_account/img/register_button.png rename to docs/usage/accounts_user_profile/new_account/img/register_button.png diff --git a/usage/accounts_user_profile/new_account/img/register_form.png b/docs/usage/accounts_user_profile/new_account/img/register_form.png similarity index 100% rename from usage/accounts_user_profile/new_account/img/register_form.png rename to docs/usage/accounts_user_profile/new_account/img/register_form.png diff --git a/docs/usage/accounts_user_profile/new_account/index.md b/docs/usage/accounts_user_profile/new_account/index.md new file mode 100644 index 00000000..2269e591 --- /dev/null +++ b/docs/usage/accounts_user_profile/new_account/index.md @@ -0,0 +1,46 @@ +# Creating a New Account + +To take full advantage of all the GeoNode features you need a user account. Follow these step to create a new one. + +1. From any page in the web interface, you will see a `Register`{.interpreted-text role="guilabel"} link. Click that link, and the register form will appear + + > ::: note + > ::: title + > Note + > ::: + > + > The registrations in GeoNode must be open, in case you don\'t see the register link then it\'s not possible to register unless the administrator of the site does that for you. + > ::: + > + >
+ > img/register_button.png + >
Register Link
+ >
+ +2. On the next page, fill out the form. Enter a username and password in the fields. Also, enter your email address for verification. + +
+ img/register_form.png +
Registering for a new account
+
+ +3. You will be automatically logged in and redirected to the Profile page. An email will be sent confirming that you have signed up. If no errors occur during the registration, the following alerts will appear on the screen: + +
+ img/register_alerts.png +
Alerts
+
+ +To log out click on the `Log out`{.interpreted-text role="guilabel"} link of the user menu. + +
+img/logout_link.png +
Logout link
+
+ +You have to confirm this action as described in the picture below. + +>
+> img/confirm_logout.png +>
Confirm Log out
+>
diff --git a/docs/usage/accounts_user_profile/updating_profile/associating_email.md b/docs/usage/accounts_user_profile/updating_profile/associating_email.md new file mode 100644 index 00000000..c72302f2 --- /dev/null +++ b/docs/usage/accounts_user_profile/updating_profile/associating_email.md @@ -0,0 +1,22 @@ +# Associating your Account with an e-mail {#associating_emails} + +Your account is automatically associated with the e-mail that you used to register yourself on the platform. + +
+img/associate_email.png +
Accounts e-mail
+
+ +By clicking on `Associated e-mails`{.interpreted-text role="guilabel"} of the *Profile* page (see `editing-profile`{.interpreted-text role="ref"}), you will have the possibility to fill up a new e-mail address. Type it in the `E-mail`{.interpreted-text role="guilabel"} input filed then click on `Add E-mail`{.interpreted-text role="guilabel"} to perform a new association. + +
+img/associate_new_email.png +
New e-mail association
+
+ +You can make it primary if necessary, in order to receive the notification on this address. To do that, select the e-mail that you want, then click on `Make Primary`{.interpreted-text role="guilabel"}. + +
+img/primary_email.png +
Primary e-mail address
+
diff --git a/docs/usage/accounts_user_profile/updating_profile/connecting_with_social.md b/docs/usage/accounts_user_profile/updating_profile/connecting_with_social.md new file mode 100644 index 00000000..fdbd41fd --- /dev/null +++ b/docs/usage/accounts_user_profile/updating_profile/connecting_with_social.md @@ -0,0 +1,3 @@ +# Connecting your Account with Social Networks + +You currently have no social network accounts connected to this account. diff --git a/docs/usage/accounts_user_profile/updating_profile/editing_profile.md b/docs/usage/accounts_user_profile/updating_profile/editing_profile.md new file mode 100644 index 00000000..496d4843 --- /dev/null +++ b/docs/usage/accounts_user_profile/updating_profile/editing_profile.md @@ -0,0 +1,18 @@ +# Editing Profile Information + +Your *Profile* contains personal information such as your address, your telephone number, your organization and so on but it is empty by default at the beginning. + +Through the `Edit profile`{.interpreted-text role="guilabel"} button of the *Profile* page (see `editing-profile`{.interpreted-text role="ref"}) you can set your details, including your avatar. + +
+img/user_profile_update_info.png +
Updating Profile information
+
+ +| When finished, click `Update profile`{.interpreted-text role="guilabel"}. You will be redirected to the *Profile* page. +| A message will confirm the profile has been correctly updated. + +
+img/profile_update_confirm.png +
Updating Profile correctly finalized
+
diff --git a/usage/accounts_user_profile/updating_profile/img/associate_email.png b/docs/usage/accounts_user_profile/updating_profile/img/associate_email.png similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/associate_email.png rename to docs/usage/accounts_user_profile/updating_profile/img/associate_email.png diff --git a/usage/accounts_user_profile/updating_profile/img/associate_new_email.png b/docs/usage/accounts_user_profile/updating_profile/img/associate_new_email.png similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/associate_new_email.png rename to docs/usage/accounts_user_profile/updating_profile/img/associate_new_email.png diff --git a/usage/accounts_user_profile/updating_profile/img/change_password.png b/docs/usage/accounts_user_profile/updating_profile/img/change_password.png similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/change_password.png rename to docs/usage/accounts_user_profile/updating_profile/img/change_password.png diff --git a/usage/accounts_user_profile/updating_profile/img/change_password_confirm.png b/docs/usage/accounts_user_profile/updating_profile/img/change_password_confirm.png similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/change_password_confirm.png rename to docs/usage/accounts_user_profile/updating_profile/img/change_password_confirm.png diff --git a/usage/accounts_user_profile/updating_profile/img/favorites_resources.png b/docs/usage/accounts_user_profile/updating_profile/img/favorites_resources.png similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/favorites_resources.png rename to docs/usage/accounts_user_profile/updating_profile/img/favorites_resources.png diff --git a/usage/accounts_user_profile/updating_profile/img/notifications_settings.png b/docs/usage/accounts_user_profile/updating_profile/img/notifications_settings.png similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/notifications_settings.png rename to docs/usage/accounts_user_profile/updating_profile/img/notifications_settings.png diff --git a/usage/accounts_user_profile/updating_profile/img/notifications_settings1.png b/docs/usage/accounts_user_profile/updating_profile/img/notifications_settings1.png similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/notifications_settings1.png rename to docs/usage/accounts_user_profile/updating_profile/img/notifications_settings1.png diff --git a/usage/accounts_user_profile/updating_profile/img/primary_email.png b/docs/usage/accounts_user_profile/updating_profile/img/primary_email.png similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/primary_email.png rename to docs/usage/accounts_user_profile/updating_profile/img/primary_email.png diff --git a/usage/accounts_user_profile/updating_profile/img/profile_link.png b/docs/usage/accounts_user_profile/updating_profile/img/profile_link.png similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/profile_link.png rename to docs/usage/accounts_user_profile/updating_profile/img/profile_link.png diff --git a/usage/accounts_user_profile/updating_profile/img/profile_update_confirm.png b/docs/usage/accounts_user_profile/updating_profile/img/profile_update_confirm.png similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/profile_update_confirm.png rename to docs/usage/accounts_user_profile/updating_profile/img/profile_update_confirm.png diff --git a/usage/accounts_user_profile/updating_profile/img/user_activities.jpg b/docs/usage/accounts_user_profile/updating_profile/img/user_activities.jpg similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/user_activities.jpg rename to docs/usage/accounts_user_profile/updating_profile/img/user_activities.jpg diff --git a/usage/accounts_user_profile/updating_profile/img/user_profile_page.jpg b/docs/usage/accounts_user_profile/updating_profile/img/user_profile_page.jpg similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/user_profile_page.jpg rename to docs/usage/accounts_user_profile/updating_profile/img/user_profile_page.jpg diff --git a/usage/accounts_user_profile/updating_profile/img/user_profile_update_info.png b/docs/usage/accounts_user_profile/updating_profile/img/user_profile_update_info.png similarity index 100% rename from usage/accounts_user_profile/updating_profile/img/user_profile_update_info.png rename to docs/usage/accounts_user_profile/updating_profile/img/user_profile_update_info.png diff --git a/docs/usage/accounts_user_profile/updating_profile/index.md b/docs/usage/accounts_user_profile/updating_profile/index.md new file mode 100644 index 00000000..c8714357 --- /dev/null +++ b/docs/usage/accounts_user_profile/updating_profile/index.md @@ -0,0 +1,49 @@ +# Updating the Profile {#editing-profile} + +Once having an account you can enrich your profile with useful information, you can also edit or delete the existing ones. +You can connect the account with your social network, associate many e-mail addresses to it and manage many options such as preferences about notifications. + +You can update these information anytime from your *Profile* page which is accessible from the user menu. + +So, click on your profile picture in the top right of the screen. A drop-down list will show. Click on `Profile`{.interpreted-text role="guilabel"} to enter the *Profile* settings page. + +
+img/profile_link.png +
Link to your profile
+
+ +The *Profile* page looks like the one shown in the picture below. + +
+img/user_profile_page.jpg +
User profile page
+
+ +Your personal information is shown under the username. At the bottom of the page are listed all the resources associated to your *Profile*, you can decide to view only datasets, maps ,documents, dashboard or geostory by clicking on the corresponding tab. + +Through the link `User datasets WMS GetCapabilities document` you can retrieve an XML document with the list of the available datasets. + +On the right side of the page there are many useful links to edit personal information to update your *Profile* settings and to get in touch with other GeoNode users. + +The `My Activities`{.interpreted-text role="guilabel"} link allows to see all your recent activities on GeoNode such as datasets uploading and maps creation. + +
+img/user_activities.jpg +
Recent activities
+
+ +All other links and their functionalities will be described in depth in the following sections. + +::: {.toctree maxdepth="1"} +editing_profile +associating_email +managing_password +notification_settings +::: + + +- [editing_profile](editing_profile.md) +- [associating_email](associating_email.md) +- [managing_password](managing_password.md) +- [notification_settings](notification_settings.md) + diff --git a/docs/usage/accounts_user_profile/updating_profile/managing_password.md b/docs/usage/accounts_user_profile/updating_profile/managing_password.md new file mode 100644 index 00000000..a6d70c17 --- /dev/null +++ b/docs/usage/accounts_user_profile/updating_profile/managing_password.md @@ -0,0 +1,18 @@ +# Managing the Password + +To change your password, click on the `Set/Change password`{.interpreted-text role="guilabel"} link of the *Profile* page (see `editing-profile`{.interpreted-text role="ref"}). +You will be asked to enter your current password and the new one (two times). Click on `Change my password`{.interpreted-text role="guilabel"} to perform the change. + +
+img/change_password.png +
Change your password
+
+ +If no errors occur you will see a confirmation message. + +
+img/change_password_confirm.png +
Change password confirmation
+
+ +Next time you signin, you will have to use the new password. diff --git a/docs/usage/accounts_user_profile/updating_profile/notification_settings.md b/docs/usage/accounts_user_profile/updating_profile/notification_settings.md new file mode 100644 index 00000000..1e3ba521 --- /dev/null +++ b/docs/usage/accounts_user_profile/updating_profile/notification_settings.md @@ -0,0 +1,21 @@ +# Setting Notification Preferences + +By default GeoNode sends notifications to the users for events that they could be subscribed to such as a new dataset uploaded or a new rate added to a map. +You can adjust your notification settings by clicking on the `Notifications`{.interpreted-text role="guilabel"} link of the *Profile* page (see `editing-profile`{.interpreted-text role="ref"}). + +::: note +::: title +Note +::: + +Make sure to have a verified email address to which notices can be sent. If not see `associating_emails`{.interpreted-text role="ref"}. +::: + +Now check/uncheck the notification types you wish to receive or not receive. It is possible to be notified for the events shown in the picture below. + +![](img/notifications_settings.png){.align-center} + +
+img/notifications_settings1.png +
Notifications settings
+
diff --git a/usage/dashboard/img/create_dashboard.png b/docs/usage/dashboard/img/create_dashboard.png similarity index 100% rename from usage/dashboard/img/create_dashboard.png rename to docs/usage/dashboard/img/create_dashboard.png diff --git a/usage/dashboard/img/create_dashboard_link.png b/docs/usage/dashboard/img/create_dashboard_link.png similarity index 100% rename from usage/dashboard/img/create_dashboard_link.png rename to docs/usage/dashboard/img/create_dashboard_link.png diff --git a/usage/dashboard/img/new_dashboard.png b/docs/usage/dashboard/img/new_dashboard.png similarity index 100% rename from usage/dashboard/img/new_dashboard.png rename to docs/usage/dashboard/img/new_dashboard.png diff --git a/docs/usage/dashboard/index.md b/docs/usage/dashboard/index.md new file mode 100644 index 00000000..2a8db9e3 --- /dev/null +++ b/docs/usage/dashboard/index.md @@ -0,0 +1,32 @@ +# Dashboard + +Dashboard is a MapStore tool integrated in GeoNode that provides the user with a space to add many Widgets, such as charts, maps, tables, texts and counters, and can create connections between them in order to: + +- Provide an overview to better visualize a specific data context +- Interact spatially and analytically with the data by creating connections between widgets +- Perform analysis on involved data/layers + +To build a new Dashboard go to `Add Resource`{.interpreted-text role="guilabel"} option on the *All Resources* page and choose option *Create dashboard* or select `New`{.interpreted-text role="guilabel"} option on the *Dashboards* page + +>
+> img/create_dashboard_link.png +>
Create dashboard from All Resources page
+>
+ +
+img/create_dashboard.png +
Create dashboard from Dashboards page
+
+ +Now you landed on the Dashboard edition page that is composed of the following sections: + +
+https://mapstore.readthedocs.io/en/latest/user-guide/img/exploring-dashboards/dashboard-1.jpg +
New Dashboard Apps option
+
+ +## Further Reading + +Follow the link below to get more detailed information about the usage of Dashboard. + +[Dashboard Documentation](https://mapstore.readthedocs.io/en/latest/user-guide/exploring-dashboards) diff --git a/usage/data/data_types.rst b/docs/usage/data/data_types.md similarity index 68% rename from usage/data/data_types.rst rename to docs/usage/data/data_types.md index 5610efe9..137ddc35 100644 --- a/usage/data/data_types.rst +++ b/docs/usage/data/data_types.md @@ -1,32 +1,34 @@ -.. _data-types: - -Data Types -========== +# Data Types | GeoNode welcome page shows a variety of information about the current GeoNode instance. -| You can explore the existing data using many search tools and filters (see :ref:`finding-data`) or through the links of the navigation bar at the top of the page. +| You can explore the existing data using many search tools and filters (see `finding-data`{.interpreted-text role="ref"}) or through the links of the navigation bar at the top of the page. | There are five main types of resources that GeoNode can manage: -#. Datasets -#. Maps -#. Documents -#. GeoStories -#. Dashboards - +1. Datasets +2. Maps +3. Documents +4. GeoStories +5. Dashboards -Each resource type has its own menu and can be reached through :guilabel:`Datasets`, :guilabel:`Maps`, :guilabel:`Documents`, :guilabel:`GeoStories` and :guilabel:`Dashboards` buttons on the navigation bar +Each resource type has its own menu and can be reached through `Datasets`{.interpreted-text role="guilabel"}, `Maps`{.interpreted-text role="guilabel"}, `Documents`{.interpreted-text role="guilabel"}, `GeoStories`{.interpreted-text role="guilabel"} and `Dashboards`{.interpreted-text role="guilabel"} buttons on the navigation bar -.. note:: :guilabel:`GeoStories` and :guilabel:`Dashboards` In the screenshot below are GeoApp resouce types which are added by the client. +::: note +::: title +Note +::: -.. figure:: img/data_menu.png - :align: center +`GeoStories`{.interpreted-text role="guilabel"} and `Dashboards`{.interpreted-text role="guilabel"} In the screenshot below are GeoApp resouce types which are added by the client. +::: - *Data menu* - -.. _document-data-type: +
+img/data_menu.png +
Data menu
+
+::: {#document-data-type} Datasets ------- +\-\-\-\-\-- +::: | Datasets are a primary component of GeoNode. | Datasets are publishable resources representing a raster or vector spatial data source. Datasets also can be associated with metadata, ratings, and comments. @@ -34,34 +36,32 @@ Datasets | By clicking the Datasets link you will get a list of all published datasets. If logged in as an administrator, you will also see the unpublished datasets in the same list. | GeoNode allows the user to upload vector and raster data in their original projections using a web form. -Vector data can be uploaded in many different formats (ESRI Shapefile, KML and so on...). Satellite imagery and other kinds of raster data can be uploaded as GeoTIFFs. +Vector data can be uploaded in many different formats (ESRI Shapefile, KML and so on\...). Satellite imagery and other kinds of raster data can be uploaded as GeoTIFFs. -Maps ----- +## Maps | Maps are a primary component of GeoNode. | Maps are comprised of various datasets and their styles. Datasets can be both local datasets in GeoNode as well as remote datasets either served from other WMS servers or by web service datasets such as Google or MapQuest. + GeoNode maps also contain other information such as map zoom and extent, dataset ordering, and style. You can create a map based on uploaded datasets, combine them with some existing datasets and a remote web service dataset, share the resulting map for public viewing. Once the data has been uploaded, GeoNode lets the user search for it geographically or via keywords and create maps. -All the datasets are automatically reprojected to web mercator for maps display, making it possible to use popular base maps such as `OpenStreetMap `_. +All the datasets are automatically reprojected to web mercator for maps display, making it possible to use popular base maps such as [OpenStreetMap](https://www.openstreetmap.org). -Documents ---------- +## Documents | GeoNode allows to publish tabular and text data and to manage metadata and associated documents. -| Documents can be uploaded directly from your disk (see :ref:`uploading-documents` for further information). -| The following documents types are allowed: `txt, .log, .doc, .docx, .ods, .odt, .sld, .qml, .xls, .xlsx, .xml, .bm, .bmp, .dwg, .dxf, .fif, .gif, .jpg, .jpe, .jpeg, .png, .tif, .tiff, .pbm, .odp, .ppt, .pptx, .pdf, .tar, .tgz, .rar, .gz, .7z, .zip, .aif, .aifc, .aiff, .au, .mp3, .mpga, .wav, .afl, .avi, .avs, .fli, .mp2, .mp4, .mpg, .ogg, .webm, .3gp, .flv, .vdo, .glb, .pcd, .gltf`. +| Documents can be uploaded directly from your disk (see `uploading-documents`{.interpreted-text role="ref"} for further information). +| The following documents types are allowed: [txt, .log, .doc, .docx, .ods, .odt, .sld, .qml, .xls, .xlsx, .xml, .bm, .bmp, .dwg, .dxf, .fif, .gif, .jpg, .jpe, .jpeg, .png, .tif, .tiff, .pbm, .odp, .ppt, .pptx, .pdf, .tar, .tgz, .rar, .gz, .7z, .zip, .aif, .aifc, .aiff, .au, .mp3, .mpga, .wav, .afl, .avi, .avs, .fli, .mp2, .mp4, .mpg, .ogg, .webm, .3gp, .flv, .vdo, .glb, .pcd, .gltf]{.title-ref}. | Through the document detailed page is possible to view, download and manage a document. GeoStories ---------- +\-\-\-\-\-\-\-\-- | GeoStory is a MapStore tool integrated in GeoNode that provides the user a way to create inspiring and immersive stories by combining text, interactive maps, and other multimedia content like images and video or other third party contents. Through this tool you can simply tell your stories on the web and then publish and share them with different groups of GeoNode users or make them public to everyone around the world. -Dashboard ---------- +## Dashboard | Dashboard is a MapStore tool integrated in GeoNode that provides the user with a space to add many Widgets, such as charts, maps, tables, texts and counters, and can create connections between them in order to: | - Provide an overview to better visualize a specific data context diff --git a/usage/data/finding_data.rst b/docs/usage/data/finding_data.md similarity index 59% rename from usage/data/finding_data.rst rename to docs/usage/data/finding_data.md index 3d76b117..24fe0423 100644 --- a/usage/data/finding_data.rst +++ b/docs/usage/data/finding_data.md @@ -1,45 +1,41 @@ -.. _finding-data: - -Finding Data -============ +# Finding Data | This section will guide you to navigate GeoNode to find datasets, maps and documents and other resource types by using different routes, filters and search functions. | On every page you can find some quick search tool. The *Search* box in the navigation bar (see the picture below) let you type a text and find all the resource which have to deal with that text. -.. figure:: img/search_tool.png - :align: center - - *Search tool in GeoNode welcome page* +
+img/search_tool.png +
Search tool in GeoNode welcome page
+
When you trigger a search you are brought to the *Search* page which shows you the search result through all data types. -.. figure:: img/search_page.png - :align: center - - *The Search page* +
+img/search_page.png +
The Search page
+
| This page contains a wealth of options for customizing a search for various information on GeoNode. This search form allows for much more fine-tuned searches than the simple search box is available at the top of every page. | It is possible to search and filter data by Text, Types, Categories, Keywords, Owners, Regions, Group, Limitations on public access, Date and Extent. Try to set some filter and see how the resulting data list changes accordingly. An interesting type of filter is *EXTENT*: you can apply a spatial filter by moving or zooming a map within a box as shown in the picture below. -.. figure:: img/search_filter_by_extent.png - :align: center - - *Search filter by EXTENT* +
+img/search_filter_by_extent.png +
Search filter by EXTENT
+
Data can be ordered by Most recent, Less recent, Name and Popularity. -.. figure:: img/ordering_link.png - :align: center - *Ordering Link* - -.. figure:: img/ordering_data.png - :align: center +![](img/ordering_link.png){.align-center} - *Ordering Data* +> *Ordering Link* +
+img/ordering_data.png +
Ordering Data
+
-For *Users* see :ref:`user-info`. +For *Users* see `user-info`{.interpreted-text role="ref"}. diff --git a/usage/data/img/data_menu.png b/docs/usage/data/img/data_menu.png similarity index 100% rename from usage/data/img/data_menu.png rename to docs/usage/data/img/data_menu.png diff --git a/usage/data/img/ordering_data.png b/docs/usage/data/img/ordering_data.png similarity index 100% rename from usage/data/img/ordering_data.png rename to docs/usage/data/img/ordering_data.png diff --git a/usage/data/img/ordering_link.png b/docs/usage/data/img/ordering_link.png similarity index 100% rename from usage/data/img/ordering_link.png rename to docs/usage/data/img/ordering_link.png diff --git a/usage/data/img/search_filter_by_extent.png b/docs/usage/data/img/search_filter_by_extent.png similarity index 100% rename from usage/data/img/search_filter_by_extent.png rename to docs/usage/data/img/search_filter_by_extent.png diff --git a/usage/data/img/search_page.png b/docs/usage/data/img/search_page.png similarity index 100% rename from usage/data/img/search_page.png rename to docs/usage/data/img/search_page.png diff --git a/usage/data/img/search_tool.png b/docs/usage/data/img/search_tool.png similarity index 100% rename from usage/data/img/search_tool.png rename to docs/usage/data/img/search_tool.png diff --git a/usage/data/index.rst b/docs/usage/data/index.md similarity index 83% rename from usage/data/index.rst rename to docs/usage/data/index.md index b20d09ae..c620e541 100644 --- a/usage/data/index.rst +++ b/docs/usage/data/index.md @@ -1,7 +1,4 @@ -.. _data: - -Data -==== +# Data Data management tools built into GeoNode allow integrated creation of resources eg datasets, documents, link to external documents, map visualizations and other configured geonode apps. Each resource in the system can be shared publicly or restricted to allow access to only specific users. @@ -9,8 +6,12 @@ Social features like user profiles and commenting and rating systems allow for t The following sections will explain more in depth what data can be managed in GeoNode and how to easily find that data. -.. toctree:: - :maxdepth: 1 +::: {.toctree maxdepth="1"} +data_types +finding_data +::: + + +- [data_types](data_types.md) +- [finding_data](finding_data.md) - data_types - finding_data diff --git a/docs/usage/download_handlers/index.md b/docs/usage/download_handlers/index.md new file mode 100644 index 00000000..cd5c5989 --- /dev/null +++ b/docs/usage/download_handlers/index.md @@ -0,0 +1,29 @@ +# Download Handlers {#data} + +With GeoNode 4.2.x has been introduced the concept of Download Handler and ofc GeoNode provides a default implementation of it which process the download via WPS + +Follow an example of how to create a custom download handler and to replace the default one or add an additional one. + +# Settings + +## DEFAULT_DATASET_DOWNLOAD_HANDLER + +> Default: `geonode.layers.download_handler.DatasetDownloadHandler` +> +> path to the download handler location + +## DATASET_DOWNLOAD_HANDLERS + +> Default: `[]` +> +> List of paths of the additional download handlers + +# CODE + +The default download handler is placed under the [geonode.layers]{.title-ref} \[package\]() + +Follow an example of an basic class for define the download handler: + +`` `python class DatasetDownloadHandler: def __str__(self): return f"{self.__module__}.{self.__class__.__name__}" def __repr__(self): return self.__str__() def __init__(self, request, resource_name) -> None: self.request = request self.resource_name = resource_name self._resource = None def get_download_response(self): """ Main method used, this method should return the response object """ return response @property def download_url(self): """ Used by the API, it should return the URL where the resource can be downloaded from """ return reverse("dataset_download", args=[resource.alternate]) ``\` + +If you prefer to inherit from the already existing one, the response is generated in the [process_dowload]{.title-ref} method diff --git a/docs/usage/extra_metadata/api.md b/docs/usage/extra_metadata/api.md new file mode 100644 index 00000000..0adbdbd1 --- /dev/null +++ b/docs/usage/extra_metadata/api.md @@ -0,0 +1,127 @@ +# Rest API {#data} + +The [api/v2/resources]{.title-ref} endpoint provide different ways to handle the metadata. + +**GET:** + +Get the metadata list of the selected resource + +``` python +URL: http://host/api/v2/resources/{resource_id}/extra_metadata + +Response: + +[ + { + "id": 1, + "field_name": "bike", + "field_label": "KTM", + "field_value": "ktm", + "filter_header": "Sports Parameters" + } +] +``` + +**POST:** + +Adding new metadata to the resource + +``` python +URL: http://host/api/v2/resources/{resource_id}/extra_metadata +data = [ + { + "field_name": "bike", + "field_label": "KTM", + "field_value": "ktm", + "filter_header": "Sports Parameters" + } +] + +Response: +status_code: 201 +response json: List of the available metadata for the resource +[ + { + "id": 1, + "field_name": "bike", + "field_label": "KTM", + "field_value": "ktm", + "filter_header": "Sports Parameters" + } +] +``` + +**PUT:** + +Update specific metadata for the selected resource. In this case the metadata **ID** is required to perform the update + +``` python +http://host/api/v2/resources/{resource_id}/extra_metadata +payload: +[ + { + "id": 1, + "field_name": "bike", + "field_label": "KTM - sport", <- this value need to be updated + "field_value": "ktm", + "filter_header": "Sports Parameters" + } +] + +Response: +status_code: 200 +response: the available payload for the selected resource +[ + { + "id": 1, + "field_name": "bike", + "field_label": "KTM - sport", + "field_value": "ktm", + "filter_header": "Sports Parameters" + } +] +``` + +**DELETE:** + +Delete the metadata for a given resource by *ID*. + +``` python +http://host/api/v2/resources/{resource_id}/extra_metadata +payload: list of ID to be deleted +[ + 1, 2, 3, 4, 5 +] + + + +Response: +status_code: 200 +response: List of the available metadata + +[] +``` + +**API search** + +Is possible to search for resources with specific metadata. This feature is available for both API v1 and API v2 + +APIv1: + +To perform the search is enough to add as query parameters the field of the metadata payload. + +Assuming that the payload is the same as the example above, the URL could be something like the following: + +[http://host/api/base/?metadata\_\_field_category=bike]{.title-ref} + +In this way, we can retrieve all the resources that have at least 1 metadata with the [field_category = \'bike\']{.title-ref} + +APIv2 + +For the API v2 is a bit different since the library doesnt have a support for the JSON field. + +To reproduce the same search above, we need to call a URL like the following one: + +[http://localhost:8000/api/v2/resources?filter{metadata.metadata.icontains}=%22field_category%22:%20%22bike%22]{.title-ref} + +In this way, we can retrieve all the resources that have at least 1 metadata with the [field_category = \'bike\']{.title-ref} diff --git a/docs/usage/extra_metadata/edit.md b/docs/usage/extra_metadata/edit.md new file mode 100644 index 00000000..c7799476 --- /dev/null +++ b/docs/usage/extra_metadata/edit.md @@ -0,0 +1,79 @@ +# Metadata manipulation {#data} + +There are two possible ways to manipulate extra metadata in geonode: + +- via Metadata Editor (Wizard and advanced) +- via Rest API + +## Metadata Editor (wizard/advanced): + +The metadata section is placed under the OPTIONAL METADATA section available for all the GeoNode resources. + +The metadata must follow two specific rules to save to the resource: + +- Must always be a list of JSON. This permits to add of more than one metadata for each resource +- The JSON must follow the schema defined in the [settings.py]{.title-ref} for the selected resource. + +For example, for my document resource, I can have something like the following: + +
+img/wizard.png +
Advanced edit wizard menù
+
+ +After pressing the save button, the system will perform the following checks: + +- Check if the text provided is a valid JSON. In case of wrong format input, the following error is shown: + +
+img/invalid_json.png +
invalid JSON error
+
+ +- Check if the metadata schema is provided for the resource if not will raise the following error + +
+img/missing_schema.png +
missing schema error
+
+ +- Check if the metadata schema is coherent with the schema defined in the settings. In case of wrong format input, the error will print the missing JSON keys + +
+img/invalid_schema.png +
invalid schema error
+
+ +## Facet Filtering + +Automatically the web interface will create dynamically the facets if there is at least 1 metadata defined for the resource. + +Suppose that a resource heve the following metadata: + +``` json +[ + { + "field_name": "bike", + "field_label": "KTM", + "field_value": "ktm", + "filter_header": "Sports Parameters" + }, + { + "field_name": "mountain", + "field_label": "Monte bianco", + "field_value": "monte_bianco", + "filter_header": "Italian Mountains" + } +] +``` + +By default GeoNode will convert this metadata info in facets available for the resource + +The facet will convert: +- \`filter_header\`: used as the header filter +- \`field_value\`: used to perform the search +- \`field_name\`: used for calculate the unique values (along with [field_value]{.title-ref}) + +After says that, the facet will be the follow: + +![](img/facet.png){.align-center} diff --git a/usage/extra_metadata/img/facet.png b/docs/usage/extra_metadata/img/facet.png old mode 100755 new mode 100644 similarity index 100% rename from usage/extra_metadata/img/facet.png rename to docs/usage/extra_metadata/img/facet.png diff --git a/usage/extra_metadata/img/invalid_json.png b/docs/usage/extra_metadata/img/invalid_json.png old mode 100755 new mode 100644 similarity index 100% rename from usage/extra_metadata/img/invalid_json.png rename to docs/usage/extra_metadata/img/invalid_json.png diff --git a/usage/extra_metadata/img/invalid_schema.png b/docs/usage/extra_metadata/img/invalid_schema.png old mode 100755 new mode 100644 similarity index 100% rename from usage/extra_metadata/img/invalid_schema.png rename to docs/usage/extra_metadata/img/invalid_schema.png diff --git a/usage/extra_metadata/img/missing_schema.png b/docs/usage/extra_metadata/img/missing_schema.png old mode 100755 new mode 100644 similarity index 100% rename from usage/extra_metadata/img/missing_schema.png rename to docs/usage/extra_metadata/img/missing_schema.png diff --git a/usage/extra_metadata/img/wizard.png b/docs/usage/extra_metadata/img/wizard.png old mode 100755 new mode 100644 similarity index 100% rename from usage/extra_metadata/img/wizard.png rename to docs/usage/extra_metadata/img/wizard.png diff --git a/usage/extra_metadata/index.rst b/docs/usage/extra_metadata/index.md similarity index 53% rename from usage/extra_metadata/index.rst rename to docs/usage/extra_metadata/index.md index a4f1d966..d889670a 100644 --- a/usage/extra_metadata/index.rst +++ b/docs/usage/extra_metadata/index.md @@ -1,13 +1,15 @@ -.. _data: - -Dynamic Extra Metadata -====================== +# Dynamic Extra Metadata {#data} In GeoNode is possible to add metadata to each resource dynamically without extending the base model provided by the application using the extra metadata field. -.. toctree:: - :maxdepth: 1 +::: {.toctree maxdepth="1"} +settings +edit +api +::: + + +- [settings](settings.md) +- [edit](edit.md) +- [api](api.md) - settings - edit - api diff --git a/docs/usage/extra_metadata/settings.md b/docs/usage/extra_metadata/settings.md new file mode 100644 index 00000000..0787dfa7 --- /dev/null +++ b/docs/usage/extra_metadata/settings.md @@ -0,0 +1,40 @@ +# Settings + +Three main settings control the extra metadata field: + +DEFAULT_EXTRA_METADATA_SCHEMA: define the schema used to store the metadata + +- \`id\`: (optional int): the identifier of the metadata. Optional for creation, required in the Upgrade phase + +- \`filter_header\`: (required object): Can be any type, is used to generate the facet filter header. Is also an identifier. + +- \`field_name\`: (required object): name of the metadata field + +- \`field_label\`: (required object): a verbose string of the name. Is used as a label in the facet filters. + +- \`field_value\`: (required object): metadata values + + > An example of metadata that can be ingested is the following: + > + > ``` json + > [ + > { + > "filter_header": "Bike Brand", + > "field_name": "name", + > "field_label": "Bike Name", + > "field_value": "KTM", + > }, + > { + > "filter_header": "Bike Brand", + > "field_name": "name", + > "field_label": "Bike Name", + > "field_value": "Bianchi", + > } + > ] + > ``` + +The above schema is valid by using the [schema]{.title-ref} \<\> + +CUSTOM_METADATA_SCHEMA: environment variable used to inject additional schema to the default one. Helpful for third-party libraries + +EXTRA_METADATA_SCHEMA: used to get the expected metadata schema for each resource_type. diff --git a/usage/geostory/img/add_media.png b/docs/usage/geostory/img/add_media.png similarity index 100% rename from usage/geostory/img/add_media.png rename to docs/usage/geostory/img/add_media.png diff --git a/usage/geostory/img/create_geostory.png b/docs/usage/geostory/img/create_geostory.png similarity index 100% rename from usage/geostory/img/create_geostory.png rename to docs/usage/geostory/img/create_geostory.png diff --git a/usage/geostory/img/create_geostory_link.png b/docs/usage/geostory/img/create_geostory_link.png similarity index 100% rename from usage/geostory/img/create_geostory_link.png rename to docs/usage/geostory/img/create_geostory_link.png diff --git a/usage/geostory/img/new_geostory.png b/docs/usage/geostory/img/new_geostory.png similarity index 100% rename from usage/geostory/img/new_geostory.png rename to docs/usage/geostory/img/new_geostory.png diff --git a/usage/geostory/img/save_geostory.png b/docs/usage/geostory/img/save_geostory.png similarity index 100% rename from usage/geostory/img/save_geostory.png rename to docs/usage/geostory/img/save_geostory.png diff --git a/docs/usage/geostory/index.md b/docs/usage/geostory/index.md new file mode 100644 index 00000000..c31cd7d0 --- /dev/null +++ b/docs/usage/geostory/index.md @@ -0,0 +1,65 @@ +# GeoStory + +GeoStory is a MapStore tool integrated in GeoNode that provides the user a way to create inspiring and immersive stories by combining text, interactive maps, and other multimedia content like images and video or other third party contents. +Through this tool you can simply tell your stories on the web and then publish and share them with different groups of GeoNode users or make them public to everyone around the world. + +To build a new GeoStory go to `Add Resource`{.interpreted-text role="guilabel"} option on the *All Resources* page and choose option *Create geostory* or select `New`{.interpreted-text role="guilabel"} option on the *Geostories* page + +>
+> img/create_geostory_link.png +>
Create geostory from All Resources page
+>
+ +
+img/create_geostory.png +
Create geostory from Geostories page
+
+ +Now you landed on the GeoStory edition page that is composed of the following sections: + +
+https://mapstore.readthedocs.io/en/latest/user-guide/img/exploring-stories/story-workspace.jpg +
New GeoStory Apps option
+
+ +| The GeoStory content is organized in Sections, that can be added with the ![add_section](https://mapstore.readthedocs.io/en/latest/user-guide/img/button/add-section.jpg){.align-middle width="30px" height="30px"} button in the *Container* area. In particular, the user can add to the story the following kind of sections: + +
+https://mapstore.readthedocs.io/en/latest/user-guide/img/exploring-stories/sections.jpg +
GeoStory Sections Types
+
+ +For more information on these specific section please follow the official MapStore documentation: + +- [Title Section](https://mapstore.readthedocs.io/en/latest/user-guide/title-section/) +- [Banner Section](https://mapstore.readthedocs.io/en/latest/user-guide/banner-section/) +- [Paragraph Section](https://mapstore.readthedocs.io/en/latest/user-guide/paragraph-section/) +- [Immersive Section](https://mapstore.readthedocs.io/en/latest/user-guide/immersive-section/) +- [Media Section](https://mapstore.readthedocs.io/en/latest/user-guide/media-section/) +- [Web Page Section](https://mapstore.readthedocs.io/en/latest/user-guide/web-section/) + +## Add GeoNode content to GeoStory + +With GeoNode you can add content to your GeoStory using internal GeoNode documents and maps as well external sources. +This ability to add internal GeoNode content makes the GeoStory creation a very usefull feature. + +| To add GeoNode content to your GeoStory use the ![edit_mode](https://mapstore.readthedocs.io/en/latest/user-guide/img/button/edit-icon-1.jpg){.align-middle width="30px" height="30px"} button on top of your GeoStory section. + +From here you can add `Images`{.interpreted-text role="guilabel"}, `Videos`{.interpreted-text role="guilabel"} and `Maps`{.interpreted-text role="guilabel"}. +To enable GeoNode internal catalog, on `Services`{.interpreted-text role="guilabel"} dropdown choose `GeoNode`{.interpreted-text role="guilabel"} as shown in picture down. +On the left you get a list of media documents available with a complementary text filter feature on top. + +
+img/add_media.png +
Add Media to GeoStory
+
+ +To save your GeoStory, on the top your Geostory content choose `Save`{.interpreted-text role="guilabel"} and then `Save as...`{.interpreted-text role="guilabel"} + +Now your GeoStory can be shared with everyone! + +## Further Reading + +Follow the link below to get more detailed information about the usage of GeoStory. + +[GeoStory Documentation](https://mapstore.readthedocs.io/en/latest/user-guide/exploring-stories) diff --git a/docs/usage/index.md b/docs/usage/index.md new file mode 100644 index 00000000..a20104ee --- /dev/null +++ b/docs/usage/index.md @@ -0,0 +1,31 @@ +# GeoNode Users Guide + +::: {.toctree maxdepth="2"} +accounts_user_profile/index +users_groups/index +data/index +managing_documents/index +managing_datasets/index +managing_maps/index +dashboard/index +geostory/index +publishing_data/index +other_apps/index +extra_metadata/index +download_handlers/index +::: + + +- [index](accounts_user_profile/index.md) +- [index](users_groups/index.md) +- [index](data/index.md) +- [index](managing_documents/index.md) +- [index](managing_datasets/index.md) +- [index](managing_maps/index.md) +- [index](dashboard/index.md) +- [index](geostory/index.md) +- [index](publishing_data/index.md) +- [index](other_apps/index.md) +- [index](extra_metadata/index.md) +- [index](download_handlers/index.md) + diff --git a/usage/managing_datasets/dataset_advanced.rst b/docs/usage/managing_datasets/dataset_advanced.md similarity index 57% rename from usage/managing_datasets/dataset_advanced.rst rename to docs/usage/managing_datasets/dataset_advanced.md index 9586cf9b..44ce77d8 100644 --- a/usage/managing_datasets/dataset_advanced.rst +++ b/docs/usage/managing_datasets/dataset_advanced.md @@ -1,13 +1,13 @@ -.. _dataset-advanced: - +::: {#dataset-advanced} Advanced Dataset Management with MapStore ======================================= +::: GeoNode provides the user with some advanced features for dataset manipulation such as dataset filtering, attribute edition and dataset export in different formats. In a nutshell, these feature are provided via MapStore and we will redirect the user to MapStore specific documentation for further details. Filtering Datasets ----------------- +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-- -With GeoNode you can filter a dataset via it's attributes, direct map filter by drawing an area of interest over the map canvas and via cross-dataset filter, allowing intersection, contained and contains overlay methods. -For more detail please check the MapStore documentation `here `_. +With GeoNode you can filter a dataset via it\'s attributes, direct map filter by drawing an area of interest over the map canvas and via cross-dataset filter, allowing intersection, contained and contains overlay methods. +For more detail please check the MapStore documentation [here](https://mapstore.readthedocs.io/en/latest/user-guide/filtering-layers/). diff --git a/docs/usage/managing_datasets/dataset_download.md b/docs/usage/managing_datasets/dataset_download.md new file mode 100644 index 00000000..ead635a3 --- /dev/null +++ b/docs/usage/managing_datasets/dataset_download.md @@ -0,0 +1,86 @@ +::: {#dataset-download} +Downloading Datasets +================== +::: + +At the top of the *Dataset Menu* there is an `Download`{.interpreted-text role="guilabel"} link where it is possible to download: + +- Dataset +- ISO Metadata +- Dublin Core Metadata + +Clicking the `ISO Metadata`{.interpreted-text role="guilabel"} or the `Dublin Core Metadata`{.interpreted-text role="guilabel"} exports the layer metadata in [xml]{.title-ref} format. + +
+img/export_metadata.png +
Downloading Metadata
+
+ +Open the `Dataset`{.interpreted-text role="guilabel"} option you will be able to select from a list of options of the supported export file formats. + +
+img/export_dataset_form.png +
Downloading Datasets
+
+ +As shown in the image above, GeoNode allows you to download a subset of data. Click on `Download filtered dataset`{.interpreted-text role="guilabel"} to download filtered data. + +On clicking Export, the file is prepared and a notification is showed when file is ready + +
+img/export_notification.png +
Export Ready
+
+ +To download the file to your machine, click on the export dataset icon. This opens the prepared export files and you can save the files on your by clicking on the save icon on each item. + +
+img/export_icon.png +
Export Results Icon
+
+ +![](img/export_list.png){.align-center} + +# Printing + +| The [MapStore](https://mapstore2.geo-solutions.it/mapstore/#/) based map viewer of GeoNode allows you to print the current view with a customizable layout. +| Click the `Print`{.interpreted-text role="guilabel"} option from the *Menu*, the **Printing Window** will open. + +
+img/printing_window.png +
The Printing Window
+
+ +From this window you can: + +- enter *Title* and *Description*; +- choose the *Resolution* in dpi; +- select the format +- select the coordinate +- add the scale +- add grid with label +- customize the *Layout* + - the *Sheet size* (A3, A4); + - if include the legend or not; + - if to put the legend in a separate page; + - the page *Orientation* (Landscape or Portrait); +- customize the *Legend* + - the *Label Font*; + - the *Font Size*; + - the *Font Emphasis* (bold, italic); + - if *Force Labels*; + - if use *Anti Aliasing Font*; + - the *Icon Size*; + - the *Legend Resolution* in dpi. + +To print the view click on `Print`{.interpreted-text role="guilabel"}. + +# Performing Measurements + +Click on the `Measure`{.interpreted-text role="guilabel"} option of the *Menu* to perform a measurement. +As you can see in the picture below, this tool allows you to measure *Distances*, *Areas* and the *Bearing* of lines. + +
+img/measuring.png +
The Measure Tool
+
diff --git a/docs/usage/managing_datasets/dataset_editing.md b/docs/usage/managing_datasets/dataset_editing.md new file mode 100644 index 00000000..4cc9dc32 --- /dev/null +++ b/docs/usage/managing_datasets/dataset_editing.md @@ -0,0 +1,70 @@ +# Dataset Editing + +The `Edit`{.interpreted-text role="guilabel"} link in the menu of the *Dataset Page* opens a list of options like ones shown in the picture below. + +
+img/dataset_editing_link.png +
Dataset Editing Link
+
+ +In that options list, you can see three options listed as: + +1. *Edit Data* +2. *Edit Styles* +3. *Edit Metadata* +4. *Upload Style* +5. *Upload Metadata* + +In this section you will learn how to edit a *Dataset*, and its data. See `dataset-metadata`{.interpreted-text role="ref"} to learn how to explore the dataset *Metadata*, how to upload and edit them. The *Styles* will be covered in a dedicated section, see `dataset-style`{.interpreted-text role="ref"}. + +## Editing the Dataset Data {#dataset-data-editing} + +The `Edit data`{.interpreted-text role="guilabel"} link of the *Dataset Editing* options opens the *Dataset* within a *Map*. + +
+img/editing_dataset_data.png +
Editing the Dataset Data
+
+ +The *Attribute Table* panel of the *Dataset* will automatically appear at the bottom of the *Map*. In that panel all the features are listed. For each feature you can zoom to its extent by clicking on the corresponding *magnifying glass* icon ![magnifying_glass_icon](img/magnifying_glass_icon.png){.align-middle width="30px" height="30px"} at the beginning of the row, you can also observe which values the feature assumes for each attribute. + +Click the *Edit Mode* ![edit_mode_button](img/edit_mode_button.png){.align-middle width="30px" height="30px"} button to start an editing session. + +Now you can: + +- *Add new Features* + + Through the *Add New Feature* button it is possible to set up a new feature for your dataset. + Fill the attributes fields and click to save your change. + Your new feature doesn\'t have a shape yet, click on to draw its shape directly on the *Map* then click on to save it. + + ::: note + ::: title + Note + ::: + + When your new feature has a multi-vertex shape you have to double-click the last vertex to finish the drawing. + ::: + +
+img/create_new_feature.png +
Create New Feature
+
+ +- *Delete Features* + + If you want to delete a feature you have to select it on the *Attribute Table* and click on . + +- *Change the Feature Shape* + + You can edit the shape of an existing geometry dragging its vertices with the mouse. A blue circle lets you know what vertex you are moving. + + Features can have *multipart shapes*. You can add parts to the shape when editing it. + +- *Change the Feature Attributes* + + When you are in *Edit Mode* you can also edit the attributes values changing them directly in the corresponding text fields. You can achieve this by going into the edit mode and double click in the values. + +Once you have finished you can end the *Editing Session* by clicking on the . + +By default the GeoNode map viewer is [MapStore](https://mapstore2.geo-solutions.it/mapstore/#/) based, see the [MapStore Documentation](https://docs.mapstore.geosolutionsgroup.com/en/latest/user-guide/attributes-table/) for further information. diff --git a/docs/usage/managing_datasets/dataset_info.md b/docs/usage/managing_datasets/dataset_info.md new file mode 100644 index 00000000..e29c6c8b --- /dev/null +++ b/docs/usage/managing_datasets/dataset_info.md @@ -0,0 +1,74 @@ +::: {#dataset-info} +Dataset Information +================= +::: + +From the *Dataset Search Page* (see `finding-data`{.interpreted-text role="ref"}) you can select the dataset you are interested in and click on its name to see an overview about it. + +![](img/dataset_overview.png){.align-center} + +The information panel reports: + +- The *Info* tab is active by default. This tab section shows some dataset metadata such as its title, the abstract, date of publication etc. The metadata also indicates the dataset owner, what are the topic categories the dataset belongs to and which regions are affected. + +
+img/dataset_info.png +
Dataset Info tab
+
+ +- The *Location* tab shows the spacial extent of the dataset. + +
+img/dataset_location.png +
Dataset Location tab
+
+ +By clicking on the copy icons you have a copy of the current *Bounding Box* or the *Center* in the clipboard which once pasted will be a WKT text. + +
+img/copy_locations_dataset.png +
Bounding Box and Center
+
+ +- The *Attributes* tab shows the data structure behind the dataset. All the attributes are listed and for each of them some statistics (e.g. the range of values) are estimated (if possible). + +
+img/dataset_attributes_tab.png +
Dataset Attributes tab
+
+ +- The *Linked Resources* tab shows the resources, that can be *Documents*, *Maps*, *GeoStories* and *Dashboards*, linked to the dataset. + +
+img/dataset_linked_resources.png +
Dataset Linked Resources tab
+
+ +::: note +::: title +Note +::: + +When a resource is [linked from]{.title-ref} the dataset the link is shown under the *Linked by* list while when a resource is [linked to]{.title-ref} the dataset the link is shown under the *Used by* list +::: + +From the right toolbar on the **Info** panel it is possible: + +
+img/dataset_info_toolbar.png +
Dataset Info toolbar
+
+ +- Add dataset in your *Favorites* (see `editing-profile`{.interpreted-text role="ref"}), clicking on star icon. +- Directly *Download* the dataset oh [ShapeFile]{.title-ref} format +- *Share* the dataset url +- Copy the *WMS Capabilities url* +- Change the dataset *Title* by clicking on it + +You can access the dataset details page by clicking on *View dataset* in the overview panel. +That page looks like the one shown in the picture below. + +
+img/dataset_detail.png +
Dataset page
+
diff --git a/docs/usage/managing_datasets/dataset_metadata.md b/docs/usage/managing_datasets/dataset_metadata.md new file mode 100644 index 00000000..1bf7db15 --- /dev/null +++ b/docs/usage/managing_datasets/dataset_metadata.md @@ -0,0 +1,170 @@ +::: {#dataset-metadata} +Datasets Metadata +=============== +::: + +In GeoNode special importance is given to *Metadata* and their standard formats. + +Editing Metadata +\-\-\-\-\-\-\-\-\-\-\-\-\-\-- + +| Metadata contains all the information related to the dataset. They provide essential information for its identification and its comprehension. Metadata also make the dataset more easily retrievable through search by other users. +| The *Metadata* of a dataset can be changed through a *Edit Metadata* form which involves four steps, one for each type of metadata considered: + +You can open the *Metadata* form of a *Dataset* by clicking the `Edit Metadata`{.interpreted-text role="guilabel"} link from the `Edit`{.interpreted-text role="guilabel"} options on the *Dataset Page*. + +- **Basic Metadata** + + The first two steps are mandatory (no datasets will be published if the required information are not provided) whereas the last two are optional. + +
+ img/basic_dataset_metadata.png +
Basic Dataset Metadata
+
+ + In the first step the system asks you to insert the following metadata: + + - *Thumbnail* of the dataset (click `Edit`{.interpreted-text role="guilabel"} to change it); + + - *Title* of the dataset, which should be clear and understandable; + + - *Abstract*; brief narrative summary of the content of the dataset + + > ::: note + > ::: title + > Note + > ::: + > + > The *Abstract* panel allows you to insert HTML code through a *wysiwyg* text editor + > ::: + + - *Creation/Publication/Revision Dates* which define the time period that is covered by the dataset; + + - *Keywords*, which should be chosen within the available list. The contributor search for available keywords by clicking on the searching bar, or on the folder logo representing, or by entering the first letters of the desired word; + + - *Category* which the dataset belongs to; + + - *Group* which the dataset is linked to. + +- **Location and Licenses** + +
+ img/location_licenses_dataset_metadata.png +
Location and Licenses Metadata for Datasets
+
+ + The following list shows what kinds of metadata you are required to enter (see also the picture below): + + - *Language* of the dataset; + + - *License* of the dataset; + + - *DOI* of the dataset; if available, this represents the [Digital Object Identifier](https://www.doi.org/) of the resource + + - *Attribution* of the dataset; authority or function assigned, as to a ruler, legislative assembly, delegate, or the like + + - *Regions*, which informs on the spatial extent covered by the dataset. Proposed extents cover the following scales: global, continental, regional, national; + + - *Data Quality statement* (general explanation of the data producer\'s knowledge about the lineage of a dataset); + + - Potential *Restrictions* on dataset sharing. + + > ::: note + > ::: title + > Note + > ::: + > + > The *Data Quality statement* and *Restrictions* panels allow you to insert HTML code through a *wysiwyg* text editor + > ::: + +- **Optional Metadata** + +
+ img/optional_dataset_metadata.png +
Optional Dtaset Metadata
+
+ + Complementary information are: + + - *Edition* to indicate the reference or the source of the dataset; + + - *Purpose* of the dataset and its objectives; + + - *Supplemental information* that can provide a better understanding of the uploaded dataset; + + - *Maintenance frequency* of the dataset; + + - users who are *Responsible* for the dataset, its *Owner*, and the *Author* of its metadata; + + - *Spatial representation type* used. + + - *Related resources* to link one or multiple resources to the document. These will be visible inside the `dataset-info`{.interpreted-text role="ref"} panel + + > ::: note + > ::: title + > Note + > ::: + > + > The *Purpose* and *Supplemental information* panels allow you to insert HTML code through a *wysiwyg* text editor + > ::: + +- **Dataset Attributes** + +
+ img/dataset_attributes_dataset_metadata.png +
Dataset Attributes Metadata for Dataset
+
+ + At this step you can enrich the dataset attributes with useful information like the following: + + - The *Label* displayed + - A detailed *Description* + - The *Display Order* + - The *Display Type*; the default value is *Label*, which means that the value of the attribute will be rendered as a plain text. + There\'s the possibility to instruct GeoNode to threat the values as different media-types. As an instance, if the values of the + selected attribute will contain image urls, by selecting the `IMAGE` *Display Type* you will allow GeoNode to render the image + directly when querying the dataset from the maps. The same for `VIDEO`, `AUDIO` or `IFRAME` mime types. + - The *Visibile* flag; allows you to instruct GeoNode wether or not hiding an attribute from the *Get Feature Type* outcomes + + It is possible to define a completely custom `HTML` template for the *Get Feature Type* outcome. That is possible by enabling the *Use a custom template* flag as shown in the figure below. + +
+ img/dataset_attributes_dataset_metadata_custom_ft.png +
Use a custom template
+
+ + By using the keywork `${properties.`, you can tell to GeoNode to render the actual value of the attribute on the map. + + As an instance, the example below + +
+ img/dataset_attributes_dataset_metadata_custom_ft_html.png +
Use a custom template: HTML
+
+ + Will render an `HTML Table` along with values as shown here below + +
+ img/dataset_attributes_dataset_metadata_custom_ft_outcome.png +
Use a custom template: Get Feature Info outcome
+
+ +Use `next >>`{.interpreted-text role="guilabel"} or `<< back`{.interpreted-text role="guilabel"} to navigate through those steps. Once you have finished click on `Update`{.interpreted-text role="guilabel"}. + +Some metadata are mandatory, if you miss any of that metadata the *Completeness* bar shows you a red message like the one in the picture below. + +
+img/completeness_progress_bar.png +
Completeness Progress Bar
+
+ +# Metadata Advanced Editing + +In the *Edit Metadata* page the `Advanced Metadata`{.interpreted-text role="guilabel"} button is also available. + +
+img/advanced_edit_button.png +
The Advanced Metadata button
+
+ +Click on it to display the *Metadata Advanced Editing Page*. That page allows you to edit all the dataset metadata described in the previous paragraph. Once you have finished to edit them click on `Update`{.interpreted-text role="guilabel"} to save your changes. diff --git a/docs/usage/managing_datasets/dataset_permissions.md b/docs/usage/managing_datasets/dataset_permissions.md new file mode 100644 index 00000000..1edb2a66 --- /dev/null +++ b/docs/usage/managing_datasets/dataset_permissions.md @@ -0,0 +1,32 @@ +# Share Options {#dataset-permissions} + +In order to modify the Dataset *Share options* settings, on the detail page of the dataset click the `Share`{.interpreted-text role="guilabel"} link in the `Resource`{.interpreted-text role="guilabel"} menu. By default only owners can edit and manage datasets, anyone can view them. + +
+img/change_dataset_permissions.png +
Change Dataset Share options
+
+ +Through the *Share options Settings Panel* you can add or remove options for users and groups. The picture below shows an example. + +
+img/dataset_permissions_settings.png +
Dataset Share options settings for users and groups
+
+ +You can set the following share options: + +- *View* (allows to view the dataset). +- *Download* (allows to view and download the dataset). +- *Edit* (allows to change the dataset metadata,change attributes and properties of the datasets features and change the dataset style). +- *Manage* (allows to update, delete, change share options, publish and unpublish the dataset). + +::: warning +::: title +Warning +::: + +When assigning options to a group, all the group members will have those options. Be careful in case of editing them. +::: + +Click on `Save`{.interpreted-text role="guilabel"} link in the menu to save these settings. diff --git a/docs/usage/managing_datasets/dataset_styling.md b/docs/usage/managing_datasets/dataset_styling.md new file mode 100644 index 00000000..2893563a --- /dev/null +++ b/docs/usage/managing_datasets/dataset_styling.md @@ -0,0 +1,77 @@ +::: {#dataset-style} +Dataset Styling +============= +::: + +Maps are helpful because they allow you gain a deeper understanding of your data by allowing you to visualize it in many different ways. +So you can tell different stories depending on how the data is presented. +For any given data or dataset, you should explore different styling options and choose the best style for that. + +In GeoNode each dataset has one style refered to as a *Default Style* which is determined by the nature of the data you\'re mapping. When uploading a new dataset (see `uploading-datasets`{.interpreted-text role="ref"}) a new default style will be associated to it. + +Editing the Dataset Style +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-- + +In order to edit a dataset style, open the *Dataset Page* (see `dataset-info`{.interpreted-text role="ref"}) and click on `Edit`{.interpreted-text role="guilabel"}. Then click the `Edit Style`{.interpreted-text role="guilabel"} link in the *options* (see the picture below). + +
+img/edit_style_button.png +
Edit Styles button
+
+ +The *Dataset* will open in a new *Map*. The *Styles Panel* will show you all the default style for the dataset and some useful tools. By default, the style will be showm in a text editor form. + +
+img/styles_map_panel.png +
The Styles Panel in the Map
+
+ +You could continue to change the style with the text editor or switch to a visualized editor using the `Visual editor`{.interpreted-text role="guilabel"} above the text editor. + +The visual editor looks like this + +
+img/visual_styles_map_panel.png +
Visual Styles Panel in the Map
+
+ +You can then edit the style by clicking on each attribute of the style. + +Creating Style Rules +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-- + +In order to create a new rule, Three options are provided. + +1. Mark rule +2. Icon rule +3. text rule + +Click on any of the buttons below to create a type of style you want. + +>
+> img/add_style_buttons.png +>
Create new rule buttons
+>
+ +The rule you have chosen is now added on the top with default attributes which you can edit to your preference\*. + +
+img/new_rule.png +
New style
+
+ +If the rule has errors, the following will be shown. + +
+img/edit_style_error.png +
Rule Error
+
+ +You can switch the rule ordering by dragging it to the top or bottom depending on your preference. + +It would be nice to change the style in order to decrease the opacity of the filling color as well as to reduce the lines width. +The embedded [MapStore](https://mapstore2.geo-solutions.it/mapstore/#/) makes available a powerful *Style Editor* to accomplish that tasks. In the next paragraph we will explain how. + +Each rule has a delete icon on the top right which you can use to remove the rule on the style. + +Click on `Save`{.interpreted-text role="guilabel"} on the top to save your changes. diff --git a/usage/managing_datasets/img/add_new_feature.gif b/docs/usage/managing_datasets/img/add_new_feature.gif similarity index 100% rename from usage/managing_datasets/img/add_new_feature.gif rename to docs/usage/managing_datasets/img/add_new_feature.gif diff --git a/usage/managing_datasets/img/add_new_feature_button.png b/docs/usage/managing_datasets/img/add_new_feature_button.png similarity index 100% rename from usage/managing_datasets/img/add_new_feature_button.png rename to docs/usage/managing_datasets/img/add_new_feature_button.png diff --git a/usage/managing_datasets/img/add_shape_to_existing_geometry.gif b/docs/usage/managing_datasets/img/add_shape_to_existing_geometry.gif similarity index 100% rename from usage/managing_datasets/img/add_shape_to_existing_geometry.gif rename to docs/usage/managing_datasets/img/add_shape_to_existing_geometry.gif diff --git a/usage/managing_datasets/img/add_shape_to_geometry_button.png b/docs/usage/managing_datasets/img/add_shape_to_geometry_button.png similarity index 100% rename from usage/managing_datasets/img/add_shape_to_geometry_button.png rename to docs/usage/managing_datasets/img/add_shape_to_geometry_button.png diff --git a/usage/managing_datasets/img/add_style_button.png b/docs/usage/managing_datasets/img/add_style_button.png similarity index 100% rename from usage/managing_datasets/img/add_style_button.png rename to docs/usage/managing_datasets/img/add_style_button.png diff --git a/usage/managing_datasets/img/add_style_buttons.png b/docs/usage/managing_datasets/img/add_style_buttons.png similarity index 100% rename from usage/managing_datasets/img/add_style_buttons.png rename to docs/usage/managing_datasets/img/add_style_buttons.png diff --git a/usage/managing_datasets/img/advanced_edit_button.png b/docs/usage/managing_datasets/img/advanced_edit_button.png similarity index 100% rename from usage/managing_datasets/img/advanced_edit_button.png rename to docs/usage/managing_datasets/img/advanced_edit_button.png diff --git a/usage/managing_datasets/img/append_layer.png b/docs/usage/managing_datasets/img/append_layer.png old mode 100755 new mode 100644 similarity index 100% rename from usage/managing_datasets/img/append_layer.png rename to docs/usage/managing_datasets/img/append_layer.png diff --git a/usage/managing_datasets/img/basic_dataset_metadata.png b/docs/usage/managing_datasets/img/basic_dataset_metadata.png similarity index 100% rename from usage/managing_datasets/img/basic_dataset_metadata.png rename to docs/usage/managing_datasets/img/basic_dataset_metadata.png diff --git a/usage/managing_datasets/img/change_dataset_permissions.png b/docs/usage/managing_datasets/img/change_dataset_permissions.png similarity index 100% rename from usage/managing_datasets/img/change_dataset_permissions.png rename to docs/usage/managing_datasets/img/change_dataset_permissions.png diff --git a/usage/managing_datasets/img/completeness_progress_bar.png b/docs/usage/managing_datasets/img/completeness_progress_bar.png similarity index 100% rename from usage/managing_datasets/img/completeness_progress_bar.png rename to docs/usage/managing_datasets/img/completeness_progress_bar.png diff --git a/usage/managing_datasets/img/copy_locations_dataset.png b/docs/usage/managing_datasets/img/copy_locations_dataset.png similarity index 100% rename from usage/managing_datasets/img/copy_locations_dataset.png rename to docs/usage/managing_datasets/img/copy_locations_dataset.png diff --git a/usage/managing_datasets/img/create_dataset_link.png b/docs/usage/managing_datasets/img/create_dataset_link.png similarity index 100% rename from usage/managing_datasets/img/create_dataset_link.png rename to docs/usage/managing_datasets/img/create_dataset_link.png diff --git a/usage/managing_datasets/img/create_new_feature.png b/docs/usage/managing_datasets/img/create_new_feature.png similarity index 100% rename from usage/managing_datasets/img/create_new_feature.png rename to docs/usage/managing_datasets/img/create_new_feature.png diff --git a/usage/managing_datasets/img/create_new_style_button.png b/docs/usage/managing_datasets/img/create_new_style_button.png similarity index 100% rename from usage/managing_datasets/img/create_new_style_button.png rename to docs/usage/managing_datasets/img/create_new_style_button.png diff --git a/usage/managing_datasets/img/create_new_style_from_templates.png b/docs/usage/managing_datasets/img/create_new_style_from_templates.png similarity index 100% rename from usage/managing_datasets/img/create_new_style_from_templates.png rename to docs/usage/managing_datasets/img/create_new_style_from_templates.png diff --git a/usage/managing_datasets/img/dataset_attributes_dataset_metadata.png b/docs/usage/managing_datasets/img/dataset_attributes_dataset_metadata.png similarity index 100% rename from usage/managing_datasets/img/dataset_attributes_dataset_metadata.png rename to docs/usage/managing_datasets/img/dataset_attributes_dataset_metadata.png diff --git a/usage/managing_datasets/img/dataset_attributes_dataset_metadata_custom_ft.png b/docs/usage/managing_datasets/img/dataset_attributes_dataset_metadata_custom_ft.png similarity index 100% rename from usage/managing_datasets/img/dataset_attributes_dataset_metadata_custom_ft.png rename to docs/usage/managing_datasets/img/dataset_attributes_dataset_metadata_custom_ft.png diff --git a/usage/managing_datasets/img/dataset_attributes_dataset_metadata_custom_ft_html.png b/docs/usage/managing_datasets/img/dataset_attributes_dataset_metadata_custom_ft_html.png similarity index 100% rename from usage/managing_datasets/img/dataset_attributes_dataset_metadata_custom_ft_html.png rename to docs/usage/managing_datasets/img/dataset_attributes_dataset_metadata_custom_ft_html.png diff --git a/usage/managing_datasets/img/dataset_attributes_dataset_metadata_custom_ft_outcome.png b/docs/usage/managing_datasets/img/dataset_attributes_dataset_metadata_custom_ft_outcome.png similarity index 100% rename from usage/managing_datasets/img/dataset_attributes_dataset_metadata_custom_ft_outcome.png rename to docs/usage/managing_datasets/img/dataset_attributes_dataset_metadata_custom_ft_outcome.png diff --git a/usage/managing_datasets/img/dataset_attributes_tab.png b/docs/usage/managing_datasets/img/dataset_attributes_tab.png similarity index 100% rename from usage/managing_datasets/img/dataset_attributes_tab.png rename to docs/usage/managing_datasets/img/dataset_attributes_tab.png diff --git a/usage/managing_datasets/img/dataset_detail.png b/docs/usage/managing_datasets/img/dataset_detail.png similarity index 100% rename from usage/managing_datasets/img/dataset_detail.png rename to docs/usage/managing_datasets/img/dataset_detail.png diff --git a/usage/managing_datasets/img/dataset_editing_link.png b/docs/usage/managing_datasets/img/dataset_editing_link.png similarity index 100% rename from usage/managing_datasets/img/dataset_editing_link.png rename to docs/usage/managing_datasets/img/dataset_editing_link.png diff --git a/usage/managing_datasets/img/dataset_info.png b/docs/usage/managing_datasets/img/dataset_info.png similarity index 100% rename from usage/managing_datasets/img/dataset_info.png rename to docs/usage/managing_datasets/img/dataset_info.png diff --git a/usage/managing_datasets/img/dataset_info_toolbar.png b/docs/usage/managing_datasets/img/dataset_info_toolbar.png similarity index 100% rename from usage/managing_datasets/img/dataset_info_toolbar.png rename to docs/usage/managing_datasets/img/dataset_info_toolbar.png diff --git a/usage/managing_datasets/img/dataset_linked_resources.png b/docs/usage/managing_datasets/img/dataset_linked_resources.png similarity index 100% rename from usage/managing_datasets/img/dataset_linked_resources.png rename to docs/usage/managing_datasets/img/dataset_linked_resources.png diff --git a/usage/managing_datasets/img/dataset_location.png b/docs/usage/managing_datasets/img/dataset_location.png similarity index 100% rename from usage/managing_datasets/img/dataset_location.png rename to docs/usage/managing_datasets/img/dataset_location.png diff --git a/usage/managing_datasets/img/dataset_menu.png b/docs/usage/managing_datasets/img/dataset_menu.png similarity index 100% rename from usage/managing_datasets/img/dataset_menu.png rename to docs/usage/managing_datasets/img/dataset_menu.png diff --git a/usage/managing_datasets/img/dataset_overview.png b/docs/usage/managing_datasets/img/dataset_overview.png similarity index 100% rename from usage/managing_datasets/img/dataset_overview.png rename to docs/usage/managing_datasets/img/dataset_overview.png diff --git a/usage/managing_datasets/img/dataset_permissions_settings.png b/docs/usage/managing_datasets/img/dataset_permissions_settings.png similarity index 100% rename from usage/managing_datasets/img/dataset_permissions_settings.png rename to docs/usage/managing_datasets/img/dataset_permissions_settings.png diff --git a/usage/managing_datasets/img/dataset_settings_button.png b/docs/usage/managing_datasets/img/dataset_settings_button.png similarity index 100% rename from usage/managing_datasets/img/dataset_settings_button.png rename to docs/usage/managing_datasets/img/dataset_settings_button.png diff --git a/usage/managing_datasets/img/dataset_upload_confirm.png b/docs/usage/managing_datasets/img/dataset_upload_confirm.png similarity index 100% rename from usage/managing_datasets/img/dataset_upload_confirm.png rename to docs/usage/managing_datasets/img/dataset_upload_confirm.png diff --git a/usage/managing_datasets/img/dataset_upload_process.png b/docs/usage/managing_datasets/img/dataset_upload_process.png similarity index 100% rename from usage/managing_datasets/img/dataset_upload_process.png rename to docs/usage/managing_datasets/img/dataset_upload_process.png diff --git a/usage/managing_datasets/img/dataset_upload_view.png b/docs/usage/managing_datasets/img/dataset_upload_view.png similarity index 100% rename from usage/managing_datasets/img/dataset_upload_view.png rename to docs/usage/managing_datasets/img/dataset_upload_view.png diff --git a/usage/managing_datasets/img/delete_feature_button.png b/docs/usage/managing_datasets/img/delete_feature_button.png similarity index 100% rename from usage/managing_datasets/img/delete_feature_button.png rename to docs/usage/managing_datasets/img/delete_feature_button.png diff --git a/usage/managing_datasets/img/delete_style_button.png b/docs/usage/managing_datasets/img/delete_style_button.png similarity index 100% rename from usage/managing_datasets/img/delete_style_button.png rename to docs/usage/managing_datasets/img/delete_style_button.png diff --git a/usage/managing_datasets/img/download_metadata.png b/docs/usage/managing_datasets/img/download_metadata.png similarity index 100% rename from usage/managing_datasets/img/download_metadata.png rename to docs/usage/managing_datasets/img/download_metadata.png diff --git a/usage/managing_datasets/img/edit_mode_button.png b/docs/usage/managing_datasets/img/edit_mode_button.png similarity index 100% rename from usage/managing_datasets/img/edit_mode_button.png rename to docs/usage/managing_datasets/img/edit_mode_button.png diff --git a/usage/managing_datasets/img/edit_style.gif b/docs/usage/managing_datasets/img/edit_style.gif similarity index 100% rename from usage/managing_datasets/img/edit_style.gif rename to docs/usage/managing_datasets/img/edit_style.gif diff --git a/usage/managing_datasets/img/edit_style_button.png b/docs/usage/managing_datasets/img/edit_style_button.png similarity index 100% rename from usage/managing_datasets/img/edit_style_button.png rename to docs/usage/managing_datasets/img/edit_style_button.png diff --git a/usage/managing_datasets/img/edit_style_error.png b/docs/usage/managing_datasets/img/edit_style_error.png similarity index 100% rename from usage/managing_datasets/img/edit_style_error.png rename to docs/usage/managing_datasets/img/edit_style_error.png diff --git a/usage/managing_datasets/img/editing_dataset_data.png b/docs/usage/managing_datasets/img/editing_dataset_data.png similarity index 100% rename from usage/managing_datasets/img/editing_dataset_data.png rename to docs/usage/managing_datasets/img/editing_dataset_data.png diff --git a/usage/managing_datasets/img/end_editing_session_button.png b/docs/usage/managing_datasets/img/end_editing_session_button.png similarity index 100% rename from usage/managing_datasets/img/end_editing_session_button.png rename to docs/usage/managing_datasets/img/end_editing_session_button.png diff --git a/usage/managing_datasets/img/export_dataset_form.png b/docs/usage/managing_datasets/img/export_dataset_form.png similarity index 100% rename from usage/managing_datasets/img/export_dataset_form.png rename to docs/usage/managing_datasets/img/export_dataset_form.png diff --git a/usage/managing_datasets/img/export_icon.png b/docs/usage/managing_datasets/img/export_icon.png similarity index 100% rename from usage/managing_datasets/img/export_icon.png rename to docs/usage/managing_datasets/img/export_icon.png diff --git a/usage/managing_datasets/img/export_list.png b/docs/usage/managing_datasets/img/export_list.png similarity index 100% rename from usage/managing_datasets/img/export_list.png rename to docs/usage/managing_datasets/img/export_list.png diff --git a/usage/managing_datasets/img/export_metadata.png b/docs/usage/managing_datasets/img/export_metadata.png similarity index 100% rename from usage/managing_datasets/img/export_metadata.png rename to docs/usage/managing_datasets/img/export_metadata.png diff --git a/usage/managing_datasets/img/export_notification.png b/docs/usage/managing_datasets/img/export_notification.png similarity index 100% rename from usage/managing_datasets/img/export_notification.png rename to docs/usage/managing_datasets/img/export_notification.png diff --git a/usage/managing_datasets/img/geometry_types.png b/docs/usage/managing_datasets/img/geometry_types.png similarity index 100% rename from usage/managing_datasets/img/geometry_types.png rename to docs/usage/managing_datasets/img/geometry_types.png diff --git a/usage/managing_datasets/img/layer_comments.png b/docs/usage/managing_datasets/img/layer_comments.png similarity index 100% rename from usage/managing_datasets/img/layer_comments.png rename to docs/usage/managing_datasets/img/layer_comments.png diff --git a/usage/managing_datasets/img/layer_images_download.png b/docs/usage/managing_datasets/img/layer_images_download.png similarity index 100% rename from usage/managing_datasets/img/layer_images_download.png rename to docs/usage/managing_datasets/img/layer_images_download.png diff --git a/usage/managing_datasets/img/layer_metadata_details_page.png b/docs/usage/managing_datasets/img/layer_metadata_details_page.png similarity index 100% rename from usage/managing_datasets/img/layer_metadata_details_page.png rename to docs/usage/managing_datasets/img/layer_metadata_details_page.png diff --git a/usage/managing_datasets/img/layer_rating.png b/docs/usage/managing_datasets/img/layer_rating.png similarity index 100% rename from usage/managing_datasets/img/layer_rating.png rename to docs/usage/managing_datasets/img/layer_rating.png diff --git a/usage/managing_datasets/img/layer_sharing.png b/docs/usage/managing_datasets/img/layer_sharing.png similarity index 100% rename from usage/managing_datasets/img/layer_sharing.png rename to docs/usage/managing_datasets/img/layer_sharing.png diff --git a/usage/managing_datasets/img/location_licenses_dataset_metadata.png b/docs/usage/managing_datasets/img/location_licenses_dataset_metadata.png similarity index 100% rename from usage/managing_datasets/img/location_licenses_dataset_metadata.png rename to docs/usage/managing_datasets/img/location_licenses_dataset_metadata.png diff --git a/usage/managing_datasets/img/magnifying_glass_icon.png b/docs/usage/managing_datasets/img/magnifying_glass_icon.png similarity index 100% rename from usage/managing_datasets/img/magnifying_glass_icon.png rename to docs/usage/managing_datasets/img/magnifying_glass_icon.png diff --git a/usage/managing_datasets/img/manage_styles.png b/docs/usage/managing_datasets/img/manage_styles.png similarity index 100% rename from usage/managing_datasets/img/manage_styles.png rename to docs/usage/managing_datasets/img/manage_styles.png diff --git a/usage/managing_datasets/img/manage_styles_button.png b/docs/usage/managing_datasets/img/manage_styles_button.png similarity index 100% rename from usage/managing_datasets/img/manage_styles_button.png rename to docs/usage/managing_datasets/img/manage_styles_button.png diff --git a/usage/managing_datasets/img/measuring.png b/docs/usage/managing_datasets/img/measuring.png similarity index 100% rename from usage/managing_datasets/img/measuring.png rename to docs/usage/managing_datasets/img/measuring.png diff --git a/usage/managing_datasets/img/metadata_upload.png b/docs/usage/managing_datasets/img/metadata_upload.png similarity index 100% rename from usage/managing_datasets/img/metadata_upload.png rename to docs/usage/managing_datasets/img/metadata_upload.png diff --git a/usage/managing_datasets/img/new_dataset_from_scratch.png b/docs/usage/managing_datasets/img/new_dataset_from_scratch.png similarity index 100% rename from usage/managing_datasets/img/new_dataset_from_scratch.png rename to docs/usage/managing_datasets/img/new_dataset_from_scratch.png diff --git a/usage/managing_datasets/img/new_rule.png b/docs/usage/managing_datasets/img/new_rule.png similarity index 100% rename from usage/managing_datasets/img/new_rule.png rename to docs/usage/managing_datasets/img/new_rule.png diff --git a/usage/managing_datasets/img/new_style_in_layer_page.png b/docs/usage/managing_datasets/img/new_style_in_layer_page.png similarity index 100% rename from usage/managing_datasets/img/new_style_in_layer_page.png rename to docs/usage/managing_datasets/img/new_style_in_layer_page.png diff --git a/usage/managing_datasets/img/new_style_in_list.png b/docs/usage/managing_datasets/img/new_style_in_list.png similarity index 100% rename from usage/managing_datasets/img/new_style_in_list.png rename to docs/usage/managing_datasets/img/new_style_in_list.png diff --git a/usage/managing_datasets/img/new_style_title.png b/docs/usage/managing_datasets/img/new_style_title.png similarity index 100% rename from usage/managing_datasets/img/new_style_title.png rename to docs/usage/managing_datasets/img/new_style_title.png diff --git a/usage/managing_datasets/img/optional_dataset_metadata.png b/docs/usage/managing_datasets/img/optional_dataset_metadata.png similarity index 100% rename from usage/managing_datasets/img/optional_dataset_metadata.png rename to docs/usage/managing_datasets/img/optional_dataset_metadata.png diff --git a/usage/managing_datasets/img/printing_window.png b/docs/usage/managing_datasets/img/printing_window.png similarity index 100% rename from usage/managing_datasets/img/printing_window.png rename to docs/usage/managing_datasets/img/printing_window.png diff --git a/usage/managing_datasets/img/remote_service_metadata.png b/docs/usage/managing_datasets/img/remote_service_metadata.png similarity index 100% rename from usage/managing_datasets/img/remote_service_metadata.png rename to docs/usage/managing_datasets/img/remote_service_metadata.png diff --git a/usage/managing_datasets/img/remote_service_resources.png b/docs/usage/managing_datasets/img/remote_service_resources.png similarity index 100% rename from usage/managing_datasets/img/remote_service_resources.png rename to docs/usage/managing_datasets/img/remote_service_resources.png diff --git a/usage/managing_datasets/img/remote_services.png b/docs/usage/managing_datasets/img/remote_services.png similarity index 100% rename from usage/managing_datasets/img/remote_services.png rename to docs/usage/managing_datasets/img/remote_services.png diff --git a/usage/managing_datasets/img/remote_services_link.png b/docs/usage/managing_datasets/img/remote_services_link.png similarity index 100% rename from usage/managing_datasets/img/remote_services_link.png rename to docs/usage/managing_datasets/img/remote_services_link.png diff --git a/usage/managing_datasets/img/save_changes_button.png b/docs/usage/managing_datasets/img/save_changes_button.png similarity index 100% rename from usage/managing_datasets/img/save_changes_button.png rename to docs/usage/managing_datasets/img/save_changes_button.png diff --git a/usage/managing_datasets/img/save_style_button.png b/docs/usage/managing_datasets/img/save_style_button.png similarity index 100% rename from usage/managing_datasets/img/save_style_button.png rename to docs/usage/managing_datasets/img/save_style_button.png diff --git a/usage/managing_datasets/img/service_type.png b/docs/usage/managing_datasets/img/service_type.png similarity index 100% rename from usage/managing_datasets/img/service_type.png rename to docs/usage/managing_datasets/img/service_type.png diff --git a/usage/managing_datasets/img/set_style_as_default_button.png b/docs/usage/managing_datasets/img/set_style_as_default_button.png similarity index 100% rename from usage/managing_datasets/img/set_style_as_default_button.png rename to docs/usage/managing_datasets/img/set_style_as_default_button.png diff --git a/usage/managing_datasets/img/style_editor.gif b/docs/usage/managing_datasets/img/style_editor.gif similarity index 100% rename from usage/managing_datasets/img/style_editor.gif rename to docs/usage/managing_datasets/img/style_editor.gif diff --git a/usage/managing_datasets/img/style_editor_button.png b/docs/usage/managing_datasets/img/style_editor_button.png similarity index 100% rename from usage/managing_datasets/img/style_editor_button.png rename to docs/usage/managing_datasets/img/style_editor_button.png diff --git a/usage/managing_datasets/img/style_editor_syntax_validator.png b/docs/usage/managing_datasets/img/style_editor_syntax_validator.png similarity index 100% rename from usage/managing_datasets/img/style_editor_syntax_validator.png rename to docs/usage/managing_datasets/img/style_editor_syntax_validator.png diff --git a/usage/managing_datasets/img/style_tab.png b/docs/usage/managing_datasets/img/style_tab.png similarity index 100% rename from usage/managing_datasets/img/style_tab.png rename to docs/usage/managing_datasets/img/style_tab.png diff --git a/usage/managing_datasets/img/style_templates.png b/docs/usage/managing_datasets/img/style_templates.png similarity index 100% rename from usage/managing_datasets/img/style_templates.png rename to docs/usage/managing_datasets/img/style_templates.png diff --git a/usage/managing_datasets/img/styles_map_panel.png b/docs/usage/managing_datasets/img/styles_map_panel.png similarity index 100% rename from usage/managing_datasets/img/styles_map_panel.png rename to docs/usage/managing_datasets/img/styles_map_panel.png diff --git a/usage/managing_datasets/img/toc_button.png b/docs/usage/managing_datasets/img/toc_button.png similarity index 100% rename from usage/managing_datasets/img/toc_button.png rename to docs/usage/managing_datasets/img/toc_button.png diff --git a/usage/managing_datasets/img/upload_SLD_button.png b/docs/usage/managing_datasets/img/upload_SLD_button.png similarity index 100% rename from usage/managing_datasets/img/upload_SLD_button.png rename to docs/usage/managing_datasets/img/upload_SLD_button.png diff --git a/usage/managing_datasets/img/upload_dataset.png b/docs/usage/managing_datasets/img/upload_dataset.png similarity index 100% rename from usage/managing_datasets/img/upload_dataset.png rename to docs/usage/managing_datasets/img/upload_dataset.png diff --git a/usage/managing_datasets/img/upload_dataset_link.png b/docs/usage/managing_datasets/img/upload_dataset_link.png similarity index 100% rename from usage/managing_datasets/img/upload_dataset_link.png rename to docs/usage/managing_datasets/img/upload_dataset_link.png diff --git a/usage/managing_datasets/img/upload_style_button.png b/docs/usage/managing_datasets/img/upload_style_button.png similarity index 100% rename from usage/managing_datasets/img/upload_style_button.png rename to docs/usage/managing_datasets/img/upload_style_button.png diff --git a/usage/managing_datasets/img/upload_styles.png b/docs/usage/managing_datasets/img/upload_styles.png similarity index 100% rename from usage/managing_datasets/img/upload_styles.png rename to docs/usage/managing_datasets/img/upload_styles.png diff --git a/usage/managing_datasets/img/visual_styles_map_panel.png b/docs/usage/managing_datasets/img/visual_styles_map_panel.png similarity index 100% rename from usage/managing_datasets/img/visual_styles_map_panel.png rename to docs/usage/managing_datasets/img/visual_styles_map_panel.png diff --git a/docs/usage/managing_datasets/index.md b/docs/usage/managing_datasets/index.md new file mode 100644 index 00000000..1198a871 --- /dev/null +++ b/docs/usage/managing_datasets/index.md @@ -0,0 +1,31 @@ +Managing Datasets +=============== + +| Datasets are published resources representing raster o vector spatial data sources. Datasets can also be associated with metadata, ratings, and comments. +| In this section, you will learn how to create a new dataset by uploading a local data set, add dataset info, change the style of the dataset, and share the results. + +::: {.toctree maxdepth="1"} +uploading_datasets +new_dataset_from_scratch +using_remote_services +dataset_permissions +dataset_info +dataset_download +dataset_editing +dataset_metadata +dataset_styling +dataset_advanced +::: + + +- [uploading_datasets](uploading_datasets.md) +- [new_dataset_from_scratch](new_dataset_from_scratch.md) +- [using_remote_services](using_remote_services.md) +- [dataset_permissions](dataset_permissions.md) +- [dataset_info](dataset_info.md) +- [dataset_download](dataset_download.md) +- [dataset_editing](dataset_editing.md) +- [dataset_metadata](dataset_metadata.md) +- [dataset_styling](dataset_styling.md) +- [dataset_advanced](dataset_advanced.md) + diff --git a/docs/usage/managing_datasets/new_dataset_from_scratch.md b/docs/usage/managing_datasets/new_dataset_from_scratch.md new file mode 100644 index 00000000..9b405e95 --- /dev/null +++ b/docs/usage/managing_datasets/new_dataset_from_scratch.md @@ -0,0 +1,31 @@ +Creating a Dataset from scratch +============================= + +An interesting tool that GeoNode makes available to you is the *Create dataset*. It allows you to create a new vector dataset from scratch. The *Dataset Creation Form* is reachable through the `Create dataset`{.interpreted-text role="guilabel"} link shown in the picture below. + +
+img/create_dataset_link.png +
Create dataset link
+
+ +In order to create the new Dataset you have to fill out the required fields: + +- *Name* + +- *Title* + +- *Geometry type* + +
+ img/geometry_types.png +
Geometry types
+
+ +Usually the datasets features should have some *Attributes* that enrich the amount of information associated with each of them. Through the `Add Attribute`{.interpreted-text role="guilabel"} button you can add new attributes. + +
+img/new_dataset_from_scratch.png +
New Dataset creation from scratch
+
+ +Once the form has been filled out, click on `Create`{.interpreted-text role="guilabel"}. You will be redirected to the *Dataset Page* (see `dataset-info`{.interpreted-text role="ref"}). Now your Dataset is created but is still empty, no features have been added yet. See the `dataset-editing`{.interpreted-text role="ref"} section to learn how to add new features. diff --git a/docs/usage/managing_datasets/uploading_datasets.md b/docs/usage/managing_datasets/uploading_datasets.md new file mode 100644 index 00000000..791afb9c --- /dev/null +++ b/docs/usage/managing_datasets/uploading_datasets.md @@ -0,0 +1,75 @@ +::: {#uploading-datasets} +Datasets Uploading +================ +::: + +| The most important resource type in GeoNode is the *Dataset*. A dataset represents spatial information so it can be displayed inside a map. +| To better understand what we are talking about lets upload your first dataset. + +It is possible to upload a *Datasets* in two ways: + +- From the `All Resources`{.interpreted-text role="guilabel"} page, by clicking *Add Resource* which displays a list including [Upload dataset]{.title-ref} link: + +
+ img/upload_dataset_link.png +
Dataset Upload from All Resources page
+
+ +- From the `Datasets`{.interpreted-text role="guilabel"} page, by clicking on *New* which displays a list including [Upload dataset]{.title-ref} link: + +
+img/upload_dataset.png +
Dataset Upload from Datasets page
+
+ +The *Datasets Uploading* page looks like the one in the picture below. + +
+img/dataset_upload_process.png +
The Datasets Uploading page
+
+ +| Through the `Select files`{.interpreted-text role="guilabel"} button you can select files from your disk, make sure they are valid raster or vector spatial data, then you can click to `Upload`{.interpreted-text role="guilabel"} button. + +A progress bar shows the operation made during the dataset upload and alerts you when the process is over. + +
+img/dataset_upload_confirm.png +
Upload dataset
+
+ +In this example the `ne_10m_airports2` ESRI Shapefile, with all its mandatory files ([.shp]{.title-ref}, [.shx]{.title-ref}, [.dbf]{.title-ref} and [.prj]{.title-ref}), has been chosen. +When the process ends click on `View`{.interpreted-text role="guilabel"} button + +
+img/dataset_upload_view.png +
Dataset uploading finished
+
+ +::: note +::: title +Note +::: + +There are lot of free spatial dataset available in the Internet. In this example, an extract of the Berlin city center roads map from the [BBBike extracts OpenStreetMap](https://extract.bbbike.org) dataset has been used. +::: + +In the next paragraphs you will learn how to create a dataset from scratch, how to set share options, how to explore the dataset properties and how to edit them. + +::: note +::: title +Note +::: + +If you get the following error message: + +`Total upload size exceeds 100.0 MB. Please try again with smaller files.` + +This means that there is an upload size limit of 100 MB. An user with administrative access can change the upload size limits at the [admin panel for size limits](../../admin/upload-size-limits/index.html#upload-size-limits). + +Similarly, for the following error message: + +`The number of active parallel uploads exceeds 5. Wait for the pending ones to finish.` + +You can modify the upload parallelism limits at the [admin panel for parallelism limits](../../admin/upload-parallelism-limit/index.html#upload-parallelism-limits). +::: diff --git a/docs/usage/managing_datasets/using_remote_services.md b/docs/usage/managing_datasets/using_remote_services.md new file mode 100644 index 00000000..0f737b3b --- /dev/null +++ b/docs/usage/managing_datasets/using_remote_services.md @@ -0,0 +1,103 @@ +# Using Remote Services + +In GeoNode you can add new datasets not only by loading them from your disk but also using *Remote Services*. +In this section you will learn how to add a new service and how to load resources in GeoNode through that. + +Let\'s try it! + +Click on the `Remote Services`{.interpreted-text role="guilabel"} link of the `New`{.interpreted-text role="guilabel"} menu in the navigation bar. + +
+img/remote_services_link.png +
Remote Services link
+
+ +The page that opens will contain the list of the available services. + +
+img/remote_services.png +
Remote Services
+
+ +To configure a new service: + +- click on `Add Remote Service`{.interpreted-text role="guilabel"} + +- type the *Service URL* + +- select the *Service Type* + +
+ img/service_type.png +
Service Types
+
+ +- click on `Create`{.interpreted-text role="guilabel"} + +GeoNode supports four **types of remote services**: + +- *Web Map Service* + + Generic Web Map Service (WMS) based on a standard protocol for serving georeferenced map images over the Internet. + These images are typically produced by a map server (like [GeoServer](http://geoserver.org/)) from data provided by one or more distributed geospatial databases. + Common operations performed by a WMS service are: *GetCapabilities* (to retrieves metadata about the service, including supported operations and parameters, and a list of the available datasets) and *GetMap* (to retrieves a map image for a specified area and content). + + ::: note + ::: title + Note + ::: + + Lots of WMS services are available on the internet, in this example we used the `https://demo.geo-solutions.it/geoserver/wms`. + ::: + +- *GeoNode Web Map Service* + + Generally a WMS is not directly invoked; client applications such as GIS-Desktop or WEB-GIS are used that provide the user with interactive controls. + A GeoNode WMS automatically performs some operations and lets you to immediately retrieve resources. + + ::: note + ::: title + Note + ::: + + An example of GeoNode WMS is available at `http://dev.geonode.geo-solutions.it/geoserver/wms`. + ::: + +- *ArcGIS REST MapServer* + + This map service provides basic information about the map, including the datasets that it contains, whether the map is cached or not, its spatial reference, initial and full extents, whether the service is allowed to export tiles and max tiles export count, etc. + A set of operations that manage the state and contents of the service are allowed: Edit Service, Refresh, Update Tiles. + The URL should follow this pattern: `https:///services//MapServer`. + + ::: note + ::: title + Note + ::: + + Try the following service to better understand how it works: `https://sampleserver6.arcgisonline.com/arcgis/rest/services/USA/MapServer`. + ::: + +- *ArcGIS REST ImageServer* + + This Image Server allows you to assemble, process, analyze, and manage large collections of overlapping, multiresolution imagery and raster data from different sensors, sources, and time periods. You can also publish dynamic image services that transform source imagery and other raster data into multiple image products on demand---without needing to preprocess the data or store intermediate results---saving time and computer resources. In addition, ArcGIS Image Server uses parallel processing across multiple machines and instances, and distributed storage to speed up large raster analysis tasks. + The URL should follow this pattern: `https:///services//ImageServer`. + + ::: note + ::: title + Note + ::: + + Try the following service to better understand how it works: `https://sampleserver6.arcgisonline.com/arcgis/rest/services/Toronto/ImageServer`. + ::: + +Once the service has been configured, you can load the resources you are interested in through the *Import Resources* page where you will be automatically redirected to. + +| From the page where the services are listed, it is possible to click on the *Title* of a service. It opens the *Service Details* page. +| Each service has its own metadata such as the *Service Type*, the *URL*, an *Abstract*, some *Keywords* and the *Contact* user. + +You can edit those metadata through the form available from the `Edit Service Metadata`{.interpreted-text role="guilabel"} link of the *Service Details* page (see the picture below). + +
+img/remote_service_metadata.png +
Remote Service metadata
+
diff --git a/docs/usage/managing_documents/document_editing.md b/docs/usage/managing_documents/document_editing.md new file mode 100644 index 00000000..8758e8a5 --- /dev/null +++ b/docs/usage/managing_documents/document_editing.md @@ -0,0 +1,88 @@ +# Document Editing + +The `document-info`{.interpreted-text role="ref"} page makes available useful menu for document editing. Click on the `Edit`{.interpreted-text role="guilabel"} link then `Edit Metadata`{.interpreted-text role="guilabel"} to see editing forms. + +## Setting the Document Thumbnail + +From the *Metadata Form*, it is possible to *Set the Thumbnail* of the document. Click on `Choose file`{.interpreted-text role="guilabel"} to open the *Thumbnail Uploading* page and chose the image that will illustrate your document. Once this is done, click on the ok button and save. If the thumbnail has been successfully uploaded you can see it by coming back to the document list. + +
+img/document_uploading_thumbnail.png +
Upload Document's Thumbnail
+
+ +If no errors occur, the new selected thumbnail will be shown. + +
+img/document_uploading_thumbnail_success.png +
Uploading success
+
+ +## Editing the Document Metadata + +You can edit the metadata of your document as described in the `document-metadata`{.interpreted-text role="ref"} section or by using THe *Advanced Metadata* option below. + +![](img/advanced_metadata_button.png){.align-center} + +Some information are mandatory such as the *Title* or the *Category* the document belongs to, some others are optional. + +
+img/document_metadata_editing.png +
Editing Metadata
+
+ +In the example shown in the picture above, the information inside the red rectangles have been changed. To save the changes click on `Update`{.interpreted-text role="guilabel"}, you will be redirected to the document page. + +## Editing the Document Info + +You can edit the information of your document by clicking on the `Info`{.interpreted-text role="guilabel"} button. + +From the Info panel, it is possible to upload the **Thumbnail** of the document. + +
+img/upload_thumbnail.png +
Upload Thumbnail
+
+ +If you want this document in your **Favorites** (see `editing-profile`{.interpreted-text role="ref"}), click on the start icon. + +
+img/favorite_document.png +
Favorite document
+
+ +You can directly **Download** the document and **Share** the document url by clicking the buttons on the top-right of the panel. + +
+img/download_share_document.png +
Share and Download document
+
+ +By clicking the document **Title**, you can change it. + +
+img/title_document.png +
Document Title
+
+ +From the `Locations`{.interpreted-text role="guilabel"} tab, you can manage the spacial extent of the document. + +
+img/locations_document.png +
Document Title
+
+ +Here numerical fields and a map show the spatial extent (WGS 84) for the document and you can edit it by Shift+click and drag to draw the new extent on the map and change its location by clicking the marker and drag. + +
+img/edit-location.png +
Document Title
+
+ +::: note +::: title +Note +::: + +The *Location* tab is also available for `dashboard`{.interpreted-text role="ref"} and `geostory`{.interpreted-text role="ref"} +::: diff --git a/docs/usage/managing_documents/document_info.md b/docs/usage/managing_documents/document_info.md new file mode 100644 index 00000000..09abd4f8 --- /dev/null +++ b/docs/usage/managing_documents/document_info.md @@ -0,0 +1,111 @@ +# Document Information {#document-info} + +From the *Documents Search Page* (see `finding-data`{.interpreted-text role="ref"}) you can select the document you are interested in and click on its name to see an overview about it. + +![](img/document_overview.png){.align-center} + +The information panel reports: + +- The **Info** tab is active by default. This tab section shows some document metadata such as its title, the abstract, date of publication etc. The metadata also indicates the document owner, what are the topic categories the document belongs to and which regions are affected. + +
+img/document_info.png +
Document Information tab
+
+ +- The **Location** tab shows the spacial extent of the document. + +
+img/document_location.png +
Document Location tab
+
+ +By clicking on the copy icons you have a copy of the current *Bounding Box* or the *Center* in the clipboard which once pasted will be a WKT text. + +
+img/copy_locations.png +
Bounding Box and Center
+
+ +::: warning +::: title +Warning +::: + +From the `document-info`{.interpreted-text role="ref"} the *Location* tab is in read only mode, to edit it see the `document-editing`{.interpreted-text role="ref"} +::: + +- The *Linked Resources* tab shows the resources, that can be *Datasets*, *Maps*, *GeoStories* and *Dashboards*, linked to the document. + +
+img/document_linked_resources.png +
Document Linked Resources tab
+
+ +::: note +::: title +Note +::: + +When a resource is [linked from]{.title-ref} the document the link is under the *Linked by* list while when a resource is [linked to]{.title-ref} the document the link is under the *Used by* list +::: + +::: warning +::: title +Warning +::: + +From the `document_metadata`{.interpreted-text role="ref"} it is possible edit the document *Linked Resources* +::: + +From the right toolbar on the **Info** panel it is possible: + +
+img/document_info_toolbar.png +
Document Info toolbar
+
+ +- Add document in your *Favorites* (see `editing-profile`{.interpreted-text role="ref"}), clicking on star icon. +- Directly *Download* the document oh [ShapeFile]{.title-ref} format +- *Share* the document url +- Change the document *Title* by clicking on it + +You can access the document details page by clicking on *View document* in the overview panel. +That page looks like the one shown in the picture below. + +
+img/document_detail.png +
Document page
+
+ +On the page of a document, the resource is either directly displayed on the page or accessible by clicking on the link provided under the title. + +Exploring the Document detail menu Sections +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-- + +As soon as a document is opened, the **Info** panel is shown. It reports the document metadata such as its title, abstract, date of publication etc. The metadata also indicates the user who is responsible for uploading and managing this content, as well as the group to which it is linked. + +![](img/document_info.png){.align-center} + +> *Document Info* + +Selecting *View Metadata* from the `View`{.interpreted-text role="guilabel"} button it is possible to visualize the metadata of the document + +
+img/view_metadata.png +
Document Metadata
+
+ +Under the `Resource`{.interpreted-text role="guilabel"} button, it is possible to select the *Share* option to provides the document link to share. + +
+img/document_sharing.png +
Document Sharing
+
+ +If you want to download the document, click on the `Download`{.interpreted-text role="guilabel"} button and the document will be downloaded automatically. + +
+img/document_download.png +
Document Metadata download
+
diff --git a/docs/usage/managing_documents/document_metadata.md b/docs/usage/managing_documents/document_metadata.md new file mode 100644 index 00000000..7f3bf435 --- /dev/null +++ b/docs/usage/managing_documents/document_metadata.md @@ -0,0 +1,61 @@ +# Filling the Document Metadata {#document-metadata} + +You can open the Metadata form by clicking on the *Edit* link in the document details menu and then *Edit Metatadata* Link. + +>
+> img/edit_metadata_link.png +>
Edit Metadata Link
+>
+ +Metadata contains all the information related to the document: they are its ID card. They provide essential information for its identification and its comprehension. Metadata also make the document more easily retrievable through search by other users. + +Editing a document\'s metadata is done in three steps (*Basic Metadata*, *Location and Licenses*, *Optional Metadata*). The first two steps are mandatory (no documents will be published if the required information are not provided) whereas the last one is optional. + +1. On the **Basic Metadata** page, the essential information that has to be filled is: + +> - *Title* of the document, which should be clear and understandable; +> - *Abstract* of the document; +> - *Creation/Publication/Revision* dates which define the time period that is covered by the document; +> - *Keywords*, which should be chosen within the available list. The contributor search for available keywords by clicking on the searching bar, or on the folder logo representing, or by entering the first letters of the desired word. Key-words should be relevant to the imported document; +> - *Category* in which the document belongs; +> - *Group* to which the document is linked. +> +>
+> img/document_basic_metadata.png +>
Document Basic Metadata
+>
+> +> Once all the fields are filled, click on the blue button `Next >>`{.interpreted-text role="guilabel"} in the bottom right corner of the page. + +2. On the **Location and Licenses** page, the following information should be filled: + +> - *Language* of the document; +> - *Regions*, which informs on the spatial extent covered by the document. Proposed extents cover the following scales: global, continental, regional, national; +> - *Data Quality statement* (general explanation of the data producer\'s knowledge about the lineage of a dataset); +> - Potential *Restrictions* to sharing the document should be provided in the Restrictions box. +> +>
+> img/document_location_licenses.png +>
Document Location and Licenses
+>
+> +> Click on the blue button `Next >>`{.interpreted-text role="guilabel"} to go ahead to the next step. + +3. On the **Optional Metadata** page, complementary information can be added: + +> - *Edition* to indicate the reference or the source of the document; +> - *Purpose* of the document and its objectives; +> - *Supplemental information* that can provide a better understanding of the uploaded document; +> - *Maintenance frequency* of the document; +> - *Spatial representation type* used. +> - *Related resources* to link one or multiple resources to the document. These will be visible inside the `document-info`{.interpreted-text role="ref"} panel +> +> *Responsible Parties*, *Owner* and *Share options* are listed on the right side of the page, you can edit them. +> +>
+> img/document_optional_metadata.png +>
Document Optional Metadata
+>
+> +> | If all the mandatory information is filled out the document can be published, if not the `Completeness` progress bar warns you that something is missing. +> | Click on the blue button `Update`{.interpreted-text role="guilabel"} to save information on the system. diff --git a/docs/usage/managing_documents/document_permissions.md b/docs/usage/managing_documents/document_permissions.md new file mode 100644 index 00000000..7ec10897 --- /dev/null +++ b/docs/usage/managing_documents/document_permissions.md @@ -0,0 +1,32 @@ +# Share Options {#edit-document-permissions} + +GeoNode encourages to publicly, share and make available for download information uploaded on the platform. By default, anyone can see and download a document. However, the document responsible can choose to limit access to the document to some contributors and/or groups. + +Through the *Share* Link shown in the menu it is possible to manage the document share options. it opens a form where set up who can: + +- None (the document is not visible). +- View Metadata (allows to view the document). +- View and Download (allows to view and download the document). +- Edit (allows to change the document metadata and attributes). +- Manage it (allows to update, delete, change share options, publish/unpublish). + +![](img/permissions_form.png){.align-center} + +See an example in the picture below. + +::: note +::: title +Note +::: + +After making changes, always save them with the *Save* Link in the menu +::: + +
+img/changing_document_permissions.png +
Changing the Document share options
+
+ +Usually those with options of editing metadata and the management of a document are in charge and responsible of the document, i.e. the contributor who uploaded it has those options by default. + +Once the share options are set, click `Save`{.interpreted-text role="guilabel"} link in the menu to save them. diff --git a/usage/managing_documents/img/add_url.png b/docs/usage/managing_documents/img/add_url.png similarity index 100% rename from usage/managing_documents/img/add_url.png rename to docs/usage/managing_documents/img/add_url.png diff --git a/usage/managing_documents/img/advanced_metadata_button.png b/docs/usage/managing_documents/img/advanced_metadata_button.png similarity index 100% rename from usage/managing_documents/img/advanced_metadata_button.png rename to docs/usage/managing_documents/img/advanced_metadata_button.png diff --git a/usage/managing_documents/img/changing_document_permissions.png b/docs/usage/managing_documents/img/changing_document_permissions.png similarity index 100% rename from usage/managing_documents/img/changing_document_permissions.png rename to docs/usage/managing_documents/img/changing_document_permissions.png diff --git a/usage/managing_documents/img/copy_locations.png b/docs/usage/managing_documents/img/copy_locations.png similarity index 100% rename from usage/managing_documents/img/copy_locations.png rename to docs/usage/managing_documents/img/copy_locations.png diff --git a/usage/managing_documents/img/document_basic_metadata.png b/docs/usage/managing_documents/img/document_basic_metadata.png similarity index 100% rename from usage/managing_documents/img/document_basic_metadata.png rename to docs/usage/managing_documents/img/document_basic_metadata.png diff --git a/usage/managing_documents/img/document_detail.png b/docs/usage/managing_documents/img/document_detail.png similarity index 100% rename from usage/managing_documents/img/document_detail.png rename to docs/usage/managing_documents/img/document_detail.png diff --git a/usage/managing_documents/img/document_download.png b/docs/usage/managing_documents/img/document_download.png similarity index 100% rename from usage/managing_documents/img/document_download.png rename to docs/usage/managing_documents/img/document_download.png diff --git a/usage/managing_documents/img/document_info.png b/docs/usage/managing_documents/img/document_info.png similarity index 100% rename from usage/managing_documents/img/document_info.png rename to docs/usage/managing_documents/img/document_info.png diff --git a/usage/managing_documents/img/document_info_toolbar.png b/docs/usage/managing_documents/img/document_info_toolbar.png similarity index 100% rename from usage/managing_documents/img/document_info_toolbar.png rename to docs/usage/managing_documents/img/document_info_toolbar.png diff --git a/usage/managing_documents/img/document_linked_resources.png b/docs/usage/managing_documents/img/document_linked_resources.png similarity index 100% rename from usage/managing_documents/img/document_linked_resources.png rename to docs/usage/managing_documents/img/document_linked_resources.png diff --git a/usage/managing_documents/img/document_location.png b/docs/usage/managing_documents/img/document_location.png similarity index 100% rename from usage/managing_documents/img/document_location.png rename to docs/usage/managing_documents/img/document_location.png diff --git a/usage/managing_documents/img/document_location_licenses.png b/docs/usage/managing_documents/img/document_location_licenses.png similarity index 100% rename from usage/managing_documents/img/document_location_licenses.png rename to docs/usage/managing_documents/img/document_location_licenses.png diff --git a/usage/managing_documents/img/document_metadata_editing.png b/docs/usage/managing_documents/img/document_metadata_editing.png similarity index 100% rename from usage/managing_documents/img/document_metadata_editing.png rename to docs/usage/managing_documents/img/document_metadata_editing.png diff --git a/usage/managing_documents/img/document_optional_metadata.png b/docs/usage/managing_documents/img/document_optional_metadata.png similarity index 100% rename from usage/managing_documents/img/document_optional_metadata.png rename to docs/usage/managing_documents/img/document_optional_metadata.png diff --git a/usage/managing_documents/img/document_overview.png b/docs/usage/managing_documents/img/document_overview.png similarity index 100% rename from usage/managing_documents/img/document_overview.png rename to docs/usage/managing_documents/img/document_overview.png diff --git a/usage/managing_documents/img/document_sharing.png b/docs/usage/managing_documents/img/document_sharing.png similarity index 100% rename from usage/managing_documents/img/document_sharing.png rename to docs/usage/managing_documents/img/document_sharing.png diff --git a/usage/managing_documents/img/document_upload_page.png b/docs/usage/managing_documents/img/document_upload_page.png similarity index 100% rename from usage/managing_documents/img/document_upload_page.png rename to docs/usage/managing_documents/img/document_upload_page.png diff --git a/usage/managing_documents/img/document_uploading_thumbnail.png b/docs/usage/managing_documents/img/document_uploading_thumbnail.png similarity index 100% rename from usage/managing_documents/img/document_uploading_thumbnail.png rename to docs/usage/managing_documents/img/document_uploading_thumbnail.png diff --git a/usage/managing_documents/img/document_uploading_thumbnail_success.png b/docs/usage/managing_documents/img/document_uploading_thumbnail_success.png similarity index 100% rename from usage/managing_documents/img/document_uploading_thumbnail_success.png rename to docs/usage/managing_documents/img/document_uploading_thumbnail_success.png diff --git a/usage/managing_documents/img/download_share_document.png b/docs/usage/managing_documents/img/download_share_document.png similarity index 100% rename from usage/managing_documents/img/download_share_document.png rename to docs/usage/managing_documents/img/download_share_document.png diff --git a/usage/managing_documents/img/edit-location.png b/docs/usage/managing_documents/img/edit-location.png similarity index 100% rename from usage/managing_documents/img/edit-location.png rename to docs/usage/managing_documents/img/edit-location.png diff --git a/usage/managing_documents/img/edit_metadata_link.png b/docs/usage/managing_documents/img/edit_metadata_link.png similarity index 100% rename from usage/managing_documents/img/edit_metadata_link.png rename to docs/usage/managing_documents/img/edit_metadata_link.png diff --git a/usage/managing_documents/img/favorite_document.png b/docs/usage/managing_documents/img/favorite_document.png similarity index 100% rename from usage/managing_documents/img/favorite_document.png rename to docs/usage/managing_documents/img/favorite_document.png diff --git a/usage/managing_documents/img/locations_document.png b/docs/usage/managing_documents/img/locations_document.png similarity index 100% rename from usage/managing_documents/img/locations_document.png rename to docs/usage/managing_documents/img/locations_document.png diff --git a/usage/managing_documents/img/permissions_form.png b/docs/usage/managing_documents/img/permissions_form.png similarity index 100% rename from usage/managing_documents/img/permissions_form.png rename to docs/usage/managing_documents/img/permissions_form.png diff --git a/usage/managing_documents/img/select_files.png b/docs/usage/managing_documents/img/select_files.png similarity index 100% rename from usage/managing_documents/img/select_files.png rename to docs/usage/managing_documents/img/select_files.png diff --git a/usage/managing_documents/img/title_document.png b/docs/usage/managing_documents/img/title_document.png similarity index 100% rename from usage/managing_documents/img/title_document.png rename to docs/usage/managing_documents/img/title_document.png diff --git a/usage/managing_documents/img/upload_document.png b/docs/usage/managing_documents/img/upload_document.png similarity index 100% rename from usage/managing_documents/img/upload_document.png rename to docs/usage/managing_documents/img/upload_document.png diff --git a/usage/managing_documents/img/upload_document_link.png b/docs/usage/managing_documents/img/upload_document_link.png similarity index 100% rename from usage/managing_documents/img/upload_document_link.png rename to docs/usage/managing_documents/img/upload_document_link.png diff --git a/usage/managing_documents/img/upload_thumbnail.png b/docs/usage/managing_documents/img/upload_thumbnail.png similarity index 100% rename from usage/managing_documents/img/upload_thumbnail.png rename to docs/usage/managing_documents/img/upload_thumbnail.png diff --git a/usage/managing_documents/img/view_metadata.png b/docs/usage/managing_documents/img/view_metadata.png similarity index 100% rename from usage/managing_documents/img/view_metadata.png rename to docs/usage/managing_documents/img/view_metadata.png diff --git a/docs/usage/managing_documents/index.md b/docs/usage/managing_documents/index.md new file mode 100644 index 00000000..963d77d4 --- /dev/null +++ b/docs/usage/managing_documents/index.md @@ -0,0 +1,20 @@ +# Managing Documents + +| In this section all the aspects concerning `document-data-type`{.interpreted-text role="ref"} will be discussed. +| You will learn how to upload a document and how to inspect its metadata and details. All the editing tools will be also explained. + +::: {.toctree maxdepth="1"} +uploading_documents +document_metadata +document_info +document_editing +document_permissions +::: + + +- [uploading_documents](uploading_documents.md) +- [document_metadata](document_metadata.md) +- [document_info](document_info.md) +- [document_editing](document_editing.md) +- [document_permissions](document_permissions.md) + diff --git a/docs/usage/managing_documents/uploading_documents.md b/docs/usage/managing_documents/uploading_documents.md new file mode 100644 index 00000000..14339c43 --- /dev/null +++ b/docs/usage/managing_documents/uploading_documents.md @@ -0,0 +1,80 @@ +::: {#uploading-documents} +Upload/Add Documents +=================== +::: + +GeoNode allows to share reports, conceptual notes, posters, spreadsheets, etc. A wide range of documents files can be hosted on the platform, including text files (.doc, .docx, .txt, .odt), spreadsheets (.xls, .xlsx, .ods), presentations (.ppt, .pptx, .odp), images (.gif, .jpg, .png, .tif, .tiff), PDF, zip files (.rar, .zip, .gz), SLD, XML, QML files or as External URL. + +::: warning +::: title +Warning +::: + +Only authenticated users can upload data into GeoNode. +::: + +It is possible to upload a *Documents* in two ways: + +- From the `All Resources`{.interpreted-text role="guilabel"} page, by clicking *Add Resource* which displays a list including [Upload document]{.title-ref} link: + +
+ img/upload_document_link.png +
Document Upload from All Resources page
+
+ +- From the `Documents`{.interpreted-text role="guilabel"} page, by clicking on *New* button. + +
+img/upload_document.png +
Document Upload from Documents page
+
+ +The *Document Upload* page looks like the one shown in the picture below. + +>
+> img/document_upload_page.png +>
Document Upload page
+>
+ +On *GeoNode* documents can be: + +- Upload from the **Local file** +- Created with reference to **External URL** + +In order to upload a document from the **Local file**, you need to: + +1. Click on `Select files`{.interpreted-text role="guilabel"} button +2. Select a file from your disk. +3. Click the `Upload`{.interpreted-text role="guilabel"} button + +>
+> img/select_files.png +>
Upload document from the select files option
+>
+ +| A document may refer to a remote document, without making a local copy of the remote resource. +| To add a document that references an **External URL** you need to: + +1. Click on `Add URL`{.interpreted-text role="guilabel"} button +2. Select an URL +3. Select an extension from the drop-down menu +4. Click the `Upload`{.interpreted-text role="guilabel"} button + +>
+> img/add_url.png +>
Add document from the add URL option
+>
+ +At the end of the uploading process, by clicking on the View button, you will be driven to the document page with the Info panel open. Here it is possible to view more info, edit metadata, share, download, and delete the document. See the next section to know more about Metatadata. + +::: note +::: title +Note +::: + +If you get the following error message: + +`Total upload size exceeds 100.0 MB. Please try again with smaller files.` + +This means that there is an upload size limit of 100 MB. An user with administrative access can change the upload limits at the [admin panel](../../admin/upload-size-limits/index.html#upload-size-limits). +::: diff --git a/docs/usage/managing_maps/creating_maps.md b/docs/usage/managing_maps/creating_maps.md new file mode 100644 index 00000000..ed528e5c --- /dev/null +++ b/docs/usage/managing_maps/creating_maps.md @@ -0,0 +1,44 @@ +# Creating Maps {#creating-map} + +In this section, we'll create a *Map* using some uploaded datasets, combine them with some other datasets from remote web services, and then share the resulting map for public viewing. + +In order to create new maps you can use: + +- The `Create map`{.interpreted-text role="guilabel"} listed after clicking the `Add Resource`{.interpreted-text role="guilabel"} button on the *All Resources* list page. + +
+ img/create_map_link.png +
The Create Map from All Resources page
+
+ +- The `New`{.interpreted-text role="guilabel"} button after clicking the `Maps`{.interpreted-text role="guilabel"} button on the menu bar. + +
+ img/new_map_link.png +
The Create Map from Maps page
+
+ +- The `Create map`{.interpreted-text role="guilabel"} link in the *Dataset Page* (it creates a map using a specific dataset) + +
+ img/create_map_button.png +
The Create map from dataset
+
+ +The new *Map* will open in a *Map Viewer* like the one in the picture below. + +
+img/map_viewer.png +
The Map Viewer
+
+ +Using the `Add dataset`{.interpreted-text role="guilabel"} link, you can add a layer by clicking on one of the layers listed in the catalog. +In the upper left corner the *TOC button* button opens the `toc`{.interpreted-text role="ref"} of the *Map*. It allows to manage all the datasets associated with the map and to add new ones from the `Add dataset`{.interpreted-text role="guilabel"}. +The *TOC* component makes possible to manage datasets overlap on the map by shifting their relative positions in the list (drag and drop them up or down in the list). +It also allows to hide/show datasets ( ![show_button](img/show_button.png){.align-middle width="30px" height="30px"} and ![hide_button](img/hide_button.png){.align-middle width="30px" height="30px"} ), to zoom to datasets extents ( ![zoom_to_dataset_extent_button](img/zoom_to_dataset_extent_button.png){.align-middle width="30px" height="30px"} ) and to manage their properties ( ![dataset_settings_button](img/dataset_settings_button.png){.align-middle width="30px" height="30px"} ). +Once the map datasets have been settled it is possible to save the *Map* by clicking on the `Save as`{.interpreted-text role="guilabel"} under the `Resources`{.interpreted-text role="guilabel"} link in the map toolbar. + +If you followed the steps above, you have just created your first *Map*. +Now you should see it in the *Explore Maps* page, see `map-info`{.interpreted-text role="ref"} for further details. + +We will take a closer look at the *Map Viewer* tools in the `exploring-maps`{.interpreted-text role="ref"} section. diff --git a/usage/managing_maps/img/advanced_edit_button.png b/docs/usage/managing_maps/img/advanced_edit_button.png similarity index 100% rename from usage/managing_maps/img/advanced_edit_button.png rename to docs/usage/managing_maps/img/advanced_edit_button.png diff --git a/usage/managing_maps/img/basic_map_metadata.png b/docs/usage/managing_maps/img/basic_map_metadata.png similarity index 100% rename from usage/managing_maps/img/basic_map_metadata.png rename to docs/usage/managing_maps/img/basic_map_metadata.png diff --git a/usage/managing_maps/img/burger_menu_button.png b/docs/usage/managing_maps/img/burger_menu_button.png similarity index 100% rename from usage/managing_maps/img/burger_menu_button.png rename to docs/usage/managing_maps/img/burger_menu_button.png diff --git a/usage/managing_maps/img/change_map_permissions.png b/docs/usage/managing_maps/img/change_map_permissions.png similarity index 100% rename from usage/managing_maps/img/change_map_permissions.png rename to docs/usage/managing_maps/img/change_map_permissions.png diff --git a/usage/managing_maps/img/completeness_progress_bar.png b/docs/usage/managing_maps/img/completeness_progress_bar.png similarity index 100% rename from usage/managing_maps/img/completeness_progress_bar.png rename to docs/usage/managing_maps/img/completeness_progress_bar.png diff --git a/usage/managing_maps/img/copy_locations_map.png b/docs/usage/managing_maps/img/copy_locations_map.png similarity index 100% rename from usage/managing_maps/img/copy_locations_map.png rename to docs/usage/managing_maps/img/copy_locations_map.png diff --git a/usage/managing_maps/img/create_map_button.png b/docs/usage/managing_maps/img/create_map_button.png similarity index 100% rename from usage/managing_maps/img/create_map_button.png rename to docs/usage/managing_maps/img/create_map_button.png diff --git a/usage/managing_maps/img/create_map_link.png b/docs/usage/managing_maps/img/create_map_link.png similarity index 100% rename from usage/managing_maps/img/create_map_link.png rename to docs/usage/managing_maps/img/create_map_link.png diff --git a/usage/managing_maps/img/create_map_menu.png b/docs/usage/managing_maps/img/create_map_menu.png similarity index 100% rename from usage/managing_maps/img/create_map_menu.png rename to docs/usage/managing_maps/img/create_map_menu.png diff --git a/usage/managing_maps/img/create_new_map_button.png b/docs/usage/managing_maps/img/create_new_map_button.png similarity index 100% rename from usage/managing_maps/img/create_new_map_button.png rename to docs/usage/managing_maps/img/create_new_map_button.png diff --git a/usage/managing_maps/img/dataset_settings_button.png b/docs/usage/managing_maps/img/dataset_settings_button.png similarity index 100% rename from usage/managing_maps/img/dataset_settings_button.png rename to docs/usage/managing_maps/img/dataset_settings_button.png diff --git a/usage/managing_maps/img/favorite_map.png b/docs/usage/managing_maps/img/favorite_map.png similarity index 100% rename from usage/managing_maps/img/favorite_map.png rename to docs/usage/managing_maps/img/favorite_map.png diff --git a/usage/managing_maps/img/hide_button.png b/docs/usage/managing_maps/img/hide_button.png similarity index 100% rename from usage/managing_maps/img/hide_button.png rename to docs/usage/managing_maps/img/hide_button.png diff --git a/usage/managing_maps/img/location_licenses_map_metadata.png b/docs/usage/managing_maps/img/location_licenses_map_metadata.png similarity index 100% rename from usage/managing_maps/img/location_licenses_map_metadata.png rename to docs/usage/managing_maps/img/location_licenses_map_metadata.png diff --git a/usage/managing_maps/img/map_comments.png b/docs/usage/managing_maps/img/map_comments.png similarity index 100% rename from usage/managing_maps/img/map_comments.png rename to docs/usage/managing_maps/img/map_comments.png diff --git a/usage/managing_maps/img/map_detail_page.png b/docs/usage/managing_maps/img/map_detail_page.png similarity index 100% rename from usage/managing_maps/img/map_detail_page.png rename to docs/usage/managing_maps/img/map_detail_page.png diff --git a/usage/managing_maps/img/map_info.png b/docs/usage/managing_maps/img/map_info.png similarity index 100% rename from usage/managing_maps/img/map_info.png rename to docs/usage/managing_maps/img/map_info.png diff --git a/usage/managing_maps/img/map_info_tab.png b/docs/usage/managing_maps/img/map_info_tab.png similarity index 100% rename from usage/managing_maps/img/map_info_tab.png rename to docs/usage/managing_maps/img/map_info_tab.png diff --git a/usage/managing_maps/img/map_info_toolbar.png b/docs/usage/managing_maps/img/map_info_toolbar.png similarity index 100% rename from usage/managing_maps/img/map_info_toolbar.png rename to docs/usage/managing_maps/img/map_info_toolbar.png diff --git a/usage/managing_maps/img/map_linked_resources.png b/docs/usage/managing_maps/img/map_linked_resources.png similarity index 100% rename from usage/managing_maps/img/map_linked_resources.png rename to docs/usage/managing_maps/img/map_linked_resources.png diff --git a/usage/managing_maps/img/map_location.png b/docs/usage/managing_maps/img/map_location.png similarity index 100% rename from usage/managing_maps/img/map_location.png rename to docs/usage/managing_maps/img/map_location.png diff --git a/usage/managing_maps/img/map_metadata_detail_button.png b/docs/usage/managing_maps/img/map_metadata_detail_button.png similarity index 100% rename from usage/managing_maps/img/map_metadata_detail_button.png rename to docs/usage/managing_maps/img/map_metadata_detail_button.png diff --git a/usage/managing_maps/img/map_metadata_details_page.png b/docs/usage/managing_maps/img/map_metadata_details_page.png similarity index 100% rename from usage/managing_maps/img/map_metadata_details_page.png rename to docs/usage/managing_maps/img/map_metadata_details_page.png diff --git a/usage/managing_maps/img/map_overview.png b/docs/usage/managing_maps/img/map_overview.png similarity index 100% rename from usage/managing_maps/img/map_overview.png rename to docs/usage/managing_maps/img/map_overview.png diff --git a/usage/managing_maps/img/map_permissions_settings.png b/docs/usage/managing_maps/img/map_permissions_settings.png similarity index 100% rename from usage/managing_maps/img/map_permissions_settings.png rename to docs/usage/managing_maps/img/map_permissions_settings.png diff --git a/usage/managing_maps/img/map_preview.gif b/docs/usage/managing_maps/img/map_preview.gif similarity index 100% rename from usage/managing_maps/img/map_preview.gif rename to docs/usage/managing_maps/img/map_preview.gif diff --git a/usage/managing_maps/img/map_rating.png b/docs/usage/managing_maps/img/map_rating.png similarity index 100% rename from usage/managing_maps/img/map_rating.png rename to docs/usage/managing_maps/img/map_rating.png diff --git a/usage/managing_maps/img/map_sharing.png b/docs/usage/managing_maps/img/map_sharing.png similarity index 100% rename from usage/managing_maps/img/map_sharing.png rename to docs/usage/managing_maps/img/map_sharing.png diff --git a/usage/managing_maps/img/map_viewer.png b/docs/usage/managing_maps/img/map_viewer.png similarity index 100% rename from usage/managing_maps/img/map_viewer.png rename to docs/usage/managing_maps/img/map_viewer.png diff --git a/usage/managing_maps/img/maps_add_layers_button.png b/docs/usage/managing_maps/img/maps_add_layers_button.png similarity index 100% rename from usage/managing_maps/img/maps_add_layers_button.png rename to docs/usage/managing_maps/img/maps_add_layers_button.png diff --git a/usage/managing_maps/img/maps_layers_add_to_map.png b/docs/usage/managing_maps/img/maps_layers_add_to_map.png similarity index 100% rename from usage/managing_maps/img/maps_layers_add_to_map.png rename to docs/usage/managing_maps/img/maps_layers_add_to_map.png diff --git a/usage/managing_maps/img/maps_layers_button.png b/docs/usage/managing_maps/img/maps_layers_button.png similarity index 100% rename from usage/managing_maps/img/maps_layers_button.png rename to docs/usage/managing_maps/img/maps_layers_button.png diff --git a/usage/managing_maps/img/maps_layers_switcher.png b/docs/usage/managing_maps/img/maps_layers_switcher.png similarity index 100% rename from usage/managing_maps/img/maps_layers_switcher.png rename to docs/usage/managing_maps/img/maps_layers_switcher.png diff --git a/usage/managing_maps/img/maps_list.png b/docs/usage/managing_maps/img/maps_list.png similarity index 100% rename from usage/managing_maps/img/maps_list.png rename to docs/usage/managing_maps/img/maps_list.png diff --git a/usage/managing_maps/img/maps_save_map_button.png b/docs/usage/managing_maps/img/maps_save_map_button.png similarity index 100% rename from usage/managing_maps/img/maps_save_map_button.png rename to docs/usage/managing_maps/img/maps_save_map_button.png diff --git a/usage/managing_maps/img/maps_save_map_thumbnail.png b/docs/usage/managing_maps/img/maps_save_map_thumbnail.png similarity index 100% rename from usage/managing_maps/img/maps_save_map_thumbnail.png rename to docs/usage/managing_maps/img/maps_save_map_thumbnail.png diff --git a/usage/managing_maps/img/new_map_creation.gif b/docs/usage/managing_maps/img/new_map_creation.gif similarity index 100% rename from usage/managing_maps/img/new_map_creation.gif rename to docs/usage/managing_maps/img/new_map_creation.gif diff --git a/usage/managing_maps/img/new_map_link.png b/docs/usage/managing_maps/img/new_map_link.png similarity index 100% rename from usage/managing_maps/img/new_map_link.png rename to docs/usage/managing_maps/img/new_map_link.png diff --git a/usage/managing_maps/img/optional_map_metadata.png b/docs/usage/managing_maps/img/optional_map_metadata.png similarity index 100% rename from usage/managing_maps/img/optional_map_metadata.png rename to docs/usage/managing_maps/img/optional_map_metadata.png diff --git a/usage/managing_maps/img/show_button.png b/docs/usage/managing_maps/img/show_button.png similarity index 100% rename from usage/managing_maps/img/show_button.png rename to docs/usage/managing_maps/img/show_button.png diff --git a/usage/managing_maps/img/toc.png b/docs/usage/managing_maps/img/toc.png similarity index 100% rename from usage/managing_maps/img/toc.png rename to docs/usage/managing_maps/img/toc.png diff --git a/usage/managing_maps/img/toc_button.png b/docs/usage/managing_maps/img/toc_button.png similarity index 100% rename from usage/managing_maps/img/toc_button.png rename to docs/usage/managing_maps/img/toc_button.png diff --git a/usage/managing_maps/img/zoom_to_dataset_extent_button.png b/docs/usage/managing_maps/img/zoom_to_dataset_extent_button.png similarity index 100% rename from usage/managing_maps/img/zoom_to_dataset_extent_button.png rename to docs/usage/managing_maps/img/zoom_to_dataset_extent_button.png diff --git a/docs/usage/managing_maps/index.md b/docs/usage/managing_maps/index.md new file mode 100644 index 00000000..4bb3d926 --- /dev/null +++ b/docs/usage/managing_maps/index.md @@ -0,0 +1,20 @@ +# Managing Maps + +| Maps are sets of datasets displayed together on an interactive web map. Maps can be composed in the map composer and saved as GeoNode resources. Maps can also be associated with metadata, ratings, and comments. +| In this section, you will learn how to create a new map and share it. + +::: {.toctree maxdepth="1"} +creating_maps +maps_configuration/index +map_info +map_metadata +map_permissions +::: + + +- [creating_maps](creating_maps.md) +- [index](maps_configuration/index.md) +- [map_info](map_info.md) +- [map_metadata](map_metadata.md) +- [map_permissions](map_permissions.md) + diff --git a/docs/usage/managing_maps/map_info.md b/docs/usage/managing_maps/map_info.md new file mode 100644 index 00000000..f1277ae1 --- /dev/null +++ b/docs/usage/managing_maps/map_info.md @@ -0,0 +1,72 @@ +# Map Information {#map-info} + +From the *Map Search Page* (see `finding-data`{.interpreted-text role="ref"}) you can select the map you are interested in and click on its name to see an overview about it. + +![](img/map_overview.png){.align-center} + +The information panel reports: + +- The *Info* tab is active by default. This tab section shows some map metadata such as its title, the abstract, date of publication etc. The metadata also indicates the map owner, what are the topic categories the map belongs to and which regions are affected. + +
+img/map_info.png +
Map Info tab
+
+ +- The *Location* tab shows the spacial extent of the map. + +
+img/map_location.png +
Map Location tab
+
+ +By clicking on the copy icons you have a copy of the current *Bounding Box* or the *Center* in the clipboard which once pasted will be a WKT text. + +
+img/copy_locations_map.png +
Bounding Box and Center
+
+ +- The *Linked Resources* tab shows resources, that can be *Documents*, *Datasets*, *GeoStories* and *Dashboards*, linked to the map. + +
+img/map_linked_resources.png +
Map Linked Resources tab
+
+ +::: note +::: title +Note +::: + +When a resource is [linked from]{.title-ref} the map the link is shown under the *Linked by* list while when a resource is [linked to]{.title-ref} the map the link is shown under the *Used by* list +::: + +From the right toolbar on the **Info** panel it is possible: + +
+img/map_info_toolbar.png +
Map Info toolbar
+
+ +- Add map in your *Favorites* (see `editing-profile`{.interpreted-text role="ref"}), clicking on star icon. +- *Share* the map url +- Change the map *Title* by clicking on it + +You can access the map details page by clicking on *View map* in the overview panel. +That page looks like the one shown in the picture below. + +
+img/map_detail_page.png +
Map page
+
+ +The *Map Viewer* shows the map with very basic functionalities: + +- the *Base Map Switcher* that allows you to change the base map; +- the *Zoom in/out* tool to enlarge and decrease the view; +- the *Zoom to max extent* tool for the zoom to fit the datasets extents; +- the *Query Objects* tool to retrieve information about the map objects by clicking on the map; +- the *Print* tool to print the preview. +- the *Measure* tool to measure Distances, Areas and the Bearing of lines. +- the *Annotations* tool to add an annotation to the map diff --git a/docs/usage/managing_maps/map_metadata.md b/docs/usage/managing_maps/map_metadata.md new file mode 100644 index 00000000..2bb623f8 --- /dev/null +++ b/docs/usage/managing_maps/map_metadata.md @@ -0,0 +1,101 @@ +# Maps Metadata {#map-metadata} + +*Maps Metadata* can be Edited by clicking the `Edit Metadata`{.interpreted-text role="guilabel"} link from the *Map Detail* page. + +
+img/map_metadata_detail_button.png +
The Map Metadata Detail link
+
+ +The *Map Metadata* Edit form will open. +\| Metadata provide essential information for the identification and the comprehension of the map. They also make the map more easily retrievable through the search tools. +\| Those *Metadata* can be filled out through three-steps in which you have to provide all mandatory information to complete the process. Those three steps are described below. + +- **Basic Metadata** + +
+ img/basic_map_metadata.png +
Basic Map Metadata
+
+ + In the first step the system asks you to insert the following metadata (required fields are highlighted with red outlines): + + - *Thumbnail* of the map (click `Edit`{.interpreted-text role="guilabel"} to change it); + + - *Title* of the map, which should be clear and understandable; + + - *Abstract*; brief narrative summary of the content of the Map + + > ::: note + > ::: title + > Note + > ::: + > + > The *Abstract* panel allows you to insert HTML code through a *wysiwyg* text editor + > ::: + + - *Creation/Publication/Revision Dates* which define the time period that is covered by the map; + + - *Keywords*, which should be chosen within the available list; + + - *Category* which the map belongs to; + + - *Group* which the map is linked to. + + Click `Next >>`{.interpreted-text role="guilabel"} to go to the next step. + +- **Location and Licenses** + +
+ img/location_licenses_map_metadata.png +
Location and Licenses Metadata for Maps
+
+ + The following list shows what kinds of metadata you are required to enter (see also the picture below): + + - *Language* of the layer; + - *License* of the dataset; + - *Regions* covered by the layers extent. Proposed extents cover the following scales: global, continental, regional, national; + - *Data Quality statement* (general explanation of the data producer\'s knowledge about the lineage of a dataset); + - Potential *Restrictions* on layer sharing. + + No further mandatory metadata are required in the next step so, once the required fields have been filled out, a green `Done`{.interpreted-text role="guilabel"} button will be visible in the screen. + Click `Next >>`{.interpreted-text role="guilabel"} to go to the next step or `<< Back`{.interpreted-text role="guilabel"} to go back to the previous step. + +- **Optional Metadata** + +
+ img/optional_map_metadata.png +
Optional Map Metadata
+
+ + Complementary information are: + + - *Edition* of the map; + - *Purpose* of the map and its objectives; + - *Supplemental information* that can provide a better understanding of the map; + - *Maintenance frequency* of the map; + - *Spatial representation type*, the method used to represent geographic information in the dataset; + - Users who are *Responsible* for the layer, its *Owner*, and the *Author* of its metadata; + - *Related resources* to link one or multiple resources to the document. These will be visible inside the `map-info`{.interpreted-text role="ref"} panel + +If you miss some mandatory metadata the *Completeness* bar shows you a red message like the one in the picture below. + +
+img/completeness_progress_bar.png +
Completeness Progress Bar
+
+ +## Metadata Advanced Editing + +The `Advanced Metadata`{.interpreted-text role="guilabel"} editing button in the Metadata Edit form allows to change the map metadata. + +
+img/advanced_edit_button.png +
The Advanced Edit button
+
+ +Click on it to display the *Metadata Advanced Editing Page*. +That page allows you to edit all the map metadata described in the previous paragraph. + +Once you have finished to edit them click on `Update`{.interpreted-text role="guilabel"} to save your changes. diff --git a/docs/usage/managing_maps/map_permissions.md b/docs/usage/managing_maps/map_permissions.md new file mode 100644 index 00000000..6a61c4f1 --- /dev/null +++ b/docs/usage/managing_maps/map_permissions.md @@ -0,0 +1,33 @@ +# Share Options {#map-permissions} + +In GeoNode the share options management system is indeed more complex. +Administrators can choose who can do what for each map. +Users can manage only the maps they own or the maps which they are authorize to manage. + +By default only owners can edit and manage maps, and anyone can view them. + +In order to modify the *Map Share Options* settings you can click the `Share`{.interpreted-text role="guilabel"} link in the *Map Detail Page*. + +Through the *Share Options Settings Panel* you can add or remove options for users and groups. The picture below shows an example. + +
+img/map_permissions_settings.png +
Map Share options settings for users and groups
+
+ +You can set the following options: + +- *View* (allows to view the map). +- *Download* (allows to view and download the map). +- *Edit* (allows to change the map\'s metadata); +- *Manage* allows to update, delete, change share options, publish and unpublish the map. + +::: warning +::: title +Warning +::: + +When assigning options to a group, all the group members will have that option. Be careful in case of editing them. +::: + +Click on `Save`{.interpreted-text role="guilabel"} link in the menu to save these settings. diff --git a/docs/usage/managing_maps/maps_configuration/attribute_table.md b/docs/usage/managing_maps/maps_configuration/attribute_table.md new file mode 100644 index 00000000..ad511db9 --- /dev/null +++ b/docs/usage/managing_maps/maps_configuration/attribute_table.md @@ -0,0 +1,88 @@ +# Attributes Table + +When clicking on the ![attribute_table_button](img/attribute_table_button.png){.align-middle width="30px" height="30px"} button of the `toc`{.interpreted-text role="ref"}, the *Attributes Table* panel opens at the bottom of the *Map* page. + +
+img/attributes_table.png +
The Attributes Table Panel
+
+ +In that panel you can navigate through the features of the dataset, zoom to their geometries by clicking on the ![zoom_to_feature_icon](img/zoom_to_feature_icon.png){.align-middle width="30px" height="30px"} icon and explore their attributes. + +| The *Attribute Tables* has a row for each feature belonging to the dataset and a column for each attribute that describes the feature. +| Each column has a *Filter* input field through which you can filter the features basing on some value or expression (depending on the data type of the field). + +
+img/filtering_feature_by_attributes.png +
Filtering Features by Attribute
+
+ +The *Attributes Table* panel contains a *Toolbar* which makes you available some useful functionalities. + +
+img/attributes_table_toolbar.png +
The Attributes Table Toolbar
+
+ +Those functionalities are: + +- | *Edit Mode* + | By clicking on you can start an editing session. It permits you to add new features, to delete or modify the existing ones, to edit geometries. See the `dataset-data-editing`{.interpreted-text role="ref"} section for further information. + +- | *Advanced Search* + | Click on , a new panel opens. That panel allows you to filter features in many different ways. This functionality will be explained in depth in the `advanced-search`{.interpreted-text role="ref"} section. + +- | *Zoom to page extent* + | Click on to zoom to the page extent. + +- | *Export Data* + | Click on to open the export/download data form. + +- | *Hide/show columns* + | When clicking on another panel opens inside the *Attributes Table*. Through that panel you can choose what columns you want to see. + +- | *Create a chart* + | Through the button you can open the *Chart Widgets* panel where many functionalities to describe and visualize the dataset data are available (see `creating-widgets`{.interpreted-text role="ref"}). + +- | *Sync map with filter* + | Click on the icon to synchronize the map with the filter. + +## Advanced Search + +As mentioned before, GeoNode allows both an attribute based and spatial filtering. +When clicking on from the dataset *Attributes Table* the *Advanced Search* panel opens and shows you three different filtering functionalities: + +
+img/advanced_search.png +
Advanced Search
+
+ +- In the **Attribute Filter** section you can compose a series of conditions about the attributes of the dataset. + Click on to insert a new empty condition. + Select the attribute you are interested in, select an operator and type a comparison value. + You can group conditions through the *Add Group* button. + Click on to perform the search. + +
+ img/filtering_by_attributes.png +
Filtering by Attributes
+
+ + You can also decide if *All* the conditions have to be met, if only *Any* or *None* of them (see the red arrow in the picture above). + +- | The \*\* Area of interest\*\* filtering allows you to filter features that have some relationship with a spatial region that you draw on the map. + | Select the *Filter Type* (Circle, Viewport, Polygon or Rectangle), draw the spatial region of interest on the map, select a *Geometric Operation* (Intersects, Bounding Box, Contains or Is contained) and then click on . + +
+ img/filtering_region_of_interest.png +
Filtering by Area Of Interest
+
+ +- Through the **Dataset Filter** you can select only those features which comply with some conditions on other datasets of the map. You can also add conditions on attributes for those datasets. + +
+ img/dataset_filtering.png +
Dataset Filtering
+
+ +You can read more about the *Attributes Table* and the *Advanced Search* on the [MapStore Documentation](https://docs.mapstore.geosolutionsgroup.com/en/latest/user-guide/filtering-layers/#query-panel). diff --git a/docs/usage/managing_maps/maps_configuration/creating_widgets.md b/docs/usage/managing_maps/maps_configuration/creating_widgets.md new file mode 100644 index 00000000..6b0e4e6b --- /dev/null +++ b/docs/usage/managing_maps/maps_configuration/creating_widgets.md @@ -0,0 +1,89 @@ +# Creating Widgets + +*Widgets* are graphical elements that describe the datasets data. They can be of different types such as *Charts*, *Texts*, *Tables* and *Counters*. +Through the ![create_widgets_button](img/create_widgets_button.png){.align-middle width="30px" height="30px"} button of the `toc`{.interpreted-text role="ref"} you can open the *Widgets* panel. + +
+img/widgets_panel.png +
Creating Widgets
+
+ +## Chart Widgets + +*Chart Widgets* are graphical representations of the dataset data. They can be *Bar Chart*, *Pie Chart* or *Line Chart* as shown in the picture below. + +
+img/chart_widgets.png +
Chart Widgets
+
+ +| Lets create a new **Bar Chart**. +| Click on *Bar Chart* then select the *X Attribute*, the *Y Attribute*, the *Operation* and the *Color* do you prefer. You can also display the *Legend*, *Hide the Y axis*, *Hide the grid* and decide what *Label* display into the legend. +| Now you can filter the data to be considered for the chart by clicking on ![advanced_search_button](img/advanced_search_button.png){.align-middle width="30px" height="30px"}. We don\'t need any filter so click ![configure_widgets_options_button](img/configure_widgets_options_button.png){.align-middle width="30px" height="30px"} to configure other widget options. Insert a *Title* and a *Description* and click on *Save* ![save_button](img/save_button.png){.align-middle width="30px" height="30px"}. + +
+img/bar_chart_widget.png +
Chart Widgets Creation
+
+ +The green ![connect_to_viewport](img/connect_to_viewport.png){.align-middle width="30px" height="30px"} icon means that the chart is connected to the viewport. + +At the top of the bar chat, there is the options menu of the widget where you can *Download grapg as png*, *Zoom the wigets* and *Reset axes*. + +
+img/chart_widget_options.png +
Chart Widgets Options
+
+ +## Text Widgets + +If you select *Text* on the *Widgets* panel you can create *Text Widgets*. Add a *Title* and the desired descriptive text, then click on ![save_button](img/save_button.png){.align-middle width="30px" height="30px"}. + +
+img/text_widgets_creation.png +
Text Widgets Creation
+
+ +The resulting widget looks like the following. + +
+img/text_widget.png +
My Text Widget
+
+ +## Table Widgets + +| Through the *Table Widgets* you can add the `attributes-table`{.interpreted-text role="ref"} of the dataset to the map. You can decide to show a subset of the features, through filters, and you can select one or more columns/attributes. + +You can also enter a *Title* for each column to be displayed as the table header in place of the Name of the layer field and enter a *Description* for each field to be displayed as a tooltip, visible moving the mouse on the column header. +\| So, choose what attributes you are interested in and click on ![configure_widgets_options_button](img/configure_widgets_options_button.png){.align-middle width="30px" height="30px"}. + +
+img/table_widget_columns.png +
Table Widgets Columns
+
+ +Insert *Title* and *Description* (optional) and click on ![save_button](img/save_button.png){.align-middle width="30px" height="30px"}. The example below shows the *Table Widget* on the map. + +
+img/table_widget.png +
Table Widget
+
+ +## Counter Widgets + +*Counter Widgets* are numeric representations of some attributes. For example you can represent the average speed limit on a road network. + +
+img/counter_widget_creation.png +
Counter Widget Creation
+
+ +Click on ![configure_widgets_options_button](img/configure_widgets_options_button.png){.align-middle width="30px" height="30px"}, insert *Title* and *Description* then click on ![save_button](img/save_button.png){.align-middle width="30px" height="30px"}. + +
+img/counter_widget.png +
Counter Widget
+
+ +The GeoNode map viewer is [MapStore](https://mapstore2.geo-solutions.it/mapstore/#/) based, see the [MapStore Documentation](https://docs.mapstore.geosolutionsgroup.com/en/latest/user-guide/widgets/) for further information. diff --git a/docs/usage/managing_maps/maps_configuration/get_fetureinfo.md b/docs/usage/managing_maps/maps_configuration/get_fetureinfo.md new file mode 100644 index 00000000..81ee6fba --- /dev/null +++ b/docs/usage/managing_maps/maps_configuration/get_fetureinfo.md @@ -0,0 +1,112 @@ +::: {#getfetureinfo-templates} +Customizing The Datasets\' GetFeatureInfo Templates +================================================ +When "clicking" over a feature of a dataset into a GeoNode Map, an info window popups showing a formatted representation of the raw data identified by the coordinates (see Fig. 1) +::: + +
+img/getfetureinfo_templates_001.png +
Fig. 1
+
+ +The way how such information is presented to the user is defined by what we call "GetFeatureInfo Template". The latter is basically an HTML snippet containing some placeholders and special inline codes that instruct GeoServer on how to generate the raw data output. + +The outcome is a rendered HTML snippet with the real values replacing the placeholders of the Template. + +Currently, GeoNode allows a very simple mechanism to customize the "GetFeatureInfo Template" of a dataset. + +It is possible, through the dataset Metadata Editor, to assign a name, a label and also set the attributes we want to display on the GetFeatureInfo output. + +As an instance, by using the example above, we can customize a bit the dataset Metadata as shown in Fig. 2 + +
+img/getfetureinfo_templates_002.png +
Fig. 2
+
+ +The "GetFeatureInfo" output will change accordingly as shown in Fig. 3 + +
+img/getfetureinfo_templates_003.png +
Fig. 3
+
+ +# Simple Template: Assigning A Media-Type To Attribute Values + +The easiest way to render a different media-type (`image`{.interpreted-text role="guilabel"}, `audio`{.interpreted-text role="guilabel"}, `video`{.interpreted-text role="guilabel"} or `iframe`{.interpreted-text role="guilabel"}) to a property value, is to change it from the `Metadata Edit`{.interpreted-text role="guilabel"} attributes panel. + +By changing the `Display Type`{.interpreted-text role="guilabel"} of an attribute from this panel as shown in Fig. 4 + +
+img/getfetureinfo_templates_004.png +
Fig. 4
+
+ +GeoNode will create automatically the HTML media type when rendering by using the **value** of the selected property. + +So, as an example, if, in the figure above, the attribute `NAME` contains values representing some `links` to other resources, GeoNode will create those links automatically for you when clicking over a geometry. + +
+img/getfetureinfo_templates_005.png +
Fig. 5
+
+ +Selecting `image`{.interpreted-text role="guilabel"} as media-type (Fig. 6) + +
+img/getfetureinfo_templates_006.png +
Fig. 6
+
+ +and editing the contents accordingly (Fig. 7) + +
+img/getfetureinfo_templates_007.png +
Fig. 7
+
+ +you will get a nice effect as shown in Fig. 8 + +
+img/getfetureinfo_templates_008.png +
Fig. 8
+
+ +# Advanced Template: Use A Custom HTML Template + +By selecting the option `Use a custom template?`{.interpreted-text role="guilabel"} as shown in Fig. 9 + +
+img/getfetureinfo_templates_009.png +
Fig. 9
+
+ +You will be able to provide your own custom HTML Template for the Feature Info output. +Attributes from the dataset can be represented inside the template using `${properties.ATTRIBUTE}`, where `ATTRIBUTE` is the name of an attribute (field) from your dataset. + +The example below uses the `wikipedia` and the `name` attributes to show the name of the airport as a link to the Wikipedia page: + +
+img/getfetureinfo_templates_010.png +
Fig. 10
+
+ +
+img/getfetureinfo_templates_011.png +
Fig. 11
+
+ +This is the result shown when you click on an airport + +
+img/getfetureinfo_templates_012.png +
Fig. 12
+
+ +# Optional: Customizing the HTML WYSIWYG Editor Menu Bar + +The `Menu Bar`{.interpreted-text role="guilabel"} and `Tool Bar`{.interpreted-text role="guilabel"} of the HTML Editor, can be easily customized by overriding the `TINYMCE_DEFAULT_CONFIG` variable on `settings.py`{.interpreted-text role="guilabel"} (see `tinyMCE Default Config Settings`{.interpreted-text role="ref"}) + +There are many plugins and options allowing you to easily customize the editor and also provides some predefined *templates* to speed up the editing. + +For more information about the Javascript tool, please refer to diff --git a/usage/managing_maps/maps_configuration/img/3d_button.png b/docs/usage/managing_maps/maps_configuration/img/3d_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/3d_button.png rename to docs/usage/managing_maps/maps_configuration/img/3d_button.png diff --git a/usage/managing_maps/maps_configuration/img/add-viewer-page.png b/docs/usage/managing_maps/maps_configuration/img/add-viewer-page.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/add-viewer-page.png rename to docs/usage/managing_maps/maps_configuration/img/add-viewer-page.png diff --git a/usage/managing_maps/maps_configuration/img/add_an_annotations.png b/docs/usage/managing_maps/maps_configuration/img/add_an_annotations.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/add_an_annotations.png rename to docs/usage/managing_maps/maps_configuration/img/add_an_annotations.png diff --git a/usage/managing_maps/maps_configuration/img/add_annoations.png b/docs/usage/managing_maps/maps_configuration/img/add_annoations.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/add_annoations.png rename to docs/usage/managing_maps/maps_configuration/img/add_annoations.png diff --git a/usage/managing_maps/maps_configuration/img/add_annotations_button.png b/docs/usage/managing_maps/maps_configuration/img/add_annotations_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/add_annotations_button.png rename to docs/usage/managing_maps/maps_configuration/img/add_annotations_button.png diff --git a/usage/managing_maps/maps_configuration/img/add_condition_button.png b/docs/usage/managing_maps/maps_configuration/img/add_condition_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/add_condition_button.png rename to docs/usage/managing_maps/maps_configuration/img/add_condition_button.png diff --git a/usage/managing_maps/maps_configuration/img/add_group_button.png b/docs/usage/managing_maps/maps_configuration/img/add_group_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/add_group_button.png rename to docs/usage/managing_maps/maps_configuration/img/add_group_button.png diff --git a/usage/managing_maps/maps_configuration/img/advanced_search.png b/docs/usage/managing_maps/maps_configuration/img/advanced_search.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/advanced_search.png rename to docs/usage/managing_maps/maps_configuration/img/advanced_search.png diff --git a/usage/managing_maps/maps_configuration/img/advanced_search_button.png b/docs/usage/managing_maps/maps_configuration/img/advanced_search_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/advanced_search_button.png rename to docs/usage/managing_maps/maps_configuration/img/advanced_search_button.png diff --git a/usage/managing_maps/maps_configuration/img/animation_buttons.png b/docs/usage/managing_maps/maps_configuration/img/animation_buttons.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/animation_buttons.png rename to docs/usage/managing_maps/maps_configuration/img/animation_buttons.png diff --git a/usage/managing_maps/maps_configuration/img/animation_settings_button.png b/docs/usage/managing_maps/maps_configuration/img/animation_settings_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/animation_settings_button.png rename to docs/usage/managing_maps/maps_configuration/img/animation_settings_button.png diff --git a/usage/managing_maps/maps_configuration/img/annotation_button.png b/docs/usage/managing_maps/maps_configuration/img/annotation_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/annotation_button.png rename to docs/usage/managing_maps/maps_configuration/img/annotation_button.png diff --git a/usage/managing_maps/maps_configuration/img/attribute_table_button.png b/docs/usage/managing_maps/maps_configuration/img/attribute_table_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/attribute_table_button.png rename to docs/usage/managing_maps/maps_configuration/img/attribute_table_button.png diff --git a/usage/managing_maps/maps_configuration/img/attributes_table.png b/docs/usage/managing_maps/maps_configuration/img/attributes_table.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/attributes_table.png rename to docs/usage/managing_maps/maps_configuration/img/attributes_table.png diff --git a/usage/managing_maps/maps_configuration/img/attributes_table_toolbar.png b/docs/usage/managing_maps/maps_configuration/img/attributes_table_toolbar.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/attributes_table_toolbar.png rename to docs/usage/managing_maps/maps_configuration/img/attributes_table_toolbar.png diff --git a/usage/managing_maps/maps_configuration/img/bar_chart_widget.png b/docs/usage/managing_maps/maps_configuration/img/bar_chart_widget.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/bar_chart_widget.png rename to docs/usage/managing_maps/maps_configuration/img/bar_chart_widget.png diff --git a/usage/managing_maps/maps_configuration/img/basemap_switcher.png b/docs/usage/managing_maps/maps_configuration/img/basemap_switcher.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/basemap_switcher.png rename to docs/usage/managing_maps/maps_configuration/img/basemap_switcher.png diff --git a/usage/managing_maps/maps_configuration/img/basemap_switching.png b/docs/usage/managing_maps/maps_configuration/img/basemap_switching.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/basemap_switching.png rename to docs/usage/managing_maps/maps_configuration/img/basemap_switching.png diff --git a/usage/managing_maps/maps_configuration/img/chart_widget_options.png b/docs/usage/managing_maps/maps_configuration/img/chart_widget_options.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/chart_widget_options.png rename to docs/usage/managing_maps/maps_configuration/img/chart_widget_options.png diff --git a/usage/managing_maps/maps_configuration/img/chart_widgets.gif b/docs/usage/managing_maps/maps_configuration/img/chart_widgets.gif similarity index 100% rename from usage/managing_maps/maps_configuration/img/chart_widgets.gif rename to docs/usage/managing_maps/maps_configuration/img/chart_widgets.gif diff --git a/usage/managing_maps/maps_configuration/img/chart_widgets.png b/docs/usage/managing_maps/maps_configuration/img/chart_widgets.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/chart_widgets.png rename to docs/usage/managing_maps/maps_configuration/img/chart_widgets.png diff --git a/usage/managing_maps/maps_configuration/img/collapsed_sidebar.png b/docs/usage/managing_maps/maps_configuration/img/collapsed_sidebar.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/collapsed_sidebar.png rename to docs/usage/managing_maps/maps_configuration/img/collapsed_sidebar.png diff --git a/usage/managing_maps/maps_configuration/img/collapsed_timeline.png b/docs/usage/managing_maps/maps_configuration/img/collapsed_timeline.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/collapsed_timeline.png rename to docs/usage/managing_maps/maps_configuration/img/collapsed_timeline.png diff --git a/usage/managing_maps/maps_configuration/img/configure_widgets_options_button.png b/docs/usage/managing_maps/maps_configuration/img/configure_widgets_options_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/configure_widgets_options_button.png rename to docs/usage/managing_maps/maps_configuration/img/configure_widgets_options_button.png diff --git a/usage/managing_maps/maps_configuration/img/connect_to_viewport.png b/docs/usage/managing_maps/maps_configuration/img/connect_to_viewport.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/connect_to_viewport.png rename to docs/usage/managing_maps/maps_configuration/img/connect_to_viewport.png diff --git a/usage/managing_maps/maps_configuration/img/counter_widget.png b/docs/usage/managing_maps/maps_configuration/img/counter_widget.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/counter_widget.png rename to docs/usage/managing_maps/maps_configuration/img/counter_widget.png diff --git a/usage/managing_maps/maps_configuration/img/counter_widget_creation.png b/docs/usage/managing_maps/maps_configuration/img/counter_widget_creation.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/counter_widget_creation.png rename to docs/usage/managing_maps/maps_configuration/img/counter_widget_creation.png diff --git a/usage/managing_maps/maps_configuration/img/create_charts_button.png b/docs/usage/managing_maps/maps_configuration/img/create_charts_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/create_charts_button.png rename to docs/usage/managing_maps/maps_configuration/img/create_charts_button.png diff --git a/usage/managing_maps/maps_configuration/img/create_widgets_button.png b/docs/usage/managing_maps/maps_configuration/img/create_widgets_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/create_widgets_button.png rename to docs/usage/managing_maps/maps_configuration/img/create_widgets_button.png diff --git a/usage/managing_maps/maps_configuration/img/dataset_filtering.png b/docs/usage/managing_maps/maps_configuration/img/dataset_filtering.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/dataset_filtering.png rename to docs/usage/managing_maps/maps_configuration/img/dataset_filtering.png diff --git a/usage/managing_maps/maps_configuration/img/dataset_settings_panel.png b/docs/usage/managing_maps/maps_configuration/img/dataset_settings_panel.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/dataset_settings_panel.png rename to docs/usage/managing_maps/maps_configuration/img/dataset_settings_panel.png diff --git a/usage/managing_maps/maps_configuration/img/dataset_toolbar.png b/docs/usage/managing_maps/maps_configuration/img/dataset_toolbar.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/dataset_toolbar.png rename to docs/usage/managing_maps/maps_configuration/img/dataset_toolbar.png diff --git a/usage/managing_maps/maps_configuration/img/datasets_catalog.png b/docs/usage/managing_maps/maps_configuration/img/datasets_catalog.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/datasets_catalog.png rename to docs/usage/managing_maps/maps_configuration/img/datasets_catalog.png diff --git a/usage/managing_maps/maps_configuration/img/delete_dataset.png b/docs/usage/managing_maps/maps_configuration/img/delete_dataset.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/delete_dataset.png rename to docs/usage/managing_maps/maps_configuration/img/delete_dataset.png diff --git a/usage/managing_maps/maps_configuration/img/delete_dataset_button.png b/docs/usage/managing_maps/maps_configuration/img/delete_dataset_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/delete_dataset_button.png rename to docs/usage/managing_maps/maps_configuration/img/delete_dataset_button.png diff --git a/usage/managing_maps/maps_configuration/img/display_settings.png b/docs/usage/managing_maps/maps_configuration/img/display_settings.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/display_settings.png rename to docs/usage/managing_maps/maps_configuration/img/display_settings.png diff --git a/usage/managing_maps/maps_configuration/img/edit_mode_button.png b/docs/usage/managing_maps/maps_configuration/img/edit_mode_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/edit_mode_button.png rename to docs/usage/managing_maps/maps_configuration/img/edit_mode_button.png diff --git a/usage/managing_maps/maps_configuration/img/enable_plugins.png b/docs/usage/managing_maps/maps_configuration/img/enable_plugins.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/enable_plugins.png rename to docs/usage/managing_maps/maps_configuration/img/enable_plugins.png diff --git a/usage/managing_maps/maps_configuration/img/expand_timeline_button.png b/docs/usage/managing_maps/maps_configuration/img/expand_timeline_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/expand_timeline_button.png rename to docs/usage/managing_maps/maps_configuration/img/expand_timeline_button.png diff --git a/usage/managing_maps/maps_configuration/img/expanded_sidebar.png b/docs/usage/managing_maps/maps_configuration/img/expanded_sidebar.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/expanded_sidebar.png rename to docs/usage/managing_maps/maps_configuration/img/expanded_sidebar.png diff --git a/usage/managing_maps/maps_configuration/img/expanded_timeline.png b/docs/usage/managing_maps/maps_configuration/img/expanded_timeline.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/expanded_timeline.png rename to docs/usage/managing_maps/maps_configuration/img/expanded_timeline.png diff --git a/usage/managing_maps/maps_configuration/img/explore_map_viewer.png b/docs/usage/managing_maps/maps_configuration/img/explore_map_viewer.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/explore_map_viewer.png rename to docs/usage/managing_maps/maps_configuration/img/explore_map_viewer.png diff --git a/usage/managing_maps/maps_configuration/img/export_data_button.png b/docs/usage/managing_maps/maps_configuration/img/export_data_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/export_data_button.png rename to docs/usage/managing_maps/maps_configuration/img/export_data_button.png diff --git a/usage/managing_maps/maps_configuration/img/filtering_by_attributes.png b/docs/usage/managing_maps/maps_configuration/img/filtering_by_attributes.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/filtering_by_attributes.png rename to docs/usage/managing_maps/maps_configuration/img/filtering_by_attributes.png diff --git a/usage/managing_maps/maps_configuration/img/filtering_feature_by_attributes.png b/docs/usage/managing_maps/maps_configuration/img/filtering_feature_by_attributes.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/filtering_feature_by_attributes.png rename to docs/usage/managing_maps/maps_configuration/img/filtering_feature_by_attributes.png diff --git a/usage/managing_maps/maps_configuration/img/filtering_region_of_interest.png b/docs/usage/managing_maps/maps_configuration/img/filtering_region_of_interest.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/filtering_region_of_interest.png rename to docs/usage/managing_maps/maps_configuration/img/filtering_region_of_interest.png diff --git a/usage/managing_maps/maps_configuration/img/full_screen_button.png b/docs/usage/managing_maps/maps_configuration/img/full_screen_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/full_screen_button.png rename to docs/usage/managing_maps/maps_configuration/img/full_screen_button.png diff --git a/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_001.png b/docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_001.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/getfetureinfo_templates_001.png rename to docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_001.png diff --git a/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_002.png b/docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_002.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/getfetureinfo_templates_002.png rename to docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_002.png diff --git a/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_003.png b/docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_003.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/getfetureinfo_templates_003.png rename to docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_003.png diff --git a/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_004.png b/docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_004.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/getfetureinfo_templates_004.png rename to docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_004.png diff --git a/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_005.png b/docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_005.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/getfetureinfo_templates_005.png rename to docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_005.png diff --git a/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_006.png b/docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_006.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/getfetureinfo_templates_006.png rename to docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_006.png diff --git a/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_007.png b/docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_007.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/getfetureinfo_templates_007.png rename to docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_007.png diff --git a/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_008.png b/docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_008.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/getfetureinfo_templates_008.png rename to docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_008.png diff --git a/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_009.png b/docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_009.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/getfetureinfo_templates_009.png rename to docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_009.png diff --git a/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_010.png b/docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_010.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/getfetureinfo_templates_010.png rename to docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_010.png diff --git a/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_011.png b/docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_011.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/getfetureinfo_templates_011.png rename to docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_011.png diff --git a/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_012.png b/docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_012.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/getfetureinfo_templates_012.png rename to docs/usage/managing_maps/maps_configuration/img/getfetureinfo_templates_012.png diff --git a/usage/managing_maps/maps_configuration/img/go_back_button.png b/docs/usage/managing_maps/maps_configuration/img/go_back_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/go_back_button.png rename to docs/usage/managing_maps/maps_configuration/img/go_back_button.png diff --git a/usage/managing_maps/maps_configuration/img/go_forward_button.png b/docs/usage/managing_maps/maps_configuration/img/go_forward_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/go_forward_button.png rename to docs/usage/managing_maps/maps_configuration/img/go_forward_button.png diff --git a/usage/managing_maps/maps_configuration/img/hide_button.png b/docs/usage/managing_maps/maps_configuration/img/hide_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/hide_button.png rename to docs/usage/managing_maps/maps_configuration/img/hide_button.png diff --git a/usage/managing_maps/maps_configuration/img/hide_show_columns.gif b/docs/usage/managing_maps/maps_configuration/img/hide_show_columns.gif similarity index 100% rename from usage/managing_maps/maps_configuration/img/hide_show_columns.gif rename to docs/usage/managing_maps/maps_configuration/img/hide_show_columns.gif diff --git a/usage/managing_maps/maps_configuration/img/hide_show_columns_button.png b/docs/usage/managing_maps/maps_configuration/img/hide_show_columns_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/hide_show_columns_button.png rename to docs/usage/managing_maps/maps_configuration/img/hide_show_columns_button.png diff --git a/usage/managing_maps/maps_configuration/img/layer_filtering.png b/docs/usage/managing_maps/maps_configuration/img/layer_filtering.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/layer_filtering.png rename to docs/usage/managing_maps/maps_configuration/img/layer_filtering.png diff --git a/usage/managing_maps/maps_configuration/img/layers_catalog.png b/docs/usage/managing_maps/maps_configuration/img/layers_catalog.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/layers_catalog.png rename to docs/usage/managing_maps/maps_configuration/img/layers_catalog.png diff --git a/usage/managing_maps/maps_configuration/img/map_coordinates.png b/docs/usage/managing_maps/maps_configuration/img/map_coordinates.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/map_coordinates.png rename to docs/usage/managing_maps/maps_configuration/img/map_coordinates.png diff --git a/usage/managing_maps/maps_configuration/img/map_scale.png b/docs/usage/managing_maps/maps_configuration/img/map_scale.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/map_scale.png rename to docs/usage/managing_maps/maps_configuration/img/map_scale.png diff --git a/usage/managing_maps/maps_configuration/img/map_scaling.png b/docs/usage/managing_maps/maps_configuration/img/map_scaling.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/map_scaling.png rename to docs/usage/managing_maps/maps_configuration/img/map_scaling.png diff --git a/usage/managing_maps/maps_configuration/img/measure_button.png b/docs/usage/managing_maps/maps_configuration/img/measure_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/measure_button.png rename to docs/usage/managing_maps/maps_configuration/img/measure_button.png diff --git a/usage/managing_maps/maps_configuration/img/measure_tool.png b/docs/usage/managing_maps/maps_configuration/img/measure_tool.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/measure_tool.png rename to docs/usage/managing_maps/maps_configuration/img/measure_tool.png diff --git a/usage/managing_maps/maps_configuration/img/measuring_areas.png b/docs/usage/managing_maps/maps_configuration/img/measuring_areas.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/measuring_areas.png rename to docs/usage/managing_maps/maps_configuration/img/measuring_areas.png diff --git a/usage/managing_maps/maps_configuration/img/my_map.png b/docs/usage/managing_maps/maps_configuration/img/my_map.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/my_map.png rename to docs/usage/managing_maps/maps_configuration/img/my_map.png diff --git a/usage/managing_maps/maps_configuration/img/options_menu_tools.png b/docs/usage/managing_maps/maps_configuration/img/options_menu_tools.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/options_menu_tools.png rename to docs/usage/managing_maps/maps_configuration/img/options_menu_tools.png diff --git a/usage/managing_maps/maps_configuration/img/play_button.png b/docs/usage/managing_maps/maps_configuration/img/play_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/play_button.png rename to docs/usage/managing_maps/maps_configuration/img/play_button.png diff --git a/usage/managing_maps/maps_configuration/img/print_button.png b/docs/usage/managing_maps/maps_configuration/img/print_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/print_button.png rename to docs/usage/managing_maps/maps_configuration/img/print_button.png diff --git a/usage/managing_maps/maps_configuration/img/printing_window.png b/docs/usage/managing_maps/maps_configuration/img/printing_window.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/printing_window.png rename to docs/usage/managing_maps/maps_configuration/img/printing_window.png diff --git a/usage/managing_maps/maps_configuration/img/query_objects_on_map_button.png b/docs/usage/managing_maps/maps_configuration/img/query_objects_on_map_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/query_objects_on_map_button.png rename to docs/usage/managing_maps/maps_configuration/img/query_objects_on_map_button.png diff --git a/usage/managing_maps/maps_configuration/img/querying_objects_on_map.png b/docs/usage/managing_maps/maps_configuration/img/querying_objects_on_map.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/querying_objects_on_map.png rename to docs/usage/managing_maps/maps_configuration/img/querying_objects_on_map.png diff --git a/usage/managing_maps/maps_configuration/img/save_as_map.png b/docs/usage/managing_maps/maps_configuration/img/save_as_map.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/save_as_map.png rename to docs/usage/managing_maps/maps_configuration/img/save_as_map.png diff --git a/usage/managing_maps/maps_configuration/img/save_button.png b/docs/usage/managing_maps/maps_configuration/img/save_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/save_button.png rename to docs/usage/managing_maps/maps_configuration/img/save_button.png diff --git a/usage/managing_maps/maps_configuration/img/scrolling_opacity.png b/docs/usage/managing_maps/maps_configuration/img/scrolling_opacity.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/scrolling_opacity.png rename to docs/usage/managing_maps/maps_configuration/img/scrolling_opacity.png diff --git a/usage/managing_maps/maps_configuration/img/search_bar_result.png b/docs/usage/managing_maps/maps_configuration/img/search_bar_result.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/search_bar_result.png rename to docs/usage/managing_maps/maps_configuration/img/search_bar_result.png diff --git a/usage/managing_maps/maps_configuration/img/search_bar_typing_name.png b/docs/usage/managing_maps/maps_configuration/img/search_bar_typing_name.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/search_bar_typing_name.png rename to docs/usage/managing_maps/maps_configuration/img/search_bar_typing_name.png diff --git a/usage/managing_maps/maps_configuration/img/settings_toc_button.png b/docs/usage/managing_maps/maps_configuration/img/settings_toc_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/settings_toc_button.png rename to docs/usage/managing_maps/maps_configuration/img/settings_toc_button.png diff --git a/usage/managing_maps/maps_configuration/img/show_button.png b/docs/usage/managing_maps/maps_configuration/img/show_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/show_button.png rename to docs/usage/managing_maps/maps_configuration/img/show_button.png diff --git a/usage/managing_maps/maps_configuration/img/show_hide_coordinates_button.png b/docs/usage/managing_maps/maps_configuration/img/show_hide_coordinates_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/show_hide_coordinates_button.png rename to docs/usage/managing_maps/maps_configuration/img/show_hide_coordinates_button.png diff --git a/usage/managing_maps/maps_configuration/img/show_hide_datasets_list_button.png b/docs/usage/managing_maps/maps_configuration/img/show_hide_datasets_list_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/show_hide_datasets_list_button.png rename to docs/usage/managing_maps/maps_configuration/img/show_hide_datasets_list_button.png diff --git a/usage/managing_maps/maps_configuration/img/sidebar_expand_button.png b/docs/usage/managing_maps/maps_configuration/img/sidebar_expand_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/sidebar_expand_button.png rename to docs/usage/managing_maps/maps_configuration/img/sidebar_expand_button.png diff --git a/usage/managing_maps/maps_configuration/img/start_search_button.png b/docs/usage/managing_maps/maps_configuration/img/start_search_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/start_search_button.png rename to docs/usage/managing_maps/maps_configuration/img/start_search_button.png diff --git a/usage/managing_maps/maps_configuration/img/stop_button.png b/docs/usage/managing_maps/maps_configuration/img/stop_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/stop_button.png rename to docs/usage/managing_maps/maps_configuration/img/stop_button.png diff --git a/usage/managing_maps/maps_configuration/img/style_settings.png b/docs/usage/managing_maps/maps_configuration/img/style_settings.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/style_settings.png rename to docs/usage/managing_maps/maps_configuration/img/style_settings.png diff --git a/usage/managing_maps/maps_configuration/img/sync_map_with_filter_button.png b/docs/usage/managing_maps/maps_configuration/img/sync_map_with_filter_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/sync_map_with_filter_button.png rename to docs/usage/managing_maps/maps_configuration/img/sync_map_with_filter_button.png diff --git a/usage/managing_maps/maps_configuration/img/table_widget.png b/docs/usage/managing_maps/maps_configuration/img/table_widget.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/table_widget.png rename to docs/usage/managing_maps/maps_configuration/img/table_widget.png diff --git a/usage/managing_maps/maps_configuration/img/table_widget_columns.png b/docs/usage/managing_maps/maps_configuration/img/table_widget_columns.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/table_widget_columns.png rename to docs/usage/managing_maps/maps_configuration/img/table_widget_columns.png diff --git a/usage/managing_maps/maps_configuration/img/text_widget.png b/docs/usage/managing_maps/maps_configuration/img/text_widget.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/text_widget.png rename to docs/usage/managing_maps/maps_configuration/img/text_widget.png diff --git a/usage/managing_maps/maps_configuration/img/text_widgets_creation.png b/docs/usage/managing_maps/maps_configuration/img/text_widgets_creation.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/text_widgets_creation.png rename to docs/usage/managing_maps/maps_configuration/img/text_widgets_creation.png diff --git a/usage/managing_maps/maps_configuration/img/tiling_settings.png b/docs/usage/managing_maps/maps_configuration/img/tiling_settings.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/tiling_settings.png rename to docs/usage/managing_maps/maps_configuration/img/tiling_settings.png diff --git a/usage/managing_maps/maps_configuration/img/time_cursor.png b/docs/usage/managing_maps/maps_configuration/img/time_cursor.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/time_cursor.png rename to docs/usage/managing_maps/maps_configuration/img/time_cursor.png diff --git a/usage/managing_maps/maps_configuration/img/time_go_backward_button.png b/docs/usage/managing_maps/maps_configuration/img/time_go_backward_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/time_go_backward_button.png rename to docs/usage/managing_maps/maps_configuration/img/time_go_backward_button.png diff --git a/usage/managing_maps/maps_configuration/img/time_go_forward_button.png b/docs/usage/managing_maps/maps_configuration/img/time_go_forward_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/time_go_forward_button.png rename to docs/usage/managing_maps/maps_configuration/img/time_go_forward_button.png diff --git a/usage/managing_maps/maps_configuration/img/time_instant_controls.png b/docs/usage/managing_maps/maps_configuration/img/time_instant_controls.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/time_instant_controls.png rename to docs/usage/managing_maps/maps_configuration/img/time_instant_controls.png diff --git a/usage/managing_maps/maps_configuration/img/time_range.png b/docs/usage/managing_maps/maps_configuration/img/time_range.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/time_range.png rename to docs/usage/managing_maps/maps_configuration/img/time_range.png diff --git a/usage/managing_maps/maps_configuration/img/time_range_button.png b/docs/usage/managing_maps/maps_configuration/img/time_range_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/time_range_button.png rename to docs/usage/managing_maps/maps_configuration/img/time_range_button.png diff --git a/usage/managing_maps/maps_configuration/img/timeline_animation.gif b/docs/usage/managing_maps/maps_configuration/img/timeline_animation.gif similarity index 100% rename from usage/managing_maps/maps_configuration/img/timeline_animation.gif rename to docs/usage/managing_maps/maps_configuration/img/timeline_animation.gif diff --git a/usage/managing_maps/maps_configuration/img/timeline_animation.png b/docs/usage/managing_maps/maps_configuration/img/timeline_animation.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/timeline_animation.png rename to docs/usage/managing_maps/maps_configuration/img/timeline_animation.png diff --git a/usage/managing_maps/maps_configuration/img/timeline_histogram.png b/docs/usage/managing_maps/maps_configuration/img/timeline_histogram.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/timeline_histogram.png rename to docs/usage/managing_maps/maps_configuration/img/timeline_histogram.png diff --git a/usage/managing_maps/maps_configuration/img/timeline_settings.png b/docs/usage/managing_maps/maps_configuration/img/timeline_settings.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/timeline_settings.png rename to docs/usage/managing_maps/maps_configuration/img/timeline_settings.png diff --git a/usage/managing_maps/maps_configuration/img/toc_button.png b/docs/usage/managing_maps/maps_configuration/img/toc_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/toc_button.png rename to docs/usage/managing_maps/maps_configuration/img/toc_button.png diff --git a/usage/managing_maps/maps_configuration/img/toc_panel.png b/docs/usage/managing_maps/maps_configuration/img/toc_panel.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/toc_panel.png rename to docs/usage/managing_maps/maps_configuration/img/toc_panel.png diff --git a/usage/managing_maps/maps_configuration/img/view_map_button.png b/docs/usage/managing_maps/maps_configuration/img/view_map_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/view_map_button.png rename to docs/usage/managing_maps/maps_configuration/img/view_map_button.png diff --git a/usage/managing_maps/maps_configuration/img/visibility_settings.png b/docs/usage/managing_maps/maps_configuration/img/visibility_settings.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/visibility_settings.png rename to docs/usage/managing_maps/maps_configuration/img/visibility_settings.png diff --git a/usage/managing_maps/maps_configuration/img/widgets_panel.png b/docs/usage/managing_maps/maps_configuration/img/widgets_panel.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/widgets_panel.png rename to docs/usage/managing_maps/maps_configuration/img/widgets_panel.png diff --git a/usage/managing_maps/maps_configuration/img/zoom_in_button.png b/docs/usage/managing_maps/maps_configuration/img/zoom_in_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/zoom_in_button.png rename to docs/usage/managing_maps/maps_configuration/img/zoom_in_button.png diff --git a/usage/managing_maps/maps_configuration/img/zoom_out_button.png b/docs/usage/managing_maps/maps_configuration/img/zoom_out_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/zoom_out_button.png rename to docs/usage/managing_maps/maps_configuration/img/zoom_out_button.png diff --git a/usage/managing_maps/maps_configuration/img/zoom_to_feature_icon.png b/docs/usage/managing_maps/maps_configuration/img/zoom_to_feature_icon.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/zoom_to_feature_icon.png rename to docs/usage/managing_maps/maps_configuration/img/zoom_to_feature_icon.png diff --git a/usage/managing_maps/maps_configuration/img/zoom_to_max_extent_button.png b/docs/usage/managing_maps/maps_configuration/img/zoom_to_max_extent_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/zoom_to_max_extent_button.png rename to docs/usage/managing_maps/maps_configuration/img/zoom_to_max_extent_button.png diff --git a/usage/managing_maps/maps_configuration/img/zoom_to_page_extent_button.png b/docs/usage/managing_maps/maps_configuration/img/zoom_to_page_extent_button.png similarity index 100% rename from usage/managing_maps/maps_configuration/img/zoom_to_page_extent_button.png rename to docs/usage/managing_maps/maps_configuration/img/zoom_to_page_extent_button.png diff --git a/docs/usage/managing_maps/maps_configuration/index.md b/docs/usage/managing_maps/maps_configuration/index.md new file mode 100644 index 00000000..ffaf654c --- /dev/null +++ b/docs/usage/managing_maps/maps_configuration/index.md @@ -0,0 +1,125 @@ +# Map tools and configuration[]{#exploring-maps} {#asemap-switcher} + +In this section, we are going to explore all tools provided on the Map View page. From the list of available maps, you can select the map you are interested in and click *View map*, the map will look like this. + +
+img/explore_map_viewer.png +
The Map View
+
+ +The Map View (based on [MapStore](https://mapstore2.geo-solutions.it/mapstore/#/)) provides the following tools: + +- the `toc`{.interpreted-text role="ref"} to manage the map contents; +- the *Basemap Switcher* to change the basemap (see the next paragraphs); +- the *Search Bar* to search by location, name and coordinates (see the paragraph below); +- the `options-menu-tools`{.interpreted-text role="ref"} which contains the link to the datasets *Catalog*; +- the *Sidebar* which contains, by default, the link to the *Print* tool and to the *Measure* tool; +- the *Navigation Bar* and its tools such as the *Zoom* tools, the *3D Navigation* tool and the *Get Features Info* tool; +- the *Footer Tools* to manage the scale of the map, to track the mouse coordinates and change the CRS (Coordinates Reference System). + +A map can be configured to use a custom `map-viewers`{.interpreted-text role="ref"}, with which the lsit of tools available in the map can be customized. + +::: {.toctree hidden="" maxdepth="1"} +map_viewers +toc +attribute_table +creating_widgets +timeline +options_menu +get_fetureinfo +::: + +## Search Bar + +| The *Search Bar* of the map viewer allows you to find point of interests (POIs), streets or locations by name. +| Lets type the name of some place then select the first record. + +
+img/search_bar_typing_name.png +
The Search Bar
+
+ +The map will automatically re-center on that area delimiting it by a polygon in the case of an area, by a line in the case of a linear shape (e.g. streets, streams) and by a marker in the case of a point. + +
+img/search_bar_result.png +
Result of a Search
+
+ +::: {#sidebar} +Navigation bar +\-\-\-\-\-\-\-\-\-\-\-\-- +::: + +| The *Map Viewer* makes also available the *Navigation bar*. It is a navigation panel containing various tools that help you to explore the map such as tools for zooming, changing the extent and querying objects on the map. +| By default the *Navigation bar* shows you the zooming buttons ![zoom_in_button](img/zoom_in_button.png){.align-middle width="30px" height="30px"}, ![zoom_out_button](img/zoom_out_button.png){.align-middle width="30px" height="30px"} and ![3d_button](img/3d_button.png){.align-middle width="30px" height="30px"}, other options can be explored by clicking on ![sidebar_expand_button](img/sidebar_expand_button.png){.align-middle width="30px" height="30px"} which expands/collapses the toolbar.\| + +
+img/collapsed_sidebar.png +
The Default Navigation bar
+
+ +
+img/expanded_sidebar.png +
The Expanded Navigation bar
+
+ +The *Navigation bar* contains the following tools: + +- The *Query Objects on map* allows you to get feature information through the ![query_objects_on_map_button](img/query_objects_on_map_button.png){.align-middle width="30px" height="30px"} button. + It allows you to retrieve information about the features of some datasets by clicking them directly on the map. + +
+ img/querying_objects_on_map.png +
Querying Objects on map
+
+ + When clicking on map a new panel opens. That panel will show you all the information about the clicked features for each active loaded dataset. + +- You can *Zoom To Max Extent* by clicking ![zoom_to_max_extent_button](img/zoom_to_max_extent_button.png){.align-middle width="30px" height="30px"}. + +## Basemap Switcher + +By default, GeoNode allows to enrich maps with many world backgrounds. You can open available backgrounds by clicking on the map tile below: + +- *OpenStreetMap* +- *OpenTopoMap* +- *Sentinel-2-cloudless* + +
+img/basemap_switcher.png +
The Basemap Switcher Tool
+
+ +You can also decide to have an *Empty Background*. + +
+img/basemap_switching.png +
Switching the Basemap
+
+ +## Footer Tools + +At the bottom of the map, the *Footer* shows you the *Scale* of the map and allows you to change it. + +
+img/map_scaling.png +
The Map Scale
+
+ +The ![show_hide_coordinates_button](img/show_hide_coordinates_button.png){.align-middle width="30px" height="30px"} button allows you to see the pointer *Coordinates* and to change the Coordinates Reference System (CRS), WGS 84 by default. + +
+img/map_coordinates.png +
The Pointer Coordinates and the CRS
+
+ + +- [map_viewers](map_viewers.md) +- [toc](toc.md) +- [attribute_table](attribute_table.md) +- [creating_widgets](creating_widgets.md) +- [timeline](timeline.md) +- [options_menu](options_menu.md) +- [get_fetureinfo](get_fetureinfo.md) + diff --git a/docs/usage/managing_maps/maps_configuration/map_viewers.md b/docs/usage/managing_maps/maps_configuration/map_viewers.md new file mode 100644 index 00000000..d4758886 --- /dev/null +++ b/docs/usage/managing_maps/maps_configuration/map_viewers.md @@ -0,0 +1,27 @@ +# Map Viewer {#map-viewers} + +GeoNode allows for the administrator of the map to configure a custom viewer by choosing the set of plugins available for the viewer + +From the `Add viewer`{.interpreted-text role="guilabel"} option under the `Edit`{.interpreted-text role="guilabel"} options of the *Menu* a page opens and the user can: + +- Select an existing viewer from the list of viewers by clicking on the `Link to viewer`{.interpreted-text role="guilabel"} +- Create a new viewer by clicking on the `Create new viewer`{.interpreted-text role="guilabel"} + +
+img/add-viewer-page.png +
Add viewer option
+
+ +Once the `Create new viewer`{.interpreted-text role="guilabel"} is selected an *Edit Plugins* page opens and, through the central vertical bar, the user can select the plugins to include in the context viewer by moving them from the **Available Plugins** list to the **Enabled Plugins** list. + +
+img/enable_plugins.png +
Enable plugins fro the viewer
+
+ +To save and enable the map viewer, the user can click `Save as`{.interpreted-text role="guilabel"} option under the `Resource`{.interpreted-text role="guilabel"} options of the *Menu*. + +The *Map Viewer* will be visible to all users who have permission to view the map and can be reused or modified by the user who has edit permissions on it. +It will be available on the list of the resources on *Home Page* + +See the [MapStore Documentation](https://docs.mapstore.geosolutionsgroup.com/en/latest/user-guide/application-context/#configure-plugins) for more information. diff --git a/docs/usage/managing_maps/maps_configuration/options_menu.md b/docs/usage/managing_maps/maps_configuration/options_menu.md new file mode 100644 index 00000000..bd3279bc --- /dev/null +++ b/docs/usage/managing_maps/maps_configuration/options_menu.md @@ -0,0 +1,81 @@ +# Other Menu Tools {#options-menu-tools} + +At the top of the *Map* and on the *SideBar* of the map there are more menu items which we are are going to explain in this section. + +## Add dataset + +All the datasets available in GeoNode, both uploaded and remote, can be loaded on the map through the *Catalog*. +Click on the `Add dataset`{.interpreted-text role="guilabel"} option of the *Menu* to take a look at the catalog panel. + +
+img/datasets_catalog.png +
The Datasets Catalog
+
+ +You can navigate through datasets and look at their *Thumbnail* images, *Title*, *Description* and *Abstract*. +Click on a dataset to load it into the map, it will be also visible in the `toc`{.interpreted-text role="ref"}. + +## Saving a map + +| Once all the customizations have been carried out, you can *Save* your map by clicking on the `Save`{.interpreted-text role="guilabel"} option under the `Resources`{.interpreted-text role="guilabel"} options of the *Menu*. + +A new popup window will open. + +
+img/save_as_map.png +
Saving as Map option
+
+ +The current map title is filled by default, You can change it to the prefered naming then click on `Save`{.interpreted-text role="guilabel"}. The page will reload and your map should be visible in the `finding-data`{.interpreted-text role="ref"} list. + +## Printing a map + +| The [MapStore](https://mapstore2.geo-solutions.it/mapstore/#/) based map viewer of GeoNode allows you to print the current view with a customizable layout. + +Click the ![print_button](img/print_button.png){.align-middle width="30px" height="30px"} button from the *SideBar*, the **Printing Window** will open. + +
+img/printing_window.png +
The Printing Window
+
+ +From this window you can: + +- enter *Title* and *Description*; +- choose the *Resolution* in dpi; +- select the format +- select the coordinate +- add the scale +- add grid with label +- customize the *Layout* + - the *Sheet size* (A3, A4); + - if include the legend or not; + - if to put the legend in a separate page; + - the page *Orientation* (Landscape or Portrait); +- customize the *Legend* + - the *Label Font*; + - the *Font Size*; + - the *Font Emphasis* (bold, italic); + - if *Force Labels*; + - if use *Anti Aliasing Font*; + - the *Icon Size*; + - the *Legend Resolution* in dpi. + +To print the view click on `Print`{.interpreted-text role="guilabel"}. + +## Performing Measurements + +Click on the ![measure_button](img/measure_button.png){.align-middle width="30px" height="30px"} button from the *SideBar* to perform a measurement. +As you can see in the picture below, this tool allows you to measure *Distances*, *Areas* and the *Bearing* of lines. + +
+img/measure_tool.png +
The Measure Tool
+
+ +| To perform a measure draw on the map the geometry you are interested in, the result will be displayed on the left of the unit of measure select menu (this tool allows you to change the unit of measure also). + +
+img/measuring_areas.png +
Measuring Areas
+
diff --git a/docs/usage/managing_maps/maps_configuration/timeline.md b/docs/usage/managing_maps/maps_configuration/timeline.md new file mode 100644 index 00000000..a7abdad8 --- /dev/null +++ b/docs/usage/managing_maps/maps_configuration/timeline.md @@ -0,0 +1,91 @@ +# Timeline + +| GeoNode can manage datasets with a *time dimension*. Those vector datasets may vary their data through time so it is useful to represent that variation on the map. +| The [MapStore](https://mapstore2.geo-solutions.it/mapstore/#/) based map viewer used in Geonode makes available the **Timeline** tool which allows you to observe the datasets\' evolution over time, to inspect the dataset configuration at a specific time instant and to view different dataset configurations time by time dynamically through animations (see the [MapStore Documentation](https://docs.mapstore.geosolutionsgroup.com/en/latest/user-guide/timeline/#animations) for further details). + +::: warning +::: title +Warning +::: + +Timeline actually works only with WMTS-Multidim extension (WMS time in capabilities is not fully supported). +::: + +When loading a temporal dataset into the map, the *Timeline* opens automatically. + +
+img/collapsed_timeline.png +
The Timeline
+
+ +On the left side of the *Timeline* panel you can set the time value in which you want to observe the data. You can type it directly filling out the corresponding input fields or by using the up/down arrows. + +
+img/time_instant_controls.png +
The Time Control Buttons
+
+ +| On the other side there are the buttons responsible for managing the animations. +| In particular you can *Play* the animation by clicking ![play_button](img/play_button.png){.align-middle width="30px" height="30px"}, go back to the previous time instant through ![time_go_backward_button](img/time_go_backward_button.png){.align-middle width="30px" height="30px"}, go forward to next time step using ![time_go_forward_button](img/time_go_forward_button.png){.align-middle width="30px" height="30px"} and stop the animation by clicking ![stop_button](img/stop_button.png){.align-middle width="30px" height="30px"}. + +
+img/animation_buttons.png +
The Animation Control Buttons
+
+ +The *Timeline* panel can be expanded through the ![expand_timeline_button](img/expand_timeline_button.png){.align-middle width="30px" height="30px"} button. + +
+img/expanded_timeline.png +
The Expanded Timeline
+
+ +The expanded section of the *Timeline* panel contains the *Time Datasets List* and an *Histogram* which shows you: + +- the distribution of the data over time + +
+ img/timeline_histogram.png +
The Timeline Histogram
+
+ +- the *Time Cursor* + +
+ img/time_cursor.png +
The Time Cursor
+
+ +You can show/hide the datasets list by clicking ![show_hide_datasets_list_button](img/show_hide_datasets_list_button.png){.align-middle width="30px" height="30px"} (it is active by default). + +Through the *Time Range* function you can observe the data in a finite temporal interval. +Click on ![time_range_button](img/time_range_button.png){.align-middle width="30px" height="30px"} and set the initial and the final times to use it. + +
+img/time_range.png +
The Time Range Settings
+
+ +## Animations + +| The *Timeline* allows you to see the data configurations (one for each time in which the data are defined) through ordered sequences of steps. +| As said before, you can play the resulting *Animation* by clicking the play button ![play_button](img/play_button.png){.align-middle width="30px" height="30px"}. The dataset data displayed on map will change accordingly to the time reach by the cursor on the *Histogram*. + +By clicking on ![animation_settings_button](img/animation_settings_button.png){.align-middle width="30px" height="30px"} you can manage some *Animation Settings*. + +
+img/timeline_settings.png +
The Timeline Settings
+
+ +| You can activate the *Snap to guide dataset* so that the time cursor will snap to the selected dataset\'s data. You can also set up the *Frame Duration* (by default 5 seconds). +| If the *Snap to guide dataset* option is disabled, you can force the animation step to be a fixed value. + +The *Animation Range* option lets you to define a temporal range within which the time cursor can move. + +
+img/timeline_animation.png +
The Timeline Animation Range
+
+ +See the [MapStore Documentation](https://docs.mapstore.geosolutionsgroup.com/en/latest/user-guide/timeline/) for more information. diff --git a/docs/usage/managing_maps/maps_configuration/toc.md b/docs/usage/managing_maps/maps_configuration/toc.md new file mode 100644 index 00000000..5fa3b5e1 --- /dev/null +++ b/docs/usage/managing_maps/maps_configuration/toc.md @@ -0,0 +1,108 @@ +# Table of Contents (TOC) {#toc} + +In the upper left corner, click on ![toc_button](img/toc_button.png){.align-middle width="30px" height="30px"} to open the *Table Of Contents*, briefly *TOC* from now on, of the map. +The *TOC* shows all the datasets involved with the *Map* and allows to manage their properties and representations on the map. + +
+img/toc_panel.png +
The Table Of Contents (TOC)
+
+ +From the *TOC* you can: + +- manage the *TOC Settings* (click on ![settings_toc_button](img/settings_toc_button.png){.align-middle width="30px" height="30px"}) (See the [MapStore Documentation](https://docs.mapstore.geosolutionsgroup.com/en/latest/user-guide/toc/#toc-settings-and-toolbar/) for more information.) +- manage the datasets *Overlap*; +- filter the datasets list by typing text in the *Filter Datasets* field; +- manage the datasets properties such as *Opacity* (scroll the opacity cursor), *Visibility* (click on ![hide_button](img/hide_button.png){.align-middle width="30px" height="30px"} to make the dataset not visible, click on ![show_button](img/show_button.png){.align-middle width="30px" height="30px"} to show it on map); +- add and manage *Annotations* (click on ![annotation_button](img/annotation_button.png){.align-middle width="30px" height="30px"}) +- manage the *Dataset Settings*, see the next paragraph. + +
+img/scrolling_opacity.png +
Scrolling the Dataset Opacity
+
+ +Select a *Dataset* from the list and click on it, the *Dataset Toolbar* should appear in the *TOC*. + +
+img/dataset_toolbar.png +
The Dataset Toolbar
+
+ +The *Toolbar* shows you many buttons: + +- the **Zoom to dataset extent** button allows you to zoom to the dataset extent; +- the **Filter layer** button that acts directly on a layer with WFS available and filter its content; +- the **Attribute Table** button to explore the features of the dataset and their attributes (more information at `attributes-table`{.interpreted-text role="ref"}); +- the **Delete** button to delete datasets (click on `Delete Dataset`{.interpreted-text role="guilabel"} to confirm your choice); +- the **Widgets** button to create Widgets (see `creating-widgets`{.interpreted-text role="ref"}). +- the **Export data** button; +- the **Settings** button drives you through the dataset settings customization (see the next paragraph); +- the **Compare tool** button to *Swipe* or *Spy* the selected layer +- the **Edit Style** button; + +Managing Dataset Settings +\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-\-- + +The *Dataset Settings* panel looks like the one below. + +
+img/dataset_settings_panel.png +
The Dataset Settings Panel
+
+ +The *Dataset Settings* are divided in three groups: + +1. *General* settings +2. *Visibility* settings +3. *Style* settings +4. *Tiling* settings + +In the **General** tab of the *Settings Panel* you can customize the dataset *Title*, insert a *Description*, change/add the *Dataset Group* and change the *Tooltip content* and the *Tooltip placement*. + +The **Visibility** tab where you can Change the *Opacity* of the layer and add the *Visibility limits* to display the layer only within certain scale limits + +
+img/visibility_settings.png +
The Visibility tab on Settings Panel
+
+ +The **Style** tab allows you to select the style from the available layer styles and change the *Width* and the *Height* of the *Legend*. + +
+img/style_settings.png +
The Style tab on Settings Panel
+
+ +Click on the **Tiling** tab to change the output *Format* of the WMS requests, the *Tile Size* and enable/disable the *Trasparent*, the *Use cache options* and the *Single Tile*. + +
+img/tiling_settings.png +
The Tiling tab on Settings Panel
+
+ +Add an Annotation +\-\-\-\-\-\-\-\-\-\-\--Click on the ![annotation_button](img/annotation_button.png){.align-middle width="30px" height="30px"} button from the *TOC Toolbar* to enrich the map with special features which expose additional information, mark particular position on the map and so on. +From here the editor can insert a *Title* and a *Description*. + +
+img/add_annoations.png +
Annotations panel
+
+ +To begin, from the annotation panel, the editor add new annotation by selecting the `Geometries`{.interpreted-text role="guilabel"} tab. + +
+img/add_an_annotations.png +
Add an Annotations
+
+ +Here the user can choose between five different types of *Geometries*: + +1. *Marker* +2. *Line* +3. *Polygon* +4. *Text* +5. *Circle* + +See the [MapStore Documentation](https://docs.mapstore.geosolutionsgroup.com/en/latest/user-guide/annotations/) for more information. diff --git a/usage/other_apps/index.rst b/docs/usage/other_apps/index.md similarity index 85% rename from usage/other_apps/index.rst rename to docs/usage/other_apps/index.md index 30e6e764..e6e76811 100644 --- a/usage/other_apps/index.rst +++ b/docs/usage/other_apps/index.md @@ -1,9 +1,11 @@ -Using GeoNode with Other Applications -===================================== +# Using GeoNode with Other Applications Your GeoNode project is based on core components which are interoperable and as such, it is straightforward for you to integrate with external applications and services. This section will walk you through how to connect to your GeoNode instance from other applications and how to integrate other services into your GeoNode project. When complete, you should have a good idea about the possibilities for integration, and have basic knowledge about how to accomplish it. You may find it necessary to dive deeper into how to do more complex integration in order to accomplish your goals, but you should feel comfortable with the basics, and feel confident reaching out to the wider GeoNode community for help. -.. toctree:: - :maxdepth: 1 +::: {.toctree maxdepth="1"} +qgis/index +::: + + +- [index](qgis/index.md) - qgis/index diff --git a/usage/other_apps/qgis/img/geonode_connection_details.PNG b/docs/usage/other_apps/qgis/img/geonode_connection_details.PNG similarity index 100% rename from usage/other_apps/qgis/img/geonode_connection_details.PNG rename to docs/usage/other_apps/qgis/img/geonode_connection_details.PNG diff --git a/usage/other_apps/qgis/img/geonode_datamanager_dialog.PNG b/docs/usage/other_apps/qgis/img/geonode_datamanager_dialog.PNG similarity index 100% rename from usage/other_apps/qgis/img/geonode_datamanager_dialog.PNG rename to docs/usage/other_apps/qgis/img/geonode_datamanager_dialog.PNG diff --git a/usage/other_apps/qgis/img/geonode_example_layer.PNG b/docs/usage/other_apps/qgis/img/geonode_example_layer.PNG similarity index 100% rename from usage/other_apps/qgis/img/geonode_example_layer.PNG rename to docs/usage/other_apps/qgis/img/geonode_example_layer.PNG diff --git a/usage/other_apps/qgis/img/geonode_load_layers.PNG b/docs/usage/other_apps/qgis/img/geonode_load_layers.PNG similarity index 100% rename from usage/other_apps/qgis/img/geonode_load_layers.PNG rename to docs/usage/other_apps/qgis/img/geonode_load_layers.PNG diff --git a/usage/other_apps/qgis/img/geonode_oauth2_001.png b/docs/usage/other_apps/qgis/img/geonode_oauth2_001.png similarity index 100% rename from usage/other_apps/qgis/img/geonode_oauth2_001.png rename to docs/usage/other_apps/qgis/img/geonode_oauth2_001.png diff --git a/usage/other_apps/qgis/img/geonode_oauth2_002.png b/docs/usage/other_apps/qgis/img/geonode_oauth2_002.png similarity index 100% rename from usage/other_apps/qgis/img/geonode_oauth2_002.png rename to docs/usage/other_apps/qgis/img/geonode_oauth2_002.png diff --git a/usage/other_apps/qgis/img/geonode_oauth2_003.png b/docs/usage/other_apps/qgis/img/geonode_oauth2_003.png similarity index 100% rename from usage/other_apps/qgis/img/geonode_oauth2_003.png rename to docs/usage/other_apps/qgis/img/geonode_oauth2_003.png diff --git a/usage/other_apps/qgis/img/geonode_oauth2_004.png b/docs/usage/other_apps/qgis/img/geonode_oauth2_004.png similarity index 100% rename from usage/other_apps/qgis/img/geonode_oauth2_004.png rename to docs/usage/other_apps/qgis/img/geonode_oauth2_004.png diff --git a/usage/other_apps/qgis/img/geonode_oauth2_005.png b/docs/usage/other_apps/qgis/img/geonode_oauth2_005.png similarity index 100% rename from usage/other_apps/qgis/img/geonode_oauth2_005.png rename to docs/usage/other_apps/qgis/img/geonode_oauth2_005.png diff --git a/usage/other_apps/qgis/img/geonode_oauth2_006.png b/docs/usage/other_apps/qgis/img/geonode_oauth2_006.png similarity index 100% rename from usage/other_apps/qgis/img/geonode_oauth2_006.png rename to docs/usage/other_apps/qgis/img/geonode_oauth2_006.png diff --git a/usage/other_apps/qgis/img/geonode_oauth2_007.png b/docs/usage/other_apps/qgis/img/geonode_oauth2_007.png similarity index 100% rename from usage/other_apps/qgis/img/geonode_oauth2_007.png rename to docs/usage/other_apps/qgis/img/geonode_oauth2_007.png diff --git a/usage/other_apps/qgis/img/geonode_oauth2_008.png b/docs/usage/other_apps/qgis/img/geonode_oauth2_008.png similarity index 100% rename from usage/other_apps/qgis/img/geonode_oauth2_008.png rename to docs/usage/other_apps/qgis/img/geonode_oauth2_008.png diff --git a/usage/other_apps/qgis/img/geonode_oauth2_009.png b/docs/usage/other_apps/qgis/img/geonode_oauth2_009.png similarity index 100% rename from usage/other_apps/qgis/img/geonode_oauth2_009.png rename to docs/usage/other_apps/qgis/img/geonode_oauth2_009.png diff --git a/usage/other_apps/qgis/img/geonode_oauth2_010.png b/docs/usage/other_apps/qgis/img/geonode_oauth2_010.png similarity index 100% rename from usage/other_apps/qgis/img/geonode_oauth2_010.png rename to docs/usage/other_apps/qgis/img/geonode_oauth2_010.png diff --git a/usage/other_apps/qgis/img/geonode_oauth2_011.png b/docs/usage/other_apps/qgis/img/geonode_oauth2_011.png similarity index 100% rename from usage/other_apps/qgis/img/geonode_oauth2_011.png rename to docs/usage/other_apps/qgis/img/geonode_oauth2_011.png diff --git a/usage/other_apps/qgis/img/geonode_oauth2_012.png b/docs/usage/other_apps/qgis/img/geonode_oauth2_012.png similarity index 100% rename from usage/other_apps/qgis/img/geonode_oauth2_012.png rename to docs/usage/other_apps/qgis/img/geonode_oauth2_012.png diff --git a/usage/other_apps/qgis/img/geonode_qgis_desktop.PNG b/docs/usage/other_apps/qgis/img/geonode_qgis_desktop.PNG similarity index 100% rename from usage/other_apps/qgis/img/geonode_qgis_desktop.PNG rename to docs/usage/other_apps/qgis/img/geonode_qgis_desktop.PNG diff --git a/usage/other_apps/qgis/img/geonode_success_connection.PNG b/docs/usage/other_apps/qgis/img/geonode_success_connection.PNG similarity index 100% rename from usage/other_apps/qgis/img/geonode_success_connection.PNG rename to docs/usage/other_apps/qgis/img/geonode_success_connection.PNG diff --git a/docs/usage/other_apps/qgis/index.md b/docs/usage/other_apps/qgis/index.md new file mode 100644 index 00000000..ed60fbe0 --- /dev/null +++ b/docs/usage/other_apps/qgis/index.md @@ -0,0 +1,152 @@ +# QGIS Desktop {#qgis} + +QGIS is a professional GIS application that is built on top of and proud to be itself Free and Open Source Software (FOSS). QGIS is a volunteer driven project if you are interested you can find more information at . + +
+img/geonode_qgis_desktop.PNG +
QGIS Desktop Main Window
+
+ +## How can I connect to Geonode? + +Open QGIS Desktop and go to **Layer Menu \> Data Source Manager**. At the bottom of Data Source Manager, you can see a tab +with the name and an icon related to Geonode. This is because Geonode is recognized as a data source inside QGIS. + +
+img/geonode_datamanager_dialog.PNG +
Data Source Manager Dialog
+
+ +::: note +::: title +Note +::: + +It\'s possible as well load Geonode instances from an existence file this is useful to share between users or to backup existence connections. +::: + +To add a new Geonode instance, in the Geonode tab selected click on **New** and you will see the following dialog: + +
+img/geonode_connection_details.PNG +
Details of Geonode instance Dialog
+
+ +In the dialog Fill the name as you like and in the URL put the link of the Geonode instance. It\'s possible edit some WFS and WMS options to optimize the connection. +If everything is ok you will receive the following successful connection dialog: + +
+img/geonode_success_connection.PNG +
Successful connection Dialog
+
+ +After the successful dialog it\'s now possible to load all layers of the Geonode instance clicking on **Connect** button. You can see both WMS and WFS connections of the Geonode and you can load to QGIS Desktop. + +
+img/geonode_load_layers.PNG +
Geonode instance layers Dialog
+
+ +After select a layer (WMS or WFS) click on the **Add** button and the layer will be displayed in the main window of QGIS. + +
+img/geonode_example_layer.PNG +
Example of Geonode layer
+
+ +::: warning +::: title +Warning +::: + +This procedure only work with public layers. If the layers are for private use is necessary to do +the standard qgis add remote WMS/WFS layers (through **Data Source Manager**) along with basic auth method and specific endpoints. +::: + +## Connect to Private layers by using OAuth2 + +### GeoNode OAuth2 Client App Setup + +Login to GeoNode as a superuser + +![](img/geonode_oauth2_001.png){.align-center} + +Browse to `http:///o/applications/`{.interpreted-text role="guilabel"} + +![](img/geonode_oauth2_002.png){.align-center} + +Create a new specific app or, better, edit the existing one ("GeoServer") based on [OAuth2 Authorization Code Grant Type](https://oauth.net/2/grant-types/authorization-code/#:~:text=The%20Authorization%20Code%20grant%20type,to%20request%20an%20access%20token.) + +Click on "Edit" and add the Redirect URI `http://127.0.0.1:7070/qgis-client`{.interpreted-text role="guilabel"} as shown below + +::: note +::: title +Note +::: + +This is an example. The port and path of the URI can be customized. They must be the same on both GeoNode and QGis Client as shown later. +::: + +![](img/geonode_oauth2_003.png){.align-center} + +![](img/geonode_oauth2_004.png){.align-center} + +Also you will need the `Client ID`{.interpreted-text role="guilabel"} and `Client Secret`{.interpreted-text role="guilabel"} keys later when configuring QGis. + +### Configure QGis Desktop Client OAuth2 Authentication + +Open the QGis Desktop Client and add a new OWS remote Layer configuration + +![](img/geonode_oauth2_005.png){.align-center} + +Create a new service connection + +![](img/geonode_oauth2_006.png){.align-center} + +Provide the connection details + +::: note +::: title +Note +::: + +*It is* `Important`{.interpreted-text role="guilabel"} *that the URL ends with* `/gs/ows`{.interpreted-text role="guilabel"} +::: + +When finished click on "+" in order to add a new auth configuration + +![](img/geonode_oauth2_007.png){.align-center} + +Provide the needed information as shown below: + +> - Name: *any descriptive string* +> - Type: *OAuth2 authentication* +> - Grant Flow: *Authorization Code* +> - Request URL: *must end with* `/o/authorize/`{.interpreted-text role="guilabel"} +> - Token URL and Refresh URL: *must end with* `/o/token/`{.interpreted-text role="guilabel"} +> - Redirect URL: *must match with the one defined on GeoNode above* +> - Client ID and Client Secret: *must match with the one defined on GeoNode above* +> - Scopes: *openid write* +> - Enable the persistent Token Session via Headers + +![](img/geonode_oauth2_008.png){.align-center} + +Save and click on `“Connect”`{.interpreted-text role="guilabel"}. QGis will redirect you on a browser page asking to GeoNode to authenticate. Approve the Claims and go back to QGis. + +![](img/geonode_oauth2_009.png){.align-center} + +### Remove Saved Token Sessions From QGis and Login with another User + +Edit the QGis configuration + +![](img/geonode_oauth2_010.png){.align-center} + +Click on the `“pencil”`{.interpreted-text role="guilabel"} + +![](img/geonode_oauth2_011.png){.align-center} + +Clean up the saved `Tokens`{.interpreted-text role="guilabel"} and save + +![](img/geonode_oauth2_012.png){.align-center} + +Try to connect again. diff --git a/usage/publishing_data/img/publishing_resource.png b/docs/usage/publishing_data/img/publishing_resource.png similarity index 100% rename from usage/publishing_data/img/publishing_resource.png rename to docs/usage/publishing_data/img/publishing_resource.png diff --git a/usage/publishing_data/index.rst b/docs/usage/publishing_data/index.md similarity index 54% rename from usage/publishing_data/index.rst rename to docs/usage/publishing_data/index.md index 97acebd0..03444d3a 100644 --- a/usage/publishing_data/index.rst +++ b/docs/usage/publishing_data/index.md @@ -1,14 +1,13 @@ -Publishing Data -=============== +# Publishing Data In GeoNode, each resource can be published in order to share it with other people. In order to publish a map, document or dataset or any other Geonode resource, Go to the settings tab in the Metadata Edit form, The check for publishing and unpublishing is available. See picture below. -.. figure:: img/publishing_resource.png - :align: center +
+img/publishing_resource.png +
Resource publishing
+
- *Resource publishing* - -Click :guilabel: `Update` to save the changes. +Click :guilabel: [Update]{.title-ref} to save the changes. diff --git a/docs/usage/users_groups/contacting_users.md b/docs/usage/users_groups/contacting_users.md new file mode 100644 index 00000000..600c3984 --- /dev/null +++ b/docs/usage/users_groups/contacting_users.md @@ -0,0 +1,58 @@ +# Contacting other users + +| GeoNode allows you to communicate by message with other registered users and groups. +| To send a message to some user and/or groups you can follow the link `Message User`{.interpreted-text role="guilabel"} from your *Profile* page (see `editing-profile`{.interpreted-text role="ref"}) or from the *Profile* details page (see the previous section `user-info`{.interpreted-text role="ref"}) of that user. + +
+img/write_message.png +
Send message to users and groups
+
+ +| Insert your content, type a subject and click on `Send message`{.interpreted-text role="guilabel"} to send the message to the users and groups you have selected. +| You will be redirected to the *Conversation* details page related to the subject. + +
+img/sent_message.png +
Your message
+
+ +## The Inbox page + +You can view your conversations in your *Inbox* page, reachable through the `Back to inbox`{.interpreted-text role="guilabel"} button (see the picture above) or from the `Inbox`{.interpreted-text role="guilabel"} link of the user menu. + +
+img/inbox_link.png +
Inbox link
+
+ +The picture below shows how your *Inbox* page should look like. + +
+img/inbox_page.png +
Inbox page
+
+ +In *Inbox* all the unread messages are listed. You haven\'t received any message yet so your *Inbox* is empty. If you switch to the *All* tab you can see all the conversations you are involved in. + +
+img/all_your_conversations.png +
All your conversations
+
+ +When some user send a reply to your message your *Inbox* shows it, see the picture below for an example. + +
+img/reply_inbox.png +
A reply to your message
+
+ +You can open the *Conversation* details by clicking on the *Subject* link. + +
+img/conversation.png +
Conversation details
+
+ +As you can see in the picture above, in the *Conversation* page you have the ability to write a quick reply. Type your message in the text box and click on `Send Reply`{.interpreted-text role="guilabel"} to do that. + +In the *Inbox* page there is also the `Create Message`{.interpreted-text role="guilabel"} button that provides you a quick link to the message creation form. diff --git a/usage/users_groups/img/all_your_conversations.png b/docs/usage/users_groups/img/all_your_conversations.png similarity index 100% rename from usage/users_groups/img/all_your_conversations.png rename to docs/usage/users_groups/img/all_your_conversations.png diff --git a/usage/users_groups/img/conversation.png b/docs/usage/users_groups/img/conversation.png similarity index 100% rename from usage/users_groups/img/conversation.png rename to docs/usage/users_groups/img/conversation.png diff --git a/usage/users_groups/img/inbox_link.png b/docs/usage/users_groups/img/inbox_link.png similarity index 100% rename from usage/users_groups/img/inbox_link.png rename to docs/usage/users_groups/img/inbox_link.png diff --git a/usage/users_groups/img/inbox_page.png b/docs/usage/users_groups/img/inbox_page.png similarity index 100% rename from usage/users_groups/img/inbox_page.png rename to docs/usage/users_groups/img/inbox_page.png diff --git a/usage/users_groups/img/invite_users.png b/docs/usage/users_groups/img/invite_users.png similarity index 100% rename from usage/users_groups/img/invite_users.png rename to docs/usage/users_groups/img/invite_users.png diff --git a/usage/users_groups/img/invite_users_confirm.png b/docs/usage/users_groups/img/invite_users_confirm.png similarity index 100% rename from usage/users_groups/img/invite_users_confirm.png rename to docs/usage/users_groups/img/invite_users_confirm.png diff --git a/usage/users_groups/img/people_link.png b/docs/usage/users_groups/img/people_link.png similarity index 100% rename from usage/users_groups/img/people_link.png rename to docs/usage/users_groups/img/people_link.png diff --git a/usage/users_groups/img/recent_activities.png b/docs/usage/users_groups/img/recent_activities.png similarity index 100% rename from usage/users_groups/img/recent_activities.png rename to docs/usage/users_groups/img/recent_activities.png diff --git a/usage/users_groups/img/recent_activities_link.png b/docs/usage/users_groups/img/recent_activities_link.png similarity index 100% rename from usage/users_groups/img/recent_activities_link.png rename to docs/usage/users_groups/img/recent_activities_link.png diff --git a/usage/users_groups/img/registered_users_list.png b/docs/usage/users_groups/img/registered_users_list.png similarity index 100% rename from usage/users_groups/img/registered_users_list.png rename to docs/usage/users_groups/img/registered_users_list.png diff --git a/usage/users_groups/img/reply_inbox.png b/docs/usage/users_groups/img/reply_inbox.png similarity index 100% rename from usage/users_groups/img/reply_inbox.png rename to docs/usage/users_groups/img/reply_inbox.png diff --git a/usage/users_groups/img/sent_message.png b/docs/usage/users_groups/img/sent_message.png similarity index 100% rename from usage/users_groups/img/sent_message.png rename to docs/usage/users_groups/img/sent_message.png diff --git a/usage/users_groups/img/user_activities.png b/docs/usage/users_groups/img/user_activities.png similarity index 100% rename from usage/users_groups/img/user_activities.png rename to docs/usage/users_groups/img/user_activities.png diff --git a/usage/users_groups/img/user_details.png b/docs/usage/users_groups/img/user_details.png similarity index 100% rename from usage/users_groups/img/user_details.png rename to docs/usage/users_groups/img/user_details.png diff --git a/usage/users_groups/img/write_message.png b/docs/usage/users_groups/img/write_message.png similarity index 100% rename from usage/users_groups/img/write_message.png rename to docs/usage/users_groups/img/write_message.png diff --git a/docs/usage/users_groups/index.md b/docs/usage/users_groups/index.md new file mode 100644 index 00000000..eaa1011c --- /dev/null +++ b/docs/usage/users_groups/index.md @@ -0,0 +1,38 @@ +# Interacting with Users and Groups + +The GeoNode platform allows you to communicate by message with other GeoNode users and groups of users. + +| You can also invite external users to join your GeoNode. In order to do that, click on `Invite Users`{.interpreted-text role="guilabel"} in the *Profile* page (see `editing-profile`{.interpreted-text role="ref"}) or in the `About`{.interpreted-text role="guilabel"} menu in the *Home* page. +| You can invite your contacts typing their email addresses in the input field as shown in the picture below. Click on `Submit`{.interpreted-text role="guilabel"} to perform the action. + +
+img/invite_users.png +
Invite users to join GeoNode
+
+ +A message will confirm that invitations have been correctly sent. + +
+img/invite_users_confirm.png +
Invitations confirm message
+
+ +::: note +::: title +Note +::: + +You can invite more than one user at the same time by typing the email addresses inline with a semi-colon separator. +::: + +The next sections will show you how to view information about other users and how to contact them. + +::: {.toctree maxdepth="1"} +users_info +contacting_users +::: + + +- [users_info](users_info.md) +- [contacting_users](contacting_users.md) + diff --git a/docs/usage/users_groups/users_info.md b/docs/usage/users_groups/users_info.md new file mode 100644 index 00000000..50679219 --- /dev/null +++ b/docs/usage/users_groups/users_info.md @@ -0,0 +1,51 @@ +# Viewing other users information {#user-info} + +| Once your account is created, you can view other accounts on the system. +| To see information about other users on the system, click the `People`{.interpreted-text role="guilabel"} link of the `About`{.interpreted-text role="guilabel"} menu in *Home* page. + +
+img/people_link.png +
About menu - People link
+
+ +You will see a list of users registered on the system. + +
+img/registered_users_list.png +
List of the registered users
+
+ +| The *Search* tool is very useful in case of many registered users, type the name of the user you are looking for in the input text field to filter the users list. +| Select a user and click on its *username* to access to the user details page. + +
+img/user_details.png +
User details
+
+ +In this page the main information about the user are shown: personal information (organization) and the resources the user owns (datasets, maps, documents and other apps). + +Through the `User Activities`{.interpreted-text role="guilabel"} link, in right side of the page, it is possible to visualize all the activities the user has been done. + +
+img/user_activities.png +
User activities
+
+ +The `Message User`{.interpreted-text role="guilabel"} link lets you to contact other users, see the next section to read more about that. + +It is also possible, in GeoNode, to see the recent activities of all users through the `Recent Activities`{.interpreted-text role="guilabel"} link of the user menu. + +
+img/recent_activities_link.png +
Recent Activities link
+
+ +In the picture below an example. + +
+img/recent_activities.png +
Recent Activities
+
+ +As you can see, you can decide whether to see only the activities related to datasets or those related to maps or comments by switching the tabs. diff --git a/install/advanced/core/docker/connect_to_external_postgres_db.rst b/install/advanced/core/docker/connect_to_external_postgres_db.rst deleted file mode 100644 index 1fe914e6..00000000 --- a/install/advanced/core/docker/connect_to_external_postgres_db.rst +++ /dev/null @@ -1,175 +0,0 @@ -Connecting GeoNode Docker to an External Database -------------------------------------------------- - -1. Make sure docker components are down -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. code:: sh - - >_ docker-compose down - -2. Disable GeoNode docker database and related commands -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -- ``@docker-compose.yml`` comment out the ``db`` section - - .. code:: yml - - # # PostGIS database. - # db: - # # use geonode official postgis 11 image - # image: geonode/postgis:latest - # container_name: db4${COMPOSE_PROJECT_NAME} - # env_file: - # - ./scripts/docker/env/${DOCKER_ENV}/db.env - # volumes: - # - dbdata:/var/lib/postgresql/data - # - dbbackups:/pg_backups - # restart: on-failure - # # uncomment to enable remote connections to postgres - # #ports: - # # - "5432:5432" - -- ``@docker-compose.yml`` comment out the related ``volumes`` section - - .. code:: yml - - # dbdata: - # name: ${COMPOSE_PROJECT_NAME}-dbdata - # dbbackups: - # name: ${COMPOSE_PROJECT_NAME}-dbbackups - -- ``@entrypoint.sh`` comment out the related ``"waitfordbs task done"`` - section - - - Note 1: **It is expected to have the external database up running - before running, So no need to wait for the databases** - - Note 2: **Please check how the script for ``wait for databases`` - could be improved at the end on the file** - - .. code:: sh - - # /usr/local/bin/invoke waitfordbs > /usr/src/geonode/invoke.log 2>&1 - # echo "waitfordbs task done" - -- Rebuild django docker service - - .. code:: sh - - >_ docker-compose build - -3. As for the external PostgreSQL database it is expected to have the following parameters: -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -- database user with password for connection -- database name for geonode django tables -- database name for geonode data tables -- exposed database IP address -- exposed database port -- Docker containers network IP address - -4. Prepare The External PostgreSQL Database: -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - - Skip this part if it is already prepared - -- Create database user ```` with ```` - - .. code:: sh - - >_ sudo -u postgres createuser -P - -- Create databases for ``geonode django tables`` with name ``geonode`` - - .. code:: sh - - >_ sudo -u postgres createdb -O geonode - -- Create databases for ``geonode django tables`` with name - ``geonode_data`` - - .. code:: sh - - >_ sudo -u postgres createdb -O geonode_data - -- Use ``PostGIS`` extension for the created databases > Replace - ```` with your username - - .. code:: sh - - >_ sudo -u postgres psql -d geonode -c 'CREATE EXTENSION postgis;' - >_ sudo -u postgres psql -d geonode -c 'GRANT ALL ON geometry_columns TO PUBLIC;' - >_ sudo -u postgres psql -d geonode -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' - >_ sudo -u postgres psql -d geonode -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO ;' - - >_ sudo -u postgres psql -d geonode_data -c 'CREATE EXTENSION postgis;' - >_ sudo -u postgres psql -d geonode_data -c 'GRANT ALL ON geometry_columns TO PUBLIC;' - >_ sudo -u postgres psql -d geonode_data -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' - >_ sudo -u postgres psql -d geonode_data -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO ;' - -- Scenario 1: PostgreSQL service with docker service on the same host - machine - - - Get the geonode docker network IP address: - - .. code:: sh - - >_ docker network inspect geonode_default | grep Subnet - # example output: - # "Subnet": "172.21.0.0/16", - -- Scenario 2: PostgreSQL is running on separate host machine: - - - Get the host machine IP address: - - .. code:: sh - - >_ ifconfig - -- Now, Having the IP address, Add two rules for both databases at - ``/etc/postgres//pg_hba.conf`` - - .. code:: conf - - # Syntax - # host DATABASE USER ADDRESS METHOD - # Ex - # host geonode_data geonode_user 172.21.0.0/16 md5 - host geonode /16 md5 - host geonode_data /16 md5 - -- Restart postgresql database service - - .. code:: sh - - >_ sudo service postgresql restart - -- Allow GeoNode docker to connect to the external database: - - - At geonode project dir, edit ``.env`` file - - Replace with your ``username``, ``password``, ``ip_address``, - ``port`` - - Replace database names if changed than ``geonode & geonode_data`` - - .. code:: conf - - POSTGRES_USER= - POSTGRES_PASSWORD= - GEONODE_DATABASE=geonode - GEONODE_DATABASE_PASSWORD=geonode - GEONODE_GEODATABASE=geonode_data - GEONODE_GEODATABASE_PASSWORD=geonode_data - DATABASE_URL=postgis://:@:/geonode - GEODATABASE_URL=postgis://:@:/geonode_data - - - Start the docker containers - - Expected to see the migrations running again - - .. code:: sh - - >_ docker-compose up -d - - - **Done!** - - diff --git a/install/advanced/core/index.rst b/install/advanced/core/index.rst deleted file mode 100644 index 36dbabd2..00000000 --- a/install/advanced/core/index.rst +++ /dev/null @@ -1,1499 +0,0 @@ -.. _geonode-vanilla: - -=============== -GeoNode Vanilla -=============== - -Overview -======== - -The following steps will guide you to a fresh setup of GeoNode. - -All guides will first install and configure the system to run it in ``DEBUG`` mode (also known as ``DEVELOPMENT`` mode) -and then by configuring an HTTPD server to serve GeoNode through the standard ``HTTP`` (``80``) port. - -.. warning:: Those guides **are not** meant to be used on a production system. - There will be dedicated chapters that will show you some *hints* to optimize GeoNode for a production-ready machine. - In any case, we strongly suggest to task an experienced *DevOp* or *System Administrator* before exposing your server to the ``WEB``. - -Ubuntu 22.04 LTS -================= - -This part of the documentation describes the complete setup process for GeoNode on an Ubuntu 22.04.1LTS **64-bit** clean environment (Desktop or Server). - -All examples use shell commands that you must enter on a local terminal or a remote shell. - -- If you have a graphical desktop environment you can open the terminal application after login; -- if you are working on a remote server the provider or sysadmin should has given you access through an ssh client. - -.. _install_dep: - -1. Install the dependencies -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In this section, we are going to install all the basic packages and tools needed for a complete GeoNode installation. - -.. warning:: To follow this guide, a basic knowledge about Ubuntu Server configuration and working with a shell is required. - -.. note:: This guide uses ``vim`` as the editor; fill free to use ``nano``, ``gedit`` or others. - -Upgrade system packages -....................... - -Check that your system is already up-to-date with the repository running the following commands: - -.. code-block:: shell - - sudo apt update -y - - -Packages Installation -..................... - -.. note:: You don't need to install the **system packages** if you want to run the project using Docker - -We will use **example.org** as fictitious Domain Name. - -First, we are going to install all the **system packages** needed for the GeoNode setup. Login to the target machine and execute the following commands: - -.. code-block:: shell - - # Install packages from GeoNode core - sudo apt install -y --allow-downgrades build-essential \ - python3-gdal=3.4.1+dfsg-1build4 gdal-bin=3.4.1+dfsg-1build4 libgdal-dev=3.4.1+dfsg-1build4 \ - python3-all-dev python3.10-dev python3.10-venv virtualenvwrapper \ - libxml2 libxml2-dev gettext libmemcached-dev zlib1g-dev \ - libxslt1-dev libjpeg-dev libpng-dev libpq-dev \ - software-properties-common build-essential \ - git unzip gcc zlib1g-dev libgeos-dev libproj-dev \ - sqlite3 spatialite-bin libsqlite3-mod-spatialite libsqlite3-dev - - # Install Openjdk - sudo apt install openjdk-11-jdk-headless default-jdk-headless -y - - # Verify GDAL version - gdalinfo --version - $> GDAL 3.4.1, released 2021/12/27 - - # Verify Python version - python3.10 --version - $> Python 3.10.4 - - which python3.10 - $> /usr/bin/python3.10 - - # Verify Java version - java -version - $> openjdk version "11.0.16" - - # Install VIM - sudo apt install -y vim - - # Cleanup the packages - sudo apt update -y; sudo apt autoremove --purge - -.. warning:: GeoNode 4.4.x is not compatible with Python < 3.7 - -.. _install_venv: - -2. GeoNode Installation -^^^^^^^^^^^^^^^^^^^^^^^ - -This is the most basic installation of GeoNode. It won't use any external server like ``Apache Tomcat``, ``PostgreSQL`` or ``HTTPD``. - -First of all we need to prepare a new Python Virtual Environment - -Since geonode needs a large number of different python libraries and packages, its recommended to use a python virtual environment to avoid conflicts on dependencies with system wide python packages and other installed software. See also documentation of `Virtualenvwrapper `_ package for more information - -.. note:: The GeoNode Virtual Environment must be created only the first time. You won't need to create it again everytime. - -.. code-block:: shell - - which python3.10 # copy the path of python executable - - # Create the GeoNode Virtual Environment (first time only) - export WORKON_HOME=~/.virtualenvs - source /usr/share/virtualenvwrapper/virtualenvwrapper.sh - mkvirtualenv --python=/usr/bin/python3.10 geonode # Use the python path from above - - # Alterantively you can also create the virtual env like below - mkdir -p ~/.virtualenvs - python3.10 -m venv ~/.virtualenvs/geonode - source ~/.virtualenvs/geonode/bin/activate - - -At this point your command prompt shows a ``(geonode)`` prefix, this indicates that your virtualenv is active. - -.. note:: The next time you need to access the Virtual Environment just run - - .. code-block:: shell - - source /usr/share/virtualenvwrapper/virtualenvwrapper.sh - workon geonode - - # Alterantively you can also create the virtual env like below - source ~/.virtualenvs/geonode/bin/activate - -.. note:: In order to save permanently the virtualenvwrapper environment - - .. code-block:: shell - - vim ~/.bashrc - - # Write to the bottom of the file the following lines - export WORKON_HOME=~/.virtualenvs - source /usr/share/virtualenvwrapper/virtualenvwrapper.sh - -.. code-block:: shell - - # Let's create the GeoNode core base folder and clone it - sudo mkdir -p /opt/geonode/; sudo usermod -a -G www-data $USER; sudo chown -Rf $USER:www-data /opt/geonode/; sudo chmod -Rf 775 /opt/geonode/ - - # Clone the GeoNode source code on /opt/geonode - cd /opt; git clone https://github.com/GeoNode/geonode.git -b 4.4.x geonode - -.. code-block:: shell - - # Install the Python packages - cd /opt/geonode - pip install -r requirements.txt --upgrade - pip install -e . --upgrade - pip install pygdal=="`gdal-config --version`.*" - -Edit ``/opt/geonode/celery-cmd``. - -.. code-block:: shell - - CELERY__STATE_DB=${CELERY__STATE_DB:-"/opt/geonode/worker@%h.state"} - -Edit ``/opt/geonode/geonode/settings.py``. - -.. code-block:: python - - FILE_UPLOAD_DIRECTORY_PERMISSIONS = 0o777 - FILE_UPLOAD_PERMISSIONS = 0o777 - -Edit ``/opt/geonode/uwsgi.ini``. - -.. code-block:: ini - - chdir = /opt/geonode/ - - touch-reload = /opt/geonode/geonode/wsgi.py - -3. Postgis database Setup -^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. warning:: - -Be sure you have successfully completed all the steps of the section :ref:`install_dep`. - -In this section, we are going to setup users and databases for GeoNode in PostgreSQL. - -Install and Configure the PostgreSQL Database System -.................................................... - -In this section we are going to install the ``PostgreSQL`` packages along with the ``PostGIS`` extension. Those steps must be done **only** if you don't have the DB already installed on your system. - -.. code-block:: shell - - # Ubuntu 22.04.1 (focal) - sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" >> /etc/apt/sources.list.d/pgdg.list' - sudo wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - - sudo apt update -y; sudo apt install -y postgresql-13 postgresql-13-postgis-3 postgresql-13-postgis-3-scripts postgresql-13 postgresql-client-13 - -We now must create two databases, ``geonode`` and ``geonode_data``, belonging to the role ``geonode``. - -.. warning:: This is our default configuration. - You can use any database or role you need. - The connection parameters must be correctly configured on ``settings``, as we will see later in this section. - -Databases and Permissions -......................... - -First, create the geonode user. GeoNode is going to use this user to access the database - -.. code-block:: shell - - sudo service postgresql start - sudo -u postgres createuser -P geonode - - # Use the password: geonode - -You will be prompted asked to set a password for the user. **Enter geonode as password**. - -.. warning:: This is a sample password used for the sake of simplicity. This password is very **weak** and should be changed in a production environment. - -Create database ``geonode`` and ``geonode_data`` with owner ``geonode`` - -.. code-block:: shell - - sudo -u postgres createdb -O geonode geonode - sudo -u postgres createdb -O geonode geonode_data - -Next let's create PostGIS extensions - -.. code-block:: shell - - sudo -u postgres psql -d geonode -c 'CREATE EXTENSION postgis;' - sudo -u postgres psql -d geonode -c 'GRANT ALL ON geometry_columns TO PUBLIC;' - sudo -u postgres psql -d geonode -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' - sudo -u postgres psql -d geonode -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO geonode;' - sudo -u postgres psql -d geonode -c 'GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO geonode;' - - sudo -u postgres psql -d geonode_data -c 'CREATE EXTENSION postgis;' - sudo -u postgres psql -d geonode_data -c 'GRANT ALL ON geometry_columns TO PUBLIC;' - sudo -u postgres psql -d geonode_data -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' - sudo -u postgres psql -d geonode_data -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO geonode;' - sudo -u postgres psql -d geonode_data -c 'GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO geonode;' - -Final step is to change user access policies for local connections in the file ``pg_hba.conf`` - -.. code-block:: shell - - sudo vim /etc/postgresql/13/main/pg_hba.conf - -Scroll down to the bottom of the document. We want to make local connection ``trusted`` for the default user. - -Make sure your configuration looks like the one below. - -.. code-block:: shell - - ... - # DO NOT DISABLE! - # If you change this first entry you will need to make sure that the - # database superuser can access the database using some other method. - # Noninteractive access to all databases is required during automatic - # maintenance (custom daily cronjobs, replication, and similar tasks). - # - # Database administrative login by Unix domain socket - local all postgres trust - - # TYPE DATABASE USER ADDRESS METHOD - - # "local" is for Unix domain socket connections only - local all all md5 - # IPv4 local connections: - host all all 127.0.0.1/32 md5 - # IPv6 local connections: - host all all ::1/128 md5 - # Allow replication connections from localhost, by a user with the - # replication privilege. - local replication all peer - host replication all 127.0.0.1/32 md5 - host replication all ::1/128 md5 - -.. warning:: If your ``PostgreSQL`` database resides on a **separate/remote machine**, you'll have to **allow** remote access to the databases in the ``/etc/postgresql/13/main/pg_hba.conf`` to the ``geonode`` user and tell PostgreSQL to **accept** non-local connections in your ``/etc/postgresql/13/main/postgresql.conf`` file - -Restart PostgreSQL to make the change effective. - -.. code-block:: shell - - sudo service postgresql restart - -PostgreSQL is now ready. To test the configuration, try to connect to the ``geonode`` database as ``geonode`` role. - -.. code-block:: shell - - psql -U postgres geonode - # This should not ask for any password - - psql -U geonode geonode - # This should ask for the password geonode - - # Repeat the test with geonode_data DB - psql -U postgres geonode_data - psql -U geonode geonode_data - - -4. Install GeoServer -^^^^^^^^^^^^^^^^^^^^ - -In this section, we are going to install the ``Apache Tomcat 8`` Servlet Java container, which will be started by default on the internal port ``8080``. - -We will also perform several optimizations to: - -1. Correctly setup the Java VM Options, like the available heap memory and the garbage collector options. -2. Externalize the ``GeoServer`` and ``GeoWebcache`` catalogs in order to allow further updates without the risk of deleting our datasets. - -.. note:: This is still a basic setup of those components. More details will be provided on sections of the documentation concerning the hardening of the system in a production environment. Nevertheless, you will need to tweak a bit those settings accordingly with your current system. As an instance, if your machine does not have enough memory, you will need to lower down the initial amount of available heap memory. **Warnings** and **notes** will be placed below the statements that will require your attention. - -Install Apache Tomcat -............................ - -The reference version of Tomcat for the Geoserver for GeoNode is **Tomcat 9**. - - -The following steps have been adapted from https://yallalabs.com/linux/ubuntu/how-to-install-apache-tomcat-9-ubuntu-20-04/ - -.. warning:: Apache Tomcat 9 and Geoserver require Java 11 or newer to be installed on the server. - Check the steps before in order to be sure you have OpenJDK 11 correctly installed on your system. - -First, it is not recommended to run Apache Tomcat as user root, so we will create a new system user which will run the Apache Tomcat server - -.. code-block:: shell - - sudo useradd -m -U -d /opt/tomcat -s /bin/bash tomcat - sudo usermod -a -G www-data tomcat - -.. warning:: Now, go to the official Apache Tomcat `website `_ and download the most recent version of the software to your server. But don't use Tomcat10 because there are still some errors between Geoserver and Tomcat. - -.. code-block:: shell - - VERSION=9.0.65; wget https://archive.apache.org/dist/tomcat/tomcat-9/v${VERSION}/bin/apache-tomcat-${VERSION}.tar.gz - - -Once the download is complete, extract the tar file to the /opt/tomcat directory: - -.. code-block:: shell - - sudo mkdir /opt/tomcat - sudo tar -xf apache-tomcat-${VERSION}.tar.gz -C /opt/tomcat/; rm apache-tomcat-${VERSION}.tar.gz - -Apache Tomcat is updated regulary. So, to have more control over versions and updates, we’ll create a symbolic link as below: - -.. code-block:: shell - - sudo ln -s /opt/tomcat/apache-tomcat-${VERSION} /opt/tomcat/latest - -Now, let’s change the ownership of all Apache Tomcat files as below: - -.. code-block:: shell - - sudo chown -R tomcat:www-data /opt/tomcat/ - -Make the shell scripts inside the bin directory executable: - -.. code-block:: shell - - sudo sh -c 'chmod +x /opt/tomcat/latest/bin/*.sh' - -Create the a systemd file with the following content: - -.. code-block:: shell - - # Check the correct JAVA_HOME location - JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::") - echo $JAVA_HOME - $> /usr/lib/jvm/java-11-openjdk-amd64/ - - # Let's create a symbolic link to the JDK - sudo ln -s /usr/lib/jvm/java-1.11.0-openjdk-amd64 /usr/lib/jvm/jre - - # Let's create the tomcat service - sudo vim /etc/systemd/system/tomcat9.service - -.. code-block:: bash - - [Unit] - Description=Tomcat 9 servlet container - After=network.target - - [Service] - Type=forking - - User=tomcat - Group=tomcat - - Environment="JAVA_HOME=/usr/lib/jvm/jre" - Environment="JAVA_OPTS=-Djava.security.egd=file:///dev/urandom -Djava.awt.headless=true" - - Environment="CATALINA_BASE=/opt/tomcat/latest" - Environment="CATALINA_HOME=/opt/tomcat/latest" - Environment="CATALINA_PID=/opt/tomcat/latest/temp/tomcat.pid" - Environment="CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC" - - ExecStart=/opt/tomcat/latest/bin/startup.sh - ExecStop=/opt/tomcat/latest/bin/shutdown.sh - - [Install] - WantedBy=multi-user.target - -Now you can start the Apache Tomcat 9 server and enable it to start on boot time using the following command: - -.. code-block:: shell - - sudo systemctl daemon-reload - sudo systemctl start tomcat9.service - sudo systemctl status tomcat9.service - sudo systemctl enable tomcat9.service - - -For verification, type the following ss command, which will show you the 8080 open port number, the default open port reserved for Apache Tomcat Server. - -.. code-block:: shell - - ss -ltn - -In a clean Ubuntu 22.04.1, the ss command may not be found and the iproute2 library should be installed first. - -.. code-block:: shell - - sudo apt install iproute2 - # Then run the ss command - ss -ltn - -In a clean Ubuntu 22.04.1, the ss command may not be found and the iproute2 library should be installed first. - -.. code-block:: shell - - sudo apt install iproute2 - # Then run the ss command - ss -ltn - -If your server is protected by a firewall and you want to access Tomcat from the outside of your local network, you need to open port 8080. - -Use the following command to open the necessary port: - -.. code-block:: shell - - sudo ufw allow 8080/tcp - -.. warning:: Generally, when running Tomcat in a production environment, you should use a load balancer or reverse proxy. - - It’s a best practice to allow access to port ``8080`` only from your internal network. - - We will use ``NGINX`` in order to provide Apache Tomcat through the standard ``HTTP`` port. - -.. note:: Alternatively you can define the Tomcat Service as follow, in case you would like to use ``systemctl`` - - .. code-block:: shell - - sudo vim /usr/lib/systemd/system/tomcat9.service - - .. code-block:: ini - - [Unit] - Description=Apache Tomcat Server - After=syslog.target network.target - - [Service] - Type=forking - User=tomcat - Group=tomcat - - Environment=JAVA_HOME=/usr/lib/jvm/jre - Environment=JAVA_OPTS=-Djava.security.egd=file:///dev/urandom - Environment=CATALINA_PID=/opt/tomcat/latest/temp/tomcat.pid - Environment=CATALINA_HOME=/opt/tomcat/latest - Environment=CATALINA_BASE=/opt/tomcat/latest - - ExecStart=/opt/tomcat/latest/bin/startup.sh - ExecStop=/opt/tomcat/latest/bin/shutdown.sh - - RestartSec=30 - Restart=always - - [Install] - WantedBy=multi-user.target - - .. code-block:: shell - - sudo systemctl daemon-reload - sudo systemctl enable tomcat9.service - sudo systemctl start tomcat9.service - -Install GeoServer on Tomcat -............................ - -Let's externalize the ``GEOSERVER_DATA_DIR`` and ``logs`` - -.. code-block:: shell - - # Create the target folders - sudo mkdir -p /opt/data - sudo chown -Rf $USER:www-data /opt/data - sudo chmod -Rf 775 /opt/data - sudo mkdir -p /opt/data/logs - sudo chown -Rf $USER:www-data /opt/data/logs - sudo chmod -Rf 775 /opt/data/logs - - # Download and extract the default GEOSERVER_DATA_DIR - GS_VERSION=2.24.2 - sudo wget "https://artifacts.geonode.org/geoserver/$GS_VERSION/geonode-geoserver-ext-web-app-data.zip" -O data-$GS_VERSION.zip - - sudo unzip data-$GS_VERSION.zip -d /opt/data/ - - sudo mv /opt/data/data/ /opt/data/geoserver_data - sudo chown -Rf tomcat:www-data /opt/data/geoserver_data - sudo chmod -Rf 775 /opt/data/geoserver_data - - sudo mkdir -p /opt/data/geoserver_logs - sudo chown -Rf tomcat:www-data /opt/data/geoserver_logs - sudo chmod -Rf 775 /opt/data/geoserver_logs - - sudo mkdir -p /opt/data/gwc_cache_dir - sudo chown -Rf tomcat:www-data /opt/data/gwc_cache_dir - sudo chmod -Rf 775 /opt/data/gwc_cache_dir - - # Download and install GeoServer - sudo wget "https://artifacts.geonode.org/geoserver/$GS_VERSION/geoserver.war" -O geoserver-$GS_VERSION.war - sudo mv geoserver-$GS_VERSION.war /opt/tomcat/latest/webapps/geoserver.war - -Let's now configure the ``JAVA_OPTS``, i.e. the parameters to run the Servlet Container, like heap memory, garbage collector and so on. - -.. code-block:: shell - - sudo sed -i -e 's/xom-\*\.jar/xom-\*\.jar,bcprov\*\.jar/g' /opt/tomcat/latest/conf/catalina.properties - - export JAVA_HOME=$(readlink -f /usr/bin/java | sed "s:bin/java::") - echo 'JAVA_HOME='$JAVA_HOME | sudo tee --append /opt/tomcat/latest/bin/setenv.sh - sudo sed -i -e "s/JAVA_OPTS=/#JAVA_OPTS=/g" /opt/tomcat/latest/bin/setenv.sh - - echo 'GEOSERVER_DATA_DIR="/opt/data/geoserver_data"' | sudo tee --append /opt/tomcat/latest/bin/setenv.sh - echo 'GEOSERVER_LOG_LOCATION="/opt/data/geoserver_logs/geoserver.log"' | sudo tee --append /opt/tomcat/latest/bin/setenv.sh - echo 'GEOWEBCACHE_CACHE_DIR="/opt/data/gwc_cache_dir"' | sudo tee --append /opt/tomcat/latest/bin/setenv.sh - echo 'GEOFENCE_DIR="$GEOSERVER_DATA_DIR/geofence"' | sudo tee --append /opt/tomcat/latest/bin/setenv.sh - echo 'TIMEZONE="UTC"' | sudo tee --append /opt/tomcat/latest/bin/setenv.sh - - echo 'JAVA_OPTS="-server -Djava.awt.headless=true -Dorg.geotools.shapefile.datetime=false -DGS-SHAPEFILE-CHARSET=UTF-8 -XX:+UseParallelGC -XX:ParallelGCThreads=4 -Dfile.encoding=UTF8 -Duser.timezone=$TIMEZONE -Xms512m -Xmx4096m -Djavax.servlet.request.encoding=UTF-8 -Djavax.servlet.response.encoding=UTF-8 -DGEOSERVER_CSRF_DISABLED=true -DPRINT_BASE_URL=http://localhost:8080/geoserver/pdf -DGEOSERVER_DATA_DIR=$GEOSERVER_DATA_DIR -Dgeofence.dir=$GEOFENCE_DIR -DGEOSERVER_LOG_LOCATION=$GEOSERVER_LOG_LOCATION -DGEOWEBCACHE_CACHE_DIR=$GEOWEBCACHE_CACHE_DIR -Dgwc.context.suffix=gwc"' | sudo tee --append /opt/tomcat/latest/bin/setenv.sh - -.. note:: After the execution of the above statements, you should be able to see the new options written at the bottom of the file ``/opt/tomcat/latest/bin/setenv.sh``. - - .. code-block:: shell - - ... - # If you run Tomcat on port numbers that are all higher than 1023, then you - # do not need authbind. It is used for binding Tomcat to lower port numbers. - # (yes/no, default: no) - #AUTHBIND=no - JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64/ - GEOSERVER_DATA_DIR="/opt/data/geoserver_data" - GEOSERVER_LOG_LOCATION="/opt/data/geoserver_logs/geoserver.log" - GEOWEBCACHE_CACHE_DIR="/opt/data/gwc_cache_dir" - GEOFENCE_DIR="$GEOSERVER_DATA_DIR/geofence" - TIMEZONE="UTC" - JAVA_OPTS="-server -Djava.awt.headless=true -Dorg.geotools.shapefile.datetime=false -DGS-SHAPEFILE-CHARSET=UTF-8 -XX:+UseParallelGC -XX:ParallelGCThreads=4 -Dfile.encoding=UTF8 -Duser.timezone=$TIMEZONE -Xms512m -Xmx4096m -Djavax.servlet.request.encoding=UTF-8 -Djavax.servlet.response.encoding=UTF-8 -DGEOSERVER_CSRF_DISABLED=true -DPRINT_BASE_URL=http://localhost:8080/geoserver/pdf -DGEOSERVER_DATA_DIR=$GEOSERVER_DATA_DIR -Dgeofence.dir=$GEOFENCE_DIR -DGEOSERVER_LOG_LOCATION=$GEOSERVER_LOG_LOCATION -DGEOWEBCACHE_CACHE_DIR=$GEOWEBCACHE_CACHE_DIR" - - Those options could be updated or changed manually at any time, accordingly to your needs. - -.. warning:: The default options we are going to add to the Servlet Container, assume you can reserve at least ``4GB`` of ``RAM`` to ``GeoServer`` (see the option ``-Xmx4096m``). You must be sure your machine has enough memory to run both ``GeoServer`` and ``GeoNode``, which in this case means at least ``4GB`` for ``GeoServer`` plus at least ``2GB`` for ``GeoNode``. A total of at least ``6GB`` of ``RAM`` available on your machine. If you don't have enough ``RAM`` available, you can lower down the values ``-Xms512m -Xmx4096m``. Consider that with less ``RAM`` available, the performances of your services will be highly impacted. - -.. code-block:: shell - - # Create the Logrotate config - sudo tee /etc/logrotate.d/geoserver < Global`` - - .. figure:: img/ubuntu-https-002.png - :align: center - - *Proxy Base URL* - - - 3. Update the ``GeoServer`` ``Role Base URL`` accordingly. - - From the ``GeoServer Admin GUI`` go to ``Security > Users, Groups, Roles > geonode REST role service`` - - .. figure:: img/ubuntu-https-003.png - :align: center - - *Role Base URL* - - 4. Update the ``GeoServer`` ``OAuth2 Service Parameters`` accordingly. - - From the ``GeoServer Admin GUI`` go to ``Security > Authentication > Authentication Filters > geonode-oauth2`` - - .. figure:: img/ubuntu-https-004.png - :align: center - - *OAuth2 Service Parameters* - - - 5. Update the ``.env`` file - - .. code-block:: shell - - vim /opt/geonode/.env - - # Change everywhere 'http' to 'https' - %s/http/https/g - - # Restart the service - sudo systemctl restart geonode-uwsgi - -8. Enabling Fully Asynchronous Tasks -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Install and configure `"rabbitmq-server" `_ -............................................................................................ - -.. seealso:: - - A `March 2021 blog post `_ from RabbitMQ provides alternative installations for other systems. - -**Install rabbitmq-server** - -Reference: `lindevs.com/install-rabbitmq-on-ubuntu/ `_ & `www.rabbitmq.com/install-debian.html/ `_ - -.. code-block:: bash - - sudo apt install curl -y - - ## Import GPG Key - sudo apt update - sudo apt install curl software-properties-common apt-transport-https lsb-release - echo 'deb [signed-by=/etc/apt/keyrings/erlang.gpg] http://binaries2.erlang-solutions.com/ubuntu/ jammy-esl-erlang-26 contrib' | sudo tee /etc/apt/sources.list.d/erlang.list - curl -fsSL https://packages.erlang-solutions.com/ubuntu/erlang_solutions.asc | sudo gpg --dearmor -o /etc/apt/keyrings/erlang.gpg - - ## Add Erlang Repository to Ubuntu - sudo apt update - sudo apt install esl-erlang - - ## Add RabbitMQ Repository to Ubuntu - curl -s https://packagecloud.io/install/repositories/rabbitmq/rabbitmq-server/script.deb.sh | sudo bash - - ## Install RabbitMQ Server - sudo apt install rabbitmq-server - - # check the status (it should already be running) - sudo systemctl status rabbitmq-server - - # check the service is enabled (it should already be enabled) - sudo systemctl is-enabled rabbitmq-server.service - - # enable the web frontend and allow access through firewall - # view this interface at http://:15672 - sudo rabbitmq-plugins enable rabbitmq_management - sudo ufw allow proto tcp from any to any port 5672,15672 - -**Create admin user** - -This is the user that GeoNode will use to communicate with rabbitmq-server. - -.. code-block:: - - sudo rabbitmqctl delete_user guest - sudo rabbitmqctl add_user admin - sudo rabbitmqctl set_user_tags admin administrator - sudo rabbitmqctl add_vhost /localhost - sudo rabbitmqctl set_permissions -p / admin ".*" ".*" ".*" - sudo rabbitmqctl set_permissions -p /localhost admin ".*" ".*" ".*" - -**Managing RabbitMQ** - -You can manage the rabbitmq-server service like any other service:: - - sudo systemctl stop rabbitmq-server - sudo systemctl start rabbitmq-server - sudo systemctl restart rabbitmq-server - -You can manage the rabbitmq-server node with `rabbitmqctl `_. -For example, to fully reset the server, use these commands:: - - sudo rabbitmqctl stop_app - sudo rabbitmqctl reset - sudo rabbitmqctl start_app - -After reset, you'll need to recreate the ``admin`` user (see above). - -Daemonize and configure Celery -............................ - -**Create the Systemd unit** - -.. code-block:: shell - - sudo vim /etc/systemd/system/celery.service - -.. code-block:: ini - - [Unit] - Description=Celery - After=network.target - - [Service] - Type=simple - # the specific user that our service will run as - EnvironmentFile=/opt/geonode/.env - User=geosolutions - Group=geosolutions - # another option for an even more restricted service is - # DynamicUser=yes - # see http://0pointer.net/blog/dynamic-users-with-systemd.html - RuntimeDirectory=celery - WorkingDirectory=/opt/geonode - ExecStart=bash -c 'source /home/geosolutions/.virtualenvs/geonode/bin/activate && /opt/geonode/celery-cmd' - ExecReload=/bin/kill -s HUP $MAINPID - Restart=always - TimeoutSec=900 - TimeoutStopSec=60 - PrivateTmp=true - - [Install] - WantedBy=multi-user.target - -.. code-block:: shell - - # Create the Logrotate config - sudo tee /etc/logrotate.d/celery <` to prepare a Ubuntu 22.04 server with Docker and Docker Compose - -1. Clone GeoNode -^^^^^^^^^^^^^^^^ - -.. code-block:: shell - - # Let's create the GeoNode core base folder and clone it - sudo mkdir -p /opt/geonode/ - sudo usermod -a -G www-data geonode - sudo chown -Rf geonode:www-data /opt/geonode/ - sudo chmod -Rf 775 /opt/geonode/ - - # Clone the GeoNode source code on /opt/geonode - cd /opt - git clone https://github.com/GeoNode/geonode.git - - -2. Prepare the .env file -^^^^^^^^^^^^^^^^^^^^^^^^^ -Follow the instructions at :ref:`Docker create env file` - -3. Build and run -^^^^^^^^^^^^^^^^^ -Follow the instructions at :ref:`Docker build and run` - - -Test the instance and follow the logs -..................................... - -If you run the containers daemonized (with the ``-d`` option), you can either run specific Docker commands to follow the ``startup and initialization logs`` or entering the image ``shell`` and check for the ``GeoNode logs``. - -In order to follow the ``startup and initialization logs``, you will need to run the following command from the repository folder - -.. code-block:: shell - - cd /opt/geonode - docker logs -f django4geonode - -Alternatively: - -.. code-block:: shell - - cd /opt/geonode - docker-compose logs -f django - -You should be able to see several initialization messages. Once the container is up and running, you will see the following statements - -.. code-block:: shell - - ... - 789 static files copied to '/mnt/volumes/statics/static'. - static data refreshed - Executing UWSGI server uwsgi --ini /usr/src/app/uwsgi.ini for Production - [uWSGI] getting INI configuration from /usr/src/app/uwsgi.ini - -To exit just hit ``CTRL+C``. - -This message means that the GeoNode containers have bee started. Browsing to ``http://localhost/`` will show the GeoNode home page. You should be able to successfully log with the default admin user (``admin`` / ``admin``) and start using it right away. - -With Docker it is also possible to run a shell in the container and follow the logs exactly the same as you deployed it on a physical host. To achieve this run - -.. code-block:: shell - - docker exec -it django4geonode /bin/bash - - # Once logged in the GeoNode image, follow the logs by executing - tail -F -n 300 /var/log/geonode.log - -Alternatively: - -.. code-block:: shell - - docker-compose exec django /bin/bash - -To exit just hit ``CTRL+C`` and ``exit`` to return to the host. - -Override the ENV variables to deploy on a public IP or domain -............................................................. - -If you would like to start the containers on a ``public IP`` or ``domain``, let's say ``www.example.org``, you can follow the instructions at :ref:`Deploy to production` - -ariables to customize the GeoNode instance. See the ``GeoNode Settings`` section in order to get a list of the available options. - - -Remove all data and bring your running GeoNode deployment to the initial stage -.............................................................................. - -This procedure allows you to stop all the containers and reset all the data with the deletion of all the volumes. - -.. code-block:: shell - - cd /opt/geonode - - # stop containers and remove volumes - docker-compose down -v - -Get rid of old Docker images and volumes (reset the environment completely) -............................................................................ - -.. note:: For more details on Docker commands, please refer to the official Docker documentation. - -It is possible to let docker show which containers are currently running (add ``-a`` for all containers, also stopped ones) - -.. code-block:: shell - - # Show the currently running containers - docker ps - - CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES - 4729b3dd1de7 geonode/geonode:4.0 "/usr/src/geonode/en…" 29 minutes ago Up 5 minutes 8000/tcp celery4geonode - 418da5579b1a geonode/geonode:4.0 "/usr/src/geonode/en…" 29 minutes ago Up 5 minutes (healthy) 8000/tcp django4geonode - d6b043f16526 geonode/letsencrypt:4.0 "./docker-entrypoint…" 29 minutes ago Up 9 seconds 80/tcp, 443/tcp letsencrypt4geonode - c77e1fa3ab2b geonode/geoserver:2.19.6 "/usr/local/tomcat/t…" 29 minutes ago Up 5 minutes (healthy) 8080/tcp geoserver4geonode - a971cedfd788 rabbitmq:3.7-alpine "docker-entrypoint.s…" 29 minutes ago Up 5 minutes 4369/tcp, 5671-5672/tcp, 25672/tcp rabbitmq4geonode - a2e4c69cb80f geonode/nginx:4.0 "/docker-entrypoint.…" 29 minutes ago Up 5 minutes 0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443- >443/tcp, :::443->443/tcp nginx4geonode - d355d34cac4b geonode/postgis:13 "docker-entrypoint.s…" 29 minutes ago Up 5 minutes 5432/tcp db4geonode - - -Stop all the containers by running - -.. code-block:: shell - - docker-compose stop - -Force kill all containers by running - -.. code-block:: shell - - docker kill $(docker ps -q) - -I you want to clean up all containers and images, without deleting the static volumes (i.e. the ``DB`` and the ``GeoServer catalog``), issue the following commands - -.. code-block:: shell - - # Remove all containers - docker rm $(docker ps -a -q) - - # Remove all docker images - docker rmi $(docker images -q) - - # Prune the old images - docker system prune -a - -If you want to remove a ``volume`` also - -.. code-block:: shell - - # List of the running volumes - docker volume ls - - # Remove the GeoServer catalog by its name - docker volume rm -f geonode-gsdatadir - - # Remove all dangling docker volumes - docker volume rm $(docker volume ls -qf dangling=true) - - # update all images, should be run regularly to fetch published updates - for i in $(docker images| awk 'NR>1{print $1":"$2}'| grep -v ''); do docker pull "$i" ;done diff --git a/install/advanced/index.rst b/install/advanced/index.rst deleted file mode 100644 index 2e7e0d0a..00000000 --- a/install/advanced/index.rst +++ /dev/null @@ -1,18 +0,0 @@ -============================= -GeoNode Advanced Installation -============================= - -.. toctree:: - :maxdepth: 3 - - core/index - -.. toctree:: - :maxdepth: 3 - - project/index - -.. toctree:: - :maxdepth: 3 - - production_enhancements/index diff --git a/install/advanced/production_enhancements/index.rst b/install/advanced/production_enhancements/index.rst deleted file mode 100644 index 6ee4000e..00000000 --- a/install/advanced/production_enhancements/index.rst +++ /dev/null @@ -1,331 +0,0 @@ -.. _geonode-production-enhancements: - -Further Production Enhancements -=============================== - -GeoServer Production Settings -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -JVM Settings: Memory And GeoServer Options -.......................................... - -The :guilabel:`.env` file provides a way to customize GeoServer JVM Options. - -The variable ``GEOSERVER_JAVA_OPTS`` allows you to tune-up the GeoServer container and to enable specific GeoServer options. - -.. code-block:: shell - - GEOSERVER_JAVA_OPTS= - -Djava.awt.headless=true -Xms4G -Xmx4G -XX:PerfDataSamplingInterval=500 - -XX:SoftRefLRUPolicyMSPerMB=36000 -XX:-UseGCOverheadLimit -XX:+UseConcMarkSweepGC - -XX:+UseParNewGC -XX:ParallelGCThreads=4 -Dfile.encoding=UTF8 -Djavax.servlet.request.encoding=UTF-8 - -Djavax.servlet.response.encoding=UTF-8 -Duser.timezone=GMT - -Dorg.geotools.shapefile.datetime=false -DGS-SHAPEFILE-CHARSET=UTF-8 -DGEOSERVER_CSRF_DISABLED=true -DPRINT_BASE_URL=http://geoserver:8080/geoserver/pdf - -``-Djava.awt.headless (true)`` - -Work with graphics-based applications in Java without an actual display, keyboard, or mouse -A typical use case of UI components running in a headless environment could be an image converter app. Though it needs graphics data for image processing, a display is not really necessary. The app could be run on a server and converted files saved or sent over the network to another machine for display. - -``-Xms4G -Xmx4G`` - -This means that your JVM will be started with Xms amount of memory and will be able to use a maximum of Xmx amount of memory. Above will start a JVM like with 2 GB of memory and will allow the process to use up to 4 GB of memory. You need to adjust this value depening on your availabnle RAM. - -``-DGEOSERVER_CSRF_DISABLED (True)`` - -The GeoServer web admin employs a CSRF (Cross-Site Request Forgery) protection filter that will block any form submissions that didn’t appear to originate from GeoServer. This can sometimes cause problems for certain proxy configurations. You can disable the CSRF filter by setting the GEOSERVER_CSRF_DISABLED property to true. -https://docs.geoserver.org/stable/en/user/security/webadmin/csrf.html - - -Whenever you need to change one or more of the JVM options, you will need to restart the GeoServer Docker container. - -.. code-block:: shell - - # Hard restart of the container: the only way to update the .env variables - docker-compose up -d geoserver - -This command will **preserve** all the GeoServer configuration and data, since the ``GEOSERVER_DATA_DIR`` is stored on a Docker static volume. - -Nevertheless, any change you have made manually to the container, e.g. added a new plugin to GeoServer or updated some JARs into the ``WEB-INF/lib`` library folder, will be lost. - -You will need to add the JARs again and restart GeoServer *softly* - -.. code-block:: shell - - # Soft restart of the container: the .env variables won't be updated - docker-compose restart geoserver - - -Global And Services Settings -............................ - - * Check the GeoServer Memory usage and status; ensure the ``GEOSERVER_DATA_DIR`` path points to the static volume - - .. figure:: img/production_geoserver_001.png - :width: 350px - :align: center - - *GeoServer Status* - - * GeoServer :guilabel:`Global Settings`; make sure the ``Proxy Base Url`` points to the publlc URL and the ``LOGGING`` levels are set to :guilabel:`Production Mode` - - .. figure:: img/production_geoserver_002.png - :width: 350px - :align: center - - *Global Settings* - - * GeoServer :guilabel:`Image Processing Settings`; unless you are using some specific renderer or GeoServer plugin, use the following recommended options - - .. note:: Further details at https://docs.geoserver.org/stable/en/user/configuration/image_processing/index.html#image-processing - - .. figure:: img/production_geoserver_003.png - :width: 350px - :align: center - - *Image Processing Settings* - - * Tune up :guilabel:`GeoServer Services Configuration`; :guilabel:`WCS`, :guilabel:`WFS`, :guilabel:`WMS` and :guilabel:`WPS`; - - - **WCS**: Update the limits accordingly to your needs. Do not use very high values, this will set GeoServer prone to DoS Attacks. - - .. figure:: img/production_geoserver_004.png - :width: 350px - :align: center - - *WCS Resource Consuption Limits* - - - **WMS**: Specify here the SRS List you are going to use. Empty means all the ones supported by GeoServer, but be carefull since the ``GetCapabilities`` output will become huge. - - .. figure:: img/production_geoserver_005.png - :width: 350px - :align: center - - *WMS Supported SRS List* - - - **WMS**: :guilabel:`Raster Rendering Options` allows you to tune up the WMS output for better performance or quality. Best Performance: ``Nearest Neighbour`` - Best Quality: ``Bicubic`` - - .. warning:: Raster Images should be always optimized before ingested into GeoNode. The general recommendation is to **never** upload a non-processed GeoTIFF image to GeoNode. - - Further details at: - - - https://geoserver.geo-solutions.it/edu/en/enterprise/raster.html - - https://geoserver.geo-solutions.it/edu/en/raster_data/advanced_gdal/index.html - - .. figure:: img/production_geoserver_006.png - :width: 350px - :align: center - - *WMS Raster Rendering Options* - - - **WMS**: Update the limits accordingly to your needs. Do not use very high values, this will set GeoServer prone to DoS Attacks. - - .. figure:: img/production_geoserver_007.png - :width: 350px - :align: center - - *WMS Resource Consuption Limits* - -GeoWebCache DiskQuota On Postgis -................................ - -By default GeoWebCache DiskQuota is disabled. That means that the layers cache might potentially grow up indefinitely. - -GeoWebCache DiskQuota should be always enabled on a production system. In the case it is enabled, this **must** be configured to make use of a DB engine like Postgis to store its indexes. - - - First of all ensure :guilabel:`Tile Caching` is enabled on all available layers - - .. note:: GeoNode tipically does this automatically for you. It is worth to double check anyway. - - .. figure:: img/production_geoserver_008.png - :width: 350px - :align: center - - *Tile Caching: Tiled Datasets* - - - Configure :guilabel:`Disk Quota` by providing the connection string to the DB Docker Container as specified in the :guilabel:`.env` file - - .. figure:: img/production_geoserver_009.png - :width: 350px - :align: center - - *Tile Caching: Disk Quota Configuration* - -GeoFence Security Rules On Postgis -.................................. - -By default GeoFence stores the security rules on an :guilabel:`H2` db. - -On a production system, this is not really recommended. You will need to update the GeoServer Docker container in order to enable GeoFence storing the rules into the DB Docker Container instead. - -In order to do that, follow the procedure below: - -.. code-block:: shell - - # Enter the GeoServer Docker Container - docker-compose exec geoserver bash - - # Install a suitable editor - apt update - apt install nano - - # Edit the GeoFence DataStore .properties file - nano /geoserver_data/data/geofence/geofence-datasource-ovr.properties - -.. note:: Make sure to provide the same connection parameters specified in the :guilabel:`.env` file - -.. code-block:: ini - - geofenceVendorAdapter.databasePlatform=org.hibernatespatial.postgis.PostgisDialect - geofenceDataSource.driverClassName=org.postgresql.Driver - geofenceDataSource.url=jdbc:postgresql://db:5432/my_geonode_data - geofenceDataSource.username=my_geonode_data - geofenceDataSource.password=******** - geofenceEntityManagerFactory.jpaPropertyMap[hibernate.default_schema]=public - -.. code-block:: shell - - # Update the GeoServer WEB-INF/lib JARs accordingly - wget --no-check-certificate "https://repo1.maven.org/maven2/org/postgis/postgis-jdbc/1.3.3/postgis-jdbc-1.3.3.jar" -O postgis-jdbc-1.3.3.jar && \ - wget --no-check-certificate "https://maven.geo-solutions.it/org/hibernatespatial/hibernate-spatial-postgis/1.1.3.2/hibernate-spatial-postgis-1.1.3.2.jar" -O hibernate-spatial-postgis-1.1.3.2.jar && \ - rm /usr/local/tomcat/webapps/geoserver/WEB-INF/lib/hibernate-spatial-h2-geodb-1.1.3.1.jar && \ - mv hibernate-spatial-postgis-1.1.3.2.jar /usr/local/tomcat/webapps/geoserver/WEB-INF/lib/ && \ - mv postgis-jdbc-1.3.3.jar /usr/local/tomcat/webapps/geoserver/WEB-INF/lib/ - -The container is ready to be restarted now. - -.. warning:: Remember to do a **soft restart** otherwise the WEB-INF/lib JARs will be reset to the original state - -.. code-block:: shell - - # Exit the GeoServer container - exit - - # Soft Restart GeoServer Docker Container - docker-compose restart geoserver - -**IMPORTANT**: The first time you perform this procedure, GeoFence won't be able to retrieve the old security rules anymore. - -You will need to :ref:`fixup_geonode_layers_permissions` in order to regenerate the security rules. - -.. _fixup_geonode_layers_permissions: - -Fixup GeoNode Datasets Permissions -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The list of the GeoFence Security Rules is available from the :guilabel:`GeoFence Data Rules` section. - -Always double check the list is accessible and the data rules are there. If empty, no layer will be accessible by standard users other than admin. - -.. figure:: img/production_geoserver_010.png - :width: 350px - :align: center - - *GeoFence Data Rules* - -In order to re-sync the GeoFence security rules, follow the procedure below: - -.. code-block:: shell - - # Enter the GeoNode Docker Container - docker-compose exec django bash - - # Run the `sync_geonode_datasets` management command - ./manage.sh sync_geonode_datasets --updatepermissions - -Regenerate GeoNode Datasets Thumbnails -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The following procedure allows you to *batch* regenerate all Datasets Thumbnails: - -.. code-block:: shell - - # Enter the GeoNode Docker Container - docker-compose exec django bash - - # Run the `sync_geonode_datasets` management command - ./manage.sh sync_geonode_datasets --updatethumbnails - -Regenerate GeoNode Datasets BBOXES -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The following procedure allows you to *batch* regenerate all Datasets BBOXES: - -.. code-block:: shell - - # Enter the GeoNode Docker Container - docker-compose exec django bash - - # Run the `sync_geonode_datasets` management command - ./manage.sh sync_geonode_datasets --updatebbox - -Fixup GeoNode Datasets Metadata And Download Links -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The following procedure allows you to fix-up broken or incorrect Metadata Links: - -.. code-block:: shell - - # Enter the GeoNode Docker Container - docker-compose exec django bash - - # Run the `set_all_datasets_metadata` management command - ./manage.sh set_all_datasets_metadata -d - -It is also possible to *force* purging the links before regenerating: - -.. code-block:: shell - - # Enter the GeoNode Docker Container - docker-compose exec django bash - - # Run the `set_all_datasets_metadata` management command - ./manage.sh set_all_datasets_metadata -d --prune - -Migrate GeoNode To A New Hostname -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In the case you will need to move your instance to another domain, as an example from ``https://my_geonode.geonode.org/`` to ``https://prod_geonode.geonode.org/``, follow the procedure below: - -- Update the :guilabel:`.env` file by specifyig the new name accordingly. - -- Restart the GeoNode Docker Container. - - .. code:: shell - - docker-compose up -d geonode - -- Run the following management commands from inside the GeoNode Docker Container. - - .. code:: shell - - # Enter the GeoNode Docker Container - docker-compose exec django bash - - # Run the `migrate_baseurl` management command - ./manage.sh migrate_baseurl --source-address=my_geonode.geonode.org --target-address=prod_geonode.geonode.org - - # Run the `set_all_datasets_metadata` management command - ./manage.sh set_all_datasets_metadata -d - - -Add Huge Or DB Datasets To Your Instance -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Uploaing huge datasets, or DB tables, to GeoNode from the :guilabel:`Web Upload Interface` is not really possible sometimes. - -The suggested procedure in such cases is the following one: - -- Add the dataset to :guilabel:`GeoServer` first directly. - - You must upload the data into the GeoServer Docker Container Static Volume first and then adding manually the layer throught the :guilabel:`GeoServer Admin GUI`. - -- Once the dataset is correctly configured on GeoServer, run the following management command from inside the GeoNode Docker Container - - .. code:: shell - - # Enter the GeoNode Docker Container - docker-compose exec django bash - - # Run the `updatelayers` management command - ./manage.sh updatelayers -w -f diff --git a/install/advanced/project/index.rst b/install/advanced/project/index.rst deleted file mode 100644 index c51b8ce5..00000000 --- a/install/advanced/project/index.rst +++ /dev/null @@ -1,523 +0,0 @@ -.. _geonode-project: - -=============== -GeoNode Project -=============== - -Overview -======== - -The following steps will guide you to a new setup of GeoNode Project. All guides will first install and configure the system to run it in ``DEBUG`` mode (also known as ``DEVELOPMENT`` mode) and then by configuring an HTTPD server to serve GeoNode through the standard ``HTTP`` (``80``) port. - -Those guides **are not** meant to be used on a production system. There will be dedicated chapters that will show you some *hints* to optimize GeoNode for a production-ready machine. In any case, we strongly suggest to task an experienced *DevOp* or *System Administrator* before exposing your server to the ``WEB``. - -Ubuntu 22.04 -============ - -This part of the documentation describes the complete setup process for GeoNode on an Ubuntu 22.04 64-bit clean environment (Desktop or Server). All examples use shell commands that you must enter on a local terminal or a remote shell. -- If you have a graphical desktop environment you can open the terminal application after login; -- if you are working on a remote server the provider or sysadmin should has given you access through an ssh client. - -.. _install_dep_proj: - -Install the dependencies -^^^^^^^^^^^^^^^^^^^^^^^^ - -In this section, we are going to install all the basic packages and tools needed for a complete GeoNode installation. To follow this guide, a piece of basic knowledge about Ubuntu Server configuration and working with a shell is required. This guide uses ``vim`` as the editor; fill free to use ``nano``, ``gedit`` or others. - -Upgrade system packages -....................... - -Check that your system is already up-to-date with the repository running the following commands: - -.. code-block:: shell - - sudo apt update - - -Create a Dedicated User -....................... - -In the following steps a User named ``geonode`` is used: to run installation commands the user must be in the ``sudo`` group. - -Create User ``geonode`` **if not present**: - -.. code-block:: shell - - # Follow the prompts to set the new user's information. - # It is fine to accept the defaults to leave all of this information blank. - sudo adduser geonode - - # The following command adds the user geonode to group sudo - sudo usermod -aG sudo geonode - - # make sure the newly created user is allowed to login by ssh - # (out of the scope of this documentation) and switch to User geonode - su geonode - -Packages Installation -..................... -Add the Ubuntu GIS packages prior to installing the other system packages. - -.. code-block:: shell - - sudo apt update - -.. note:: You don't need to install the **system packages** if you want to run the project using Docker - -First, we are going to install all the **system packages** needed for the GeoNode setup. - -.. code-block:: shell - - # Install packages from GeoNode core - sudo apt install -y python3-gdal=3.3.2+dfsg-2~focal2 gdal-bin=3.3.2+dfsg-2~focal2 libgdal-dev=3.3.2+dfsg-2~focal2 - sudo apt install -y python3-pip python3-dev python3-virtualenv python3-venv virtualenvwrapper - sudo apt install -y libxml2 libxml2-dev gettext - sudo apt install -y libxslt1-dev libjpeg-dev libpng-dev libpq-dev libmemcached-dev - sudo apt install -y software-properties-common build-essential - sudo apt install -y git unzip gcc zlib1g-dev libgeos-dev libproj-dev - sudo apt install -y sqlite3 spatialite-bin libsqlite3-mod-spatialite - - # Install Openjdk - sudo -i apt update - sudo apt install openjdk-8-jdk-headless default-jdk-headless -y - - # Remember to select the correct java version /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java - sudo update-alternatives --config java - - sudo apt update -y - sudo apt autoremove -y - sudo apt autoclean -y - sudo apt purge -y - sudo apt clean -y - - # Install Packages for Virtual environment management - sudo apt install -y virtualenv virtualenvwrapper - - # Install text editor - sudo apt install -y vim - -Geonode Project Installation -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Geonode project is the proper way to run a customized installation of Geonode. The repository of geonode-project contains a minimal set of files following the structure of a django-project. Geonode itself will be installed as a requirement of your project. -Inside the project structure is possible to extend, replace or modify all geonode components (e.g. css and other static files, templates, models..) and even register new django apps **without touching the original Geonode code**. - - -.. note:: You can call your geonode project whatever you like following the naming conventions for python packages (generally lower case with underscores (_). In the examples below, replace ``my_geonode`` with whatever you would like to name your project. - -See also the `README `_ file on geonode-project repository - -First of all we need to prepare a new Python Virtual Environment - -Prepare the environment - -.. code-block:: shell - - sudo mkdir -p /opt/geonode_custom/ - sudo usermod -a -G www-data geonode - sudo chown -Rf geonode:www-data /opt/geonode_custom/ - sudo chmod -Rf 775 /opt/geonode_custom/ - -Clone the source code - -.. code-block:: shell - - cd /opt/geonode_custom/ - git clone https://github.com/GeoNode/geonode-project.git - -This will clone the ``master`` branch. You will have to checkout the desidered branch or tag. -As an example, if you want to generate a propject for GeoNode 4.2.2 you will docker - -.. code-block:: shell - - git checkout -b 4.2.2 - -Make an instance out of the ``Django Template`` - -.. note:: We will call our instance ``my_geonode``. You can change the name at your convenience. - -.. code-block:: shell - - vim ~/.bashrc - # add the following line to the bottom - source /usr/share/virtualenvwrapper/virtualenvwrapper.sh - -.. code-block:: shell - - source /usr/share/virtualenvwrapper/virtualenvwrapper.sh - mkvirtualenv --python=/usr/bin/python3 my_geonode - - Alterantively you can also create the virtual env like below - python3.8 -m venv /home/geonode/dev/.venvs/my_geonode - source /home/geonode/dev/.venvs/my_geonode/bin/activate - - pip install Django==3.2.13 - - django-admin startproject --template=./geonode-project -e py,sh,md,rst,json,yml,ini,env,sample,properties -n monitoring-cron -n Dockerfile my_geonode - - # Install the Python packages - cd /opt/geonode_custom/my_geonode - pip install -r src/requirements.txt --upgrade - pip install -e src/ --upgrade - - # Install GDAL Utilities for Python - pip install pygdal=="`gdal-config --version`.*" - - # Dev scripts - mv .override_dev_env.sample src/.override_dev_env - mv src/manage_dev.sh.sample src/manage_dev.sh - mv src/paver_dev.sh.sample src/paver_dev.sh - -Install and Configure the PostgreSQL Database System -.................................................... - -In this section we are going to install the ``PostgreSQL`` packages along with the ``PostGIS`` extension. Those steps must be done **only** if you don't have the DB already installed on your system. - -.. code-block:: shell - - # Ubuntu 22.04 - sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main" >> /etc/apt/sources.list.d/pgdg.list' - sudo wget --no-check-certificate --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add - - sudo apt update -y; sudo apt install -y postgresql-13 postgresql-13-postgis-3 postgresql-13-postgis-3-scripts postgresql-13 postgresql-client-13 - -We now must create two databases, ``my_geonode`` and ``my_geonode_data``, belonging to the role ``my_geonode``. - -.. warning:: This is our default configuration. - You can use any database or role you need. - The connection parameters must be correctly configured on ``settings``, as we will see later in this section. - -Databases and Permissions -......................... - -First, create the geonode user. GeoNode is going to use this user to access the database - -.. code-block:: shell - - sudo service postgresql start - sudo -u postgres createuser -P my_geonode - - # Use the password: geonode - -You will be prompted asked to set a password for the user. **Enter geonode as password**. - -.. warning:: This is a sample password used for the sake of simplicity. This password is very **weak** and should be changed in a production environment. - -Create database ``my_geonode`` and ``my_geonode_data`` with owner ``my_geonode`` - -.. code-block:: shell - - sudo -u postgres createdb -O my_geonode my_geonode - sudo -u postgres createdb -O my_geonode my_geonode_data - -Next let's create PostGIS extensions - -.. code-block:: shell - - sudo -u postgres psql -d my_geonode -c 'CREATE EXTENSION postgis;' - sudo -u postgres psql -d my_geonode -c 'GRANT ALL ON geometry_columns TO PUBLIC;' - sudo -u postgres psql -d my_geonode -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' - sudo -u postgres psql -d my_geonode -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO my_geonode;' - - sudo -u postgres psql -d my_geonode_data -c 'CREATE EXTENSION postgis;' - sudo -u postgres psql -d my_geonode_data -c 'GRANT ALL ON geometry_columns TO PUBLIC;' - sudo -u postgres psql -d my_geonode_data -c 'GRANT ALL ON spatial_ref_sys TO PUBLIC;' - sudo -u postgres psql -d my_geonode_data -c 'GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO my_geonode;' - -Final step is to change user access policies for local connections in the file ``pg_hba.conf`` - -.. code-block:: shell - - sudo vim /etc/postgresql/13/main/pg_hba.conf - -Scroll down to the bottom of the document. We want to make local connection ``trusted`` for the default user. - -Make sure your configuration looks like the one below. - -.. code-block:: shell - - ... - # DO NOT DISABLE! - # If you change this first entry you will need to make sure that the - # database superuser can access the database using some other method. - # Noninteractive access to all databases is required during automatic - # maintenance (custom daily cronjobs, replication, and similar tasks). - # - # Database administrative login by Unix domain socket - local all postgres trust - - # TYPE DATABASE USER ADDRESS METHOD - - # "local" is for Unix domain socket connections only - local all all md5 - # IPv4 local connections: - host all all 127.0.0.1/32 md5 - # IPv6 local connections: - host all all ::1/128 md5 - # Allow replication connections from localhost, by a user with the - # replication privilege. - local replication all peer - host replication all 127.0.0.1/32 md5 - host replication all ::1/128 md5 - -.. warning:: If your ``PostgreSQL`` database resides on a **separate/remote machine**, you'll have to **allow** remote access to the databases in the ``/etc/postgresql/13/main/pg_hba.conf`` to the ``geonode`` user and tell PostgreSQL to **accept** non-local connections in your ``/etc/postgresql/13/main/postgresql.conf`` file - -Restart PostgreSQL to make the change effective. - -.. code-block:: shell - - sudo service postgresql restart - -PostgreSQL is now ready. To test the configuration, try to connect to the ``geonode`` database as ``geonode`` role. - -.. code-block:: shell - - psql -U postgres my_geonode - # This should not ask for any password - - psql -U my_geonode my_geonode - # This should ask for the password geonode - - # Repeat the test with geonode_data DB - psql -U postgres my_geonode_data - psql -U my_geonode my_geonode_data - - - - -Run GeoNode Project for the first time in DEBUG Mode -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. warning:: - - Be sure you have successfully completed all the steps of the section :ref:`install_dep_proj`. - -This command will run both GeoNode and GeoServer locally after having prepared the Spatialite database. The server will start in ``DEBUG`` (or ``DEVELOPMENT``) mode, and it will start the following services: - -#. GeoNode on ``http://localhost:8000/`` -#. GeoServer on ``http://localhost:8080/geoserver/`` - -This modality is beneficial to debug issues and/or develop new features, but it cannot be used on a production system. - -.. code-block:: shell - - # Prepare the GeoNode Spatialite database (the first time only) - cd src/ - chmod +x paver_dev.sh - ./paver_dev.sh setup - ./paver_dev.sh sync - -.. note:: - - In case you want to start again from a clean situation, just run - - .. code:: shell - - ./paver_dev.sh reset_hard - -.. warning:: This will blow up completely your ``local_settings``, delete the SQLlite database and remove the GeoServer data dir. - -.. code-block:: shell - - # Run the server in DEBUG mode - ./paver_dev.sh start - -Once the server has finished the initialization and prints on the console the sentence ``GeoNode is now available.``, you can open a browser and go to:: - - http://localhost:8000/ - -Sign-in with:: - - user: admin - password: admin - -From now on, everything already said for GeoNode Core (please refer to the section :ref:`configure_dbs_core` and following), applies to a -GeoNode Project. - -**Be careful** to use the **new** paths and names everywhere: - -* Everytime you'll find the keyword ``geonode``, you'll need to use your geonode custom name instead (in this example ``my_geonode``). - -* Everytime you'll find paths pointing to ``/opt/geonode/``, you'll need to update them to point to your custom project instead (in this example ``/opt/geonode_custom/my_geonode``). - -Docker -====== - -.. warning:: Before moving with this section, you should have read and clearly understood the ``INSTALLATION > GeoNode Core`` sections, and in particular the ``Docker`` one. Everything said for the GeoNode Core Vanilla applies here too, except that the Docker container names will be slightly different. As an instance if you named your project ``my_geonode``, your containers will be called: - - .. code-block:: shell - - 'django4my_geonode' instead of 'django4geonode' and so on... - -Deploy an instance of a geonode-project Django template with Docker on localhost -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Prepare the environment - -.. code-block:: shell - - sudo mkdir -p /opt/geonode_custom/ - sudo usermod -a -G www-data geonode - sudo chown -Rf geonode:www-data /opt/geonode_custom/ - sudo chmod -Rf 775 /opt/geonode_custom/ - -Clone the source code - -.. code-block:: shell - - cd /opt/geonode_custom/ - git clone https://github.com/GeoNode/geonode-project.git - -This will clone the ``master`` branch. You will have to checkout the desidered branch or tag. -As an example, if you want to generate a propject for GeoNode 4.2.2 you will docker - -.. code-block:: shell - - git checkout -b 4.2.2 - -Make an instance out of the ``Django Template`` - -.. note:: We will call our instance ``my_geonode``. You can change the name at your convenience. - -.. code-block:: shell - - source /usr/share/virtualenvwrapper/virtualenvwrapper.sh - mkvirtualenv --python=/usr/bin/python3 my_geonode - - Alterantively you can also create the virtual env like below - python3.8 -m venv /home/geonode/dev/.venvs/my_geonode - source /home/geonode/dev/.venvs/my_geonode/bin/activate - - pip install Django==3.2.13 - - django-admin startproject --template=./geonode-project -e py,sh,md,rst,json,yml,ini,env,sample,properties -n monitoring-cron -n Dockerfile my_geonode - cd /opt/geonode_custom/my_geonode - -Create the .env file - -An `.env` file is requird to run the application. It can be created from the `.env.sample` either manually or with the create-envfile.py script. - -The script accepts several parameters to create the file, in detail: - - - *hostname*: e.g. master.demo.geonode.org, default localhost - - *https*: (boolean), default value is False - - *email*: Admin email (this is required if https is set to True since a valid email is required by Letsencrypt certbot) - - *emv_type*: `prod`, `test` or `dev`. It will set the `DEBUG` variable to `False` (`prod`, `test`) or `True` (`dev`) - - *geonodepwd*: GeoNode admin password (required inside the .env) - - *geoserverpwd*: Geoserver admin password (required inside the .env) - - *pgpwd*: PostgreSQL password (required inside the .env) - - *dbpwd*: GeoNode DB user password (required inside the .env) - - *geodbpwd*: Geodatabase user password (required inside the .env) - - *clientid*: Oauth2 client id (required inside the .env) - - *clientsecret*: Oauth2 client secret (required inside the .env) - - *secret key*: Django secret key (required inside the .env) - - *sample_file*: absolute path to a env_sample file used to create the env_file. If not provided, the one inside the GeoNode project is used. - - *file*: absolute path to a json file that contains all the above configuration - -.. note:: if the same configuration is passed in the json file and as an argument, the CLI one will overwrite the one in the JSON file. If some value is not provided, a random string is used - - Example USAGE - -.. code-block:: shell - - ```bash - python create-envfile.py -f /opt/core/geonode-project/file.json \ - --hostname localhost \ - --https \ - --email random@email.com \ - --geonodepwd gn_password \ - --geoserverpwd gs_password \ - --pgpwd pg_password \ - --dbpwd db_password \ - --geodbpwd _db_password \ - --clientid 12345 \ - --clientsecret abc123 - ``` - -Example JSON expected: - -.. code-block:: shell - - ```JSON - { - "hostname": "value", - "https": "value", - "email": "value", - "geonodepwd": "value", - "geoserverpwd": "value", - "pgpwd": "value", - "dbpwd": "value", - "geodbpwd": "value", - "clientid": "value", - "clientsecret": "value" - } - ``` - -Modify the code and the templates and rebuild the Docker Containers - -.. code-block:: shell - - docker-compose -f docker-compose.yml build --no-cache - -Finally, run the containers - -.. code-block:: shell - - docker-compose -f docker-compose.yml up -d - -Deploy an instance of a geonode-project Django template with Docker on a domain -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. note:: We will use ``www.example.org`` as an example. You can change the name at your convenience. - -Stop the containers - -.. code-block:: shell - - cd /opt/geonode_custom/my_geonode - - docker-compose -f docker-compose.yml stop - -Edit the ``ENV`` override file in order to deploy on ``www.example.org`` - -Replace everywhere ``localhost`` with ``www.example.org`` - -.. code-block:: shell - - vim .env - -.. code-block:: shell - - # e.g.: :%s/localhost/www.example.org/g - -.. note:: It is possible to override here even more variables to customize the GeoNode instance. See the ``GeoNode Settings`` section in order to get a list of the available options. - -Run the containers in daemon mode - -.. code-block:: shell - - docker-compose -f docker-compose.yml -f docker-compose.override.example-org.yml up --build -d - -Test geonode-project with vagrant -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. note:: Inside geonode-project files you will find one file named `Vagrantfile.compose` and one named `Vagrantfile.stack`, copy one of them onto file `Vagrantfile` to use them with vagrant. - -.. code-block:: shell - - apt-get install -y vagrant - #choose what to test (in this case docker-compose.yml) - cp Vagrantfile.compose Vagrantfile - #this will start a vargant virtual machine, generate and build geonode-project - vagrant up - # check services are up upon reboot - vagrant ssh geonode-compose -c 'docker ps' - vagrant destroy -f - # test docker swarm - cp Vagrantfile.stack Vagrantfile - vagrant up - # check services are up upon reboot - vagrant ssh geonode-vagrant -c 'docker service ls' - vagrant destroy -f - -.. note:: Vagrant will generate a dummi project named "antani" inside vagrant, starting with the geonode-project codebase, this way it is possible to test inside vagrant almost instantly what one modifies into geonode-project diff --git a/install/basic/index.rst b/install/basic/index.rst deleted file mode 100644 index cb655da0..00000000 --- a/install/basic/index.rst +++ /dev/null @@ -1,376 +0,0 @@ -.. _geonode-project-basic: - -========================== -GeoNode Basic Installation -========================== - -Overview -======== - -The followings are the easiest and recommended ways to deploy a full-stack GeoNode server on your host. - -#. **First Step**: Deploy :guilabel:`GeoNode on a local server`, running as ``http://localhost/`` service. :guilabel:`GeoServer` will be also available at ``http://localhost/geoserver/`` - -#. **Second Step**: Deploy :guilabel:`GeoNode on a production server`, running as ``https://my_geonode.geonode.org/`` service. :guilabel:`GeoServer` will be also available at ``https://my_geonode.geonode.org/geoserver/`` - -#. **Third Step**: Customize :guilabel:`.env` to match your needs - -#. **Fourth Step**: Secure your production deployment; change the :guilabel:`admin` passwords and :guilabel:`OAUth2` keys - -#. **Further Production Enhancements** - -First Step: Deploy GeoNode on a local server -============================================ - -.. _Ubuntu (22.04) Basic Setup: - -Docker Setup for Ubuntu (22.04) -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. code-block:: shell - - # install OS level packages.. - sudo add-apt-repository universe - sudo apt-get update -y - sudo apt-get install -y git-core git-buildpackage debhelper devscripts python3.10-dev python3.10-venv virtualenvwrapper - sudo apt-get install -y apt-transport-https ca-certificates curl lsb-release gnupg gnupg-agent software-properties-common vim - - # add docker repo and packages... - sudo mkdir -p /etc/apt/keyrings - curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg - sudo chmod a+r /etc/apt/keyrings/docker.gpg - echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null - - sudo apt-get update -y - sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose - sudo apt autoremove --purge - - # add your user to the docker group... - sudo usermod -aG docker ${USER} - su ${USER} - -Create an instance of your ``geonode-project`` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Let's say your project is named :guilabel:`my_geonode` perform the following steps: - -.. code-block:: shell - - git clone https://github.com/GeoNode/geonode-project.git - - # Checkout the desired branch. For example, 4.2.2 to generate a project for GeoNode 4.2.2 - - git checkout -b 4.2.2 - - # Ubuntu - source /usr/share/virtualenvwrapper/virtualenvwrapper.sh - mkvirtualenv --python=/usr/bin/python3 my_geonode - - Alterantively you can also create the virtual env like below - python3.8 -m venv /home/geonode/dev/.venvs/my_geonode - source /home/geonode/dev/.venvs/my_geonode/bin/activate - - pip install Django==3.2.13 - - # CentOS - virtualenv -p python3 my_geonode - source my_geonode/bin/activate - - django-admin startproject --template=./geonode-project -e py,sh,md,rst,json,yml,ini,env,sample,properties -n monitoring-cron -n Dockerfile my_geonode - - # If the previous command does not work for some reason, try the following one - python -m django startproject --template=./geonode-project -e py,sh,md,rst,json,yml,ini,env,sample,properties -n monitoring-cron -n Dockerfile my_geonode - -.. _Docker create env file: - -Prepare the .env file -^^^^^^^^^^^^^^^^^^^^^ -GeoNode uses an ``.env`` where the environment variables required by Docker and GeoNode itself must be configured. -A simple way to generate a basic ``.env`` is using the ``create-envfile.py`` utility command. If run without arguments it will generate a valid ``.env`` for a local deployment. - -.. code-block:: shell - cd my_geonode - python create-envfile.py - -The command accepts the following arguments: - -* `--https`: Enable SSL. It's disabled by default -* `--env_type`: - * When set to `prod` `DEBUG` is disabled and the creation of a valid `SSL` is requested to Letsencrypt's ACME server - * When set to `test` `DEBUG` is disabled and a test `SSL` certificate is generated for local testing - * When set to `dev` `DEBUG` is enabled and no `SSL` certificate is generated -* `--hostname`: The URL that will serve GeoNode (`localhost` by default) -* `--email`: The administrator's email. Notice that a real email and valid SMPT configurations are required if `--env_type` is set to `prod`. Letsencrypt uses email for issuing the SSL certificate -* `--geonodepwd`: GeoNode's administrator password. A random value is set if left empty -* `--geoserverpwd`: GeoNode's administrator password. A random value is set if left empty -* `--pgpwd`: PostgreSQL's administrator password. A random value is set if left empty -* `--dbpwd`: GeoNode DB user role's password. A random value is set if left empty -* `--geodbpwd`: GeoNode data DB user role's password. A random value is set if left empty -* `--clientid`: Client id of Geoserver's GeoNode Oauth2 client. A random value is set if left empty -* `--clientsecret`: Client secret of Geoserver's GeoNode Oauth2 client. A random value is set if left empty - -.. _Docker build and run: - -Build and run -^^^^^^^^^^^^^ -.. code-block:: shell - - docker compose build - docker compose up -d - -- You can follow the containers startup by running the following commands from ``my_geonode`` root folder: - - .. code-block:: shell - - # GeoNode Container - docker-compose logs -f django - - # GeoServer Container - docker-compose logs -f geoserver - - # DB Container - docker-compose logs -f db - - # NGINX Container - docker-compose logs -f geonode - -- If any error occurs, try to catch the error stacktrace by running the following commands from ``my_geonode`` root folder: - - .. code-block:: shell - - # GeoNode “entrypoint.sh” Logs - tail -F -n 300 invoke.log - - -Connect to :guilabel:`http://localhost/` -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The startup typically takes some time, so be patient… - -If everything goes well, you should be able to see from the ``geonode startup logs`` a line similar to the following one: - -.. code-block:: shell - - [UWSGI] Uwsgi running... - -Connect to :guilabel:`http://localhost/` - -The default credentials are: - - * GeoNode (:guilabel:`http://localhost/`) :guilabel:`admin`: - - ``username: admin`` - ``password: admin`` - - * GeoServer (:guilabel:`http://localhost/geoserver/`) :guilabel:`admin`: - - ``username: admin`` - ``password: geoserver`` - -.. _Docker deploy to production: - -Second Step: Deploy GeoNode on a production server -================================================== - -In the case you would like to deploy to, let's say, :guilabel:`https://my_geonode.geonode.org/`, you will need to change ``.env`` as follows: - -.. code-block:: diff - - DOCKER_ENV=production - SITEURL=https://my_geonode.geonode.org/ - NGINX_BASE_URL=https://my_geonode.geonode.org/ - ALLOWED_HOSTS=['django',] - GEOSERVER_WEB_UI_LOCATION=https://my_geonode.geonode.org/geoserver/ - GEOSERVER_PUBLIC_LOCATION=https://my_geonode.geonode.org/geoserver/ - HTTP_HOST= - HTTPS_HOST=my_geonode.geonode.org - HTTP_PORT=80 - HTTPS_PORT=443 - LETSENCRYPT_MODE=production # This will use Letsencrypt and the ACME server to generate valid SSL certificates - -These variables are automatically set by the ``create-envfile.py`` script if the ``--https`` and ``--hostname`` variables are used. - -.. warning:: When ``LETSENCRYPT_MODE`` is set to ``production`` a valid email and email SMPT server are required to make the system generate a valid certificate. - -Restart the containers -^^^^^^^^^^^^^^^^^^^^^^ - -Whenever you change someting on :guilabel:`.env` file, you will need to rebuild the container - -.. warning:: Notche that the following command drops any change you might have done manually inside the containers, except for the static volumes. - -.. code-block:: shell - - docker-compose up -d - -Troubleshooting -^^^^^^^^^^^^^^^ - -If for some reason you are not able to reach the server on the :guilabel:`HTTPS` channel, please check the :guilabel:`NGINX` configuration files below: - -1. Enter the :guilabel:`NGINX` container - - .. code-block:: shell - - docker-compose exec geonode sh - -2. Install an editor if not present - - .. code-block:: shell - - apk add nano - -3. Double check that the ``nginx.https.enabled.conf`` link has been correctly created - - .. code-block:: shell - - ls -lah - - .. figure:: img/throubleshooting_prod_001.png - :align: center - - If the list does not match exactly the figure above, please run the following commands, and check again - - .. code-block:: shell - - rm nginx.https.enabled.conf - ln -s nginx.https.available.conf nginx.https.enabled.conf - -4. Inspect the ``nginx.https.enabled.conf`` contents - - .. code-block:: shell - - nano nginx.https.enabled.conf - - Make sure the contents match the following - - .. warning:: - - Change the :guilabel:`Hostname` accordingly. **This is only an example!** - - .. code-block:: shell - - # NOTE : $VARIABLES are env variables replaced by entrypoint.sh using envsubst - # not to be mistaken for nginx variables (also starting with $, but usually lowercase) - - # This file is to be included in the main nginx.conf configuration if HTTPS_HOST is set - ssl_session_cache shared:SSL:10m; - ssl_session_timeout 10m; - - # this is the actual HTTPS host - server { - listen 443 ssl; - server_name my_geonode.geonode.org; - keepalive_timeout 70; - - ssl_certificate /certificate_symlink/fullchain.pem; - ssl_certificate_key /certificate_symlink/privkey.pem; - ssl_protocols TLSv1 TLSv1.1 TLSv1.2; - ssl_ciphers HIGH:!aNULL:!MD5; - - include sites-enabled/*.conf; - } - - # if we try to connect from http, we redirect to https - server { - listen 80; - server_name my_geonode.geonode.org; # TODO : once geoserver supports relative urls, we should allow access though both HTTP and HTTPS at the same time and hence remove HTTP_HOST from this line - - # Except for let's encrypt challenge - location /.well-known { - alias /geonode-certificates/.well-known; - include /etc/nginx/mime.types; - } - - # Redirect to https - location / { - return 302 https://my_geonode.geonode.org/$request_uri; # TODO : we should use 301 (permanent redirect, but not practical for debug) - } - } - - .. warning:: - - **Save the changes, if any, and exit!** - -5. Reload the NGINX configuration - - .. code-block:: shell - - nginx -s reload - 2020/06/24 10:00:11 [notice] 112#112: signal process started - /etc/nginx# exit - -6. It may be helpful to disable https to isolate the source of errors. After reverting the HTTPS-related changes in the `.env` file, repeat the above steps and ensure that the ``nginx.http.enabled.conf`` link has been correctly created. - - .. code-block:: shell - - ln -s nginx.conf nginx.http.enabled.conf - nano nginx.http.enabled.conf - -Customize :guilabel:`.env` to match your needs -=========================================================== - -In the case you would like to modify the GeoNode behavior, always use the :guilabel:`.env` file in order to update the :guilabel:`settings`. - -If you need to change a setting which does not exist in :guilabel:`.env`, you can force the values inside :guilabel:`my_geonode/settings.py` - -Refer to the section: :ref:`settings` - -You can add here any property referred as - - | Env: ``PROPERTY_NAME`` - - -Restart the containers -^^^^^^^^^^^^^^^^^^^^^^ - -Whenever you change someting on :guilabel:`.env` file, you will need to rebuild the containers. - -.. warning:: **Be careful!** The following command drops any change you might have done manually inside the containers, except for the static volumes. - -.. code-block:: shell - - docker-compose up -d django - - -[Optional] Configure your SSL Certificates -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -In production deployment mode, GeoNode uses by default :guilabel:`Let's Encrypt` certificates - -You may want to provide your own certificates to GeoNode - -.. code-block:: shell - - docker exec -it nginx4my_geonode_geonode sh -c 'mkdir /geonode-certificates/my_geonode' - - wget --no-check-certificate 'http://' \ - -O chain.crt - - wget --no-check-certificate 'http://' \ - -O my_geonode.key - - docker cp chain.crt nginx4my_geonode_geonode:/geonode-certificates/my_geonode - - docker cp my_geonode.key nginx4my_geonode_geonode:/geonode-certificates/my_geonode - - docker-compose exec geonode sh - apk add vim - - vim nginx.https.enabled.conf - - -.. code-block:: diff - - -ssl_certificate /certificate_symlink/fullchain.pem; - -ssl_certificate_key /certificate_symlink/privkey.pem; - +ssl_certificate /geonode-certificates/my_geonode/chain.crt; - +ssl_certificate_key /geonode-certificates/my_geonode/my_geonode.key; - - -.. code-block:: shell - - nginx -s reload - exit - - diff --git a/install/index.rst b/install/index.rst deleted file mode 100644 index ffa7d06b..00000000 --- a/install/index.rst +++ /dev/null @@ -1,10 +0,0 @@ - -.. toctree:: - :maxdepth: 3 - - basic/index - -.. toctree:: - :maxdepth: 3 - - advanced/index diff --git a/intermediate/backup/index.rst b/intermediate/backup/index.rst deleted file mode 100644 index f9986ea9..00000000 --- a/intermediate/backup/index.rst +++ /dev/null @@ -1,606 +0,0 @@ -.. _backup_restore_geonode: - -============================= -Full GeoNode Backup & Restore -============================= - -The admin command to backup and restore GeoNode, allows to extract consistently the GeoNode and GeoServer data models in a serializable -meta-format which is being interpreted later by the restore procedure in order to exactly rebuild the whole structure. - -In particular the tool helps developers and administrators to correctly extract and serialize the following resources: - -* **GeoNode** (Resource Base Model): - - #. Layers (both raster and vectors) - - #. Maps - - #. Documents - - #. People with Credentials - - #. Permissions - - #. Associated Styles - - #. Static data and templates - -* **GeoServer** (Catalog): - - #. OWS Services configuration and limits - - #. Security model along with auth filters configuration, users and credentials - - #. Workspaces - - #. Stores (both DataStores and CoverageStores) - - #. Layers - - #. Styles - -The tool exposes two GeoNode Management Commands, 'backup' and 'restore'. - -The commands allow to: - -#. Fully backup GeoNode data and fixtures on a zip archive - -#. Fully backup GeoServer configuration (physical datasets - tables, shapefiles, geotiffs) - -#. Fully restore GeoNode and GeoServer fixtures and catalog from the zip archive - -The usage of those commands is quite easy and straightforward. - -The first step is to ensure that everything is correctly configured and the requisites respected in order to successfully -perform a backup and restore of GeoNode. - -.. warning:: It is worth to notice that this functionality requires the latest `GeoServer Extension `_ (2.9.x or greater) for GeoNode in order to correctly work. - -.. note:: GeoServer full documentation is also available here `GeoServer Docs `_ - -Requisites and Setup -==================== - -**Before** running a GeoNode backup / restore, it is necessary to ensure everything is correctly configured and setup. - -Settings --------- - -Accordingly to the admin needs, the file ``settings.ini`` must be ceated before running a backup or restore. - -The default files can be found at ``geonode/br/management/commands/settings_sample.ini`` and ``geonode/br/management/commands/settings_docker_sample.ini`` -for the classic and Docker environments accordingly. The content is similar in both of them (an example from ``settings_sample.ini``): - -.. code-block:: ini - - [database] - pgdump = pg_dump - pgrestore = pg_restore - - [geoserver] - datadir = geoserver/data - dumpvectordata = yes - dumprasterdata = yes - - [fixtures] - # NOTE: Order is important - apps = contenttypes,auth,people,groups,account,guardian,admin,actstream,announcements,avatar,base,dialogos,documents,geoserver,invitations,pinax_notifications,layers,maps,oauth2_provider,services,sites,socialaccount,taggit,tastypie,upload,user_messages - dumps = contenttypes,auth,people,groups,account,guardian,admin,actstream,announcements,avatar,base,dialogos,documents,geoserver,invitations,pinax_notifications,layers,maps,oauth2_provider,services,sites,socialaccount,taggit,tastypie,upload,user_messages - -The ``settings.ini`` file can be created in any directory accessible by GeoNode, and it's path can be passed to the backup / restore -procedures using `-c` (`--config`) argument. - -There are few different sections of the configuration file, that must be carefully checked before running a backup / restore command. - -Settings: [database] Section -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. code-block:: ini - - [database] - pgdump = pg_dump - pgrestore = pg_restore - -This section is quite simple. It contains only two properties: - -* *pgdump*; the path of the ``pg_dump`` local command. - -* *pgrestore*; the path of the ``pg_restore`` local command. - -.. warning:: Those properties are ignored in case GeoNode is not configured to use a DataBase as backend (see ``settings.py`` and ``local_settings.py`` sections) - -.. note:: Database connection settings (both for GeoNode and GeoServer) will be taken from ``settings.py`` and ``local_settings.py`` configuration files. Make sure they are correctly configured (on the target GeoNode instance, too) and the DataBase server is accessible while executing a backup / restore command. - -Settings: [geoserver] Section -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. code-block:: ini - - [geoserver] - datadir = /opt/gs_data_dir - datadir_exclude_file_path = - dumpvectordata = yes - dumprasterdata = yes - data_dt_filter = - data_layername_filter = - data_layername_exclude_filter = - -This section allows to enable / disable a full data backup / restore of GeoServer. - -* *datadir*: the full path of GeoServer Data Dir, by default ``/opt/gs_data_dir``. The path **must** be accessible and **fully writable** by the ``geonode`` and / or ``httpd server`` users when executing a backup / restore command. - -* *datadir_exclude_file_path*: comma separated list of paths to exclude from ``geoserver_catalog.zip``; This list will be sent and managed directly by the GeoServer Backup REST API. - -* *dumpvectordata*: a boolean flag enabling or disabling creation of a vector data dump from GeoServer (shapefiles or DB tables). If ``false`` (or ``no``) vector data won't be stored / re-stored. - -* *dumprasterdata*: a boolean flag enabling or disabling creation of a raster data dump from GeoServer (geotiffs). If ``false`` (or ``no``) raster data won't be stored / re-stored. - -* *data_dt_filter*: {cmp_operator} {ISO8601} e.g. > 2019-04-05T24:00 which means "include on backup archive only the files that have been modified later than 2019-04-05T24:00 - -* *data_layername_filter*: comma separated list of ``layer names``, optionally with glob syntax e.g.: tuscany_*,italy; Only ``RASTER`` original data and ``VECTORIAL`` table dumps matching those filters will be **included** into the backup ZIP archive - -* *data_layername_exclude_filter*: comma separated list of ``layer names``, optionally with glob syntax e.g.: tuscany_*,italy; The ``RASTER`` original data and ``VECTORIAL`` table dumps matching those filters will be **excluded** from the backup ZIP archive - - -.. warning:: Enabling these options **requires** the GeoServer Data Dir to be accessible and **fully writable** for the ``geonode`` and / or ``httpd server`` users when executing a backup / restore command. - -Settings: [fixtures] Section -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. code-block:: ini - - [fixtures] - #NOTE: Order is important - apps = people,account,avatar.avatar,base.backup,base.license,base.topiccategory,base.region,base.resourcebase,base.contactrole,base.link,base.restrictioncodetype,base.spatialrepresentationtype,guardian.userobjectpermission,guardian.groupobjectpermission,layers.uploadsession,layers.style,layers.layer,layers.attribute,layers.layerfile,maps.map,maps.maplayer,maps.mapsnapshot,documents.document,taggit - - dumps = people,accounts,avatars,backups,licenses,topiccategories,regions,resourcebases,contactroles,links,restrictioncodetypes,spatialrepresentationtypes,useropermissions,groupopermissions,uploadsessions,styles,layers,attributes,layerfiles,maps,maplayers,mapsnapshots,documents,tags - -This section is the most complex one. Usually you don't need to modify it. Only an expert user who knows Python and GeoNode model structure should modify this section. - -What its properties mean: - -* *apps*; an ordered list of GeoNode Django applications. The backup / restore procedure will dump / restore the fixtures in a portable format. - -* *dumps*; this is the list of ``files`` associated to the Django applications. The order **must** be the same as in the *apps* property above. Each name represents the ``file name`` where to dump to / read from the single app's fixtures. - - -Executing from the CLI -====================== - -The following sections shows instructions on how to perform backup / restore from the command line by using the Django Admin Management Commands. - -In order to obtain a basic user guide for the management command from the command line, just run - - .. code-block:: shell - - python manage.py backup --help - - python manage.py restore --help - -``--help`` will provide the list of available command line options with a brief description. - -By default both procedures activate `Read Only` mode, disabling any content modifying requests, which is reverted -to the previous state (from before the execution) after finish, regardless of the command's result (success or failure). -To disable activation of this mode, ``--skip-read-only`` argument can be passed to the command. - -It is worth notice that both commands allows the following option - - .. code-block:: shell - - python manage.py backup --force / -f - - python manage.py restore --force / -f - -Which enables a non-interactive mode, meaning the user will not be asked for an explicit confirmation. - -Backup ------- - -In order to perform a backup just run the command: - - .. code-block:: shell - - python manage.py backup --backup-dir= --config= - -The management command will automatically generate a ``.zip`` archive file on the target folder in case of success. In the target directory -``.md5`` file with the same name as backup will be created. It contains the MD5 hash of the backup file, which can be used to check archive's -integrity before restoration. - -It is worth to mention that ``br`` (Backup & Restore GeoNode application) will not be dumped, even if specified in the ``settings.ini`` as -its content is strictly related to the certain GeoNode instance. - -Currently, GeoNode does not support any automatic extraction of the backup file. It should be manually transferred, if needed to the target -instance environment. - -Restore -------- - -The ``restore`` command has a number of arguments, modifying its execution: - -# ``-c`` / ``--config``: path to the ``settings.ini`` configuration file. If the Backup archive is provided with his settings, the latter will be used by the restore command and this option won't be mandatory anymore - -#. ``--skip-geoserver``: the GeoServer backup restoration won't be performed - -#. ``--skip-geoserver-info``: {Default: True} Skips GeoServer Global Infos, like the proxy base url and other global GeoServer metadata info - -#. ``--skip-geoserver-security``: {Default: True} Skips GeoServer all the Security Settings - -#. ``--backup-file``: (exclusive together with ``--backup-files-dir``) path to the backup ``.zip`` archive - -#. ``--backup-files-dir``: (exclusive together with ``--backup-file``) directory containing backup archives. The directory may contain a number of files, but **only** backup archives are allowed with a ``.zip`` extension. In case multiple archives are present in the directory, the newest one, created after the last already restored backup creation time, will be restored. This option was implemented with a thought of automated restores. - -#. ``--recovery-file``: Backup archive containing GeoNode data to restore in case of failure. - -#. ``-l`` / ``--with-logs``: the backup file will be checked against the restoration logs (history). In case this backup has already been restored (MD5 based comparision), RuntimeError is raised, preventing restore execution. - -#. ``-n`` / ``--notify``: the restore procedure outcome will be send by an e-mail notification to the superusers of the instance (note: notification will be sent to the superusers of the instance before restoration). - -#. ``--skip-read-only``: the restore procedure will be conducted without setting `Read Only` mode during execution. - -#. ``--soft-reset``: the restore procedure will preserve geoserver table / resources during the restore. By default the procedure will drop tables and resources - -In order to perform a default backup restoration just run the command: - - .. code-block:: shell - - python manage.py restore --backup-file= --config= - -For restore to run it requires either ``--backup-file`` or ``--backup-files-dir`` argument defined. - -.. warning:: The Restore will **overwrite** the whole target instances of GeoNode (and by default GeoServer) including users, catalog and database, so be very careful. - -GeoNode Admin GUI Inspection -============================ - -The history of restored backups can be verified in the admin panel. - -Login to the admin panel and select ``Restored backups`` table from ``BACKUP/RESTORE`` application. - -.. figure:: img/br_1.png - :align: center - - -A list will be displayed with a history of all restored backups. You can select a certain backup to view it's data. - -.. figure:: img/br_2.png - :align: center - - -The detailed view of the restored backup shows backup archive's name, it's MD5 hash, it's creation/modification date (in the target folder), and the date of the restoration. Please note Restored Backup history cannot be modified. - -.. figure:: img/br_3.png - :align: center - -B/R in Docker environment -========================= - -When executing B/R in the Docker environment, creation backup to / restoration from should be executed in ``/backup_restore`` directory. -It is a shared volume between Geoserver and Geonode images, created for this purpose only. Pointing at another -location will fail, as one of the images won't have an access to the files. - -.. warning:: When executing B/R in Docker environment **remember** to create ``settings.ini`` file basing on ``settings_docker_sample.ini`` to point at a proper Geoserver data directory! In other case configuration mismatch may cause unexpected errors. - -.. warning:: The only other volume shared between images is ``/geoserver_data/data``, but backup creation **should not** be performed there, as the recursive Geoserver backups may be created in such case. - -B/R Jenkins Job in Docker environment -===================================== - -When installing GeoNode through the :guilabel:`geonode-project` Docker (see :ref:`geonode-project-basic`), an instance of `Jenkins CI/CD `_ is also automatically deployed and available through :guilabel:`http:///jenkins`. - - -.. figure:: img/br_jenkins_1.png - :align: center - -Configure Jenkins at first startup ----------------------------------- - -The very first time you try to access Jenkins, you will need to unlock it and generate a new administrator username and password. - -In order to do that, you need to print the contents of the auto-generated file ``/var/jenkins_home/secrets/initialAdminPassword`` - -#. First of all search for the Jenkins container ID, usually :guilabel:`jenkins4{{project_name}}` where ``{{project_name}}`` is the name of your :guilabel:`geonode-project` instance (e.g. ``my_geonode``) - -.. code:: shell - - $> docker ps - - CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES - e9fc97a75d1a geonode/nginx:geoserver "/docker-entrypoint.…" 2 hours ago Up 2 hours 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp nginx4my_geonode - c5496400b1b9 my_geonode_django "/bin/sh -c 'service…" 2 hours ago Up 2 hours django4my_geonode - bc899f81fa28 my_geonode_celery "/bin/sh -c 'service…" 2 hours ago Up 2 hours celery4my_geonode - 3b213400d630 geonode/geoserver:2.17.1 "/usr/local/tomcat/t…" 2 hours ago Up 2 hours 8080/tcp geoserver4my_geonode - d2f59d70a0d3 geonode/postgis:11 "docker-entrypoint.s…" 2 hours ago Up 2 hours 5432/tcp db4my_geonode - 3f9ce0be7f88 rabbitmq "docker-entrypoint.s…" 2 hours ago Up 2 hours 4369/tcp, 5671-5672/tcp, 25672/tcp rabbitmq4my_geonode - 02fdbce9ae73 geonode/letsencrypt:latest "./docker-entrypoint…" 2 hours ago Up 14 seconds my_geonode_letsencrypt_1 - c745520fd551 jenkins/jenkins:lts "/sbin/tini -- /usr/…" 2 hours ago Up 2 hours 0.0.0.0:9080->9080/tcp, 8080/tcp, 0.0.0.0:50000->50000/tcp, 0.0.0.0:9443->8443/tcp jenkins4my_geonode - -#. Now just ``cat`` the file above inside the Jenkins container - -.. code:: shell - - $> docker container exec -u 0 -it jenkins4my_geonode sh -c 'cat /var/jenkins_home/secrets/initialAdminPassword' - - b91e9d*****************373834 - -#. Copy the hash code you just got form the print above, and copy-and-paste to the browser window - -.. figure:: img/br_jenkins_2.png - :align: center - -In the next step just install the :guilabel:`Default Plugins`. You can install more of them later on from the management page. - -.. figure:: img/br_jenkins_3.png - :align: center - -Wait until Jenkins has finished configuring the plugins - -.. figure:: img/br_jenkins_4.png - :align: center - -Provide the administrator credentials as requested - -.. figure:: img/br_jenkins_5.png - :align: center - -Confirm the Jenkins instance URL, this can be changed form the configuration later in case you will need to update the server address - -.. figure:: img/br_jenkins_6.png - :align: center - -Well done, Jenkins is ready now - -.. figure:: img/br_jenkins_7.png - :align: center - -The next step is to configure a Jenkins Job able to interact with the Django Docker container and run a full backup - -.. figure:: img/br_jenkins_8.png - :align: center - -Configure a Jenkins Job to run a full backup on the Django Container --------------------------------------------------------------------- - -Before creating the new Jenkins job, we need to install and configure a new plugin, `Publish over SSH `_ - -In order to do that, once logged in as ``admin``, go to the Jenkins :guilabel:`Management Page > Manage Plugins` tab - -.. figure:: img/br_jenkins_9.png - :align: center - -.. figure:: img/br_jenkins_10.png - :align: center - -Click on :guilabel:`Available` tab and search for ``SSH`` available plugins - -.. figure:: img/br_jenkins_11.png - :align: center - -Select and check the ``Publish over SSH`` one - -.. figure:: img/br_jenkins_12.png - :align: center - -Install the plugins and restart Jenkins - -.. figure:: img/br_jenkins_13.png - :align: center - -The next step is to configure the ``SSH Server Connection`` for the :guilabel:`Publish over SSH` plugin. - -Move to :guilabel:`Jenkins Configuration` - -.. figure:: img/br_jenkins_14.png - :align: center - -Scroll down until you find the :guilabel:`Publish over SSH` plugin section - -.. figure:: img/br_jenkins_15.png - :align: center - -Depending on how your ``HOST SSH service`` has been configured, you might need several information in order to setup the connection. - -Here below an example using a global host (``master.demo.geonode.org``) accepting ``SSH`` connections via ``RSA keys`` - -.. figure:: img/br_jenkins_16.png - :align: center - -.. note:: - Before saving the configuration always ensure the connection is ok by using the :guilabel:`Test Configuration` button - - .. figure:: img/br_jenkins_17.png - :align: center - -It is possible also to run and configure :guilabel:`Jenkins` to run locally, as an instance on :guilabel:`localhost`. -In that case you will need to change few things in order to allow :guilabel:`Jenkins` to access your local network. - -#. First of all, be sure :guilabel:`OpenSSH Server` is correctly installed and running on your PC. Eventually check any firewall rules. - - .. code:: shell - - $> sudo apt install openssh-server - - # Test your connection locally - $> ssh -p 22 user@localhost - user@localhost's password: - -#. You will need to do some changed to your ``docker-compose.yml`` file in order to enable the :guilabel:`host network` configuration. - - .. note:: Enable ``network_mode: "host"`` on Jenkins container - - .. code:: shell - - $> vim docker-compose.yml - - ... - jenkins: - image: jenkins/jenkins:lts - # image: istresearch/jenkins:latest - container_name: jenkins4${COMPOSE_PROJECT_NAME} - user: jenkins - ports: - - '${JENKINS_HTTP_PORT}:${JENKINS_HTTP_PORT}' - - '${JENKINS_HTTPS_PORT}:${JENKINS_HTTPS_PORT}' - - '50000:50000' - network_mode: "host" - volumes: - - jenkins_data:/var/jenkins_home - - backup-restore:/backup_restore - # - /var/run/docker.sock:/var/run/docker.sock - environment: - - 'JENKINS_OPTS=--httpPort=${JENKINS_HTTP_PORT} --httpsPort=${JENKINS_HTTPS_PORT} --prefix=/jenkins' - ... - - # Recreate the Jenkins container - $> docker-compose stop jenkins - $> docker-compose rm jenkins - $> docker-compose up -d jenkins - - .. warning:: From now on, your local Jenkins instance will be accessible from :guilabel:`http://localhost:9080/jenkins` - -#. Add ``localhost`` Server to the :guilabel:`Publish over SSH` plugin configuration - - Mode to :guilabel:`http://localhost:9080/jenkins/configure` and fill the required information - - .. figure:: img/br_jenkins_18.png - :align: center - - .. note:: - Before saving the configuration always ensure the connection is ok by using the :guilabel:`Test Configuration` button - - .. figure:: img/br_jenkins_17.png - :align: center - -We are now ready to create the Jenkins Job which will run a full backup & restore of our GeoNode dockerized instance. - -#. Move to the Jenkins Home and click on :guilabel:`Create a Job` button - - .. figure:: img/br_jenkins_19.png - :align: center - -#. Provide a name to the Job and select :guilabel:`Freestyle project` - - .. figure:: img/br_jenkins_20.png - :align: center - -#. Enable the :guilabel:`Log rotation` strategy if needed - - .. figure:: img/br_jenkins_21.png - :align: center - -#. Configure the :guilabel:`Job Parameters` which will be used by the script later on. - - Add three :guilabel:`String Parameters` - - .. figure:: img/br_jenkins_22.png - :align: center - - as shown below - - #. :guilabel:`BKP_FOLDER_NAME` - - .. figure:: img/br_jenkins_23.png - :align: center - - #. :guilabel:`SOURCE_URL` - - .. warning:: Provide the correct URL of your GeoNode instance - - .. figure:: img/br_jenkins_24.png - :align: center - - #. :guilabel:`TARGET_URL` - - .. warning:: Provide the correct URL of your GeoNode instance - - .. figure:: img/br_jenkins_25.png - :align: center - -#. Enable the :guilabel:`Delete workspace before build starts` and :guilabel:`Add timestamps to the Console Output` :guilabel:`Build Environment` options - - .. figure:: img/br_jenkins_26.png - :align: center - -#. Finally let's create the :guilabel:`SSH Build Step` - - .. figure:: img/br_jenkins_27.png - :align: center - - Select the correct :guilabel:`SSH Server` and provide the :guilabel:`Exec Command` below - - .. warning:: Replace :guilabel:`{{project_name}}` with your :guilabel:`geonode-project instance name` (e.g. :guilabel:`my_geonode`) - - .. code:: shell - - # Replace {{project_name}} with your geonode-project instance name (e.g. my_geonode) - # docker exec -u 0 -it django4{{project_name}} sh -c 'SOURCE_URL=$SOURCE_URL TARGET_URL=$TARGET_URL ./{{project_name}}/br/backup.sh $BKP_FOLDER_NAME' - # e.g.: - docker exec -u 0 -it django4my_geonode sh -c 'SOURCE_URL=$SOURCE_URL TARGET_URL=$TARGET_URL ./my_geonode/br/backup.sh $BKP_FOLDER_NAME' - - .. figure:: img/br_jenkins_28.png - :align: center - - Click on :guilabel:`Advanced` and change the parameters as shown below - - .. figure:: img/br_jenkins_29.png - :align: center - -**Save!** You are ready to run the Job... - -.. figure:: img/br_jenkins_30.png - :align: center - -.. figure:: img/br_jenkins_31.png - :align: center - -.. figure:: img/br_jenkins_32.png - :align: center - -.. figure:: img/br_jenkins_33.png - :align: center - -Link the :guilabel:`backup_restore` folder to a local folder on the :guilabel:`HOST` ------------------------------------------------------------------------------------- - -In the case you need to save the backup archives outside the docker container, there's the possibility to directly link the :guilabel:`backup_restore` folder to a local folder on the :guilabel:`HOST`. - -In that case you won't need to :guilabel:`docker cp` the files everytime from the containers, they will be directly available on the host filesystem. - -.. warning:: Always keep an eye to the disk space. Backups archives may be huge. - -.. note:: You might want also to consider filtering the files through the backup dt filters on the :guilabel:`settings.ini` in order to reduce the size of the archive files, including only the new ones. - -Modify the ``docker-compose.override.yml`` as follows in order to link the backup folders outside. - -.. note:: ``/data/backup_restore`` is a folder physically located into the host filesystem. - -.. code:: shell - - $> vim docker-compose.override.yml - - version: '2.2' - services: - - django: - build: . - # Loading the app is defined here to allow for - # autoreload on changes it is mounted on top of the - # old copy that docker added when creating the image - volumes: - - '.:/usr/src/my_geonode' - - '/data/backup_restore:/backup_restore' # Link to local volume in the HOST - - celery: - volumes: - - '/data/backup_restore:/backup_restore' # Link to local volume in the HOST - - geoserver: - volumes: - - '/data/backup_restore:/backup_restore' # Link to local volume in the HOST - - jenkins: - volumes: - - '/data/backup_restore:/backup_restore' # Link to local volume in the HOST - - # Restart the containers - $> docker-compose up -d - diff --git a/intermediate/harvesting/index.rst b/intermediate/harvesting/index.rst deleted file mode 100644 index d97cb1ed..00000000 --- a/intermediate/harvesting/index.rst +++ /dev/null @@ -1,507 +0,0 @@ -========================================= -Harvesting resources from remote services -========================================= - -GeoNode is able to harvest resource metadata from multiple remote services. - -Harvesting is the process by which a metadata catalogue, *i.e.* GeoNode, is able to connect to other remote catalogues -and retrieve information about their resources. This process is usually performed periodically, in order to keep the -local catalogue in sync with the remote. - -When appropriately configured, GeoNode will contact the remote service, extract a list of relevant resources that can be -harvested and create local resources for each remote resource. It will also keep the resources synchronized with the -remote service by periodically updating them. - -Out of the box, GeoNode ships with support for harvesting from: - -#. :ref:`Other remote GeoNode instances `; -#. :ref:`OGC WMS servers `; -#. :ref:`ArcGIS REST services `. - -Adding support for :ref:`additional harvesting sources ` is also possible. - - -GeoNode harvesting concepts -=========================== - -When a **harvester** is configured, GeoNode is able to use its corresponding **harvester worker** to contact -the remote service and generate a list of **harvestable resources**. -The user is then able to select which of those resources are of interest. Depending on its configured update -frequency, sometime later, the **harvesting scheduler** will create new **harvesting sessions** in order to create -local GeoNode resources from the remote harvestable resources that had been marked as relevant by the user. - -The above description uses the following key concepts: - -.. _harvester-label: - -harvester - This is the configuration object that is used to parametrize harvesting of a remote service. It is configurable - at runtime and is preserved in the GeoNode database. - - Harvesters and their properties can be managed by visiting the ``Harvesting -> Harvesters`` section of the GeoNode - admin area, or by visiting the ``api/v2/harvesters/`` API endpoint with an admin user. - - Among other parameters, a harvester holds: - - remote_url - Base URL of the remote service being harvested, *e.g.* ``https://stable.demo.geonode.org`` - - harvester_type - Type of harvester worker that will be used to perform harvesting. See the - :ref:`Harvester worker concept ` and the :ref:`standard harvester workers - ` sections below for more detail. Example: ``geonode.harvesting.harvesters.geonodeharvester.GeonodeUnifiedHarvesterWorker``. - - scheduling_enabled - Whether harvesting shall be performed periodically by the - :ref:`harvesting scheduler ` or not. - - harvesting_session_update_frequency - How often (in minutes) should new :ref:`harvesting sessions ` be - automatically scheduled? - - refresh_harvestable_resources_update_frequency - How often (in minutes) should new :ref:`refresh sessions ` be automatically scheduled? - - default_owner - Which GeoNode user shall be made the owner of harvested resources - - harvest_new_resources_by_default - Should new remote resources be harvested automatically? When this option is selected, the user does not - need to specify which :ref:`harvestable resources ` should be harvested, - as all of them will be automatically marked for harvesting by GeoNode. - - delete_orphan_resources_automatically - Orphan resources are those that have previously been created by means of a harvesting operation but that - GeoNode can no longer find on the remote service being harvested. Should these resources be deleted from - GeoNode automatically? This also applies to when a harvester configuration is deleted, in which case all of - the resources that originated from that harvester are now considered to be orphan. - -.. _harvester-worker-label: - -harvester worker - Harvester workers implement retrieval for concrete remote service types. Each harvester uses a specific worker, - depending on the type of remote service that it gets data from. Harvester workers may accept their own additional - configuration parameters. - - Harvester workers are set as the ``harvester_type`` attribute on a :ref:`harvester `. Their configuration is set as a JSON - object on the ``harvester_type_specific_configuration`` attribute of the harvester. - - GeoNode ships with the following harvester workers: - - #. :ref:`GeoNode ` - Enables harvesting from other GeoNode deployments - #. :ref:`WMS ` - Enables harvesting from OGC WMS servers - #. :ref:`ArcGIS REST services ` - Enables harvesting from ArcGIS REST services - - :ref:`Adding new harvester workers ` is also possible. This allows custom GeoNode - deployments to add support for harvesting from other remote sources. - - -.. _harvestable-resource-label: - -harvestable resource - A resource that is available on the remote server. Harvestable resources are persisted in the GeoNode DB. They are - created during :ref:`refresh operations `, when the harvester worker - interacts with the remote service in order to discover which remote resources can be harvested. - - Harvestable resources can be managed by visiting the ``Harvesting -> Harvestable resources`` section of the - GeoNode admin area, or by visiting the ``api/v2/harvesters/{harvester-id}/harvestable-resources`` API endpoint - with an admin user. - - In order to be harvested by the :ref:`harvesting scheduler `, a harvestable resource - must have its ``should_be_harvested`` attribute set to ``True``. This attribute can be set manually by the user - or it can be set automatically by the harvester worker, in case the corresponding harvester is configured with - ``harvest_new_resources_by_default = True`` - - -.. _asynchronous-session-label: - -harvesting session - In GeoNode, discovering remote resources and harvesting them is always done under the scope of a harvesting - session. These sessions are stored in the GeoNode DB and can be inspected by visiting the - ``Harvesting -> Asynchronous harvesting sessions`` section of the GeoNode admin area. - - Harvesting sessions are used to keep track of the progress of execution of the relevant harvesting operations. They - are updated while each operation is running. There are two types of sessions: - - - .. _refresh-session-label: - - refresh session - This session is created during the :ref:`update of harvestable resources operation `. - It has ``type=discover-harvestable-resources``. During a refresh session, the harvester worker discovers remote - resources and creates their respective :ref:`harvestable resources ` on the GeoNode - DB. After such session is finished, the user can inspect the found harvestable resources and mark those that - are relevant with ``should_be_harvester=True``. - - .. _harvesting-session-label: - - harvesting session - This session is created during the :ref:`perform harvesting operation `. It has - ``type=harvesting``. During a harvesting session, the harvester worker creates or updates new GeoNode resources - based on the harvestable resources that have been configured with ``should_be_harvested=True``. - - In addition to the aforementioned ``type``, harvesting sessions also carry the ``status`` attribute, which provides - context on the current status of the session (and consequently of the underlying harvesting operation). - - -.. _harvesting-scheduler-label: - -harvesting scheduler - The scheduler is responsible for initiating new :ref:`harvesting operations ` - in an automated fashion. Periodically, the scheduler goes through the list of existing harvesters, checking if it - is time to dispatch one of the harvesting operations mentioned in the next section. - - The scheduler's operation frequency is configurable by defining a ``HARVESTER_SCHEDULER_FREQUENCY_MINUTES`` setting - - the default is to trigger the scheduler every 30 seconds. - - .. note:: - - Since the harvesting scheduler only checks if there is work to do once every ``x`` seconds - (defaulting to 30 seconds, as mentioned above), there will usually be a delay between the time a harvesting - operation is supposed to be scheduled and the actual time when it is indeed scheduled. Moreover, the harvesting - scheduler is implemented as a celery task. This means that, if the celery worker is busy, that may also cause a - delay in scheduling harvesting operations, as the scheduler's celery task may not be triggered immediately. - - -Harvesting workflows -==================== - -There are two main possible harvesting workflows: - -#. :ref:`Continuous harvesting ` -#. :ref:`One-time harvesting ` - - -.. _continuous-harvesting-label: - -Continuous harvesting ---------------------- - - This workflow relies on the :ref:`harvesting scheduler ` in order to ensure harvested - resources are continuously kept up to date with their remote counterparts. - - #. User creates harvester and sets its ``scheduling_enabled`` attribute to ``True``; - #. When the time comes, the harvesting scheduler calls the - :ref:`update list of harvestable resources operation `. Alternatively, - the user may call this operation manually the first time. - #. When the previous operation is done, user goes through the list of generated - :ref:`harvestable resources ` and, for each relevant harvestable resource, sets - it ``should_be_harvested`` attribute to ``True``. Alternatively, if the harvester has its - ``harvest_new_resources_automatically`` attribute set to ``True``, the harvestable resources will already be - marked as to be harvested, without requiring manual user intervention; - #. When the time comes, the harvesting scheduler calls the - :ref:`perform harvesting operation `. This causes - the remote resources to be harvested. These now show up as resources on the local GeoNode. - - -.. _one-time-harvesting-label: - -One-time harvesting -------------------- - - This workflow is mostly executed manually by the user. - - #. User creates harvester and sets its ``scheduling_enabled`` attribute to ``False``; - #. User calls the :ref:`update list of harvestable resources operation `; - #. When the previous operation is done, user goes through the list of generated - :ref:`harvestable resources ` and, for each relevant harvestable resource, sets - it ``should_be_harvested`` attribute to ``True``; - #. User then proceeds to call the :ref:`perform harvesting operation `. This causes - the remote resources to be harvested. These now show up as resources on the local GeoNode. - - - -.. _harvesting-operations-label: - -Harvester operations -==================== - -Each GeoNode harvester is able to perform a finite set of operations. These can be performed either: - -#. In an **automated fashion**, being dispatched by the harvesting scheduler. Automated harvesting is only performed when - the corresponding :ref:`harvester ` has ``scheduling_enabled=True``; - -#. **On-demand**, by explicit request of the user. On-demand execution can be requested by one of two ways: - - #. By selecting the relevant harvester(s) in the ``Harvesting -> Harvesters`` section of the GeoNode admin area and - then selecting and running an action from the drop-down menu; - - #. By interacting with the GeoNode REST API. Harvester actions are requested by issuing ``HTTP PATCH`` requests to - the ``/api/v2/harvesters/{harvester-id}/`` endpoint. The payload of such requests must specify the corresponding - status. For example, by issuing a request like:: - - curl -X PATCH http:/localhost/api/v2/harvesters/1/ \ - -H "Content-Type: application/json" \ - -u "myuser:mypass" \ - --data '{"status": "updating-harvestable-resources"}' - - We are asking that the harvester's status be changed to ``updating-harvestable-resources``. If the server accepts - this request, then the :ref:`update list of harvestable resources operation ` - is triggered. - - .. note:: - - The server will not accept the API request if the harvester's current status is not ``ready``. - - -While performing an action, the harvester's ``status`` property transitions from ``ready`` to whatever action-related -status is appropriate (as indicated below). As the operation finishes execution, the harvester's status transitions -back to ``ready``. If the harvester has any status other than ``ready``, then it is currently busy. When a harvester -is busy it cannot execute other operations, you'll need to wait until the current operation finishes. - - -.. _check-remote-available-action-label: - -Check if the remote service is available operation --------------------------------------------------- - -This operation causes the harvester to perform a simple health check on the remote service, in order to check whether it -responds successfully. The response is stored in the harvester's ``remote_available`` property. This operation is -performed in the same process of the main GeoNode (*i.e.* it runs synchronously). - -When triggered, this operation causes the harvester's status to transition to ``checking-availability``. -As the operation finishes, the harvester's status transitions back to ``ready``. - -Invocation via the GeoNode admin is performed by selecting the -``Check availability of selected harvesters`` command. - -Invocation via the GeoNode REST API is performed by issuing an HTTP PATCH request with a payload that sets the -harvester status. - - -.. _update-harvestable-resources-action-label: - -Update the list of harvestable resources operation --------------------------------------------------- - -This operation causes the harvester to interact with the remote service in order to discover which resources are -available for being harvested. Existing remote resources are then saved as -:ref:`harvestable resources `. - -Since this operation can potentially take a long time to complete (as we don't know how may resources may exist on the -remote service), it is run using a background process. GeoNode creates a new :ref:`refresh session ` -and uses it to track the progress of this operation. - -When triggered, this operation causes the harvester's status to transition to ``updating-harvestable-resources``. -As the operation finishes, the harvester's status transitions back to ``ready``. - -Invocation via the GeoNode admin is performed by selecting the -``Update harvestable resources for selected harvesters`` command. - -Invocation via the GeoNode REST API is performed by issuing an HTTP PATCH request with a payload that sets the -harvester status. - - -.. _perform-harvesting-action-label: - -Perform harvesting operation ----------------------------- - -This operation causes the harvester to check which harvestable resources are currently marked as being harvestable -and then, for each one, harvest the resource from the remote server. - -Since this operation can potentially take a long time to complete (as we don't know how may resources may exist on the -remote service), it is run using a background process. GeoNode creates a new :ref:`harvesting session ` -and uses it to track the progress of this operation. - -When triggered, this operation causes the harvester's status to transition to ``harvesting-resources``. -As the operation finishes, the harvester's status transitions back to ``ready``. - -Invocation via the GeoNode admin is performed by selecting the ``Perform harvesting on selected harvesters`` command. - -Invocation via the GeoNode REST API is performed by issuing an HTTP PATCH request with a payload that sets the -harvester status. - - -.. _abort-refresh-action-label: - -Abort update of harvestable resources operation ------------------------------------------------- - -This operation causes the harvester to abort an on-going -:ref:`update list of harvestable resources operation `. - -When triggered, this operation causes the harvester's status to transition to ``aborting-update-harvestable-resources``. -As the operation finishes, the harvester's status transitions back to ``ready``. - -Invocation via the GeoNode admin is performed by selecting the -``Abort on-going update of harvestable resources for selected harvesters`` command. - -Invocation via the GeoNode REST API is performed by issuing an HTTP PATCH request with a payload that sets the -harvester status. - - -.. _abort-harvesting-action-label: - -Abort harvesting operation --------------------------- - -This operation causes the harvester to abort an on-going -:ref:`perform harvesting operation `. - -When triggered, this operation causes the harvester's status to transition to ``aborting-performing-harvesting``. -As the operation finishes, the harvester's status transitions back to ``ready``. - -Invocation via the GeoNode admin is performed by selecting the -``Abort on-going harvesting sessions for selected harvesters`` command. - -Invocation via the GeoNode REST API is performed by issuing an HTTP PATCH request with a payload that sets the -harvester status. - - -.. _reset-harvester-action-label: - -Reset harvester operation -------------------------- - -This operation causes the harvester's status to be reset back to ``ready``. It is mainly useful for troubleshooting -potential errors, in order to unlock harvesters that may get stuck in a non-actionable status when some unforeseen -error occurs. - -When triggered, this operation causes the harvester's status to transition to ``ready`` immediately. - -Invocation via the GeoNode admin is performed by selecting the ``Reset harvester status`` command. - -This operation cannot be called via the GeoNode API. - - -.. _standard-harvester-workers-label: - -Standard harvester workers -========================== - -.. note:: - Remember that, as stated above, a harvester worker is configured by means of setting the ``harvester_type`` and - ``harvester_type_specific_configuration`` attributes on the :ref:`harvester `. - - Moreover, the format of the ``harvester_type_specific_configuration`` attribute must be a JSON object. - - -.. _geonode-harvester-worker-label: - -GeoNode harvester worker ------------------------- - -This worker is able to harvest remote GeoNode deployments. In addition to creating local resources by retrieving -the remote metadata, this harvester is also able to copy remote datasets over to the local GeoNode. This means -that this harvester can even be used in order to generate replicated GeoNode instances. - -This harvester can be used by setting ``harvester_type=geonode.harvesting.harvesters.geonodeharvester.GeonodeUnifiedHarvesterWorker`` -in the harvester configuration. - -It recognizes the following ``harvester_type_specific_configuration`` parameters: - -harvest_datasets - Whether to harvest remote resources of type ``dataset`` or not. Acceptable values: ``true`` (the default) or ``false``. - -copy_datasets - Whether to copy remote resources of type ``dataset`` over to the local GeoNode. Acceptable values: ``true`` or ``false`` (the default). - -harvest_documents - Whether to harvest remote resources of type ``document`` or not. Acceptable values: ``true`` (the default) or ``false``. - -copy_documents - Whether to copy remote resources of type ``document`` over to the local GeoNode. Acceptable values: ``true`` or ``false`` (the default). - -resource_title_filter - A string that must be present in the remote resources' ``title`` in order for them to be acknowledged as - harvestable resources. This allows filtering out resources that are not relevant. Acceptable values: any - alphanumeric value. - - Example: setting this to a value of ``"water"`` would mean that the harvester would generate harvestable resources - for remote resources that are titled *water basins*, *Water territories*, etc. The harvester would not generate - harvestable resources for remote resources whose title does not contain the word *water*. - -start_date_filter - A string specifying a datetime that is used to filter out resources by their start_date. This is parsed with - :ref:`dateutil.parser.parse() `, - which means that it accepts many different formats (e.g. `2021-06-31T13:04:05Z`) - -end_date_filter - Similar to ``start_date_filter`` but uses resources' `end_date` as a filter parameter. - -keywords_filter - A list of keywords that are used to filter remote resources. - -categories_filter - A list of categories that are used to filter remote resources. - - -.. _wms-harvester-worker-label: - -WMS harvester worker --------------------- - -This worker is able to harvest from remote OGC WMS servers. - -This harvester can be used by setting ``harvester_type=geonode.harvesting.harvesters.wms.OgcWmsHarvester`` -in the harvester configuration. - -It recognizes the following ``harvester_type_specific_configuration`` parameters: - -dataset_title_filter - A string that is used to filter remote WMS layers by their ``title`` property. If a remote layer's title contains - the string defined by this parameter, then the layer is recognized by the harvester worker. - - -.. _arcgis-harvester-worker-label: - -ArcGIS REST Services harvester worker -------------------------------------- - -This worker is able to harvest from remote ArcGIS REST Services catalogs. - -This worker is able to recognize two types of ``remote_url``: - -#. URL of the ArcGIS REST services catalog. This URL usually ends in ``rest/services``. A catalog may expose several - different services. This harvester worker is able to descend into the available ArcGIS Rest services and retrieve - their respective resources. Example:: - - https://sampleserver6.arcgisonline.com/arcgis/rest/services - -#. URL of the ArcGIS REST services Service. This URL usually takes the form ``{base-url}/rest/services/{service-name}/{service-type}``. Example:: - - https://sampleserver6.arcgisonline.com/arcgis/rest/services/CharlotteLAS/ImageServer - -This harvester worker can be used by setting ``harvester_type=geonode.harvesting.harvesters.arcgis.ArcgisHarvesterWorker`` -in the harvester configuration. - -It recognizes the following ``harvester_type_specific_configuration`` parameters: - -harvest_map_services - Whether services of type `MapServer` ought to be harvested. Defaults to ``True``. - -harvest_image_services - Whether services of type `ImageServer` ought to be harvested. Defaults to ``True``. - -resource_name_filter - A string that is used to filter remote WMS layers by their ``title`` property. If a remote layer's name contains - the string defined by this parameter, then the layer is recognized by the harvester worker. - -service_names_filter - A list of names that are used to filter the remote ArcGIS catalog. - - -.. _creating-new-workers-label: - -Creating new harvesting workers -=============================== - -New harvesting workers can be created by writing classes derived from -:ref:`geonode.harvesting.harvesters.base.BaseGeonodeHarvesterWorker `. -This class defines an abstract interface that must be implemented. All methods decorated with ``abc.abstractmethod`` -must be implemented in the custom harvester worker class. Study the implementation of the standard GeoNode harvester worker -classes in order to gain insight on how to implement custom ones. - -After writing a custom harvester worker class, it can be added to the list of known harvester workers by defining -the ``HARVESTER_CLASSES`` GeoNode setting. This setting is a list of strings, containing the Python class path to each -harvester worker class. It has a default value of:: - - HARVESTER_CLASSES = [ - "geonode.harvesting.harvesters.geonodeharvester.GeonodeUnifiedHarvesterWorker", - "geonode.harvesting.harvesters.wms.OgcWmsHarvester", - "geonode.harvesting.harvesters.arcgis.ArcgisHarvesterWorker", - ] - -These are the standard harvester worker classes shipped by GeoNode. If this setting is defined, its value will simply -extend the default list. This means that it is not possible to disable the standard worker classes, only to add new ones. diff --git a/intermediate/index.rst b/intermediate/index.rst deleted file mode 100644 index 1a512a3a..00000000 --- a/intermediate/index.rst +++ /dev/null @@ -1,26 +0,0 @@ -.. _geonode_monitoring: - -Read-Only and Maintenance Mode -============================== - -.. toctree:: - :maxdepth: 3 - - modes/index - -Harvesting resources from remote services -========================================= - -.. toctree:: - :maxdepth: 3 - - harvesting/index - -GeoNode Backup and Restore -========================== - -.. toctree:: - :caption: GeoNode Backup and Restore - :maxdepth: 3 - - backup/index diff --git a/intermediate/modes/index.rst b/intermediate/modes/index.rst deleted file mode 100644 index 2f20d001..00000000 --- a/intermediate/modes/index.rst +++ /dev/null @@ -1,50 +0,0 @@ -=============================== -Read-Only and Maintenance Modes -=============================== - -Overview -======== - -GeoNode gives an option to operate in different modes, according to the needs and demands of the certain application system. - -Changing the currently used mode can be done in the admin panel by the user with super-user privileges, by modifying -``Configuration`` singleton model in the ``BASE`` application: - - .. figure:: img/configuration_admin_panel.png - :align: center - - *Configuration change in the admin panel* - -Read-Only Mode -============== - -Activating the Read-Only Mode (by setting ``Read only`` True in the ``Configuration``) activates a middleware rejecting all modifying requests -(POST/PUT/DELETE), with an exception for: - -- POST to login view -- POST to logout view -- POST to admin login view -- POST to admin logout view -- all requests to OWS endpoint -- all requests ordered by a super-user - -Additionally, all UI elements allowing modifying GeoNode's content are hidden, so e.g. the button "Upload Layer" is not rendered in the templates. - -In case a user tries to perform a forbidden request, they will be presented with a static HTML page informing them, the GeoNode is in the Read-Only -mode and this action is currently forbidden. - -Maintenance Mode -================ - -Activating the Maintenance Mode (by setting ``Maintenance`` True in the ``Configuration``) activates the highest level middleware -(the one executed as the first) rejecting all requests to the GeoNode instance, with an exception for: - -- POST to admin login view -- POST to admin logout view -- all requests ordered by a super-user - -In case a user tries to perform any request against the GeoNode (including GET requests), they will be presented with a static HTML page informing -them, the maintenance actions are taken on the GeoNode instance, and asking them to try again soon. - -The maintenance mode was implemented with a thought of the backup and restore procedures without a necessity to put down the instance, -but at the same time with a restriction of any outer interference. diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 00000000..2e1d4b72 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,15 @@ +site_name: GeoNode Documentation +docs_dir: docs + +nav: + - Home: index.md + - About: about/index.md + - Getting Started: start/index.md + - Using: usage/index.md + - Installing: install/index.md + - Configuring (Basic): basic/index.md + - Configuring (Intermediate): intermediate/index.md + - Configuring (Advanced): advanced/index.md + - Administering: admin/index.md + - Contributing: contribute/index.md + - Developing: devel/index.md diff --git a/start/index.rst b/start/index.rst deleted file mode 100644 index 77bfa0e9..00000000 --- a/start/index.rst +++ /dev/null @@ -1,137 +0,0 @@ -.. _geonode_basics: - -GeoNode Basics -============== - - .. figure:: img/geonode.png - :align: center - -is a platform for the management and publication of geospatial data. -It brings together mature open-source software projects under an easy to use interface. - - .. figure:: img/gn_simplified_architecture.png - :align: center - - *GeoNode simplified architecture* - -*With GeoNode, non-specialized users can share data and create interactive maps.* -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - - .. figure:: img/gn_is_made_for.png - :align: center - .. figure:: img/gn_publication_data.png - :align: center - .. figure:: img/gn_publication_data_2.png - :align: center - -Geospatial data storage -^^^^^^^^^^^^^^^^^^^^^^^ - -GeoNode allows users to upload vector data (currently shapefiles, json, csv, kml and kmz) and raster data in their original projections -using a web form. - -Vector data is converted into geospatial tables on a DB, satellite imagery and other kinds of raster data are retained as GeoTIFFs. - -Special importance is given to standard metadata formats like ISO 19139:2007 / ISO 19115 metadata standards. - -As soon as the upload is finished, the user can fill the resource metadata in order to make it suddenly available through the `CSW`_ (OGC Catalogue Service) -endpoints and APIs. - -Users may also upload a metadata XML document (ISO, FGDC, and Dublin Core format) to fill key GeoNode metadata elements automatically. - -Similarly, GeoNode provides a web based styler that lets the users to change the data portrayals and preview the changes at real time. - -.. _CSW: http://www.opengeospatial.org/standards/cat - - -Data mixing, maps creation -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Once the data has been uploaded, GeoNode lets the user search for it geographically or via keywords in order to create fancy maps. - -All the datasets are automatically re-projected to web Mercator for maps display, making it possible to use different popular base datasets, -like Open Street Map, Google Satellite or Bing datasets. - -Once the maps are saved, it is possible to embed them in any web page or get a PDF version for printing. - -GeoNode as a building block -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -A handful of other Open Source projects extend GeoNode's functionality by tapping into the re-usability of Django applications. - -Visit our gallery to see how the community uses GeoNode: `GeoNode Projects `_. - -The development community is very supportive of new projects and contributes ideas and guidance for newcomers. - -Convinced! Where do I sign? -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The next steps are: - -1. Make a ride on the :ref:`online_demo` - -2. Follow the :ref:`quick_setup_guide` in order to play with your own local instance and access all the admin functionalities - -3. Read the documentation starting from the :doc:`user guide ` to the :doc:`admin guide ` - -4. Subscribe to the `geonode-users`_ and/or `geonode-devel`_ mailing lists to join the community. - See also the section :ref:`get_in_touch` for more info. - -Thanks for your interest! - -.. _geonode-users: https://lists.osgeo.org/mailman/listinfo/geonode-users -.. _geonode-devel: https://lists.osgeo.org/mailman/listinfo/geonode-devel - -.. _supported_browsers: - -Supported Browsers -================== - -GeoNode is known to be working on all modern web browsers. - -This list includes (but is not limited to): - -- `Google Chrome `_. -- `Apple Safari `_. -- `Mozilla Firefox `_. -- `Microsoft Edge `_. - -.. note:: The vast majority of GeoNode developers prefer using Google Chrome. - -.. _online_demo: - -Online Demo -=========== - -.. note:: **Disclaimer** we do not guarantee for any data published on this Demo Site. Publish the data at your own risk. - Every dataset will be removed automatically every Sunday. If you find some dataset that shouldn't be there, please - write suddenly to developers and maintainers. - - See the section :ref:`get_in_touch` for details. - -A live demo of the latest stable build is available at http://stable.demo.geonode.org/. - - .. figure:: img/online_demo-001.png - :align: center - - *Online Demo @ master.demo.geonode.org* - -Anyone may sign up for a user account, upload and style data, create and share maps, and change permissions. - -Since it is a demo site, every sunday all the datasets will be wiped out. Users, passwords and groups will be preserved. - -It should hopefully allow you to easily and quickly make a tour of the main capabilities of GeoNode. - -.. warning:: This GeoNode instance is configured with standards settings and a very low security level. - This is a demo only not to be considered a really production ready system. - For a complete list of settings, refer to the section: :ref:`settings` - -.. _quick_setup_guide: - -Quick Installation Guide -======================== - -.. toctree:: - :maxdepth: 3 - - quick/index diff --git a/start/quick/index.rst b/start/quick/index.rst deleted file mode 100644 index 15e6f5e5..00000000 --- a/start/quick/index.rst +++ /dev/null @@ -1,72 +0,0 @@ -Introduction -^^^^^^^^^^^^ - -The following is a quick guide to get started with GeoNode in most common operating systems. - -.. note:: For a full setup and deployment, please refer to the :doc:`complete installation guides ` - -This is meant to be run on a fresh machine with no previously installed packages or GeoNode versions. - -.. warning:: The methods presented here are meant to be used for a limited internal demo only. - Before exposing your GeoNode instance to a public server, please read carefully the :doc:`hardening guide ` - -Recommended Minimum System Requirements -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -A definite specification of technical requirements is difficult to recommend. Accepted performance is highly subjective. Furthermore, the performance depends on factors such as concurrent users, records in the database or the network connectivity of your infrastructure. - -For deployment of GeoNode on a single server, the following are the *bare minimum* system requirements: - -* 8GB of RAM (16GB or more preferred for a production deployment). -* 2.2GHz processor with 4 cores. (Additional processing power may be required for multiple concurrent styling renderings) -* 30 GB software disk usage (Reserved to OS and source code only). -* Additional disk space for any data hosted with GeoNode, data stored on the DataBase and tiles cached with GeoWebCache. - For db, spatial data, cached tiles, and "scratch space" useful for administration, a decent baseline size for GeoNode deployments is between 50GB and 100GB. -* 64-bit hardware **strongly** recommended. - - -.. OSGEO Live CD -.. ^^^^^^^^^^^^^ -.. -.. .. figure:: img/osgeo_live.png -.. :align: center -.. -.. `OSGeoLive `_ is a self-contained bootable DVD, USB thumb drive or Virtual Machine based on Lubuntu, -.. that allows you to try a wide variety of open source geospatial software without installing anything. -.. -.. It is composed entirely of free software, allowing it to be freely distributed, duplicated and passed around. -.. -.. It provides pre-configured applications for a range of geospatial use cases, including storage, publishing, viewing, analysis and -.. manipulation of data. -.. -.. It also contains sample datasets and documentation. -.. -.. To try out the applications, simply: -.. -.. * Insert DVD or USB thumb drive in computer or virtual machine. -.. * Reboot computer. (verify boot device order if necessary) -.. * Press :guilabel:`Enter` to startup & login. -.. * Select and run applications from the :guilabel:`Geospatial` menu. -.. -.. `OSGeoLive `_ is an `OSGeo Foundation `_ project. -.. The `OSGeo Foundation `_ is a not-for-profit supporting Geospatial Open Source Software development, promotion and `education `_. - -Install via Docker -^^^^^^^^^^^^^^^^^^ - -`Docker `_ is a free software platform used for packaging software into standardized units for development, shipment and deployment. - - .. figure:: img/docker.png - .. note:: credits to Docker - -A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings. - -Docker containers running on a single machine share that machine's operating system kernel; they start instantly and use less compute and RAM. - -Containers can share a single kernel, and the only information that needs to be in a container image is the executable and its package dependencies, which never need to be installed on the host system. - -Multiple containers can run on the same machine and share the OS kernel with other containers, each running as isolated processes in user space. - -This documentation will refer to the usage of Docker on a Ubuntu host, but you can of course run Docker also on other Linux distributions, Windows and Mac. - -For the steps to setup Docker on Ubuntu you can jump ahead to :ref:`Ubuntu 22.04 ` diff --git a/usage/accounts_user_profile/new_account/index.rst b/usage/accounts_user_profile/new_account/index.rst deleted file mode 100644 index d278c9fd..00000000 --- a/usage/accounts_user_profile/new_account/index.rst +++ /dev/null @@ -1,41 +0,0 @@ -Creating a New Account -====================== - -To take full advantage of all the GeoNode features you need a user account. Follow these step to create a new one. - -#. From any page in the web interface, you will see a :guilabel:`Register` link. Click that link, and the register form will appear - - .. note:: The registrations in GeoNode must be open, in case you don't see the register link then it's not possible to register unless the administrator of the site does that for you. - - .. figure:: img/register_button.png - :align: center - - *Register Link* - -#. On the next page, fill out the form. Enter a username and password in the fields. Also, enter your email address for verification. - - .. figure:: img/register_form.png - :align: center - - *Registering for a new account* - -#. You will be automatically logged in and redirected to the Profile page. An email will be sent confirming that you have signed up. If no errors occur during the registration, the following alerts will appear on the screen: - - .. figure:: img/register_alerts.png - :align: center - - *Alerts* - -To log out click on the :guilabel:`Log out` link of the user menu. - -.. figure:: img/logout_link.png - :align: center - - *Logout link* - -You have to confirm this action as described in the picture below. - - .. figure:: img/confirm_logout.png - :align: center - - *Confirm Log out* diff --git a/usage/accounts_user_profile/updating_profile/associating_email.rst b/usage/accounts_user_profile/updating_profile/associating_email.rst deleted file mode 100644 index 95a3ab86..00000000 --- a/usage/accounts_user_profile/updating_profile/associating_email.rst +++ /dev/null @@ -1,25 +0,0 @@ -.. _associating_emails: - -Associating your Account with an e-mail -======================================= - -Your account is automatically associated with the e-mail that you used to register yourself on the platform. - -.. figure:: img/associate_email.png - :align: center - - *Accounts e-mail* - -By clicking on :guilabel:`Associated e-mails` of the *Profile* page (see :ref:`editing-profile`), you will have the possibility to fill up a new e-mail address. Type it in the :guilabel:`E-mail` input filed then click on :guilabel:`Add E-mail` to perform a new association. - -.. figure:: img/associate_new_email.png - :align: center - - *New e-mail association* - -You can make it primary if necessary, in order to receive the notification on this address. To do that, select the e-mail that you want, then click on :guilabel:`Make Primary`. - -.. figure:: img/primary_email.png - :align: center - - *Primary e-mail address* diff --git a/usage/accounts_user_profile/updating_profile/connecting_with_social.rst b/usage/accounts_user_profile/updating_profile/connecting_with_social.rst deleted file mode 100644 index 63670219..00000000 --- a/usage/accounts_user_profile/updating_profile/connecting_with_social.rst +++ /dev/null @@ -1,4 +0,0 @@ -Connecting your Account with Social Networks -============================================ - -You currently have no social network accounts connected to this account. diff --git a/usage/accounts_user_profile/updating_profile/editing_profile.rst b/usage/accounts_user_profile/updating_profile/editing_profile.rst deleted file mode 100644 index cad76c33..00000000 --- a/usage/accounts_user_profile/updating_profile/editing_profile.rst +++ /dev/null @@ -1,19 +0,0 @@ -Editing Profile Information -=========================== - -Your *Profile* contains personal information such as your address, your telephone number, your organization and so on but it is empty by default at the beginning. - -Through the :guilabel:`Edit profile` button of the *Profile* page (see :ref:`editing-profile`) you can set your details, including your avatar. - -.. figure:: img/user_profile_update_info.png - :align: center - - *Updating Profile information* - -| When finished, click :guilabel:`Update profile`. You will be redirected to the *Profile* page. -| A message will confirm the profile has been correctly updated. - -.. figure:: img/profile_update_confirm.png - :align: center - - *Updating Profile correctly finalized* diff --git a/usage/accounts_user_profile/updating_profile/index.rst b/usage/accounts_user_profile/updating_profile/index.rst deleted file mode 100644 index 8293505c..00000000 --- a/usage/accounts_user_profile/updating_profile/index.rst +++ /dev/null @@ -1,47 +0,0 @@ -.. _editing-profile: - -Updating the Profile -===================== - -Once having an account you can enrich your profile with useful information, you can also edit or delete the existing ones. -You can connect the account with your social network, associate many e-mail addresses to it and manage many options such as preferences about notifications. - -You can update these information anytime from your *Profile* page which is accessible from the user menu. - -So, click on your profile picture in the top right of the screen. A drop-down list will show. Click on :guilabel:`Profile` to enter the *Profile* settings page. - -.. figure:: img/profile_link.png - :align: center - - *Link to your profile* - -The *Profile* page looks like the one shown in the picture below. - -.. figure:: img/user_profile_page.jpg - :align: center - - *User profile page* - -Your personal information is shown under the username. At the bottom of the page are listed all the resources associated to your *Profile*, you can decide to view only datasets, maps ,documents, dashboard or geostory by clicking on the corresponding tab. - -Through the link ``User datasets WMS GetCapabilities document`` you can retrieve an XML document with the list of the available datasets. - -On the right side of the page there are many useful links to edit personal information to update your *Profile* settings and to get in touch with other GeoNode users. - -The :guilabel:`My Activities` link allows to see all your recent activities on GeoNode such as datasets uploading and maps creation. - -.. figure:: img/user_activities.jpg - :align: center - - *Recent activities* - - -All other links and their functionalities will be described in depth in the following sections. - -.. toctree:: - :maxdepth: 1 - - editing_profile - associating_email - managing_password - notification_settings diff --git a/usage/accounts_user_profile/updating_profile/managing_password.rst b/usage/accounts_user_profile/updating_profile/managing_password.rst deleted file mode 100644 index b65cccc3..00000000 --- a/usage/accounts_user_profile/updating_profile/managing_password.rst +++ /dev/null @@ -1,19 +0,0 @@ -Managing the Password -===================== - -To change your password, click on the :guilabel:`Set/Change password` link of the *Profile* page (see :ref:`editing-profile`). -You will be asked to enter your current password and the new one (two times). Click on :guilabel:`Change my password` to perform the change. - -.. figure:: img/change_password.png - :align: center - - *Change your password* - -If no errors occur you will see a confirmation message. - -.. figure:: img/change_password_confirm.png - :align: center - - *Change password confirmation* - -Next time you signin, you will have to use the new password. diff --git a/usage/accounts_user_profile/updating_profile/notification_settings.rst b/usage/accounts_user_profile/updating_profile/notification_settings.rst deleted file mode 100644 index bbe4d4f7..00000000 --- a/usage/accounts_user_profile/updating_profile/notification_settings.rst +++ /dev/null @@ -1,16 +0,0 @@ -Setting Notification Preferences -================================ - -By default GeoNode sends notifications to the users for events that they could be subscribed to such as a new dataset uploaded or a new rate added to a map. -You can adjust your notification settings by clicking on the :guilabel:`Notifications` link of the *Profile* page (see :ref:`editing-profile`). - -.. note:: Make sure to have a verified email address to which notices can be sent. If not see :ref:`associating_emails`. - -Now check/uncheck the notification types you wish to receive or not receive. It is possible to be notified for the events shown in the picture below. - -.. figure:: img/notifications_settings.png - :align: center -.. figure:: img/notifications_settings1.png - :align: center - - *Notifications settings* diff --git a/usage/dashboard/index.rst b/usage/dashboard/index.rst deleted file mode 100644 index 7007ea21..00000000 --- a/usage/dashboard/index.rst +++ /dev/null @@ -1,36 +0,0 @@ -.. _dashboard: - -Dashboard -========= - -Dashboard is a MapStore tool integrated in GeoNode that provides the user with a space to add many Widgets, such as charts, maps, tables, texts and counters, and can create connections between them in order to: - -* Provide an overview to better visualize a specific data context -* Interact spatially and analytically with the data by creating connections between widgets -* Perform analysis on involved data/layers - -To build a new Dashboard go to :guilabel:`Add Resource` option on the *All Resources* page and choose option *Create dashboard* or select :guilabel:`New` option on the *Dashboards* page - - .. figure:: img/create_dashboard_link.png - :align: center - - *Create dashboard from All Resources page* - -.. figure:: img/create_dashboard.png - :align: center - - *Create dashboard from Dashboards page* - -Now you landed on the Dashboard edition page that is composed of the following sections: - -.. figure:: https://mapstore.readthedocs.io/en/latest/user-guide/img/exploring-dashboards/dashboard-1.jpg - :align: center - - *New Dashboard Apps option* - -Further Reading ---------------- - -Follow the link below to get more detailed information about the usage of Dashboard. - -`Dashboard Documentation `_ diff --git a/usage/download_handlers/index.rst b/usage/download_handlers/index.rst deleted file mode 100644 index 4483ae58..00000000 --- a/usage/download_handlers/index.rst +++ /dev/null @@ -1,67 +0,0 @@ -.. _data: - -Download Handlers -================= - -With GeoNode 4.2.x has been introduced the concept of Download Handler and ofc GeoNode provides a default implementation of it which process the download via WPS - -Follow an example of how to create a custom download handler and to replace the default one or add an additional one. - - -Settings -======== - -DEFAULT_DATASET_DOWNLOAD_HANDLER --------------------------------- - - Default: ``geonode.layers.download_handler.DatasetDownloadHandler`` - - path to the download handler location - -DATASET_DOWNLOAD_HANDLERS -------------------------- - - Default: ``[]`` - - List of paths of the additional download handlers - - -CODE -==== - - -The default download handler is placed under the `geonode.layers` [package](https://github.com/GeoNode/geonode/blob/master/geonode/layers/download_handler.py) - -Follow an example of an basic class for define the download handler: - -```python - -class DatasetDownloadHandler: - def __str__(self): - return f"{self.__module__}.{self.__class__.__name__}" - - def __repr__(self): - return self.__str__() - - def __init__(self, request, resource_name) -> None: - self.request = request - self.resource_name = resource_name - self._resource = None - - def get_download_response(self): - """ - Main method used, this method should return the response object - """ - return response - @property - - def download_url(self): - """ - Used by the API, it should return the URL where the resource can be downloaded from - """ - return reverse("dataset_download", args=[resource.alternate]) - -``` - - -If you prefer to inherit from the already existing one, the response is generated in the `process_dowload` method diff --git a/usage/extra_metadata/api.rst b/usage/extra_metadata/api.rst deleted file mode 100644 index abb1ff9d..00000000 --- a/usage/extra_metadata/api.rst +++ /dev/null @@ -1,136 +0,0 @@ -.. _data: - - -Rest API -======== - -The `api/v2/resources` endpoint provide different ways to handle the metadata. - -**GET:** - -Get the metadata list of the selected resource - -.. code-block:: python - - URL: http://host/api/v2/resources/{resource_id}/extra_metadata - - Response: - - [ - { - "id": 1, - "field_name": "bike", - "field_label": "KTM", - "field_value": "ktm", - "filter_header": "Sports Parameters" - } - ] - - -**POST:** - -Adding new metadata to the resource - -.. code-block:: python - - URL: http://host/api/v2/resources/{resource_id}/extra_metadata - data = [ - { - "field_name": "bike", - "field_label": "KTM", - "field_value": "ktm", - "filter_header": "Sports Parameters" - } - ] - - Response: - status_code: 201 - response json: List of the available metadata for the resource - [ - { - "id": 1, - "field_name": "bike", - "field_label": "KTM", - "field_value": "ktm", - "filter_header": "Sports Parameters" - } - ] - -**PUT:** - -Update specific metadata for the selected resource. In this case the metadata **ID** is required to perform the update - -.. code-block:: python - - http://host/api/v2/resources/{resource_id}/extra_metadata - payload: - [ - { - "id": 1, - "field_name": "bike", - "field_label": "KTM - sport", <- this value need to be updated - "field_value": "ktm", - "filter_header": "Sports Parameters" - } - ] - - Response: - status_code: 200 - response: the available payload for the selected resource - [ - { - "id": 1, - "field_name": "bike", - "field_label": "KTM - sport", - "field_value": "ktm", - "filter_header": "Sports Parameters" - } - ] - - -**DELETE:** - -Delete the metadata for a given resource by *ID*. - -.. code-block:: python - - http://host/api/v2/resources/{resource_id}/extra_metadata - payload: list of ID to be deleted - [ - 1, 2, 3, 4, 5 - ] - - - - Response: - status_code: 200 - response: List of the available metadata - - [] - - -**API search** - -Is possible to search for resources with specific metadata. This feature is available for both API v1 and API v2 - -APIv1: - -To perform the search is enough to add as query parameters the field of the metadata payload. - -Assuming that the payload is the same as the example above, the URL could be something like the following: - -`http://host/api/base/?metadata__field_category=bike` - -In this way, we can retrieve all the resources that have at least 1 metadata with the `field_category = 'bike'` - - -APIv2 - -For the API v2 is a bit different since the library doesnt have a support for the JSON field. - -To reproduce the same search above, we need to call a URL like the following one: - -`http://localhost:8000/api/v2/resources?filter{metadata.metadata.icontains}=%22field_category%22:%20%22bike%22` - -In this way, we can retrieve all the resources that have at least 1 metadata with the `field_category = 'bike'` - diff --git a/usage/extra_metadata/edit.rst b/usage/extra_metadata/edit.rst deleted file mode 100644 index 128b34fe..00000000 --- a/usage/extra_metadata/edit.rst +++ /dev/null @@ -1,86 +0,0 @@ -.. _data: - -Metadata manipulation -===================== - -There are two possible ways to manipulate extra metadata in geonode: - -- via Metadata Editor (Wizard and advanced) -- via Rest API - - -Metadata Editor (wizard/advanced): ----------------------------------- - -The metadata section is placed under the OPTIONAL METADATA section available for all the GeoNode resources. - -The metadata must follow two specific rules to save to the resource: - -- Must always be a list of JSON. This permits to add of more than one metadata for each resource -- The JSON must follow the schema defined in the `settings.py` for the selected resource. - -For example, for my document resource, I can have something like the following: - -.. figure:: img/wizard.png - :align: center - - *Advanced edit wizard menù* - -After pressing the save button, the system will perform the following checks: - -- Check if the text provided is a valid JSON. In case of wrong format input, the following error is shown: - -.. figure:: img/invalid_json.png - :align: center - - *invalid JSON error* - -- Check if the metadata schema is provided for the resource if not will raise the following error - -.. figure:: img/missing_schema.png - :align: center - - *missing schema error* - -- Check if the metadata schema is coherent with the schema defined in the settings. In case of wrong format input, the error will print the missing JSON keys - -.. figure:: img/invalid_schema.png - :align: center - - *invalid schema error* - -Facet Filtering ---------------- - -Automatically the web interface will create dynamically the facets if there is at least 1 metadata defined for the resource. - -Suppose that a resource heve the following metadata: - -.. code-block:: json - - [ - { - "field_name": "bike", - "field_label": "KTM", - "field_value": "ktm", - "filter_header": "Sports Parameters" - }, - { - "field_name": "mountain", - "field_label": "Monte bianco", - "field_value": "monte_bianco", - "filter_header": "Italian Mountains" - } - ] - -By default GeoNode will convert this metadata info in facets available for the resource - -The facet will convert: -- `filter_header`: used as the header filter -- `field_value`: used to perform the search -- `field_name`: used for calculate the unique values (along with `field_value`) - -After says that, the facet will be the follow: - -.. figure:: img/facet.png - :align: center diff --git a/usage/extra_metadata/settings.rst b/usage/extra_metadata/settings.rst deleted file mode 100644 index 51f45aa5..00000000 --- a/usage/extra_metadata/settings.rst +++ /dev/null @@ -1,40 +0,0 @@ - -Settings -======== - -Three main settings control the extra metadata field: - -DEFAULT_EXTRA_METADATA_SCHEMA: define the schema used to store the metadata - -- `id`: (optional int): the identifier of the metadata. Optional for creation, required in the Upgrade phase -- `filter_header`: (required object): Can be any type, is used to generate the facet filter header. Is also an identifier. -- `field_name`: (required object): name of the metadata field -- `field_label`: (required object): a verbose string of the name. Is used as a label in the facet filters. -- `field_value`: (required object): metadata values - - An example of metadata that can be ingested is the following: - - .. code-block:: json - - [ - { - "filter_header": "Bike Brand", - "field_name": "name", - "field_label": "Bike Name", - "field_value": "KTM", - }, - { - "filter_header": "Bike Brand", - "field_name": "name", - "field_label": "Bike Name", - "field_value": "Bianchi", - } - ] - - -The above schema is valid by using the `schema` - - -CUSTOM_METADATA_SCHEMA: environment variable used to inject additional schema to the default one. Helpful for third-party libraries - -EXTRA_METADATA_SCHEMA: used to get the expected metadata schema for each resource_type. diff --git a/usage/geostory/index.rst b/usage/geostory/index.rst deleted file mode 100644 index 6a54ae35..00000000 --- a/usage/geostory/index.rst +++ /dev/null @@ -1,80 +0,0 @@ -.. _geostory: - -GeoStory -======== - -GeoStory is a MapStore tool integrated in GeoNode that provides the user a way to create inspiring and immersive stories by combining text, interactive maps, and other multimedia content like images and video or other third party contents. -Through this tool you can simply tell your stories on the web and then publish and share them with different groups of GeoNode users or make them public to everyone around the world. - -To build a new GeoStory go to :guilabel:`Add Resource` option on the *All Resources* page and choose option *Create geostory* or select :guilabel:`New` option on the *Geostories* page - - .. figure:: img/create_geostory_link.png - :align: center - - *Create geostory from All Resources page* - -.. figure:: img/create_geostory.png - :align: center - - *Create geostory from Geostories page* - -Now you landed on the GeoStory edition page that is composed of the following sections: - -.. figure:: https://mapstore.readthedocs.io/en/latest/user-guide/img/exploring-stories/story-workspace.jpg - :align: center - - *New GeoStory Apps option* - -.. |add_section| image:: https://mapstore.readthedocs.io/en/latest/user-guide/img/button/add-section.jpg - :width: 30px - :height: 30px - :align: middle - -| The GeoStory content is organized in Sections, that can be added with the |add_section| button in the *Container* area. In particular, the user can add to the story the following kind of sections: - -.. figure:: https://mapstore.readthedocs.io/en/latest/user-guide/img/exploring-stories/sections.jpg - :align: center - - *GeoStory Sections Types* - -For more information on these specific section please follow the official MapStore documentation: - -* `Title Section `_ -* `Banner Section `_ -* `Paragraph Section `_ -* `Immersive Section `_ -* `Media Section `_ -* `Web Page Section `_ - -Add GeoNode content to GeoStory -------------------------------- - -With GeoNode you can add content to your GeoStory using internal GeoNode documents and maps as well external sources. -This ability to add internal GeoNode content makes the GeoStory creation a very usefull feature. - -.. |edit_mode| image:: https://mapstore.readthedocs.io/en/latest/user-guide/img/button/edit-icon-1.jpg - :width: 30px - :height: 30px - :align: middle - -| To add GeoNode content to your GeoStory use the |edit_mode| button on top of your GeoStory section. - -From here you can add :guilabel:`Images`, :guilabel:`Videos` and :guilabel:`Maps`. -To enable GeoNode internal catalog, on :guilabel:`Services` dropdown choose :guilabel:`GeoNode` as shown in picture down. -On the left you get a list of media documents available with a complementary text filter feature on top. - -.. figure:: img/add_media.png - :align: center - - *Add Media to GeoStory* - -To save your GeoStory, on the top your Geostory content choose :guilabel:`Save` and then :guilabel:`Save as...` - -Now your GeoStory can be shared with everyone! - -Further Reading ---------------- - -Follow the link below to get more detailed information about the usage of GeoStory. - -`GeoStory Documentation `_ diff --git a/usage/index.rst b/usage/index.rst deleted file mode 100644 index 31a71067..00000000 --- a/usage/index.rst +++ /dev/null @@ -1,18 +0,0 @@ -GeoNode Users Guide -=================== - -.. toctree:: - :maxdepth: 2 - - accounts_user_profile/index - users_groups/index - data/index - managing_documents/index - managing_datasets/index - managing_maps/index - dashboard/index - geostory/index - publishing_data/index - other_apps/index - extra_metadata/index - download_handlers/index diff --git a/usage/managing_datasets/dataset_download.rst b/usage/managing_datasets/dataset_download.rst deleted file mode 100644 index f62c1ad3..00000000 --- a/usage/managing_datasets/dataset_download.rst +++ /dev/null @@ -1,97 +0,0 @@ - -.. _dataset-download: - -Downloading Datasets -================== - -At the top of the *Dataset Menu* there is an :guilabel:`Download` link where it is possible to download: - -* Dataset - -* ISO Metadata - -* Dublin Core Metadata - -Clicking the :guilabel:`ISO Metadata` or the :guilabel:`Dublin Core Metadata` exports the layer metadata in `xml` format. - -.. figure:: img/export_metadata.png - :align: center - - *Downloading Metadata* - -Open the :guilabel:`Dataset` option you will be able to select from a list of options of the supported export file formats. - -.. figure:: img/export_dataset_form.png - :align: center - - *Downloading Datasets* - -As shown in the image above, GeoNode allows you to download a subset of data. Click on :guilabel:`Download filtered dataset` to download filtered data. - -On clicking Export, the file is prepared and a notification is showed when file is ready - -.. figure:: img/export_notification.png - :align: center - - *Export Ready* - -To download the file to your machine, click on the export dataset icon. This opens the prepared export files and you can save the files on your by clicking on the save icon on each item. - -.. figure:: img/export_icon.png - :align: center - - *Export Results Icon * - -.. figure:: img/export_list.png - :align: center - - -Printing --------------- - -| The `MapStore `_ based map viewer of GeoNode allows you to print the current view with a customizable layout. -| Click the :guilabel:`Print` option from the *Menu*, the **Printing Window** will open. - -.. figure:: img/printing_window.png - :align: center - - *The Printing Window* - -From this window you can: - -* enter *Title* and *Description*; -* choose the *Resolution* in dpi; -* select the format -* select the coordinate -* add the scale -* add grid with label -* customize the *Layout* - - + the *Sheet size* (A3, A4); - + if include the legend or not; - + if to put the legend in a separate page; - + the page *Orientation* (Landscape or Portrait); - -* customize the *Legend* - - + the *Label Font*; - + the *Font Size*; - + the *Font Emphasis* (bold, italic); - + if *Force Labels*; - + if use *Anti Aliasing Font*; - + the *Icon Size*; - + the *Legend Resolution* in dpi. - -To print the view click on :guilabel:`Print`. - - -Performing Measurements ------------------------ - -Click on the :guilabel:`Measure` option of the *Menu* to perform a measurement. -As you can see in the picture below, this tool allows you to measure *Distances*, *Areas* and the *Bearing* of lines. - -.. figure:: img/measuring.png - :align: center - - *The Measure Tool* diff --git a/usage/managing_datasets/dataset_editing.rst b/usage/managing_datasets/dataset_editing.rst deleted file mode 100644 index 65f32128..00000000 --- a/usage/managing_datasets/dataset_editing.rst +++ /dev/null @@ -1,106 +0,0 @@ -.. _dataset-editing: - -Dataset Editing -=============== - -The :guilabel:`Edit` link in the menu of the *Dataset Page* opens a list of options like ones shown in the picture below. - -.. figure:: img/dataset_editing_link.png - :align: center - - *Dataset Editing Link* - -In that options list, you can see three options listed as: - -1. *Edit Data* -2. *Edit Styles* -3. *Edit Metadata* -4. *Upload Style* -5. *Upload Metadata* - - -In this section you will learn how to edit a *Dataset*, and its data. See :ref:`dataset-metadata` to learn how to explore the dataset *Metadata*, how to upload and edit them. The *Styles* will be covered in a dedicated section, see :ref:`dataset-style`. - -.. _dataset-data-editing: - -Editing the Dataset Data ------------------------- - -The :guilabel:`Edit data` link of the *Dataset Editing* options opens the *Dataset* within a *Map*. - -.. figure:: img/editing_dataset_data.png - :align: center - - *Editing the Dataset Data* - -The *Attribute Table* panel of the *Dataset* will automatically appear at the bottom of the *Map*. In that panel all the features are listed. For each feature you can zoom to its extent by clicking on the corresponding *magnifying glass* icon |magnifying_glass_icon| at the beginning of the row, you can also observe which values the feature assumes for each attribute. - -.. |magnifying_glass_icon| image:: img/magnifying_glass_icon.png - :width: 30px - :height: 30px - :align: middle - -Click the *Edit Mode* |edit_mode_button| button to start an editing session. - -.. |edit_mode_button| image:: img/edit_mode_button.png - :width: 30px - :height: 30px - :align: middle - -Now you can: - -* *Add new Features* - - Through the *Add New Feature* button |add_new_feature_button| it is possible to set up a new feature for your dataset. - Fill the attributes fields and click |save_changes_button| to save your change. - Your new feature doesn't have a shape yet, click on |add_shape_to_geometry_button| to draw its shape directly on the *Map* then click on |save_changes_button| to save it. - - .. |add_new_feature_button| image:: img/add_new_feature_button.png - :width: 30px - :height: 30px - :align: middle - - .. |save_changes_button| image:: img/save_changes_button.png - :width: 30px - :height: 30px - :align: middle - - .. |add_shape_to_geometry_button| image:: img/add_shape_to_geometry_button.png - :width: 30px - :height: 30px - :align: middle - - .. note:: When your new feature has a multi-vertex shape you have to double-click the last vertex to finish the drawing. - -.. figure:: img/create_new_feature.png - :align: center - - *Create New Feature* - -* *Delete Features* - - If you want to delete a feature you have to select it on the *Attribute Table* and click on |delete_feature_button|. - - .. |delete_feature_button| image:: img/delete_feature_button.png - :width: 30px - :height: 30px - :align: middle - -* *Change the Feature Shape* - - You can edit the shape of an existing geometry dragging its vertices with the mouse. A blue circle lets you know what vertex you are moving. - - Features can have *multipart shapes*. You can add parts to the shape when editing it. - -* *Change the Feature Attributes* - - When you are in *Edit Mode* you can also edit the attributes values changing them directly in the corresponding text fields. You can achieve this by going into the edit mode and double click in the values. - -Once you have finished you can end the *Editing Session* by clicking on the |end_editing_session_button|. - - .. |end_editing_session_button| image:: img/end_editing_session_button.png - :width: 30px - :height: 30px - :align: middle - -By default the GeoNode map viewer is `MapStore `_ based, see the `MapStore Documentation `_ for further information. diff --git a/usage/managing_datasets/dataset_info.rst b/usage/managing_datasets/dataset_info.rst deleted file mode 100644 index 3ba9d70c..00000000 --- a/usage/managing_datasets/dataset_info.rst +++ /dev/null @@ -1,73 +0,0 @@ -.. _dataset-info: - -Dataset Information -================= - -From the *Dataset Search Page* (see :ref:`finding-data`) you can select the dataset you are interested in and click on its name to see an overview about it. - -.. figure:: img/dataset_overview.png - :align: center - -The information panel reports: - -* The *Info* tab is active by default. This tab section shows some dataset metadata such as its title, the abstract, date of publication etc. The metadata also indicates the dataset owner, what are the topic categories the dataset belongs to and which regions are affected. - -.. figure:: img/dataset_info.png - :align: center - - *Dataset Info tab* - -* The *Location* tab shows the spacial extent of the dataset. - -.. figure:: img/dataset_location.png - :align: center - - *Dataset Location tab* - -By clicking on the copy icons you have a copy of the current *Bounding Box* or the *Center* in the clipboard which once pasted will be a WKT text. - -.. figure:: img/copy_locations_dataset.png - :align: center - - *Bounding Box and Center* - -* The *Attributes* tab shows the data structure behind the dataset. All the attributes are listed and for each of them some statistics (e.g. the range of values) are estimated (if possible). - -.. figure:: img/dataset_attributes_tab.png - :align: center - - *Dataset Attributes tab* - -* The *Linked Resources* tab shows the resources, that can be *Documents*, *Maps*, *GeoStories* and *Dashboards*, linked to the dataset. - -.. figure:: img/dataset_linked_resources.png - :align: center - - *Dataset Linked Resources tab* - -.. note:: When a resource is `linked from` the dataset the link is shown under the *Linked by* list while when a resource is `linked to` the dataset the link is shown under the *Used by* list - -From the right toolbar on the **Info** panel it is possible: - -.. figure:: img/dataset_info_toolbar.png - :align: center - - *Dataset Info toolbar* - -* Add dataset in your *Favorites* (see :ref:`editing-profile`), clicking on star icon. - -* Directly *Download* the dataset oh `ShapeFile` format - -* *Share* the dataset url - -* Copy the *WMS Capabilities url* - -* Change the dataset *Title* by clicking on it - -You can access the dataset details page by clicking on *View dataset* in the overview panel. -That page looks like the one shown in the picture below. - -.. figure:: img/dataset_detail.png - :align: center - - *Dataset page* \ No newline at end of file diff --git a/usage/managing_datasets/dataset_metadata.rst b/usage/managing_datasets/dataset_metadata.rst deleted file mode 100644 index df857d55..00000000 --- a/usage/managing_datasets/dataset_metadata.rst +++ /dev/null @@ -1,144 +0,0 @@ -.. _dataset-metadata: - -Datasets Metadata -=============== - -In GeoNode special importance is given to *Metadata* and their standard formats. - -Editing Metadata ---------------- - -| Metadata contains all the information related to the dataset. They provide essential information for its identification and its comprehension. Metadata also make the dataset more easily retrievable through search by other users. -| The *Metadata* of a dataset can be changed through a *Edit Metadata* form which involves four steps, one for each type of metadata considered: - -You can open the *Metadata* form of a *Dataset* by clicking the :guilabel:`Edit Metadata` link from the :guilabel:`Edit` options on the *Dataset Page*. - -* **Basic Metadata** - - The first two steps are mandatory (no datasets will be published if the required information are not provided) whereas the last two are optional. - - .. figure:: img/basic_dataset_metadata.png - :align: center - - *Basic Dataset Metadata* - - In the first step the system asks you to insert the following metadata: - - * *Thumbnail* of the dataset (click :guilabel:`Edit` to change it); - * *Title* of the dataset, which should be clear and understandable; - * *Abstract*; brief narrative summary of the content of the dataset - - .. note:: The *Abstract* panel allows you to insert HTML code through a *wysiwyg* text editor - - * *Creation/Publication/Revision Dates* which define the time period that is covered by the dataset; - * *Keywords*, which should be chosen within the available list. The contributor search for available keywords by clicking on the searching bar, or on the folder logo representing, or by entering the first letters of the desired word; - * *Category* which the dataset belongs to; - * *Group* which the dataset is linked to. - - -* **Location and Licenses** - - .. figure:: img/location_licenses_dataset_metadata.png - :align: center - - *Location and Licenses Metadata for Datasets* - - The following list shows what kinds of metadata you are required to enter (see also the picture below): - - * *Language* of the dataset; - * *License* of the dataset; - * *DOI* of the dataset; if available, this represents the `Digital Object Identifier `_ of the resource - * *Attribution* of the dataset; authority or function assigned, as to a ruler, legislative assembly, delegate, or the like - * *Regions*, which informs on the spatial extent covered by the dataset. Proposed extents cover the following scales: global, continental, regional, national; - * *Data Quality statement* (general explanation of the data producer's knowledge about the lineage of a dataset); - * Potential *Restrictions* on dataset sharing. - - .. note:: The *Data Quality statement* and *Restrictions* panels allow you to insert HTML code through a *wysiwyg* text editor - - -* **Optional Metadata** - - .. figure:: img/optional_dataset_metadata.png - :align: center - - *Optional Dtaset Metadata* - - Complementary information are: - - * *Edition* to indicate the reference or the source of the dataset; - * *Purpose* of the dataset and its objectives; - * *Supplemental information* that can provide a better understanding of the uploaded dataset; - * *Maintenance frequency* of the dataset; - * users who are *Responsible* for the dataset, its *Owner*, and the *Author* of its metadata; - * *Spatial representation type* used. - * *Related resources* to link one or multiple resources to the document. These will be visible inside the :ref:`dataset-info` panel - - .. note:: The *Purpose* and *Supplemental information* panels allow you to insert HTML code through a *wysiwyg* text editor - - -* **Dataset Attributes** - - .. figure:: img/dataset_attributes_dataset_metadata.png - :align: center - - *Dataset Attributes Metadata for Dataset* - - At this step you can enrich the dataset attributes with useful information like the following: - - * The *Label* displayed - * A detailed *Description* - * The *Display Order* - * The *Display Type*; the default value is *Label*, which means that the value of the attribute will be rendered as a plain text. - There's the possibility to instruct GeoNode to threat the values as different media-types. As an instance, if the values of the - selected attribute will contain image urls, by selecting the ``IMAGE`` *Display Type* you will allow GeoNode to render the image - directly when querying the dataset from the maps. The same for ``VIDEO``, ``AUDIO`` or ``IFRAME`` mime types. - * The *Visibile* flag; allows you to instruct GeoNode wether or not hiding an attribute from the *Get Feature Type* outcomes - - - It is possible to define a completely custom ``HTML`` template for the *Get Feature Type* outcome. That is possible by enabling the *Use a custom template* flag as shown in the figure below. - - - .. figure:: img/dataset_attributes_dataset_metadata_custom_ft.png - :align: center - - *Use a custom template* - - By using the keywork ``${properties.``, you can tell to GeoNode to render the actual value of the attribute on the map. - - - As an instance, the example below - - .. figure:: img/dataset_attributes_dataset_metadata_custom_ft_html.png - :align: center - - *Use a custom template: HTML* - - - Will render an ``HTML Table`` along with values as shown here below - - .. figure:: img/dataset_attributes_dataset_metadata_custom_ft_outcome.png - :align: center - - *Use a custom template: Get Feature Info outcome* - -Use :guilabel:`next >>` or :guilabel:`<< back` to navigate through those steps. Once you have finished click on :guilabel:`Update`. - -Some metadata are mandatory, if you miss any of that metadata the *Completeness* bar shows you a red message like the one in the picture below. - -.. figure:: img/completeness_progress_bar.png - :align: center - :width: 200px - - *Completeness Progress Bar* - -Metadata Advanced Editing -------------------------- - -In the *Edit Metadata* page the :guilabel:`Advanced Metadata` button is also available. - -.. figure:: img/advanced_edit_button.png - :align: center - - *The Advanced Metadata button* - -Click on it to display the *Metadata Advanced Editing Page*. That page allows you to edit all the dataset metadata described in the previous paragraph. Once you have finished to edit them click on :guilabel:`Update` to save your changes. diff --git a/usage/managing_datasets/dataset_permissions.rst b/usage/managing_datasets/dataset_permissions.rst deleted file mode 100644 index 0d4dff3c..00000000 --- a/usage/managing_datasets/dataset_permissions.rst +++ /dev/null @@ -1,29 +0,0 @@ -.. _dataset-permissions: - -Share Options -============= - -In order to modify the Dataset *Share options* settings, on the detail page of the dataset click the :guilabel:`Share` link in the :guilabel:`Resource` menu. By default only owners can edit and manage datasets, anyone can view them. - -.. figure:: img/change_dataset_permissions.png - :align: center - - *Change Dataset Share options* - -Through the *Share options Settings Panel* you can add or remove options for users and groups. The picture below shows an example. - -.. figure:: img/dataset_permissions_settings.png - :align: center - - *Dataset Share options settings for users and groups* - -You can set the following share options: - -* *View* (allows to view the dataset). -* *Download* (allows to view and download the dataset). -* *Edit* (allows to change the dataset metadata,change attributes and properties of the datasets features and change the dataset style). -* *Manage* (allows to update, delete, change share options, publish and unpublish the dataset). - -.. warning:: When assigning options to a group, all the group members will have those options. Be careful in case of editing them. - -Click on :guilabel:`Save` link in the menu to save these settings. diff --git a/usage/managing_datasets/dataset_styling.rst b/usage/managing_datasets/dataset_styling.rst deleted file mode 100644 index 4afa0378..00000000 --- a/usage/managing_datasets/dataset_styling.rst +++ /dev/null @@ -1,79 +0,0 @@ -.. _dataset-style: - -Dataset Styling -============= - -Maps are helpful because they allow you gain a deeper understanding of your data by allowing you to visualize it in many different ways. -So you can tell different stories depending on how the data is presented. -For any given data or dataset, you should explore different styling options and choose the best style for that. - -In GeoNode each dataset has one style refered to as a *Default Style* which is determined by the nature of the data you're mapping. When uploading a new dataset (see :ref:`uploading-datasets`) a new default style will be associated to it. - -Editing the Dataset Style ------------------------ - -In order to edit a dataset style, open the *Dataset Page* (see :ref:`dataset-info`) and click on :guilabel:`Edit`. Then click the :guilabel:`Edit Style` link in the *options* (see the picture below). - -.. figure:: img/edit_style_button.png - :align: center - - *Edit Styles button* - -The *Dataset* will open in a new *Map*. The *Styles Panel* will show you all the default style for the dataset and some useful tools. By default, the style will be showm in a text editor form. - -.. figure:: img/styles_map_panel.png - :align: center - - *The Styles Panel in the Map* - -You could continue to change the style with the text editor or switch to a visualized editor using the :guilabel:`Visual editor` above the text editor. - -The visual editor looks like this - -.. figure:: img/visual_styles_map_panel.png - :align: center - - *Visual Styles Panel in the Map* - -You can then edit the style by clicking on each attribute of the style. - -Creating Style Rules -------------------- - -In order to create a new rule, Three options are provided. - -1. Mark rule -2. Icon rule -3. text rule - -Click on any of the buttons below to create a type of style you want. - - .. figure:: img/add_style_buttons.png - :align: center - - *Create new rule buttons* - -The rule you have chosen is now added on the top with default attributes which you can edit to your preference*. - -.. figure:: img/new_rule.png - :align: center - - *New style* - -If the rule has errors, the following will be shown. - -.. figure:: img/edit_style_error.png - :align: center - - *Rule Error* - - -You can switch the rule ordering by dragging it to the top or bottom depending on your preference. - - -It would be nice to change the style in order to decrease the opacity of the filling color as well as to reduce the lines width. -The embedded `MapStore `_ makes available a powerful *Style Editor* to accomplish that tasks. In the next paragraph we will explain how. - -Each rule has a delete icon on the top right which you can use to remove the rule on the style. - -Click on :guilabel:`Save` on the top to save your changes. diff --git a/usage/managing_datasets/index.rst b/usage/managing_datasets/index.rst deleted file mode 100644 index 6017c300..00000000 --- a/usage/managing_datasets/index.rst +++ /dev/null @@ -1,19 +0,0 @@ -Managing Datasets -=============== - -| Datasets are published resources representing raster o vector spatial data sources. Datasets can also be associated with metadata, ratings, and comments. -| In this section, you will learn how to create a new dataset by uploading a local data set, add dataset info, change the style of the dataset, and share the results. - -.. toctree:: - :maxdepth: 1 - - uploading_datasets - new_dataset_from_scratch - using_remote_services - dataset_permissions - dataset_info - dataset_download - dataset_editing - dataset_metadata - dataset_styling - dataset_advanced diff --git a/usage/managing_datasets/new_dataset_from_scratch.rst b/usage/managing_datasets/new_dataset_from_scratch.rst deleted file mode 100644 index 397c2c4d..00000000 --- a/usage/managing_datasets/new_dataset_from_scratch.rst +++ /dev/null @@ -1,30 +0,0 @@ -Creating a Dataset from scratch -============================= - -An interesting tool that GeoNode makes available to you is the *Create dataset*. It allows you to create a new vector dataset from scratch. The *Dataset Creation Form* is reachable through the :guilabel:`Create dataset` link shown in the picture below. - -.. figure:: img/create_dataset_link.png - :align: center - - *Create dataset link* - -In order to create the new Dataset you have to fill out the required fields: - -* *Name* -* *Title* -* *Geometry type* - - .. figure:: img/geometry_types.png - :align: center - - *Geometry types* - -Usually the datasets features should have some *Attributes* that enrich the amount of information associated with each of them. Through the :guilabel:`Add Attribute` button you can add new attributes. - -.. figure:: img/new_dataset_from_scratch.png - :align: center - - *New Dataset creation from scratch* - - -Once the form has been filled out, click on :guilabel:`Create`. You will be redirected to the *Dataset Page* (see :ref:`dataset-info`). Now your Dataset is created but is still empty, no features have been added yet. See the :ref:`dataset-editing` section to learn how to add new features. diff --git a/usage/managing_datasets/uploading_datasets.rst b/usage/managing_datasets/uploading_datasets.rst deleted file mode 100644 index 4df9751f..00000000 --- a/usage/managing_datasets/uploading_datasets.rst +++ /dev/null @@ -1,64 +0,0 @@ -.. _uploading-datasets: - -Datasets Uploading -================ - -| The most important resource type in GeoNode is the *Dataset*. A dataset represents spatial information so it can be displayed inside a map. -| To better understand what we are talking about lets upload your first dataset. - -It is possible to upload a *Datasets* in two ways: - -- From the :guilabel:`All Resources` page, by clicking *Add Resource* which displays a list including `Upload dataset` link: - - .. figure:: img/upload_dataset_link.png - :align: center - - *Dataset Upload from All Resources page* - -- From the :guilabel:`Datasets` page, by clicking on *New* which displays a list including `Upload dataset` link: - -.. figure:: img/upload_dataset.png - :align: center - - *Dataset Upload from Datasets page* - -The *Datasets Uploading* page looks like the one in the picture below. - -.. figure:: img/dataset_upload_process.png - :align: center - - *The Datasets Uploading page* - - -| Through the :guilabel:`Select files` button you can select files from your disk, make sure they are valid raster or vector spatial data, then you can click to :guilabel:`Upload` button. -A progress bar shows the operation made during the dataset upload and alerts you when the process is over. - -.. figure:: img/dataset_upload_confirm.png - :align: center - - *Upload dataset* - - -In this example the ``ne_10m_airports2`` ESRI Shapefile, with all its mandatory files (`.shp`, `.shx`, `.dbf` and `.prj`), has been chosen. -When the process ends click on :guilabel:`View` button - -.. figure:: img/dataset_upload_view.png - :align: center - - *Dataset uploading finished* - -.. note:: There are lot of free spatial dataset available in the Internet. In this example, an extract of the Berlin city center roads map from the `BBBike extracts OpenStreetMap `_ dataset has been used. - -In the next paragraphs you will learn how to create a dataset from scratch, how to set share options, how to explore the dataset properties and how to edit them. - -.. note:: If you get the following error message: - - ``Total upload size exceeds 100.0 MB. Please try again with smaller files.`` - - This means that there is an upload size limit of 100 MB. An user with administrative access can change the upload size limits at the `admin panel for size limits <../../admin/upload-size-limits/index.html#upload-size-limits>`__. - - Similarly, for the following error message: - - ``The number of active parallel uploads exceeds 5. Wait for the pending ones to finish.`` - - You can modify the upload parallelism limits at the `admin panel for parallelism limits <../../admin/upload-parallelism-limit/index.html#upload-parallelism-limits>`__. diff --git a/usage/managing_datasets/using_remote_services.rst b/usage/managing_datasets/using_remote_services.rst deleted file mode 100644 index 6448211e..00000000 --- a/usage/managing_datasets/using_remote_services.rst +++ /dev/null @@ -1,78 +0,0 @@ -Using Remote Services -===================== - -In GeoNode you can add new datasets not only by loading them from your disk but also using *Remote Services*. -In this section you will learn how to add a new service and how to load resources in GeoNode through that. - -Let's try it! - -Click on the :guilabel:`Remote Services` link of the :guilabel:`New` menu in the navigation bar. - -.. figure:: img/remote_services_link.png - :align: center - - *Remote Services link* - -The page that opens will contain the list of the available services. - -.. figure:: img/remote_services.png - :align: center - - *Remote Services* - -To configure a new service: - -* click on :guilabel:`Add Remote Service` -* type the *Service URL* -* select the *Service Type* - - .. figure:: img/service_type.png - :align: center - - *Service Types* - -* click on :guilabel:`Create` - -GeoNode supports four **types of remote services**: - -* *Web Map Service* - - Generic Web Map Service (WMS) based on a standard protocol for serving georeferenced map images over the Internet. - These images are typically produced by a map server (like `GeoServer `_) from data provided by one or more distributed geospatial databases. - Common operations performed by a WMS service are: *GetCapabilities* (to retrieves metadata about the service, including supported operations and parameters, and a list of the available datasets) and *GetMap* (to retrieves a map image for a specified area and content). - - .. note:: Lots of WMS services are available on the internet, in this example we used the ``https://demo.geo-solutions.it/geoserver/wms``. - -* *GeoNode Web Map Service* - - Generally a WMS is not directly invoked; client applications such as GIS-Desktop or WEB-GIS are used that provide the user with interactive controls. - A GeoNode WMS automatically performs some operations and lets you to immediately retrieve resources. - - .. note:: An example of GeoNode WMS is available at ``http://dev.geonode.geo-solutions.it/geoserver/wms``. - -* *ArcGIS REST MapServer* - - This map service provides basic information about the map, including the datasets that it contains, whether the map is cached or not, its spatial reference, initial and full extents, whether the service is allowed to export tiles and max tiles export count, etc. - A set of operations that manage the state and contents of the service are allowed: Edit Service, Refresh, Update Tiles. - The URL should follow this pattern: ``https:///services//MapServer``. - - .. note:: Try the following service to better understand how it works: ``https://sampleserver6.arcgisonline.com/arcgis/rest/services/USA/MapServer``. - -* *ArcGIS REST ImageServer* - - This Image Server allows you to assemble, process, analyze, and manage large collections of overlapping, multiresolution imagery and raster data from different sensors, sources, and time periods. You can also publish dynamic image services that transform source imagery and other raster data into multiple image products on demand—without needing to preprocess the data or store intermediate results—saving time and computer resources. In addition, ArcGIS Image Server uses parallel processing across multiple machines and instances, and distributed storage to speed up large raster analysis tasks. - The URL should follow this pattern: ``https:///services//ImageServer``. - - .. note:: Try the following service to better understand how it works: ``https://sampleserver6.arcgisonline.com/arcgis/rest/services/Toronto/ImageServer``. - -Once the service has been configured, you can load the resources you are interested in through the *Import Resources* page where you will be automatically redirected to. - -| From the page where the services are listed, it is possible to click on the *Title* of a service. It opens the *Service Details* page. -| Each service has its own metadata such as the *Service Type*, the *URL*, an *Abstract*, some *Keywords* and the *Contact* user. - -You can edit those metadata through the form available from the :guilabel:`Edit Service Metadata` link of the *Service Details* page (see the picture below). - -.. figure:: img/remote_service_metadata.png - :align: center - - *Remote Service metadata* diff --git a/usage/managing_documents/document_editing.rst b/usage/managing_documents/document_editing.rst deleted file mode 100644 index da954f04..00000000 --- a/usage/managing_documents/document_editing.rst +++ /dev/null @@ -1,91 +0,0 @@ -.. _document-editing: - -Document Editing -================ - -The :ref:`document-info` page makes available useful menu for document editing. Click on the :guilabel:`Edit` link then :guilabel:`Edit Metadata` to see editing forms. - -Setting the Document Thumbnail ------------------------------- - -From the *Metadata Form*, it is possible to *Set the Thumbnail* of the document. Click on :guilabel:`Choose file` to open the *Thumbnail Uploading* page and chose the image that will illustrate your document. Once this is done, click on the ok button and save. If the thumbnail has been successfully uploaded you can see it by coming back to the document list. - -.. figure:: img/document_uploading_thumbnail.png - :align: center - - *Upload Document's Thumbnail* - -If no errors occur, the new selected thumbnail will be shown. - -.. figure:: img/document_uploading_thumbnail_success.png - :align: center - - *Uploading success* - -Editing the Document Metadata ------------------------------ - -You can edit the metadata of your document as described in the :ref:`document-metadata` section or by using THe *Advanced Metadata* option below. - -.. figure:: img/advanced_metadata_button.png - :align: center - -|The :guilabel:`Advanced Metadata` button takes you to a big form where all the available metadata of the document can be edited. -| Some information are mandatory such as the *Title* or the *Category* the document belongs to, some others are optional. - -.. figure:: img/document_metadata_editing.png - :align: center - - *Editing Metadata* - -In the example shown in the picture above, the information inside the red rectangles have been changed. To save the changes click on :guilabel:`Update`, you will be redirected to the document page. - -Editing the Document Info ------------------------------ - -You can edit the information of your document by clicking on the :guilabel:`Info` button. - -From the Info panel, it is possible to upload the **Thumbnail** of the document. - -.. figure:: img/upload_thumbnail.png - :align: center - - *Upload Thumbnail* - -If you want this document in your **Favorites** (see :ref:`editing-profile`), click on the start icon. - -.. figure:: img/favorite_document.png - :align: center - - *Favorite document* - -You can directly **Download** the document and **Share** the document url by clicking the buttons on the top-right of the panel. - -.. figure:: img/download_share_document.png - :align: center - - *Share and Download document* - -By clicking the document **Title**, you can change it. - -.. figure:: img/title_document.png - :align: center - - *Document Title* - -From the :guilabel:`Locations` tab, you can manage the spacial extent of the document. - -.. figure:: img/locations_document.png - :align: center - - *Document Title* - - -Here numerical fields and a map show the spatial extent (WGS 84) for the document and you can edit it by Shift+click and drag to draw the new extent on the map and change its location by clicking the marker and drag. - -.. figure:: img/edit-location.png - :align: center - - *Document Title* - -.. note:: The *Location* tab is also available for :ref:`dashboard` and :ref:`geostory` \ No newline at end of file diff --git a/usage/managing_documents/document_info.rst b/usage/managing_documents/document_info.rst deleted file mode 100644 index 7647a070..00000000 --- a/usage/managing_documents/document_info.rst +++ /dev/null @@ -1,104 +0,0 @@ -.. _document-info: - -Document Information -==================== - -From the *Documents Search Page* (see :ref:`finding-data`) you can select the document you are interested in and click on its name to see an overview about it. - -.. figure:: img/document_overview.png - :align: center - -The information panel reports: - -* The **Info** tab is active by default. This tab section shows some document metadata such as its title, the abstract, date of publication etc. The metadata also indicates the document owner, what are the topic categories the document belongs to and which regions are affected. - -.. figure:: img/document_info.png - :align: center - - *Document Information tab* - -* The **Location** tab shows the spacial extent of the document. - -.. figure:: img/document_location.png - :align: center - - *Document Location tab* - -By clicking on the copy icons you have a copy of the current *Bounding Box* or the *Center* in the clipboard which once pasted will be a WKT text. - -.. figure:: img/copy_locations.png - :align: center - - *Bounding Box and Center* - -.. warning:: From the :ref:`document-info` the *Location* tab is in read only mode, to edit it see the :ref:`document-editing` - -* The *Linked Resources* tab shows the resources, that can be *Datasets*, *Maps*, *GeoStories* and *Dashboards*, linked to the document. - -.. figure:: img/document_linked_resources.png - :align: center - - *Document Linked Resources tab* - -.. note:: When a resource is `linked from` the document the link is under the *Linked by* list while when a resource is `linked to` the document the link is under the *Used by* list - -.. warning:: From the :ref:`document_metadata` it is possible edit the document *Linked Resources* - -From the right toolbar on the **Info** panel it is possible: - -.. figure:: img/document_info_toolbar.png - :align: center - - *Document Info toolbar* - -* Add document in your *Favorites* (see :ref:`editing-profile`), clicking on star icon. - -* Directly *Download* the document oh `ShapeFile` format - -* *Share* the document url - -* Change the document *Title* by clicking on it - -You can access the document details page by clicking on *View document* in the overview panel. -That page looks like the one shown in the picture below. - -.. figure:: img/document_detail.png - :align: center - - *Document page* - -On the page of a document, the resource is either directly displayed on the page or accessible by clicking on the link provided under the title. - -Exploring the Document detail menu Sections ---------------------------- - -As soon as a document is opened, the **Info** panel is shown. It reports the document metadata such as its title, abstract, date of publication etc. The metadata also indicates the user who is responsible for uploading and managing this content, as well as the group to which it is linked. - -.. figure:: img/document_info.png - :align: center - *Document Info* - - - -Selecting *View Metadata* from the :guilabel:`View` button it is possible to visualize the metadata of the document - -.. figure:: img/view_metadata.png - :align: center - - *Document Metadata* - -Under the :guilabel:`Resource` button, it is possible to select the *Share* option to provides the document link to share. - -.. figure:: img/document_sharing.png - :align: center - - *Document Sharing* - -If you want to download the document, click on the :guilabel:`Download` button and the document will be downloaded automatically. - -.. figure:: img/document_download.png - :align: center - - *Document Metadata download* - - diff --git a/usage/managing_documents/document_metadata.rst b/usage/managing_documents/document_metadata.rst deleted file mode 100644 index ddbd87f2..00000000 --- a/usage/managing_documents/document_metadata.rst +++ /dev/null @@ -1,63 +0,0 @@ -.. _document-metadata: - -Filling the Document Metadata -============================= -You can open the Metadata form by clicking on the *Edit* link in the document details menu and then *Edit Metatadata* Link. - - .. figure:: img/edit_metadata_link.png - :align: center - - *Edit Metadata Link* - -Metadata contains all the information related to the document: they are its ID card. They provide essential information for its identification and its comprehension. Metadata also make the document more easily retrievable through search by other users. - -Editing a document's metadata is done in three steps (*Basic Metadata*, *Location and Licenses*, *Optional Metadata*). The first two steps are mandatory (no documents will be published if the required information are not provided) whereas the last one is optional. - -1. On the **Basic Metadata** page, the essential information that has to be filled is: - - * *Title* of the document, which should be clear and understandable; - * *Abstract* of the document; - * *Creation/Publication/Revision* dates which define the time period that is covered by the document; - * *Keywords*, which should be chosen within the available list. The contributor search for available keywords by clicking on the searching bar, or on the folder logo representing, or by entering the first letters of the desired word. Key-words should be relevant to the imported document; - * *Category* in which the document belongs; - * *Group* to which the document is linked. - - .. figure:: img/document_basic_metadata.png - :align: center - - *Document Basic Metadata* - - Once all the fields are filled, click on the blue button :guilabel:`Next >>` in the bottom right corner of the page. - -2. On the **Location and Licenses** page, the following information should be filled: - - * *Language* of the document; - * *Regions*, which informs on the spatial extent covered by the document. Proposed extents cover the following scales: global, continental, regional, national; - * *Data Quality statement* (general explanation of the data producer's knowledge about the lineage of a dataset); - * Potential *Restrictions* to sharing the document should be provided in the Restrictions box. - - .. figure:: img/document_location_licenses.png - :align: center - - *Document Location and Licenses* - - Click on the blue button :guilabel:`Next >>` to go ahead to the next step. - -3. On the **Optional Metadata** page, complementary information can be added: - - * *Edition* to indicate the reference or the source of the document; - * *Purpose* of the document and its objectives; - * *Supplemental information* that can provide a better understanding of the uploaded document; - * *Maintenance frequency* of the document; - * *Spatial representation type* used. - * *Related resources* to link one or multiple resources to the document. These will be visible inside the :ref:`document-info` panel - - *Responsible Parties*, *Owner* and *Share options* are listed on the right side of the page, you can edit them. - - .. figure:: img/document_optional_metadata.png - :align: center - - *Document Optional Metadata* - - | If all the mandatory information is filled out the document can be published, if not the ``Completeness`` progress bar warns you that something is missing. - | Click on the blue button :guilabel:`Update` to save information on the system. diff --git a/usage/managing_documents/document_permissions.rst b/usage/managing_documents/document_permissions.rst deleted file mode 100644 index 326a0b50..00000000 --- a/usage/managing_documents/document_permissions.rst +++ /dev/null @@ -1,33 +0,0 @@ -.. _edit-document-permissions: - -Share Options -=================================================== - -GeoNode encourages to publicly, share and make available for download information uploaded on the platform. By default, anyone can see and download a document. However, the document responsible can choose to limit access to the document to some contributors and/or groups. - -Through the *Share* Link shown in the menu it is possible to manage the document share options. it opens a form where set up who can: - -* None (the document is not visible). -* View Metadata (allows to view the document). -* View and Download (allows to view and download the document). -* Edit (allows to change the document metadata and attributes). -* Manage it (allows to update, delete, change share options, publish/unpublish). - - -.. figure:: img/permissions_form.png - :align: center - - -See an example in the picture below. - -.. note:: - After making changes, always save them with the *Save* Link in the menu - -.. figure:: img/changing_document_permissions.png - :align: center - - *Changing the Document share options* - -Usually those with options of editing metadata and the management of a document are in charge and responsible of the document, i.e. the contributor who uploaded it has those options by default. - -Once the share options are set, click :guilabel:`Save` link in the menu to save them. diff --git a/usage/managing_documents/index.rst b/usage/managing_documents/index.rst deleted file mode 100644 index b8d24c62..00000000 --- a/usage/managing_documents/index.rst +++ /dev/null @@ -1,14 +0,0 @@ -Managing Documents -================== - -| In this section all the aspects concerning :ref:`document-data-type` will be discussed. -| You will learn how to upload a document and how to inspect its metadata and details. All the editing tools will be also explained. - -.. toctree:: - :maxdepth: 1 - - uploading_documents - document_metadata - document_info - document_editing - document_permissions diff --git a/usage/managing_documents/uploading_documents.rst b/usage/managing_documents/uploading_documents.rst deleted file mode 100644 index f9734c2c..00000000 --- a/usage/managing_documents/uploading_documents.rst +++ /dev/null @@ -1,70 +0,0 @@ -.. _uploading-documents: - -Upload/Add Documents -=================== - -GeoNode allows to share reports, conceptual notes, posters, spreadsheets, etc. A wide range of documents files can be hosted on the platform, including text files (.doc, .docx, .txt, .odt), spreadsheets (.xls, .xlsx, .ods), presentations (.ppt, .pptx, .odp), images (.gif, .jpg, .png, .tif, .tiff), PDF, zip files (.rar, .zip, .gz), SLD, XML, QML files or as External URL. - -.. warning:: Only authenticated users can upload data into GeoNode. - -It is possible to upload a *Documents* in two ways: - -- From the :guilabel:`All Resources` page, by clicking *Add Resource* which displays a list including `Upload document` link: - - .. figure:: img/upload_document_link.png - :align: center - - *Document Upload from All Resources page* - -- From the :guilabel:`Documents` page, by clicking on *New* button. - -.. figure:: img/upload_document.png - :align: center - - *Document Upload from Documents page* - -The *Document Upload* page looks like the one shown in the picture below. - - .. figure:: img/document_upload_page.png - :align: center - - *Document Upload page* - -On *GeoNode* documents can be: - -* Upload from the **Local file** - -* Created with reference to **External URL** - - -In order to upload a document from the **Local file**, you need to: - -#. Click on :guilabel:`Select files` button -#. Select a file from your disk. -#. Click the :guilabel:`Upload` button - - .. figure:: img/select_files.png - :align: center - - *Upload document from the select files option* - -| A document may refer to a remote document, without making a local copy of the remote resource. -| To add a document that references an **External URL** you need to: - -#. Click on :guilabel:`Add URL` button -#. Select an URL -#. Select an extension from the drop-down menu -#. Click the :guilabel:`Upload` button - - .. figure:: img/add_url.png - :align: center - - *Add document from the add URL option* - -At the end of the uploading process, by clicking on the View button, you will be driven to the document page with the Info panel open. Here it is possible to view more info, edit metadata, share, download, and delete the document. See the next section to know more about Metatadata. - -.. note:: If you get the following error message: - - ``Total upload size exceeds 100.0 MB. Please try again with smaller files.`` - - This means that there is an upload size limit of 100 MB. An user with administrative access can change the upload limits at the `admin panel <../../admin/upload-size-limits/index.html#upload-size-limits>`__. \ No newline at end of file diff --git a/usage/managing_maps/creating_maps.rst b/usage/managing_maps/creating_maps.rst deleted file mode 100644 index c1f49ffa..00000000 --- a/usage/managing_maps/creating_maps.rst +++ /dev/null @@ -1,68 +0,0 @@ -.. _creating-map: - -Creating Maps -============= - -In this section, we’ll create a *Map* using some uploaded datasets, combine them with some other datasets from remote web services, and then share the resulting map for public viewing. - -In order to create new maps you can use: - -* The :guilabel:`Create map` listed after clicking the :guilabel:`Add Resource` button on the *All Resources* list page. - - .. figure:: img/create_map_link.png - :align: center - - *The Create Map from All Resources page* - -- The :guilabel:`New` button after clicking the :guilabel:`Maps` button on the menu bar. - - .. figure:: img/new_map_link.png - :align: center - - *The Create Map from Maps page* - -* The :guilabel:`Create map` link in the *Dataset Page* (it creates a map using a specific dataset) - - .. figure:: img/create_map_button.png - :align: center - - *The Create map from dataset* - -The new *Map* will open in a *Map Viewer* like the one in the picture below. - -.. figure:: img/map_viewer.png - :align: center - - *The Map Viewer* - -Using the :guilabel:`Add dataset` link, you can add a layer by clicking on one of the layers listed in the catalog. -In the upper left corner the *TOC button* button opens the :ref:`toc` of the *Map*. It allows to manage all the datasets associated with the map and to add new ones from the :guilabel:`Add dataset`. -The *TOC* component makes possible to manage datasets overlap on the map by shifting their relative positions in the list (drag and drop them up or down in the list). -It also allows to hide/show datasets ( |show_button| and |hide_button| ), to zoom to datasets extents ( |zoom_to_dataset_extent_button| ) and to manage their properties ( |dataset_settings_button| ). -Once the map datasets have been settled it is possible to save the *Map* by clicking on the :guilabel:`Save as` under the :guilabel:`Resources` link in the map toolbar. - - -If you followed the steps above, you have just created your first *Map*. -Now you should see it in the *Explore Maps* page, see :ref:`map-info` for further details. - -We will take a closer look at the *Map Viewer* tools in the :ref:`exploring-maps` section. - -.. |show_button| image:: img/show_button.png - :width: 30px - :height: 30px - :align: middle - -.. |hide_button| image:: img/hide_button.png - :width: 30px - :height: 30px - :align: middle - -.. |zoom_to_dataset_extent_button| image:: img/zoom_to_dataset_extent_button.png - :width: 30px - :height: 30px - :align: middle - -.. |dataset_settings_button| image:: img/dataset_settings_button.png - :width: 30px - :height: 30px - :align: middle \ No newline at end of file diff --git a/usage/managing_maps/index.rst b/usage/managing_maps/index.rst deleted file mode 100644 index c28f3157..00000000 --- a/usage/managing_maps/index.rst +++ /dev/null @@ -1,16 +0,0 @@ -Managing Maps -============= - -| Maps are sets of datasets displayed together on an interactive web map. - Maps can be composed in the map composer and saved as GeoNode resources. - Maps can also be associated with metadata, ratings, and comments. -| In this section, you will learn how to create a new map and share it. - -.. toctree:: - :maxdepth: 1 - - creating_maps - maps_configuration/index - map_info - map_metadata - map_permissions \ No newline at end of file diff --git a/usage/managing_maps/map_info.rst b/usage/managing_maps/map_info.rst deleted file mode 100644 index 4409a13c..00000000 --- a/usage/managing_maps/map_info.rst +++ /dev/null @@ -1,73 +0,0 @@ -.. _map-info: - -Map Information -=============== - -From the *Map Search Page* (see :ref:`finding-data`) you can select the map you are interested in and click on its name to see an overview about it. - -.. figure:: img/map_overview.png - :align: center - -The information panel reports: - -* The *Info* tab is active by default. This tab section shows some map metadata such as its title, the abstract, date of publication etc. The metadata also indicates the map owner, what are the topic categories the map belongs to and which regions are affected. - -.. figure:: img/map_info.png - :align: center - - *Map Info tab* - -* The *Location* tab shows the spacial extent of the map. - -.. figure:: img/map_location.png - :align: center - - *Map Location tab* - -By clicking on the copy icons you have a copy of the current *Bounding Box* or the *Center* in the clipboard which once pasted will be a WKT text. - -.. figure:: img/copy_locations_map.png - :align: center - - *Bounding Box and Center* - -* The *Linked Resources* tab shows resources, that can be *Documents*, *Datasets*, *GeoStories* and *Dashboards*, linked to the map. - -.. figure:: img/map_linked_resources.png - :align: center - - *Map Linked Resources tab* - -.. note:: When a resource is `linked from` the map the link is shown under the *Linked by* list while when a resource is `linked to` the map the link is shown under the *Used by* list - - -From the right toolbar on the **Info** panel it is possible: - -.. figure:: img/map_info_toolbar.png - :align: center - - *Map Info toolbar* - -* Add map in your *Favorites* (see :ref:`editing-profile`), clicking on star icon. - -* *Share* the map url - -* Change the map *Title* by clicking on it - -You can access the map details page by clicking on *View map* in the overview panel. -That page looks like the one shown in the picture below. - -.. figure:: img/map_detail_page.png - :align: center - - *Map page* - -The *Map Viewer* shows the map with very basic functionalities: - -* the *Base Map Switcher* that allows you to change the base map; -* the *Zoom in/out* tool to enlarge and decrease the view; -* the *Zoom to max extent* tool for the zoom to fit the datasets extents; -* the *Query Objects* tool to retrieve information about the map objects by clicking on the map; -* the *Print* tool to print the preview. -* the *Measure* tool to measure Distances, Areas and the Bearing of lines. -* the *Annotations* tool to add an annotation to the map \ No newline at end of file diff --git a/usage/managing_maps/map_metadata.rst b/usage/managing_maps/map_metadata.rst deleted file mode 100644 index cb0d8171..00000000 --- a/usage/managing_maps/map_metadata.rst +++ /dev/null @@ -1,97 +0,0 @@ -.. _map-metadata: - -Maps Metadata -============= - -*Maps Metadata* can be Edited by clicking the :guilabel:`Edit Metadata` link from the *Map Detail* page. - -.. figure:: img/map_metadata_detail_button.png - :align: center - - *The Map Metadata Detail link* - -The *Map Metadata* Edit form will open. -| Metadata provide essential information for the identification and the comprehension of the map. They also make the map more easily retrievable through the search tools. -| Those *Metadata* can be filled out through three-steps in which you have to provide all mandatory information to complete the process. Those three steps are described below. - -* **Basic Metadata** - - .. figure:: img/basic_map_metadata.png - :align: center - - *Basic Map Metadata* - - In the first step the system asks you to insert the following metadata (required fields are highlighted with red outlines): - - * *Thumbnail* of the map (click :guilabel:`Edit` to change it); - * *Title* of the map, which should be clear and understandable; - * *Abstract*; brief narrative summary of the content of the Map - - .. note:: The *Abstract* panel allows you to insert HTML code through a *wysiwyg* text editor - - * *Creation/Publication/Revision Dates* which define the time period that is covered by the map; - * *Keywords*, which should be chosen within the available list; - * *Category* which the map belongs to; - * *Group* which the map is linked to. - - Click :guilabel:`Next >>` to go to the next step. - - -* **Location and Licenses** - - .. figure:: img/location_licenses_map_metadata.png - :align: center - - *Location and Licenses Metadata for Maps* - - The following list shows what kinds of metadata you are required to enter (see also the picture below): - - * *Language* of the layer; - * *License* of the dataset; - * *Regions* covered by the layers extent. Proposed extents cover the following scales: global, continental, regional, national; - * *Data Quality statement* (general explanation of the data producer's knowledge about the lineage of a dataset); - * Potential *Restrictions* on layer sharing. - - No further mandatory metadata are required in the next step so, once the required fields have been filled out, a green :guilabel:`Done` button will be visible in the screen. - Click :guilabel:`Next >>` to go to the next step or :guilabel:`<< Back` to go back to the previous step. - - -* **Optional Metadata** - - .. figure:: img/optional_map_metadata.png - :align: center - - *Optional Map Metadata* - - Complementary information are: - - * *Edition* of the map; - * *Purpose* of the map and its objectives; - * *Supplemental information* that can provide a better understanding of the map; - * *Maintenance frequency* of the map; - * *Spatial representation type*, the method used to represent geographic information in the dataset; - * Users who are *Responsible* for the layer, its *Owner*, and the *Author* of its metadata; - * *Related resources* to link one or multiple resources to the document. These will be visible inside the :ref:`map-info` panel - -If you miss some mandatory metadata the *Completeness* bar shows you a red message like the one in the picture below. - -.. figure:: img/completeness_progress_bar.png - :align: center - :width: 200px - - *Completeness Progress Bar* - -Metadata Advanced Editing -------------------------- - -The :guilabel:`Advanced Metadata` editing button in the Metadata Edit form allows to change the map metadata. - -.. figure:: img/advanced_edit_button.png - :align: center - - *The Advanced Edit button* - -Click on it to display the *Metadata Advanced Editing Page*. -That page allows you to edit all the map metadata described in the previous paragraph. - -Once you have finished to edit them click on :guilabel:`Update` to save your changes. diff --git a/usage/managing_maps/map_permissions.rst b/usage/managing_maps/map_permissions.rst deleted file mode 100644 index 8fa4f559..00000000 --- a/usage/managing_maps/map_permissions.rst +++ /dev/null @@ -1,30 +0,0 @@ -.. _map-permissions: - -Share Options -============= - -In GeoNode the share options management system is indeed more complex. -Administrators can choose who can do what for each map. -Users can manage only the maps they own or the maps which they are authorize to manage. - -By default only owners can edit and manage maps, and anyone can view them. - -In order to modify the *Map Share Options* settings you can click the :guilabel:`Share` link in the *Map Detail Page*. - -Through the *Share Options Settings Panel* you can add or remove options for users and groups. The picture below shows an example. - -.. figure:: img/map_permissions_settings.png - :align: center - - *Map Share options settings for users and groups* - -You can set the following options: - -* *View* (allows to view the map). -* *Download* (allows to view and download the map). -* *Edit* (allows to change the map's metadata); -* *Manage* allows to update, delete, change share options, publish and unpublish the map. - -.. warning:: When assigning options to a group, all the group members will have that option. Be careful in case of editing them. - -Click on :guilabel:`Save` link in the menu to save these settings. diff --git a/usage/managing_maps/maps_configuration/attribute_table.rst b/usage/managing_maps/maps_configuration/attribute_table.rst deleted file mode 100644 index e8da6dfe..00000000 --- a/usage/managing_maps/maps_configuration/attribute_table.rst +++ /dev/null @@ -1,155 +0,0 @@ -.. _attributes-table: - -Attributes Table -================ - -.. |attribute_table_button| image:: img/attribute_table_button.png - :width: 30px - :height: 30px - :align: middle - -When clicking on the |attribute_table_button| button of the :ref:`toc`, the *Attributes Table* panel opens at the bottom of the *Map* page. - -.. figure:: img/attributes_table.png - :align: center - - *The Attributes Table Panel* - -.. |zoom_to_feature_icon| image:: img/zoom_to_feature_icon.png - :width: 30px - :height: 30px - :align: middle - -In that panel you can navigate through the features of the dataset, zoom to their geometries by clicking on the |zoom_to_feature_icon| icon and explore their attributes. - -| The *Attribute Tables* has a row for each feature belonging to the dataset and a column for each attribute that describes the feature. -| Each column has a *Filter* input field through which you can filter the features basing on some value or expression (depending on the data type of the field). - -.. figure:: img/filtering_feature_by_attributes.png - :align: center - - *Filtering Features by Attribute* - -The *Attributes Table* panel contains a *Toolbar* which makes you available some useful functionalities. - -.. figure:: img/attributes_table_toolbar.png - :align: center - - *The Attributes Table Toolbar* - -Those functionalities are: - -* | *Edit Mode* - | By clicking on |edit_mode_button| you can start an editing session. It permits you to add new features, to delete or modify the existing ones, to edit geometries. See the :ref:`dataset-data-editing` section for further information. - - .. |edit_mode_button| image:: img/edit_mode_button.png - :width: 30px - :height: 30px - :align: middle - -* | *Advanced Search* - | Click on |advanced_search_button|, a new panel opens. That panel allows you to filter features in many different ways. This functionality will be explained in depth in the :ref:`advanced-search` section. - - .. |advanced_search_button| image:: img/advanced_search_button.png - :width: 30px - :height: 30px - :align: middle - -* | *Zoom to page extent* - | Click on |zoom_to_page_extent_button| to zoom to the page extent. - - .. |zoom_to_page_extent_button| image:: img/zoom_to_page_extent_button.png - :width: 30px - :height: 30px - :align: middle - -* | *Export Data* - | Click on |export_data_button| to open the export/download data form. - - .. |export_data_button| image:: img/export_data_button.png - :width: 30px - :height: 30px - :align: middle - - -* | *Hide/show columns* - | When clicking on |hide_show_columns_button| another panel opens inside the *Attributes Table*. Through that panel you can choose what columns you want to see. - - .. |hide_show_columns_button| image:: img/hide_show_columns_button.png - :width: 30px - :height: 30px - :align: middle - -* | *Create a chart* - | Through the |create_charts_button| button you can open the *Chart Widgets* panel where many functionalities to describe and visualize the dataset data are available (see :ref:`creating-widgets`). - - .. |create_charts_button| image:: img/create_charts_button.png - :width: 30px - :height: 30px - :align: middle - -* | *Sync map with filter* - | Click on the |sync_map_with_filter_button| icon to synchronize the map with the filter. - - .. |sync_map_with_filter_button| image:: img/sync_map_with_filter_button.png - :width: 30px - :height: 30px - :align: middle - -.. _advanced-search: - -Advanced Search ---------------- - -As mentioned before, GeoNode allows both an attribute based and spatial filtering. -When clicking on |advanced_search_button| from the dataset *Attributes Table* the *Advanced Search* panel opens and shows you three different filtering functionalities: - -.. figure:: img/advanced_search.png - :align: center - - *Advanced Search* - -* In the **Attribute Filter** section you can compose a series of conditions about the attributes of the dataset. - Click on |add_condition_button| to insert a new empty condition. - Select the attribute you are interested in, select an operator and type a comparison value. - You can group conditions through the *Add Group* |add_group_button| button. - Click on |start_search_button| to perform the search. - - .. figure:: img/filtering_by_attributes.png - :align: center - - *Filtering by Attributes* - - .. |add_condition_button| image:: img/add_condition_button.png - :width: 30px - :height: 30px - :align: middle - - .. |add_group_button| image:: img/add_group_button.png - :width: 30px - :height: 30px - :align: middle - - You can also decide if *All* the conditions have to be met, if only *Any* or *None* of them (see the red arrow in the picture above). - -* | The ** Area of interest** filtering allows you to filter features that have some relationship with a spatial region that you draw on the map. - | Select the *Filter Type* (Circle, Viewport, Polygon or Rectangle), draw the spatial region of interest on the map, select a *Geometric Operation* (Intersects, Bounding Box, Contains or Is contained) and then click on |start_search_button|. - - .. figure:: img/filtering_region_of_interest.png - :align: center - - *Filtering by Area Of Interest* - - .. |start_search_button| image:: img/start_search_button.png - :width: 30px - :height: 30px - :align: middle - -* Through the **Dataset Filter** you can select only those features which comply with some conditions on other datasets of the map. You can also add conditions on attributes for those datasets. - - .. figure:: img/dataset_filtering.png - :align: center - - *Dataset Filtering* - -You can read more about the *Attributes Table* and the *Advanced Search* on the `MapStore Documentation `_. diff --git a/usage/managing_maps/maps_configuration/creating_widgets.rst b/usage/managing_maps/maps_configuration/creating_widgets.rst deleted file mode 100644 index 76d49ec2..00000000 --- a/usage/managing_maps/maps_configuration/creating_widgets.rst +++ /dev/null @@ -1,125 +0,0 @@ -.. _creating-widgets: - -Creating Widgets -================ - -.. |create_widgets_button| image:: img/create_widgets_button.png - :width: 30px - :height: 30px - :align: middle - -*Widgets* are graphical elements that describe the datasets data. They can be of different types such as *Charts*, *Texts*, *Tables* and *Counters*. -Through the |create_widgets_button| button of the :ref:`toc` you can open the *Widgets* panel. - -.. figure:: img/widgets_panel.png - :align: center - - *Creating Widgets* - -Chart Widgets -------------- - -*Chart Widgets* are graphical representations of the dataset data. They can be *Bar Chart*, *Pie Chart* or *Line Chart* as shown in the picture below. - -.. figure:: img/chart_widgets.png - :align: center - - *Chart Widgets* - -.. |advanced_search_button| image:: img/advanced_search_button.png - :width: 30px - :height: 30px - :align: middle - -.. |configure_widgets_options_button| image:: img/configure_widgets_options_button.png - :width: 30px - :height: 30px - :align: middle - -.. |save_button| image:: img/save_button.png - :width: 30px - :height: 30px - :align: middle - -.. |connect_to_viewport| image:: img/connect_to_viewport.png - :width: 30px - :height: 30px - :align: middle - -| Lets create a new **Bar Chart**. -| Click on *Bar Chart* then select the *X Attribute*, the *Y Attribute*, the *Operation* and the *Color* do you prefer. - You can also display the *Legend*, *Hide the Y axis*, *Hide the grid* and decide what *Label* display into the legend. -| Now you can filter the data to be considered for the chart by clicking on |advanced_search_button|. We don't need any filter so click |configure_widgets_options_button| to configure other widget options. - Insert a *Title* and a *Description* and click on *Save* |save_button|. - -.. figure:: img/bar_chart_widget.png - :align: center - - *Chart Widgets Creation* - -The green |connect_to_viewport| icon means that the chart is connected to the viewport. - -At the top of the bar chat, there is the options menu of the widget where you can *Download grapg as png*, *Zoom the wigets* and *Reset axes*. - -.. figure:: img/chart_widget_options.png - :align: center - :width: 500px - - *Chart Widgets Options* - -Text Widgets ------------- - -If you select *Text* on the *Widgets* panel you can create *Text Widgets*. Add a *Title* and the desired descriptive text, then click on |save_button|. - - -.. figure:: img/text_widgets_creation.png - :align: center - - *Text Widgets Creation* - -The resulting widget looks like the following. - -.. figure:: img/text_widget.png - :align: center - - *My Text Widget* - -Table Widgets -------------- - -| Through the *Table Widgets* you can add the :ref:`attributes-table` of the dataset to the map. You can decide to show a subset of the features, through filters, and you can select one or more columns/attributes. -You can also enter a *Title* for each column to be displayed as the table header in place of the Name of the layer field and enter a *Description* for each field to be displayed as a tooltip, visible moving the mouse on the column header. -| So, choose what attributes you are interested in and click on |configure_widgets_options_button|. - -.. figure:: img/table_widget_columns.png - :align: center - - *Table Widgets Columns* - -Insert *Title* and *Description* (optional) and click on |save_button|. The example below shows the *Table Widget* on the map. - -.. figure:: img/table_widget.png - :align: center - - *Table Widget* - -Counter Widgets ---------------- - -*Counter Widgets* are numeric representations of some attributes. For example you can represent the average speed limit on a road network. - -.. figure:: img/counter_widget_creation.png - :align: center - :width: 400px - - *Counter Widget Creation* - -Click on |configure_widgets_options_button|, insert *Title* and *Description* then click on |save_button|. - -.. figure:: img/counter_widget.png - :align: center - - *Counter Widget* - -The GeoNode map viewer is `MapStore `_ based, see the `MapStore Documentation `_ for further information. diff --git a/usage/managing_maps/maps_configuration/get_fetureinfo.rst b/usage/managing_maps/maps_configuration/get_fetureinfo.rst deleted file mode 100644 index 890913db..00000000 --- a/usage/managing_maps/maps_configuration/get_fetureinfo.rst +++ /dev/null @@ -1,115 +0,0 @@ -.. _getfetureinfo-templates: - -Customizing The Datasets' GetFeatureInfo Templates -================================================ -When “clicking” over a feature of a dataset into a GeoNode Map, an info window popups showing a formatted representation of the raw data identified by the coordinates (see Fig. 1) - -.. figure:: img/getfetureinfo_templates_001.png - :align: center - - *Fig. 1* - -The way how such information is presented to the user is defined by what we call “GetFeatureInfo Template”. The latter is basically an HTML snippet containing some placeholders and special inline codes that instruct GeoServer on how to generate the raw data output. - -The outcome is a rendered HTML snippet with the real values replacing the placeholders of the Template. - -Currently, GeoNode allows a very simple mechanism to customize the “GetFeatureInfo Template” of a dataset. - -It is possible, through the dataset Metadata Editor, to assign a name, a label and also set the attributes we want to display on the GetFeatureInfo output. - -As an instance, by using the example above, we can customize a bit the dataset Metadata as shown in Fig. 2 - -.. figure:: img/getfetureinfo_templates_002.png - :align: center - - *Fig. 2* - -The “GetFeatureInfo” output will change accordingly as shown in Fig. 3 - -.. figure:: img/getfetureinfo_templates_003.png - :align: center - - *Fig. 3* - -Simple Template: Assigning A Media-Type To Attribute Values -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The easiest way to render a different media-type (:guilabel:`image`, :guilabel:`audio`, :guilabel:`video` or :guilabel:`iframe`) to a property value, is to change it from the :guilabel:`Metadata Edit` attributes panel. - -By changing the :guilabel:`Display Type` of an attribute from this panel as shown in Fig. 4 - -.. figure:: img/getfetureinfo_templates_004.png - :align: center - - *Fig. 4* - -GeoNode will create automatically the HTML media type when rendering by using the **value** of the selected property. - -So, as an example, if, in the figure above, the attribute ``NAME`` contains values representing some ``links`` to other resources, GeoNode will create those links automatically for you when clicking over a geometry. - -.. figure:: img/getfetureinfo_templates_005.png - :align: center - - *Fig. 5* - -Selecting :guilabel:`image` as media-type (Fig. 6) - -.. figure:: img/getfetureinfo_templates_006.png - :align: center - - *Fig. 6* - -and editing the contents accordingly (Fig. 7) - -.. warning: Pay attention to the **length** of the values you put into the properties. They must respect you data schema. - -.. figure:: img/getfetureinfo_templates_007.png - :align: center - - *Fig. 7* - -you will get a nice effect as shown in Fig. 8 - -.. figure:: img/getfetureinfo_templates_008.png - :align: center - - *Fig. 8* - -Advanced Template: Use A Custom HTML Template -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -By selecting the option :guilabel:`Use a custom template?` as shown in Fig. 9 - -.. figure:: img/getfetureinfo_templates_009.png - :align: center - - *Fig. 9* - -You will be able to provide your own custom HTML Template for the Feature Info output. -Attributes from the dataset can be represented inside the template using ``${properties.ATTRIBUTE}``, where ``ATTRIBUTE`` is the name of an attribute (field) from your dataset. - -The example below uses the ``wikipedia`` and the ``name`` attributes to show the name of the airport as a link to the Wikipedia page: - -.. figure:: img/getfetureinfo_templates_010.png - :align: center - - *Fig. 10* - -.. figure:: img/getfetureinfo_templates_011.png - :align: center - - *Fig. 11* - -This is the result shown when you click on an airport - -.. figure:: img/getfetureinfo_templates_012.png - :align: center - - *Fig. 12* - -Optional: Customizing the HTML WYSIWYG Editor Menu Bar -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The :guilabel:`Menu Bar` and :guilabel:`Tool Bar` of the HTML Editor, can be easily customized by overriding the ``TINYMCE_DEFAULT_CONFIG`` variable on :guilabel:`settings.py` (see :ref:`tinyMCE Default Config Settings`) - -There are many plugins and options allowing you to easily customize the editor and also provides some predefined *templates* to speed up the editing. - -For more information about the Javascript tool, please refer to https://www.tiny.cloud/ \ No newline at end of file diff --git a/usage/managing_maps/maps_configuration/index.rst b/usage/managing_maps/maps_configuration/index.rst deleted file mode 100644 index 626d879e..00000000 --- a/usage/managing_maps/maps_configuration/index.rst +++ /dev/null @@ -1,182 +0,0 @@ -.. _exploring-maps: -.. _asemap-switcher: - -Map tools and configuration -================================ - -In this section, we are going to explore all tools provided on the Map View page. From the list of available maps, you can select the map you are interested in and click *View map*, the map will look like this. - -.. figure:: img/explore_map_viewer.png - :align: center - - *The Map View* - -The Map View (based on `MapStore `_) provides the following tools: - -* the :ref:`toc` to manage the map contents; -* the *Basemap Switcher* to change the basemap (see the next paragraphs); -* the *Search Bar* to search by location, name and coordinates (see the paragraph below); -* the :ref:`options-menu-tools` which contains the link to the datasets *Catalog*; -* the *Sidebar* which contains, by default, the link to the *Print* tool and to the *Measure* tool; -* the *Navigation Bar* and its tools such as the *Zoom* tools, the *3D Navigation* tool and the *Get Features Info* tool; -* the *Footer Tools* to manage the scale of the map, to track the mouse coordinates and change the CRS (Coordinates Reference System). - -A map can be configured to use a custom :ref:`map-viewers`, with which the lsit of tools available in the map can be customized. - -.. toctree:: - :hidden: - :maxdepth: 1 - - map_viewers - toc - attribute_table - creating_widgets - timeline - options_menu - get_fetureinfo - - -Search Bar ----------- - -| The *Search Bar* of the map viewer allows you to find point of interests (POIs), streets or locations by name. -| Lets type the name of some place then select the first record. - -.. figure:: img/search_bar_typing_name.png - :align: center - - *The Search Bar* - -The map will automatically re-center on that area delimiting it by a polygon in the case of an area, by a line in the case of a linear shape (e.g. streets, streams) and by a marker in the case of a point. - -.. figure:: img/search_bar_result.png - :align: center - - *Result of a Search* - -.. _sidebar: - -Navigation bar -------------- - -.. |sidebar_expand_button| image:: img/sidebar_expand_button.png - :width: 30px - :height: 30px - :align: middle - -.. |zoom_in_button| image:: img/zoom_in_button.png - :width: 30px - :height: 30px - :align: middle - -.. |zoom_out_button| image:: img/zoom_out_button.png - :width: 30px - :height: 30px - :align: middle - -.. |3d_button| image:: img/3d_button.png - :width: 30px - :height: 30px - :align: middle - -.. |query_objects_on_map_button| image:: img/query_objects_on_map_button.png - :width: 30px - :height: 30px - :align: middle - -.. |zoom_to_max_extent_button| image:: img/zoom_to_max_extent_button.png - :width: 30px - :height: 30px - :align: middle - -.. |go_back_button| image:: img/go_back_button.png - :width: 30px - :height: 30px - :align: middle - -.. |go_forward_button| image:: img/go_forward_button.png - :width: 30px - :height: 30px - :align: middle - -.. |full_screen_button| image:: img/full_screen_button.png - :width: 30px - :height: 30px - :align: middle - -.. |show_hide_coordinates_button| image:: img/show_hide_coordinates_button.png - :width: 30px - :height: 30px - :align: middle - -| The *Map Viewer* makes also available the *Navigation bar*. - It is a navigation panel containing various tools that help you to explore the map such as tools for zooming, changing the extent and querying objects on the map. -| By default the *Navigation bar* shows you the zooming buttons |zoom_in_button|, |zoom_out_button| and |3d_button|, other options can be explored by clicking on |sidebar_expand_button| which expands/collapses the toolbar.| - -.. figure:: img/collapsed_sidebar.png - :align: center - :width: 300px - - *The Default Navigation bar* - -.. figure:: img/expanded_sidebar.png - :align: center - :width: 300px - - *The Expanded Navigation bar* - -The *Navigation bar* contains the following tools: - -* The *Query Objects on map* allows you to get feature information through the |query_objects_on_map_button| button. - It allows you to retrieve information about the features of some datasets by clicking them directly on the map. - - .. figure:: img/querying_objects_on_map.png - :align: center - - *Querying Objects on map* - - When clicking on map a new panel opens. That panel will show you all the information about the clicked features for each active loaded dataset. - -* You can *Zoom To Max Extent* by clicking |zoom_to_max_extent_button|. - - -Basemap Switcher ----------------- - -By default, GeoNode allows to enrich maps with many world backgrounds. You can open available backgrounds by clicking on the map tile below: - -* *OpenStreetMap* -* *OpenTopoMap* -* *Sentinel-2-cloudless* - -.. figure:: img/basemap_switcher.png - :align: center - :height: 400px - - *The Basemap Switcher Tool* - -You can also decide to have an *Empty Background*. - -.. figure:: img/basemap_switching.png - :align: center - :height: 400px - - *Switching the Basemap* - -Footer Tools ------------- - -At the bottom of the map, the *Footer* shows you the *Scale* of the map and allows you to change it. - -.. figure:: img/map_scaling.png - :align: center - :height: 600px - - *The Map Scale* - -The |show_hide_coordinates_button| button allows you to see the pointer *Coordinates* and to change the Coordinates Reference System (CRS), WGS 84 by default. - -.. figure:: img/map_coordinates.png - :align: center - - *The Pointer Coordinates and the CRS* diff --git a/usage/managing_maps/maps_configuration/map_viewers.rst b/usage/managing_maps/maps_configuration/map_viewers.rst deleted file mode 100644 index 6fdf7877..00000000 --- a/usage/managing_maps/maps_configuration/map_viewers.rst +++ /dev/null @@ -1,32 +0,0 @@ -.. _map-viewers: - -Map Viewer ------------------- - -GeoNode allows for the administrator of the map to configure a custom viewer by choosing the set of plugins available for the viewer - -From the :guilabel:`Add viewer` option under the :guilabel:`Edit` options of the *Menu* a page opens and the user can: - -- Select an existing viewer from the list of viewers by clicking on the :guilabel:`Link to viewer` - -- Create a new viewer by clicking on the :guilabel:`Create new viewer` - -.. figure:: img/add-viewer-page.png - :align: center - - *Add viewer option* - -Once the :guilabel:`Create new viewer` is selected an *Edit Plugins* page opens and, through the central vertical bar, the user can select the plugins to include in the context viewer by moving them from the **Available Plugins** list to the **Enabled Plugins** list. - -.. figure:: img/enable_plugins.png - :align: center - - *Enable plugins fro the viewer* - -To save and enable the map viewer, the user can click :guilabel:`Save as` option under the :guilabel:`Resource` options of the *Menu*. - -The *Map Viewer* will be visible to all users who have permission to view the map and can be reused or modified by the user who has edit permissions on it. -It will be available on the list of the resources on *Home Page* - - -See the `MapStore Documentation `_ for more information. \ No newline at end of file diff --git a/usage/managing_maps/maps_configuration/options_menu.rst b/usage/managing_maps/maps_configuration/options_menu.rst deleted file mode 100644 index 843ca30b..00000000 --- a/usage/managing_maps/maps_configuration/options_menu.rst +++ /dev/null @@ -1,107 +0,0 @@ -.. _options-menu-tools: - -Other Menu Tools -================== - -.. |burger_menu_button| image:: ../img/burger_menu_button.png - :width: 30px - :height: 30px - :align: middle - -At the top of the *Map* and on the *SideBar* of the map there are more menu items which we are are going to explain in this section. - -Add dataset ------------------- - -All the datasets available in GeoNode, both uploaded and remote, can be loaded on the map through the *Catalog*. -Click on the :guilabel:`Add dataset` option of the *Menu* to take a look at the catalog panel. - -.. figure:: img/datasets_catalog.png - :align: center - - *The Datasets Catalog* - -You can navigate through datasets and look at their *Thumbnail* images, *Title*, *Description* and *Abstract*. -Click on a dataset to load it into the map, it will be also visible in the :ref:`toc`. - -Saving a map ------------- - -| Once all the customizations have been carried out, you can *Save* your map by clicking on the :guilabel:`Save` option under the :guilabel:`Resources` options of the *Menu*. - -|You could create a new map from the existing view by clicking :guilabel:`Save As..`. -| A new popup window will open. - -.. figure:: img/save_as_map.png - :align: center - - *Saving as Map option* - -The current map title is filled by default, You can change it to the prefered naming then click on :guilabel:`Save`. The page will reload and your map should be visible in the :ref:`finding-data` list. - -Printing a map --------------- - -.. |print_button| image:: img/print_button.png - :width: 30px - :height: 30px - :align: middle - -| The `MapStore `_ based map viewer of GeoNode allows you to print the current view with a customizable layout. - -Click the |print_button| button from the *SideBar*, the **Printing Window** will open. - -.. figure:: img/printing_window.png - :align: center - - *The Printing Window* - -From this window you can: - -* enter *Title* and *Description*; -* choose the *Resolution* in dpi; -* select the format -* select the coordinate -* add the scale -* add grid with label -* customize the *Layout* - - + the *Sheet size* (A3, A4); - + if include the legend or not; - + if to put the legend in a separate page; - + the page *Orientation* (Landscape or Portrait); - -* customize the *Legend* - - + the *Label Font*; - + the *Font Size*; - + the *Font Emphasis* (bold, italic); - + if *Force Labels*; - + if use *Anti Aliasing Font*; - + the *Icon Size*; - + the *Legend Resolution* in dpi. - -To print the view click on :guilabel:`Print`. - -Performing Measurements ------------------------ - -.. |measure_button| image:: img/measure_button.png - :width: 30px - :height: 30px - :align: middle - -Click on the |measure_button| button from the *SideBar* to perform a measurement. -As you can see in the picture below, this tool allows you to measure *Distances*, *Areas* and the *Bearing* of lines. - -.. figure:: img/measure_tool.png - :align: center - - *The Measure Tool* - -| To perform a measure draw on the map the geometry you are interested in, the result will be displayed on the left of the unit of measure select menu (this tool allows you to change the unit of measure also). - -.. figure:: img/measuring_areas.png - :align: center - - *Measuring Areas* diff --git a/usage/managing_maps/maps_configuration/timeline.rst b/usage/managing_maps/maps_configuration/timeline.rst deleted file mode 100644 index c1c347a9..00000000 --- a/usage/managing_maps/maps_configuration/timeline.rst +++ /dev/null @@ -1,134 +0,0 @@ -.. _timeline: - -Timeline -======== - - -| GeoNode can manage datasets with a *time dimension*. Those vector datasets may vary their data through time so it is useful to represent that variation on the map. -| The `MapStore `_ based map viewer used in Geonode makes available the **Timeline** tool which allows you to observe the datasets' evolution over time, to inspect the dataset configuration at a specific time instant and to view different dataset configurations time by time dynamically through animations (see the `MapStore Documentation `_ for further details). - -.. warning:: Timeline actually works only with WMTS-Multidim extension (WMS time in capabilities is not fully supported). - -When loading a temporal dataset into the map, the *Timeline* opens automatically. - -.. figure:: img/collapsed_timeline.png - :align: center - - *The Timeline* - -On the left side of the *Timeline* panel you can set the time value in which you want to observe the data. You can type it directly filling out the corresponding input fields or by using the up/down arrows. - -.. figure:: img/time_instant_controls.png - :align: center - - *The Time Control Buttons* - -| On the other side there are the buttons responsible for managing the animations. -| In particular you can *Play* the animation by clicking |play_button|, go back to the previous time instant through |time_go_backward_button|, go forward to next time step using |time_go_forward_button| and stop the animation by clicking |stop_button|. - -.. figure:: img/animation_buttons.png - :align: center - - *The Animation Control Buttons* - -The *Timeline* panel can be expanded through the |expand_timeline_button| button. - -.. figure:: img/expanded_timeline.png - :align: center - - *The Expanded Timeline* - -The expanded section of the *Timeline* panel contains the *Time Datasets List* and an *Histogram* which shows you: - -* the distribution of the data over time - - .. figure:: img/timeline_histogram.png - :align: center - - *The Timeline Histogram* - -* the *Time Cursor* - - .. figure:: img/time_cursor.png - :align: center - - *The Time Cursor* - -You can show/hide the datasets list by clicking |show_hide_datasets_list_button| (it is active by default). - -Through the *Time Range* function you can observe the data in a finite temporal interval. -Click on |time_range_button| and set the initial and the final times to use it. - -.. figure:: img/time_range.png - :align: center - - *The Time Range Settings* - -Animations ----------- - -| The *Timeline* allows you to see the data configurations (one for each time in which the data are defined) through ordered sequences of steps. -| As said before, you can play the resulting *Animation* by clicking the play button |play_button|. - The dataset data displayed on map will change accordingly to the time reach by the cursor on the *Histogram*. - -By clicking on |animation_settings_button| you can manage some *Animation Settings*. - -.. figure:: img/timeline_settings.png - :align: center - :height: 400px - - *The Timeline Settings* - -| You can activate the *Snap to guide dataset* so that the time cursor will snap to the selected dataset's data. You can also set up the *Frame Duration* (by default 5 seconds). -| If the *Snap to guide dataset* option is disabled, you can force the animation step to be a fixed value. - -The *Animation Range* option lets you to define a temporal range within which the time cursor can move. - -.. figure:: img/timeline_animation.png - :align: center - - *The Timeline Animation Range* - - -See the `MapStore Documentation `_ for more information. - -.. |play_button| image:: img/play_button.png - :width: 30px - :height: 30px - :align: middle - -.. |time_go_backward_button| image:: img/time_go_backward_button.png - :width: 30px - :height: 30px - :align: middle - -.. |time_go_forward_button| image:: img/time_go_forward_button.png - :width: 30px - :height: 30px - :align: middle - -.. |stop_button| image:: img/stop_button.png - :width: 30px - :height: 30px - :align: middle - -.. |expand_timeline_button| image:: img/expand_timeline_button.png - :width: 30px - :height: 30px - :align: middle - -.. |show_hide_datasets_list_button| image:: img/show_hide_datasets_list_button.png - :width: 30px - :height: 30px - :align: middle - -.. |time_range_button| image:: img/time_range_button.png - :width: 30px - :height: 30px - :align: middle - -.. |animation_settings_button| image:: img/animation_settings_button.png - :width: 30px - :height: 30px - :align: middle - diff --git a/usage/managing_maps/maps_configuration/toc.rst b/usage/managing_maps/maps_configuration/toc.rst deleted file mode 100644 index 3735745c..00000000 --- a/usage/managing_maps/maps_configuration/toc.rst +++ /dev/null @@ -1,152 +0,0 @@ -.. _toc: - -Table of Contents (TOC) -======================= - -In the upper left corner, click on |toc_button| to open the *Table Of Contents*, briefly *TOC* from now on, of the map. -The *TOC* shows all the datasets involved with the *Map* and allows to manage their properties and representations on the map. - -.. figure:: img/toc_panel.png - :align: center - :height: 200px - - *The Table Of Contents (TOC)* - -From the *TOC* you can: - -* manage the *TOC Settings* (click on |settings_toc_button|) (See the `MapStore Documentation `_ for more information.) -* manage the datasets *Overlap*; -* filter the datasets list by typing text in the *Filter Datasets* field; -* manage the datasets properties such as *Opacity* (scroll the opacity cursor), *Visibility* (click on |hide_button| to make the dataset not visible, click on |show_button| to show it on map); -* add and manage *Annotations* (click on |annotation_button|) -* manage the *Dataset Settings*, see the next paragraph. - -.. figure:: img/scrolling_opacity.png - :align: center - :height: 400px - - *Scrolling the Dataset Opacity* - -Select a *Dataset* from the list and click on it, the *Dataset Toolbar* should appear in the *TOC*. - -.. figure:: img/dataset_toolbar.png - :align: center - :height: 200px - - *The Dataset Toolbar* - - -The *Toolbar* shows you many buttons: - -* the **Zoom to dataset extent** button allows you to zoom to the dataset extent; -* the **Filter layer** button that acts directly on a layer with WFS available and filter its content; -* the **Attribute Table** button to explore the features of the dataset and their attributes (more information at :ref:`attributes-table`); -* the **Delete** button to delete datasets (click on :guilabel:`Delete Dataset` to confirm your choice); -* the **Widgets** button to create Widgets (see :ref:`creating-widgets`). -* the **Export data** button; -* the **Settings** button drives you through the dataset settings customization (see the next paragraph); -* the **Compare tool** button to *Swipe* or *Spy* the selected layer -* the **Edit Style** button; - - -Managing Dataset Settings ------------------------ - -The *Dataset Settings* panel looks like the one below. - -.. figure:: img/dataset_settings_panel.png - :align: center - :height: 600px - - *The Dataset Settings Panel* - -The *Dataset Settings* are divided in three groups: - -1. *General* settings -2. *Visibility* settings -3. *Style* settings -4. *Tiling* settings - -In the **General** tab of the *Settings Panel* you can customize the dataset *Title*, insert a *Description*, change/add the *Dataset Group* and change the *Tooltip content* and the *Tooltip placement*. - -The **Visibility** tab where you can Change the *Opacity* of the layer and add the *Visibility limits* to display the layer only within certain scale limits - -.. figure:: img/visibility_settings.png - :align: center - :height: 400px - - *The Visibility tab on Settings Panel* - -The **Style** tab allows you to select the style from the available layer styles and change the *Width* and the *Height* of the *Legend*. - -.. figure:: img/style_settings.png - :align: center - :height: 400px - - *The Style tab on Settings Panel* - -Click on the **Tiling** tab to change the output *Format* of the WMS requests, the *Tile Size* and enable/disable the *Trasparent*, the *Use cache options* and the *Single Tile*. - -.. figure:: img/tiling_settings.png - :align: center - :height: 400px - - *The Tiling tab on Settings Panel* - -.. |toc_button| image:: img/toc_button.png - :width: 30px - :height: 30px - :align: middle - -.. |hide_button| image:: img/hide_button.png - :width: 30px - :height: 30px - :align: middle - -.. |show_button| image:: img/show_button.png - :width: 30px - :height: 30px - :align: middle - -.. |settings_toc_button| image:: img/settings_toc_button.png - :width: 30px - :height: 30px - :align: middle - -Add an Annotation ------------- -Click on the |annotation_button| button from the *TOC Toolbar* to enrich the map with special features which expose additional information, mark particular position on the map and so on. -From here the editor can insert a *Title* and a *Description*. - -.. figure:: img/add_annoations.png - :align: center - - *Annotations panel* - -.. |add_annotations_button| image:: img/add_annotations_button.png - :width: 30px - :height: 30px - :align: middle - -.. |annotation_button| image:: img/annotation_button.png - :width: 30px - :height: 30px - :align: middle - -To begin, from the annotation panel, the editor add new annotation by selecting the :guilabel:`Geometries` tab. - -.. figure:: img/add_an_annotations.png - :align: center - - *Add an Annotations* - - -Here the user can choose between five different types of *Geometries*: - -1. *Marker* -2. *Line* -3. *Polygon* -4. *Text* -5. *Circle* - -See the `MapStore Documentation `_ for more information. \ No newline at end of file diff --git a/usage/other_apps/qgis/index.rst b/usage/other_apps/qgis/index.rst deleted file mode 100644 index 7d0e94dd..00000000 --- a/usage/other_apps/qgis/index.rst +++ /dev/null @@ -1,153 +0,0 @@ -.. _qgis: - -QGIS Desktop -============ - -QGIS is a professional GIS application that is built on top of and proud to be itself Free and Open Source Software (FOSS). QGIS is a volunteer driven project if you are interested you can find more information at https://www.qgis.org. - -.. figure:: img/geonode_qgis_desktop.PNG - :align: center - - *QGIS Desktop Main Window* - -How can I connect to Geonode? ------------------------------ - -Open QGIS Desktop and go to **Layer Menu > Data Source Manager**. At the bottom of Data Source Manager, you can see a tab -with the name and an icon related to Geonode. This is because Geonode is recognized as a data source inside QGIS. - -.. figure:: img/geonode_datamanager_dialog.PNG - :align: center - - *Data Source Manager Dialog* - -.. note:: - It's possible as well load Geonode instances from an existence file this is useful to share between users or to backup existence connections. - -To add a new Geonode instance, in the Geonode tab selected click on **New** and you will see the following dialog: - -.. figure:: img/geonode_connection_details.PNG - :align: center - - *Details of Geonode instance Dialog* - - -In the dialog Fill the name as you like and in the URL put the link of the Geonode instance. It's possible edit some WFS and WMS options to optimize the connection. -If everything is ok you will receive the following successful connection dialog: - -.. figure:: img/geonode_success_connection.PNG - :align: center - - *Successful connection Dialog* - -After the successful dialog it's now possible to load all layers of the Geonode instance clicking on **Connect** button. You can see both WMS and WFS connections of the Geonode and you can load to QGIS Desktop. - -.. figure:: img/geonode_load_layers.PNG - :align: center - - *Geonode instance layers Dialog* - -After select a layer (WMS or WFS) click on the **Add** button and the layer will be displayed in the main window of QGIS. - -.. figure:: img/geonode_example_layer.PNG - :align: center - - *Example of Geonode layer* - -.. warning:: - This procedure only work with public layers. If the layers are for private use is necessary to do - the standard qgis add remote WMS/WFS layers (through **Data Source Manager**) along with basic auth method and specific endpoints. - -Connect to Private layers by using OAuth2 ------------------------------------------ - -GeoNode OAuth2 Client App Setup -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Login to GeoNode as a superuser - -.. figure:: img/geonode_oauth2_001.png - :align: center - -Browse to :guilabel:`http:///o/applications/` - -.. figure:: img/geonode_oauth2_002.png - :align: center - -Create a new specific app or, better, edit the existing one (“GeoServer”) based on `OAuth2 Authorization Code Grant Type `_ - -Click on “Edit” and add the Redirect URI :guilabel:`http://127.0.0.1:7070/qgis-client` as shown below - -.. note:: - This is an example. The port and path of the URI can be customized. They must be the same on both GeoNode and QGis Client as shown later. - -.. figure:: img/geonode_oauth2_003.png - :align: center - -.. figure:: img/geonode_oauth2_004.png - :align: center - -Also you will need the :guilabel:`Client ID` and :guilabel:`Client Secret` keys later when configuring QGis. - -Configure QGis Desktop Client OAuth2 Authentication -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Open the QGis Desktop Client and add a new OWS remote Layer configuration - -.. figure:: img/geonode_oauth2_005.png - :align: center - -Create a new service connection - -.. figure:: img/geonode_oauth2_006.png - :align: center - -Provide the connection details - -.. note:: - *It is* :guilabel:`Important` *that the URL ends with* :guilabel:`/gs/ows` - -When finished click on “+” in order to add a new auth configuration - -.. figure:: img/geonode_oauth2_007.png - :align: center - -Provide the needed information as shown below: - - - Name: *any descriptive string* - - Type: *OAuth2 authentication* - - Grant Flow: *Authorization Code* - - Request URL: *must end with* :guilabel:`/o/authorize/` - - Token URL and Refresh URL: *must end with* :guilabel:`/o/token/` - - Redirect URL: *must match with the one defined on GeoNode above* - - Client ID and Client Secret: *must match with the one defined on GeoNode above* - - Scopes: *openid write* - - Enable the persistent Token Session via Headers - -.. figure:: img/geonode_oauth2_008.png - :align: center - -Save and click on :guilabel:`“Connect”`. QGis will redirect you on a browser page asking to GeoNode to authenticate. Approve the Claims and go back to QGis. - -.. figure:: img/geonode_oauth2_009.png - :align: center - -Remove Saved Token Sessions From QGis and Login with another User -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Edit the QGis configuration - -.. figure:: img/geonode_oauth2_010.png - :align: center - -Click on the :guilabel:`“pencil”` - -.. figure:: img/geonode_oauth2_011.png - :align: center - -Clean up the saved :guilabel:`Tokens` and save - -.. figure:: img/geonode_oauth2_012.png - :align: center - -Try to connect again. \ No newline at end of file diff --git a/usage/users_groups/contacting_users.rst b/usage/users_groups/contacting_users.rst deleted file mode 100644 index 54c4716d..00000000 --- a/usage/users_groups/contacting_users.rst +++ /dev/null @@ -1,60 +0,0 @@ -Contacting other users -====================== - -| GeoNode allows you to communicate by message with other registered users and groups. -| To send a message to some user and/or groups you can follow the link :guilabel:`Message User` from your *Profile* page (see :ref:`editing-profile`) or from the *Profile* details page (see the previous section :ref:`user-info`) of that user. - -.. figure:: img/write_message.png - :align: center - - *Send message to users and groups* - -| Insert your content, type a subject and click on :guilabel:`Send message` to send the message to the users and groups you have selected. -| You will be redirected to the *Conversation* details page related to the subject. - -.. figure:: img/sent_message.png - :align: center - - *Your message* - -The Inbox page --------------- - -You can view your conversations in your *Inbox* page, reachable through the :guilabel:`Back to inbox` button (see the picture above) or from the :guilabel:`Inbox` link of the user menu. - -.. figure:: img/inbox_link.png - :align: center - - *Inbox link* - -The picture below shows how your *Inbox* page should look like. - -.. figure:: img/inbox_page.png - :align: center - - *Inbox page* - -In *Inbox* all the unread messages are listed. You haven't received any message yet so your *Inbox* is empty. If you switch to the *All* tab you can see all the conversations you are involved in. - -.. figure:: img/all_your_conversations.png - :align: center - - *All your conversations* - -When some user send a reply to your message your *Inbox* shows it, see the picture below for an example. - -.. figure:: img/reply_inbox.png - :align: center - - *A reply to your message* - -You can open the *Conversation* details by clicking on the *Subject* link. - -.. figure:: img/conversation.png - :align: center - - *Conversation details* - -As you can see in the picture above, in the *Conversation* page you have the ability to write a quick reply. Type your message in the text box and click on :guilabel:`Send Reply` to do that. - -In the *Inbox* page there is also the :guilabel:`Create Message` button that provides you a quick link to the message creation form. diff --git a/usage/users_groups/index.rst b/usage/users_groups/index.rst deleted file mode 100644 index bbf640da..00000000 --- a/usage/users_groups/index.rst +++ /dev/null @@ -1,29 +0,0 @@ -Interacting with Users and Groups -================================= - -The GeoNode platform allows you to communicate by message with other GeoNode users and groups of users. - -| You can also invite external users to join your GeoNode. In order to do that, click on :guilabel:`Invite Users` in the *Profile* page (see :ref:`editing-profile`) or in the :guilabel:`About` menu in the *Home* page. -| You can invite your contacts typing their email addresses in the input field as shown in the picture below. Click on :guilabel:`Submit` to perform the action. - -.. figure:: img/invite_users.png - :align: center - - *Invite users to join GeoNode* - -A message will confirm that invitations have been correctly sent. - -.. figure:: img/invite_users_confirm.png - :align: center - - *Invitations confirm message* - -.. note:: You can invite more than one user at the same time by typing the email addresses inline with a semi-colon separator. - -The next sections will show you how to view information about other users and how to contact them. - -.. toctree:: - :maxdepth: 1 - - users_info - contacting_users diff --git a/usage/users_groups/users_info.rst b/usage/users_groups/users_info.rst deleted file mode 100644 index 1cae9eff..00000000 --- a/usage/users_groups/users_info.rst +++ /dev/null @@ -1,54 +0,0 @@ -.. _user-info: - -Viewing other users information -=============================== - -| Once your account is created, you can view other accounts on the system. -| To see information about other users on the system, click the :guilabel:`People` link of the :guilabel:`About` menu in *Home* page. - -.. figure:: img/people_link.png - :align: center - - *About menu - People link* - -You will see a list of users registered on the system. - -.. figure:: img/registered_users_list.png - :align: center - - *List of the registered users* - -| The *Search* tool is very useful in case of many registered users, type the name of the user you are looking for in the input text field to filter the users list. -| Select a user and click on its *username* to access to the user details page. - -.. figure:: img/user_details.png - :align: center - - *User details* - -In this page the main information about the user are shown: personal information (organization) and the resources the user owns (datasets, maps, documents and other apps). - -Through the :guilabel:`User Activities` link, in right side of the page, it is possible to visualize all the activities the user has been done. - -.. figure:: img/user_activities.png - :align: center - - *User activities* - -The :guilabel:`Message User` link lets you to contact other users, see the next section to read more about that. - -It is also possible, in GeoNode, to see the recent activities of all users through the :guilabel:`Recent Activities` link of the user menu. - -.. figure:: img/recent_activities_link.png - :align: center - - *Recent Activities link* - -In the picture below an example. - -.. figure:: img/recent_activities.png - :align: center - - *Recent Activities* - -As you can see, you can decide whether to see only the activities related to datasets or those related to maps or comments by switching the tabs.