Add some small pages

Author: flyingsilverfin

new_maintenance_operation/modules/ROOT/pages/database-export-import.adoc

@@ -1,4 +1,101 @@
 = Database Export & Import
 
-[placeholder]
-Export and import TypeDB databases. 
+Use TypeDB's built-in export functionality to export a database as a complete TypeQL schema file and a binary containing the instances in the database.
+
+The built-in import functionality can be used to create a database from an exported data binary and a complete schema.
+
+Exported binary data is designed to be identical across any version of TypeDB. This means you can reload your database in versions TypeDB that are no longer compatible on the data layer (such as moving from TypeDB 2.x to 3.x).
+
+[IMPORTANT]
+====
+This feature is introduced in TypeDB 3.4.0. If you want to export or import a database from earlier versions of TypeDB 3.x, simply update your server and client and follow this manual.
+
+You can import your databases exported from TypeDB 2.x following xref:{page-version}@manual::migration/2_to_3/process.adoc[this guide].
+====
+
+== Understanding export and import
+
+TypeDB database can be represented as two files:
+
+[#_files]
+1. A text file with its TypeQL schema description: a complete `define` query for the whole schema.
+2. A binary file with its data.
+
+An exported database can be imported into another version of TypeDB using these two files.
+This is the best tool you can use to migrate a database between two incompatible versions of TypeDB (e.g., a migration xref:{page-version}@manual::migration/2_to_3/process.adoc[from TypeDB 2.x to TypeDB 3.x]).
+
+Both operations are performed through the network, but the server and the client can be used on the same host.
+
+Examples below use xref:{page-version}@new_home::install/console-cli.adoc[TypeDB Console] as the most accessible client for these operations.
+Please check the xref:{page-version}@new_reference::typedb-grpc-drivers/index.html[gRPC Drivers API] to explore available programmatic commands.
+
+[#_export]
+== Export
+
+Database export allows a client to download xref:#_files[database schema and data files] from a TypeDB server for future import to the same or higher TypeDB version.
+
+The files are created on the client side.
+The exported binary file might be up to two times larger than the original database.
+If the client connects to a remote host, the server's disk remains unchanged, thus introducing zero risks to the stability of the server.
+Please ensure the client's disk has enough free capacity to store a file of this size.
+
+While the database data is being exported, parallel transactions are allowed, but none of them will affect the exported data thanks to TypeDB's xref:{page-version}@new_core_concepts::typedb/transactions.adoc[transactionality].
+However, the database will not be available for operations such as deletion.
+
+[NOTE]
+====
+The files can be incomplete and corrupted if the export operation is interrupted by user. Please restart and obtain a complete data file in one attempt.
+====
+
+=== Using TypeDB Console
+
+1. Make sure that your TypeDB Server is running.
+
+2. Follow xref:{page-version}@new_tools::console.adoc[Console page] to set up a connection
+
+3. Execute `database export database-name schema.typeql data.typedb`, where
+
+- `database-name` is the name of the exported database;
+- `schema.typeql` is the exported schema definition file path;
+- `data.typedb` is the exported data file path.
+
+4. The result (either `Successfully exported database` or an error) will be printed to the Console.
+
+== Import
+
+Database import allows a client to upload previously exported xref:#_files[database schema and data files] to a TypeDB server.
+
+For your safety, the imported database should not exist on the target server.
+It is possible to assign any new name to the imported database.
+
+[NOTE]
+====
+The database will be incomplete (and thus not recommended for usage) if the import operation is interrupted.
+It is expected to be automatically removed from the disk, but a small number of unexpected technical issues can prevent this from happening.
+A server restart will resolve any related issues.
+====
+
+=== Using TypeDB Console
+
+1. Make sure that your TypeDB Server is running.
+
+2. Follow xref:{page-version}@new_tools::console.adoc[Console page] to set up a connection
+
+3. Execute `database import database-name schema.typeql data.typedb`, where
+
+- `database-name` is the new name of the imported database;
+- `schema.typeql` is the imported schema definition file path;
+- `data.typedb` is the imported data file path.
+
+4. The result (either `Successfully imported database` or an error) will be printed to the Console.
+
+== Troubleshooting
+
+Follow this checklist if any errors occur:
+
+* Ensure that you use xref:{page-version}@new_tools::console.adoc#version-compatiblity[compatible] TypeDB Server and TypeDB Console.
+* Ensure that the correct file paths were specified for the operations.
+* If you import a TypeDB 2.x database, some errors may be expected.
+Check out xref:{page-version}@manual::migration/2_to_3/process.adoc[this page] for a detailed guide.
+
+If any errors persist, please join one of our communities and ask for assistance.

new_maintenance_operation/modules/ROOT/pages/troubleshooting.adoc

@@ -0,0 +1,31 @@
+= Connecting
+
+This page contains the most common issues that may be encountered during development with respect to connecting to a TypeDB server.
+
+== Connection refused
+
+[source]
+----
+$ ./typedb console --address localhost:1729 --username admin --password password
+tcp connect error: Connection refused (os error 61)
+----
+
+Receiving this error message from Console or Driver indicates that they are unable to establish a connection to the server for some reason. The most common reason could be:
+
+- The server is either not running for some reason
+- The server is configured at a different host or port to what the client is attempting to connect to
+- The connection is blocked by a firewall
+
+== TLS errors
+
+[source]
+----
+$ ./typedb console --address https://localhost:1729 --username admin --password password --tls-disabled
+Connecting to HTTPS without TLS enabled
+----
+
+Receiving this error message from Console or Driver indicates that TLS is not properly configured from the Console / Driver side.
+
+When using TLS encryption, ensure it is configured on both the server and the driver. **Drivers connecting to TLS-secured servers must use the `https://` prefix.**
+
+Similarly, when disabling TLS encryption, ensure both server and driver are configured to not use TLS, and the driver connects without an `https://` prefix.

new_maintenance_operation/modules/ROOT/pages/troubleshooting/connection.adoc

@@ -0,0 +1,39 @@
+= Debugging queries
+
+This page summarizes general steps and ideas for debugging TypeDB queries.
+
+== Locating errors
+
+When your query returns an error, the TypeDB server will report a stacktrace of error messages; you will find the most relevant message on the top of that trace.
+
+* In the case of a syntax, the error message will indicate the line and character position where the error occurred.
+
+* In other cases, the error message should give an indication of which logical conflicts caused the error in the database. A natural debugging approach is to isolate the part of the query that you believe caused that conflict, and then make the necessary changes before returning to your larger query.
+
+* Unexpected pipeline result. When a query results in an unexpected output or data write, the simplest way to debug the query is to take advantage of TypeQL's compositionality. Simply remove query clauses to inspect intermediate query operations when debugging a pipeline.
+
+* Unexpected clause result. To debug an individual clause, again you can take advantage of TypeQL's compositionality. Split the clause into smaller clauses and execute them one by one to see their results, pinpointing the part that misbehaves.
+
+== Dissecting stages
+
+In complex pipelines, it is often useful to truncate your query to specific stages. This allows you to inspect "`intermediate results`". You can output results for specific variables of those stages using the `select` operator. It is also often useful to simply output the number of results of (a part of) your pipeline via the `reduce` operator:
+
+[,typeql]
+----
+...                # incoming pipeline
+reduce $c = count; # outputs answer count of incoming pipeline
+----
+
+== Limiting queries
+
+If query executes indefinitely and does not return results, a sanity check that it can potentially produce at least one result in a reasonable time is always a good idea:
+
+[,typeql]
+----
+...      # incoming pipeline
+limit 1; # stops the execution after consuming one answer
+----
+
+== New issues
+
+Send us a message on https://discord.com/channels/665254494820368395/983769458269114408[Discord]!

new_maintenance_operation/modules/ROOT/pages/troubleshooting/debugging-queries.adoc

@@ -0,0 +1,39 @@
+= Troubleshooting
+
+This section provides general recommendations and solutions to common issues faced while interacting with TypeDB.
+
+Please refer to the section closest to your problem.
+If you cannot find a solution, do not hesitate to send us a message on https://discord.com/channels/665254494820368395/983769458269114408[Discord]!
+
+[cols-2]
+--
+.xref:{page-version}@new_maintenance_operation::troubleshooting/installation.adoc[]
+[.clickable]
+****
+Addressing setup issues.
+****
+
+.xref:{page-version}@new_maintenance_operation::troubleshooting/connection.adoc[]
+[.clickable]
+****
+Resolve connection issues.
+****
+
+.xref:{page-version}@new_maintenance_operation::troubleshooting/stopping-queries.adoc[]
+[.clickable]
+****
+Query takes too long.
+****
+
+.xref:{page-version}@new_maintenance_operation::troubleshooting/optimising-queries.adoc[]
+[.clickable]
+****
+Optimize your queries for the best performance.
+****
+
+.xref:{page-version}@new_maintenance_operation::troubleshooting/debugging-queries.adoc[]
+[.clickable]
+****
+Tips for query debugging.
+****
+--

new_maintenance_operation/modules/ROOT/pages/troubleshooting/index.adoc

@@ -0,0 +1,26 @@
+= Installation
+
+== GLIBC not found
+
+*The server does not start with error `version 'GLIBC_2.25' not found`*
+
+TypeDB for Linux currently requires glibc v2.25 or later. Updating your Linux distribution may resolve this issue.
+
+Supported versions of some popular Linux distributions:
+
+* Debian 10 buster or newer,
+* Ubuntu 17.10 artful or newer,
+* RHEL-8.10 or newer,
+* Fedora 26 or newer,
+* CentOS 8.5 or newer,
+* Amazon Linux 2,
+* Amazon Linux 2023.
+
+If your distribution is not listed above, you can consult https://distrowatch.com/[DistroWatch.com] to find out if
+the newest version ships with glibc >=2.25.
+
+TIP: Alpine Linux does not ship with glibc by default. Additional setup is required to run TypeDB server on Alpine Linux.
+
+== New issues
+
+Send us a message on https://discord.com/channels/665254494820368395/983762496798204024[Discord]!

new_maintenance_operation/modules/ROOT/pages/troubleshooting/installation.adoc

@@ -0,0 +1,119 @@
+= Optimizing queries
+
+== TypeDB's optimizer
+
+The built-in optimizer of TypeDB will decide the optimal order of evaluation of your query on a *per-stage* basis, while also respecting the boundaries of subqueries (created via `or`, `not`, or function calls). This gives user direct control over the evaluation order in their queries. For example, the query
+
+[,typeql]
+----
+match
+  $x isa user;
+  $x has username "john_1";
+----
+and the query pipeline
+
+[,typeql]
+----
+match
+  $x isa user;
+match
+  $x has username "john_1";
+----
+are semantically equivalent (i.e., they yield the same results), but the first query will likely run much faster than the second query.
+
+1. In the first query, the optimizer will optimize evaluation order of the first (and only) `match` stage: it'll decide that it is most efficient to first look up all entities with `username` equal to `"john_1"` and then filter the result to contain only `user` entities.
+2. The second query comprises two stages, to be executed one after the other: the database will first retrieve _all_ users, thus constructing a very large set of answers. Only in the second stage to we filter this set to contain only users with `username` equal to `"john_1"`.
+
+In other words, the second query suffers from have _large intermediate results_.
+
+== Avoiding large intermediate results
+
+A standard method for avoiding large intermediate results is to declare/enforce constraints as early as possible: this is also known as *predicate-pushdown*. The above queries provide an example of this principle.
+
+In TypeQL predicate-pushdown is extended to the principle of *data-predicate-locality*: create data only at the stage where you need it, jointly with all applicable predicates. Pipelines give you fine-grained control for ensure that data is created where you need it. For example, a schematic pipeline of the form:
+[,typeql]
+----
+<match_data($a,$b)>
+<processing($b)>
+<processing($a,$b)>
+----
+could be rewritten to be of the form
+[,typeql]
+----
+<match_data($b)>
+<processing($b)>
+<match_data($a,$b)>
+<processing($a,$b)>
+----
+This defers matching of data to-be-stored in the variables `$a` until after we have finished processing `$b` (assuming the latter is fully independent of results for `$a`).
+
+== Optimizing branches
+
+Another source of sub-optimality in queries comes from the usage of `or`-branches, and could be called *pattern-pushdown*. TypeDB performs minimal transformation of your `or`-branches, assuming that each disjunction is meant to be a *branching point* of your query, at which a set of nested subqueries is retrieved.
+
+Thus, when working with branches, we should ensure that data is retrieved a minimal number of times. Schematically, this can be understood as follows: the query
+[,typeql]
+----
+match
+  <root-pattern>;
+  { <expensive-pattern1>; <cond1>; }
+  or { <expensive-pattern1>; <cond2>; }
+  or { <expensive-pattern2>; };
+----
+is equivalent to the pattern
+[,typeql]
+----
+match
+  <root-pattern>;
+  {
+    <expensive-pattern1>;
+    { <cond1>; } or { <cond2>; };
+  } or {
+    <expensive-pattern2>;
+  };
+----
+We expect the second query to run faster, as it evaluates the `<expensive-pattern1>` only in one subquery (which itself has two further subqueries).
+
+== Optimizing negations
+
+Negations, especially nested negations, can be extremely expensive query operates: indeed, to ensure that something does not exist, we usually must search the entire to domain of possibilities. It is therefore paramount to reduce the "`size of the domain`" prior to using negation. For example, the query
+[,typeql]
+----
+match
+  $x isa user;
+  not {
+    $y isa user;
+    not { friendship ($x, $y) };
+  };
+limit 1;
+----
+does the following:
+
+> Find me a user `$x` for which there does not exist a user `$y` that is not friends with `$y`.
+
+This can equivalently be achieved by the following query:
+[,typeql]
+----
+match
+  $x isa user;
+  friendship ($x, $y);
+reduce $f_count = count groupby $x;
+sort $f_count desc;
+limit 1;
+match $f_count == user_count(); # function returning user count
+----
+While the first query needs to consider data whose size is quadratic in the number of users, the second query considers data linear in the size of friendships.
+
+== Optimizing recursive functions
+
+[NOTE]
+====
+Coming soon.
+====
+
+== Roadmap
+
+[NOTE]
+====
+While the above advice is timeless in any case, future versions of TypeDB may apply more aggressive query transformation techniques that will shift the burden of optimizing nested subqueries from the user further to the database.
+====