diff --git a/site/content/amp/deployments/_index.md b/site/content/amp/deployments/_index.md index d8e67ac5ec..b331ed8163 100644 --- a/site/content/amp/deployments/_index.md +++ b/site/content/amp/deployments/_index.md @@ -273,7 +273,7 @@ attached to your role. Read more about [roles and permissions](../security-and-a ## How to connect a driver to your deployment -[ArangoDB drivers](../../arangodb/3.12/develop/drivers/_index.md) allow you to use your AMP +[ArangoDB drivers](../../ecosystem/drivers/) allow you to use your AMP deployment as a database system for your applications. Drivers act as interfaces between different programming languages and ArangoDB, which enable you to connect to and manipulate ArangoDB deployments from within compiled programs diff --git a/site/content/amp/security-and-access-control/x-509-certificates.md b/site/content/amp/security-and-access-control/x-509-certificates.md index 21e1e8493b..f17cf87e97 100644 --- a/site/content/amp/security-and-access-control/x-509-certificates.md +++ b/site/content/amp/security-and-access-control/x-509-certificates.md @@ -152,7 +152,7 @@ unique ID that is part of your AMP deployment endpoint URL. ## How to connect to your application -[ArangoDB drivers](../../arangodb/3.12/develop/drivers/_index.md), also called connectors, allow you to +[ArangoDB drivers](../../ecosystem/drivers/_index.md), also called connectors, allow you to easily connect AMP deployments to your application. 1. Navigate to **Deployments** and click the **View** button to show the diff --git a/site/content/arangodb/3.10/develop/javascript-api/_index.md b/site/content/arangodb/3.10/develop/javascript-api/_index.md index ac10a520f7..3ad9a70302 100644 --- a/site/content/arangodb/3.10/develop/javascript-api/_index.md +++ b/site/content/arangodb/3.10/develop/javascript-api/_index.md @@ -26,7 +26,7 @@ It communicates with the server via the HTTP API. {{< tip >}} The JavaScript API cannot be used in browsers, Node.js, or other JavaScript -environments. You can use the [arangojs driver](../drivers/nodejs.md) instead. +environments. You can use the [arangojs driver](../../develop/drivers/nodejs.md) instead. Note that it has a different interface. {{< /tip >}} diff --git a/site/content/arangodb/3.11/develop/drivers/_index.md b/site/content/arangodb/3.11/develop/drivers/_index.md index cba7ac5de0..6fc7ee3507 100644 --- a/site/content/arangodb/3.11/develop/drivers/_index.md +++ b/site/content/arangodb/3.11/develop/drivers/_index.md @@ -37,7 +37,7 @@ language. - [Tutorial](go.md#tutorial) - Repository: [github.com/arangodb/go-driver](https://github.com/arangodb/go-driver/tree/master/v2) -## Node.js driver +## JavaScript driver The [**ArangoJS driver**](javascript.md) lets you work with ArangoDB in Node.js, using the JavaScript scripting language. You can also use it in web browsers. diff --git a/site/content/arangodb/3.11/develop/integrations/arangodb-datasource-for-apache-spark.md b/site/content/arangodb/3.11/develop/integrations/arangodb-datasource-for-apache-spark.md index 2cbf5ac7ca..01f39ce044 100644 --- a/site/content/arangodb/3.11/develop/integrations/arangodb-datasource-for-apache-spark.md +++ b/site/content/arangodb/3.11/develop/integrations/arangodb-datasource-for-apache-spark.md @@ -25,8 +25,6 @@ This library works with all the non-EOLed [ArangoDB versions](https://www.arango There are several variants of this library, each one compatible with different Spark and Scala versions: -- `com.arangodb:arangodb-spark-datasource-3.3_2.12` (Spark 3.3, Scala 2.12) -- `com.arangodb:arangodb-spark-datasource-3.3_2.13` (Spark 3.3, Scala 2.13) - `com.arangodb:arangodb-spark-datasource-3.4_2.12` (Spark 3.4, Scala 2.12) (compatible with Spark `3.4.2+`) - `com.arangodb:arangodb-spark-datasource-3.4_2.13` (Spark 3.4, Scala 2.13) (compatible with Spark `3.4.2+`) - `com.arangodb:arangodb-spark-datasource-3.5_2.12` (Spark 3.5, Scala 2.12) @@ -39,6 +37,8 @@ The following variants are no longer supported: - `com.arangodb:arangodb-spark-datasource-3.1_2.12` (Spark 3.1, Scala 2.12) - `com.arangodb:arangodb-spark-datasource-3.2_2.12` (Spark 3.2, Scala 2.12) - `com.arangodb:arangodb-spark-datasource-3.2_2.13` (Spark 3.2, Scala 2.13) +- `com.arangodb:arangodb-spark-datasource-3.3_2.12` (Spark 3.3, Scala 2.12) +- `com.arangodb:arangodb-spark-datasource-3.3_2.13` (Spark 3.3, Scala 2.13) Since version `1.7.0`, due to [breaking changes](https://github.com/apache/spark/commit/ad29290a02fb94a958fd21e301100338c9f5b82a#diff-b25c8acff88c1b4850c6642e80845aac4fb882c664795c3b0aa058e37ed732a0L42-R52) in Spark `3.4.2`, `arangodb-spark-datasource-3.4` is not compatible anymore with Spark versions `3.4.0` and `3.4.1`. diff --git a/site/content/arangodb/3.11/develop/javascript-api/_index.md b/site/content/arangodb/3.11/develop/javascript-api/_index.md index fb7146ff1b..d797cb71f1 100644 --- a/site/content/arangodb/3.11/develop/javascript-api/_index.md +++ b/site/content/arangodb/3.11/develop/javascript-api/_index.md @@ -26,7 +26,7 @@ It communicates with the server via the HTTP API. {{< tip >}} The JavaScript API cannot be used in browsers, Node.js, or other JavaScript -environments. You can use the [arangojs driver](../drivers/javascript.md) instead. +environments. You can use the [arangojs driver](../../develop/drivers/javascript.md) instead. Note that it has a different interface. {{< /tip >}} diff --git a/site/content/arangodb/3.12/aql/functions/document-object.md b/site/content/arangodb/3.12/aql/functions/document-object.md index a7272366aa..70496ccca7 100644 --- a/site/content/arangodb/3.12/aql/functions/document-object.md +++ b/site/content/arangodb/3.12/aql/functions/document-object.md @@ -477,7 +477,7 @@ Only [`HAS()`](#has) can differentiate between an attribute being absent and hav a stored `null` value. An empty object `{}` will match all documents. Be careful not to ask for all -documents accidentally. For example, the [arangojs](../../develop/drivers/javascript.md) driver +documents accidentally. For example, the [arangojs](../../../../ecosystem/drivers/javascript.md) driver skips attributes with a value of `undefined`, turning `{attr: undefined}` into `{}`. {{< info >}} diff --git a/site/content/arangodb/3.12/aql/how-to-invoke-aql/_index.md b/site/content/arangodb/3.12/aql/how-to-invoke-aql/_index.md index b44a6ed75a..1276ab8755 100644 --- a/site/content/arangodb/3.12/aql/how-to-invoke-aql/_index.md +++ b/site/content/arangodb/3.12/aql/how-to-invoke-aql/_index.md @@ -11,8 +11,8 @@ You can execute AQL queries using different interfaces: - The web interface - The `db` object of the JavaScript API (either in arangosh or in a Foxx service) - The raw HTTP REST API -- Through a [driver](../../develop/drivers/_index.md) or - [integration](../../develop/integrations/_index.md) as an abstraction over the +- Through a [driver](../../../../ecosystem/drivers/_index.md) or + [integration](../../../../ecosystem/integrations/_index.md) as an abstraction over the HTTP REST API There are always calls to the server's API under the hood, but the web interface, diff --git a/site/content/arangodb/3.12/components/tools/arango-datasets.md b/site/content/arangodb/3.12/components/tools/arango-datasets.md index a24903550f..10a4626f48 100644 --- a/site/content/arangodb/3.12/components/tools/arango-datasets.md +++ b/site/content/arangodb/3.12/components/tools/arango-datasets.md @@ -27,7 +27,7 @@ You can find the source code repository of the module on GitHub: Once you have installed the `arango-datasets` package, you can use it to download and import datasets into your deployment with `arango_datasets.Datasets`. -The `Datasets` constructor requires a valid [python-arango](../../develop/drivers/python.md) +The `Datasets` constructor requires a valid [python-arango](../../../../ecosystem/drivers/python.md) database object as input. It defines the target deployment, database, and credentials to load a dataset. diff --git a/site/content/arangodb/3.12/develop/drivers/java/_index.md b/site/content/arangodb/3.12/develop/drivers/java/_index.md deleted file mode 100644 index dc7d37d9fd..0000000000 --- a/site/content/arangodb/3.12/develop/drivers/java/_index.md +++ /dev/null @@ -1,480 +0,0 @@ ---- -title: ArangoDB Java driver -menuTitle: Java driver -weight: 10 -description: '' ---- -The official ArangoDB Java Driver. - -- Repository: -- [Code examples](https://github.com/arangodb/arangodb-java-driver/tree/main/test-non-functional/src/test/java/example) -- [Reference](reference-version-7/_index.md) (driver setup, serialization, changes in version 7) -- [JavaDoc](https://www.javadoc.io/doc/com.arangodb/arangodb-java-driver/latest/index.html) (generated reference documentation) -- [ChangeLog](https://github.com/arangodb/arangodb-java-driver/blob/main/ChangeLog.md) - -## Supported versions - -Version 7 is the latest supported and actively developed release. - -The driver is compatible with all supported stable versions of ArangoDB server, see -[Product Support End-of-life Announcements](https://arangodb.com/subscriptions/end-of-life-notice/). - -The driver is compatible with JDK 8 and higher versions. - -{{< warning >}} -Version 6 reached End of Life (EOL) and is not actively developed anymore. -Upgrading to version 7 is recommended. - -The API changes between version 6 and 7 are documented in -[Changes in version 7](reference-version-7/changes-in-version-7.md). -{{< /warning >}} - -## Project configuration - -To use the ArangoDB Java driver, you need to import `arangodb-java-driver` as a -library into your project. This is described below for the popular Java build -automation systems Maven and Gradle. - -### Maven - -To add the driver to your project with Maven, add the following code to your -`pom.xml`: - -```xml - - - com.arangodb - arangodb-java-driver - 7.x.x - - -``` - -Substitute `7.x.x` with the latest driver version. - -### Gradle - -To add the driver to your project with Gradle, add the following code to your -`build.gradle` (substitute `7.x.x` with the latest driver version): - -```groovy -repositories { - mavenCentral() -} - -dependencies { - implementation 'com.arangodb:arangodb-java-driver:7.x.x' -} -``` - -## Tutorial - -### Connect to ArangoDB - -Let's configure and open a connection to ArangoDB. The default connection is to -`127.0.0.1:8529`. Change the connection details to point to your specific instance. - -```java -ArangoDB arangoDB = new ArangoDB.Builder() - .host("localhost", 8529) - .user("root") - .password("") - .build(); -``` - -For more connections options and details, see -[Driver setup](reference-version-7/driver-setup.md). - -### Create a database - -Let's create a new database: - -```java -ArangoDatabase db = arangoDB.db("mydb"); -System.out.println("Creating database..."); -db.create(); -``` - -### Create a collection - -Now let's create our first collection: - -```java -ArangoCollection collection = db.collection("firstCollection"); -System.out.println("Creating collection..."); -collection.create(); -``` - -### Create a document - -Let's create a document in the collection. Any object can be added as a document -to the database and be retrieved from the database as an object. - -This example uses the `BaseDocument` class, provided with the driver. The -attributes of the document are stored in a map as `key`/`value` pair: - -```java -String key = "myKey"; -BaseDocument doc = new BaseDocument(key); -doc.addAttribute("a", "Foo"); -doc.addAttribute("b", 42); -System.out.println("Inserting document..."); -collection.insertDocument(doc); -``` - -Some details you should know about the code: - -- The document key is passed to the `BaseDocument` constructor -- The `addAttribute()` method puts a new key/value pair into the document -- Each attribute is stored as a single key value pair in the document root - -### Read a document - -Read the created document: - -```java -System.out.println("Reading document..."); -BaseDocument readDocument = collection.getDocument(key, BaseDocument.class); -System.out.println("Key: " + readDocument.getKey()); -System.out.println("Attribute a: " + readDocument.getAttribute("a")); -System.out.println("Attribute b: " + readDocument.getAttribute("b")); -``` - -After executing this program, the console output should be: - -```text -Key: myKey -Attribute a: Foo -Attribute b: 42 -``` - -Some details you should know about the code: - -- The `getDocument()` method reads the stored document data and deserializes it - into the given class (`BaseDocument`) - -### Create a document from Jackson JsonNode - -You can also create a document from a Jackson -[JsonNode](https://fasterxml.github.io/jackson-databind/javadoc/2.13/com/fasterxml/jackson/databind/JsonNode.html) -object: - -```java -System.out.println("Creating a document from Jackson JsonNode..."); -String keyJackson = "myJacksonKey"; -JsonNode jsonNode = JsonNodeFactory.instance.objectNode() - .put("_key", keyJackson) - .put("a", "Bar") - .put("b", 53); -System.out.println("Inserting document from Jackson JsonNode..."); -collection.insertDocument(jsonNode); -``` - -### Read a document as Jackson JsonNode - -You can also read a document as a Jackson -[JsonNode](https://fasterxml.github.io/jackson-databind/javadoc/2.13/com/fasterxml/jackson/databind/JsonNode.html): - -```java -System.out.println("Reading document as Jackson JsonNode..."); -JsonNode readJsonNode = collection.getDocument(keyJackson, JsonNode.class); -System.out.println("Key: " + readJsonNode.get("_key").textValue()); -System.out.println("Attribute a: " + readJsonNode.get("a").textValue()); -System.out.println("Attribute b: " + readJsonNode.get("b").intValue()); -``` - -After executing this program, the console output should be: - -```text -Key: myKey -Attribute a: Bar -Attribute b: 53 -``` - -Some details you should know about the code: - -- The `getDocument()` method returns the stored document as instance of - `com.fasterxml.jackson.databind.JsonNode`. - -### Create a document from JSON String - -You can also create a document from raw JSON string: - -```java -System.out.println("Creating a document from JSON String..."); -String keyJson = "myJsonKey"; -RawJson json = RawJson.of("{\"_key\":\"" + keyJson + "\",\"a\":\"Baz\",\"b\":64}"); -System.out.println("Inserting document from JSON String..."); -collection.insertDocument(json); -``` - -### Read a document as JSON String - -You can also read a document as raw JSON string: - -```java -System.out.println("Reading document as JSON String..."); -RawJson readJson = collection.getDocument(keyJson, RawJson.class); -System.out.println(readJson.get()); -``` - -After executing this program, the console output should be: - -```text -{"_key":"myJsonKey","_id":"firstCollection/myJsonKey","_rev":"_e0nEe2y---","a":"Baz","b":64} -``` - -### Update a document - -Let's update the document: - -```java -doc.addAttribute("c", "Bar"); -System.out.println("Updating document ..."); -collection.updateDocument(key, doc); -``` - -### Read the document again - -Let's read the document again: - -```java -System.out.println("Reading updated document ..."); -BaseDocument updatedDocument = collection.getDocument(key, BaseDocument.class); -System.out.println("Key: " + updatedDocument.getKey()); -System.out.println("Attribute a: " + updatedDocument.getAttribute("a")); -System.out.println("Attribute b: " + updatedDocument.getAttribute("b")); -System.out.println("Attribute c: " + updatedDocument.getAttribute("c")); -``` - -After executing this program, the console output should look like this: - -```text -Key: myKey -Attribute a: Foo -Attribute b: 42 -Attribute c: Bar -``` - -### Delete a document - -Let's delete a document: - -```java -System.out.println("Deleting document ..."); -collection.deleteDocument(key); -``` - -### Execute AQL queries - -First, you need to create some documents with the name `Homer` in the -collection called `firstCollection`: - -```java -for (int i = 0; i < 10; i++) { - BaseDocument value = new BaseDocument(String.valueOf(i)); - value.addAttribute("name", "Homer"); - collection.insertDocument(value); -} -``` - -Get all documents with the name `Homer` from the collection using an AQL query -and iterate over the results: - -```java -String query = "FOR t IN firstCollection FILTER t.name == @name RETURN t"; -Map bindVars = Collections.singletonMap("name", "Homer"); -System.out.println("Executing read query ..."); -ArangoCursor cursor = db.query(query, BaseDocument.class, bindVars); -cursor.forEach(aDocument -> System.out.println("Key: " + aDocument.getKey())); -``` - -After executing this program, the console output should look something like this: - -```text -Key: 1 -Key: 0 -Key: 5 -Key: 3 -Key: 4 -Key: 9 -Key: 2 -Key: 7 -Key: 8 -Key: 6 -``` - -Some details you should know about the code: - -- The AQL query uses the placeholder `@name` that has to be bound to a value -- The `query()` method executes the defined query and returns an `ArangoCursor` - with the given class (here: `BaseDocument`) -- The order of is not guaranteed - -### Delete documents with AQL - -Delete previously created documents: - -```java -String query = "FOR t IN firstCollection FILTER t.name == @name " - + "REMOVE t IN firstCollection LET removed = OLD RETURN removed"; -Map bindVars = Collections.singletonMap("name", "Homer"); -System.out.println("Executing delete query ..."); -ArangoCursor cursor = db.query(query, BaseDocument.class, bindVars); -cursor.forEach(aDocument -> System.out.println("Removed document " + aDocument.getKey())); -``` - -After executing this program, the console output should look something like this: - -```text -Removed document: 1 -Removed document: 0 -Removed document: 5 -Removed document: 3 -Removed document: 4 -Removed document: 9 -Removed document: 2 -Removed document: 7 -Removed document: 8 -Removed document: 6 -``` - -### Learn more - -- Have a look at the [AQL documentation](../../../aql/) to lear about the - query language -- See [Serialization](reference-version-7/serialization.md) for details about - user-data serde -- For the full reference documentation, see - [JavaDoc](https://www.javadoc.io/doc/com.arangodb/arangodb-java-driver/latest/index.html) - -## GraalVM Native Image - -The driver supports GraalVM Native Image compilation. -To compile with `--link-at-build-time` when `http-protocol` module is present in -the classpath, additional substitutions are required for transitive dependencies -`Netty` and `Vert.x`. See this -[example](https://github.com/arangodb/arangodb-java-driver/tree/main/test-functional/src/test-default/java/graal) -for reference. Such substitutions are not required when compiling the shaded driver. - -### Framework compatibility - -The driver can be used in the following frameworks that support -GraalVM Native Image generation: - -- [Quarkus](https://quarkus.io), see [arango-quarkus-native-example](https://github.com/arangodb-helper/arango-quarkus-native-example) -- [Helidon](https://helidon.io), see [arango-helidon-native-example](https://github.com/arangodb-helper/arango-helidon-native-example) -- [Micronaut](https://micronaut.io), see [arango-micronaut-native-example](https://github.com/arangodb-helper/arango-micronaut-native-example) - -## ArangoDB Java Driver Shaded - -A shaded variant of the driver is also published with -Maven coordinates: `com.arangodb:arangodb-java-driver-shaded`. - -It bundles and relocates the following packages: -- `com.fasterxml.jackson` -- `com.arangodb.jackson.dataformat.velocypack` -- `io.vertx` -- `io.netty` - -Note that the **internal serde** internally uses Jackson classes from -`com.fasterxml.jackson` that are relocated to `com.arangodb.shaded.fasterxml.jackson`. -Therefore, the **internal serde** of the shaded driver is not compatible with -Jackson annotations and modules from package`com.fasterxml.jackson`, but only -with their relocated variants. In case the **internal serde** is used as -**user-data serde**, the annotations from package `com.arangodb.serde` can be -used to annotate fields, parameters, getters and setters for mapping values -representing ArangoDB documents metadata (`_id`, `_key`, `_rev`, `_from`, `_to`): -- `@InternalId` -- `@InternalKey` -- `@InternalRev` -- `@InternalFrom` -- `@InternalTo` - -These annotations are compatible with relocated Jackson classes. -Note that the **internal serde** is not part of the public API and could change -in future releases without notice, thus breaking client applications relying on -it to serialize or deserialize user-data. It is therefore recommended also in -this case either: -- using the default user-data serde `JacksonSerde` - (from packages `com.arangodb:jackson-serde-json` or `com.arangodb:jackson-serde-vpack`), or -- providing a custom user-data serde implementation via `ArangoDB.Builder.serde(ArangoSerde)`. - -## Support for extended naming constraints - -The driver supports ArangoDB's **extended** naming constraints/convention, -allowing most UTF-8 characters in the names of: -- Databases -- Collections -- Views -- Indexes - -These names must be NFC-normalized, otherwise the server returns an error. -To normalize a string, use the function -`com.arangodb.util.UnicodeUtils.normalize(String): String`: - -```java -String normalized = UnicodeUtils.normalize("π”Έπ•£π•’π•Ÿπ•˜π• π”»π”Ή"); -``` - -To check if a string is already normalized, use the -function `com.arangodb.util.UnicodeUtils.isNormalized(String): boolean`: - -```java -boolean isNormalized = UnicodeUtils.isNormalized("π”Έπ•£π•’π•Ÿπ•˜π• π”»π”Ή"); -``` - -## Async API - -The asynchronous API is accessible via `ArangoDB#async()`, for example: - -```java -ArangoDB adb = new ArangoDB.Builder() - // ... - .build(); -ArangoDBAsync adbAsync = adb.async(); -CompletableFuture version = adbAsync.getVersion(); -// ... -``` - -Under the hood, both synchronous and asynchronous API use the same internal -communication layer, which has been reworked and re-implemented in an -asynchronous way. The synchronous API blocks and waits for the result, while the -asynchronous one returns a `CompletableFuture<>` representing the pending -operation being performed. -Each asynchronous API method is equivalent to the corresponding synchronous -variant, except for the Cursor API. - -### Async Cursor API - -The Cursor API (`ArangoCursor` and `ArangoCursorAsync`) is intrinsically different, -because the synchronous Cursor API is based on Java's `java.util.Iterator`, which -is an interface only suitable for synchronous scenarios. -On the other side, the asynchronous Cursor API provides a method -`com.arangodb.ArangoCursorAsync#nextBatch()`, which returns a -`CompletableFuture>` and can be used to consume the next -batch of the cursor, for example: - -```java -CompletableFuture> future1 = adbAsync.db() - .query("FOR i IN i..10000", Integer.class); -CompletableFuture> future2 = future1 - .thenCompose(c -> { - List batch = c.getResult(); - // ... - // consume batch - // ... - return c.nextBatch(); - }); -// ... -``` - -## Data Definition Classes - -Classes used to exchange data definitions, in particular classes in the packages -`com.arangodb.entity.**` and `com.arangodb.model.**`, are meant to be serialized -and deserialized internally by the driver. - -The behavior to serialize and deserialize these classes is considered an internal -implementation detail, and as such, it might change without prior notice. -The API with regard to the public members of these classes is kept compatible. diff --git a/site/content/arangodb/3.12/develop/drivers/javascript.md b/site/content/arangodb/3.12/develop/drivers/javascript.md deleted file mode 100644 index 5161b596b2..0000000000 --- a/site/content/arangodb/3.12/develop/drivers/javascript.md +++ /dev/null @@ -1,224 +0,0 @@ ---- -title: ArangoDB JavaScript driver -menuTitle: JavaScript driver -weight: 25 -description: >- - arangojs is the JavaScript driver to access ArangoDB from outside the - database system, primarily with Node.js -aliases: - - nodejs # 3.12 -> 3.12 ---- -The official ArangoDB low-level JavaScript client. - -- Repository: -- Reference: -- [Changelog](https://github.com/arangodb/arangojs/blob/main/CHANGELOG.md) - -{{< info >}} -If you are looking for the ArangoDB JavaScript API in -[Foxx](https://www.arangodb.com/community-server/foxx/) or the `arangosh` -interactive shell, please refer to the documentation about the -[`@arangodb` module](../javascript-api/@arangodb/_index.md) instead. - -The JavaScript driver is **only** meant to be used when accessing ArangoDB from -**outside** the database. -{{< /info >}} - -## Compatibility - -The arangojs driver is compatible with the latest stable version of ArangoDB -available at the time of the driver release and remains compatible with the two -most recent Node.js LTS versions in accordance with the official -[Node.js long-term support schedule](https://github.com/nodejs/LTS). -Versions of ArangoDB that have reached their [end of life](https://arangodb.com/subscriptions/end-of-life-notice/) -by the time of a driver release are explicitly not supported. - -The [_arangoVersion_ option](https://arangodb.github.io/arangojs/latest/types/configuration.ConfigOptions.html) -can be used to tell arangojs to target a specific -ArangoDB version. Depending on the version this will enable or disable certain -methods and change behavior to maintain compatibility with the given version. -The oldest version of ArangoDB supported by arangojs when using this option -is 2.8.0 (using `arangoVersion: 20800`). - -## Versions - -The version number of this driver does not indicate supported ArangoDB versions! - -This driver uses semantic versioning: - -- A change in the bugfix version (e.g. X.Y.0 -> X.Y.1) indicates internal - changes and should always be safe to upgrade. -- A change in the minor version (e.g. X.1.Z -> X.2.0) indicates additions and - backwards-compatible changes that should not affect your code. -- A change in the major version (e.g. 1.Y.Z -> 2.0.0) indicates _breaking_ - changes that require changes in your code to upgrade. - -If you are getting weird errors or functions seem to be missing, make sure you -are using the latest version of the driver and following documentation written -for a compatible version. If you are following a tutorial written for an older -version of arangojs, you can install that version using the `@` -syntax: - -```sh -# for version 9.x.x -yarn add arangojs@9 -# - or - -npm install --save arangojs@9 -``` - -You can find the documentation for each version by clicking on the corresponding -date on the left in -[the list of version tags](https://github.com/arangodb/arangojs/tags). - -## Install - -{{< tabs "js-install" >}} - -{{< tab "Yarn" >}} -```sh -yarn add arangojs -``` -{{< /tab >}} - -{{< tab "NPM" >}} -```sh -npm install --save arangojs -``` -{{< /tab >}} - -{{< tab "From source" >}} -```bash -git clone https://github.com/arangodb/arangojs.git -cd arangojs -npm install -npm run build -``` - -Building natively on Windows is not supported but you can use a virtual Linux -or Linux container. -{{< /tab >}} - -{{< tab "For browsers" >}} -When using modern JavaScript tooling with a bundler and compiler (e.g. Babel), -arangojs can be installed using NPM or Yarn like any other dependency. - -You can also use [jsDelivr CDN](https://www.jsdelivr.com/) during development: - -```js - - -``` -{{< /tab >}} - -{{< /tabs >}} - -## Basic usage example - -Modern JavaScript/TypeScript with async/await and ES Modules: - -```js -import { Database, aql } from "arangojs"; - -const db = new Database(); -const Pokemons = db.collection("my-pokemons"); - -async function main() { - try { - const pokemons = await db.query(aql` - FOR pokemon IN ${Pokemons} - FILTER pokemon.type == "fire" - RETURN pokemon - `); - console.log("My pokemans, let me show you them:"); - for await (const pokemon of pokemons) { - console.log(pokemon.name); - } - } catch (err) { - console.error(err.message); - } -} - -main(); -``` - -Using a different database: - -```js -const db = new Database({ - url: "http://127.0.0.1:8529", - databaseName: "pancakes", - auth: { username: "root", password: "hunter2" }, -}); - -// The credentials can be swapped at any time -db.useBasicAuth("admin", "maplesyrup"); -``` - -Old-school JavaScript with promises and CommonJS: - -```js -var arangojs = require("arangojs"); -var Database = arangojs.Database; - -var db = new Database(); -var pokemons = db.collection("pokemons"); - -db.query({ - query: "FOR p IN @@c FILTER p.type == 'fire' RETURN p", - bindVars: { "@c": "pokemons" }, -}) - .then(function (cursor) { - console.log("My pokemons, let me show you them:"); - return cursor.forEach(function (pokemon) { - console.log(pokemon.name); - }); - }) - .catch(function (err) { - console.error(err.message); - }); -``` - -For AQL, please check out the -[aql template tag](https://arangodb.github.io/arangojs/latest/functions/aql.aql.html) -for writing parametrized AQL queries without making your code vulnerable to -injection attacks. - -## Error responses - -If the server returns an ArangoDB error response, arangojs throws an `ArangoError` -with an `errorNum` property indicating the -[ArangoDB error code](../error-codes.md) and expose the response body -as the response property of the error object. - -For all other errors during the request/response cycle arangojs throws a -`NetworkError` or a more specific subclass thereof and expose the originating -request object as the `request` property of the error object. - -If the server responded with a non-2xx status code, this `NetworkError` is an -`HttpError` with a code property indicating the HTTP status code of the response -and a response property containing the `response` object itself. - -If the error is caused by an exception, the originating exception is available -as the `cause` property of the error object thrown by arangojs. -For network errors, this is often a `TypeError`. - -**Example** - -```js -try { - const info = await db.createDatabase("mydb"); - // database created -} catch (err) { - console.error(err.stack); -} -``` diff --git a/site/content/arangodb/3.12/develop/http-api/_index.md b/site/content/arangodb/3.12/develop/http-api/_index.md index e42c7b49f3..e52e55ee54 100644 --- a/site/content/arangodb/3.12/develop/http-api/_index.md +++ b/site/content/arangodb/3.12/develop/http-api/_index.md @@ -13,7 +13,7 @@ world wide web. All interactions with a server are ultimately carried out via this HTTP API. You can use the API by sending HTTP requests to the server directly, but the -more common way of communicating with the server is via a [database driver](../drivers/_index.md). +more common way of communicating with the server is via a [database driver](../../../../ecosystem/drivers/_index.md). A driver abstracts the complexity of the API away by providing a simple interface for your programming language or environment and handling things like authentication, connection pooling, asynchronous requests, and multi-part replies diff --git a/site/content/arangodb/3.12/develop/integrations/arangodb-datasource-for-apache-spark.md b/site/content/arangodb/3.12/develop/integrations/arangodb-datasource-for-apache-spark.md deleted file mode 100644 index aeaf9d379d..0000000000 --- a/site/content/arangodb/3.12/develop/integrations/arangodb-datasource-for-apache-spark.md +++ /dev/null @@ -1,422 +0,0 @@ ---- -title: ArangoDB Datasource for Apache Spark -menuTitle: Datasource for Apache Spark -weight: 10 -description: >- - ArangoDB Datasource for Apache Spark allows batch reading and writing Spark DataFrame data -aliases: -- arangodb-spark-connector -- arangodb-spark-connector/getting-started -- arangodb-spark-connector/reference -- arangodb-spark-connector/reference/java -- arangodb-spark-connector/reference/scala ---- -ArangoDB Datasource for Apache Spark allows batch reading and writing Spark DataFrame data from and to ArangoDB, by implementing the Spark Data Source V2 API. - -Reading tasks are parallelized according to the number of shards of the related ArangoDB collection, and the writing ones - depending on the source DataFrame partitions. The network traffic is load balanced across the available DB Coordinators. - -Filter predicates and column selections are pushed down to the DB by dynamically generating AQL queries, which will fetch only the strictly required data, thus saving network and computational resources both on the Spark and the DB side. - -The connector is usable from all the Spark supported client languages: Scala, Python, Java, and R. - -This library works with all the non-EOLed [ArangoDB versions](https://www.arangodb.com/subscriptions/end-of-life-notice/). - -## Supported versions - -There are several variants of this library, each one compatible with different Spark and Scala versions: - -- `com.arangodb:arangodb-spark-datasource-3.3_2.12` (Spark 3.3, Scala 2.12) -- `com.arangodb:arangodb-spark-datasource-3.3_2.13` (Spark 3.3, Scala 2.13) -- `com.arangodb:arangodb-spark-datasource-3.4_2.12` (Spark 3.4, Scala 2.12) (compatible with Spark `3.4.2+`) -- `com.arangodb:arangodb-spark-datasource-3.4_2.13` (Spark 3.4, Scala 2.13) (compatible with Spark `3.4.2+`) -- `com.arangodb:arangodb-spark-datasource-3.5_2.12` (Spark 3.5, Scala 2.12) -- `com.arangodb:arangodb-spark-datasource-3.5_2.13` (Spark 3.5, Scala 2.13) - -The following variants are no longer supported: - -- `com.arangodb:arangodb-spark-datasource-2.4_2.11` (Spark 2.4, Scala 2.11) -- `com.arangodb:arangodb-spark-datasource-2.4_2.12` (Spark 2.4, Scala 2.12) -- `com.arangodb:arangodb-spark-datasource-3.1_2.12` (Spark 3.1, Scala 2.12) -- `com.arangodb:arangodb-spark-datasource-3.2_2.12` (Spark 3.2, Scala 2.12) -- `com.arangodb:arangodb-spark-datasource-3.2_2.13` (Spark 3.2, Scala 2.13) - -Since version `1.7.0`, due to [breaking changes](https://github.com/apache/spark/commit/ad29290a02fb94a958fd21e301100338c9f5b82a#diff-b25c8acff88c1b4850c6642e80845aac4fb882c664795c3b0aa058e37ed732a0L42-R52) -in Spark `3.4.2`, `arangodb-spark-datasource-3.4` is not compatible anymore with Spark versions `3.4.0` and `3.4.1`. - -In the following sections the `${sparkVersion}` and `${scalaVersion}` placeholders refer to the Spark and Scala versions. - -## Setup - -To import ArangoDB Datasource for Apache Spark in a Maven project: - -```xml - - - com.arangodb - arangodb-spark-datasource-${sparkVersion}_${scalaVersion} - x.y.z - - -``` - -Substitute `x.y.z` with the latest available version that is compatible. - -To use in an external Spark cluster, submit your application with the following parameter: - -```sh ---packages="com.arangodb:arangodb-spark-datasource-${sparkVersion}_${scalaVersion}:x.y.z" -``` - -## General Configuration - -- `user`: db user, `root` by default -- `password`: db password -- `endpoints`: list of Coordinators, e.g. `c1:8529,c2:8529` (required) -- `acquireHostList`: acquire the list of all known hosts in the cluster (`true` or `false`), `false` by default -- `protocol`: communication protocol (`vst`, `http`, or `http2`), `http2` by default -- `contentType`: content type for driver communication (`json` or `vpack`), `json` by default -- `timeout`: driver connect and request timeout in ms, `300000` by default -- `ssl.enabled`: ssl secured driver connection (`true` or `false`), `false` by default -- `ssl.verifyHost`: whether TLS hostname verification is enabled, `true` by default -- `ssl.cert.value`: Base64 encoded certificate -- `ssl.cert.type`: certificate type, `X.509` by default -- `ssl.cert.alias`: certificate alias name, `arangodb` by default -- `ssl.algorithm`: trust manager algorithm, `SunX509` by default -- `ssl.keystore.type`: keystore type, `jks` by default -- `ssl.protocol`: SSLContext protocol, `TLS` by default -- `ssl.trustStore.path`: trust store path -- `ssl.trustStore.password`: trust store password - -### SSL - -To use TLS-secured connections to ArangoDB, set `ssl.enabled` to `true` and -configure the certificate to use. This can be achieved in one of the following ways: - -- Provide the Base64-encoded certificate as the `ssl.cert.value` configuration entry: - - ```scala - val spark: SparkSession = SparkSession.builder() - // ... - .config("ssl.enabled", "true") - .config("ssl.cert.value", "") - .getOrCreate() - ``` - -- Set the trust store to use in the `ssl.trustStore.path` configuration entry and - optionally set `ssl.trustStore.password`: - - ```scala - val spark: SparkSession = SparkSession.builder() - // ... - .config("ssl.enabled", "true") - .config("ssl.trustStore.path", "") - .config("ssl.trustStore.password", "") - .getOrCreate() - ``` - -- Start the Spark driver and workers with a properly configured trust store: - - ```scala - val spark: SparkSession = SparkSession.builder() - // ... - .config("ssl.enabled", "true") - .getOrCreate() - ``` - - Set the following in the Spark configuration file: - - ```properties - spark.executor.extraJavaOptions=-Djavax.net.ssl.trustStore= -Djavax.net.ssl.trustStorePassword= - spark.driver.extraJavaOptions=-Djavax.net.ssl.trustStore= -Djavax.net.ssl.trustStorePassword= - ``` - - Alternatively, you can set this in the command-line when submitting the Spark job: - - ```sh - ./bin/spark-submit \ - --conf "spark.driver.extraJavaOptions=-Djavax.net.ssl.trustStore= -Djavax.net.ssl.trustStorePassword=" \ - --conf "spark.executor.extraJavaOptions=-Djavax.net.ssl.trustStore= -Djavax.net.ssl.trustStorePassword=" \ - ... - ``` - -### Supported deployment topologies - -The connector can work with a single server and cluster deployments of ArangoDB. - -## Batch Read - -The connector implements support for batch reading from an ArangoDB collection. - -```scala -val df: DataFrame = spark.read - .format("com.arangodb.spark") - .options(options) // Map[String, String] - .schema(schema) // StructType - .load() -``` - -The connector can read data from: -- a collection -- an AQL cursor (query specified by the user) - -When reading data from a **collection**, the reading job is split into many Spark tasks, one for each shard in the ArangoDB source collection. The resulting Spark DataFrame has the same number of partitions as the number of shards in the ArangoDB collection, each one containing data from the respective collection shard. The reading tasks consist of AQL queries that are load balanced across all the available ArangoDB Coordinators. Each query is related to only one shard, therefore it will be executed locally in the DB-Server holding the related shard. - -When reading data from an **AQL cursor**, the reading job cannot be partitioned or parallelized, so it will be less scalable. This mode can be used for reading data coming from different tables, i.e. resulting from an AQL traversal query. - -**Example** - -```scala -val spark: SparkSession = SparkSession.builder() - .appName("ArangoDBSparkDemo") - .master("local[*]") - .config("spark.driver.host", "127.0.0.1") - .getOrCreate() - -val df: DataFrame = spark.read - .format("com.arangodb.spark") - .options(Map( - "password" -> "test", - "endpoints" -> "c1:8529,c2:8529,c3:8529", - "table" -> "users" - )) - .schema(new StructType( - Array( - StructField("likes", ArrayType(StringType, containsNull = false)), - StructField("birthday", DateType, nullable = true), - StructField("gender", StringType, nullable = false), - StructField("name", StructType( - Array( - StructField("first", StringType, nullable = true), - StructField("last", StringType, nullable = false) - ) - ), nullable = true) - ) - )) - .load() - -usersDF.filter(col("birthday") === "1982-12-15").show() -``` - -### Read Configuration - -- `database`: database name, `_system` by default -- `table`: datasource ArangoDB collection name, ignored if `query` is specified. Either `table` or `query` is required. -- `query`: custom AQL read query. If set, `table` will be ignored. Either `table` or `query` is required. -- `batchSize`: reading batch size, `10000` by default -- `sampleSize`: sample size prefetched for schema inference, only used if read schema is not provided, `1000` by default -- `fillBlockCache`: specifies whether the query should store the data it reads in the RocksDB block cache (`true` or `false`), `false` by default -- `stream`: specifies whether the query should be executed lazily, `true` by default -- `ttl`: cursor time to live in seconds, `30` by default -- `mode`: allows setting a mode for dealing with corrupt records during parsing: - - `PERMISSIVE` : win case of a corrupted record, the malformed string is put into a field configured by - `columnNameOfCorruptRecord`, and sets malformed fields to null. To keep corrupt records, a user can set a string - type field named `columnNameOfCorruptRecord` in a user-defined schema. If a schema does not have the field, it drops - corrupt records during parsing. When inferring a schema, it implicitly adds the `columnNameOfCorruptRecord` field in - an output schema - - `DROPMALFORMED`: ignores the whole corrupted records - - `FAILFAST`: throws an exception in case of corrupted records -- `columnNameOfCorruptRecord`: allows renaming the new field having malformed string created by the `PERMISSIVE` mode - -### Predicate and Projection Pushdown - -The connector can convert some Spark SQL filter predicates into AQL predicates and push their execution down to the data source. In this way, ArangoDB can apply the filters and return only the matching documents. - -The following filter predicates (implementations of `org.apache.spark.sql.sources.Filter`) are pushed down: -- `And` -- `Or` -- `Not` -- `EqualTo` -- `EqualNullSafe` -- `IsNull` -- `IsNotNull` -- `GreaterThan` -- `GreaterThanOrEqualFilter` -- `LessThan` -- `LessThanOrEqualFilter` -- `StringStartsWithFilter` -- `StringEndsWithFilter` -- `StringContainsFilter` -- `InFilter` - -Furthermore, the connector will push down the subset of columns required by the Spark query, so that only the relevant documents fields will be returned. - -Predicate and projection pushdowns are only performed while reading an ArangoDB collection (set by the `table` configuration parameter). In case of a batch read from a custom query (set by the `query` configuration parameter), no pushdown optimizations are performed. - -### Read Resiliency - -The data of each partition is read using an AQL cursor. If any error occurs, the read task of the related partition will fail. Depending on the Spark configuration, the task could be retried. - -## Batch Write - -The connector implements support for batch writing to ArangoDB collection. - -```scala -import org.apache.spark.sql.DataFrame - -val df: DataFrame = //... -df.write - .format("com.arangodb.spark") - .mode(SaveMode.Append) - .options(Map( - "password" -> "test", - "endpoints" -> "c1:8529,c2:8529,c3:8529", - "table" -> "users" - )) - .save() -``` - -Write tasks are load balanced across the available ArangoDB Coordinators. The data saved into the ArangoDB is sharded according to the related target collection definition and is different from the Spark DataFrame partitioning. - -### SaveMode - -On writing, `org.apache.spark.sql.SaveMode` is used to specify the expected behavior in case the target collection already exists. - -The following save modes are supported: -- `Append`: the target collection is created, if it does not exist. -- `Overwrite`: the target collection is created, if it does not exist, otherwise it is truncated. Use it in combination with the - `confirmTruncate` write configuration parameter. - -Save modes `ErrorIfExists` and `Ignore` behave the same as `Append`. - -Use the `overwriteMode` write configuration parameter to specify the document overwrite behavior (if a document with the same `_key` already exists). - -### Write Configuration - -- `database`: database name, `_system` by default -- `table`: target ArangoDB collection name (required) -- `batchSize`: writing batch size, `10000` by default -- `byteBatchSize`: byte batch size threshold, only considered for `contentType=json`, `8388608` by default (8 MB) -- `table.shards`: number of shards of the created collection (in case of the `Append` or `Overwrite` SaveMode) -- `table.type`: type (`document` or `edge`) of the created collection (in case of the `Append` or `Overwrite` SaveMode), `document` by default -- `waitForSync`: specifies whether to wait until the documents have been synced to disk (`true` or `false`), `false` by default -- `confirmTruncate`: confirms to truncate table when using the `Overwrite` SaveMode, `false` by default -- `overwriteMode`: configures the behavior in case a document with the specified `_key` value already exists. It is only considered for `Append` SaveMode. - - `ignore` (default for SaveMode other than `Append`): it will not be written - - `replace`: it will be overwritten with the specified document value - - `update`: it will be patched (partially updated) with the specified document value. The overwrite mode can be - further controlled via the `keepNull` and `mergeObjects` parameter. `keepNull` will also be automatically set to - `true`, so that null values are kept in the saved documents and not used to remove existing document fields (as for - default ArangoDB upsert behavior). - - `conflict` (default for the `Append` SaveMode): return a unique constraint violation error so that the insert operation fails -- `mergeObjects`: in case `overwriteMode` is set to `update`, controls whether objects (not arrays) will be merged. - - `true` (default): objects will be merged - - `false`: existing document fields will be overwritten -- `keepNull`: in case `overwriteMode` is set to `update` - - `true` (default): `null` values are saved within the document (by default) - - `false`: `null` values are used to delete the corresponding existing attributes -- `retry.maxAttempts`: max attempts for retrying write requests in case they are idempotent, `10` by default -- `retry.minDelay`: min delay in ms between write requests retries, `0` by default -- `retry.maxDelay`: max delay in ms between write requests retries, `0` by default - -### Write Resiliency - -The data of each partition is saved in batches using the ArangoDB API for -[inserting multiple documents](../http-api/documents.md#multiple-document-operations). -This operation is not atomic, therefore some documents could be successfully written to the database, while others could fail. To make the job more resilient to temporary errors (i.e. connectivity problems), in case of failure the request will be retried (with another Coordinator), if the provided configuration allows idempotent requests, namely: -- the schema of the dataframe has a **not nullable** `_key` field and -- `overwriteMode` is set to one of the following values: - - `replace` - - `ignore` - - `update` with `keep.null=true` - -A failing batch-saving request is retried once for every Coordinator. After that, if still failing, the write task for the related partition is aborted. According to the Spark configuration, the task can be retried and rescheduled on a different executor, if the provided write configuration allows idempotent requests (as described above). - -If a task ultimately fails and is aborted, the entire write job will be aborted as well. Depending on the `SaveMode` configuration, the following cleanup operations will be performed: -- `Append`: no cleanup is performed and the underlying data source may require manual cleanup. - `DataWriteAbortException` is thrown. -- `Overwrite`: the target collection will be truncated. -- `ErrorIfExists`: the target collection will be dropped. -- `Ignore`: if the collection did not exist before, it will be dropped; otherwise, nothing will be done. - -### Write requirements - -When writing to an edge collection (`table.type=edge`), the schema of the Dataframe being written must have: -- a non nullable string field named `_from`, and -- a non nullable string field named `_to` - -### Write Limitations - -- Batch writes are not performed atomically, so sometimes (i.e. in case of `overwrite.mode: conflict`) several documents in the batch may be written and others may return an exception (i.e. due to a conflicting key). -- Writing records with the `_key` attribute is only allowed on collections sharded by `_key`. -- In case of the `Append` save mode, failed jobs cannot be rolled back and the underlying data source may require manual cleanup. -- Speculative execution of tasks only works for idempotent write configurations. See [Write Resiliency](#write-resiliency) for more details. -- Speculative execution of tasks can cause concurrent writes to the same documents, resulting in write-write conflicts or lock timeouts - -## Mapping Configuration - -Serialization and deserialization of Spark Dataframe Row to and from JSON (or Velocypack) can be customized using the following options: -- `ignoreNullFields`: whether to ignore null fields during serialization, `false` by default (only supported in Spark 3.x) - -## Supported Spark data types - -The following Spark SQL data types (subtypes of `org.apache.spark.sql.types.Filter`) are supported for reading, writing and filter pushdown. - -- Numeric types: - - `ByteType` - - `ShortType` - - `IntegerType` - - `LongType` - - `FloatType` - - `DoubleType` - -- String types: - - `StringType` - -- Boolean types: - - `BooleanType` - -- Datetime types: - - `TimestampType` - - `DateType` - -- Complex types: - - `ArrayType` - - `MapType` (only with key type `StringType`) - - `StructType` - -## Connect to the Arango Managed Platform (AMP) - -To connect to SSL-secured deployments using X.509 Base64-encoded CA certificate (AMP): - -```scala -val options = Map( - "database" -> "", - "user" -> "", - "password" -> "", - "endpoints" -> ":", - "ssl.cert.value" -> "", - "ssl.enabled" -> "true", - "table" -> "" -) - -// read -val myDF = spark.read - .format("com.arangodb.spark") - .options(options) - .load() - -// write -import org.apache.spark.sql.DataFrame -val df: DataFrame = //... -df.write - .format("com.arangodb.spark") - .options(options) - .save() -``` - -## Current limitations - -- For `contentType=vpack`, implicit deserialization casts don't work well, i.e. - reading a document having a field with a numeric value whereas the related - read schema requires a string value for such a field. -- Dates and timestamps fields are interpreted to be in a UTC time zone. -- In read jobs using `stream=true` (default), possible AQL warnings are only - logged at the end of each read task (BTS-671). -- Spark SQL `DecimalType` fields are not supported in write jobs when using `contentType=json`. -- Spark SQL `DecimalType` values are written to the database as strings. -- `byteBatchSize` is only considered for `contentType=json` (DE-226) - -## Demo - -Check out our [demo](https://github.com/arangodb/arangodb-spark-datasource/tree/main/demo) -to learn more about ArangoDB Datasource for Apache Spark. diff --git a/site/content/arangodb/3.12/develop/integrations/arangodb-tinkerpop-provider.md b/site/content/arangodb/3.12/develop/integrations/arangodb-tinkerpop-provider.md deleted file mode 100644 index a1a68550bf..0000000000 --- a/site/content/arangodb/3.12/develop/integrations/arangodb-tinkerpop-provider.md +++ /dev/null @@ -1,841 +0,0 @@ ---- -title: ArangoDB TinkerPop Provider -menuTitle: TinkerPop Provider -weight: 10 -description: >- - Build graph applications using TinkerPop API with ArangoDB's high-performance backend, combining Gremlin traversals and native AQL queries ---- -ArangoDB TinkerPop Provider is an implementation of the [Apache TinkerPop OLTP Provider](https://tinkerpop.apache.org/docs/3.7.4/dev/provider) API for -ArangoDB. - -It allows using the standard TinkerPop API with ArangoDB as the backend storage. It supports creating, -querying, and manipulating graph data using the Gremlin traversal language, while offering the possibility to use native -AQL (ArangoDB Query Language) for complex queries. - -- Repository: -- [Code examples](https://github.com/arangodb/arangodb-tinkerpop-provider/tree/main/src/test/java/example) -- [Demo](https://github.com/arangodb/arangodb-tinkerpop-provider/tree/main/demo) -- [JavaDoc](https://www.javadoc.io/doc/com.arangodb/arangodb-tinkerpop-provider/latest/index.html) (generated reference documentation) -- [ChangeLog](https://github.com/arangodb/arangodb-tinkerpop-provider/blob/main/CHANGELOG.md) - -## Compatibility - -This Provider is compatible with: - -- Apache TinkerPop 3.7 -- ArangoDB 3.12+ -- ArangoDB Java Driver 7.22+ -- Java 8+ - -## Installation - -### Maven - -1. Check the [latest version](https://search.maven.org/artifact/com.arangodb/arangodb-tinkerpop-provider) available. -2. Add the following dependency to your `pom.xml` file: - -```xml - - - com.arangodb - arangodb-tinkerpop-provider - x.y.z - - -``` - -### Gradle - -1. Add the following dependency to your `build.gradle`: - -```groovy -implementation 'com.arangodb:arangodb-tinkerpop-provider:x.y.z' -``` - -### Gremlin Console - -1. Install the TinkerPop Provider in the Gremlin Console: - -```text -:install com.arangodb arangodb-tinkerpop-provider x.y.z -``` - -2. Restart the console to load the provider. - -3. Create a configuration for your ArangoDB connection. - -```text -gremlin> conf = [ -......1> "gremlin.graph":"com.arangodb.tinkerpop.gremlin.structure.ArangoDBGraph", -......2> "gremlin.arangodb.conf.graph.enableDataDefinition":"true", -......3> "gremlin.arangodb.conf.driver.hosts":"172.28.0.1:8529", -......4> "gremlin.arangodb.conf.driver.password":"test", -......5> ] -==>gremlin.graph=com.arangodb.tinkerpop.gremlin.structure.ArangoDBGraph -==>gremlin.arangodb.conf.graph.enableDataDefinition=true -==>gremlin.arangodb.conf.driver.hosts=172.28.0.1:8529 -==>gremlin.arangodb.conf.driver.password=test -``` -4. Open the graph using the configuration. - -```text -gremlin> graph = GraphFactory.open(conf) -==>arangodbgraph[ArangoDBGraphConfig{...}] -``` - -5. Create a traversal source and start using it: - -```text -gremlin> g = graph.traversal() -==>graphtraversalsource[arangodbgraph[...], standard] - -gremlin> g.addV("person").property("name", "marko") -==>v[4586117] - -gremlin> g.V().hasLabel("person").values("name") -==>marko -``` - -### Server Plugin - -Follow the steps below to set up the provider as a Gremlin Server plugin. - -1. Install the provider in the Gremlin Server: - - ```bash - ./bin/gremlin-server.sh install com.arangodb arangodb-tinkerpop-provider x.y.z - ``` - -2. Create a graph configuration file (e.g., `conf/arangodb.yaml`): - - ```yaml - gremlin: - graph: "com.arangodb.tinkerpop.gremlin.structure.ArangoDBGraph" - arangodb: - conf: - graph: - enableDataDefinition: true - driver: - hosts: - - "172.28.0.1:8529" - password: test - ``` - -3. Create a server configuration file to load the plugin and graph configuration (e.g., `conf/gremlin-server-arangodb.yaml`): - -```yaml -host: 0.0.0.0 -port: 8182 -graphs: { - graph: conf/arangodb.yaml} -scriptEngines: { - gremlin-groovy: { - plugins: { org.apache.tinkerpop.gremlin.server.jsr223.GremlinServerGremlinPlugin: {}, - org.apache.tinkerpop.gremlin.tinkergraph.jsr223.TinkerGraphGremlinPlugin: {}, - com.arangodb.tinkerpop.gremlin.jsr223.ArangoDBGremlinPlugin: {}, - org.apache.tinkerpop.gremlin.jsr223.ImportGremlinPlugin: {classImports: [java.lang.Math], methodImports: [java.lang.Math#*]}, - org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin: {files: [scripts/empty-sample.groovy]}}}} -serializers: - - { className: org.apache.tinkerpop.gremlin.util.ser.GraphSONMessageSerializerV3, config: { ioRegistries: [org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerIoRegistryV3] }} # application/json - - { className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1 } # application/vnd.graphbinary-v1.0 - - { className: org.apache.tinkerpop.gremlin.util.ser.GraphBinaryMessageSerializerV1, config: { serializeResultToString: true }} # application/vnd.graphbinary-v1.0-stringd -processors: - - { className: org.apache.tinkerpop.gremlin.server.op.session.SessionOpProcessor, config: { sessionTimeout: 28800000 }} - - { className: org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor} -``` - -4. Start the Gremlin Server with your configuration: - - ```bash - ./bin/gremlin-server.sh conf/gremlin-server-arangodb.yaml - ``` - -5. Connect to the server using the Gremlin Console: - - ```text - gremlin> :remote connect tinkerpop.server conf/remote.yaml - - gremlin> :remote console - ==>All scripts will now be sent to Gremlin Server - [localhost/127.0.0.1:8182] - type ':remote console' to return to local mode - ``` - -6. Start using the graph: - - ```text - gremlin> g.addV("person").property("name", "marko") - ==>v[4587713] - - gremlin> g.V().hasLabel("person").values("name") - ==>marko - ``` - -You can find the reference documentation [here](https://tinkerpop.apache.org/docs/3.7.4/reference/#_configuring_2). - - -## Quick Start - -Follow the steps below to get started with creating vertices, edges, and querying the graph. - -### Step 1: Configure and Create the Graph - -[//]: <> (@formatter:off) -```java -// Create a configuration -Configuration conf = new ArangoDBConfigurationBuilder() - .hosts("localhost:8529") - .user("root") - .password("test") - .db("myDatabase") - .name("myGraph") - .enableDataDefinition(true) // Allow creating database and graph if they don't exist - .build(); - -// Create a graph -ArangoDBGraph graph = (ArangoDBGraph) GraphFactory.open(conf); - -// Get a traversal source -GraphTraversalSource g = graph.traversal(); -``` -### Step 2: Add Vertices and Edges - -```java -// Add vertices with properties -Vertex person = g.addV("person") - .property("name", "Alice") - .property("age", 30) - .property("country", "Germany") - .next(); - -Vertex software = g.addV("software") - .property("name", "JArango") - .property("lang", "Java") - .next(); - -// Create relationships between vertices -Edge created = g.addE("created") - .from(person) - .to(software) - .property("year", 2025) - .next(); -``` - -### Step 3: Query the Graph - -```java -// Query the graph -List creators = g.V() - .hasLabel("software") - .has("name", "JArango") - .in("created") - .values("name") - .toList(); - -System.out.println("Creators: " + creators); - -// Query: Find all software created by Alice -List aliceSoftware = g.V() - .hasLabel("person") - .has("name", "Alice") - .out("created") - .values("name") - .toList(); - -System.out.println("aliceSoftware: " + aliceSoftware); - -``` - -### Step 4: Update Data - -```java -// Update a property -g.V() - .hasLabel("person") - .has("name", "Alice") - .property("age", 31) - .iterate(); - -// Remove a property -g.V() - .hasLabel("person") - .has("name", "Alice") - .properties("country") - .drop() - .iterate(); - -// Check the updated vertex -Map alice = g.V() - .hasLabel("person") - .has("name", "Alice") - .valueMap() - .next(); - -System.out.println("alice: " + alice); -``` - -### Step 5: Delete Data and Clean Up - -```java -// Remove an edge -g.E() - .hasLabel("created") - .where(__.outV() - .has("name", "Alice")) - .where(__.inV() - .has("name", "JArango")) - .drop() - .iterate(); - -// Remove a vertex (and its incident edges) -g.V() - .hasLabel("person") - .has("name", "Alice") - .drop() - .iterate(); - -// Close the graph when done -graph.close(); -``` -[//]: <> (@formatter:on) - -## Configuration - -The graph can be created using the methods from `org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(...)` -(see [javadoc](https://tinkerpop.apache.org/javadocs/3.7.4/full/org/apache/tinkerpop/gremlin/structure/util/GraphFactory.html)). -These methods accept a configuration file (e.g., YAML or properties file), a Java Map, or an Apache Commons Configuration object. - -The property `gremlin.graph` must be set to: `com.arangodb.tinkerpop.gremlin.structure.ArangoDBGraph`. - -Configuration examples can be found [here](https://github.com/arangodb/arangodb-tinkerpop-provider/tree/main/src/test/java/example). - -### Graph Configuration Properties - -Graph configuration properties are prefixed with `gremlin.arangodb.conf.graph`: - -| Property | Description | Default | -|----------------------------------------------------|---------------------------------------|-------------| -| `gremlin.arangodb.conf.graph.db` | ArangoDB database name | `_system` | -| `gremlin.arangodb.conf.graph.name` | ArangoDB graph name | `tinkerpop` | -| `gremlin.arangodb.conf.graph.enableDataDefinition` | Flag to allow data definition changes | `false` | -| `gremlin.arangodb.conf.graph.type` | Graph type: `SIMPLE` or `COMPLEX` | `SIMPLE` | -| `gremlin.arangodb.conf.graph.labelField` | Label field name | `_label` | -| `gremlin.arangodb.conf.graph.orphanCollections` | List of orphan collections names | - | -| `gremlin.arangodb.conf.graph.edgeDefinitions` | List of edge definitions | - | - -### Driver Configuration Properties - -Driver configuration properties are prefixed with `gremlin.arangodb.conf.driver`. All properties from -`com.arangodb.config.ArangoConfigProperties` are supported. See -the [ArangoDB Java Driver documentation](../drivers/java/reference-version-7/driver-setup.md#config-file-properties) -for details. - -### YAML Configuration - -```yaml -gremlin: - graph: "com.arangodb.tinkerpop.gremlin.structure.ArangoDBGraph" - arangodb: - conf: - graph: - db: "testDb" - name: "myFirstGraph" - enableDataDefinition: true - type: COMPLEX - orphanCollections: [ "x", "y", "z" ] - edgeDefinitions: - - "e1:[a]->[b]" - - "e2:[a,b]->[c,d]" - driver: - user: "root" - password: "test" - hosts: - - "172.28.0.1:8529" - - "172.28.0.1:8539" - - "172.28.0.1:8549" -``` - -Loading from a YAML file: - -[//]: <> (@formatter:off) -```java -ArangoDBGraph graph = (ArangoDBGraph) GraphFactory.open(""); -``` -[//]: <> (@formatter:on) - -### Programmatic Configuration - -1. Build the configuration using the configuration builder: - - [//]: <> (@formatter:off) - ```java - Configuration conf = new ArangoDBConfigurationBuilder() - .hosts("172.28.0.1:8529") - .user("root") - .password("test") - .database("testDb") - .name("myGraph") - .graphType(GraphType.SIMPLE) - .enableDataDefinition(true) - .build(); - ``` - [//]: <> (@formatter:on) - -2. Create the graph using the configuration: - - [//]: <> (@formatter:off) - ```java - ArangoDBGraph graph = (ArangoDBGraph) GraphFactory.open(conf); - ``` - [//]: <> (@formatter:on) - -### SSL Configuration - -To use TLS-secured connections to ArangoDB, follow these steps: - -1. Enable SSL by setting `gremlin.arangodb.conf.driver.useSsl` to `true` in your configuration. - -2. Configure SSL properties as needed (see [ArangoDB Java Driver documentation](../drivers/java/reference-version-7/driver-setup.md#config-file-properties) for all available options): - - ```yaml - gremlin: - graph: "com.arangodb.tinkerpop.gremlin.structure.ArangoDBGraph" - arangodb: - conf: - driver: - hosts: - - "172.28.0.1:8529" - useSsl: true - verifyHost: false - sslCertValue: "MIIDezCCAmOgAwIBAgIEeDCzXzANBgkqhkiG9w0BAQsFADBuMRAwDgYDVQQGEwdVbmtub3duMRAwDgYDVQQIEwdVbmtub3duMRAwDgYDVQQHEwdVbmtub3duMRAwDgYDVQQKEwdVbmtub3duMRAwDgYDVQQLEwdVbmtub3duMRIwEAYDVQQDEwlsb2NhbGhvc3QwHhcNMjAxMTAxMTg1MTE5WhcNMzAxMDMwMTg1MTE5WjBuMRAwDgYDVQQGEwdVbmtub3duMRAwDgYDVQQIEwdVbmtub3duMRAwDgYDVQQHEwdVbmtub3duMRAwDgYDVQQKEwdVbmtub3duMRAwDgYDVQQLEwdVbmtub3duMRIwEAYDVQQDEwlsb2NhbGhvc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC1WiDnd4+uCmMG539ZNZB8NwI0RZF3sUSQGPx3lkqaFTZVEzMZL76HYvdc9Qg7difyKyQ09RLSpMALX9euSseD7bZGnfQH52BnKcT09eQ3wh7aVQ5sN2omygdHLC7X9usntxAfv7NzmvdogNXoJQyY/hSZff7RIqWH8NnAUKkjqOe6Bf5LDbxHKESmrFBxOCOnhcpvZWetwpiRdJVPwUn5P82CAZzfiBfmBZnB7D0l+/6Cv4jMuH26uAIcixnVekBQzl1RgwczuiZf2MGO64vDMMJJWE9ClZF1uQuQrwXF6qwhuP1Hnkii6wNbTtPWlGSkqeutr004+Hzbf8KnRY4PAgMBAAGjITAfMB0GA1UdDgQWBBTBrv9Awynt3C5IbaCNyOW5v4DNkTANBgkqhkiG9w0BAQsFAAOCAQEAIm9rPvDkYpmzpSIhR3VXG9Y71gxRDrqkEeLsMoEyqGnw/zx1bDCNeGg2PncLlW6zTIipEBooixIE9U7KxHgZxBy0Et6EEWvIUmnr6F4F+dbTD050GHlcZ7eOeqYTPYeQC502G1Fo4tdNi4lDP9L9XZpf7Q1QimRH2qaLS03ZFZa2tY7ah/RQqZL8Dkxx8/zc25sgTHVpxoK853glBVBs/ENMiyGJWmAXQayewY3EPt/9wGwV4KmU3dPDleQeXSUGPUISeQxFjy+jCw21pYviWVJTNBA9l5ny3GhEmcnOT/gQHCvVRLyGLMbaMZ4JrPwb+aAtBgrgeiK4xeSMMvrbhw==" - ``` - -3. Alternatively, use system properties for truststore configuration. If no `sslCertValue` is provided, the default SSL context is used. In such cases, you can specify the truststore using system properties: - - `javax.net.ssl.trustStore` - - `javax.net.ssl.trustStorePassword` - -### Data Definition Management - -The provider follows this process when instantiating a graph: - -1. **Validation**: The provider compares existing data definitions in ArangoDB with the structure expected by your configuration. It checks whether: - - The database exists - - The graph exists - - The graph structure has the same edge definitions and orphan collections - -2. **Error handling**: If there's a mismatch, an error is thrown and the graph is not instantiated. - -3. **Automatic creation** (optional): To automatically create missing data definitions, set `gremlin.arangodb.conf.graph.enableDataDefinition` to `true`. This allows: - - Creating a new database if it doesn't exist - - Creating a new graph if it doesn't exist (along with vertex and edge collections) - -{{< info >}} -Existing graphs are never modified automatically for safety reasons. -{{< /info >}} - -## Graph Types - -The ArangoDB TinkerPop Provider supports two graph types, which can be configured with the property -`gremlin.arangodb.conf.graph.type`: `SIMPLE` and `COMPLEX`. - -### SIMPLE Graph Type - -From an application perspective, this is the most flexible graph type that is backed by an ArangoDB graph composed of only one vertex collection and one edge definition. - -The `label` of each element is stored in a database document field. The label field name is configurable by setting the configuration property `graph.labelField`, `_label` by default. - -The `SIMPLE` graph type is the default graph type. - -It has the following advantages: - -- It closely matches the TinkerPop property graph -- It is simpler to get started and run examples -- It imposes no restrictions about element IDs -- It supports arbitrary labels, i.e., labels not known at graph construction time - -It has the following disadvantages: - -- All vertex types will be stored in the same vertex collection -- All edge types will be stored in the same edge collection -- It could not leverage the full potential of ArangoDB graph traversal -- It could require an index on the label field to improve performance - -Example configuration: - -```yaml -gremlin: - graph: "com.arangodb.tinkerpop.gremlin.structure.ArangoDBGraph" - arangodb: - conf: - graph: - db: "db" - name: "myGraph" - type: SIMPLE - edgeDefinitions: - - "e:[v]->[v]" -``` - -If `edgeDefinitions` are not configured, the default names are used: - -- `vertex` will be used for the vertex collection -- `edge` will be used for the edge collection - -Using a `SIMPLE` graph configured as in the example above and creating a new element like: - -[//]: <> (@formatter:off) -```java -graph.addVertex(T.label, "person", T.id, "foo"); -``` -[//]: <> (@formatter:on) - -would result in creating a document in the vertex collection `v` with `_key` equals to `foo` (and `_id` equals -to `v/foo`). - -### COMPLEX Graph Type - -The `COMPLEX` graph type is backed by an ArangoDB graph composed potentially of multiple vertex collections and multiple -edge definitions. - -The `label` of each element is used as name for the related database collection. - -It has the following advantages: - -- It closely matches the ArangoDB graph structure -- It allows multiple vertex collections and multiple edge collections -- It partitions the data in a finer way -- It allows indexing and sharding collections independently -- It is more flexible to match pre-existing database graph structures - -But on the other side has the following constraints: - -- Element IDs must have the format: `