From c6de61982dfe1d855baccb792f518c250b6a511b Mon Sep 17 00:00:00 2001 From: alexpeters1208 <80283343+alexpeters1208@users.noreply.github.com> Date: Sat, 3 Jun 2023 09:52:54 -0500 Subject: [PATCH] R Client 1.0 (#3879) * First round of files for R client. * Commit first round of package files created from RStudio * Package compiles and links, does not load * Modify client caller with wrapper and manual Rcpp integration, not working * Add comments for clarity * Halfway working glue * Another halfway working package * Debug segfault in R side for getStream, modify test_connect to reflect real use * Add authentication, add TODO comments, better helper functions, appropriate renaming * Implement pushing dataframe to table * Delete .gitignore * Progress on documentation, no testing yet * Add docs and functionality to client.cpp, add terrible error handling in helper functions * Update docs and first unit tests, might be for not as API is likely to change * Rework API, still need to overhaul testing and docs * Created R CODEOWNERS * Add tibble support, update Description, remove junk * Add client_options, comment on bash script * Update client_wrapper doc * Upload .rd file for client documentation * Fix docs, improve error messages, add licenses * Update licensing, remove prototyping directory * Delete LICENSE from C++ wrapper * Delete LICENSE * Create LICENSE * Delete LICENSE * Create LICENSE --------- Co-authored-by: Chip Kent <5250374+chipkent@users.noreply.github.com> --- .github/CODEOWNERS | 1 + .gitignore | 1 + R/rdeephaven/.gitignore | 5 + R/rdeephaven/DESCRIPTION | 21 + R/rdeephaven/LICENSE | 202 +++++++++ R/rdeephaven/NAMESPACE | 3 + R/rdeephaven/R/client_options_wrapper.R | 59 +++ R/rdeephaven/R/client_wrapper.R | 129 ++++++ R/rdeephaven/R/exports.R | 5 + R/rdeephaven/R/table_handle_wrapper.R | 50 +++ R/rdeephaven/Read-and-delete-me | 8 + R/rdeephaven/lib/build-cpp.sh | 24 ++ .../cpp-dependencies/build-dependencies.sh | 390 ++++++++++++++++++ R/rdeephaven/man/Client.Rd | 134 ++++++ R/rdeephaven/man/ClientOptions.Rd | 105 +++++ R/rdeephaven/man/rdeephaven-package.Rd | 34 ++ R/rdeephaven/src/CMakeLists.txt | 16 + R/rdeephaven/src/Makevars | 67 +++ R/rdeephaven/src/RcppExports.cpp | 24 ++ R/rdeephaven/src/client.cpp | 224 ++++++++++ R/rdeephaven/tests/testthat.R | 4 + .../tests/testthat/client_infrastructure.R | 32 ++ R/rdeephaven/tests/testthat/test_client.R | 1 + cpp-client/LICENSE | 202 +++++++++ 24 files changed, 1741 insertions(+) create mode 100644 R/rdeephaven/.gitignore create mode 100644 R/rdeephaven/DESCRIPTION create mode 100644 R/rdeephaven/LICENSE create mode 100644 R/rdeephaven/NAMESPACE create mode 100644 R/rdeephaven/R/client_options_wrapper.R create mode 100644 R/rdeephaven/R/client_wrapper.R create mode 100644 R/rdeephaven/R/exports.R create mode 100644 R/rdeephaven/R/table_handle_wrapper.R create mode 100644 R/rdeephaven/Read-and-delete-me create mode 100755 R/rdeephaven/lib/build-cpp.sh create mode 100755 R/rdeephaven/lib/cpp-dependencies/build-dependencies.sh create mode 100644 R/rdeephaven/man/Client.Rd create mode 100644 R/rdeephaven/man/ClientOptions.Rd create mode 100644 R/rdeephaven/man/rdeephaven-package.Rd create mode 100644 R/rdeephaven/src/CMakeLists.txt create mode 100644 R/rdeephaven/src/Makevars create mode 100644 R/rdeephaven/src/RcppExports.cpp create mode 100644 R/rdeephaven/src/client.cpp create mode 100644 R/rdeephaven/tests/testthat.R create mode 100644 R/rdeephaven/tests/testthat/client_infrastructure.R create mode 100644 R/rdeephaven/tests/testthat/test_client.R create mode 100644 cpp-client/LICENSE diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 7cccd019962..ed440a0c5cc 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -22,4 +22,5 @@ /docker @devinrsmith @jcferretti @rcaudy /engine/function/ @chipkent @kosak @rcaudy /py @chipkent @jmao-denver @rcaudy +/R @chipkent @alexpeters1208 @rcaudy *.proto @devinrsmith @nbauernfeind @niloc132 @rcaudy diff --git a/.gitignore b/.gitignore index 8c1c4d98f1c..b769e45c7cb 100644 --- a/.gitignore +++ b/.gitignore @@ -55,3 +55,4 @@ node_modules .project .classpath .settings +.Rproj.user diff --git a/R/rdeephaven/.gitignore b/R/rdeephaven/.gitignore new file mode 100644 index 00000000000..9f651736392 --- /dev/null +++ b/R/rdeephaven/.gitignore @@ -0,0 +1,5 @@ +lib/cpp-client/ +lib/cpp-examples/ +lib/cpp-dependencies/local +lib/cpp-dependencies/src +lib/cpp-dependencies/env.sh \ No newline at end of file diff --git a/R/rdeephaven/DESCRIPTION b/R/rdeephaven/DESCRIPTION new file mode 100644 index 00000000000..77afb3e3cff --- /dev/null +++ b/R/rdeephaven/DESCRIPTION @@ -0,0 +1,21 @@ +Package: rdeephaven +Type: Package +Title: R Client for Deephaven Core +Version: 1.0 +Date: 2023-05-12 +Author: Deephaven Data Labs +Maintainer: Alex Peters +Description: The `rdeephaven` package provides an R API for communicating with the Deephaven server and working with Deephaven tables. + In this release, we support connecting to the Deephaven server with three authentication methods including anonymous, + username/password, and general key/value authentication. Once the connection has been established, we provide the tools to + pull Deephaven tables from the server directly into R Data Frames, Tibbles, Arrow Tables, or Arrow RecordBatchReaders. + You can also import local data with any of those types directly to the server, retrieve a reference to the new table, + and bind it to a server-side variable so you can access it from any Deephaven client. Finally, you can run Python or Groovy + scripts on the Deephaven server, so long as your server is equipped with that capability. +License: Apache License (== 2.0) +Depends: R (> 4.2.0), Rcpp (>= 1.0.10), arrow (>= 12.0.0), R6 (>= 2.5.0), dplyr (>= 1.1.0) +Imports: Rcpp (>= 1.0.10), arrow (>= 12.0.0), R6 (>= 2.5.0), dplyr (>= 1.1.0) +LinkingTo: Rcpp +Suggests: testthat (>= 3.0.0) +Config/testthat/edition: 3 +RoxygenNote: 7.2.3 diff --git a/R/rdeephaven/LICENSE b/R/rdeephaven/LICENSE new file mode 100644 index 00000000000..d6456956733 --- /dev/null +++ b/R/rdeephaven/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/R/rdeephaven/NAMESPACE b/R/rdeephaven/NAMESPACE new file mode 100644 index 00000000000..0a33cb56cd5 --- /dev/null +++ b/R/rdeephaven/NAMESPACE @@ -0,0 +1,3 @@ +useDynLib(rdeephaven, .registration=TRUE) +importFrom(Rcpp, evalCpp) +exportPattern("^[[:alpha:]]+") \ No newline at end of file diff --git a/R/rdeephaven/R/client_options_wrapper.R b/R/rdeephaven/R/client_options_wrapper.R new file mode 100644 index 00000000000..9b965c20d22 --- /dev/null +++ b/R/rdeephaven/R/client_options_wrapper.R @@ -0,0 +1,59 @@ +#' @title Deephaven ClientOptions +#' @description Client options provide a simple interface to the Deephaven server's authentication protocols. +#' This makes it easy to connect to a Deephaven server with any flavor of authentication, and shields the API from +#' any future changes to the underlying implementation. +#' +#' Currently, we support three different kinds of authentication that your Deephaven server might be using: +#' +#' - "default": Default (or anonymous) authentication does not require any username or password. If you are +#' running the Deephaven server locally, this is probably the kind of authentication you are using. +#' +#' - "basic": Basic authentication requires a standard username and password pair. +#' +#' - "custom": Custom authentication requires general key-value pairs. +#' +#' In addition to setting your authentication parameters when you connect to a client, you can also start +#' a console in one of our supported server languages. We currently support Python and Groovy, and if you +#' want start a console upon client connection, you must ensure that the server you're connecting to was +#' started with support for the language you want to use. +#' +#' @section Methods +#' +#' - `$set_default_authentication()` +#' - `$set_basic_authentication()` +#' - `$set_custom_authentication()` +#' +#' @examples +#' +#' # connect to the Deephaven server running on "localhost:10000" with anonymous 'default' authentication +#' client_options <- ClientOptions$new() +#' client_options$set_default_authentication() +#' client <- Client$new(target="localhost:10000", client_options=client_options) + + +ClientOptions <- R6Class("ClientOptions", + public = list( + + initialize = function() { + self$internal_client_options <- new(INTERNAL_ClientOptions) + }, + + set_default_authentication = function() { + self$internal_client_options$set_default_authentication() + }, + + set_basic_authentication = function(username, password) { + self$internal_client_options$set_basic_authentication(username, password) + }, + + set_custom_authentication = function(auth_key, auth_value) { + self$internal_client_options$set_custom_authentication(auth_key, auth_value) + }, + + set_session_type = function(session_type) { + self$internal_client_options$set_session_type(session_type) + }, + + internal_client_options = NULL + ) +) \ No newline at end of file diff --git a/R/rdeephaven/R/client_wrapper.R b/R/rdeephaven/R/client_wrapper.R new file mode 100644 index 00000000000..dd03b17e709 --- /dev/null +++ b/R/rdeephaven/R/client_wrapper.R @@ -0,0 +1,129 @@ +#' @title The Deephaven Client +#' @description The Deephaven Client class is responsible for establishing and maintaining +#' a connection to a running Deephaven server and facilitating basic server requests. +#' +#' @usage NULL +#' @format NULL +#' @docType class +#' +#' @section Methods +#' +#' - `$open_table(name)` +#' - `$import_table(table_object)` +#' - `$run_script(script)` +#' +#' @examples +#' +#' # connect to the Deephaven server running on "localhost:10000" with anonymous 'default' authentication +#' client_options <- ClientOptions$new() +#' client_options$set_default_authentication() +#' client <- Client$new(target="localhost:10000", client_options=client_options) +#' +#' # open a table that already exists on the server +#' new_table_handle1 <- client$open_table("table_on_the_server") +#' +#' # create a new dataframe, import onto the server, and retrieve a reference +#' new_data_frame <- data.frame(matrix(rnorm(10 * 1000), nrow = 10)) +#' new_table_handle2 <- client$import_table(new_data_frame) +#' +#' # run a python script on the server (default client options specify a Python console) +#' client$run_script("print([i for i in range(10)])") + + +Client <- R6Class("Client", + public = list( + + #' @description + #' Connect to a running Deephaven server. + #' @param target The address of the Deephaven server. + #' @param client_options ClientOptions instance with the parameters needed to connect to the server. + #' See ?ClientOptions for more information. + initialize = function(target, client_options) { + private$internal_client <- new(INTERNAL_Client, target=target, + client_options=client_options$internal_client_options) + }, + + #' @description + #' Opens a table named 'name' from the server if it exists. + #' @param name Name of the table to open from the server as a string. + #' @return TableHandle reference to the requested table. + open_table = function(name) { + private$verify_string(name, "name") + if (!private$check_for_table(name)) { + stop(paste0("The table '", name, "' you're trying to pull does not exist on the server.")) + } + return(TableHandle$new(private$internal_client$open_table(name))) + }, + + #' @description + #' Imports a new table to the Deephaven server. Note that this new table is not automatically bound to + #' a variable name on the server. See `?TableHandle` for more information. + #' @param table_object An R Data Frame, an R Tibble, an Arrow Table, or an Arrow RecordBatchReader + #' containing the data to import to the server. + #' @return TableHandle reference to the new table. + import_table = function(table_object) { + table_object_class = class(table_object) + if (table_object_class[[1]] == "data.frame") { + return(TableHandle$new(private$df_to_dh_table(table_object))) + } + if (table_object_class[[1]] == "tbl_df") { + return(TableHandle$new(private$tibble_to_dh_table(table_object))) + } + else if (table_object_class[[1]] == "RecordBatchReader") { + return(TableHandle$new(private$rbr_to_dh_table(table_object))) + } + else if ((length(table_object_class) == 4 && + table_object_class[[1]] == "Table" && + table_object_class[[3]] == "ArrowObject")) { + return(TableHandle$new(private$arrow_to_dh_table(table_object))) + } + else { + stop(paste0("'table_object' must be either an R Data Frame, an R Tibble, an Arrow Table, or an Arrow Record Batch Reader. + Got object of class ", table_object_class[[1]], " instead.")) + } + }, + + #' @description + #' Runs a script on the server. The script must be in the language that the server console was started with. + #' @param script Code to be executed on the server as a string. + run_script = function(script) { + private$verify_string(script, "script") + private$internal_client$run_script(script) + } + ), + private = list( + + internal_client = NULL, + + check_for_table = function(name) { + return(private$internal_client$check_for_table(name)) + }, + + verify_string = function(string_candidate, arg_name) { + if (class(string_candidate) != "character") { + stop(paste0("'", arg_name, "' must be passed as a string. Got object of class ", class(string_candidate)[[1]], " instead.")) + } + }, + + rbr_to_dh_table = function(rbr) { + ptr = private$internal_client$new_arrow_array_stream_ptr() + rbr$export_to_c(ptr) + return(private$internal_client$new_table_from_arrow_array_stream_ptr(ptr)) + }, + + arrow_to_dh_table = function(arrow_tbl) { + rbr = as_record_batch_reader(arrow_tbl) + return(private$rbr_to_dh_table(rbr)) + }, + + tibble_to_dh_table = function(tibbl) { + arrow_tbl = arrow_table(tibbl) + return(private$arrow_to_dh_table(arrow_tbl)) + }, + + df_to_dh_table = function(data_frame) { + arrow_tbl = arrow_table(data_frame) + return(private$arrow_to_dh_table(arrow_tbl)) + } + ) +) diff --git a/R/rdeephaven/R/exports.R b/R/rdeephaven/R/exports.R new file mode 100644 index 00000000000..a36953209ed --- /dev/null +++ b/R/rdeephaven/R/exports.R @@ -0,0 +1,5 @@ +#' @import Rcpp +#' @useDynLib rdeephaven, .registration = TRUE +#' @importFrom Rcpp evalCpp + +loadModule("DeephavenInternalModule", TRUE) \ No newline at end of file diff --git a/R/rdeephaven/R/table_handle_wrapper.R b/R/rdeephaven/R/table_handle_wrapper.R new file mode 100644 index 00000000000..1428962277d --- /dev/null +++ b/R/rdeephaven/R/table_handle_wrapper.R @@ -0,0 +1,50 @@ +# TODO: Document this guy + + +TableHandle <- R6Class("TableHandle", + public = list( + + initialize = function(table_handle) { + + if (class(table_handle)[[1]] != "Rcpp_INTERNAL_TableHandle") { + stop("'table_handle' should be an internal Deephaven TableHandle. If you're seeing this, + you are trying to call the constructor of TableHandle directly, which is not advised.") + } + private$internal_table_handle <- table_handle + }, + + bind_to_variable = function(name) { + if (class(name)[[1]] != "character") { + stop(paste("'name' should be a character or a string. Got object of type ", class(name)[[1]], " instead.")) + } + private$internal_table_handle$bind_to_variable(name) + }, + + to_arrow_record_batch_stream_reader = function() { + ptr = private$internal_table_handle$get_arrow_array_stream_ptr() + rbsr = RecordBatchStreamReader$import_from_c(ptr) + return(rbsr) + }, + + to_arrow_table = function() { + rbsr = self$to_arrow_record_batch_stream_reader() + arrow_tbl = rbsr$read_table() + return(arrow_tbl) + }, + + to_tibble = function() { + rbsr = self$to_arrow_record_batch_stream_reader() + arrow_tbl = rbsr$read_table() + return(as_tibble(arrow_tbl)) + }, + + to_data_frame = function() { + arrow_tbl = self$to_arrow_table() + return(as.data.frame(as.data.frame(arrow_tbl))) # TODO: for some reason as.data.frame on arrow table returns a tibble, not a dataframe + } + + ), + private = list( + internal_table_handle = NULL + ) +) diff --git a/R/rdeephaven/Read-and-delete-me b/R/rdeephaven/Read-and-delete-me new file mode 100644 index 00000000000..dcaca43dc12 --- /dev/null +++ b/R/rdeephaven/Read-and-delete-me @@ -0,0 +1,8 @@ +* Edit the help file skeletons in 'man', possibly combining help files for multiple functions. +* Edit the exports in 'NAMESPACE', and add necessary imports. +* Put any C/C++/Fortran code in 'src'. +* If you have compiled code, add a useDynLib() directive to 'NAMESPACE'. +* Run R CMD build to build the package tarball. +* Run R CMD check to check the package tarball. + +Read "Writing R Extensions" for more information. diff --git a/R/rdeephaven/lib/build-cpp.sh b/R/rdeephaven/lib/build-cpp.sh new file mode 100755 index 00000000000..d8f6b73f75d --- /dev/null +++ b/R/rdeephaven/lib/build-cpp.sh @@ -0,0 +1,24 @@ +#!/bin/bash + +# This script calls build-dependencies.sh to build all the C++ client dependencies, then downloads and builds the C++ client from source. +# The R client is entirely dependent on the C++ client, so this script must be run for a successful install from source. + +export SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) +export CLIENT=$SCRIPT_DIR/cpp-client + +# First, we build all C++ dependencies +cd cpp-dependencies +chmod +x ./build-dependencies.sh +./build-dependencies.sh +cd $SCRIPT_DIR + +# Next, we download, build, and install the Deephaven C++ client +svn checkout https://github.com/deephaven/deephaven-core/trunk/cpp-client +cd cpp-client/deephaven +mkdir build && cd build + +export CLIENT=$SCRIPT_DIR/cpp-client +export DEEPHAVEN_LOCAL=$SCRIPT_DIR/cpp-dependencies/local +export CMAKE_PREFIX_PATH=${DEEPHAVEN_LOCAL}/abseil:${DEEPHAVEN_LOCAL}/boost:${DEEPHAVEN_LOCAL}/cares:${DEEPHAVEN_LOCAL}/flatbuffers:${DEEPHAVEN_LOCAL}/gflags:${DEEPHAVEN_LOCAL}/immer:${DEEPHAVEN_LOCAL}/protobuf:${DEEPHAVEN_LOCAL}/re2:${DEEPHAVEN_LOCAL}/zlib:${DEEPHAVEN_LOCAL}/grpc:${DEEPHAVEN_LOCAL}/arrow:${DEEPHAVEN_LOCAL}/deephaven +export NCPUS=$(getconf _NPROCESSORS_ONLN) +cmake -DCMAKE_INSTALL_PREFIX=${DEEPHAVEN_LOCAL}/deephaven .. && make -j$NCPUS install diff --git a/R/rdeephaven/lib/cpp-dependencies/build-dependencies.sh b/R/rdeephaven/lib/cpp-dependencies/build-dependencies.sh new file mode 100755 index 00000000000..608d26f9591 --- /dev/null +++ b/R/rdeephaven/lib/cpp-dependencies/build-dependencies.sh @@ -0,0 +1,390 @@ +#!/bin/bash + +# +# Copyright (c) 2016-2022 Deephaven Data Labs and Patent Pending +# + +# +# Tested on Ubuntu 20.04 +# + +# Fail on first error; echo each command before executing. +set -eux + +clean="no" +if [ "$#" -ge 1 ]; then + if [ "$1" == "--clean" ]; then + clean="yes" + shift + fi +fi + +step=all +if [ "$#" -gt 0 ]; then + if [ "$#" -gt 1 ]; then + echo "Usage: $0 [checkout|protobuf|re2|gflags|absl|flatbuffers|cares|zlib|grpc|arrow|immer|boost|env]" + exit 1 + fi + step=$1 +fi + +# Add anything to PATH that should take precendence here. +# export PATH=/l/cmake/3.21.2/bin:/l/gcc/11.2.0/bin:$PATH + +# Edit to reflect your compiler preferences, or comment out for using system versions. +# export CC=/l/gcc/11.2.0/bin/gcc +# export CXX=/l/gcc/11.2.0/bin/g++ + +# Set to "Debug" or "Release" +: ${BUILD_TYPE:=Debug} + +# Set to where you intend the sources for and installed depdenencies to live. +: ${DHDEPS_HOME:=$(pwd)} + +if [ "$step" = "all" ]; then +# If you want to rebuild only certain parts or skip phases change "yes" to "no" below. +# Note this assumues you at least once built everything; otherwise dependencies among libraries +# may fail. + : ${BUILD_PROTOBUF:=yes} + : ${BUILD_RE2:=yes} + : ${BUILD_GFLAGS:=yes} + : ${BUILD_ABSL:=yes} + : ${BUILD_FLATBUFFERS:=no} + : ${BUILD_CARES:=yes} + : ${BUILD_ZLIB:=yes} + : ${BUILD_GRPC:=yes} + : ${BUILD_ARROW:=yes} + : ${BUILD_IMMER:=yes} + : ${GENERATE_ENV:=yes} +else + BUILD_PROTOBUF=no + BUILD_RE2=no + BUILD_GFLAGS=no + BUILD_ABSL=no + BUILD_FLATBUFFERS=no + BUILD_CARES=no + BUILD_ZLIB=no + BUILD_GRPC=no + BUILD_ARROW=no + BUILD_IMMER=no + GENERATE_ENV=no + case "$step" in + protobuf) + BUILD_PROTOBUF=yes + ;; + re2) + BUILD_RE2=yes + ;; + gflags) + BUILD_GFLAGS=yes + ;; + absl) + BUILD_ABSL=yes + ;; + flatbuffers) + BUILD_FLATBUFFERS=yes + ;; + cares) + BUILD_CARES=yes + ;; + zlib) + BUILD_ZLIB=yes + ;; + grpc) + BUILD_GRPC=yes + ;; + arrow) + BUILD_ARROW=yes + ;; + immer) + BUILD_IMMER=yes + ;; + env) + GENERATE_ENV=yes + ;; + *) + echo "$0: unrecognized option: $step" 1>&2 + exit 1 + ;; + esac +fi + +# At the point of this writing, the latest immer release is pretty old. +# We want something a lot more recent, but don't want to track head as is a moving +# target and we can't guarantee things will continue to compile/be consistent. +# So we select a particular SHA. +: ${IMMER_SHA:=e5d79ed80ec74d511cc4f52fb68feeac66507f2c} + +# +# End of user customization section; you should not need to modify the code below +# unless you need to do partial re-builds. +# + +# How many CPUs to use in -j arguments to make. +: ${NCPUS:=$(getconf _NPROCESSORS_ONLN)} + +# Where the checked out sources for dependencies will go +: ${SRC:=$DHDEPS_HOME/src} + +# Where the install prefix paths will go +: ${PFX:=$DHDEPS_HOME/local} + +# Let's get make to print out commands as they run +export VERBOSE=1 + +export CMAKE_PREFIX_PATH=\ +${PFX}/abseil:\ +${PFX}/cares:\ +${PFX}/flatbuffers:\ +${PFX}/gflags:\ +${PFX}/protobuf:\ +${PFX}/re2:\ +${PFX}/zlib:\ +${PFX}/grpc:\ +${PFX}/arrow:\ +${PFX}/immer:\ +${PFX}/deephaven + +if [ ! -d $SRC ]; then + mkdir -p $SRC +fi + +if [ ! -d $PFX ]; then + mkdir -p $PFX +fi + +# +# Each phase below should explicitly change to expected current working directory before starting; +# there is no guarantee where the CWD is after a prior phase. +# +: ${GIT_FLAGS:="--quiet -c advice.detachedHead=false"} + +### Protobuf +if [ "$BUILD_PROTOBUF" = "yes" ]; then + echo + echo "*** Clone protobuf" + cd $SRC + git clone $GIT_FLAGS -b v3.20.1 --depth 1 https://github.com/protocolbuffers/protobuf.git + echo + echo "*** Building protobuf" + cd $SRC/protobuf + mkdir -p cmake/build-dir && cd cmake/build-dir + cmake -Dprotobuf_BUILD_TESTS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DCMAKE_INSTALL_PREFIX=${PFX}/protobuf .. + make -j$NCPUS + make install + if [ "$clean" = "yes" ]; then + rm -fr "$SRC/protobuf" + fi + echo "*** protobuf DONE" +fi + +### re2 +if [ "$BUILD_RE2" = "yes" ]; then + echo + echo "*** Clone re2" + cd $SRC + git clone $GIT_FLAGS -b 2022-04-01 --depth 1 https://github.com/google/re2.git + echo + echo "*** Building re2" + cd $SRC/re2 + mkdir -p build-dir && cd build-dir + cmake -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DCMAKE_INSTALL_PREFIX=${PFX}/re2 .. + make -j$NCPUS + make install + if [ "$clean" = "yes" ]; then + rm -fr "$SRC/re2" + fi + echo "*** re2 DONE" +fi + +### gflags +if [ "$BUILD_GFLAGS" = "yes" ]; then + echo + echo "*** Clone gflags" + cd $SRC + git clone $GIT_FLAGS -b v2.2.2 --depth 1 https://github.com/gflags/gflags.git + echo + echo "*** Building gflags" + cd $SRC/gflags + mkdir -p build-dir && cd build-dir + cmake -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DCMAKE_INSTALL_PREFIX=${PFX}/gflags .. + make -j$NCPUS + make install + if [ "$clean" = "yes" ]; then + rm -fr "$SRC/gflags" + fi + echo "*** gflags DONE" +fi + +### absl +if [ "$BUILD_ABSL" = "yes" ]; then + echo + echo "*** Clone abseil" + cd $SRC + git clone $GIT_FLAGS -b 20210324.2 --depth 1 https://github.com/abseil/abseil-cpp.git + echo + echo "*** Building abseil" + cd $SRC/abseil-cpp + mkdir -p cmake/build-dir && cd cmake/build-dir + cmake -DCMAKE_CXX_STANDARD=11 -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DCMAKE_INSTALL_PREFIX=${PFX}/abseil ../.. + make -j$NCPUS + make install + if [ "$clean" = "yes" ]; then + rm -fr "$SRC/abseil-cpp" + fi + echo "*** abseil DONE" +fi + +### flatbuffers +if [ "$BUILD_FLATBUFFERS" = "yes" ]; then + echo + echo "*** Clone flatbuffers" + cd $SRC + git clone $GIT_FLAGS -b v2.0.6 --depth 1 https://github.com/google/flatbuffers.git + echo + echo "*** Building flatbuffers" + cd $SRC/flatbuffers + mkdir -p build-dir && cd build-dir + cmake -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DCMAKE_INSTALL_PREFIX=${PFX}/flatbuffers .. + make -j$NCPUS + make install + if [ "$clean" = "yes" ]; then + rm -fr "$SRC/flatbuffers" + fi + echo "*** flatbuffers DONE" +fi + +### c-ares +if [ "$BUILD_CARES" = "yes" ]; then + echo + echo "*** Clone ares" + cd $SRC + git clone $GIT_FLAGS -b cares-1_18_1 --depth 1 https://github.com/c-ares/c-ares.git + echo + echo "*** Building c-ares" + cd $SRC/c-ares + mkdir -p build-dir && cd build-dir + cmake -DCARES_STATIC=ON -DCARES_SHARED=OFF -DCARES_STATIC_PIC=ON -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DCMAKE_INSTALL_PREFIX=${PFX}/cares .. + make -j$NCPUS + make install + if [ "$clean" = "yes" ]; then + rm -fr "$SRC/c-ares" + fi + echo "*** ares DONE" +fi + +### zlib +if [ "$BUILD_ZLIB" = "yes" ]; then + echo + echo "*** Clone zlib" + cd $SRC + git clone $GIT_FLAGS -b v1.2.11 --depth 1 https://github.com/madler/zlib + echo + echo "*** Building zlib" + cd $SRC/zlib + mkdir -p build-dir && cd build-dir + cmake -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DCMAKE_INSTALL_PREFIX=${PFX}/zlib .. + make -j$NCPUS + make install + if [ "$clean" = "yes" ]; then + rm -fr "$SRC/zlib" + fi + echo "*** zlib DONE" +fi + +### grpc +if [ "$BUILD_GRPC" = "yes" ]; then + echo + echo "*** Clone grpc" + cd $SRC + git clone $GIT_FLAGS -b v1.45.2 --depth 1 https://github.com/grpc/grpc + echo + echo "*** Building grpc" + cd $SRC/grpc + mkdir -p cmake/build-dir && cd cmake/build-dir + cmake -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DCMAKE_POSITION_INDEPENDENT_CODE=TRUE -DCMAKE_INSTALL_PREFIX=${PFX}/grpc -DgRPC_INSTALL=ON \ + -DgRPC_ABSL_PROVIDER=package -DgRPC_CARES_PROVIDER=package -DgRPC_PROTOBUF_PROVIDER=package \ + -DgRPC_RE2_PROVIDER=package -DgRPC_SSL_PROVIDER=package -DgRPC_ZLIB_PROVIDER=package ../.. + make -j$NCPUS + make install + if [ "$clean" = "yes" ]; then + rm -fr "$SRC/grpc" + fi + echo "*** grpc DONE" +fi + +### arrow +if [ "$BUILD_ARROW" = "yes" ]; then + echo + echo "*** Clone arrow" + cd $SRC + git clone $GIT_FLAGS -b apache-arrow-7.0.0 --depth 1 https://github.com/apache/arrow + echo + echo "*** Patching arrow" + # Apply apache arrow patch. + (cd arrow && patch -p1 <> LoadRecordBatchSubset( + auto column = std::make_shared(); + RETURN_NOT_OK(loader.Load(&field, column.get())); + if (metadata->length() != column->length) { +- return Status::IOError("Array length did not match record batch length"); ++ // return Status::IOError("Array length did not match record batch length"); + } + columns[i] = std::move(column); + if (inclusion_mask) { +EOF +) + echo + echo "*** Building arrow" + export CPATH=${PFX}/abseil/include${CPATH+:$CPATH} + export CPATH=${PFX}/protobuf/include${CPATH+:$CPATH} + cd $SRC/arrow/cpp + mkdir -p build-dir && cd build-dir + cmake -DARROW_BUILD_STATIC=ON -DARROW_FLIGHT=ON -DARROW_CSV=ON -DARROW_FILESYSTEM=ON -DARROW_DATASET=ON -DARROW_PARQUET=ON \ + -DARROW_WITH_BZ2=ON -DARROW_WITH_ZLIB=ON -DARROW_WITH_LZ4=ON -DARROW_WITH_SNAPPY=ON -DARROW_WITH_ZSTD=ON -DARROW_WITH_BROTLI=ON \ + -DARROW_SIMD_LEVEL=NONE \ + -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DCMAKE_INSTALL_PREFIX=${PFX}/arrow .. + make -j$NCPUS + make install + if [ "$clean" = "yes" ]; then + rm -fr "$SRC/arrow" + fi + echo "*** arrow DONE" +fi + +### immer +if [ "$BUILD_IMMER" = "yes" ]; then + echo + echo "*** Clone immer" + cd $SRC + git clone $GIT_FLAGS https://github.com/arximboldi/immer.git && (cd immer && git checkout "${IMMER_SHA}") + echo + echo "*** Building immer" + cd $SRC/immer + mkdir -p build-dir && cd build-dir + cmake -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DCMAKE_INSTALL_PREFIX=${PFX}/immer .. + make -j$NCPUS + make install + if [ "$clean" = "yes" ]; then + rm -fr "$SRC/immer" + fi + echo "*** immer DONE" +fi + +echo DONE. +echo + +if [ "$GENERATE_ENV" = "yes" ]; then + echo -n "Creating env.sh..." + cd $DHDEPS_HOME + (echo "export CMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH" +# Ensure this is evaluated not now, but when the generated code is read. + echo 'export NCPUS=`getconf _NPROCESSORS_ONLN`') > env.sh + echo DONE. +fi + +exit 0 diff --git a/R/rdeephaven/man/Client.Rd b/R/rdeephaven/man/Client.Rd new file mode 100644 index 00000000000..5699dc1508e --- /dev/null +++ b/R/rdeephaven/man/Client.Rd @@ -0,0 +1,134 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/client_wrapper.R +\docType{class} +\name{Client} +\alias{Client} +\title{The Deephaven Client} +\description{ +The Deephaven Client class is responsible for establishing and maintaining +a connection to a running Deephaven server and facilitating basic server requests. +} +\examples{ + +# connect to the Deephaven server running on "localhost:10000" with anonymous 'default' authentication +client_options <- ClientOptions$new() +client_options$set_default_authentication() +client <- Client$new(target="localhost:10000", client_options=client_options) + +# open a table that already exists on the server +new_table_handle1 <- client$open_table("table_on_the_server") + +# create a new dataframe, import onto the server, and retrieve a reference +new_data_frame <- data.frame(matrix(rnorm(10 * 1000), nrow = 10)) +new_table_handle2 <- client$import_table(new_data_frame) + +# run a python script on the server (default client options specify a Python console) +client$run_script("print([i for i in range(10)])") +} +\section{Methods}{ +\subsection{Public methods}{ +\itemize{ +\item \href{#method-Client-new}{\code{Client$new()}} +\item \href{#method-Client-open_table}{\code{Client$open_table()}} +\item \href{#method-Client-import_table}{\code{Client$import_table()}} +\item \href{#method-Client-run_script}{\code{Client$run_script()}} +\item \href{#method-Client-clone}{\code{Client$clone()}} +} +} +\if{html}{\out{
}} +\if{html}{\out{}} +\if{latex}{\out{\hypertarget{method-Client-new}{}}} +\subsection{Method \code{new()}}{ +Connect to a running Deephaven server. +\subsection{Usage}{ +\if{html}{\out{
}}\preformatted{Client$new(target, client_options)}\if{html}{\out{
}} +} + +\subsection{Arguments}{ +\if{html}{\out{
}} +\describe{ +\item{\code{target}}{The address of the Deephaven server.} + +\item{\code{client_options}}{ClientOptions instance with the parameters needed to connect to the server. +See ?ClientOptions for more information.} +} +\if{html}{\out{
}} +} +} +\if{html}{\out{
}} +\if{html}{\out{}} +\if{latex}{\out{\hypertarget{method-Client-open_table}{}}} +\subsection{Method \code{open_table()}}{ +Opens a table named 'name' from the server if it exists. +\subsection{Usage}{ +\if{html}{\out{
}}\preformatted{Client$open_table(name)}\if{html}{\out{
}} +} + +\subsection{Arguments}{ +\if{html}{\out{
}} +\describe{ +\item{\code{name}}{Name of the table to open from the server as a string.} +} +\if{html}{\out{
}} +} +\subsection{Returns}{ +TableHandle reference to the requested table. +} +} +\if{html}{\out{
}} +\if{html}{\out{}} +\if{latex}{\out{\hypertarget{method-Client-import_table}{}}} +\subsection{Method \code{import_table()}}{ +Imports a new table to the Deephaven server. Note that this new table is not automatically bound to +a variable name on the server. See `?TableHandle` for more information. +\subsection{Usage}{ +\if{html}{\out{
}}\preformatted{Client$import_table(table_object)}\if{html}{\out{
}} +} + +\subsection{Arguments}{ +\if{html}{\out{
}} +\describe{ +\item{\code{table_object}}{An R Data Frame, an R Tibble, an Arrow Table, or an Arrow RecordBatchReader +containing the data to import to the server.} +} +\if{html}{\out{
}} +} +\subsection{Returns}{ +TableHandle reference to the new table. +} +} +\if{html}{\out{
}} +\if{html}{\out{}} +\if{latex}{\out{\hypertarget{method-Client-run_script}{}}} +\subsection{Method \code{run_script()}}{ +Runs a script on the server. The script must be in the language that the server console was started with. +\subsection{Usage}{ +\if{html}{\out{
}}\preformatted{Client$run_script(script)}\if{html}{\out{
}} +} + +\subsection{Arguments}{ +\if{html}{\out{
}} +\describe{ +\item{\code{script}}{Code to be executed on the server as a string.} +} +\if{html}{\out{
}} +} +} +\if{html}{\out{
}} +\if{html}{\out{}} +\if{latex}{\out{\hypertarget{method-Client-clone}{}}} +\subsection{Method \code{clone()}}{ +The objects of this class are cloneable with this method. +\subsection{Usage}{ +\if{html}{\out{
}}\preformatted{Client$clone(deep = FALSE)}\if{html}{\out{
}} +} + +\subsection{Arguments}{ +\if{html}{\out{
}} +\describe{ +\item{\code{deep}}{Whether to make a deep clone.} +} +\if{html}{\out{
}} +} +} +} diff --git a/R/rdeephaven/man/ClientOptions.Rd b/R/rdeephaven/man/ClientOptions.Rd new file mode 100644 index 00000000000..308181c2c63 --- /dev/null +++ b/R/rdeephaven/man/ClientOptions.Rd @@ -0,0 +1,105 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/client_options_wrapper.R +\name{ClientOptions} +\alias{ClientOptions} +\title{Deephaven ClientOptions} +\description{ +Client options provide a simple interface to the Deephaven server's authentication protocols. +This makes it easy to connect to a Deephaven server with any flavor of authentication, and shields the API from +any future changes to the underlying implementation. + +Currently, we support three different kinds of authentication that your Deephaven server might be using: + +- "default": Default (or anonymous) authentication does not require any username or password. If you are + running the Deephaven server locally, this is probably the kind of authentication you are using. + +- "basic": Basic authentication requires a standard username and password pair. + +- "custom": Custom authentication requires general key-value pairs. + +In addition to setting your authentication parameters when you connect to a client, you can also start +a console in one of our supported server languages. We currently support Python and Groovy, and if you +want start a console upon client connection, you must ensure that the server you're connecting to was +started with support for the language you want to use. +} +\examples{ + +# connect to the Deephaven server running on "localhost:10000" with anonymous 'default' authentication +client_options <- ClientOptions$new() +client_options$set_default_authentication() +client <- Client$new(target="localhost:10000", client_options=client_options) +} +\section{Methods}{ +\subsection{Public methods}{ +\itemize{ +\item \href{#method-ClientOptions-new}{\code{ClientOptions$new()}} +\item \href{#method-ClientOptions-set_default_authentication}{\code{ClientOptions$set_default_authentication()}} +\item \href{#method-ClientOptions-set_basic_authentication}{\code{ClientOptions$set_basic_authentication()}} +\item \href{#method-ClientOptions-set_custom_authentication}{\code{ClientOptions$set_custom_authentication()}} +\item \href{#method-ClientOptions-set_session_type}{\code{ClientOptions$set_session_type()}} +\item \href{#method-ClientOptions-clone}{\code{ClientOptions$clone()}} +} +} +\if{html}{\out{
}} +\if{html}{\out{}} +\if{latex}{\out{\hypertarget{method-ClientOptions-new}{}}} +\subsection{Method \code{new()}}{ +\subsection{Usage}{ +\if{html}{\out{
}}\preformatted{ClientOptions$new()}\if{html}{\out{
}} +} + +} +\if{html}{\out{
}} +\if{html}{\out{}} +\if{latex}{\out{\hypertarget{method-ClientOptions-set_default_authentication}{}}} +\subsection{Method \code{set_default_authentication()}}{ +\subsection{Usage}{ +\if{html}{\out{
}}\preformatted{ClientOptions$set_default_authentication()}\if{html}{\out{
}} +} + +} +\if{html}{\out{
}} +\if{html}{\out{}} +\if{latex}{\out{\hypertarget{method-ClientOptions-set_basic_authentication}{}}} +\subsection{Method \code{set_basic_authentication()}}{ +\subsection{Usage}{ +\if{html}{\out{
}}\preformatted{ClientOptions$set_basic_authentication(username, password)}\if{html}{\out{
}} +} + +} +\if{html}{\out{
}} +\if{html}{\out{}} +\if{latex}{\out{\hypertarget{method-ClientOptions-set_custom_authentication}{}}} +\subsection{Method \code{set_custom_authentication()}}{ +\subsection{Usage}{ +\if{html}{\out{
}}\preformatted{ClientOptions$set_custom_authentication(auth_key, auth_value)}\if{html}{\out{
}} +} + +} +\if{html}{\out{
}} +\if{html}{\out{}} +\if{latex}{\out{\hypertarget{method-ClientOptions-set_session_type}{}}} +\subsection{Method \code{set_session_type()}}{ +\subsection{Usage}{ +\if{html}{\out{
}}\preformatted{ClientOptions$set_session_type(session_type)}\if{html}{\out{
}} +} + +} +\if{html}{\out{
}} +\if{html}{\out{}} +\if{latex}{\out{\hypertarget{method-ClientOptions-clone}{}}} +\subsection{Method \code{clone()}}{ +The objects of this class are cloneable with this method. +\subsection{Usage}{ +\if{html}{\out{
}}\preformatted{ClientOptions$clone(deep = FALSE)}\if{html}{\out{
}} +} + +\subsection{Arguments}{ +\if{html}{\out{
}} +\describe{ +\item{\code{deep}}{Whether to make a deep clone.} +} +\if{html}{\out{
}} +} +} +} diff --git a/R/rdeephaven/man/rdeephaven-package.Rd b/R/rdeephaven/man/rdeephaven-package.Rd new file mode 100644 index 00000000000..c9b873e2a62 --- /dev/null +++ b/R/rdeephaven/man/rdeephaven-package.Rd @@ -0,0 +1,34 @@ +\name{rdeephaven-package} +\alias{rdeephaven-package} +\alias{rdeephaven} +\docType{package} +\title{ + A short title line describing what the package does +} +\description{ + A more detailed description of what the package does. A length + of about one to five lines is recommended. +} +\details{ + This section should provide a more detailed overview of how to use the + package, including the most important functions. +} +\author{ +Your Name, email optional. + +Maintainer: Your Name +} +\references{ + This optional section can contain literature or other references for + background information. +} +\keyword{ package } +\seealso{ + Optional links to other man pages +} +\examples{ + \dontrun{ + ## Optional simple examples of the most important functions + ## These can be in \dontrun{} and \donttest{} blocks. + } +} diff --git a/R/rdeephaven/src/CMakeLists.txt b/R/rdeephaven/src/CMakeLists.txt new file mode 100644 index 00000000000..3508280ca15 --- /dev/null +++ b/R/rdeephaven/src/CMakeLists.txt @@ -0,0 +1,16 @@ +cmake_minimum_required(VERSION 3.16) +project(client_caller) + +set(CMAKE_CXX_STANDARD 17) + +find_package(Arrow REQUIRED) +find_package(ArrowFlight REQUIRED HINTS ${Arrow_DIR}) +find_package(Immer REQUIRED) +find_package(Protobuf REQUIRED) +find_package(gRPC REQUIRED) +find_package(Threads REQUIRED) +find_package(deephaven REQUIRED) + +add_executable(client_caller client_caller.cpp) + +target_link_libraries(client_caller deephaven::client) \ No newline at end of file diff --git a/R/rdeephaven/src/Makevars b/R/rdeephaven/src/Makevars new file mode 100644 index 00000000000..4c9455810c4 --- /dev/null +++ b/R/rdeephaven/src/Makevars @@ -0,0 +1,67 @@ +$(info The name of the shared library to be created is: $(SHLIB)) +rwildcard=$(foreach d,$(wildcard $(1:=/*)),$(call rwildcard,$d,$2) $(filter $(subst *,%,$2),$d)) + +DEPENDENCY_DIRS = -L../lib/cpp-dependencies/local/abseil/lib \ + -L../lib/cpp-dependencies/local/cares/lib \ + -L../lib/cpp-dependencies/local/gflags/lib \ + -L../lib/cpp-dependencies/local/grpc/lib \ + -L../lib/cpp-dependencies/local/immer/lib \ + -L../lib/cpp-dependencies/local/protobuf/lib \ + -L../lib/cpp-dependencies/local/re2/lib \ + -L../lib/cpp-dependencies/local/zlib/lib + + +DEPENDENCY_LIBS = -lclient -lprotobufd \ + ../lib/cpp-dependencies/local/arrow/lib/libarrow_flight.a \ + ../lib/cpp-dependencies/local/arrow/lib/libarrow.a \ + ../lib/cpp-dependencies/local/arrow/lib/libarrow_bundled_dependencies.a \ + -lgrpc++ -lgrpc -laddress_sorting -lgpr -lupb -labsl_raw_hash_set -labsl_hashtablez_sampler -labsl_exponential_biased -labsl_hash -labsl_city \ + -labsl_wyhash -labsl_statusor -labsl_bad_variant_access -lgpr -lupb \ + -labsl_status -labsl_random_distributions -labsl_random_seed_sequences -labsl_random_internal_pool_urbg -labsl_random_internal_randen \ + -labsl_random_internal_randen_hwaes -labsl_random_internal_randen_hwaes_impl -labsl_random_internal_randen_slow -labsl_random_internal_platform \ + -labsl_random_internal_seed_material -labsl_random_seed_gen_exception -labsl_cord -labsl_bad_optional_access -labsl_str_format_internal \ + -labsl_synchronization -labsl_stacktrace -labsl_symbolize -labsl_debugging_internal -labsl_demangle_internal -labsl_graphcycles_internal \ + -labsl_malloc_internal -labsl_time -labsl_strings -labsl_throw_delegate -labsl_int128 -labsl_strings_internal -labsl_base -labsl_raw_logging_internal \ + -labsl_log_severity -labsl_spinlock_wait -labsl_civil_time -labsl_time_zone -lssl -lre2 -lcares + + +# tells the compiler where to look for additional include directories +PKG_CXXFLAGS = -I"../lib/cpp-client/deephaven/client/include/public" \ + -I"../lib/cpp-client/deephaven/dhcore/include/public" \ + -I"../lib/cpp-dependencies/local/arrow/include" \ + -I"/usr/share/R/include" \ + -I"/usr/local/lib/R/site-library/Rcpp/include" \ + -I"arrow_c_api.h" + +# list of required libraries, including deephaven and associated dependencies +PKG_LIBS = $(LAPACK_LIBS) \ + $(BLAS_LIBS) \ + $(FLIBS) \ + $(SHLIB_OPENMP_CXXFLAGS) \ + $(R_CPPFLAGS) \ + -L"/usr/lib/R/lib" -lR \ + $(R_CXXFLAGS) \ + $(DEPENDENCY_DIRS) \ + $(DEPENDENCY_LIBS) \ + -L"../lib/cpp-client/deephaven/build/client/dhcore_dir" -ldhcore \ + -L"../lib/cpp-client/deephaven/build/client" -lclient + +CC = `"${R_HOME}/bin/R" CMD config CC` +CFLAGS = `"${R_HOME}/bin/R" CMD config CFLAGS` +CXX17 = `"${R_HOME}/bin/R" CMD config CXX17` +CXX17STD=`"${R_HOME}/bin/R" CMD config CXX17STD` +CXXFLAGS = `"${R_HOME}/bin/R" CMD config CXX17FLAGS` + +# set C++ standard +CXX_STD = CXX17 +# all src directory c++ source files +SOURCES = $(wildcard *.cpp) +# compiled objects to link +OBJECTS = $(SOURCES:.cpp=.o) + +all: $(SHLIB) + +clean: + find . -type f -name '*.o' -delete + find . -type f -name '*.so' -delete + rm RcppExports.cpp diff --git a/R/rdeephaven/src/RcppExports.cpp b/R/rdeephaven/src/RcppExports.cpp new file mode 100644 index 00000000000..adebb304614 --- /dev/null +++ b/R/rdeephaven/src/RcppExports.cpp @@ -0,0 +1,24 @@ +// Generated by using Rcpp::compileAttributes() -> do not edit by hand +// Generator token: 10BE3573-1514-4C36-9D1C-5A225CD40393 + +#include + +using namespace Rcpp; + +#ifdef RCPP_USE_GLOBAL_ROSTREAM +Rcpp::Rostream& Rcpp::Rcout = Rcpp::Rcpp_cout_get(); +Rcpp::Rostream& Rcpp::Rcerr = Rcpp::Rcpp_cerr_get(); +#endif + + +RcppExport SEXP _rcpp_module_boot_DeephavenInternalModule(); + +static const R_CallMethodDef CallEntries[] = { + {"_rcpp_module_boot_DeephavenInternalModule", (DL_FUNC) &_rcpp_module_boot_DeephavenInternalModule, 0}, + {NULL, NULL, 0} +}; + +RcppExport void R_init_rdeephaven(DllInfo *dll) { + R_registerRoutines(dll, NULL, CallEntries, NULL, NULL); + R_useDynamicSymbols(dll, FALSE); +} diff --git a/R/rdeephaven/src/client.cpp b/R/rdeephaven/src/client.cpp new file mode 100644 index 00000000000..2a38c4c0f03 --- /dev/null +++ b/R/rdeephaven/src/client.cpp @@ -0,0 +1,224 @@ +#include +#include +#include +#include +#include + +#include "deephaven/client/client.h" +#include "deephaven/client/flight.h" + +#include +#include + +#include + +// forward declaration of classes +class ClientOptionsWrapper; +class ClientWrapper; + +// ######################### DH WRAPPERS ######################### + +class TableHandleWrapper { +public: + TableHandleWrapper(deephaven::client::TableHandle ref_table) : internal_tbl_hdl(std::move(ref_table)) {}; + + // TODO: DEEPHAVEN QUERY METHODS WILL GO HERE + + /** + * Binds the table referenced by this table handle to a variable on the server called tableName. + * Without this call, new tables are not accessible from the client. + * @param tableName Name for the new table on the server. + */ + void bindToVariable(std::string tableName) { + internal_tbl_hdl.bindToVariable(tableName); + } + + /** + * Creates and returns a pointer to an ArrowArrayStream C struct containing the data from the table referenced by internal_tbl_hdl. + * Intended to be used for creating an Arrow RecordBatchReader in R via RecordBatchReader$import_from_c(ptr). + */ + SEXP getArrowArrayStreamPtr() { + + std::shared_ptr fsr = internal_tbl_hdl.getFlightStreamReader(); + + std::vector> empty_record_batches; + DEEPHAVEN_EXPR_MSG(fsr->ReadAll(&empty_record_batches)); // TODO: need to add OK or throw + + std::shared_ptr record_batch_reader = arrow::RecordBatchReader::Make(empty_record_batches).ValueOrDie(); + ArrowArrayStream* stream_ptr = new ArrowArrayStream(); + arrow::ExportRecordBatchReader(record_batch_reader, stream_ptr); + + // XPtr is needed here to ensure Rcpp can properly handle type casting, as it does not like raw pointers + return Rcpp::XPtr(stream_ptr, true); + }; + +private: + deephaven::client::TableHandle internal_tbl_hdl; +}; + + +// TODO: Document this guy +class ClientOptionsWrapper { +public: + + ClientOptionsWrapper() { + internal_options = new deephaven::client::ClientOptions(); + }; + + void setDefaultAuthentication() { + internal_options->setDefaultAuthentication(); + }; + + void setBasicAuthentication(const std::string &username, const std::string &password) { + internal_options->setBasicAuthentication(username, password); + }; + + void setCustomAuthentication(const std::string &authenticationKey, const std::string &authenticationValue) { + internal_options->setCustomAuthentication(authenticationKey, authenticationValue); + }; + + void setSessionType(const std::string &sessionType) { + internal_options->setSessionType(sessionType); + }; + +private: + + deephaven::client::ClientOptions* internal_options; + friend ClientWrapper* newClientWrapper(const std::string &target, const ClientOptionsWrapper &client_options); +}; + + + +class ClientWrapper { +public: + + /** + * Fetches a reference to a table named tableName on the server if it exists. + * @param tableName Name of the table to search for. + * @return TableHandle reference to the fetched table. + */ + TableHandleWrapper* openTable(std::string tableName) { + return new TableHandleWrapper(internal_tbl_hdl_mngr.fetchTable(tableName)); + }; + + /** + * Runs a script on the server in the console language if a console was created. + * @param code String of the code to be executed on the server. + */ + void runScript(std::string code) { + internal_tbl_hdl_mngr.runScript(code); + }; + + /** + * Checks for the existence of a table named tableName on the server. + * @param tableName Name of the table to search for. + * @return Boolean indicating whether tableName exists on the server or not. + */ + bool checkForTable(std::string tableName) { + // we have to first fetchTable to check existence, fetchTable does not fail on its own, but .observe() will fail if table doesn't exist + deephaven::client::TableHandle table_handle = internal_tbl_hdl_mngr.fetchTable(tableName); + try { + table_handle.observe(); + } catch(...) { + return false; + } + return true; + }; + + /** + * Allocates memory for an ArrowArrayStream C struct and returns a pointer to the new chunk of memory. + * Intended to be used to get a pointer to pass to Arrow's R library RecordBatchReader$export_to_c(ptr). + */ + SEXP newArrowArrayStreamPtr() { + ArrowArrayStream* stream_ptr = new ArrowArrayStream(); + return Rcpp::XPtr(stream_ptr, true); + }; + + /** + * Uses a pointer to a populated ArrowArrayStream C struct to create a new table on the server from the data in the C struct. + * @param stream_ptr Pointer to an existing and populated ArrayArrayStream, populated by a call to RecordBatchReader$export_to_c(ptr) from R. + */ + TableHandleWrapper* newTableFromArrowArrayStreamPtr(Rcpp::XPtr stream_ptr) { + + auto wrapper = internal_tbl_hdl_mngr.createFlightWrapper(); + arrow::flight::FlightCallOptions options; + wrapper.addAuthHeaders(&options); + + // extract RecordBatchReader from the struct pointed to by the passed tream_ptr + std::shared_ptr record_batch_reader = arrow::ImportRecordBatchReader(stream_ptr.get()).ValueOrDie(); + auto schema = record_batch_reader.get()->schema(); + + // write RecordBatchReader data to table on server with DoPut + std::unique_ptr fsw; + std::unique_ptr fmr; + auto [new_tbl_hdl, fd] = internal_tbl_hdl_mngr.newTableHandleAndFlightDescriptor(); + DEEPHAVEN_EXPR_MSG(wrapper.flightClient()->DoPut(options, fd, schema, &fsw, &fmr)); // TODO: need to add okOrThrow + while(true) { + std::shared_ptr this_batch; + DEEPHAVEN_EXPR_MSG(record_batch_reader->ReadNext(&this_batch)); // TODO: need to add ok or throw + if (this_batch == nullptr) { + break; + } + DEEPHAVEN_EXPR_MSG(fsw->WriteRecordBatch(*this_batch)); // TODO: need to add okOrThrow + } + DEEPHAVEN_EXPR_MSG(fsw->DoneWriting()); // TODO: need to add okOrThrow + DEEPHAVEN_EXPR_MSG(fsw->Close()); // TODO: need to add okOrThrow + + return new TableHandleWrapper(new_tbl_hdl); + }; + +private: + ClientWrapper(deephaven::client::Client ref) : internal_client(std::move(ref)) {}; + + const deephaven::client::Client internal_client; + const deephaven::client::TableHandleManager internal_tbl_hdl_mngr = internal_client.getManager(); + + friend ClientWrapper* newClientWrapper(const std::string &target, const ClientOptionsWrapper &client_options); +}; + +// factory method for calling private constructor, Rcpp does not like in constructor +// the current implementation of passing authentication args to C++ client is terrible and needs to be redone. Only this could make Rcpp happy in a days work + +/** + * Factory method for creating a new ClientWrapper, which is responsible for maintaining a connection to the client. + * @param target URL that the server is running on. + * @param client_options A ClientOptionsWrapper containing the server connection information. See deephaven::client::ClientOptions for more information. + */ +ClientWrapper* newClientWrapper(const std::string &target, const ClientOptionsWrapper &client_options) { + return new ClientWrapper(deephaven::client::Client::connect(target, *client_options.internal_options)); +}; + + + +// ######################### RCPP GLUE ######################### + +using namespace Rcpp; + +RCPP_EXPOSED_CLASS(ClientOptionsWrapper) +RCPP_EXPOSED_CLASS(TableHandleWrapper) +RCPP_EXPOSED_CLASS(ArrowArrayStream) + +RCPP_MODULE(DeephavenInternalModule) { + + class_("INTERNAL_TableHandle") + .method("bind_to_variable", &TableHandleWrapper::bindToVariable) + .method("get_arrow_array_stream_ptr", &TableHandleWrapper::getArrowArrayStreamPtr) + ; + + class_("INTERNAL_ClientOptions") + .constructor() + .method("set_default_authentication", &ClientOptionsWrapper::setDefaultAuthentication) + .method("set_basic_authentication", &ClientOptionsWrapper::setBasicAuthentication) + .method("set_custom_authentication", &ClientOptionsWrapper::setCustomAuthentication) + .method("set_session_type", &ClientOptionsWrapper::setSessionType) + ; + + class_("INTERNAL_Client") + .factory(newClientWrapper) + .method("open_table", &ClientWrapper::openTable) + .method("check_for_table", &ClientWrapper::checkForTable) + .method("run_script", &ClientWrapper::runScript) + .method("new_arrow_array_stream_ptr", &ClientWrapper::newArrowArrayStreamPtr) + .method("new_table_from_arrow_array_stream_ptr", &ClientWrapper::newTableFromArrowArrayStreamPtr) + ; +} diff --git a/R/rdeephaven/tests/testthat.R b/R/rdeephaven/tests/testthat.R new file mode 100644 index 00000000000..4a424a22250 --- /dev/null +++ b/R/rdeephaven/tests/testthat.R @@ -0,0 +1,4 @@ +library(testthat) +library(rdeephaven) + +test_check("rdeephaven") \ No newline at end of file diff --git a/R/rdeephaven/tests/testthat/client_infrastructure.R b/R/rdeephaven/tests/testthat/client_infrastructure.R new file mode 100644 index 00000000000..309e719a52e --- /dev/null +++ b/R/rdeephaven/tests/testthat/client_infrastructure.R @@ -0,0 +1,32 @@ +generate_dataframes <- function() { + df1 <- data.frame(string_col = c("I", "am", "a", "string", "column"), + int_col = c(0, 1, 2, 3, 4), + dbl_col = c(1.65, 3.1234, 100000.5, 543.234567, 0.00)) + + df2 <- data.frame(col1 = rep(3.14, 100), + col2 = rep("hello!", 100), + col3 = rnorm(100)) + + df3 <- data.frame(matrix(rnorm(10 * 1000), nrow = 10)) + + df4 <- data.frame(time_col = seq.POSIXt(as.POSIXct(Sys.Date()), as.POSIXct(Sys.Date()+30), by = "1 sec")[2500000], + bool_col = sample(c(TRUE, FALSE), 2500000, TRUE), + int_col = sample(0:100000, 2500000, TRUE)) + + return(list(df1, df2, df3, df4)) +} + +setup_client <- function(target, dataframes=NULL) { + + client_options <- ClientOptions$new() + client_options$set_default_authentication() + client <- Client$new(target=target, client_options=client_options)) + + if (!is.null(dataframes)) { + for (i in 1:length(dataframes)) { + th = client$import_table(dataframes[i]) + th$bind_to_variable(paste0("table", i)) + } + } + return(client) +} diff --git a/R/rdeephaven/tests/testthat/test_client.R b/R/rdeephaven/tests/testthat/test_client.R new file mode 100644 index 00000000000..17021e26580 --- /dev/null +++ b/R/rdeephaven/tests/testthat/test_client.R @@ -0,0 +1 @@ +# TODO: TEST CLIENT CLASS \ No newline at end of file diff --git a/cpp-client/LICENSE b/cpp-client/LICENSE new file mode 100644 index 00000000000..d6456956733 --- /dev/null +++ b/cpp-client/LICENSE @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License.