As of the 2020.1 release, SAS® Event Stream Processing no longer supports the loadStreams and espCluster action sets. This github repository has been deprecated and will no longer be updated.
How to use the SAS CAS loadStream actionset to ingest data from an ESP running project into CAS.
Two different techniques are available to ingest data from SAS ESP directly into a CAS environment:
- Data ingestion through the SAS ESP CAS adapter, or
- Data ingestion through the loadStream CAS actionSet
The attached document focuses on the CAS actionSet method and will demonstrate how CAS actions can be used programmatically to process any amounts of data in a very simple way. This method applies to all versions of Viya, cloud and non-cloud based. The sample file referenced in the document is also provided to make it easy to replicate the process on your own environment.
Performing the steps documented in the paper requires familiarity with the following SAS products:
- SAS Studio
- SAS Event Stream Processing (ESP)
- SAS Event Stream Processing Studio (ESP Studio)
- For Viya 4 installations, a general knowledge of the Kubernetes architecture
With Viya 4, the code snippet provided in the paper and used to retrieve the IP address of the Kubernetes POD running the ESP project, requires the following:
- Execution of a command through a shell
- Availability of the nslookup OS command on the kubernetes cluster
Enabling the execution of commands through a shell is done through SAS Environment Manager, and it requires administrative privileges. Following is a list of the steps to perform:
-
Log on to SAS Environment Manager;
-
On the left side of the screen, click on "Contexts", then select "Compute contexts" from the "View" box;
-
Select "SAS Studio compute context" from the list of available contexts for your Viya installation;
-
On the next screen, click on the pencil icon on the far side of the lne that reads "Basic Properties";
-
The "Edit Compute Context" pop-up window appears. Scroll down to the "Attribute" section, then click on the plus (+) sign to add a new attribute;
-
Type "allowXCMD" in the "Attribute:" field, and "true" in the "Value:" field. Select "Save" when done;
-
Click on "Save" again to close the "Edit Compute Context" window;
-
Start or restart a SAS Studio session, and run a code snippet similar to the following to verify that the previous steps have been completed successfully:
%macro currdate; %let command = %bquote(")%str(date)%bquote("); filename shell pipe %unquote(&command); data _null_; length buffer $150.; infile shell dlm="¬"; input buffer; call symputx('currdate', buffer, 'G'); run; %put The current date and time is &currdate; %mend currdate; %currdate;The SAS log should display an output similar to the following:
The current date and time is Thu Feb 11 21:47:28 UTC 2021
As for nslookup, if the utility is not available on your cluster, the getent command can be used in its place as shown in the paper.
With Viya 4, the sample data file (measures.csv) has to be pre-loaded on a folder that is local to the POD where the ESP project runs. As an example, the Viya 4 environment used to test the code had access to an ephemeral NFS volume, which is a NFS-mounted folder belonging to a server external to the Kubernetes cluster. Placing any file on that folder means that the POD running the ESP project has access to it. Please check with your Kubernetes administrator to see what storage options are available to you.
We welcome your contributions! Please read CONTRIBUTING.md for details on how to submit contributions to this project.
This project is licensed under the Apache 2.0 License.