CLP Setup issue

[RajeshAccount]

I'm able to exxcute start-clp and compress scripts, I have imported the sample Hive Data set, when I try to search for following sbin % ./search "Task *" query, I'm getting below error:

My understanding: as per https://github.com/y-scope/clp/releases, clp-package-ubuntu-focal-x86_64-v0.0.2.tar.gz is built for ubuntu-focal. Do I need to have EC2 instance with ubuntu-focal only ? can you guide me to resolve the below error:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
Traceback (most recent call last):
File "/opt/clp/sbin/native/search", line 261, in
sys.exit(main(sys.argv))
File "/opt/clp/sbin/native/search", line 248, in main
for ip in set(socket.gethostbyname_ex(socket.gethostname())[2]):
socket.gaierror: [Errno -2] Name or service not known
Traceback (most recent call last):
File "./search", line 135, in
sys.exit(main(sys.argv))
File "./search", line 126, in main
subprocess.run(cmd, check=True)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'run', '-i', '--rm', '--network', 'host', '-w', '/opt/clp', '-u', '504:20', '--name', 'clp-search-61c9', '--mount', 'type=bind,src=/Users/rerajesh/Downloads/clp-package-ubuntu-focal-x86_64-v0.0.2,dst=/opt/clp', '--mount', 'type=bind,src=/Users/rerajesh/Downloads/clp-package-ubuntu-focal-x86_64-v0.0.2/var/log/bcd07464dac2,dst=/opt/clp/var/log', '--mount', 'type=bind,src=/Users/rerajesh/Downloads/clp-package-ubuntu-focal-x86_64-v0.0.2/var/data/archives,dst=/mnt/archive-output', 'ghcr.io/y-scope/clp/clp-execution-x86-ubuntu-focal:main', '/opt/clp/sbin/native/search', '--config', '/opt/clp/var/log/.clp-search-61c9-config.yml', 'Task * deprecation']' returned non-zero exit status 1.

[kirkrodrigues]

Hey @RajeshAccount,

Running on Ubuntu Focal isn't a requirement but the container image the package is using is built for x86_64 whereas it looks like you're using an ARM EC2 instance:

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

We'll add some ARM container images but before that, the fastest work around would be to use an x86 EC2 instance. In case that's not possible, we will also write some instructions on how to build and use a local container image.

[RajeshAccount]

Thanks for quick response, mean while I tried with x86 EC2 instance, now I'm getting below error while executing start-clp

ubuntu@ip----:~/clp/clp-package-ubuntu-focal-x86_64-v0.0.2$ sudo ./sbin/start-clp
2023-12-21 16:06:59,383 [INFO] [clp] Starting database...
Unable to find image 'mariadb:10.6.4-focal' locally
10.6.4-focal: Pulling from library/mariadb
7b1a6ab2e44d: Pull complete
034655750c88: Pull complete
f0b757a2a0f0: Pull complete
5c37daf8b6b5: Pull complete
b4cd9409b0f6: Pull complete
dbcda06785eb: Pull complete
a34cd90f184c: Pull complete
fd6cef4ce489: Pull complete
3cb89a1550ea: Pull complete
df9f153bd930: Pull complete
Digest: sha256:c014ba1efc5dbd711d0520c7762d57807f35549de3414eb31e942a420c8a2ed2
Status: Downloaded newer image for mariadb:10.6.4-focal
2023-12-21 16:07:16,692 [INFO] [clp] Started database.
2023-12-21 16:07:16,692 [INFO] [clp] Creating database tables...
Unable to find image 'ghcr.io/y-scope/clp/clp-execution-x86-ubuntu-focal:main' locally
main: Pulling from y-scope/clp/clp-execution-x86-ubuntu-focal
30ecab32a3b6: Pull complete
4f4fb700ef54: Pull complete
786ee64951df: Pull complete
7c312673afb9: Pull complete
bf3db92d4f61: Pull complete
d6b7527ac0a3: Pull complete
fff3b44df51e: Pull complete
5cfca84536c9: Pull complete
Digest: sha256:f419462db34cbc1daf0c626a8012bfb4892c85ffa9434a9f95f7c9b7d71eda27
Status: Downloaded newer image for ghcr.io/y-scope/clp/clp-execution-x86-ubuntu-focal:main
2023-12-21 16:07:27,304 [INFO] [clp] Created database tables.
2023-12-21 16:07:27,304 [INFO] [clp] Starting queue...
Unable to find image 'rabbitmq:3.9.8' locally
3.9.8: Pulling from library/rabbitmq
7b1a6ab2e44d: Already exists
37f453d83d8f: Pull complete
33be96f1328e: Pull complete
d3c5c3f5c49a: Pull complete
a3d2d8a82978: Pull complete
633da6037b0d: Pull complete
a619e412c37f: Pull complete
338c71e3d8cd: Pull complete
1daa7c79a268: Pull complete
Digest: sha256:19e69a7a65fa6b1d0a5c658bad8ec03d2c9900a98ebbc744c34d49179ff517bf
Status: Downloaded newer image for rabbitmq:3.9.8
2023-12-21 16:07:37,883 [INFO] [clp] Stopping clp-queue-589a...
2023-12-21 16:07:37,896 [INFO] [clp] Stopped clp-queue-589a.
2023-12-21 16:07:37,910 [INFO] [clp] Stopping clp-db-589a...
2023-12-21 16:07:38,426 [INFO] [clp] Stopped clp-db-589a.
2023-12-21 16:07:38,440 [ERROR] [clp] Failed to start CLP.
Traceback (most recent call last):
File "/home/ubuntu/clp/clp-package-ubuntu-focal-x86_64-v0.0.2/./sbin/start-clp", line 462, in main
start_queue(instance_id, clp_config)
File "/home/ubuntu/clp/clp-package-ubuntu-focal-x86_64-v0.0.2/./sbin/start-clp", line 271, in start_queue
subprocess.run(cmd, stdout=subprocess.DEVNULL, check=True)
File "/usr/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['docker', 'exec', '-it', 'clp-queue-589a', 'rabbitmqctl', 'wait', '/tmp/rabbitmq.pid']' returned non-zero exit status 137.

[RajeshAccount]

looks like its related to OOM, let me try with higher EC2 instance. Let me know if you have any other suggestions.

[kirkrodrigues]

Could you post the log output of <package>/var/log/<host>/rabbitmq*.log? This sounds similar to the issue we saw here.

[RajeshAccount]

no files present under <package>/var/log/<host>/ Can i only see two empty directory (<package>/var/log/<host>/db and <package>/var/log/<host>/queue).

[kirkrodrigues]

Are you running as a normal user or as root? Also what EC2 instance type are you using? I will try and replicate your setup to debug.

[RajeshAccount]

EC2 -> c5.9xlarge
normal user

[kirkrodrigues]

I'm able to run the v0.0.2 release successfully on a c5.9xlarge instance. If you want to compare, here are the commands I ran on the machine after logging in:

• Install Docker

  sudo snap install docker
  sudo addgroup --system docker
  sudo adduser $USER docker
  sudo snap disable docker
  sudo snap enable docker

• Log out and log back in

• Extract & start clp:

  curl -fsSLO https://github.com/y-scope/clp/releases/download/v0.0.2/clp-package-ubuntu-focal-x86_64-v0.0.2.tar.gz
  tar xzf clp-package-ubuntu-focal-x86_64-v0.0.2.tar.gz
  cd clp-package-ubuntu-focal-x86_64-v0.0.2
  sbin/start-clp

• Test compression and search on /var/log/dpkg.log

  sbin/compress /var/log/dpkg.log
  sbin/search purge

Here are some troubleshooting steps you could try:

• Run docker ps and then docker stop <container-name> on any containers with the prefix clp-.
• sudo rm -r <package>/var/log <package>/var/data
• Then try starting the package again

[RajeshAccount]

I'm able to successfully import hive data set, I installed docker-ce-24.0.7-1 latest version of docker earlier I had 24.0.5

[RajeshAccount]

kirkrodrigues Thanks for quick response and helping me setting up the clp.

I have queries on the supported search patterns, request to share any documentation on the search query pattern. Patterns I'm looking for are date range search (searching specific date range instead of entire time range), AND, OR search support.

[kirkrodrigues]

Hey @RajeshAccount, glad you got it working.

CLP currently supports wildcard queries where:

• * matches 0 or more characters, and
• ? matches any single character

We do support filtering by timestamp in the clp binary but we haven't exposed that yet in the package version of CLP. We'll look into adding that quickly so that you can test with it; we actually have a rewrite of the distributed compression and search code (that improves search performance and adds some of these missing features) coming in the 1st/2nd week of January.

If you just want to test single-threaded compression and search performance, you can use the clp binaries directly instead of the package:

• Start a container containing the clp binaries:

  docker run \
    -u$(id -u):$(id -g) \
    -v<dir-containing-logs>:/mnt/logs \
    -v<output-dir>:/mnt/output \
    --rm \
    -it \
    clp/clp-core-x86-ubuntu-focal /bin/bash

  • Change <dir-containing-logs> and <output-dir> to the relevant paths on your system.

• Then follow the usage instructions here.

  • You should set archives-dir to a path inside /mnt/output/
  • Your input logs should be a path inside /mnt/logs
  • clg supports time-range filtering using a syntax like ./clg --tge 1703242608000 --tlt 1703242608999 my-archives-dir which means search for logs which have a millisecond epoch timestamp in the range [1703242608000, 1703242608999). You can see complete usage instructions with ./clg --help.

CLP's search doesn't currently support boolean operators (AND, OR, etc.). This is on our roadmap, but we don't have an exact date for it yet. That said, we are open-sourcing a new storage engine for semi-structured logs (JSON, etc.) in the next few weeks and that supports a KQL query language (which itself supports boolean operators, range filtering, etc.). We'll ping you when that's ready to try.

[kirkrodrigues]

Hi @RajeshAccount,

We've added support for filtering by timestamps to the package. To try it, you can build the package from main by using the packager tool. Let us know if you have any trouble building the package.

Once built, the search command (<package>/sbin/search.sh) will show new options for filtering by timestamp (--begin-time and --end-time). Note that currently, timestamps must be specified as milliseconds since the UNIX epoch.