rebase by Abhinavpv28 · Pull Request #36 · rdkcentral/common_utilities

Abhinavpv28 · 2025-07-04T07:36:19Z

No description provided.

.github/workflows/L1-Test.yaml

+    name: Execute L1 test suite in test container environment
    runs-on: ubuntu-latest
-    container:
-      image: ghcr.io/rdkcentral/docker-rdk-ci:latest

    steps:
      - name: Checkout code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
+
+      - name: Log in to GitHub Container Registry
+        uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Pull test container image
+        run: docker pull ghcr.io/rdkcentral/docker-device-mgt-service-test/native-platform:latest
+
+      - name: Start test container
+        run: |
+          docker run -d --name native-platform -v ${{ github.workspace }}:/mnt/L1_CONTAINER_SHARED_VOLUME ghcr.io/rdkcentral/docker-device-mgt-service-test/native-platform:latest

-      - name: Run unit tests
-        run: sh unit_test.sh
+      - name: Run L1 Unit Tests inside container
+        run: docker exec -i native-platform /bin/bash -c "cd /mnt/L1_CONTAINER_SHARED_VOLUME/ && sh unit_test.sh"

-      - name: Upload test results to automatic test result management system
+      - name: Copy L1 test results to runner
+        run: |
+          docker cp native-platform:/tmp/Gtest_Report /tmp/Gtest_Report
+          ls -l /tmp/Gtest_Report
+
+  upload-test-results:


To fix the problem, explicitly add a permissions block to the workflow. This can be added at the root level (applies to all jobs), or individually to jobs. The simplest, most maintainable approach is to add it at the root, immediately after the name field and before the on: field, to set minimal permissions (usually contents: read) for the workflow. If a job requires additional permissions, you can specify them at the job level, overriding the root. In your workflow, none of the visible steps appear to require anything beyond reading contents (e.g., there are no pushes or PR creations). Thus, adding permissions: contents: read immediately after the name line is the best fix.

#68) * RDKEMW-12237: Uploadlib use same file to get S3 url output for all the component Reason for change: Pass output file name as a parameter to API Test Procedure: Make sure S3 url output store inside the file passed to the API Risks: Medium Priority: P1 --------- Signed-off-by: Sahu <satyasundar_sahu@comcast.com> Co-authored-by: Abhinavpv28 <162570454+Abhinavpv28@users.noreply.github.com> Co-authored-by: Gomathi Shankar <gomathishankar37@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR performs a major refactoring of the RDK common_utilities codebase to transform rdkfwupgrader into a library-driven architecture. The changes include extracting upload functionality into a separate module, introducing new logging macros for common utilities, improving error handling, and adding comprehensive unit tests.

Changes:

Extracted upload functionality from rdkfwupgrader into new uploadutils library with support for mTLS, CodeBig, and S3 uploads
Replaced SWLOG_* logging macros with COMMONUTILITIES_* macros throughout the codebase to distinguish common utility logs from firmware upgrade logs
Added robust error handling for system calls (chdir, lstat, stat, fclose) and CURL operations
Added comprehensive unit tests for the new upload utilities

Reviewed changes

Copilot reviewed 45 out of 45 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
utils/system_utils.c	Added error checking for chdir/lstat calls, fixed resource leaks, added construct_full_path helper
utils/rdkv_cdl_log_wrapper.h	Added COMMONUTILITIES_* logging macros, removed trailing newlines from SWLOG_* macros
utils/rdk_fwdl_utils.*	Increased MAX_DEVICE_PROP_BUFF_SIZE, added null checks, changed logging to COMMONUTILITIES_*
utils/common_device_api.*	Added new utility functions (GetUTCTime, GetCapabilities, GetTimezone, etc.), fixed GetPartnerId
uploadutils/*	New upload utility module with mTLS, CodeBig, and S3 support
unit-test/*	Added comprehensive unit tests for upload utilities
dwnlutils/urlHelper.*	Added urlEncodeString function, improved error handling, added retry logic for CURL 56/18/28

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-16T18:31:37Z

utils/system_utils.c

+    if (chdir(dir) != 0) {
+        COMMONUTILITIES_ERROR("Failed to change directory to %s \n", dir);
+        closedir(dp);
+        return 0;
+    }


Extra space before newline in error message. Should be "Failed to change directory to %s\n" instead of "Failed to change directory to %s \n".

Copilot · 2026-01-16T18:31:37Z

utils/system_utils.c

+            COMMONUTILITIES_ERROR("lstat failed for %s: %s\n", entry->d_name, strerror(errno));
+            closedir(dp);
+            return 0;
+        }       


Trailing whitespace after the closing brace.

Copilot · 2026-01-16T18:31:37Z

utils/system_utils.c

+                    strncpy(out, constructed_path, strlen(constructed_path) + 1); // Copy path to out
+                    COMMONUTILITIES_DEBUG(" findPFile : Constructed path out : %s", out);
+                    free(constructed_path);
+                    constructed_path =NULL;


Missing space before NULL. Should be constructed_path = NULL;.

Copilot · 2026-01-16T18:31:38Z

utils/system_utils.c

    struct dirent *entry;
-    char filePath[RDK_APP_PATH_LEN];
-
+    char filePath[RDK_APP_PATH_LEN+1] = {0};  


Trailing whitespace at the end of the line.

Copilot · 2026-01-16T18:31:38Z

utils/system_utils.c

+
+            if (emptyFolder(filePath) != RDK_API_SUCCESS) {


Line 704 contains only whitespace and should be removed.

Copilot · 2026-01-16T18:31:38Z

dwnlutils/urlHelper.c

    FILE *fp = userdata;
    if(fp != NULL && buffer != NULL) {
-        SWLOG_INFO("header_callback():=%s\n",buffer);
+        COMMONUTILITIES_INFO("header_callback():=%s",buffer); // No need to end with new line \n as buffer already has \r\n


Comment uses informal contraction. Should be "newline" instead of "new line".

Suggested change

COMMONUTILITIES_INFO("header_callback():=%s",buffer); // No need to end with new line \n as buffer already has \r\n

COMMONUTILITIES_INFO("header_callback():=%s",buffer); // No need to end with newline \n as buffer already has \r\n

Copilot · 2026-01-16T18:31:39Z

utils/rdk_fwdl_utils.c

+				if( pTmp)
+				{
+                                    ++pTmp;
+				}
+				else
+				{
+				    continue;
+				}


Inconsistent indentation mixing tabs and spaces. The code should use consistent indentation throughout.

Copilot · 2026-01-16T18:31:39Z

utils/rdk_fwdl_utils.c

    }
-    if (buff_size == 0 || buff_size > MAX_DEVICE_PROP_BUFF_SIZE) {
-        SWLOG_ERROR("%s : buff size not in the range. size should be < %d\n", __FUNCTION__, MAX_DEVICE_PROP_BUFF_SIZE);
+    if (buff_size == 0 || buff_size >= MAX_DEVICE_PROP_BUFF_SIZE) {


Changed comparison from > to >= which may break existing code that passes MAX_DEVICE_PROP_BUFF_SIZE as the buffer size. This is a subtle API behavior change that should be documented or reconsidered.

Copilot · 2026-01-16T18:31:39Z

utils/common_device_api.c

                        ;
                    }
-                    snprintf( pPartnerId, szBufSize, "%s", buf );
+                    snprintf( pPartnerId, szBufSize, "%s", pTmp );


Changed from copying buf to copying pTmp. This appears intentional to skip leading whitespace, but verify this is the correct behavior change as it modifies what gets copied to the output buffer.

Common Utils 1.5.0 release

RDKEMW-12849: Update to print logs to console Reason for change: RDKLogger sends the output directly to syslog to gain the performance. But the platform components like `rdkfwupdater`, `rfc`, `rdm`, `logupload` & `crashupload` are spun by scripts and console output are redirected to file manually. So, for these components, we have to configure logger to print to console instead syslog. Once these components become standalone and launched by systemd service independently, this change can be removed to gain performance. Test Procedure: Ensure logs from above mentioned components are printed. Risks: None. In the past also output was redirected; so whatever gained in last 2 weeks is not present anymore. Signed-off-by: Karunakaran A <karunakaran_amirthalingam@cable.comcast.com>

…ts To C Implementation (#73)

Copilot

Pull request overview

Copilot reviewed 45 out of 45 changed files in this pull request and generated 11 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-03T09:04:07Z

uploadutils/uploadUtil.c

+    ret_code = curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
+    if (ret_code != CURLE_OK) {
+        COMMONUTILITIES_ERROR("%s: CURLOPT_VERBOSE failed: %s\n",
+                __FUNCTION__, curl_easy_strerror(ret_code));
+        fclose(resp_fp);
+        if (headers) curl_slist_free_all(headers);
+        return (int)ret_code;
+    }


performHttpMetadataPost(): CURLOPT_VERBOSE is enabled unconditionally. This is very noisy in production and can leak sensitive request/response details into logs. Gate this behind a build-time/debug flag (e.g., CURL_DEBUG) similar to performS3PutUpload().

Copilot · 2026-02-03T09:04:07Z

unit-test/utils/common_device_api_gtest.cpp

    char data[32];
-    EXPECT_NE(GetPartnerId(data, 7),0);
-    printf("Pertner ID = %s\n", data);
+    EXPECT_EQ(GetPartnerId(data, 7), 0);  


This test now expects GetPartnerId(...) to return 0, but GetPartnerId returns the number of characters copied (stripinvalidchar result). In the no_source_file case it typically falls back to "comcast" and should return a non-zero length. Update the assertion to match the API contract (and optionally validate the resulting string).

Suggested change

EXPECT_EQ(GetPartnerId(data, 7), 0);

EXPECT_NE(GetPartnerId(data, 7), 0);

Copilot · 2026-02-03T09:04:08Z

utils/system_utils.c

+    if (chdir(dir) != 0) {
+        COMMONUTILITIES_ERROR("Failed to change directory to %s \n", dir);
+        closedir(dp);
+        return 0;
+    }


findFile() changes the process working directory with chdir(dir) but only attempts to restore it with chdir(".."). If dir is an absolute path (or a relative path not under the original cwd), this will not restore the original working directory and can break subsequent file operations (and is not thread-safe). Save/restore the original cwd (e.g., open(".") + fchdir) or avoid chdir entirely by building full paths / using openat-based traversal.

Copilot · 2026-02-03T09:04:08Z

utils/system_utils.c

+        if (lstat(entry->d_name, &statbuf) == -1) {
+            COMMONUTILITIES_ERROR("lstat failed for %s: %s\n", entry->d_name, strerror(errno));
+            closedir(dp);
+            return 0;
+        }       


On lstat() failure, findFile() closes the directory and returns 0 immediately. This makes a single unreadable entry (permissions, transient ENOENT, broken symlink, etc.) abort the entire search and can yield false negatives. Consider logging and continuing to the next entry instead of returning early.

Copilot · 2026-02-03T09:04:08Z

uploadutils/codebigUtils.c

+int doCodeBigSigning( int server_type, const char *SignInput, char *signurl, size_t signurlsize, char *outhheader, size_t outHeaderSize )
+{
+    return 0;
+}


doCodeBigSigning() (and related helpers) are currently stubbed to return success (0) without populating outputs. Since this file is compiled into libuploadutil when USE_CPC_CODE is disabled, it will override any real implementation and cause CodeBig signing/URL discovery to silently “succeed” with empty/invalid data. Either provide the real implementation here, or remove these stubs from the build and link against the intended implementation/library.

Copilot · 2026-02-03T09:04:09Z

utils/system_utils.c

+                char* constructed_path = construct_full_path(path, entry->d_name);
+                if (constructed_path) {
+                    strncpy(out, constructed_path, strlen(constructed_path) + 1); // Copy path to out
+                    COMMONUTILITIES_DEBUG(" findPFile : Constructed path out : %s", out);


findPFile(): copying the constructed path into out uses strncpy(out, constructed_path, strlen(constructed_path)+1) without knowing the capacity of out. If the constructed path is longer than the caller-allocated buffer, this will overflow out. Consider changing the API to accept out buffer size and use snprintf, or enforce a documented max length and cap the copy with explicit NUL termination.

Copilot · 2026-02-03T09:04:09Z

utils/system_utils.c

+    if (!strtokvalue || !string || !outValue) { 
+        COMMONUTILITIES_ERROR("Invalid Parameters %p %p %p", strtokvalue, string, outValue); 
+        return; 
+    }
    FILE *file = fopen(path,"r");
-
-    if( (strtokvalue != NULL) && (string != NULL) && ( outValue != NULL) ){
-        if( file ){
-            while(fgets(lines, sizeof(lines), file)){
-                token = strtok(lines, strtokvalue);
-
-                while(token != NULL){
-                    if(strcmp(token,string) == 0 ){
-                        strncpy(outValue, strtok(NULL,strtokvalue),128);
-                        break;
-                    }
-                    token = strtok(NULL,strtokvalue);
-                }
+    if( file ){


getStringValueFromFile(): parameter validation checks strtokvalue/string/outValue, but path is still used unconditionally in fopen(path, ...) and in the error log. If path is NULL this will crash. Please include path in the validation check.

Copilot · 2026-02-03T09:04:09Z

utils/system_utils.c

+        while(token != NULL){
+            if(strcmp(token,string) == 0 ){
+                strncpy(outValue, strtok(NULL,strtokvalue),128);
+                break;


getStringValueFromFile(): strtok(NULL, strtokvalue) can return NULL (e.g., key present but no value). Passing that NULL to strncpy will crash. Guard the return value of strtok before copying, and ensure outValue is always NUL-terminated (strncpy does not guarantee this when truncating).

Copilot · 2026-02-03T09:04:10Z

utils/common_device_api.c

+                    while( ((*pPtr=(char)fgetc( fp )) != EOF) && !feof( fp ) )
+                    {
+                        ++pPtr;
+                    }
+                    *pPtr = 0;


GetFileContents(): this loop casts fgetc() to char and compares the stored char to EOF. EOF is an int and can’t be represented safely in a char; this can mis-handle bytes like 0xFF and can fail to terminate correctly. Use an int to hold fgetc()’s return value and stop on EOF before storing into the buffer.

Copilot · 2026-02-03T09:04:10Z

uploadutils/codebig_upload.c

+    /* Step 1: Get CodeBig signed URL and authorization header */
+    if (doCodeBigSigningForUpload(server_type, filepath, codebig_url,
+                                  sizeof(codebig_url), auth_header, sizeof(auth_header)) != 0) {
+        COMMONUTILITIES_ERROR("%s: CodeBig signing failed\n", __FUNCTION__);
+        return -1;
+    }


performCodeBigMetadataPost(): CodeBig signing returns both a signed URL (codebig_url) and an authorization header (auth_header), but auth_header is never applied to the request. If the backend expects this header, the metadata POST will fail or be unauthenticated. Consider extending performHttpMetadataPost() to accept additional headers or setting CURLOPT_HTTPHEADER here with auth_header.

Common Utils 1.5.1 release

RDKEMW-12849: Update to print logs to console For coverity workflow fail we will create cmf ticket for check.

Copilot

Pull request overview

Copilot reviewed 46 out of 46 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

rdklogger issue fix

shibu-kv and others added 30 commits March 20, 2025 06:40

Merge branch 'release/1.1.0'

28b1c24

Merge branch 'release/1.2.0'

50c8b58

Update system_utils.c

4a05dc9

Update system_utils.c

d21d9e0

Update system_utils.c

1a72f21

Update system_utils.c

ad77473

Update system_utils.c

d9b881b

Update system_utils.c

fa7915a

Update system_utils.c

ed184df

Update urlHelper.c

7304fb1

Update system_utils.c

27884bc

Update system_utils.c

3dcfcad

Update urlHelper.c

4fa9e08

Update system_utils.c

d3a156f

Update system_utils.c

59acf28

Update system_utils.c

e4da7fa

Update system_utils.c

e441664

Update system_utils.c

734c4d1

Update rdk_fwdl_utils.c

8b13d48

Update urlHelper.c

ed6a6fb

Update urlHelper.c

8003362

Update urlHelper.c

4308d22

Update urlHelper.c

8c58c06

Update system_utils.c

baec8ea

Update system_utils.c

a51865b

Update system_utils.c

a2aed3f

Update system_utils.c

e538b48

Update system_utils.c

de3d994

Update system_utils.c

7813f70

Update system_utils.c

44d02c3

Abhinavpv28 added 5 commits December 16, 2025 13:50

Update configure.ac

caa815a

Update Makefile.am

e6ab839

Update Makefile.am

6271faa

Update configure.ac

5b4c463

Update configure.ac

969474b

github-advanced-security bot found potential problems Dec 16, 2025

View reviewed changes

Copilot AI review requested due to automatic review settings January 16, 2026 18:29

Copilot started reviewing on behalf of shibu-kv January 16, 2026 18:29 View session

Copilot AI reviewed Jan 16, 2026

View reviewed changes

naveenkumarhanasi and others added 5 commits January 20, 2026 20:27

Common Utils 1.5.0 release changelog updates

35467b6

Merge branch 'release/1.5.0'

95f179f

Merge tag '1.5.0' into develop

665cba3

Common Utils 1.5.0 release

RDK-57502 - [RDKE] Migrate Operation Support Log Upload Related Scrip…

9ddea87

…ts To C Implementation (#73)

Copilot AI review requested due to automatic review settings February 3, 2026 08:55

Copilot started reviewing on behalf of tdeva14 February 3, 2026 08:55 View session

Copilot AI reviewed Feb 3, 2026

View reviewed changes

mkadinti and others added 6 commits February 3, 2026 17:32

Merge branch 'develop' into topic/RDKEMW-12849

0244fc7

Common Utils 1.5.1 release changelog updates

c53186c

Merge branch 'release/1.5.1'

5a4e196

Merge tag '1.5.1' into develop

48a8eef

Common Utils 1.5.1 release

Merge branch 'develop' into topic/RDKEMW-12849

57c726f

Merge pull request #74 from rdkcentral/topic/RDKEMW-12849

f8769f0

RDKEMW-12849: Update to print logs to console For coverity workflow fail we will create cmf ticket for check.

Copilot AI review requested due to automatic review settings February 6, 2026 11:15

Copilot started reviewing on behalf of satya200 February 6, 2026 11:15 View session

Copilot AI reviewed Feb 6, 2026

View reviewed changes

satya200 added 3 commits February 6, 2026 12:43

1.5.2 release changelog updates

584486e

Merge branch 'release/1.5.2'

aea4d38

rdklogger issue fix

Merge tag '1.5.2' into develop

ae37771

rdklogger issue fix

@@ -1,4 +1,6 @@
             name: L1 Unit Tests
+            permissions:
+              contents: read
             on:
               pull_request:

	COMMONUTILITIES_INFO("header_callback():=%s",buffer); // No need to end with new line \n as buffer already has \r\n
	COMMONUTILITIES_INFO("header_callback():=%s",buffer); // No need to end with newline \n as buffer already has \r\n

	EXPECT_EQ(GetPartnerId(data, 7), 0);
	EXPECT_NE(GetPartnerId(data, 7), 0);

Conversation

Abhinavpv28 commented Jul 4, 2025

Uh oh!

Check warning

Copilot Autofix

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone