kafka: consolidate the mixed usage the kafka and kafka-c (#217)

Related: #212
Fixes: #216

Author: HofiOne

_data/external_links.yml

@@ -748,6 +748,10 @@ gh-websense:
   url: https://github.com/syslog-ng/syslog-ng/blob/master/scl/websense/plugin.conf
   title: [ "Websense configuration snippet on GitHub" ]
 
+gh-syslog-ng-cfg-helper:
+  id: gh-syslog-ng-cfg-helper
+  url: https://github.com/syslog-ng/syslog-ng-cfg-helper
+  title: [ "syslog-ng-cfg-helper utility" ]
 
 ### Destinations, sources, parsers, templates ###
 

doc/_admin-guide/050_The_configuration_file/006_Modules_in_syslog-ng/001_Listing_configuration_options.md

@@ -2,18 +2,18 @@
 title: Listing configuration options
 id: adm-mod-list
 description: >-
-    Starting with {{ site.product.short_name }} 3.25, you can use the syslog-ng-cfg-db.py
+    Starting with {{ site.product.short_name }} 3.25, you can use the syslog-ng-cfg-helper
     utility to list the available options of configuration objects. For
     example, you can list all the options that can be set in the file
     source, and so on.
 ---
 
-The syslog-ng-cfg-db.py utility has the following options:
+The syslog-ng-cfg-helper utility has the following options:
 
 - The following command lists the contexts that the utility supports.
 
     ```bash
-    syslog-ng-cfg-db.py
+    syslog-ng-cfg-helper
     ```
 
     **NOTE:** Currently, sources and destinations are supported.
@@ -22,60 +22,88 @@ The syslog-ng-cfg-db.py utility has the following options:
 - The following command lists the available drivers of a context:
 
     ```bash
-    syslog-ng-cfg-db.py -c <source|destination>
+    syslog-ng-cfg-helper -c <source|destination>
     ```
 
 - The following command lists the available options of a specific
     driver and specifies the context and the driver:
 
     ```bash
-    syslog-ng-cfg-db.py -c <source|destination> -d <driver>
+    syslog-ng-cfg-helper -c <source|destination> -d <driver>
     ```
 
-    For example, to list the options of the kafka-c() destination
+    For example, to list the options of the kafka() destination
     driver:
 
     ```bash
-    syslog-ng-cfg-db.py -c destination -d kafka-c
+    syslog-ng-cfg-helper -c destination -d kafka
     ```
 
     The output includes the available options of the driver in
     alphabetical order, and the type of the option. For example:
 
     ```config
-    destination kafka-c(
-        bootstrap-servers/kafka-bootstrap-servers(<string>)
-        client-lib-dir(<string>)
-        config/option()
-        config/option(<string> <arrow> <string-or-number>)
-        config/option(<string> <string-or-number>)
-        flush-timeout-on-reload(<number>)
-        flush-timeout-on-shutdown(<number>)
-        frac-digits(<number>)
-        key(<string>)
-        local-time-zone/time-zone(<string>)
-        log-fifo-size(<number>)
-        message/template(<string>)
+    kafka-c(
+        batch-lines(<nonnegative-integer>)
+        batch-timeout(<positive-integer>)
+        bootstrap-servers(<string>)
+        config(
+            <empty>
+            <string> <string-or-number>
+            <string> => <string-or-number>
+            <string>(<string-or-number>)
+        )
+        disk-buffer(
+            capacity-bytes(<number>)
+            compaction(<yesno>)
+            dir(<string>)
+            disk-buf-size(<number>)
+            flow-control-window-bytes(<nonnegative-integer>)
+            flow-control-window-size(<nonnegative-integer>)
+            front-cache-size(<nonnegative-integer>)
+            mem-buf-length(<nonnegative-integer>)
+            mem-buf-size(<nonnegative-integer>)
+            prealloc(<yesno>)
+            qout-size(<nonnegative-integer>)
+            reliable(<yesno>)
+            truncate-size-ratio(<nonnegative-float>)
+        )
+        fallback-topic(<string>)
+        flags(
+            <empty>
+            <string>
+        )
+        flush-timeout-on-reload(<nonnegative-integer>)
+        flush-timeout-on-shutdown(<nonnegative-integer>)
+        frac-digits(<nonnegative-integer>)
+        hook-commands(
+            setup(<string>)
+            shutdown(<string>)
+            startup(<string>)
+            teardown(<string>)
+        )
+        internal(<yesno>)
+        key(<template-content>)
+        local-time-zone(<string>)
+        log-fifo-size(<positive-integer>)
+        message(
+            <template-content>
+            <template-reference>
+        )
         on-error(<string>)
         persist-name(<string>)
-        poll-timeout(<number>)
-        properties-file(<path>)
+        poll-timeout(<nonnegative-integer>)
+        retries(<positive-integer>)
         send-time-zone(<string>)
         sync-send(<yesno>)
-        throttle(<number>)
+        template-escape(<yesno>)
+        throttle(<nonnegative-integer>)
+        time-reopen(<positive-integer>)
         time-zone(<string>)
-        topic(<string>)
+        topic(<template-content>)
         ts-format(<string>)
-        workers(<number>)
-        config/option(
-            <string>(<string-or-number>)
-        )
-        key(
-            <identifier>(<string>)
-        )
-        message/template(
-            <identifier>(<string>)
-        )
+        worker-partition-key(<template-content>)
+        workers(<positive-integer>)
     )
     ```
 

doc/_admin-guide/070_Destinations/100_Kafka-c/000_Before_you_begin.md

@@ -43,7 +43,6 @@ The following example defines a kafka destination in the new C
 implementation, using only the required parameters.
 
 ```config
-@define kafka-implementation kafka-c 
 @include "scl.conf"
 
 destination d_kafka {

doc/_admin-guide/070_Destinations/100_Kafka-c/001_Shifting_from_Java_to_C.md

@@ -7,6 +7,8 @@ description: >-
     configuration file and considerations are necessary.
 ---
 
+- `kafka()` and `kafka-c()` is now interchangable as the first one is just an alias of the latter
+
 - Unlike the old one, the new topic() option can not handle templates.
     It must be a string.
 
@@ -15,7 +17,7 @@ description: >-
 - The kafka-bootstrap-servers() option has been renamed
     bootstrap-servers().
 
-- The properties-file() was replaced with the config() option of kafka-c, which is similiar but not identical to the properties-file() option.
+- The properties-file() was replaced with the config() option which is similiar but not identical to the properties-file() option.
 
 - The sync-send() option has been deprecated. Remove it from the
     configuration file.

doc/_admin-guide/070_Destinations/100_Kafka-c/003_Kafka-c_options.md

@@ -1,16 +1,19 @@
 ---
-title: "Options of the kafka-c() destination"
+title: "Options of the kafka() destination"
 id: adm-dest-kafkac-opt
 description: >-
-    This section describes the options of the kafka-c() destination in {{ site.product.short_name }}.
+    This section describes the options of the kafka() destination in {{ site.product.short_name }}.
 ---
 
-The kafka-c() destination of {{ site.product.short_name }} can
+The kafka() destination of {{ site.product.short_name }} can
 directly publish log messages to the Apache Kafka message bus, where subscribers can access them. The destination has the following options.
 
+**NOTE:** `kafka()` and `kafka-c()` is now interchangable as the first one is just an alias of the latter
+{: .notice--info}
+
 ## Required options
 
-To use the kafka-c() destination, the following two options are required: `bootstrap-servers()` and `topic()`. Both must appear at the beginning of your {{ site.product.short_name }} configuration.
+To use the kafka() destination, the following two options are required: `bootstrap-servers()` and `topic()`. Both must appear at the beginning of your {{ site.product.short_name }} configuration.
 
 You can specify multiple, comma-separated addresses, demonstrated in the following example:
 

doc/_admin-guide/070_Destinations/100_Kafka-c/README.md

@@ -1,5 +1,5 @@
 ---
-title: 'kafka-c(): Publishing messages to Apache Kafka using the
+title: 'kafka(): Publishing messages to Apache Kafka using the
   librdkafka client (C implementation)'
 short_title: kafka
 id: adm-dest-kafkac
@@ -13,7 +13,10 @@ As of {{ site.product.short_name }} version 3.21, the new C implementation of th
 destination is available. The new implementation uses the librdkafka
 client and has several advantages, such as scalability, more efficient
 memory usage and simpler setup. The options of this implementation are
-compatible with those of the old Java implementation.
+not exactly compatible with those of the old Java implementation, see
+Shifting from Java implementation to C implementation for the differences.
+Version 4.9.0 has dropped the Java implementation. kafka() and kafka-c() can now
+be used interchangeably; both refer to the single remaining C implementation.
 
 ## Figure 11: How the C implementation of the kafka destination works with {{ site.product.short_name }}