in_tail: Add a description and note for Unicode.Encoding parameter

Signed-off-by: Hiroshi Hatake <[email protected]>

Author: cosmo0920

pipeline/inputs/tail.md

@@ -37,9 +37,23 @@ The plugin supports the following configuration parameters:
 | `Static_Batch_Size` | Set the maximum number of bytes to process per iteration for the monitored static files (files that already exist upon Fluent Bit start). | `50M` |
 | `File_Cache_Advise` | Set the `posix_fadvise` in `POSIX_FADV_DONTNEED` mode. This reduces the usage of the kernel file cache. This option is ignored if not running on Linux. | `On` |
 | `Threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
+| `Unicode.Encoding` | Set the encoding which the origin of character encoding. Currently, UTF-16LE, UTF-16BE, and auto is supported. | _none_ |
+
+{% hint style="info" %} If the database parameter `DB` isn't
+specified, by default the plugin reads each target file from the
+beginning. This might cause unwanted behavior. For example, when a
+line is bigger than `Buffer_Chunk_Size` and `Skip_Long_Lines` isn't
+turned on, the file will be read from the beginning of each
+`Refresh_Interval` until the file is rotated.  {% endhint %}
 
 {% hint style="info" %}
-If the database parameter `DB` isn't specified, by default the plugin reads each target file from the beginning. This might cause unwanted behavior. For example, when a line is bigger than `Buffer_Chunk_Size` and `Skip_Long_Lines` isn't turned on, the file will be read from the beginning of each `Refresh_Interval` until the file is rotated.
+Note that `Unicode.Encoding` depends on simdutf library which is written in C++11 or above.
+So, the older platforms are not supported for this feature.
+In addition, `Unicode.Encoding auto` is not covered for the all of the usages.
+This is because sometimes this auto-detecting for character encodings makes a mistake to guess the correct encoding.
+We recommend to use `UTF-16LE` or `UTF-16BE` if the target file encoding is pre-determined or known beforehand.
+In details, this parameter requests to use 2-bytes aligned chunk and buffer sizes.
+If they are not aligned for 2 bytes, Fluent Bit will use 2-bytes alignments automatically to avoid character breakages on consuming boundaries.
 {% endhint %}
 
 ## Monitor a large number of files