You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: development/developer-guide.md
+45-49Lines changed: 45 additions & 49 deletions
Original file line number
Diff line number
Diff line change
@@ -1,42 +1,43 @@
1
1
# Developer guide for beginners on contributing to Fluent Bit
2
2
3
-
Assuming you have some basic knowledge of C, this guide should help you understand how to make code changes to Fluent Bit.
3
+
If you have some knowledge of C, this guide should help you understand how to make code changes to Fluent Bit.
4
4
5
5
## Libraries
6
6
7
-
Most external libraries are embedded in the project in the [/lib](https://github.com/fluent/fluent-bit/tree/master/lib) folder. To keep its footprint low and make cross-platform builds simple, Fluent Bit attempts keep its dependency graph small.
7
+
Most external libraries are embedded in the project in the [/lib](https://github.com/fluent/fluent-bit/tree/master/lib) folder. To keep its footprint low and maximize compatibility in cross-platform builds, Fluent Bit attempts to keep its dependency graph small.
8
8
9
-
The external library you are mostly likely to interact with is [msgpack](https://github.com/msgpack/msgpack-c).
9
+
The external library that you're mostly likely to interact with is [MessagePack](https://github.com/msgpack/msgpack-c).
10
10
11
-
For cryptographic support, Fluent Bit uses the system installed version of OpenSSL.
12
-
Please make sure to install openssl libraries and headers before building Fluent Bit.
11
+
For cryptographic support, Fluent Bit uses the system installed version of OpenSSL. You must install OpenSSL libraries and headers before Building Fluent Bit.
13
12
14
-
### Memory Management
13
+
### Memory management
15
14
16
-
When you write Fluent Bit code, you will use Fluent Bit's versions of the standard C functions for working with memory:
15
+
When you write Fluent Bit code, you'll use the Fluent Bit versions of the standard C functions for working with memory:
17
16
18
-
*[`flb_malloc()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h) - equivalent to `malloc`, allocates memory.
19
-
*[`flb_calloc()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h) - equivalent to `calloc`, allocates memory and initializes it to zero.
20
-
*[`flb_realloc()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h) - equivalent to `realloc`.
21
-
*[`flb_free()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h) - equivalent to `free`, releases allocated memory.
17
+
*[`flb_malloc()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h): Equivalent to `malloc`, allocates memory.
18
+
*[`flb_calloc()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h): Equivalent to `calloc`, allocates memory and initializes it to zero.
19
+
*[`flb_realloc()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h): Equivalent to `realloc`.
20
+
*[`flb_free()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_mem.h): Equivalent to `free`, releases allocated memory.
22
21
23
-
Note that many types have a specialized create and destroy function. For example, [`flb_sds_create()` and `flb_sds_destroy()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_sds.h)\(more about this in the next section\).
22
+
{% hint style="info" %}
23
+
Many types have specialized create and destroy functions, like [`flb_sds_create()` and `flb_sds_destroy()`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_sds.h).
24
+
{% endhint %}
24
25
25
26
### Strings
26
27
27
-
Fluent Bit has a stripped down version of the popular [SDS](https://github.com/antirez/sds) string library. See [flb\_sds.h](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_sds.h) for the API.
28
+
Fluent Bit has a stripped down version of the popular [SDS](https://github.com/antirez/sds) string library. See [`flb_sds.h`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_sds.h) for the API.
28
29
29
-
In general, you should use SDS strings in any string processing code. SDS strings are fully compatible with any C function that accepts a null-terminated sequence of characters; to understand how they work, see the [explanation on Github](https://github.com/antirez/sds#how-sds-strings-work).
30
+
In general, you should use SDS strings in any string processing code. SDS strings are fully compatible with any C function that accepts a null-terminated sequence of characters. To understand how they work, see the [explanation on Github](https://github.com/antirez/sds#how-sds-strings-work).
30
31
31
-
### HTTP Client
32
+
### HTTP client
32
33
33
34
Fluent Bit has its own network connection library. The key types and functions are defined in the following header files:
An `flb_upstream` structure represents a host/endpoint that you want to call. Normally, you'd store this structure somewhere so that it can be re-used. An `flb_upstream_conn` represents a connection to that host for a single HTTP request. The connection structure should not be used for more than one request.
117
+
The `flb_upstream` structure represents the host/endpoint that you want to call. Normally, you'd store this structure somewhere so that it can be reused. An `flb_upstream_conn` represents a connection to that host for a single HTTP request. This connection structure shouldn't be used for more than one request.
117
118
118
-
### Linked Lists
119
+
### Linked lists
119
120
120
-
Fluent Bit contains a library for constructing linked lists- [cfl\_list](https://github.com/fluent/fluent-bit/blob/master/lib/cfl/include/cfl/cfl_list.h). The type stores data as a circular linked list.
121
+
Fluent Bit contains a library for constructing linked lists: [`cfl_list`](https://github.com/fluent/fluent-bit/blob/master/lib/cfl/include/cfl/cfl_list.h). The type stores data as a circular linked list.
121
122
122
-
The [`cfl_list.h`](https://github.com/fluent/fluent-bit/blob/master/lib/cfl/include/cfl/cfl_list.h) header file contains several macros and functions for use with the lists. The example below shows how to create a list, iterate through it, and delete an element.
123
+
The [`cfl_list.h`](https://github.com/fluent/fluent-bit/blob/master/lib/cfl/include/cfl/cfl_list.h) header file contains several macros and functions for use with the lists. The following example shows how to create a list, iterate through it, and delete an element.
123
124
124
125
```c
125
126
#include <cfl/cfl.h>
@@ -174,11 +175,11 @@ static int example()
174
175
}
175
176
```
176
177
177
-
### Message Pack
178
+
### MessagePack
178
179
179
-
Fluent Bit uses [msgpack](https://msgpack.org/index.html) to internally store data. If you write code for Fluent Bit, it is almost certain that you will interact with msgpack.
180
+
Fluent Bit uses [MessagePack](https://msgpack.org/index.html) to internally store data. If you write code for Fluent Bit, it's almost certain that you will interact with MessagePack.
180
181
181
-
Fluent Bit embeds the [msgpack-c](https://github.com/msgpack/msgpack-c) library. The example below shows manipulating message pack to add a new key-value pair to a record. In Fluent Bit, the [filter\_record\_modifier](https://github.com/fluent/fluent-bit/tree/master/plugins/filter_record_modifier) plugin adds or deletes keys from records. See its code for more.
182
+
Fluent Bit embeds the [`msgpack-c`](https://github.com/msgpack/msgpack-c) library. The following example shows how to manipulate message pack to add a new key/value pair to a record. In Fluent Bit, the [`filter_record_modifier`](https://github.com/fluent/fluent-bit/tree/master/plugins/filter_record_modifier) plugin adds or deletes keys from records. See its code for more.
Please also check out the message pack examples on the [msgpack-c GitHub repo](https://github.com/msgpack/msgpack-c).
266
+
For more info, see the MessagePack examples in the [msgpack-c GitHub repository](https://github.com/msgpack/msgpack-c).
266
267
267
268
## Plugin API
268
269
269
-
Each plugin is a shared object which is [loaded into Fluent Bit](https://github.com/fluent/fluent-bit/blob/1.3/src/flb_plugin.c#L70) using dlopenand dlsym.
270
+
Each plugin is a shared object which is [loaded into Fluent Bit](https://github.com/fluent/fluent-bit/blob/1.3/src/flb_plugin.c#L70) using `dlopen`and `dlsym`.
270
271
271
272
### Input
272
273
273
-
The input plugin structure is defined in [flb\_input.h](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_input.h#L62). There are a number of functions which a plugin can implement, most only implement `cb_init`, `cb_collect`, and `cb_exit`.
274
+
The input plugin structure is defined in [`flb_input.h`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_input.h#L62). There are a number of functions which a plugin can implement, but most only implement `cb_init`, `cb_collect`, and `cb_exit`.
274
275
275
-
The [`"dummy"` input plugin](https://github.com/fluent/fluent-bit/tree/master/plugins/in_dummy) is very simple and is an excellent example to review to understand more.
276
+
The [`"dummy"` input plugin](https://github.com/fluent/fluent-bit/tree/master/plugins/in_dummy) is very basic and is an excellent example to review to understand more.
276
277
277
-
Note that input plugins can use threaded mode if the flag `FLB_INPUT_THREADED` is provided.
278
+
Input plugins can use threaded mode if the flag `FLB_INPUT_THREADED` is provided.
278
279
To enable threading in your plugin, add the `FLB_INPUT_THREADED` to the set of `flags` when registering:
The structure for filter plugins is defined in [flb\_filter.h](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_filter.h#L44). Each plugin must implement `cb_init`, `cb_filter`, and `cb_exit`.
299
+
The structure for filter plugins is defined in [`flb_filter.h`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_filter.h#L44). Each plugin must implement `cb_init`, `cb_filter`, and `cb_exit`.
299
300
300
-
The [filter\_record\_modifier](https://github.com/fluent/fluent-bit/tree/master/plugins/filter_record_modifier) is a good example of a filter plugin.
301
+
The [`filter_record_modifier`](https://github.com/fluent/fluent-bit/tree/master/plugins/filter_record_modifier) is a good example of a filter plugin.
301
302
302
-
Note that filter plugins can not asynchronously make HTTP requests. If your plugin needs to make a request, add the following code when you initialize your `flb_upstream`:
303
+
Filter plugins can not asynchronously make HTTP requests. If your plugin needs to make a request, add the following code when you initialize your `flb_upstream`:
Output plugins are defined in [flb\_output.h](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_output.h#L57). Each plugin must implement `cb_init`, `cb_flush`, and `cb_exit`.
312
+
Output plugins are defined in [`flb_output.h`](https://github.com/fluent/fluent-bit/blob/master/include/fluent-bit/flb_output.h#L57). Each plugin must implement `cb_init`, `cb_flush`, and `cb_exit`.
312
313
313
-
The [stdout plugin](https://github.com/fluent/fluent-bit/tree/master/plugins/out_stdout) is very simple; review its code to understand how output plugins work.
314
+
The [stdout plugin](https://github.com/fluent/fluent-bit/tree/master/plugins/out_stdout) is very basic; review its code to understand how output plugins work.
314
315
315
-
## Development Environment
316
+
## Development environment
316
317
317
-
Fluent Bit provides a standalone environment for development.
318
-
Developers who use different OS or distributions can develop on a simple, common stack.
319
-
The development environment provides the required libraries and tools for you.
318
+
Fluent Bit provides a standalone environment for development. Developers who use different operating systems or distributions can develop on a basic, common stack. The development environment provides the required libraries and tools for you.
Internal tests are for the internal libraries of Fluent Bit. Runtime tests are for the plugins.
345
346
346
-
You can run the unit tests with `make test`, however, this is inconvenient in practice. Each test file will create an executable in the `build/bin` directory which you can run directly. For example, if you want to run the SDS tests, you can invoke them as follows:
347
+
You can run the unit tests with `make test`. However, this is inconvenient in practice. Each test file will create an executable in the `build/bin` directory, which you can run directly. For example, if you want to run the SDS tests, you can invoke them as follows:
347
348
348
349
```text
349
350
$ ./bin/flb-it-sds
350
351
Test sds_usage... [ OK ]
351
352
Test sds_printf... [ OK ]
352
353
SUCCESS: All unit tests have passed.
353
354
```
354
-
355
-
## Need more help?
356
-
357
-
The best way to learn how Fluent Bit code works is to read it. If you need help understanding the code, reach out to the community, or open a PR with changes that are a work in progress.
0 commit comments