[FLINK-33392][docs] Add the documentation pages for balanced tasks scheduling.

[RocMarshal]

What is the purpose of the change

[FLINK-33392][docs] Add the documentation pages for balanced tasks scheduling.

Brief change log

[FLINK-33392][docs] Add the documentation pages for balanced tasks scheduling.

Verifying this change

N.A

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): (yes / no)
• The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
• The serializers: (yes / no / don't know)
• The runtime per-record code paths (performance sensitive): (yes / no / don't know)
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know)
• The S3 file system connector: (yes / no / don't know)

Documentation

• Does this pull request introduce a new feature? (yes / no)
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[flinkbot]

CI report:

• a84ef5d Azure: FAILURE

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[RocMarshal]

The CH edition page will added a copy in the corresponding location after the EN edition page is ready.

[davidradl]

I think the situation this help in is when the parallelism of a vertex is less in the subsequent vertex in the job. Not when parallelism is inconsistent (which could mean that we have 2 kafka sources with different number of partitions - I guess this change would not effect them). Could this be a badly configured job, if not it would be useful to detail when this can occur.

Are there other topologies that this effects?

[RocMarshal]

which could mean that we have 2 kafka sources with different number of partitions - I guess this change would not effect them

This algorithm only focuses on balancing the number of tasks, and specific cases require individual discussion.

Could this be a badly configured job, if not it would be useful to detail when this can occur.

The business scenarios faced by Flink jobs are complex, so whether the configuration is reasonable depends on whether the job's performance meets the throughput requirements for processing business data.
Therefore, we cannot simply discuss whether the parallelism is properly configured.

Are there other topologies that this effects?

No, its are not.

CC @davidradl

[davidradl]

That makes sense: I would expand a bit around inconsistent, mentioning parallelism. The worry is that it could be read that inconsistent parallelism requires the use of this strategy, which is not always the case.

[RocMarshal]

@davidradl
Precise!
What about add a note item content like follows?

<span class="label label-info">Note</span> The presence of inconsistent parallelism does not imply that this strategy must be used, as this is not always the case in practice.

[davidradl]

nit: I would remove the words "theoretically" "effectively" - I think it would make the statement stronger.

[RocMarshal]

Thanks @davidradl

I removed the 'effectively' and still kept 'theoretically' for rigor.
After all, each job has unique business characteristics and load patterns, making it difficult to guarantee that no scenario exists where a flink job theoretically fits but actually causes performance degradation when enabled. What do you think?

[davidradl]

I would remove the theoretically, and be explicit about the considerations. Maybe:

If you are seeing performance bottle necks of the sort described above, then using this load balancing tasks scheduling strategy can improve performance.
Be aware that you should not use this strategy, if you are not seeing these bottle necks, as you may experience performance degradation.

[RocMarshal]

@davidradl
Thanks

Updated as:

Therefore, if you are seeing performance bottlenecks of the sort described above,
then using this load balancing tasks scheduling strategy can improve performance.
Be aware that you should not use this strategy, if you are not seeing these bottlenecks,
as you may experience performance degradation.