execution.batch.adaptive.auto-parallelism.avg-data-volume-per-task |
16 mb |
MemorySize |
The average size of data volume to expect each task instance to process if jobmanager.scheduler has been set to AdaptiveBatch . Note that when data skew occurs or the decided parallelism reaches the execution.batch.adaptive.auto-parallelism.max-parallelism (due to too much data), the data actually processed by some tasks may far exceed this value. |
execution.batch.adaptive.auto-parallelism.default-source-parallelism |
(none) |
Integer |
The default parallelism of source vertices or the upper bound of source parallelism to set adaptively if jobmanager.scheduler has been set to AdaptiveBatch . Note that execution.batch.adaptive.auto-parallelism.max-parallelism will be used if this configuration is not configured. If execution.batch.adaptive.auto-parallelism.max-parallelism is not set either, then the default parallelism set via parallelism.default will be used instead. |
execution.batch.adaptive.auto-parallelism.enabled |
true |
Boolean |
If true, Flink will automatically decide the parallelism of operators in batch jobs. |
execution.batch.adaptive.auto-parallelism.max-parallelism |
128 |
Integer |
The upper bound of allowed parallelism to set adaptively if jobmanager.scheduler has been set to AdaptiveBatch |
execution.batch.adaptive.auto-parallelism.min-parallelism |
1 |
Integer |
The lower bound of allowed parallelism to set adaptively if jobmanager.scheduler has been set to AdaptiveBatch |
execution.batch.speculative.block-slow-node-duration |
1 min |
Duration |
Controls how long an detected slow node should be blocked for. |
execution.batch.speculative.enabled |
false |
Boolean |
Controls whether to enable speculative execution. |
execution.batch.speculative.max-concurrent-executions |
2 |
Integer |
Controls the maximum number of execution attempts of each operator that can execute concurrently, including the original one and speculative ones. |