Exploring the Apache Beam capability matrix

Beam makes sure that runners adhere to the Beam model to ensure that pipelines are portable between different runners. There are tests specifically designed to validate the compatibility between runners – these tests are called ValidatesRunner suites and are annotated using the @ValidatesRunner annotation. Any runner author can verify the compatibility of their runner against these tests. However, because runners are developed at different rates, it is possible that a specific runner at a certain moment in time does not support all of the features of the Beam model. Therefore, the complete set of ValidatesRunner tests can be narrowed down by toggling specific features of the model. The tests in the complete suite are then annotated using features such as UsesSetState or UsesSchema. A runner is then allowed to specify features that should be excluded from the tests. These excluded features should then match what is documented in the...