apache/beam

58 Pull requests merged by 34 people

• Merged #8617 [BEAM-7360] Fix static analysis issues for HCatalogIO May 20, 2019
• Merged #8616 [BEAM-7359] Fix static analysis issues for HadoopFormatIO May 20, 2019
• Merged #8600 [BEAM-7349] Get back to "provided" scope for "kafka_clients" May 20, 2019
• Merged #8563 [BEAM-7275] ParDoLifeCycleTest: collect lifecycle info for DoFn insta… May 19, 2019
• Merged #8575 [release-2.13.0] Apply StringUtf8 coder related fixes for Dataflow/Flink May 19, 2019
• Merged #8554 This change updates python SDK GCP client API in sync with the current Google internal definition. May 17, 2019
• Merged #8608 [BEAM-7339] Enable 1Gb input for Python wordcount benchmark May 17, 2019
• Merged #8602 [BEAM-7347] Pass dpb_service_zone flag to benchmarks to fix breakage May 17, 2019
• Merged #8601 Fix a typo in settings.gradle May 17, 2019
• Merged #8594 [BEAM-7302] ensure artifactId set on pom also with unconfigured project May 17, 2019
• Merged #8518 [BEAM-6908] Refactor Python performance test groovy file for easy configuration May 17, 2019
• Merged #8596 [BEAM-7339] Make input and checksum configurable for Python WordCountIT May 17, 2019
• Merged #8531 [BEAM-6959] Add Flink tests for Go SDK May 16, 2019
• Merged #8583 Add retry strategy when TableRow or TableCell is null from BQ response. May 16, 2019
• Merged #8564 [BEAM-7277] Add PostCommit test suite for Python 3.7 May 16, 2019
• Merged #8588 [BEAM-7283] Update aggregated javadoc versions during linking to point to javadoc.io May 16, 2019
• Merged #8595 [BEAM-7230] Add javadoc on provided poolable data sources on JdbcIO May 16, 2019
• Merged #8591 [website] Add design doc on Portable Beam Schemas May 16, 2019
• Merged #8513 [BEAM-7240] Kinesis IO Watermark Computation Improvements May 16, 2019
• Merged #8584 [BEAM-7287] Change JAVA_HOME to match new Jenkins agents May 16, 2019
• Merged #8589 Simple fix: make use of Reify in Latest to replace duplicate code May 16, 2019
• Merged #8587 [BEAM-7331] Create PAssert util function for late panes. May 15, 2019
• Merged #8560 [BEAM-7154] Updating Go SDK errors (Part 3 - Final) May 15, 2019
• Merged #8586 Categorize tests from UsesTestStream to UsesTestStreamWithProcessingTime May 15, 2019
• Merged #8280 [BEAM-6138] Update java SDK to report user distribution tuple metrics over the FN API May 15, 2019
• Merged #8566 [SQL] Upgrade to Avatica 1.15.0 May 15, 2019
• Merged #8342 [BEAM-7103] Adding AvroCoderTranslator for cross-language data transfer May 15, 2019
• Merged #8582 [BEAM-7311] merge internal commits to beam open source trunk to prepare for the security patch May 15, 2019
• Merged #8491 [BEAM-7220] add JavaReadViaImpulse substitution to Java expansion service May 15, 2019
• Merged #8377 [BEAM-7121] Add deterministic proto coder May 15, 2019
• Merged #8499 [BEAM-5865] Create optional auto-balancing sharding function for Flink May 15, 2019
• Merged #8425 [BEAM-7174] Add schema modification transforms May 15, 2019
• Merged #8536 [BEAM-7251] Increase timeout for test BQ queries. May 14, 2019
• Merged #8581 [BEAM-4046] revert test infra to be compatible with pending PRs May 14, 2019
• Merged #8577 [BEAM-7302] Fix project dependencies in pom.xml outputs May 14, 2019
• Merged #8358 Beam Kata for Java and Python SDKs May 14, 2019
• Merged #8439 [BEAM-6493] : Add cookbook and snippets examples in Kotlin May 14, 2019
• Merged #8574 [cleanup] Remove UnboundedSocketSource from Flink Runner May 14, 2019
• Merged #8573 [website] Update Flink Runner page for version 1.8 May 14, 2019
• Merged #8410 [BEAM-7305] Add first version of Hazelcast Jet based Java Runner May 14, 2019
• Merged #8539 [BEAM-7028] Change ParDo load test according to the proposal May 14, 2019
• Merged #8483 Fix trigger paths filters to execute precommit tests only if needed May 14, 2019
• Merged #8557 [BEAM-7271] adding UTF8 String coder to Java SDK ModelCoder May 14, 2019
• Merged #8341 Use a singleton PIPELINE_OPTION_PARAMETER in DoFnSignature.Parameter May 14, 2019
• Merged #8472 [BEAM-7126] Remove duplicate key encoding for state backend May 14, 2019
• Merged #8571 [release-2.13.0][BEAM-7239] Do not close DataSource on Teardown on JdbcIO May 14, 2019
• Merged #8565 [BEAM-7239] Do not close DataSource on Teardown on JdbcIO May 14, 2019
• Merged #8559 Beam community update blogpost May 14, 2019
• Merged #8569 [BEAM-7282] portable Spark: don't fuse already optimized graph May 14, 2019
• Merged #7994 [BEAM-562] Add DoFn.setup and DoFn.teardown to Python SDK May 14, 2019
• Merged #8550 [BEAM-7181][BEAM-7243] add streaming it tests python 3.6 May 14, 2019
• Merged #8537 [BEAM-7253] test_with_jar_packages_invalid_file_name test fails on Windows May 13, 2019
• Merged #8568 [BEAM-7281] Update Python deps page to current release May 13, 2019
• Merged #8194 [BEAM-4046] decouple gradle project names and maven artifact ids May 13, 2019
• Merged #8567 Small doc clean up May 13, 2019
• Merged #8555 [BEAM-7269] remove StateSpec from hashCode of SimpleStateTag May 13, 2019
• Merged #8546 [BEAM-7243] Updating python containers to beam-master-20190509 May 13, 2019
• Merged #8561 [BEAM-6627] Add item and byte counters to HadoopFormatIOIT May 13, 2019

25 Pull requests proposed by 22 people

• Proposed #8570 [BEAM-7286] RedisIO support for INCRBY/DECRBY operations May 14, 2019
• Proposed #8576 [BEAM-2888] Add the not-yet-fully-designed drain and checkpoint to runner comparison May 14, 2019
• Proposed #8579 [DO NOT MERGE] Run all PostCommit and PreCommit Tests against Release Branch 2.13.0 May 14, 2019
• Proposed #8580 [NOT FOR REVIEW] Add empty file in order to create PR May 14, 2019
• Proposed #8585 [BEAM-6829] Use transform name for metric namespace if none provided May 15, 2019
• Proposed #8590 [BEAM-6988] Implement a Python 3 version of getcallargs_forhints May 16, 2019
• Proposed #8592 [BEAM-7305] Improve and extend Hazelcast Jet based Java Runner May 16, 2019
• Proposed #8593 [BEAM-7364] Avoid possible signed integer overflow in hash. May 16, 2019
• Proposed #8597 [BEAM-7190] Enable file based token auth for samza portable runner May 17, 2019
• Proposed #8598 [BEAM-7322] Add threshold to PubSub unbounded source May 17, 2019
• Proposed #8599 [BEAM-7348] Support environment expiration May 17, 2019
• Proposed #8603 Add aliases for machine_type and disk_type flags to match existing documentation May 17, 2019
• Proposed #8605 [BEAM-7135] Spark: close job bundle factory May 17, 2019
• Proposed #8606 [BEAM-6959] add beam_PostCommit_Go_VR_Flink job May 17, 2019
• Proposed #8607 [BEAM-7342] Extend SyntheticPipeline map steps to be splittable. May 17, 2019
• Proposed #8609 Update .test-infra/jenkins/README May 17, 2019
• Proposed #8610 [BEAM-7154] Fixing small error bug. May 17, 2019
• Proposed #8611 [BEAM-7354] Starcgen fix when no identifiers specified. May 17, 2019
• Proposed #8612 [BEAM-7101] Remove usages of deprecated self.assertEquals in Python codebase May 18, 2019
• Proposed #8613 [BEAM-7356] Support generic field selections in Filter. May 18, 2019
• Proposed #8614 Fix the ContainsKv assertion that didn't work with some JDKs May 19, 2019
• Proposed #8615 fix dependency check May 20, 2019
• Proposed #8618 [BEAM-7341] Elements produced by bounded GBK are always ON_TIME and O… May 20, 2019
• Proposed #8620 [BEAM-6673] Add schema support to BigQuery reads May 20, 2019
• Proposed #8621 [BEAM-6769][BEAM-7327] add it test for writing and reading with bigqu… May 20, 2019

26 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• 19 new comments Open #8535 [BEAM-6693] ApproximateUnique transform for Python SDK May 15, 2019
• 17 new comments Open #8558 [BEAM-7131] Spark: cache executable stage output to prevent re-computation May 15, 2019
• 15 new comments Open #8174 [BEAM-6683] add createCrossLanguageValidatesRunner task May 16, 2019
• 11 new comments Open #8104 [BEAM-6872] Add hook for user-defined JVM initialization in workers May 15, 2019
• 10 new comments Open #8074 Added new example on how to create a custom unbounded streaming source May 16, 2019
• 10 new comments Open #8408 [BEAM-7141] Add window & timestamp in timer callback method argument May 20, 2019
• 9 new comments Open #8494 [SQL] Simplify limit collect in BeamEnumerableConverter May 17, 2019
• 8 new comments Open #8363 Adding Python samples to the Stateful Processing post. May 20, 2019
• 7 new comments Open #8390 [BEAM-7043] Add DynamoDBIO May 17, 2019
• 7 new comments Open #8530 [BEAM-6988] solved problem related to updates of the str object May 15, 2019
• 5 new comments Open #8455 [BEAM-6916] Reorg Beam SQL docs and add Calcite section May 15, 2019
• 4 new comments Open #8394 [BEAM-2857] Implementing WriteToFiles transform for fileio (Python) May 14, 2019
• 4 new comments Open #8457 [BEAM-3342] Initial version of Google Cloud Bigtable IO connector May 19, 2019
• 4 new comments Open #8552 [BEAM-7268] make sorter extension Hadoop-free May 19, 2019
• 3 new comments Open #8496 [BEAM-5967] Add handling of DynamicMessage in ProtoCoder May 15, 2019
• 2 new comments Open #7777 Handle empty source May 15, 2019
• 2 new comments Open #8206 [BEAM-6695] Latest PTransform for Python SDK May 19, 2019
• 2 new comments Open #8380 [BEAM-6880] Remove deprecated Reference Runner code. May 18, 2019
• 2 new comments Open #8453 [BEAM-6985] TypeHints Py3 Error: Native type compatibility tests fail on Python 3.7+ Updates May 16, 2019
• 1 new comment Open #7868 [BEAM-4775] MonitoringInfo URN tweaks May 13, 2019
• 1 new comment Open #8357 [BEAM-5822] Vendor bytebuddy May 17, 2019
• 1 new comment Open #8376 [BEAM-7130]Support Datetime value conversion in convertAvroFieldStrict May 20, 2019
• 1 new comment Open #8416 [BEAM-6138] Add the Sampled Byte Count counters to the Java SDK May 18, 2019
• 1 new comment Open #8505 [BEAM-5878] add tests for kwonly args in python 3 May 20, 2019
• 1 new comment Open #8534 Transform catalog for Python May 15, 2019
• 0 new comments Open #8486 [BEAM-6766] Add Sort Merge Bucket sink and join source May 19, 2019