Release v0.8.0

Apache Uniffle (Incubating) Release v0.8.0

Highlight

• Support TEZ
• Introduce Netty for shuffle data transmission
• Use off heap memory to store shuffle data.
• Introduce REST API for cluster management.
• Introduce command line for cluster management.

ChangeLog

• Change license owner to ASF by @kaijchen in #5
• Trivial code improvements by @wForget in #7
• [Minor] Store shuffleId int to be consistent with other data structure by @zuston in #10
• Introduce the asList method in ConfigOptions by @zuston in #9
• Rename package by @jerqi in #6
• Minimize apache-rat excluded files by @kaijchen in #11
• Update module names by @kaijchen in #12
• Covert PartitionAssignmentInfo to static inner class by @pan3793 in #15
• [Followup] Migrate to Junit5 by @zuston in #14
• [Bug] Fix NPE problem when process the event if application was cleared already by @colinmjj in #16
• [CI] Enable codecov report by @kaijchen in #17
• Correct the config description and fix typo by @zuston in #19
• Add CI and Codecov badges in README by @kaijchen in #20
• [Followup] Use asList method in some existing configOptions by @zuston in #18
• Move rss-integration-spark-common-test module package by @wForget in #23
• [INFRA] Improve asf.yaml to reduce the notifications by @jerryshao in #25
• [TEST] Improve code coverage in rss-common by @kaijchen in #26
• Remove redundant package by @wForget in #27
• [CI] Switch to temurin JDK by @kaijchen in #24
• [INFRA] Improve asf.yaml to reduce the notifications (another-try) by @jerryshao in #33
• Bump commons-lang3 from 3.5 to 3.10 by @wForget in #28
• Fix the log of incorrectly bound class by @wForget in #35
• [TYPO] Fix misspelled word "integration" by @kaijchen in #34
• Fix some hyperlink in README.md by @daugraph in #32
• Upgrade gRPC to support Apple Silicon by @pan3793 in #13
• Allow to specify custom tags to decide the assignment of servers by @zuston in #30
• Optimize the bash script by @zuston in #29
• [Improvement] reduce compiler warnings by @advancedxy in #46
• [Chore]: document update and build time optimize by @advancedxy in #45
• Supplement doc about assignment tags by @zuston in #47
• [Bug] Fix skip() api maybe skip unexpected bytes which makes inconsistent data by @colinmjj in #40
• [improvement] Remove experimental feature with ShuffleUploader by @colinmjj in #51
• [Improvement] Provides utility classes for creating thread factories by @smallzhongfeng in #49
• Enable spotbugs and fix high priority bugs by @kaijchen in #38
• [CI] Change default checkstyle severity to error by @kaijchen in #57
• [Style] Check indentation by @kaijchen in #56
• [Experimental Feature] MR Supports Remote Spill by @frankliee in #55
• [Improvement] Log indicate the shuffle server host:port when doing re… by @zuston in #58
• Send commit concurrently in client side by @zuston in #59
• Explicitly set the constructor with AccessManager when extending AccessChecker by @zuston in #43
• [DOC] Replace Firestorm with Uniffle by @jerqi in #60
• Introduce the extraProperties to support user-defined pluggable accessCheckers by @zuston in #42
• Log enhancement: Merge multiple logs into oneline and add more description by @zuston in #62
• [TEST] Add more unit tests in rss-common by @kaijchen in #63
• [MINOR] Comments of PartitionBalanceAssignmentStrategy miss byte units by @smallzhongfeng in #68
• [Minor] Make config keys and default values finalized by @kaijchen in #70
• [Log Improvment] Add more detailed debug info for MR client by @frankliee in #84
• [Improvement] Shutdown the grpc executors pool when closing by @zuston in #83
• Log enhancement: return error message when getting assignment servers and log exception when initializing by @zuston in #64
• [ISSUE-48] [Feature] Init Kubernetes operator directory by @jerqi in #75
• [Improvement] No need to use synchronized lock of the method scope when getting client by @zuston in #82
• [DOC] Remove Wechat group in README by @jerqi in #88
• [Performance Optimization] Improve the speed of writing index file in shuffle server by @zuston in #91
• [DOC] Update title and description in README by @kaijchen in #94
• [Improvement] ShuffleBlock should be release when finished reading by @xianjingfeng in #74
• [IMPROVEMENT][COMMON] Fix common module code style by @jerqi in #99
• [Improvement]LocalStorage init use multi thread #71 by @xianjingfeng in #72
• [Improvement] Use OR operation instead of serialization for cloning BitMaps by @kaijchen in #103
• [Improvement] Ignore partial failure on initializing local storage in shuffle server side by @zuston in #102
• [CI] Test compile in Java 11 and Java 17 by @kaijchen in #105
• Sleep less time but try more times when stopping by @xianjingfeng in #112
• [Improvement] Use ConfigBuilder to rewrite the class RssSparkConfig by @smallzhongfeng in #104
• [Improvement] Introduce config to customize assignment server numbers in client side by @zuston in #100
• Assign partition again if registerShuffleServers failed by @xianjingfeng in #115
• [ISSUE-106][IMPROVEMENT] Set rpc timeout for all rpc interface by @xianjingfeng in #113
• [MINOR][IMPROVEMENT] Avoid CoordinatorServer#initialization multiple new Configuration() by @zwangsheng in #118
• [Improve] Remove useless server id from StorageManagerFactory#createStorageManager by @zwangsheng in #119
• [MINOR][IMPROVEMENT][COORD] Fix coordinator module code style by @jerqi in #122
• [Improvement] Set heartBeatExecutorService as daemon thread by @smallzhongfeng in #121
• [JUnit] Introduce the property of trimStackTrace to show error stacktrace in mvn-test by @zuston in #126
• Make the conf of rss.storage.basePath as list by @zuston in #130
• [MINOR][IMPROVEMENT][STORAGE] Fix storage module code style by @jerqi in #131
• [Improvement] Add timeout reconnection when DelegationRssShuffleManager send the request of AccessCluster by @smallzhongfeng in #139
• [MINOR] Fix flaky test testGetHostIp by @izchen in #141
• [Improvement] Add the number of unhealthy nodes in CoordinatorMetrics by @smallzhongfeng in #147
• [ISSUE-48][FEATURE] Add Uniffle Dockerfile by @wangao1236 in #132
• [BUGFIX] Fix memory leak which cause oom by @summaryzb in #145
• [Log Improvement] Output the registering/lost/exclude nodes in log by @zuston in #148
• [MINOR] Tagged spark hadoop version in release package by @izchen in #149
• [DOC] Migrate the coordinator doc from README to docs page by @zuston in #153
• [MINOR][DOC] Remove spaces when reading file of excluded nodes by @smallzhongfeng in #155
• [Improvement] Filter null value when selecting remote storage in ApplicationManager by @smallzhongfeng in #156
• Introduce more grpc server metrics by @zuston in #150
• [Improvement] Introduce a new class ShuffleTaskInfo by @smallzhongfeng in #158
• [ISSUE-76] Disallow sendShuffleData if requireBufferId expired by @xianjingfeng in #159
• Support storing shuffle data to secured dfs cluster by @zuston in #53
• [FOLLOWUP] Delete hdfs shuffle data files using proxy user by @zuston in #170
• [ISSUE-48][FEATURE] Init Operator Directory by @wangao1236 in #161
• PID file name should contains program name by @zuston in #165
• [BUGFIX] Fix resource leak when shuffle read by @izchen in #174
• [Improvement] ShuffleBufferManager supports triggering flush according to the size of single ShuffleBuffer by @leixm in #176
• [Improvement] Should match from pathToStorages when appId does not exist in appIdToStorages by @smallzhongfeng in #168
• [ISSUE-173][FOLLOWUP] The size of single buffer flush should reach rss.server.flush.cold.storage.threshold.size by @leixm in #178
• Revert "[ISSUE-173][FOLLOWUP] The size of single buffer flush should reach rss.server.flush.cold.storage.threshold.size " by @jerqi in #179
• [ISSUE-173][FOLLOWUP] The size of single buffer flush should reach rss.server.flush.cold.storage.threshold.size by @leixm in #180
• [FOLLOWUP] Store app user in ShuffleTaskInfo by @smallzhongfeng in #181
• [ISSUE-123] Fix all test code style by @macroguo-ghy in #185
• [ISSUE-48][FEATURE][FOLLOW UP] Add RemoteShuffleService CRD by @wangao1236 in #175
• [FOLLOWUP] Add the conf of rss.security.hadoop.krb5-conf.file by @zuston in #184
• Fix flaky test about kerberos by @zuston in #191
• [Improvement] Add optional environment variables by @izchen in #187
• [MINOR] Fix some bad practices reported by spotbugs by @kaijchen in #177
• [ISSUE-48][FEATURE][FOLLOW UP] Add webhook component by @wangao1236 in #188
• [Log-Improvement] Log the newly registered app id by @zuston in #193
• [MINOR] Replace HashSet with ImmutableSet in configs by @kaijchen in #195
• [IMPROVEMENT] Introduce the enumType in ConfigOptions by @zuston in #199
• [ISSUE-48][FEATURE][FOLLOW UP] Generate informer and lister for crd by @wangao1236 in #202
• [ISSUE-144] Fix flaky test RssShuffleUtilsTest#testDestroyDirectByteBuffer by @LuciferYang in #203
• [Issue-194][Feature] Support spark 3.2.0 by @leixm in #201
• [ISSUE-186][Feature] Use I/O cost time to select storage paths by @smallzhongfeng in #192
• [Improvement][AQE] Avoid calling getShuffleResult multiple times by @leixm in #190
• Fix flaky test of heartbeatTimeoutTest by @zuston in #206
• [IMPROVEMENT] Add more metrics about local storage info by @zuston in #205
• [MINOR][IMPROVEMENT] Return index-file size of n*SEGMENT_SIZE in HDFS reading by @zuston in #204
• Add DISCLAIMER by @jerqi in #212
• [TEST] Improve SimpleClusterManagerTest by @kaijchen in #216
• [Minor] Modify the format of DISCLAIMER by @jerqi in #217
• Add Notice and DISCLAMER file by @frankliee in #215
• Add more badges in README by @kaijchen in #219
• Fix incorrect log format strings by @kaijchen in #220
• Change total lines badge url to sloc.xyz in README by @kaijchen in #222
• [MINOR] Fix warnings reported by lgtm by @kaijchen in #223
• [MINOR] Simplify creating buffer logic by @zuston in #227
• Support cancelling previous ci actions by @zuston in #225
• Use the conf of shuffleNodesNumber from jobs to be as checking factor by @zuston in #208
• Output the stderr and stdout to output file in startup script by @zuston in #226
• [ISSUE-48][FEATURE][FOLLOW UP] Add controller component by @wangao1236 in #214
• Add more metrics about requiring read memory by @zuston in #231
• Adjust the memory required times to match grpc max deadline conf by @zuston in #218
• [MINOR] Fix flaky test by @jerqi in #238
• [ISSUE-48][FEATURE][FOLLOW UP] Add yaml of components and crd exampes by @wangao1236 in #236
• Fix Flaky test GetShuffleReportForMultiPartTest by @leixm in #241
• Set the default disk capacity to the total space by @zuston in #237
• Add issue template by @jerqi in #8
• [MINOR] Fix inefficient map iteration by @kaijchen in #245
• Support deploy multiple shuffle servers in a single node by @xianjingfeng in #166
• Fast fail when reading failed in ComposedClientReadHandler by @zuston in #213
• Fix startup shell problem by @jerqi in #251
• New version 0.7.0-snapshot by @jerqi in #252
• [ISSUE-196] Fix flaky test about kerberos by @zuston in #250
• [ISSUE-48][FEATURE][FOLLOW UP] add unit test for validating rss objects by @wangao1236 in #248
• [Improvement] Add hdfs path health check to AppBalanceSelectStorageStrategy by @smallzhongfeng in #210
• [TYPO] Replace Chinese colon by ASCII colon by @kaijchen in #255
• Introduce startup-silent-period mechanism to avoid partial assignments by @zuston in #247
• Replace DISCLAIMER with DISCLAIMER-WIP by @jerqi in #258
• [ISSUE-244] Fix flaky test of CoordinatorGrpcTest.rpcMetricsTest by @zuston in #256
• Fix flaky test of ClientConfManagerTest by @smallzhongfeng in #260
• [Refactor] Optimize creating shuffle handlers by @zuston in #259
• Introduce data cleanup mechanism on stage level by @zuston in #249
• [ISSUE-48][FEATURE][FOLLOW UP] add docs for operator by @wangao1236 in #261
• Fix potenial missing reads of exclude nodes by @zuston in #269
• [ISSUE-257] RssMRUtils#getBlockId change the partitionId of int type to long by @fpkgithub in #266
• [ISSUE-273][BUG] Get shuffle result failed caused by concurrent calls to registerShuffle by @leixm in #274
• Add enum type test about case insensitive by @zuston in #280
• Support ZSTD by @zuston in #254
• [ISSUE-239][BUG] RssUtils#transIndexDataToSegments should consider the length of the data file by @leixm in #275
• Remove code quality badge and add release badge by @kaijchen in #284
• [ISSUE-163][FEATURE] Write to hdfs when local disk can't be write by @xianjingfeng in #235
• Upgrade Github actions for Node.js 16 by @kaijchen in #292
• Fix NPE in WriteBufferManager.addRecord by @wForget in #296
• Fix AbstractStorage#containsWriteHandler by @xianjingfeng in #281
• Add more test cases on LocalStorageManager.selectStorage by @zuston in #298
• [ISSUE-137][Improvement][AQE] Sort MapId before the data are flushed by @zuston in #293
• [ISSUE-283][FEATURE] Support snappy compression/decompression by @amaliujia in #304
• [ISSUE-290] Make RpcNodePort and HttpNodePort optional by @amaliujia in #305
• [ISSUE-301][Subtask][Improvement][AQE] Merge continuous ShuffleDataSegment into single one by @zuston in #303
• Cleanup RuntimeException and fetchRemoteStorage logic in ClientUtils by @kaijchen in #295
• [ISSUE-135][FOLLOWUP][Improvement][AQE] Assign adjacent partitions to the same ShuffleServer by @leixm in #307
• Correct the contributing guide link in pull-request template by @zuston in #314
• Fix bug of "Comparison method violates its general contract" by @zuston in #315
• [AQE][LocalOrder] Fix potenial bug when merging continuous segments by @zuston in #318
• [AQE][LocalOrder] Fix wrong param of expectedTaskIds in LocalOrderSegmentSplit by @zuston in #319
• [Feature] Support the estimated number of ShuffleServers required. by @leixm in #322
• [Bug] Fix potenial bug when the index reading offset is greater than data length by @zuston in #320
• [ISSUE-154][Improvement] Support Empty assignment to Shuffle Server by @rhh777 in #325
• [Bug] Fix invalid owner of host path volumes by @wangao1236 in #330
• [ISSUE-309][FEATURE] Support ShuffleServer latency metrics. by @leixm in #327
• [ISSUE-329]Catch NPE in ShuffleTaskManager#addFinishedBlockIds by @xianjingfeng in #331
• [BUG] Fix wrong method name by @leixm in #335
• [ISSUE-328] Cleanup unused shuffle servers after stage completed by @xianjingfeng in #334
• [MINOR] Migrate RankValue to the package of the common class by @smallzhongfeng in #265
• [BUG] Fix incorrect spark metrics by @zuston in #324
• [Improvement][LocalOrder] Add tests about keeping consistent with FixedSize when no skew optimization by @zuston in #336
• [INFRA] Add k8s pipeline by @jerqi in #340
• Remove unused class of RssShuffleUtils by @zuston in #345
• [ISSUE-342][Improvement] Check Spark Serializer type by @chong0929 in #344
• [Feature] Support user's app quota level limit by @smallzhongfeng in #311
• [BUG][AQE][LocalOrder] Fix the bug of missed data due to block sorting by @zuston in #347
• [ISSUE-364] Fix indexWriter don't close if exception thrown when close dataWriter by @xianjingfeng in #349
• [BUG] Fix flaky test of AQESkewedJoinWithLocalOrderTest by @zuston in #350
• Add collaborators by @jerqi in #351
• [BUG][FOLLOWUP] Fix flaky test of AQESkewedJoinWithLocalOrderTest by @zuston in #355
• [BUG][AQE][LocalOrder] Remove check of discontinuous map task ids by @zuston in #354
• [Improvement] Task fast fail once blocks fail to send by @zuston in #332
• [ISSUE-228][FEATURE] Add a period local storage cleaner thread by @sfwang218 in #357
• [Improvement] Optimize the use of QuotaManager by @smallzhongfeng in #359
• [ISSUE-339] Optimize retry logic in send shuffle data by @xianjingfeng in #361
• [ISSUE-300] Make config type of RSS_CLIENT_TYPE as enum by @selectbook in #310
• [ISSUE-228] Fix the problem of protobuf-java incorrect dependency at compile time by @tsface in #362
• [ISSUE-124] Add fallback mechanism for blocks read inconsistent by @xianjingfeng in #276
• [BUG] Fix potential memory leak when encountering disk unhealthy by @zuston in #370
• [Improvement][AQE] Support getting memory data skip by upstream task ids by @zuston in #358
• [ISSUE-369] Don't throw exception if blocks are corrupted but have multi replicas by @xianjingfeng in #374
• [ISSUE-285][Improvement] Only use HDFS and LOCALFILE storageType in the test by @tiantingting5435 in #360
• Fix typo of PreferDiffHostAssignmentStrategy by @zuston in #379
• [ISSUE-376] Fix concurrency problems may occur when the ApplicationManager register app by @smallzhongfeng in #382
• [ISSUE-380] Refactor the flush process to fix fallback fail by @zuston in #383
• [Refactor] Make coordinator class more organized by @smallzhongfeng in #386
• [ISSUE-392] Fix the bug in the shuffle data cleanup checker that causes false reports of disk corruption by @zuston in #393
• [ISSUE-390] Print more infos after read finished by @xianjingfeng in #395
• [Improvement] Skip blocks when read from memory by @xianjingfeng in #294
• [Improvement] Small refactor for code quality by @advancedxy in #394
• [BUG] Fix incorrect block info statistics after read finished by @xianjingfeng in #401
• Revert "[Improvement] Skip blocks when read from memory (#294)" by @xianjingfeng in #403
• [ISSUE-388][ISSUE-244][Bug] Fix incorrect usage of GRPCMetrics#setGauge by @xianjingfeng in #404
• [MINOR] If there is no data flush to hdfs, return directly instead of throw exception by @xianjingfeng in #406
• Support writing multi files of single partition to improve speed in HDFS storage by @zuston in #396
• Fix incorrect metrics of event_queue_size and total_write_handler by @zuston in #411
• [Improvement] Support skip memory data when use multiple replicas by @xianjingfeng in #400
• [ISSUE-402] Flaky Test: QuorumTest#case1 by @Rembrant777 in #422
• Fix flaky test QuotaManagerTest#testDetectUserResource by @xianjingfeng in #421
• Improve README by @kaijchen in #427
• Remove unused data structure and method by @zuston in #429
• [FOLLOWUP] Remove unused methods in Storage interface by @zuston in #431
• Flaky Test: AppBalanceSelectStorageStrategyTest#selectStorageTest by @smallzhongfeng in #438
• [Minor] Move GrpcServerTest to common.rpc package by @kaijchen in #439
• [Minor] refactor test code by @advancedxy in #432
• [Minor][Improvement] Introduce FileWriter interface for Localfile/HDFS file writer by @zuston in #444
• [ISSUE-169] Support metric reporter and Support promethues push gateway by @xianjingfeng in #415
• [Improvement] Avoid selecting storage which has reached the high watermark by @zuston in #424
• [Bug] Fix potential negative preAllocatedSize variable by @advancedxy in #428
• [Feature] add a configuration to control shuffle data flush by @advancedxy in #445
• [Improvement] Refactor getPartitionRange to calculate range directly by @a-li in #447
• Fix Flaky Test: AppBalanceSelectStorageStrategyTest#selectStorageTest by @smallzhongfeng in #450
• Fix Flaky Test: QuotaManagerTest#testDetectUserResource by @smallzhongfeng in #453
• [ISSUE-451][Improvement] Read HDFS data files with random sequence to distribute pressure by @zuston in #452
• [ISSUE-455] Lazily create uncompressedData by @xianjingfeng in #457
• [ISSUE-378][HugePartition][Part-1] Record every partition data size for one app by @zuston in #458
• [ISSUE-456] Avoid removeResources for multiple times by @xianjingfeng in #459
• [ISSUE-461] Support Spark 3.3 by @kaijchen in #463
• [Deps] Bump slf4j to 1.7.36 to fix vulnerability in slf4j-log4j12 by @kaijchen in #464
• [ISSUE-448][Feature] shuffle server report storage info by @advancedxy in #449
• [ISSUE-472] Fix Flaky Test: LocalFileServerReadHandlerTest#testDataInconsistent by @zuston in #473
• [Improvement] Remove some unused empty server metrics by @zuston in #474
• [Improvement] Add more logs about data flush by @zuston in #482
• Fix potential race condition when registering remote storage info by @zuston in #481
• [Minor] Improve readability by replacing lambda with method reference by @iwangjie in #488
• [ISSUE-489][Minor] Cleanup some code by @iwangjie in #490
• [Test] Assume unknown blockID in LocalFileHandlerTestBase by @kaijchen in #478
• [ISSUE-475][Improvement] It's unnecessary to use ConcurrentHashMap for "partitionToBlockIds" in RssShuffleWriter by @jiafuzha in #480
• [ISSUE-378][HugePartition][Part-2] Introduce memory usage limit and data flush by @zuston in #471
• [ISSUE-484] Fix accidentally remove the storage of appId when unregistering partial shuffle in HdfsStorageManager by @zuston in #485
• [Test] Cleanup tests with Files#createTempDir() by @kaijchen in #492
• [Test] Add ConfigUtilsTest by @kaijchen in #500
• [Deps] Bump protobuf to 3.19.6 to address vulnerability by @kaijchen in #499
• [Minor] Make Constants final by @kaijchen in #501
• Cleanup UnitConverter and improve UnitConverterTest by @kaijchen in #504
• Fixes errors in doc header and operator install command by @zuston in #506
• [ISSUE-378][HugePartition][Part-3] Introduce more metrics about huge partition by @zuston in #494
• [ISSUE-378][HugePartition][Part-4] Supplement doc about huge partitions by @zuston in #505
• [Deps] Switch to slf4j-reload4j by @kaijchen in #508
• [ISSUE-512][Operator] Bump golang to 1.17 by @kaijchen in #515
• [ISSUE-514] Fix flaky test: ShuffleServerGrpcTest#clearResourceTest by @xianjingfeng in #516
• [ISSUE-507] Fix Flaky Test: ShuffleBufferManagerTest#cacheShuffleDataTest by @xianjingfeng in #511
• [ISSUE-509] Fix Flaky Test: ShuffleBufferManagerTest#shuffleFlushThreshold by @xianjingfeng in #510
• [ISSUE-479] [operator] refine operator's build system by @advancedxy in #491
• [ISSUE_479][operator][followup] use exec form instead of shell form in Dockerfile by @advancedxy in #518
• [SpotBugs] Set threshold to middle with exceptions by @kaijchen in #517
• [ISSUE-522] [operator] pass RSS_IP to coordinator container env by @advancedxy in #523
• [ISSUE-496][operator] infer resource request/limit from spec for init container by @advancedxy in #521
• [SpotBugs] Remove unread protected field in Checker by @kaijchen in #520
• [ISSUE-468] Put unavailable servers to the end of the list when sending shuffle data by @xianjingfeng in #470
• chore: add new collaborator by @advancedxy in #535
• [SpotBugs] Fix REC_CATCH_EXCEPTION by @kaijchen in #527
• [ISSUE-524][operator] Upgrading rss could also be deleted by @advancedxy in #531
• [ISSUE-553] Avoid removing buffer multiple times when clearing resources by @xianjingfeng in #534
• [ISSUE-469][operator] feat: supports adding labels to rss pods. by @wangao1236 in #528
• [SpotBugs] Fix UWF_UNWRITTEN_PUBLIC_OR_PROTECTED_FIELD by @kaijchen in #536
• Result of mkdirs() is ignored in LocalFileWriteHandler#createBasePath() by @kaijchen in #537
• [Minor] Optimize ShuffleServerInfo#hashCode by @xianjingfeng in #538
• [SpotBugs] Enable SC_START_IN_CTOR check by @kaijchen in #541
• [operator] fix error kind of ownerreference by @wangao1236 in #540
• [ISSUE-542] Ensure the elements of StatusCode and RssProtos.StatusCode are the same by @xianjingfeng in #543
• Flaky Test: LowestIOSampleCostSelectStorageStrategyTest#selectStorageTest by @smallzhongfeng in #544
• [Improvement] Only report to the shuffle servers that owns the blocks by @zuston in #539
• [Minor] Cleanup "throws RuntimeException" by @kaijchen in #549
• [ISSUE-546] Replace ResponseStatusCode with StatusCode by @xianjingfeng in #547
• [ISSUE-476][FEATURE] Respect spark.shuffle.compress configuration in Uniffle by @jiafuzha in #495
• [ISSUE-545][operator] feat: support setting runtime class name and env for rss by @wangao1236 in #548
• [ISSUE #525][operator] refine svc creations by @advancedxy in #530
• [#552] docs: add more doc about spark.serializer requirement by @advancedxy in #556
• [#559] test: use withEnvironmentVariable to replace RssUtilsTest#setEnv by @advancedxy in #560
• [Followup #249] refactor: cleanup code and unify interfaces by @kaijchen in #558
• [#554] feat: infer rss base storage conf from env by @advancedxy in #555
• [MINOR] Remove commiters from collaborators by @jerqi in #563
• [#525][FOLLOWUP] fix: add omitempty tag by @advancedxy in #565
• [#545][FOLLOWUP] update rbac rule for webhook by @advancedxy in #566
• [#571] feat: skip init for empty writable base dir by @advancedxy in #573
• [#567] feat: add a shuffle-server metric about read_used_buffer_size by @zuston in #568
• [MINOR] test: fix assertion in tests by @kaijchen in #574
• [#575] refactor: replace switch-case with EnumMap in ComposedClientReadHandler by @kaijchen in #570
• [#580] chore: improve CI workflows by @kaijchen in #579
• [#410] feat: support the hot reload of coordinator's configuration by @jerqi in #572
• [MINOR] chore(deps): bump go-restful to 2.16.0 in operator by @dependabot in #577
• [#580] chore: move deploy/kubernetes to a standalone workflow by @kaijchen in #578
• [MINOR] refactor: simplify ShuffleWriteClientImpl#genServerToBlocks() by @kaijchen in #594
• [MINOR] chore: remove duplicated dependency in rss-client-mr by @kaijchen in #599
• [#580] chore: speed up CI workflows by @kaijchen in #602
• [MINOR] chore(operator): bump prometheus/client_golang to 1.11.1 by @dependabot in #601
• [MINOR] docs: correct the format of server_guide doc by @zuston in #608
• [#611] chore: update project version to 0.8.0-SNAPSHOT by @zuston in #609
• [#613] test: SimpleClusterManagerTest#updateExcludeNodesTest by @xianjingfeng in #614
• [MINOR] fix: allow mountPoint not containing '/' by @xianjingfeng in #607
• [#551] docs: update templates for flaky test and pull request by @kaijchen in #588
• [#571][FOLLOWUP] fix: optimize base dir init process by @advancedxy in #616
• [#618] chore(docker): support downloads latest hadoop archives and mirror url by @advancedxy in #619
• [#580] chore(ci): disable parallel build in maven by @kaijchen in #631
• [#626] chore(ci): skip build operator if no code changes by @advancedxy in #628
• [#627] fix(operator): support specifying custom ports by @wangao1236 in #629
• [#626][FOLLOWUP] chore(ci): fix typo in build.yml by @advancedxy in #633
• [#600] chore(operator): change JDK base from openjdk to eclipse-temurin by @advancedxy in #617
• [#632]fix: respect volumes in rss spec by @advancedxy in #634
• [#630] feat(client): Disable the localShuffleReader by default in Spark3 client by @zuston in #636
• [#80][Part-1] feat: Add decommisson logic to shuffle server by @xianjingfeng in #606
• [MINOR] refactor: address unchecked conversions by @kaijchen in #623
• [#545][FOLLOWUP]feat(operator): support specifying custom affinity & tolerations by @crain-cn in #641
• [#647] fix: Multiple coordinator produce conflicts when they delect the same file by @jerqi in #648
• [#635] feat(client): enable LOCAL_ORDER by default for Spark AQE by @zuston in #644
• [#576] test: increase timeout and remove initialization time in storage strategy tests by @smallzhongfeng in #646
• [#649] test: remove mini-cluster in ClientConfManagerTest by @smallzhongfeng in #650
• [#647][FOLLOWUP] set coordinator id before ApplicationMaster by @advancedxy in #654
• [MINOR] docs: update the project introduction in README by @kaijchen in #653
• [#655]feat(coordinator): heap size configurable and add gc log by @advancedxy in #656
• [#659] fix(server): fix NPE of ShuffleDataFlushEvent's underStorage in some cases by @zuston in #660
• [#612] test: cleanup shuffleServer instance for each test by @zuston in #658
• [#408] test: fix memory check failure in ShuffleBufferManagerTest#bufferSizeTest by @zuston in #657
• [MINOR] docs: fix flaky-test-report.yml by @xianjingfeng in #664
• [#665] fix(client): keep consistent with vanilla spark when key or value is null by @zuston in #666
• [#642]feat(server): better default options for shuffle server by @advancedxy in #662
• [MINOR] test: address unchecked conversions in tests by @kaijchen in #624
• [#378] feat: introduce storage manager selector by @zuston in #621
• [#80][Part-2] feat: Add RPC logic and heartbeat logic for decommisson by @xianjingfeng in #663
• [#675] fix: filter no space exception in checkStorageReadAndWrite by @xumanbu in #677
• [#671] feat(coordinator): Metrics of the number of apps submitted by users by @smallzhongfeng in #672
• [#645][Improvement] feat(operator): support manager parameter configuration by @crain-cn in #670
• [#678] improvement: Write hdfs files asynchronously when detectStorage by @smallzhongfeng in #680
• [MINOR] Use multithreading to detect multiple disks by @smallzhongfeng in #687
• [#397] docs: add the usage of AccessQuotaChecker by @smallzhongfeng in #692
• [#585] feat(netty): Add MaxDirectMemorySize option for shuffle Server by @jerqi in #690
• [#697] fix: use the naive equals method to avoid introducing additional dependencies by @zuston in #698
• [#691] test :fix flaky test CoordinatorMetricsTest#testCoordinatorMetrics by @smallzhongfeng in #694
• [#585] fix: avoid unbound variable errors in start-shuffle-server.sh by @zuston in #696
• build(deps): bump golang.org/x/net to 0.7.0 in kubernetes operator by @dependabot in #676
• chore: remove unused log info by @zuston in #700
• [#674] feat(docker): use JDK11 as the default java version in Dockerfile by @zuston in #683
• [MINOR] improvement: Reduce the size of Spark patch by @jerqi in #699
• [MINOR] fix: Add method close for ApplicationManager by @jerqi in #704
• [#80][Part-3] feat: add REST API for decommisson by @xianjingfeng in #684
• [MINOR] test: fix static field initialized before TempDir field injected by @advancedxy in #707
• [#708] test: do not assume hostname of hdfs mini-cluster by @kaijchen in #709
• [#483] test: fix flaky test ShuffleServerFaultToleranceTest by @jerqi in #705
• [MINOR] Removed unused methods and variable by @smallzhongfeng in #702
• [#615] improvement: Reduce task binary by removing 'partitionToServers' from RssShuffleHandle by @jiafuzha in #637
• [#564] test(operator): add end-to-end test by @wangao1236 in #581
• [#711] feat(netty): Add Netty port information for Shuffle Server by @jerqi in #712
• [MINOR] test: fix tempdir leak in KerberizedHdfs tests by @kaijchen in #721
• [MINOR] test: test class name should end with Test by @kaijchen in #724
• [#133] feat(netty): Add StreamServer. by @leixm in #718
• [#625] improvement: Package sun.security.krb5 is not visible in Java 11 and 17. by @slfan1989 in #726
• [MINOR] chore(ci): test compile in java 11 and 17 with source/target version set by @kaijchen in #728
• [MINOR] improvement(client): update log level from INFO to DEBUG to avoid noisy by @zuston in #725
• Revert "[MINOR] test: fix tempdir leak in KerberizedHdfs tests" by @jerqi in #732
• [#733] test: fix LocalStorageManagerTest#testGetLocalStorageInfo on Linux SSD platform by @kaijchen in #734
• [#729] improvement: use foreach when iterate Roaring64NavigableMap for better performance by @zuston in #730
• [MINOR] test: fix CoordinatorGrpcTest#shuffleServerHeartbeatTest on Linux SSD platform by @kaijchen in #738
• [#133] feat(netty): Add Netty Utils by @leixm in #727
• [#669] improvement: refresh application when reading memory data by @xianjingfeng in #741
• [#719] feat(netty): Optimize allocation strategy by @smallzhongfeng in #739
• [#736] feat(storage): best effort to write same hdfs file when no race condition by @zuston in #744
• [#747] feat(tez): Add Tez Framework by @jerqi in #748
• [#750] feat: add RssFetchFailedException by @advancedxy in #751
• [MINOR] test: untracked files created in ShuffleFlushManagerTest by @kaijchen in #745
• [#752] refactor: replace RuntimeException with RssException by @kaijchen in #753
• [MINOR] chore: Remove committers from collaborators by @jerqi in #754
• [MINOR] test: do not wrap test in try-catch block by @kaijchen in #746
• [#762] If the storage path is not exist, get file store for its parent. by @xianjingfeng in #763
• [#133] feat(netty): Add Encoder and Decoder. by @leixm in #742
• [MINOR] chore(operator): remove useless configuration items in configuration example by @xianjingfeng in #764
• [#519] Speed up ConcurrentHashMap#computeIfAbsent by @turboFei in #766
• [#716] improvement(operator): support specifying imagePullSecrets by @xianjingfeng in #765
• [#755] refactor: Add the method of creating thread pool by @smallzhongfeng in #767
• [#706] Implement spill method to avoid memory deadlock by @zuston in #714
• [#772] fix(kerberos): cache proxy user ugi to avoid memory leak by @zuston in #773
• [MINOR] chore(spark): remove noisy log in client side by @zuston in #776
• [#477][part-0] feat: add ShuffleManagerServer impl by @advancedxy in #777
• [MINOR] Avoid returning null in defaultUserApps when quota file does't config user by @smallzhongfeng in #786
• [#584] feat(netty): Add transport client pool for netty by @xumanbu in #771
• [#720] feat(netty): support random port for netty by @xumanbu in #723
• [#782]refactor: restrict rss.rpc.server.type to an enum by @advancedxy in #783
• [#720][FOLLOW-UP] Correct the shuffle server id by @jerqi in #792
• [#794] feat(operator): support delete ShuffleServer pod with Evicted status by @xianjingfeng in #795
• [MINOR] fix: the description of Spark patches in the README.md by @jerqi in #801
• [#799] feat: use storage host label for remote storage write metrics by @advancedxy in #800
• [#796] bug:fix the issues of MetricReporter by @xianjingfeng in #797
• [BUG][MINOR] Fix scripts compatible with jdk8 & jdk11 by @summaryzb in #798
• [MINOR] test: Fix the flaky test GrpcServerTest by @java-codehunger in #789
• [#477][part-1] feat: support stage resubmit in spark clients by @advancedxy in #787
• [#804] improvement: Optimize CRC calculation of ByteBuffer by @jerqi in #805
• [MINOR] chore: update import order rule to make scala a separated group by @advancedxy in #818
• [#779] feat: Grpc server support random port by @xumanbu in #820
• [#596] feat(netty): Use off heap memory to read HDFS data by @jerqi in #806
• [#416] feat(hdfs): lazy initialization of hdfsShuffleWriteHandler when per-partition concurrent write is enabled by @zuston in https://github.com/apache/incubator-uniffle/pull/816
• [MINOR] Add WORKDIR by @smallzhongfeng in https://github.com/apache/incubator-uniffle/pull/823
• [#778] feat: Separate ShuffleServer metrics through tags by @smallzhongfeng in https://github.com/apache/incubator-uniffle/pull/812
• [MINOR] Modify MD format by @smallzhongfeng in https://github.com/apache/incubator-uniffle/pull/829
• [#827] feat(operator): support generating hpa objects by @wangao1236 in https://github.com/apache/incubator-uniffle/pull/828
• [MINOR] Remove unused config SHUFFLE_EXPIRED_TIMEOUT_MS by @smallzhongfeng in https://github.com/apache/incubator-uniffle/pull/835
• [#133] feat(netty): Rewrite protocol. by @leixm in https://github.com/apache/incubator-uniffle/pull/826
• [#593][part-1] feat: Codec compress support ByteBuffer by @xumanbu in https://github.com/apache/incubator-uniffle/pull/830
• [#389] Improvement: Make codec to be a singleton. by @tobehardest in https://github.com/apache/incubator-uniffle/pull/840
• [MINOR] build(operator): update clusterrole of controller and webhook by @wangao1236 in https://github.com/apache/incubator-uniffle/pull/842
• fix(client): disable spark memory spill by @zuston in https://github.com/apache/incubator-uniffle/pull/844
• [#841] feat(config): Support deprecated and fallback keys for ConfigOptions by @zuston in https://github.com/apache/incubator-uniffle/pull/843
• [#414] feat(client): support specifying per-partition's max concurrency to write in client side by @zuston in https://github.com/apache/incubator-uniffle/pull/815
• [#770] feat(cli): Introduce apache.commons.cli basic framework by @slfan1989 in https://github.com/apache/incubator-uniffle/pull/833
• [#133] feat(netty): Introduce ShuffleServerGrpcNettyClient. by @leixm in https://github.com/apache/incubator-uniffle/pull/839
• [#596][FOLLOWUP] Index data support offheap read by @jerqi in https://github.com/apache/incubator-uniffle/pull/852
• [#859][Improvement] Set MALLOC_ARENA_MAX in start-shuffle-server.sh by @rhh777 in https://github.com/apache/incubator-uniffle/pull/860
• [#593][FOLLOWUP] Fix zstd compress dest ByteBuffer position by @xumanbu in https://github.com/apache/incubator-uniffle/pull/857
• [MINOR] chore(ci): Add Tez pipeline by @jerqi in https://github.com/apache/incubator-uniffle/pull/862
• [#774] docs: Fix the metadata.annotations: Too long in install.md by @cchung100m in https://github.com/apache/incubator-uniffle/pull/853
• [#863] feat(coordinator): support comments in exclude node files by @yl09099 in https://github.com/apache/incubator-uniffle/pull/874
• [MINOR] refactor: Reduce the usage of memory in the ShuffleWriter by @jerqi in https://github.com/apache/incubator-uniffle/pull/877
• [#493] improvement: replace putIfAbsent with computeIfAbsent to avoid performance loss in some critical paths by @cchung100m in https://github.com/apache/incubator-uniffle/pull/876
• [MINOR] chore: delete checkstyle-suppressions.xml by @kaijchen in https://github.com/apache/incubator-uniffle/pull/878
• [MINOR] Remove unused code of shuffle upload by @jerqi in https://github.com/apache/incubator-uniffle/pull/883
• [#884] improvement: Make start and stop scripts executable under the bin folder by @jiafuzha in https://github.com/apache/incubator-uniffle/pull/885
• [#133] feat(netty): Implement ShuffleServer interface. by @leixm in https://github.com/apache/incubator-uniffle/pull/879
• [#715] fix(mr): The container does not exit because shuffleclient is not closed by @zhaobing001 in https://github.com/apache/incubator-uniffle/pull/882
• [#872] feat(tez): Add the common and utils class by @lifeSo in https://github.com/apache/incubator-uniffle/pull/890
• [#886] fix(mr): MR Client may lost data or throw exception when rss.storage.type without MEMORY. by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/887
• [#133] feat(netty): Fix IllegalReferenceCountException. by @leixm in https://github.com/apache/incubator-uniffle/pull/899
• [MINOR] fix: Fix LocalStorageManager divide by zero exception by @leixm in https://github.com/apache/incubator-uniffle/pull/900
• [#895] improvement: Rename Hdfs*.java to Hadoop*.java to support other Hadoop FS-compatible distributed filesystem by @jiafuzha in https://github.com/apache/incubator-uniffle/pull/898
• [MINOR] refactor: Add overwrite annotation by @lifeSo in https://github.com/apache/incubator-uniffle/pull/905
• [MINOR] docs: Add benchmark results by @jerqi in https://github.com/apache/incubator-uniffle/pull/904
• [#417] refactor: Eliminate raw use of parameterized class by @cchung100m in https://github.com/apache/incubator-uniffle/pull/891
• [#881] fix: Ensure LocalStorageMeta disk size is correctly updated when events are processed by @awdavidson in https://github.com/apache/incubator-uniffle/pull/902
• [#896] Improvement: Support Hadoop-3.2 by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/897
• [MINOR] docs: update document and build script for Hadoop-3.2 by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/912
• [MINOR] refactor: Fix tez workflow and checkstyle by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/911
• [#908] feat(tez): Write byte array shuffle data to MapOutput by @lifeSo in https://github.com/apache/incubator-uniffle/pull/909
• [#872] feat(tez): Get parameter from Inputcontext then provide util function by @lifeSo in https://github.com/apache/incubator-uniffle/pull/915
• [#872][FOLLOWUP] feat(tez): Add the common and utils class by @lifeSo in https://github.com/apache/incubator-uniffle/pull/894
• [#872][FOLLOWUP] feat(tez): Modify utils and add test case by @lifeSo in https://github.com/apache/incubator-uniffle/pull/916
• [#872][FOLLOWUP] feat(tez): Add UmbilicalUtils to get Worker info from AM by @lifeSo in https://github.com/apache/incubator-uniffle/pull/919
• [MINOR] fix: Error logs upload fail by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/917
• [MINOR] improvement(mr): Add @ Test to activate test case by @lifeSo in https://github.com/apache/incubator-uniffle/pull/923
• [#927] Improvement: improve the control of server heartbeat by @summaryzb in https://github.com/apache/incubator-uniffle/pull/928
• [#819 ] feat(tez): Tez ApplicationMaster supporting RemoteShuffle by @baitian77 in https://github.com/apache/incubator-uniffle/pull/918
• [#854][FOLLOWUP] feat(tez): add RssTezFetcher to fetch data from worker. by @lifeSo in https://github.com/apache/incubator-uniffle/pull/920
• [MINOR] fix: Fix kerberos ut error caused by Config.singleton is not refresh. by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/932
• [#933] fix: incorrect metric grpc_server_connection_number by @xianjingfeng in https://github.com/apache/incubator-uniffle/pull/934
• [#855] feat(tez): Support Tez Output OrderedPartitionedKVOutput by @bin41215 in https://github.com/apache/incubator-uniffle/pull/930
• [#343] improvement(build): Shade Netty Native lib by @cchung100m in https://github.com/apache/incubator-uniffle/pull/924
• [MINOR] Add new collaborators by @jerqi in https://github.com/apache/incubator-uniffle/pull/942
• [#854][FOLLOWUP] feat(tez): Add RssTezFetcherTask to fetch data from worker for OrderedInput by @lifeSo in https://github.com/apache/incubator-uniffle/pull/935
• [#855] feat(tez): Support Tez Output UnorderedPartitionedKVOutput by @bin41215 in https://github.com/apache/incubator-uniffle/pull/943
• [#855] feat(tez): Support Tez Output RssUnorderedKVOutput by @bin41215 in https://github.com/apache/incubator-uniffle/pull/944
• [#854][FOLLOWUP] feat(tez): Add Simple Fetched Allocator to allocation memory or disk for shuffle data by @lifeSo in https://github.com/apache/incubator-uniffle/pull/922
• [#590][part-1] ManagedBuffer instead ByteBuf to hold ShuffleData by @xumanbu in https://github.com/apache/incubator-uniffle/pull/906
• [#854][FOLLOWUP] feat(tez): Add RssShuffleScheduler to run and manager shuffle work by @lifeSo in https://github.com/apache/incubator-uniffle/pull/948
• [#854][FOLLOWUP] feat(tez): Add RssShuffle to handle event and generate fetch task by @lifeSo in https://github.com/apache/incubator-uniffle/pull/929
• [#940] feat: Support columnar shuffle with gluten by @summaryzb in https://github.com/apache/incubator-uniffle/pull/950
• [#937] feat: Add rest api for servernode list of losing connection and unhealthy by @yl09099 in https://github.com/apache/incubator-uniffle/pull/938
• [#768] feat(cli): Cli method for blacklist update by @Kwafoor in https://github.com/apache/incubator-uniffle/pull/931
• [#381]imporvement(server): Check JAVA_HOME and HADOOP_HOME in start-shuffle-server.sh by @cchung100m in https://github.com/apache/incubator-uniffle/pull/954
• [#854][FOLLOWUP] feat(tez): Add RssShuffleManager to run and manager shuffle work by @lifeSo in https://github.com/apache/incubator-uniffle/pull/947
• [#956] refactor: Changes the Boolean flag that determines whether a Node is healthy to a state by @yl09099 in https://github.com/apache/incubator-uniffle/pull/959
• [#854][FOLLOWUP] feat(tez): Add Rss Input Class to begin Tez input task by @lifeSo in https://github.com/apache/incubator-uniffle/pull/949
• [MINOR] fix(tez): fix thread factory name by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/975
• [#886] fix(tez): Tez Client may lost data or throw exception when rss… by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/976
• [#940] improvement: Optimize columnar shuffle integration by @summaryzb in https://github.com/apache/incubator-uniffle/pull/958
• [#961] refactor(coordinator): Improve Coordinator Log Format. by @slfan1989 in https://github.com/apache/incubator-uniffle/pull/962
• [#965] feat(tez): support remote shuffle for tez framework by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/966
• [#978][Improvement] Provides a tool class to format CLI output content by @yl09099 in https://github.com/apache/incubator-uniffle/pull/979
• [#768] [Follow Up] feat(cli): Cli method for blacklist update. by @slfan1989 in https://github.com/apache/incubator-uniffle/pull/968
• [MINOR] docs: update document for tez client plugin. by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/987
• [#983] improvement(tez): Optimize tez client delivery configuration by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/985
• [MINOR] Add new collaborators by @jerqi in https://github.com/apache/incubator-uniffle/pull/995
• [MINOR] docs: update the document and scripts of build tez. by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/997
• [#986] improvement(tez): Optimize the method of obtain the vertex id. by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/990
• [#864] feat(server): Introduce Jersey to strengthen REST API by @xianjingfeng in https://github.com/apache/incubator-uniffle/pull/939
• [#989] fix(tez): parition class is not set for RssUnorderedKVOutput. by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/994
• [#757] feat(server): separate flush thread pools for different storage types by @leixm in https://github.com/apache/incubator-uniffle/pull/775
• [#1001] improvement: support get all metrics by one request by @xianjingfeng in https://github.com/apache/incubator-uniffle/pull/1002
• [#957] feat(tez): Tez examples integration test by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/982
• [#991] Improvement(tez): TezRemoteShuffleManager support secure cluster. by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/1005
• [#1010]chore: enable and enforce spotless by @advancedxy in https://github.com/apache/incubator-uniffle/pull/1009
• [#992] fix(tez): convertTaskAttemptIdToLong should not consider appattemptId by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/1007
• [#999] improvement: apply REST principles in the urls of the http interfaces by @xianjingfeng in https://github.com/apache/incubator-uniffle/pull/1000
• [MINOR] test(tez): add storage type configs in TezRemoteShuffleManagerTest by @kaijchen in https://github.com/apache/incubator-uniffle/pull/1015
• [#972 ] fix(tez): Add output mapOutputByteCounter metrics by @bin41215 in https://github.com/apache/incubator-uniffle/pull/1016
• [MINOR] refactor: use general method to check the remote storage existence by @zuston in https://github.com/apache/incubator-uniffle/pull/980
• [#993] feat(tez): Optimize the method of obtain the application attem… by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/1021
• [#388][FOLLOWUP] fix: Fix the flaky test GrpcServerTest by @Alisha-0321 in https://github.com/apache/incubator-uniffle/pull/1023
• [#846] feat(cli): Service's start & stop & restart through Cli. by @slfan1989 in https://github.com/apache/incubator-uniffle/pull/925
• [#973] improvement: Make shuffle manager client RPC timeout configurable by @cchung100m in https://github.com/apache/incubator-uniffle/pull/1017
• [#477] feat(spark): support getShuffleResult throws FetchFailedException. by @leixm in https://github.com/apache/incubator-uniffle/pull/1004
• [#1019] test(tez) RssOrderedPartitionedKVOutputTest add close func unit test by @bin41215 in https://github.com/apache/incubator-uniffle/pull/1025
• [#808] feat(spark): ensure thread safe and data consistency when spilling by @zuston in https://github.com/apache/incubator-uniffle/pull/848
• [MINOR] Add new collabrator by @jerqi in https://github.com/apache/incubator-uniffle/pull/1031
• [#1018] test(tez) RssUnorderedPartitionedKVOutputTest add close func ut by @bin41215 in https://github.com/apache/incubator-uniffle/pull/1034
• [#133] feat(netty): integration-test supports netty. by @leixm in https://github.com/apache/incubator-uniffle/pull/1008
• [#889] improvement: Modify the default value of the rss.coordinator.select.partition.strategy parameter to CONTINUOUS. by @leixm in https://github.com/apache/incubator-uniffle/pull/1036
• [#889] improvement: Modify default value of single buffer flush. by @leixm in https://github.com/apache/incubator-uniffle/pull/1039
• [#889] improvement: Modify default value of rss.server.max.concurrency.of.per-partition.write to 30. by @leixm in https://github.com/apache/incubator-uniffle/pull/1037
• [MINOR] doc: fix metrics document by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/1040
• [#133] feat(netty): local shuffle read support zero-copy. by @leixm in https://github.com/apache/incubator-uniffle/pull/1047
• [#640] feat(netty): Metric system for netty server by @xumanbu in https://github.com/apache/incubator-uniffle/pull/1041
• [#1048] fix(coord): ExcludeNodes does not take effect when the coordinator restarts. by @leixm in https://github.com/apache/incubator-uniffle/pull/1049
• [MINOR] chore(asf): stop forwarding GitHub issues to dev mail list by @kaijchen in https://github.com/apache/incubator-uniffle/pull/1032
• [#1011] feat(tez): Avoid recompute succeeded task. by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/1033
• [#477] feat(spark): Fix rss.resubmit.stage does not support dynamic client conf. by @leixm in https://github.com/apache/incubator-uniffle/pull/1050
• [MINOR] doc: Add JDK benchmark tests by @jerqi in https://github.com/apache/incubator-uniffle/pull/1059
• [#1064] improvement(tez): Make shuffle data send thread pool configurable in WriteBufferManager. by @zhuyaogai in https://github.com/apache/incubator-uniffle/pull/1065
• [MINOR] fix(mr): fix default value by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/1035
• [#1068] improvement(tez): Fail fast in WriteBufferManager when failed to send shuffle data to shuffle sever. by @zhuyaogai in https://github.com/apache/incubator-uniffle/pull/1069
• [#1070] fix(tez): shuffle server may leak if not register remote stor… by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/1076
• [#1066] fix: The Jetty server fails to start when compiled with JDK 8 but runtime version is JDK 11. by @qijiale76 in https://github.com/apache/incubator-uniffle/pull/1067
• [#1095] doc: Add slack link to the readme.md by @ducksblock in https://github.com/apache/incubator-uniffle/pull/1099
• [#1100] improvement(mr): Fail fast in SortWriteBufferManager when failed to send shuffle data to shuffle server. by @zhuyaogai in https://github.com/apache/incubator-uniffle/pull/1103
• [#299] improvement: Make config type of RSS_STORAGE_TYPE as enum by @cchung100m in https://github.com/apache/incubator-uniffle/pull/1052
• Revert "[#299] improvement: Make config type of RSS_STORAGE_TYPE as enum" by @jerqi in https://github.com/apache/incubator-uniffle/pull/1106
• [#1074] feat: Introduce the metric of local_storage_uniffle_used_space by @zuston in https://github.com/apache/incubator-uniffle/pull/1075
• [MINOR] fix(tez): Add output mapOutputRecordCounter metrics by @bin41215 in https://github.com/apache/incubator-uniffle/pull/1093
• [#1101] improvement(tez): Release server resources as soon as possible in RssDAGAppMaster. by @zhuyaogai in https://github.com/apache/incubator-uniffle/pull/1102
• [#1043] chore: Add NOTICE-binary by @jerqi in https://github.com/apache/incubator-uniffle/pull/1097
• [#1006] feat(spark): Support Spark 3.4 by @summaryzb in https://github.com/apache/incubator-uniffle/pull/1082
• [#434] refactor: New utility method to cover dynamic class loading in RSSUtils by @pegasas in https://github.com/apache/incubator-uniffle/pull/1104
• [#1060] doc: Add the document for Netty. by @leixm in https://github.com/apache/incubator-uniffle/pull/1116
• [#1045] fix(server): shuffle server may hang when restart worker due to multi require-momery and no require-momery release by @lifeSo in https://github.com/apache/incubator-uniffle/pull/1058
• [#299] improvement: Make config type of RSS_STORAGE_TYPE as enum by @cchung100m in https://github.com/apache/incubator-uniffle/pull/1123
• [#1081] fix(tez): shuffle can not read the data which is flushed to hdfs by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/1118
• [#1124] docs: Add the document for tez-client by @bin41215 in https://github.com/apache/incubator-uniffle/pull/1125
• [#1109] fix(tez): Fix the user of remote storage. by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/1128
• [#1114] feat: introduce hdfs host as the total_hadoop_write_data metric label by @zuston in https://github.com/apache/incubator-uniffle/pull/1107
• [#1124] docs: modify the document for tez-client by @bin41215 in https://github.com/apache/incubator-uniffle/pull/1126
• [#1111] fix: Shuffle server can not delete remote storage path in security cluster by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/1122
• [#1115] improvement: Unregister shuffle explicitly when application i… by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/1131
• [#951] build(operator): Add the HADOOP_VERSION for Docker image. by @qijiale76 in https://github.com/apache/incubator-uniffle/pull/1027
• [#837] feat: Display information of all application through Cli. by @slfan1989 in https://github.com/apache/incubator-uniffle/pull/964
• [#1124] docs(tez): Add the document for tez-client by @bin41215 in https://github.com/apache/incubator-uniffle/pull/1140
• [#1124] docs(tez): Add the document of config option tez.rss.client.send.thread.num by @bin41215 in https://github.com/apache/incubator-uniffle/pull/1142
• [#1143] docs: Correct sequence number text by reducing paragraph indentation by 1 space by @bowenliang123 in https://github.com/apache/incubator-uniffle/pull/1144
• [#1044] chore: Add LICENSE-binary by @jerqi in https://github.com/apache/incubator-uniffle/pull/1054
• [#1132] improvement(spark): Unregister shuffle explicitly when Spark application is stopped. by @zhuyaogai in https://github.com/apache/incubator-uniffle/pull/1139
• [#1127] fix(netty): incorrect bytebuf release for ShuffleBlockInfo data in client side by @zuston in https://github.com/apache/incubator-uniffle/pull/1150
• [#1051] fix(mr): Ensure configurations in both mapred-site.xml and dynamic_client.conf take effect. by @qijiale76 in https://github.com/apache/incubator-uniffle/pull/1112
• [#1155] fix(netty): io.netty.util.internal.OutOfDirectMemoryError. by @leixm in https://github.com/apache/incubator-uniffle/pull/1151
• [#1152] fix: Direct memory may leak in shuffle server. by @zhengchenyu in https://github.com/apache/incubator-uniffle/pull/1154
• [MINOR] Remove leixm from collaborators by @jerqi in https://github.com/apache/incubator-uniffle/pull/1158
• [#1108] feat(server): Add labels with disk path for local storage total_localfile_write_data metrics. by @zhuyaogai in https://github.com/apache/incubator-uniffle/pull/1160
• [#722] test: cleanup residue files in tmp directory after tests by @cchung100m in https://github.com/apache/incubator-uniffle/pull/1134
• [#1161] improvement: Reduce the data copy by @jerqi in https://github.com/apache/incubator-uniffle/pull/1162
• [#1062] fix(tez): Fix to use TezIdHelper to getTaskAttemptId instead of default IdHelper. by @lifeSo in https://github.com/apache/incubator-uniffle/pull/1078
• [#1013] fix(tez): Wait when return MapOutput of type wait by @lifeSo in https://github.com/apache/incubator-uniffle/pull/1063
• [#1165] improvement(tez): Unregister shuffle data after completing the execution of a DAG. by @zhuyaogai in https://github.com/apache/incubator-uniffle/pull/1166
• [#1165][FOLLOWUP] Fix some spelling mistakes. by @zhuyaogai in https://github.com/apache/incubator-uniffle/pull/1172
• [#1164] refactor: Exposing the getDataLen method for ShuffleDataResult by @pegasas in https://github.com/apache/incubator-uniffle/pull/1170
• [#1173] fix: incorrect shuffle server status by @xianjingfeng in https://github.com/apache/incubator-uniffle/pull/1174
• [#1086] [Doc] Simplify the Gluten code and add the doc by @summaryzb in https://github.com/apache/incubator-uniffle/pull/1322