Merge branch 'master' into master

Author: dingxin-tech

.gitignore

@@ -6,4 +6,6 @@
 target/
 .DS_Store
 *.pyc
-replace_config.sh
+replace_config.sh
+ajcore.*
+.vscode/

CHANGELOG.md

@@ -1,249 +1,26 @@
-# 0.41.0
-- ArrayRecord support ZonedDateTime/LocalDate/Instant API
+# Changelog
+
+## [0.47.0-public] - 2024-04-08
+
+### Added
+- Support for new Stream type that enables incremental queries.
+- `preview` method to the `TableTunnel` for data preview purposes.
+- `OdpsRecordConverter` for parsing and formatting records.
+- Enhancements to the `Projects` class with `create` and `delete` methods now available, and `update` method made public. Operations related to the `group-api` package are now marked as deprecated.
+- Improved `Schemas` class to support filtering schemas with `SchemaFilter`, listing schemas, and retrieving detailed schema metadata.
+- `DownloadSession` introduces new parameter `disableModifiedCheck` to bypass modification checks and `fetchBlockId` to skip block ID list retrieval.
+- `TableWriteSession` supports writing `TIMESTAMP_NTZ` / `JSON` types and adds a new parameter `MaxFieldSize`.
+
+### Changed
+- The implementation of the `read` method in the `Table` class is now replaced with `TableTunnel.preview`, supporting new types in MaxCompute and time types switched to Java 8 time types without timezone.
+- The default `MapWritable` implementation switched from `HashMap` to `LinkedHashMap` to ensure order.
+- `Column` class now supports creation using the Builder pattern.
+
+### Improved
+- `TableReadSession` now supports predicate pushdown and introduces new parameters `maxBatchRawSize` and `splitMaxFileNum`.
+- `UpsertSession` enhancements:
+  - Supports writing partial columns.
+  - Allows setting the number of Netty thread pools with the default changed to 1.
+  - Enables setting maximum concurrency with the default value changed to 16.
+- `TableTunnel` now supports setting `quotaName` option.
 
-# 0.40.x
-- fix DateTime using java.time
-Fixes
-- fix incorrect value of schemaName value in two-tier project
-
-# 0.36.4-public
-## Enhencements:
-  - [Core] Support iterate over projects with filter. See Projects#iteratorByFilter(ProjectFilter filter)
-  - [Tunnel] Remove the 8 MB limit on field size. The field size limit can now be configured by "odps.sql.cfile2.field.maxsize".
-  - [Tunnel] Support write timeout
-  - [Tunnel] Support auto partition creation in StreamTunnel  
-  - [SQLExecutor] Add tunnel endpoint cache
-  - [SQLExecutor] Support result download in batch
-## Fixes:
-  1. [Core] Fix incorrect return value from Instance#getTaskCost(String taskName)  
-  2. [SQLExecutor] Fix hang when downloading result
-
-# 0.34.4
-- session v2
-- support SQL function
-- support stream tunnel
-
-# 0.33.7
-- support list table
-- table tunnel supports overwrite mode
-- support global settings
-- support enable/disable download privilege
-
-# 0.32.5
-- java8+ required
-- replace jaxb with simple xml
-- add parameter to enable/disable type validation in ArrayRecord
-
-# 0.31.5
-- delete dateutils related in tunnel r/w, to unify timezone logic with oversea version
-
-# 0.31.4
-- support tunnel download session async mode
-
-# 0.31.3
-- import maven source plugin explicitly to avoid problems
-
-# 0.31.2
-- support optional strict data validation for String and datetime
-- support new typeinfo
-
-# 0.31.0
-- remove fastjson
-- async create download session
-
-# 0.30.1
-- support java8_161+ xml problem
-- biggraph support set running cluster
-
-# 0.30.8
-- move updateProject/listProject to odps-sdk-core
-
-# 0.30.0
-- add extended labels
-- support java9
-- support biggraph flash job
-- expand decimal scale
-
-# 0.29.10
-- provide join key for udx
-
-# 0.29.9
-- add tunnel transform header
-
-# 0.29.8
-- increase openmr volume input/output limit from 256 to 1024
-- add user env in task settings, include user ip and mac
-
-# 0.29.6
-- revert xflow priority
-
-# 0.29.4
-- SQLTask.getResultSet support tunnel endpoint
-
-# 0.29.3
-- add sqlrt task
-- support java1.8_161 xml problem, force to use cdata
-
-# 0.29.2
-- add biggraph
-- biggraph support setting running cluster
-
-# 0.29.1
-- delete project.getSystemVersion
-- commons-logging version to 1.2
-- add AuthorizationQueryInstance to support auth async query
-
-# 0.28.0
-- quota support gpu
-- refactor tunnel buffered writer retry
-- add volume lifecycle
-- support parent and son quota
-- tunnel add logger
-- openmr support sprint-boot jar
-- support create external table
-- remove remove jcabi dep
-
-# 0.27.0 
-- add InstanceQueueingInfo
-- support check permission for select columns
-
-# 0.26.5
-- remove jcabi, fix 0.26.2-public duplicated field bug
-
-# 0.26.4
-- fix mr secondary partition bug
-
-# 0.26.3
-- fastjson update to 1.2.28.odps
-
-# 0.26.2
-- fastjson update to 1.2.25
-
-# 0.26.0 
-- add SQLTask.getResultSet
-- copytask support new type
-- mv xflow, xflows, ml from core-internal to core
-
-# 0.25.2
-- refactor Date io in tunnel
-- refactor Java classes for new type
-- fix graph classloader bug in sandbox
-
-# 0.25.0
-- add `Instance.getTaskInfo`
-- add `SQLTASK.getSqlWarning`
-- add `Instance.getTaskQuotaJson`
-- specify 'Asia/Shanghai' to be default timezon in DateUtils
-- add type info system and extend `OdpsType` to support more data type
-- refactor tunnel sdk to support more data type
-
-# 0.24.0
-- improve sync instance
-- support external table
-- copy task support tunnel endpoint router 
-- fix `user.reload` bug, mk it work
-- optimize mr multi output
-- add `BearerTokenAccount`
-
-# 0.23.4
-- always do reload in onlinemodel
-
-# 0.23.3
-- use utf-8 charset in event notification
-
-# 0.23.2
-- make `OdpsHooks` thread safe
-
-# 0.23.1
-- add `tables.loadTables` and `tables.reloadTables`
-
-# 0.22.3                                                                            
-- add `listRolesForUserName` and `listRolesForUserID`                               
-- refine `OnlineModel`                                                              
-
-# 0.22.2                                                                            
-- fix mr get summary hang  
-
-# 0.22.1
-- revert instance retry, rm guid in job model
-
-# 0.22.0 
-- `SecurityConfiguration` support AclV2 and PackageV2
-- add `Instance.getTaskCost`
-- switch to fastjson for parsing json, almost
-- remove dependency bouncycastle
-
-# 0.21.2
-- add retry logic in `TunnelRecordReader`
-- add `TunnelBufferedWriter`
-- add `InstanceTunnel` for downloading result of select statement
-- default logview host change to logview.odps.aliyun-inc.com
-- default connection timeout change from 5s to 10s
-
-# 0.21.1 
-- add `Function.getProject`, `Volume.getProject` and `StreamJob.getProject`
-- add `UploadSession.writeBlock`, `ProtobufRecordStreamWriter.write(RecordPack)` is deprecated
-- add `InstanceTunnel`
-- fix `Function.getResourceNames` returning wrong resource name
-- return partition info in `PackReader.read`
-- add `ServerTask` support
-
-# 0.20.7
-- security improvement
-
-# 0.20.1
-- fix pipeline combiner
-
-# 0.20.0
-- add ArrayRecord.clear()
-- add onInstanceCreated hook
-- array|map is supported in TableTunnel
-- add volumefs sdk
-- add Table.getTableID()
-
-# 0.19.3
-- fix tunnel download with specified columns
-
-# 0.19.2
-- fix DateUtils threads bug.
-
-# 0.19.1
-- add matrix sdk
-
-# 0.19.0
-- add volume resource sdk
-- `StreamRecordPack.clear()` do not throw exception any more
-- add `StreamClient.loadShard(long)`, deprecate `StreamClient.loadShard(int)`
-- add `StreamClient.waitForShardLoad()`
-- add `SQLTask.getResult`
-- add `XFlows.getXFlowInstance`
-- add `ArrayRecord(TableSchema)`
-- add `CheckPermission`
-- CopyTask support GroupAccount
-
-# 0.18.0
-- remove odps-sdk-ml(unused code)
-
-# 0.17.8
-- decimal range check in sdk
-- support TableSchema set Columns
-- Tables.create 支持comment
-
-# 0.17.4
-- fix domain account bug
-
-# 0.17.2
-- fix WritableRecord.getBytes
-
-# 0.17.0
-- fix get resource as streammulti
-- tmp resource
-- Get Project Clusters
-- public CheckSum
-- Table support if (not) exists
-
-# 0.16.6
-- project cluster API
-
-# 0.16.5
-- RecordPack memory size

CHANGELOG_CN.md

@@ -0,0 +1,26 @@
+# 更新日志
+
+## [0.47.0-public] - 2024-04-08
+
+### 新增
+- 对 Stream 新类型的支持，可用于进行增量查询。
+- 在 `TableTunnel` 中增加了 `preview` 方法，用于数据预览。
+- 引入 `OdpsRecordConverter`，用于对 Record 进行解析和格式化。
+- `Projects` 类增加了 `create`（创建）和 `delete`（删除）方法，`update` 方法现已公开。`group-api` 包下的相关操作已被标记为弃用。
+- `Schemas` 类增强，支持通过设置 `SchemaFilter` 来过滤 schema，支持 `listSchema` 以及获取 schema 的详细元信息。
+- `DownloadSession` 新增参数 `disableModifiedCheck`，用于跳过修改检查。新增参数 `fetchBlockId`，用于跳过获取 block ID 列表。
+- `TableWriteSession` 支持写入 `TIMESTAMP_NTZ` / `JSON` 类型，新增参数 `MaxFieldSize`。
+
+### 变更
+- `Table` 类的 `read` 方法实现现已更换为 `TableTunnel.preview` 方法，会支持 MaxCompute 新类型，时间类型切换为 Java 8 无时区类型。
+- 默认的 `MapWritable` 实现从 `HashMap` 改为 `LinkedHashMap`，以确保有序。
+- `Column` 类现支持使用建造者模式（Builder pattern）进行创建。
+
+### 改进
+- `TableReadSession` 现支持谓词下推，新增参数 `maxBatchRawSize` 和 `splitMaxFileNum`。
+- `UpsertSession` 现支持：
+  - 写入部分列。
+  - 设置 Netty 线程池的数量（默认更改为 1）。
+  - 设置最大并发量（默认值更改为 16）。
+- `TableTunnel` 支持设置 `quotaName` 选项。
+