v0.48.1-public

Changelog

[0.48.1-public] - 2024-05-07

Added

Arrow and ANTLR Libraries: Added new includes to the Maven Shade Plugin configuration for better handling and packaging of specific libraries. These includes ensure that certain essential libraries are correctly packaged into the final shaded artifact. The newly included libraries are:

• org.apache.arrow:arrow-format:jar
• org.apache.arrow:arrow-memory-core:jar
• org.apache.arrow:arrow-memory-netty:jar
• org.antlr:ST4:jar
• org.antlr:antlr-runtime:jar
• org.antlr:antlr4:jar
• org.antlr:antlr4-runtime:jar

Relocation Adjustments

Shaded Relocation for ANTLR and StringTemplate: The configuration now includes updated relocation rules for org.antlr and org.stringtemplate.v4 packages to prevent potential conflicts with other versions of these libraries that may exist in the classpath. The new shaded patterns are:
org.stringtemplate.v4 relocated to com.aliyun.odps.thirdparty.org.stringtemplate.v4
org.antlr relocated to com.aliyun.odps.thirdparty.antlr

更新日志

[0.48.1-public] - 2024-05-07

新增

Arrow和ANTLR库：在 Maven Shade 插件配置中添加了新的包含项，以更好地处理和打包特定库。这些包含项确保某些关键库被正确地打包进最终的遮蔽(Shaded)构件中。新加入的库包括：

• org.apache.arrow:arrow-format:jar
• org.apache.arrow:arrow-memory-core:jar
• org.apache.arrow:arrow-memory-netty:jar
• org.antlr:ST4:jar
• org.antlr:antlr-runtime:jar
• org.antlr:antlr4:jar
• org.antlr:antlr4-runtime:jar

位置调整

ANTLR和StringTemplate的遮蔽重定位：配置现在包括针对 org.antlr 和 org.stringtemplate.v4 包的更新重定位规则，以防止可能在类路径中存在的这些库的其他版本的潜在冲突。新的遮蔽模式是：
org.stringtemplate.v4 重定位至 com.aliyun.odps.thirdparty.org.stringtemplate.v4
org.antlr 重定位至 com.aliyun.odps.thirdparty.antlr

0.48.0-public

Changelog

[0.48.0-public] - 2024-04-22

Added

• Introduced odps-sdk-udf module to allow batch data reading in UDFs for MaxCompute, significantly improving performance in high-volume data scenarios.
• Table now supports retrieving ColumnMaskInfo, aiding in data desensitization scenarios and relevant information acquisition.
• Support for setting proxies through the use of odps.getRestClient().setProxy(Proxy) method.
• Implementation of iterable RecordReader and RecordReader.stream() method, enabling conversion to a Stream of Record objects.
• Added new parameters upsertConcurrentNum and upsertNetworkNum in TableAPI RestOptions for more detailed control for users performing upsert operations via the TableAPI.
• Support for Builder pattern in constructing TableSchema.
• Support for toString method in ArrayRecord.

Improved

• UploadSession now supports configuration of the GET_BLOCK_ID parameter to speed up session creation when the client does not need blockId.
• Enhanced table creation method using the builder pattern (TableCreator), making table creation simpler.

Fixed

• Fixed a bug in Upsert Session where the timeout setting was configured incorrectly.
• Fixed the issue where TimestampWritable computed one second less when nanoseconds were negative.

更新日志

[0.48.0-public] - 2024-04-22

新增

• 引入了odps-sdk-udf模块，支持在UDF中按批读取MaxCompute数据，能在大数据量场景下显著提高性能。
• Table现支持获取ColumnMaskInfo，用于数据脱敏场景，方便相关信息的获取。
• 新增通过odps.getRestClient().setProxy(Proxy)方法设置代理的支持。
• 实现了可迭代的RecordReader以及RecordReader.stream()方法，允许将其转换为Record对象的流。
• 在TableAPI RestOptions中新增upsertConcurrentNum和upsertNetworkNum参数，为使用TableAPI进行upsert操作的用户提供更细致的控制。
• 支持使用Builder模式来构建TableSchema。
• ArrayRecord支持toString方法。

变更

• 现在，当用户使用StsAccount但不传递StsToken时，将被视作使用AliyunAccount。

改进

• UploadSession现支持配置GET_BLOCK_ID参数，当客户端不需要blockId时，可以加速创建Session的速度。
• 使用builder模式(TableCreator)加强了表的创建方法，现在可以更简单地创建表了。

修复

• 修复了Upsert Session获取连接时，超时时间配置错误的问题。
• 修复了TimestampWritable在纳秒为负数时计算出错一秒的问题。

v0.47.0-public

Changelog

[0.47.0-public] - 2024-04-08

Added

• Support for new Stream type that enables incremental queries.
• preview method to the TableTunnel for data preview purposes.
• OdpsRecordConverter for parsing and formatting records.
• Enhancements to the Projects class with create and delete methods now available, and update method made public. Operations related to the group-api package are now marked as deprecated.
• Improved Schemas class to support filtering schemas with SchemaFilter, listing schemas, and retrieving detailed schema metadata.
• DownloadSession introduces new parameter disableModifiedCheck to bypass modification checks and fetchBlockId to skip block ID list retrieval.
• TableWriteSession supports writing TIMESTAMP_NTZ / JSON types and adds a new parameter MaxFieldSize.
• TABLE_API adds predicate related classes to support predicate pushdown in the future.

Changed

• The implementation of the read method in the Table class is now replaced with TableTunnel.preview, supporting new types in MaxCompute and time types switched to Java 8 time types without timezone.
• The default MapWritable implementation switched from HashMap to LinkedHashMap to ensure order.
• Column class now supports creation using the Builder pattern.

Improved

• TableReadSession now introduces new parameters maxBatchRawSize and splitMaxFileNum.
• UpsertSession enhancements:
  • Supports writing partial columns.
  • Allows setting the number of Netty thread pools with the default changed to 1.
  • Enables setting maximum concurrency with the default value changed to 16.
• TableTunnel now supports setting quotaName option.

更新日志

[0.47.0-public] - 2024-04-08

新增

• 对 Stream 新类型的支持，可用于进行增量查询。
• 在 TableTunnel 中增加了 preview 方法，用于数据预览。
• 引入 OdpsRecordConverter，用于对 Record 进行解析和格式化。
• Projects 类增加了 create（创建）和 delete（删除）方法，update 方法现已公开。group-api 包下的相关操作已被标记为弃用。
• Schemas 类增强，支持通过设置 SchemaFilter 来过滤 schema，支持 listSchema 以及获取 schema 的详细元信息。
• DownloadSession 新增参数 disableModifiedCheck，用于跳过修改检查。新增参数 fetchBlockId，用于跳过获取 block ID 列表。
• TableWriteSession 支持写入 TIMESTAMP_NTZ / JSON 类型，新增参数 MaxFieldSize。
• TABLE_API 新增 predicate 相关类，用于后续支持谓词下推。

变更

• Table 类的 read 方法实现现已更换为 TableTunnel.preview 方法，会支持 MaxCompute 新类型，时间类型切换为 Java 8 无时区类型。
• 默认的 MapWritable 实现从 HashMap 改为 LinkedHashMap，以确保有序。
• Column 类现支持使用建造者模式（Builder pattern）进行创建。

改进

• TableReadSession 新增参数 maxBatchRawSize 和 splitMaxFileNum。
• UpsertSession 现支持：
  • 写入部分列。
  • 设置 Netty 线程池的数量（默认更改为 1）。
  • 设置最大并发量（默认值更改为 16）。
• TableTunnel 支持设置 quotaName 选项。

Important Correction Notice for Version 0.47.0

We have identified an issue with the recent release of version 0.47.0, where code intended for the upcoming release of version 0.48.0 was inadvertently included. We understand that this may have caused some confusion, and we sincerely apologize for any inconvenience this may have caused.

Impact Assessment

After assessing all the code, we can confirm that all changes are backward-compatible. Therefore, there is no need to worry about the current version negatively impacting usage. You can rest assured that it is safe to use.

Recommended Action

Considering that the released version 0.47.0 actually contains the content of version 0.48.0, we advise you to review the complete release notes for version 0.48.0 to prepare for the upcoming features and improvements. If you have any questions or require assistance during usage, please do not hesitate to contact us.

Acknowledgments and Appreciation

We deeply apologize for any inconvenience this matter may have caused and thank you for your understanding and support. The MaxCompute team is committed to earning your trust and we will continue working to ensure that such issues are avoided in future releases.

Should you encounter any issues or require further assistance, please visit our support page at help.aliyun.com, or raise an issue in our GitHub repository.

Thank you for your continued support.

Best Regards,
The MaxCompute Team

关于0.47.0版本重要更正通知

我们发现了最近发布的0.47.0版本中的一个问题，即原本计划在即将发布的0.48.0版本中包含的代码不慎被包含在了此次发布中。我们理解这可能会导致混淆，并且对由此可能造成的不便表示诚挚的歉意。

影响评估

我们已经评估了所有代码，确认所有变更均属于兼容性变更，因此用户不必担心当前版本会对使用产生负面影响，请您放心。

建议的行动

考虑到已经发布的0.47.0实际上包含了0.48.0版本的内容，我们建议您阅读完整的0.48.0版本发布说明，并为即将到来的特性与改进做好准备。如果您在使用过程中有任何疑问或需要帮助，请及时与我们取得联系。

致谢和感激

我们为可能给您带来的任何不便表示诚挚的歉意，并感谢您的理解与支持。MaxCompute团队值得您的信任，并会继续致力于确保未来的版本发布避免此类问题。

如果您遇到任何问题或需要更多帮助，请访问我们的支持页面 help.aliyun.com，或者在我们的GitHub仓库中提交问题。

感谢您的持续支持。

此致，
MaxCompute团队

v0.45.6-public-SNAPSHOT

update version to 0.45.6-public-SNAPSHOT

v0.45.6-public

update version to 0.45.6-public

v0.45.5-public

update version to 0.45.5-public

v0.45.3-public: update version to 0.45.3-public

sync code

v0.45.2-public

udpate to 0.45.2-public

v0.45.1-public: Merge pull request #57 from zhangdingxin484/update_dependencies

Update dependencies.