[SPARK-44223][CORE] Remove LevelDB support #43138
Conversation
| .version("3.4.0") | ||
| .stringConf | ||
| .transform(_.toUpperCase(Locale.ROOT)) | ||
| .checkValues(DBBackend.values.map(_.toString).toSet) | ||
| .createWithDefault(DBBackend.LEVELDB.name) | ||
| .createWithDefault(DBBackend.ROCKSDB.name) |
There was a problem hiding this comment.
I'm not against this change, but I'm quite conflicted about it.
For shuffle db, we didn't make ROCKSDB the default in Spark 3.5.0, so I'm not sure if it's appropriate to just remove it in Spark 4.0. This is why I haven't submitted a pr to address this issue.
Let's wait for other people's opinions.
There was a problem hiding this comment.
Agree, let us change to rocksdb as default first.
We can always remove it in 4.1 or later
There was a problem hiding this comment.
Got it, let me change to rocksdb as default with another PR.
There was a problem hiding this comment.
@Hisoka-X Spark 4.0 has already been released. Are you still interested in this work?
There was a problem hiding this comment.
Sure, let me clean it up again.
There was a problem hiding this comment.
Hi @LuciferYang , I did some update to this PR. Could you help to reopen it?
There was a problem hiding this comment.
As indicated in the above prompt, it seems that I'm unable to reopen this pr either.
There was a problem hiding this comment.
Ok, I created another pr #51027
|
@Hisoka-X Can you restart this job? I believe we can still remove Leveldb's support for HybridStore in Spark 4.0. |
Hisoka-X
force-pushed
the
SPARK-44223-remove-leveldb
branch
from
92a9a47 to
23f571e
Compare
|
We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. |
What changes were proposed in this pull request?
This PR remove LevelDB support
Why are the changes needed?
The leveldb project seems to be no longer maintained, and we can always replace it with rocksdb.
Does this PR introduce any user-facing change?
Yes, the LevelDB will not supported in 4.1.0
How was this patch tested?
exist test.
Was this patch authored or co-authored using generative AI tooling?
No