Who is using WAL?

[roseduan]

Calling all WAL(Write Ahead Log) users!

Share a brief description of your project/use case, how WAL benefits you, and any encountered challenges.

Your insights will strengthen our community and inspire others.

Comment below with:

• project overview
• what scenario do you use WAL?
• what new feature do you expect for WAL?

[roseduan]

rosedb and lotusdb are using wal as file storage backend.🎉🎉

[amityahav]

I started using this wal for my personal vector database project. So far no complaints. Thanks!

[hitesh22rana]

Hi @roseduan, 🙇🏼

Thank you for creating and maintaining this amazing package! 🎉

I'm currently building a message queue broker and utilizing this WAL for ensuring consistency. Here's the link to my project: mq

So far, the experience has been great—no complaints at all! That said, I have a feature request:

Would it be possible to introduce a way to asynchronously sync data to disk at configurable intervals?

If this feature isn’t currently supported, I believe it could be a valuable addition. I’d love to contribute by working on its implementation if it’s feasible. Please let me know your thoughts!

[roseduan]

Hi @roseduan, 🙇🏼

Thank you for creating and maintaining this amazing package! 🎉

I'm currently building a message queue broker and utilizing this WAL for ensuring consistency. Here's the link to my project: mq

So far, the experience has been great—no complaints at all! That said, I have a feature request:

Would it be possible to introduce a way to asynchronously sync data to disk at configurable intervals?

If this feature isn’t currently supported, I believe it could be a valuable addition. I’d love to contribute by working on its implementation if it’s feasible. Please let me know your thoughts!

Great!

I also think this is a valuable addition, looking forward to your PR!

[hitesh22rana]

Hi @roseduan, 🙇🏼

Sorry for being late.
PTAL: #44

As requested, this introduces the asynchronous way to sync data to disk at configurable time interval.

[mikepop7]

I'm looking for a WAL component for safety recovery when system crash and support auto purge. In my project data write memory (In memory part) first and persist to disk regularity(e.g. 5 seconds). We consider write WAL first，and persist disk successful we can purge unnecessary segment. But this project isn't support auto clear segments (like commit position), this maybe a common requirements?

[roseduan]

I'm looking for a WAL component for safety recovery when system crash and support auto purge. In my project data write memory (In memory part) first and persist to disk regularity(e.g. 5 seconds). We consider write WAL first，and persist disk successful we can purge unnecessary segment. But this project isn't support auto clear segments (like commit position), this maybe a common requirements?

What do you mean 'auto clear segments'?

[mikepop7]

I'm looking for a WAL component for safety recovery when system crash and support auto purge. In my project data write memory (In memory part) first and persist to disk regularity(e.g. 5 seconds). We consider write WAL first，and persist disk successful we can purge unnecessary segment. But this project isn't support auto clear segments (like commit position), this maybe a common requirements?

What do you mean 'auto clear segments'?

such as:

wal, _ := wal.Open(opts) wal.ClearSegmentsBefore(&chunkPosition)

[hitesh22rana]

I'm looking for a WAL component for safety recovery when system crash and support auto purge. In my project data write memory (In memory part) first and persist to disk regularity(e.g. 5 seconds). We consider write WAL first，and persist disk successful we can purge unnecessary segment. But this project isn't support auto clear segments (like commit position), this maybe a common requirements?

What do you mean 'auto clear segments'?

such as:

wal, _ := wal.Open(opts) wal.ClearSegmentsBefore(&chunkPosition)

So, if I’m getting it right, you’re talking about checkpointing. We can control how many WAL segments we keep, and as new segments are added, older ones can be archived or deleted, which balances durability with efficient storage management.

[mikepop7]

I'm looking for a WAL component for safety recovery when system crash and support auto purge. In my project data write memory (In memory part) first and persist to disk regularity(e.g. 5 seconds). We consider write WAL first，and persist disk successful we can purge unnecessary segment. But this project isn't support auto clear segments (like commit position), this maybe a common requirements?

What do you mean 'auto clear segments'?

such as:
wal, _ := wal.Open(opts) wal.ClearSegmentsBefore(&chunkPosition)

So, if I’m getting it right, you’re talking about checkpointing. We can control how many WAL segments we keep, and as new segments are added, older ones can be archived or deleted, which balances durability with efficient storage management.

yeah, only keep fix segment size or age, such zap.Logger config:
MaxSize: 1GB, MaxAge: 30 days