isSeek true when pausing

[serenity81]

Hi,
The isSeek is returning true when pause button is pressed and no actual seeking is done. It also occurs when we first load without any user interaction. Is this behaviour correct?
https://alphatab.net/docs/reference/api/playerpositionchanged#positionchangedeventargs-properties

I also tried to couple it with playerstatechanged to get stopped parameter but didn't help!

My question is how can I only getting if seeking (both while playing or being paused) without pause or score load triggering it initially?

[Danielku15]

Yes the behavior is correct due to this behavior: When you pause alphaTab will actually seek back to the "start" of the current beat.

e.g. if you have a full note, and press pause on half of it. the cursor and time jumps back to the start of the full note and it will be played as a whole again.

My question is how can I only getting if seeking (both while playing or being paused) without pause or score load triggering it initially?

There is no differenciation "what" triggered a seek. From a player perspective it is simply a seek to a different position. If you share more insights on what you plan to do I might be able to give some better hints how to achieve it.

[serenity81]

I want to detect user is actively seeking and give chosen time in seconds and not when a position change occurs due to pause or score load.

[serenity81]

To explain better:
// Modified by me
const songPosition = wrapper.querySelector(".at-song-position");
let previousTime = -1;
api.playerPositionChanged.on((e) => {
// reduce number of UI updates to second changes.
const currentSeconds = (e.currentTime / 1000) | 0;
if (currentSeconds == previousTime) {
return;
}

songPosition.innerText =
formatDuration(e.currentTime) + " / " + formatDuration(e.endTime);

// Added by me to detect when seeking is being done by the user
if (e.isSeek && e.currentTime != 0 && playbackState == 1) { //So to fire only when seeking && not on load & only when playing
changePlayback(currentSeconds);
console.warn(Seeking: ${e.currentTime});
} else if (e.isSeek && e.currentTime != 0 && playbackState == 0) { //e.isSeek is also true on pause button press because it auto adjust to begining of music note so it basically change spostion
changePlayback(currentSeconds);
console.warn(Seeking (While Paused): ${e.currentTime});
}

I want to detect user is actively seeking and give chosen time in seconds and not when a position change occurs due to pause or score load.

});

[serenity81]

I understand your point, but I am syncing and external source and dont want it to start on score load due to cursor moving on note or tick or such.

[Danielku15]

Now we're getting closer to the real need 😉 Seems you are already in the situation of building workarounds and you try to handle even more edge cases.

Let's chat about this real use case you're addressing. Is my understanding correct?

You have an external audio/video and you are trying to sync it with alphaTab. You are not using the "synthesizer" for your audio, but rather you have it on volume 0. The you likely use some known time-tags and time-update events to control the synth:

1. You regularly seek to the actual position you determine from the audio/video based on some custom logic
2. You adjust the playback speed so that the highlighting and cursor positioning matches the audio/video.

Is this correct?

[Danielku15]

I already went some steps ahead (with assumptions) and prepared #1948 which allows disabling of the automatic seek back to the beat start with ensuring the correct beat cursor position.

Aiming for the future feature of synchronizing alphaTab with external audio/video sources ¹²³ the logical next step would be to create an API which allows better cursor control without having the audio synth active.

But before implementing such an API I'd need some more insights if my assumptions are correct and how you exactly do the synchronization currently. The goal for alphaTab will be to:

1. Have a list of "sync points" defined which link the time positions in the source. (this is the Guitar Pro approach)
2. Provide a way to pass in the playback state (play/pause, playback speed,..) and time changes into alphaTab so it can place the cursor, highlight elements and signal events like today.
3. Provide an automatic linking with audio/video elements.

Hence there might still not be 100% manual control with that plan.

Footnotes

1. https://github.com/CoderLine/alphaTab/issues/348 ↩

2. https://github.com/orgs/CoderLine/projects/12?pane=issue&itemId=21406605 ↩

3. https://www.alphatab.net/docs/formats/guitar-pro-8/#:~:text=Audio%20File ↩

[serenity81]

Hi Daniel,

Your assumptions are mostly right. I currently have the functionality to Sync audio play/pause/stop and seek, but not beat for beat (as it might not be necessary at this stage). Its just simple crude way to play backing audio. I don't plan to zero volume of the AlphaTab synth but give the user the ability to do it if they wish to. So its a mixed playback type of scenario where for example there is a Guitar tab and guitar synth sound that goes along with it, but the user can add real mp3 drums sound as backing track. This I have accomplished so far.

The assumptions you mentioned are my plan for the next step, as I have previously read how Guitar Pro does it in:
#348:

1. You regularly seek to the actual position you determine from the audio/video based on some custom logic
2. You adjust the playback speed so that the highlighting and cursor positioning matches the audio/video.

But what I though is do it in a way to generate/encode mp3 with slowed and fasten parts and play that file as is and not necessarily do the speed changes in real time. Your approach might make changes more fluid and in real time but I think a new edited file will be more precise. But might even have to do a hybrid approach as for marking system the user has to hear the changes and result they have done. Heck maybe we can use AI sometime in the future to detect notes and beats etc automatically, I know there is some progress in this field but not sure how accurate it is currently. Not sure would love to hear your thoughts.

[Danielku15]

Thanks for sharing this valueable insight. Especially the hybrid/mixed scenario makes things a bit more tricky and simply offering an intermediate "cursor api" will not serve your needs.

It's interesting that you rather slow down the external audio. This approach makes the music notation more accurate and when people play along. Audio/Video tags would have also a playbackRate we could use to synchronize the synth with sync points.

Going back to your initial request I somehow understand now better how you need to differenciate why the time has changed but my initial stand point remains:

Considering the currenty available version without #1948, you always always should to update your external audio if isSeek is true. As alphaTab actively does a seek back to the start of the beat, it is quasi equal to the user going back manually and the backing track needs to seek too. Otherwise with every pause/play the backing track would drift out-of-sync.

If isSeek is false, you maybe just do some checks whether to speed-up / slow-down the external audio to eliminate the "drift" without the user experience "jumps".

I generally have put some thoughts into how such a sync could happen best but I haven't settled yet what's the best approach.

a) If there is only an external audio/video, there should "only" be a mechanism of placing the cursor correctly without a sync being active. This was the planned feature until now.
b) If the synth should be active, we need to define who is defining the "correct" time axis: the synth or the external audio/video. This is a new approach I haven't considered yet.

Here some thoughts all around this.

• Technically it is possible to use an external audio/video element as source in Web Audio via createMediaElementSource. This way I thought we could really "mix" together the synthesized audio with an external element. Synchronizing the audio streams is very tricky unless alphaTab would synthesize the whole song audio in advance (seeking, tempo changes etc. have to be considered). But if done correctly, there is no latency or drift between the audio streams as we mix together the audio samples correctly.
• Neglecting a bit the latency and computing power needed (e.g. smartphone battery), a very easy way to acheive a hybrid sound is to rely on the playbackRate/speed.
  • If the synth defines the correct time axis, we synchronize the time by speeding up or slowing down the media via playbackRate.
    • The playerPositionChanged and playerStateChanged events of alphaTab result in changes on the media element (setting time position on seek, calculating the required playback speed by comparing the alphatab player position with the media time position. sync points and time drifts would be considered in the speed).
    • The timeupdate event of the media element could be used to detect additional time drifts. But playerPositionChanged fires often enough to handle everything there.
  • If the external media defines the correct time axis, we just do it a bit reverse:
    • The seeked event triggers a seek in alphaTab.
    • The timeupdate and currentTime are used to calculate the playback speed of alphaTab by mapping the time again.
    • The play, pause events also cause play/pause in alphaTab.
    • The playerPositionChanged with isSeek=true (e.g. through user interaction) also should map back the time to the external media and change the audio.
    • The playerPositionChanged with isSeek=false could be used to do additional syncing on time drifts timeupdate doesn't fire on a high rate. Some interpolation during this event could help achieving higher accuracy if needed.

If your current logic is not proprietary or you are free to share your code, you could set-up a git repository with an example project. Then we can look really on that basis how to improve the situation. I'm not sure if I directly want to focus on implementing such a sync-point mapping module. My next focus point in alphaTab is: MusicXML. The more you share from your side, the better I can help.

[serenity81]

I am using the Web Audio Api coupled with buffering because audio tag alone is a nightmare when syncing multiple audio files. As for isSeek you are right that it should really work and its kinda working but feel it will add precision to know that the change came directly from the user not the script (not a big problem though).

As for your points:
a) If there is only an external audio/video, there should "only" be a mechanism of placing the cursor correctly without a sync being active. This was the planned feature until now.
In my case i could in certain cases not use a cursor at all as its not note for note (for time being) and just treat the score as it was a pdf file and scroll through it automatically. Your approach is more accurate and will have to do it in the future for sure, but feel its much harder to accomplish. At time being I wanna keep it as simple as possible (Which is hard to do :)

b) If the synth should be active, we need to define who is defining the "correct" time axis: the synth or the external audio/video. This is a new approach I haven't considered yet.
I have it both ways currently if there is a both a score & audio the score will take lead but if its audio i will show an audio player to coontrol audio

Now after I wrote all this I saw you mentioning Web Audio :) but with "Mix" as you suggested I am not sure how it will work with Alphatab Context as they might not be accessible via Api easily for me to sync which my own code, but if you do it internally in your code it will be more tightly integrated. Currently I don't use a single audio context as destination as I need each audio track to be outputted to a different audio device vis SetSinkId. I would love for each AlphaTab tracks to also be outputted to a different audio deice but you told me it's very hard, so now I can select one output device for entire Alphatab output. I do have a version where I merge destination gains, so it offers better syncing but that's when user doesn't select any non-default audio device. I hope I make sense.

[serenity81]

I have read #1961
and #1962 That seems very promising. But the marking system will be done by one of the two APIs or is it something for the future.

[Danielku15]

Do you mean by "marking" the colored highlighting of individual notes during playback? This would be part of #1961

[serenity81]

I mean putting markers at different audio file positions inorder to sync to music notes

[Danielku15]

This is the goal in #1961 ;) I refer to them as "sync point" in there.

[serenity81]

Oh I see makes sense. An unrelated inquiry if you can help, do you know of any rock/metal song that has tabs and also music audio in public domain or creative commons I can use to showcase a demo project for my app and not have copy right issue. I know you personally have Cannon Rock but is there an alternative that is safe to use and display on my website?

[serenity81]

Here's my website, just so you understand what I mean with all my questions:
🔗 https://gigmixer.com/

Please don’t judge too harshly 😅 — this is something I put together in about a month and a half. It’s still a work in progress and hasn’t been fully tested yet.

⚠️ Best viewed on Chromium-based browsers on Window like Chrome or Edge, as others may not work correctly.

[serenity81]

yes you are correct about aligning vs syncing in current state. perfect let me know when you have something ready, appreciated.

[Danielku15]

@serenity81 A first step towards your needs is done. I merged today a first version for supporting the audio tracks in guitar pro 8 files. Under the hood it extends the data model to hold information about a backing track and the related sync points. Then <audio /> tag is used for playback synchornizing the cursor.

Its still a bit jumpy and might have some hickups but the base is working and generally working fine.

https://demo.alphatab.net/audio-track

With the same foundation I'll also add a mechanism to allow controlling the media part outside (e.g. via youtube video). Here already an early demo wher most bits are working (play/pause sync, playback speed, seeking):

https://demo.alphatab.net/youtube/#

[serenity81]

Great news.

1. Is there a way to upload an audio file to test?
   What are next steps?

2. I also see you added the audio device output selection functionality, Nice!! Is there a way you think in the future you can add it per track similar to how I do it for audio files on gigmixe.com ? (I currently dont have it per track for music score tracks)

[Danielku15]

1. Is there a way to upload an audio file to test?
   What are next steps?

The progress can be followed here: #1961 and the linked sub-issues. There will eventually be documentation around it. For now to fiddle around some early infos:

• alphaTab supports already using Guitar Pro 8 files with audio tracks. If you own guitar pro 8 you can just attach the audio track there, sync it, and then load it in alphaTab.
• For external audio sync (e.g. youtube) there is already a small sample in the playground: https://github.com/CoderLine/alphaTab/blob/develop/playground-template/youtube.html
• I'm currently working on an online playground for the website providing advanced capabilities of fiddling with alphaTab features. This playground will then receive an "editor" for syncing alphaTab with an audio file.
• Internally the data model is organized like this:
  • Score has a .backingTrack property which defines the general audio data and shift of the audio track. This should be the bytes of the file which can be played by an audio tag (e.g. ogg, mp3. Not raw PCM samples).
  • On master bars you can add sync points.
  • A sync point is an automation (placed at a relative position of a masterbar) defining:
    • On which repeat the sync point is valid (for handling repeats)
    • Which alternative tempo the song should have to align with the audio.
    • At which millisecond position the audio is played.

Defining sync points manually can be tricky, that's why I'm building an editor for it. While building this editor I'll also fine-tune the public API and data model a bit to make things easier to handle. For now you could modify any song during the scoreLoaded event.

2. I also see you added the audio device output selection functionality, Nice!! Is there a way you think in the future you can add it per track similar to how I do it for audio files on gigmixe.com ? (I currently dont have it per track for music score tracks)

This feature was added in 1.5.0: https://www.alphatab.net/docs/releases/release1_5#player-allow-changing-of-output-device

It's unlikely alphaTab will get such a multi-track output switching built-in. Its a too specific usecase and implies some challenges to solve. The synth mixes the audio samples together (according to aspects like configured volume pan etc.) and the final audio samples are buffered and sent to an AudioContext. The output device is set on the audio context. Handling every track as own audio stream, and configuring devices individually requires special synchronization of the different AudioContext

But I plan to ship features which allow you to build something on your side. 😉 The #1962 will allow you to "pull" synthesized audio according to custom configurations. You'' be able to setup multiple "audio generators" and set volumes of the tracks accordingly. Then you can pull the samples for the audio tracks and mix it together to your preference. e.g. setup multiple AudioContext and use the raw samples, send it to <audio> elements (via OGG and blob URI), however you need to handle it.

Then you would use the external cursor API to place the cursor (like the youtube video does it).

[serenity81]

That is all very promising impressive work

[Danielku15]

@serenity81 The audio export is also done now.

#2160

https://next.alphatab.net/docs/guides/audio-export

With this you might be able to build your app in the way you want.

[serenity81]

Thanks for the update :)

…

On Sun, Jun 15, 2025 at 2:47 AM Daniel Kuschny ***@***.***> wrote: @serenity81 <https://github.com/serenity81> The audio export is also done now. #2160 <#2160> https://next.alphatab.net/docs/guides/audio-export With this you might be able to build your app in the way you want. — Reply to this email directly, view it on GitHub <#1935 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AQT4GPV7FF25ATXSXR5MC4T3DSYA3AVCNFSM6AAAAABYSX6PTKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTGNBXGE3TOMI> . You are receiving this because you were mentioned.Message ID: ***@***.***>