How many long sequences are there?

[chathasphere]

How many sequences are greater than 512 in length? If there are a bunch, it might be worth looking into some of the "long transformer" frameworks (e.g. sparse attention, combiner, BigBird)

[chathasphere]

Consider truncating extra-long sequences from the end

[chathasphere]

This would be a useful table. Also get a sense of how many visits there are per patient, & over how many years a sequence runs