Skip to content

Adding tutorial for data loaders in multi device setups - #110

Open
selamw1 wants to merge 16 commits into
jax-ml:mainfrom
selamw1:data_loading_tpu
Open

Adding tutorial for data loaders in multi device setups #110
selamw1 wants to merge 16 commits into
jax-ml:mainfrom
selamw1:data_loading_tpu

Conversation

@selamw1

@selamw1 selamw1 commented Nov 21, 2024

Copy link
Copy Markdown
Contributor

Introduction to Data Loaders for Multi-Device Training with JAX.

Comment thread docs/conf.py
'JAX_examples_image_segmentation.md',
'JAX_Vision_transformer.md',
'data_loaders_on_cpu_with_jax.md',
'data_loaders_on_gpu_with_jax.md',

@jakevdp jakevdp Nov 21, 2024

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll need to add the data_loaders_for_multi_device_setups_with_jax.md filename to the contents here.


+++ {"id": "PUFGZggH49zp"}

## Introduction to Data Loaders for Multi-Device Training with JAX

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First heading should be H1

## Introduction to Data Loaders for Multi-Device Training with JAX

+++ {"id": "3ia4PKEV5Dr8"}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As with the GPU tutorial, it would be useful here to give a quick summary of what differs between this tutorial and the others in the series.

@selamw1

selamw1 commented Nov 22, 2024

Copy link
Copy Markdown
Contributor Author

Headings adjusted and new overview paragraph added into the introduction.
Below files are also modified:
docs/conf.py
docs/tutorials.md

@trallard

Copy link
Copy Markdown
Contributor

@selamw1 there seem to be a couple of merge conflicts here would you be able to fix them, please?

@selamw1

selamw1 commented Nov 26, 2024

Copy link
Copy Markdown
Contributor Author

File conflict resolved.

@trallard

Copy link
Copy Markdown
Contributor

Note the docs build is failing atm - I opened a PR against this branch that should fix this selamw1#5 so needs @selamw1 to merge it

@jakevdp

jakevdp commented Dec 2, 2024

Copy link
Copy Markdown
Collaborator

Similar comment here as in #109 (comment) – this will be the third in a sequence of very similar tutorials on data loading – it's hard to know from reading this what the expected takeaway is compared to the previous two. We should think of this not as a standalone notebook, but as a third chapter in a sequence of documents that builds on the previous ones – orienting the reader around that would make this a much more effective doc I think.

@selamw1

selamw1 commented Dec 4, 2024

Copy link
Copy Markdown
Contributor Author

Similar comment here as in #109 (comment) – this will be the third in a sequence of very similar tutorials on data loading – it's hard to know from reading this what the expected takeaway is compared to the previous two. We should think of this not as a standalone notebook, but as a third chapter in a sequence of documents that builds on the previous ones – orienting the reader around that would make this a much more effective doc I think.

#109 (comment) has been adjusted by new PR. Accordingly, this notebooks have been adjusted to highlight the key differences between each other, and their respective references have been added to make them standalone tutorials.

@jakevdp

jakevdp commented Dec 4, 2024

Copy link
Copy Markdown
Collaborator

Thanks - we'll have to rebase this PR to reflect the fact that docs has moved to docs/source.

@selamw1

selamw1 commented Dec 4, 2024

Copy link
Copy Markdown
Contributor Author
  • Rebase has been done to move files from docs to docs/source.
  • Old files removed from docs.

@selamw1

selamw1 commented Dec 6, 2024

Copy link
Copy Markdown
Contributor Author

Due to file conflicts and rebasing several commits has been done.

@rcrowe-google

Copy link
Copy Markdown
Contributor

It looks like the builds are still failing on this. Could we please fix the builds and merge this PR? Currently we have broken links pointing to this tutorial, and this is a valuable tutorial for developers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants