-
Notifications
You must be signed in to change notification settings - Fork 5
Add Euclid MER HATS Parquet notebook #73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor comments. If you plan to push many commits here while developing, we should consider temporarily turning off execution for the rendering, too.
I'm not sure we should do the numpy uninstall trick in a notebook, it's bad enough that we have installs in there 😅 (That said, I wonder why the install command is not picking up on the numpy upgrade, after all the minimum dependency is changed due to lash/hats requirements) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, some suggestions for swapping out the pip uninstall line.
Sorry about the conf.py conflict, you may want to rebase now. |
Co-authored-by: Brigitta Sipőcz <[email protected]>
Co-authored-by: Brigitta Sipőcz <[email protected]>
Rebased and force pushed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@troyraen this notebook was a good exercise for me to learn how to access HATS-format data.
The code looks good and was easy to follow, I mostly have comments about text. Please note that my comments are coming from a POV of someone who is new to HATS, LSDB, Dask, etc. so feel free to ignore the ones you think are too obvious for an average reader of this tutorial.
Co-authored-by: Jaladh Singhal <[email protected]> Co-authored-by: Brigitta Sipőcz <[email protected]>
Please rebase for sortinh out the conflicting file. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From the discussion on my HATS notebook, I wonder if we can remove hats import from here and replace hats calls in your code with equivalent lsdb calls?
I have added several more Euclid Q1 tables (>3x more columns) to this dataset plus a couple of ancillary HATS products (nasa-fornax/fornax-demo-notebooks#416) since this notebook was first drafted. I am redrafting the notebook now and it will be quite a bit different in order to explain the full product and demonstrate things. So I will close this PR and open a new one in order to ease the review process. Thanks for your feedback everyone! |
Superseded by #108 |
This PR adds a notebook with an introduction to the HATS version of the Euclid Q1 MER catalogs that IRSA is preparing to release. The dataset is currently in a testing bucket that is available from Fornax and IPAC networks only (see nasa-fornax/fornax-demo-notebooks#394 for details).
Note: Before release, I plan to update both the dataset and the notebook to include the data from the Q1 PHZ (photo-z) catalogs along with the MER data that is already there. Many Euclid use cases will require a redshift -- making this product will give users easier access to that information because they won't have to join the tables themselves. We are interested in adding the spectroscopy catalogs as well but that may or may not happen in this first round.