Skip to content

add tagset support for select-by-tag #86

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

add tagset support for select-by-tag #86

wants to merge 3 commits into from

Conversation

@jperez999
Copy link
Collaborator

@jperez999 jperez999 commented May 6, 2022

No description provided.

@jperez999 jperez999 self-assigned this May 6, 2022
@jperez999 jperez999 added the enhancement New feature or request label May 6, 2022
@jperez999 jperez999 added this to the Merlin 22.05 milestone May 6, 2022
@jperez999 jperez999 linked an issue May 6, 2022 that may be closed by this pull request
[FEA] select_by_tag by should accept a TagSet as a param #85
Open
2 tasks
@jperez999 jperez999 requested a review from karlhigley May 6, 2022 16:19
@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented May 6, 2022

Click to view CI Results
GitHub pull request #86 of commit 5ab18313ac06faa3f88773f50a8228488bb406c6, no merge conflicts.
Running as SYSTEM
Setting status of 5ab18313ac06faa3f88773f50a8228488bb406c6 to PENDING with url https://10.20.13.93:8080/job/merlin_core/48/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_core
using credential ce87ff3c-94f0-400a-8303-cb4acb4918b5
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/core # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/core
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems username and pass
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/core +refs/pull/86/*:refs/remotes/origin/pr/86/* # timeout=10
 > git rev-parse 5ab18313ac06faa3f88773f50a8228488bb406c6^{commit} # timeout=10
Checking out Revision 5ab18313ac06faa3f88773f50a8228488bb406c6 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 5ab18313ac06faa3f88773f50a8228488bb406c6 # timeout=10
Commit message: "add tagset support for select-by-tag"
 > git rev-list --no-walk cb5157148aa2ec6872698ede29cd960cc9f44aae # timeout=10
[merlin_core] $ /bin/bash /tmp/jenkins5337440993692448990.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (61.0.0)
Collecting setuptools
  Downloading setuptools-62.1.0-py3-none-any.whl (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 17.4 MB/s eta 0:00:00
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 61.0.0
    Uninstalling setuptools-61.0.0:
      Successfully uninstalled setuptools-61.0.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-auth 1.35.0 requires cachetools<5.0,>=2.0.0, but you have cachetools 5.0.0 which is incompatible.
tensorflow-gpu 2.8.0 requires keras<2.9,>=2.8.0rc0, but you have keras 2.6.0 which is incompatible.
tensorflow-gpu 2.8.0 requires tensorboard<2.9,>=2.8, but you have tensorboard 2.6.0 which is incompatible.
Successfully installed setuptools-62.1.0
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_core/core, configfile: pyproject.toml
plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 342 items / 1 skipped

tests/unit/core/test_dispatch.py ..                                      [  0%]

tests/unit/dag/test_base_operator.py ....                                [  1%]

tests/unit/dag/test_column_selector.py ..........................        [  9%]

tests/unit/dag/test_tags.py ......                                       [ 11%]

tests/unit/dag/ops/test_selection.py ...                                 [ 11%]

tests/unit/io/test_io.py ............................................... [ 25%]

................................................................         [ 44%]

tests/unit/schema/test_column_schemas.py ............................... [ 53%]

........................................................................ [ 74%]

.......................................................................  [ 95%]

tests/unit/schema/test_schema.py ......                                  [ 97%]

tests/unit/schema/test_schema_io.py ..                                   [ 97%]

tests/unit/utils/test_utils.py ........                                  [100%]


=============================== warnings summary ===============================

tests/unit/dag/test_base_operator.py: 4 warnings

tests/unit/io/test_io.py: 72 warnings

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/io/test_io.py::test_validate_and_regenerate_dataset

/var/jenkins_home/workspace/merlin_core/core/merlin/io/parquet.py:551: DeprecationWarning: 'ParquetDataset.pieces' attribute is deprecated as of pyarrow 5.0.0 and will be removed in a future version. Specify 'use_legacy_dataset=False' while constructing the ParquetDataset, and then use the '.fragments' attribute instead.

paths = [p.path for p in pa_dataset.pieces]


tests/unit/utils/test_utils.py::test_nvt_distributed[True-True]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 39285 instead

warnings.warn(


tests/unit/utils/test_utils.py::test_nvt_distributed[True-False]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 32909 instead

warnings.warn(


tests/unit/utils/test_utils.py::test_nvt_distributed[False-True]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 33219 instead

warnings.warn(


tests/unit/utils/test_utils.py::test_nvt_distributed[False-False]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 45695 instead

warnings.warn(


tests/unit/utils/test_utils.py::test_nvt_distributed_force[True]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 36981 instead

warnings.warn(


tests/unit/utils/test_utils.py::test_nvt_distributed_force[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 41083 instead

warnings.warn(


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

================= 342 passed, 1 skipped, 83 warnings in 52.24s =================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/core/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_core] $ /bin/bash /tmp/jenkins2340693349220524093.sh

@github-actions
Copy link

github-actions bot commented May 6, 2022

Documentation preview

https://nvidia-merlin.github.io/core/review/pr-86

@karlhigley
Copy link
Contributor

karlhigley commented May 6, 2022

Looks good, needs a test

@karlhigley karlhigley modified the milestones: Merlin 22.05, Merlin 22.06 May 9, 2022
karlhigley
karlhigley suggested changes May 10, 2022
View reviewed changes
Copy link
Contributor

@karlhigley karlhigley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a test

@nvidia-merlin-bot
Copy link

nvidia-merlin-bot commented Aug 15, 2022

Click to view CI Results
GitHub pull request #86 of commit 4ee61b6db84b1e38cf51e02e00fb36a615b6980a, no merge conflicts.
Running as SYSTEM
Setting status of 4ee61b6db84b1e38cf51e02e00fb36a615b6980a to PENDING with url https://10.20.13.93:8080/job/merlin_core/105/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_core
using credential ce87ff3c-94f0-400a-8303-cb4acb4918b5
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/core # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/core
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems username and pass
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/core +refs/pull/86/*:refs/remotes/origin/pr/86/* # timeout=10
 > git rev-parse 4ee61b6db84b1e38cf51e02e00fb36a615b6980a^{commit} # timeout=10
Checking out Revision 4ee61b6db84b1e38cf51e02e00fb36a615b6980a (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 4ee61b6db84b1e38cf51e02e00fb36a615b6980a # timeout=10
Commit message: "Merge branch 'main' into fix-select-tag"
 > git rev-list --no-walk 5498b20bf5a9ce69f58777d7e5abefa2a7707c7c # timeout=10
[merlin_core] $ /bin/bash /tmp/jenkins13653228622583212179.sh
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_core/core, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 343 items / 1 skipped

tests/unit/core/test_dispatch.py ..                                      [  0%]

tests/unit/dag/test_base_operator.py ....                                [  1%]

tests/unit/dag/test_column_selector.py ..........................        [  9%]

tests/unit/dag/test_graph.py .                                           [  9%]

tests/unit/dag/test_tags.py ......                                       [ 11%]

tests/unit/dag/ops/test_selection.py ...                                 [ 12%]

tests/unit/io/test_io.py ............................................... [ 25%]

................................................................         [ 44%]

tests/unit/schema/test_column_schemas.py ............................... [ 53%]

........................................................................ [ 74%]

.......................................................................  [ 95%]

tests/unit/schema/test_schema.py ......                                  [ 97%]

tests/unit/schema/test_schema_io.py ..                                   [ 97%]

tests/unit/utils/test_utils.py ........                                  [100%]


=============================== warnings summary ===============================

tests/unit/dag/test_base_operator.py: 4 warnings

tests/unit/io/test_io.py: 71 warnings

/usr/local/lib/python3.8/dist-packages/cudf/core/frame.py:384: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/io/test_io.py::test_validate_and_regenerate_dataset

/var/jenkins_home/workspace/merlin_core/core/merlin/io/parquet.py:551: DeprecationWarning: 'ParquetDataset.pieces' attribute is deprecated as of pyarrow 5.0.0 and will be removed in a future version. Specify 'use_legacy_dataset=False' while constructing the ParquetDataset, and then use the '.fragments' attribute instead.

paths = [p.path for p in pa_dataset.pieces]


tests/unit/utils/test_utils.py::test_serial_context[True]

/usr/local/lib/python3.8/dist-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first

self.make_current()


tests/unit/utils/test_utils.py::test_nvt_distributed[True-True]

/usr/local/lib/python3.8/dist-packages/distributed/node.py:180: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 32893 instead

warnings.warn(


tests/unit/utils/test_utils.py::test_nvt_distributed[True-False]

/usr/local/lib/python3.8/dist-packages/distributed/node.py:180: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 45651 instead

warnings.warn(


tests/unit/utils/test_utils.py::test_nvt_distributed[False-True]

/usr/local/lib/python3.8/dist-packages/distributed/node.py:180: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 38109 instead

warnings.warn(


tests/unit/utils/test_utils.py::test_nvt_distributed[False-False]

/usr/local/lib/python3.8/dist-packages/distributed/node.py:180: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 35445 instead

warnings.warn(


tests/unit/utils/test_utils.py::test_nvt_distributed_force[True]

/usr/local/lib/python3.8/dist-packages/distributed/node.py:180: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 43763 instead

warnings.warn(


tests/unit/utils/test_utils.py::test_nvt_distributed_force[False]

/usr/local/lib/python3.8/dist-packages/distributed/node.py:180: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 40607 instead

warnings.warn(


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

================= 343 passed, 1 skipped, 83 warnings in 54.66s =================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/core/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_core] $ /bin/bash /tmp/jenkins4710867292400428964.sh

@karlhigley karlhigley modified the milestones: Merlin 22.06, Merlin 23.04 Apr 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@karlhigley karlhigley karlhigley requested changes

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

@jperez999 jperez999

Labels

enhancement New feature or request

Projects

None yet

Milestone

Merlin 23.04

Development

Successfully merging this pull request may close these issues.

[FEA] select_by_tag by should accept a TagSet as a param

3 participants

@jperez999 @nvidia-merlin-bot @karlhigley