Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
532 commits
Select commit Hold shift + click to select a range
0136ba5
Deprecate value_list_observer (#408)
Mar 6, 2021
7a6d666
Get close to supporting autodep on reagent/TARGETS (#410)
Mar 8, 2021
bbb5ef1
Start integrating model manager into world model workflows
Mar 8, 2021
f6b737b
Generalized ips_use_cases.py, added some comments and printing to oth…
Mar 8, 2021
636fe6b
Fix a bunch of pyre errors
Mar 10, 2021
0085626
move root code into core/ (#411)
Mar 11, 2021
fcd2ca3
Add more metrics to report in validation for discrete crr
czxttkl Mar 11, 2021
ab9ac76
@final is only supported for python3.8 (#417)
czxttkl Mar 12, 2021
196b782
Upgrade Pyre version for `reagent`
grievejia Mar 16, 2021
06945cb
Convert MDNRNN to lightning
czxttkl Mar 16, 2021
7747b7a
suppress errors in `reagent`
Mar 16, 2021
101f0da
Polish Seq2Reward (#420)
czxttkl Mar 16, 2021
ed8c60c
Fix CEM Trainer (#424)
czxttkl Mar 18, 2021
82484f7
fix SAC reporting bug
kaiwenw Mar 19, 2021
2cf5f63
Make create_df_from_replay_buffer callable in oss by moving related f…
gji1 Mar 19, 2021
2c122b8
Add a unit test for Seq2Reward (#419)
gji1 Mar 19, 2021
485320d
Add option filter_short_sequence to seq2reward unit test (#422)
gji1 Mar 19, 2021
7cf9628
various updates to learnedvm (#426)
kaiwenw Mar 19, 2021
a8c3c94
fix OSS errors from D27034687 (#428)
kaiwenw Mar 20, 2021
ad64447
Remove/Resolve import duplicates + add `from_dict` classmethod to `Me…
Mar 23, 2021
9fc389b
move model managers out of workflow (#429)
czxttkl Mar 24, 2021
4b1d04a
Daily `arc lint --take BLACK`
Mar 24, 2021
5065a82
jit.trace seq2slate wrapper when possible (#431)
czxttkl Mar 25, 2021
3c52c26
Remove a useless field in parametric dqn trainer (#433)
czxttkl Mar 29, 2021
03a8541
Correct computation of eval model_propensities for CRR (#434)
Mar 29, 2021
43158c1
add comments for RB
kaiwenw Mar 29, 2021
0e5fc53
Remove additional layer of aggregation from logging (#435)
Mar 31, 2021
b14fd1f
Integration tests for model-based spark transform
czxttkl Apr 1, 2021
c6db9e1
Added some comments to the code (#436)
Apr 1, 2021
7c90858
Simplify CRR reporting (#440)
Apr 6, 2021
ec8dfb7
Enable pytorch lightning logging on all trainers (#439)
Apr 6, 2021
3db7b56
suppress errors in `reagent`
Apr 7, 2021
08a4234
Send tensorboard plots to fblearner (#443)
Apr 7, 2021
766cdcc
Migrate Seq2Reward to PyTorch Lightning (#438)
gji1 Apr 7, 2021
9cd616f
suppress errors in `reagent`
Apr 7, 2021
c133d7b
Move data fetcher out of workflow (#445)
Apr 7, 2021
da4a2e3
suppress errors in `reagent`
Apr 8, 2021
8f6ffe8
Correct eval_td_loss graph for DQN (#446)
Apr 8, 2021
cebd822
Simplify model manager unions (#444)
Apr 9, 2021
67434f4
Add binary-cross-entropy-with-logits loss for myopic values
Apr 12, 2021
5811ec4
suppress errors in `reagent`
Apr 14, 2021
e8a8607
Add time_line_operator to notifications, enable printing of CPE estim…
Apr 14, 2021
f5f5acb
Optimize the sparse_to_dense preprocessing logic.
Apr 14, 2021
211e9eb
Train Reels LearnedVM as a residual boost (#449)
kaiwenw Apr 14, 2021
61e4125
Towards RLwC + improvements (#437)
kaiwenw Apr 14, 2021
ebd84b3
Move batch_size of trainer_param to mini_batchsize of reader_options …
gji1 Apr 15, 2021
a137e80
Migrate Seq2Reward Compress Model to PyTorch Lightning (#448)
gji1 Apr 15, 2021
ad96a6b
Add unit tests for Seq2Reward Compress model (#453)
gji1 Apr 15, 2021
cffd418
suppress errors in `reagent`
Apr 15, 2021
54911d7
Fix FastRL
czxttkl Apr 15, 2021
db61132
Clone dqn reporter logging on tensorboard (#451)
Apr 17, 2021
470f860
inc. training time for sparse dqn test (#454)
kaiwenw Apr 17, 2021
edc1c24
Remove some pyre fixmes
Apr 17, 2021
6c551e9
Update the ExprTk library commit (#384)
ArashPartow Apr 18, 2021
d507a57
Make sure that trainer.train(batch) throws an error for Lightining tr…
alexnikulkov Apr 20, 2021
fd387a5
Fix discrete samplers (#393)
wall-ed-coder Apr 20, 2021
61157f3
Oss test fixes jjg (#455)
MisterTea Apr 21, 2021
322a542
Four fixes for RASP Tutorial (#458)
roelbertens Apr 21, 2021
acb98f8
Add model type to privacy context and enforce model type on publish. …
Apr 22, 2021
c51cfc0
Remove minibatch_size in seq2slate reward trainer (#459)
czxttkl Apr 22, 2021
1b10d49
Computing numbers needed for CFEval in validation_step (#460)
gji1 Apr 22, 2021
36a66fa
suppress errors in `reagent`
Apr 22, 2021
7a0ffb5
Move reagent/workflow/data -> reagent/data and reagent/data/fb (#450)
Apr 23, 2021
5eda5d7
Add logging of all considered parameters to hyperparam tuning script
alexnikulkov Apr 23, 2021
48ba183
move use_gpu out of model manager (#447)
Apr 23, 2021
76bc0ba
Migrate Klotski to Lightning
alexnikulkov Apr 29, 2021
5877d57
Remove model_values_for_logged_action, mc_loss and mc_plot from CFEva…
gji1 May 1, 2021
93df3be
add binary difference scorer and publisher (#462)
kaiwenw May 3, 2021
4d41324
suppress errors in `reagent`
May 3, 2021
c8719c8
Single Step Synthetic Reward
czxttkl May 4, 2021
f1e0624
suppress errors in `fbcode/reagent` - batch 1
May 4, 2021
cb4b6c1
apply upgraded black 21.4b2 formatting to fbsource
amyreese May 5, 2021
1da971b
Fix ReAgent Optimizer wrapper (#465)
gji1 May 5, 2021
02244c5
Inherit ReAgent optimizer from PyTorch optimizer (#466)
alexnikulkov May 6, 2021
e002808
Use lightning automatic schedulers (#467)
May 8, 2021
0bdb42c
fix predictor test (#469)
czxttkl May 9, 2021
08cb1a2
Force specifying rerank_topk in internal diversity predictor (#473)
czxttkl May 11, 2021
3f37727
Create `get_data_module()` on OSS WorldModelBase AND on FB FbWorldMod…
May 11, 2021
38e897e
Move reporters out of workflow (#474)
May 12, 2021
e710f7e
Add ngram synthetic reward network. (#476)
May 13, 2021
e301092
Migrate Seq2SlateReward to PyTorch Lightning (#472)
gji1 May 13, 2021
1427af6
Move SAC to lightning logging (#478)
May 14, 2021
cb2d14d
Fix distributed training for pytorch lightning models (#479)
czxttkl May 14, 2021
9b93fa8
Make synthetic reward tests stricter (#480)
czxttkl May 17, 2021
caa863e
Tune SAC and CRR Models. Initial support for batch gym training (#470)
May 18, 2021
91ad0d3
Add conv net to n-gram synthetic reward. (#477)
May 19, 2021
cbaa386
Add LSTM synthetic reward. (#481)
May 19, 2021
3249a00
add e2e synthetic reward test (#482)
czxttkl May 20, 2021
7bc46ba
Fix the flaky test: ngram_conv_net_synthetic_reward. (#483)
May 20, 2021
a8dbf6a
Enable bulk eval for synthetic reward models (#484)
czxttkl May 25, 2021
b5c6e38
Add binary_difference_scorer to discrete_dqn.py
May 26, 2021
1c330fa
Fix the error of "Expected all tensors to be on the same device". (#485)
Jun 1, 2021
8accc9d
disable self.log() commands for MDNRNNTrainer until LoggerConnector i…
colin2328 Jun 3, 2021
dce5a58
CFEvalDataModule
kittipatv Jun 7, 2021
d8bfb2c
Add batch norm and layer norm to synthetic reward network. (#486)
Jun 8, 2021
1d2c2a4
Upgrade ReAgent to use Python 3.8 (#415)
czxttkl Jun 10, 2021
1683459
Model manager for training contextual-bandit reward models
gji1 Jun 15, 2021
c1bbf57
Support spark in query_data_parametric
czxttkl Jun 15, 2021
9a5b9b3
Create `get_data_module()` on OSS ParametricDQNBase and FB Parametric…
Jun 16, 2021
c538992
Synchronize PyTorchLightning/pytorch-lightning (revision cdcc483e@mas…
Jun 18, 2021
0c7a89c
Try to fix world model simulation test and datamodule tests (#490)
czxttkl Jun 22, 2021
d470c4b
Add missing __init__.py to fix importing from synthetic reward (#496)
Jun 22, 2021
0ff3634
To add Rectified Adam Algorithm to Optimizers (#58968)
iramazanli Jun 22, 2021
57967dc
Revert D29241736: To add Rectified Adam Algorithm to Optimizers
samestep Jun 22, 2021
1e985fd
Simplify PPO (#487)
kittipatv Jun 22, 2021
e19cf31
Add discount_time_scale to SlateQTrainer
Jun 23, 2021
c387b5a
To add Nesterov Adam Algorithm to Optimizers (#59009)
iramazanli Jun 23, 2021
d5394c5
To add Rectified Adam Algorithm to Optimizers (#58968)
iramazanli Jun 24, 2021
395b079
ActorCriticDataModule (#491)
kittipatv Jun 25, 2021
18f5917
SlateQDataModule (#492)
kittipatv Jun 25, 2021
98cf8c9
Remove run_feature_identification() from ModelManager
kittipatv Jun 25, 2021
d499a21
Remove should_generate_eval_dataset() from ModelManager
kittipatv Jun 25, 2021
d99d165
Remove build_batch_preprocessor() from model managers
kittipatv Jun 25, 2021
9b802c7
Remove query_data() from ModelManager
kittipatv Jun 25, 2021
e23e20e
Remove trainer property from modelmanager
kittipatv Jun 25, 2021
bb53fb4
Explicitly passing normalization_data_map to make model manager state…
kittipatv Jun 25, 2021
2175b10
Explicitly passing normalization_data_map to build_trainer()
kittipatv Jun 25, 2021
d8b6ccc
Remove normalization_data accessor from ModelManager (#493)
kittipatv Jun 25, 2021
d6fd98f
Remove reward_options accessor
kittipatv Jun 25, 2021
190cb5a
Completely remove state from model managers (#494)
kittipatv Jun 25, 2021
e72494f
Make Sed2Reward model manager functional
kittipatv Jun 25, 2021
47a6a8b
suppress errors in `reagent`
Jun 25, 2021
3f61744
Fix test errors (#497)
kittipatv Jun 27, 2021
f11fbca
suppress errors in `fbcode/reagent` - batch 1
Jun 29, 2021
84e898e
DQNTrainerBase check input (#498)
czxttkl Jul 1, 2021
97a4422
Remove Seq2SlateDifferentiableRewardTrainer (#499)
gji1 Jul 1, 2021
e99d036
Migrate Seq2slate to PyTorch Lightning (#500)
gji1 Jul 4, 2021
7412be7
Fix validation_step in DQNTrainer
kittipatv Jul 6, 2021
e6d1e5c
suppress errors in `reagent`
Jul 7, 2021
d79d9fc
Add state_feature_config to continuous action models (#489)
kittipatv Jul 7, 2021
f4ea509
Implementation of a transformer-based return decomposition model (#502)
Jul 9, 2021
b1f52db
Daily `arc lint --take BLACK`
Jul 12, 2021
15e5c78
Migrate Seq2Slate Pairwise Attention to PyTorch Lightning (#501)
gji1 Jul 13, 2021
1f3cf07
Offlline Entropy Regularization for CRR (#503)
TengyuX Jul 13, 2021
0e3fd9b
Reducing log noise
Jul 15, 2021
cfe1de3
Nuke Trainer
kittipatv Jul 16, 2021
35da394
suppress errors in `reagent`
Jul 16, 2021
ba06d68
Fixing CEM tests (#508)
kittipatv Jul 21, 2021
f4c3062
Fix DiscreteDqnInput.from_dict (#506)
kittipatv Jul 21, 2021
cceee33
Seq2SlateDataModule
kittipatv Jul 21, 2021
41dab61
implement Max-Q learning method (#507)
Jul 21, 2021
812c415
Remove some unused imports (#511)
kittipatv Jul 21, 2021
eb70686
QR-DQN Integration Test (#510)
igfox Jul 21, 2021
b39e3e7
Fix PPOTrainer (#512)
kittipatv Jul 21, 2021
5074749
Fix CRR Training
igfox Jul 26, 2021
f9be943
Change blacklist -> blocklist and whitelist -> allowlist
igfox Jul 26, 2021
99082af
Synchronize PyTorchLightning/pytorch-lightning (revision 000fbe63@000…
Jul 27, 2021
3665680
suppress errors in `reagent`
Jul 27, 2021
5351f63
Use actual slate_size when not single_select (#513)
Jul 27, 2021
2463f06
Add RBF Kernel for diversity reranking (#514)
czxttkl Jul 27, 2021
0df116f
Migrating from fvcore to iopath
igfox Jul 29, 2021
6887798
Add unit tests for qr-dqn trainer (#515)
igfox Jul 30, 2021
0a085eb
Add unit tests for CRR trainer (#516)
igfox Jul 30, 2021
ada9470
fix another manual_backward call (#519)
kittipatv Aug 2, 2021
aefc19c
add option for configuring slate_size in multi selection (#517)
Aug 3, 2021
f28d515
Add result history to recurring training
igfox Aug 4, 2021
7d5bdbf
suppress errors in `fbcode/reagent` - batch 1
Aug 5, 2021
97e3da3
Remove loss_reporter.py (#521)
igfox Aug 5, 2021
28e07db
Add feature importance in star search publishing
czxttkl Aug 9, 2021
e3ea217
CRR with additional hyperparameter beta for KL-divergence (policy con…
TengyuX Aug 9, 2021
db010a9
Add resource requirements and repeats for star search feature importa…
czxttkl Aug 10, 2021
0948f01
ARS utility (#525)
xuruiyang Aug 12, 2021
6c7cd4f
fix random seed
xuruiyang Aug 13, 2021
8d00eb1
Add PPOTrainer Unit Test (#520)
igfox Aug 16, 2021
04fab8f
Revert D30114686: Add PPOTrainer Unit Test
igfox Aug 16, 2021
39ea5bd
Upgrade `requirements.txt` for downstream projects to latest fbcode r…
Aug 16, 2021
9b25610
Add PPOTrainer Unit Test V2 (#526)
igfox Aug 17, 2021
747803f
ID-list feature support
kittipatv Aug 19, 2021
0f3b1cd
DiscreteDqnDataModule
kittipatv Aug 19, 2021
0a387c1
SAC w/ ID-list features
kittipatv Aug 19, 2021
8bc799c
Minor fixes (#527)
kittipatv Aug 19, 2021
06991fe
Fix seq2reward test
igfox Aug 19, 2021
9190030
Enabling forcing BOXCOX preprocessing
kittipatv Aug 21, 2021
81eda73
Update ARS comments (#528)
czxttkl Aug 23, 2021
22b5d42
suppress errors in `fbcode/reagent` - batch 1
Aug 24, 2021
b4028d8
Fix action expression
kittipatv Aug 25, 2021
d3d0ef4
Use updated PyTorch Loading API (#529)
igfox Aug 27, 2021
938da1e
towards validation
kaiwenw Aug 29, 2021
a6d5394
Minor typing fixes
kittipatv Aug 30, 2021
cf72bf1
Adding transform unit tests (#532)
igfox Aug 30, 2021
cc6a4a3
Adding modulo ID-list mapping
kittipatv Aug 31, 2021
ca2dc4c
Ensure feature type override works as expected
kittipatv Aug 31, 2021
e690184
update CircleCI config (#533)
kittipatv Sep 2, 2021
d52b64e
Disable parallel policy evaluation by default (#534)
kittipatv Sep 2, 2021
7b4374d
Add max_weight parameter to CRR
igfox Sep 3, 2021
ab1ebc3
ReAgent Lite API (#531)
czxttkl Sep 3, 2021
fd32017
Read partitioned data by Koski when distributed training is turned on…
czxttkl Sep 5, 2021
e66d29a
suppress errors in `reagent`
Sep 8, 2021
a8c9b70
Fix type hint in Optimizers (#536)
czxttkl Sep 9, 2021
345be18
Add a function to convert idx to raw choices. More tests with probabi…
Sep 9, 2021
7f5dfe7
Fix data loader identity (#537)
czxttkl Sep 12, 2021
5d2f27d
Type fix for lite optimizer
czxttkl Sep 14, 2021
8b9b242
Add constructor method for nevergrad (#538)
czxttkl Sep 15, 2021
60f23d0
write test for test_MaskByPresence
alh99 Sep 17, 2021
0d2f8c7
Create a Unit Test for MapIDListFeatures (#540)
Sep 17, 2021
a94e01e
Refactor reagent lite (#539)
czxttkl Sep 20, 2021
8ae9850
Enforce that the optimizer closure is executed when `optimizer_step` …
Sep 21, 2021
5918384
Update Lightning version (#542)
Sep 21, 2021
7dc90f3
Fix preprocessor error (#541)
igfox Sep 21, 2021
fd11fe3
Create a Unit Test for FixedLengthSequences (#543)
PavlosApo Sep 21, 2021
042820a
Adding transform.StackDenseFixedSizeArray unit test (#544)
espellman Sep 22, 2021
312cf97
Share the training step output data via `ClosureResult` (#9349)
four4fish Sep 23, 2021
99e3c0d
Add unit test for FixedLengthSequenceDenseNormalization. (#545)
Sep 24, 2021
67c0a55
Unit Test for SlateView (#546)
Sep 24, 2021
e6b2e6e
Super net config sampling
Sep 27, 2021
5f0b21e
Add a unit test for OneHotActions
Sep 29, 2021
57f27db
Adds unit test for columnvector function
Sep 29, 2021
c703915
Update docstring for transforms.py
igfox Sep 30, 2021
b5afcc0
Allow obj_func be optional (#548)
czxttkl Sep 30, 2021
c41b961
Fix rasp tests (#550)
czxttkl Sep 30, 2021
0517902
Add test_gym_replay_buffer (#549)
czxttkl Sep 30, 2021
2e71682
Remove `ABC` from `LightningModule` (#9517)
Sep 30, 2021
603387e
Fix gym_cpu_unittest (#551)
czxttkl Sep 30, 2021
48a5a28
Deprecate TrainerProperties Mixin and move property definitions direc…
Oct 1, 2021
9b7281d
Fix last two circle ci tests (#552)
czxttkl Oct 1, 2021
f8bb0bf
Change clampping of probability feature preprocessing. (#553)
Oct 1, 2021
d219a0c
Change fb core types from namedtuple to dataclass (#554)
czxttkl Oct 7, 2021
46de5c3
add basic MAB classes to reagent
alexnikulkov Oct 7, 2021
bb357dc
Move ReAgent MAB from numpy to PyTorch
alexnikulkov Oct 7, 2021
34fe167
suppress errors in `reagent`
Oct 8, 2021
4808562
copy possible_action_maks from the env at each step instead of re-usi…
alexnikulkov Oct 9, 2021
b70c43e
Improve REINFORCE trainer (#558)
czxttkl Oct 11, 2021
dba2fd9
Convert possible_actions_mask to a Tensor (#556)
alexnikulkov Oct 12, 2021
4f8fe65
Fix ReAgentLightningModule (#559)
czxttkl Oct 12, 2021
2b65e91
suppress errors in `reagent`
Oct 13, 2021
1e2b265
Adding Bayesian Optimization Optimizer (#560)
PavlosApo Oct 13, 2021
4ce275b
Adding Bayesian Optimization Optimizer with ensemble of feedforward n…
PavlosApo Oct 13, 2021
57b58a8
add assertion for non-empty possible action mask (#557)
alexnikulkov Oct 14, 2021
103893c
suppress errors in `reagent`
Oct 18, 2021
471defa
Add Thompson Sampling to ReAgent MAB and refactor the UCB classes and…
alexnikulkov Oct 19, 2021
b60b23d
Add basic MAB simulation tools to ReAgent (#566)
alexnikulkov Oct 19, 2021
263a7ff
Add variance estimates to UCB
alexnikulkov Oct 19, 2021
9531e9c
Add MAB unittests to CircleCI test config (#567)
alexnikulkov Oct 19, 2021
25a2692
Add support for `len(datamodule)` (#9895)
tangbinh Oct 21, 2021
453d362
Add typing for `LightningOptimizer` (#9990)
tangbinh Oct 21, 2021
6cf1949
Fix `optimizers` overloads typing annotation (#10069)
tangbinh Oct 26, 2021
63bbb92
fix CircleCI test config for MAB (#568)
alexnikulkov Oct 27, 2021
5b09e5a
expose output layer activation in FC network and DQN (#572)
alexnikulkov Oct 28, 2021
cab64f8
use known batch size when sampling (#569)
Oct 30, 2021
e2c2674
Add support for BCE loss for reward decomposition. (#573)
Nov 4, 2021
02cfe37
add LinUCB trainer to reagent (#574)
alexnikulkov Nov 5, 2021
b1a3c17
Remove deprecated dataloader arguments in Trainer methods (#10325) (#…
edward-io Nov 10, 2021
756e441
Fix report coverage command
czxttkl Nov 11, 2021
ba25ae3
Update ReAgent docs (#577)
czxttkl Nov 11, 2021
e3ac3d2
Add info to arguments of post_episode_callback in Agent (#576)
alexnikulkov Nov 11, 2021
2e9e639
Remove deprecated accelerator pass through functions in Accelerator (…
aazzolini Nov 17, 2021
efe4784
update Java version
Nov 18, 2021
a8b947e
add optimize=False to reagent optimizer configs
Nov 18, 2021
fb793bc
remove deterministic=True from gym tests
Nov 18, 2021
a33e32b
add datamodule to trainer.test() in OSS train_eval_lightning
alexnikulkov Nov 18, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
273 changes: 224 additions & 49 deletions .circleci/config.yml

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions .codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
ignore:
# These are more experimental stuffs
- "reagent/ope/**/*"
- "reagent/training/gradient_free/**/*"
78 changes: 0 additions & 78 deletions docs/api/ml.rl.evaluation.rst

This file was deleted.

150 changes: 0 additions & 150 deletions docs/api/ml.rl.models.rst

This file was deleted.

30 changes: 0 additions & 30 deletions docs/api/ml.rl.prediction.rst

This file was deleted.

78 changes: 0 additions & 78 deletions docs/api/ml.rl.preprocessing.rst

This file was deleted.

46 changes: 0 additions & 46 deletions docs/api/ml.rl.readers.rst

This file was deleted.

Loading