Issue 2338 #1
Open
Issue 2338 #1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix to "perhaps there is a misprint at line 40 pytorch#2111"; review of referenced paper https://arxiv.org/pdf/1706.03762.pdf section 3.2.3 suggests: "Similarly, self-attention layers in the decoder allow each position in the decoder to attend to all positions in the decoder up to and including that position. We need to prevent leftward information flow in the decoder to preserve the auto-regressive property. We implement this inside of scaled dot-product attention by masking out (setting to −∞) all values in the input of the softmax which correspond to illegal connections. See Figure 2." Thus the suggested change in reference from nn.Transform.Encoder to nn.Transform.Decoder seems reasonable.
As per suggestion in pytorch#1114
Fix for pytorch#1781 Rather than manually update the version number with the current stable version (e.g., 2.0.1), as long as ONNX maintains compatibility with the lastest stable version that reference should be sufficient and constantly up to date.
* Update nn_tutorial.py Fix to pytorch#1303 "add pyplot.show() in beginner tutorial." Comments to issue suggested manually commenting out pyplot.show for users not using colab. --------- Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
* refactored train loop in trainingyt.py, resolves issue pytorch#2230 * Simplified numpy function call, resolves issue pytorch#1038
* Added matplotlib dependency to blitz tutorial. * Removed a modified file from pull request --------- Co-authored-by: Carl Parker <carljparker@meta.com>
* removed ### lines and numbered in headlines * removed numbered from titles * added blank lines to show code * Remove the empty TODO placeholder --------- Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
…tion Finetuning Tutorial" (pytorch#2378)
Co-authored-by: Carl Parker <carljparker@meta.com>
Co-authored-by: sekyondaMeta <127536312+sekyondaMeta@users.noreply.github.com>
* Add temporary fix for embeddings bug Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
…rch#2401) Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
* Updates pytorch#836 as suggested in pytorch/pytorch#16885 (comment)
* address bug; do a little editing Signed-off-by: Mike Brown <brownwm@us.ibm.com> * Update intermediate_source/char_rnn_classification_tutorial.py Signed-off-by: Mike Brown <brownwm@us.ibm.com> Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
…ansforms.Normalize (pytorch#2405) * Fixes pytorch#2083 - explain model.eval, torch.no_grad * set norm to mean & std of CIFAR10(pytorch#1818) --------- Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
… Tutorial, resolves issue pytorch#2332 (pytorch#2404)
Co-authored-by: noqqaqq <noqqaqq@users.noreply.github.com> Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
…orch#1971 (pytorch#2403) Co-authored-by: Carl Parker <carljparker@meta.com>
…n_tutorial.py (pytorch#2380) * changed the loss init to make it less confusing --------- Co-authored-by: Nicolas Hug <contact@nicolas-hug.com> Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
…h#2402) * Update transformer_tutorial.py Add description for positional encoding calculation for Transformers * Update Positional Encoding description in transformer_tutorial.py * Update transformer_tutorial.py --------- Co-authored-by: Carl Parker <carljparker@meta.com>
In the function demo_model_parallel, dev0 and dev1 are computed in a way that assigns two distinct GPUs to each process. This is achieved by doubling the rank and applying modulus operation with twice the world_size. Assuming 8 gpus world_size is set to 4, leading to the creation of 4 processes. Each of these processes is allocated two distinct GPUs. For instance, the first process (process 0) is assigned GPUs 0 and 1, the second process (process 1) is assigned GPUs 2 and 3, and so forth.
* Update captum dependencies (matplotlib and flask-compress) * Use resnet18 due to RAM limitation Google Colab crashes due to insufficient RAM (more than 12 GB is required) if resnet101 or resnet50 are used. Thus, resnet18 is used instead (approximately 6 GB is used).
Remove `global_rng` and use `torch.randint` to feel the tensor of shape `shape` with values in range `[0, vocab_size)` Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
I noticed when reading through these docs that the two examples did not use the parameter 'y'. I assume it was meant to be used so I updated the code in the examples. Another possibility is that we don't need param 'y' and only need 'x'. Let me know if that is the case and I will fix this :)
* Update mario_rl_tutorial.py Fixes pytorch#1620 --------- Co-authored-by: Vincent Moens <vincentmoens@gmail.com> Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
Update the github username of an author
To make tutorial builds predictable, but still keep randomness when one rans it on Collab. Also, reset default_device after every tutorial runCo-authored-by: Nikita Shulga <nshulga@meta.com> Co-authored-by: Nikita Shulga <nshulga@meta.com>
* created original copy of the model by loading from disk * Update fx_graph_mode_ptq_dynamic.py --------- Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
"evaluating and training ResNet-18 on random data" --> "evaluating and training a ``torchvision`` model on random data", since speedups are no longer demonstrated on resnet18.
Fixes pytorch#1642 Signed-off-by: BJ Hargrave <hargrave@us.ibm.com> Co-authored-by: sekyondaMeta <127536312+sekyondaMeta@users.noreply.github.com>
We also fix the code to use the scripted_cell just created. Fixes pytorch#1449 Signed-off-by: BJ Hargrave <hargrave@us.ibm.com>
Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
Co-authored-by: NM512 <morihira3513@gmailcom> Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
…#2452) * replace old decoder diagram with new one * remove 1 from encoder1 and decoder1 * fix attention in AttnDecoderRNN * Fix formatting going over max character count --------- Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
* Image prediction using trained model * Inference on custom images * Updated the PR following the PEP8 guidelines and made the requested changes --------- Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
Fixes: pytorch#800 Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
* add quantization 2.0 document --------- Co-authored-by: Svetlana Karslioglu <svekars@fb.com>
Signed-off-by: Onur Berk Töre <onurberk_t@hotmail.com>
Signed-off-by: Onur Berk Töre <onurberk_t@hotmail.com>
Signed-off-by: Onur Berk Töre <onurberk_t@hotmail.com>
Signed-off-by: Onur Berk Töre <onurberk_t@hotmail.com>
Signed-off-by: Onur Berk Töre <onurberk_t@hotmail.com>
Signed-off-by: Onur Berk Töre <onurberk_t@hotmail.com>
Signed-off-by: Onur Berk Töre <onurberk_t@hotmail.com>
Signed-off-by: Onur Berk Töre <onurberk_t@hotmail.com>
Signed-off-by: Onur Berk Töre <onurberk_t@hotmail.com>
Signed-off-by: Onur Berk Töre <onurberk_t@hotmail.com>
Signed-off-by: Onur Berk Töre <onurberk_t@hotmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #ISSUE_NUMBER
Description
Checklist