Python/C++ API Parity: torch.nn modules and functional #25883

yf225 · 2019-09-09T22:35:40Z

Currently, PyTorch C++ API is missing many torch::nn layers that are available in the Python API. As part of the Python/C++ API parity work, we would like to add the following torch::nn modules and utilities in C++ API:

Containers

Module (TODO: some APIs are missing in C++, e.g. register_forward_hook / register_forward_pre_hook)
Sequential (@ShahriarSS)
ModuleList (@ShahriarSS)
ModuleDict (@ShahriarSS #28652)
ParameterList
ParameterDict

Convolution layers

Pooling layers

Padding layers

ReflectionPad1d (@yf225 #28538)
ReflectionPad2d (@yf225 #28538)
ReplicationPad1d (@yf225 #28539)
ReplicationPad2d (@yf225 #28539)
ReplicationPad3d (@yf225 #28539)
ZeroPad2d (@yf225 #28540)
ConstantPad1d (@yf225 #28541)
ConstantPad2d (@yf225 #28541)
ConstantPad3d (@yf225 #28541)

Non-linear activations (weighted sum, nonlinearity)

Non-linear activations (other)

Softmin (@nuka137 #27459)
Softmax (@nuka137 #27446)
Softmax2d (@nuka137 #27509)
LogSoftmax (@nuka137 #27462)
AdaptiveLogSoftmaxWithLoss (@mansoorcheema #29076)

Normalization layers

Recurrent layers

RNN (We need to make sure it works inside a Sequential module)
LSTM (We need to make sure it works inside a Sequential module)
GRU (We need to make sure it works inside a Sequential module)
RNNCell (@CarMiranda) (We need to make sure it works inside a Sequential module)
LSTMCell (@CarMiranda) (We need to make sure it works inside a Sequential module)
GRUCell (@CarMiranda) (We need to make sure it works inside a Sequential module)

Transformer layers

Transformer (@anjali411)
TransformerEncoder (@anjali411)
TransformerDecoder (@anjali411)
TransformerEncoderLayer (@anjali411)
TransformerDecoderLayer (@anjali411)

Linear layers

Identity (@jon-tow #26713)
Linear (@pbelevich #27382)
Bilinear (@MJ10 #26082)
Flatten (@mrsalehi #28072)

Dropout layers

Dropout (@pbelevich #29761)
Dropout2d (@pbelevich #29761)
Dropout3d (@pbelevich #29761)
AlphaDropout (@Suyash458 #28424)
FeatureAlphaDropout (@Suyash458 #28424)

Sparse layers

Embedding (@anjali411 #26358)
EmbeddingBag (@anjali411 #26358)

Distance functions

CosineSimilarity (@jon-tow #26424)
PairwiseDistance (@jon-tow #26424)

Loss functions

Vision layers

PixelShuffle (@CarMiranda #28140)
Upsample (along with torch::nn::functional::interpolate) (@jon-tow #28413)

Utilities

torch.nn.functional

normalize (please see details in #27048) (@gtamba #27280)
gumbel_softmax (please see details in #27078) (@Naresh1318 #28121)
one_hot (please see details in #27081) (@bzinodev #27177)
pdist (please see details in #27082) (@jon-tow #27122)
grid_sample (@lsrock1 #28354)
affine_grid (please see details in #27196) (@jon-tow #27263)
gelu (@anjali411 #28433)

Implementation Notes:

For torch.nn modules, Python and C++ implementation must only differ in language-specific syntax, and their data member fields, control flow and logic must be exactly the same.
- You might see that some of the torch.nn modules call the corresponding torch.nn.functional functions. When implementing the C++ version of those modules, we should also add the corresponding torch::nn::functional functions, in order to preserve the call structure.
When you are adding a new C++ torch::nn module:
- The new torch::nn module must subclass from public Cloneable<ModuleName>. For example, class TORCH_API LinearImpl : public Cloneable<LinearImpl> (in torch/csrc/api/include/torch/nn/modules/linear.h).
- How do we decide which file to put the new C++ torch::nn module in? Answer: We should look at where the Python version of module is located and try to mirror that file structure. For example, the Python version of torch.nn.Linear lives in torch/nn/modules/linear.py, so the C++ version torch::nn::Linear should live in torch/csrc/api/include/torch/nn/modules/linear.h.
- If you add a new module header, you must also add the include statement for the new header in torch/csrc/api/include/torch/nn/modules.h.
- If you add a new module .cpp file, you must also add it into caffe2/CMakeLists.txt and tools/build_variables.py. (Please search for other modules in those files to see how this should be done.)
- You must add tests for the new module in test/cpp/api/modules.cpp. In particular, make sure the module's pretty_print is tested and it outputs the same value as the Python version.
- if the module is templatized (e.g. MaxPoolImpl), you must add explicit template instantiation in the module’s .cpp file (e.g. search for template class in torch/csrc/api/src/nn/modules/pooling.cpp to see how this is done).
When you are adding a new C++ torch::nn::functional function:
- How do we decide which file to put the new C++ torch::nn::functional function? Answer: We should look at where the corresponding torch::nn module is located. For example, torch::nn::PairwiseDistance lives in torch/csrc/api/include/torch/nn/modules/distance.h (which, following the rule above, is determined by torch/nn/modules/distance.py), so torch::nn::functional::pairwise_distance should live in torch/csrc/api/include/torch/nn/functional/distance.h.
- You must add tests for the new functional in test/cpp/api/functional.cpp.
- If you add a new header file for the functional, you must add include statement for the header file in torch/csrc/api/include/torch/nn/functional.h.
- The name of the functional must match the Python version.
When you are adding a new module/functional options:
- If the functional options only has optional arguments, the functional's function signature should have options = {}, to allow users to call the functional without passing options. And we must add a test for this.
- If the functional / module options only has one non-optional argument, we need the implicit constructor for the options that only takes the non-optional argument (i.e. /* implicit */ Options(value_type value);), so that we are able to do functional(input, options_arg_value) / auto m = Module(options_arg_value) instead of functional(input, Options(options_arg_value)) / auto m = Module(Options(options_arg_value)). And we must add a test for this. (See DropoutOptions in torch/csrc/api/include/torch/nn/options/dropout.h as example.)
- for optional Tensor argument, we should use torch::Tensor (with Tensor() as default “null" value), and don’t use c10::optional<Tensor>.
- If the options is templatized (e.g. MaxPoolOptions in torch/csrc/api/include/torch/nn/options/pooling.h), you must add explicit template instantiation in the options’ .cpp file (search for template struct in torch/csrc/api/src/nn/options/pooling.cpp to see how this is done).
Do not use torch::IntArrayRef, use std::vector<int64_t> instead.

How do I run tests?

Follow https://github.com/pytorch/pytorch#from-source to build your branch of PyTorch from source.
To test test/cpp/api/modules.cpp, run ./build/bin/test_api --gtest_filter=ModulesTest* --gtest_stack_trace_depth=10 --gmock_verbose=info.
To test test/cpp/api/functional.cpp, run ./build/bin/test_api --gtest_filter=FunctionalTest* --gtest_stack_trace_depth=10 --gmock_verbose=info.

Please ask for @yf225's review when you open a PR to add a torch::nn module from this list.

Tracking post on PyTorch forum: https://discuss.pytorch.org/t/55650

cc @yf225


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. Note: this is a WIP PR. Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. Note: this is a WIP PR. Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/) ghstack-source-id: 89882567 Pull Request resolved: #25981


        Add clip_grad_norm_ to c++ api (pytorch#25981)

Summary: Pull Request resolved: pytorch#25981 Per pytorch#25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. Note: this is a WIP PR. ghstack-source-id: 89882567 Test Plan: Added a unit test. Differential Revision: D17312367 fbshipit-source-id: e249b401002b6e01536acaa29fadd71706ce6209


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 89882567 Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 89882567 Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/) ghstack-source-id: 90026828 Pull Request resolved: #26140


        Update on "[pytorch] Add clip_grad_norm_ to c++ api"

Per #25883, we want to work towards C++/Python API parity. This diff adds `clip_grad_norm_` to the C++ API to improve parity. The implementation and tests are ported over directly from the python api (see https://pytorch.org/docs/stable/_modules/torch/nn/utils/clip_grad.html) Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Pull Request resolved: #26140 Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 90029046 ghstack-source-id: 90029046 Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        Update on "[pytorch] Add clip_grad_norm_ to c++ api"

Per #25883, we want to work towards C++/Python API parity. This diff adds `clip_grad_norm_` to the C++ API to improve parity. The implementation and tests are ported over directly from the python api (see https://pytorch.org/docs/stable/_modules/torch/nn/utils/clip_grad.html) Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Pull Request resolved: #26140 Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 90033213 ghstack-source-id: 90033213 Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        Update on "[pytorch] Add clip_grad_norm_ to c++ api"

Per #25883, we want to work towards C++/Python API parity. This diff adds `clip_grad_norm_` to the C++ API to improve parity. The implementation and tests are ported over directly from the python api (see https://pytorch.org/docs/stable/_modules/torch/nn/utils/clip_grad.html) Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Pull Request resolved: #26140 Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 90033526 ghstack-source-id: 90033526 Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        Update on "[pytorch] Add clip_grad_norm_ to c++ api"

Per #25883, we want to work towards C++/Python API parity. This diff adds `clip_grad_norm_` to the C++ API to improve parity. The implementation and tests are ported over directly from the python api (see https://pytorch.org/docs/stable/_modules/torch/nn/utils/clip_grad.html) Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Pull Request resolved: #26140 Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 90039953 ghstack-source-id: 90039953 Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        Update on "[pytorch] Add clip_grad_norm_ to c++ api"

Per #25883, we want to work towards C++/Python API parity. This diff adds `clip_grad_norm_` to the C++ API to improve parity. The implementation and tests are ported over directly from the python api (see https://pytorch.org/docs/stable/_modules/torch/nn/utils/clip_grad.html) Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Pull Request resolved: #26140 Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 90077057 ghstack-source-id: 90077057 Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        Update on "[pytorch] Add clip_grad_norm_ to c++ api"

Per #25883, we want to work towards C++/Python API parity. This diff adds `clip_grad_norm_` to the C++ API to improve parity. The implementation and tests are ported over directly from the python api (see https://pytorch.org/docs/stable/_modules/torch/nn/utils/clip_grad.html) Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Pull Request resolved: #26140 Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 90094458 ghstack-source-id: 90094458 Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        Update on "[pytorch] Add clip_grad_norm_ to c++ api"

Per #25883, we want to work towards C++/Python API parity. This diff adds `clip_grad_norm_` to the C++ API to improve parity. The implementation and tests are ported over directly from the python api (see https://pytorch.org/docs/stable/_modules/torch/nn/utils/clip_grad.html) Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Pull Request resolved: #26140 Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 90103121 ghstack-source-id: 90103121 Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        Update on "[pytorch] Add clip_grad_norm_ to c++ api"

Per #25883, we want to work towards C++/Python API parity. This diff adds `clip_grad_norm_` to the C++ API to improve parity. The implementation and tests are ported over directly from the python api (see https://pytorch.org/docs/stable/_modules/torch/nn/utils/clip_grad.html) Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Pull Request resolved: #26140 Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 90109145 ghstack-source-id: 90109145 Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        Update on "[pytorch] Add clip_grad_norm_ to c++ api"

Per #25883, we want to work towards C++/Python API parity. This diff adds `clip_grad_norm_` to the C++ API to improve parity. The implementation and tests are ported over directly from the python api (see https://pytorch.org/docs/stable/_modules/torch/nn/utils/clip_grad.html) Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)


        [pytorch][wip] Add clip_grad_norm_ to c++ api

Pull Request resolved: #26140 Per #25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 90122570 ghstack-source-id: 90122570 Differential Revision: [D17312367](https://our.internmc.facebook.com/intern/diff/D17312367/)

yf225 · 2019-11-26T19:27:55Z

Then, If Spectral_norm is open Could you tell me? If you could do that, I'd appreciate it.
Or if there's anything else I can do, I'd like to help. Please Mention me :)

@Yunseong-Jeong Yup I am working on the module hooks now and will let you know as soon as I finished it. Thanks so much for your help!


        Add C++ torch::nn::HingeEmbeddingLoss (pytorch#27101)

Summary: Adds `torch::nn::HingeEmbeddingLoss` module support for the C++ API. **Issue**: pytorch#25883 **Reviewer**: yf225 Pull Request resolved: pytorch#27101 Differential Revision: D17680489 Pulled By: yf225 fbshipit-source-id: 1f8f41775a9e1272a98232c8f899418b2b907eca


        Add C++ torch::nn::functional::pdist (pytorch#27122)

Summary: Adds `torch::nn::functional::pdist` module support for the C++ API. Issue: pytorch#25883, pytorch#27082 Reviewer: yf225 Pull Request resolved: pytorch#27122 Differential Revision: D17685823 Pulled By: yf225 fbshipit-source-id: f8ceb09635385ef2e16a002e5fc255be8eb2ebf4


        Add clip_grad_norm_ to c++ api (pytorch#26140)

Summary: Pull Request resolved: pytorch#26140 Per pytorch#25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 91334333 ghstack-source-id: 91334333 Test Plan: Added a unit test Differential Revision: D17312367 fbshipit-source-id: 753ba3a4d084d01f3cc8919da3108e67c809ad65

RithvikSharma · 2019-12-30T13:52:05Z

@yf225, I'd like to work on pack_sequence. Is it available? _

spectral_norm
remove_spectral_norm
PackedSequence
pack_padded_sequence
pad_packed_sequence
pad_sequence
pack_sequence

Thanks so much for your help!

Suyash458 · 2020-01-14T03:44:51Z

@yf225 Where should the tests for the utility functions like pad_sequence reside?

yf225 · 2020-01-17T19:04:40Z

@yf225 Where should the tests for the utility functions like pad_sequence reside?

@Suyash458 It would be awesome to put them in test/cpp/api/nn_utils.cpp. Thanks so much for your help!

yf225 · 2020-01-17T19:08:58Z

@yf225, I'd like to work on pack_sequence. Is it available? _

@RithvikSharma Thanks a lot for your interest! We are going through some internal planning at the moment, and I will update on its availability at the earliest possible :D

pandeykartikey · 2020-01-30T11:21:27Z

Hi, I would like to work on PackedSequence. If it is available. @yf225


        Add C++ torch::nn::HingeEmbeddingLoss (pytorch#27101)

Summary: Adds `torch::nn::HingeEmbeddingLoss` module support for the C++ API. **Issue**: pytorch#25883 **Reviewer**: yf225 Pull Request resolved: pytorch#27101 Differential Revision: D17680489 Pulled By: yf225 fbshipit-source-id: 1f8f41775a9e1272a98232c8f899418b2b907eca


        Add C++ torch::nn::functional::pdist (pytorch#27122)

Summary: Adds `torch::nn::functional::pdist` module support for the C++ API. Issue: pytorch#25883, pytorch#27082 Reviewer: yf225 Pull Request resolved: pytorch#27122 Differential Revision: D17685823 Pulled By: yf225 fbshipit-source-id: f8ceb09635385ef2e16a002e5fc255be8eb2ebf4


        Add clip_grad_norm_ to c++ api (pytorch#26140)

Summary: Pull Request resolved: pytorch#26140 Per pytorch#25883, we want to work towards C++/Python API parity. This diff adds clip_grad_norm_ to the c++ API to improve parity. ghstack-source-id: 91334333 ghstack-source-id: 91334333 Test Plan: Added a unit test Differential Revision: D17312367 fbshipit-source-id: 753ba3a4d084d01f3cc8919da3108e67c809ad65


        Add C++ torch::nn::CosineEmbeddingLoss (pytorch#27345)

Summary: Adds `torch::nn::CosineEmbeddingLoss` module and functional support for the C++ API. Issue: pytorch#25883 Reviewer: yf225 Pull Request resolved: pytorch#27345 Differential Revision: D17801402 Pulled By: yf225 fbshipit-source-id: 0eabe80d7d36397e6667b331c3fa2f56d7a15962


        C++ API: torch::nn::Softmax (pytorch#27446)

Summary: Add torch::nn::Softmax module support for the C++ API Related Issue: pytorch#25883 Reviewer: yf225 Pull Request resolved: pytorch#27446 Differential Revision: D17839546 Pulled By: yf225 fbshipit-source-id: 7c7fb55111b261614de7c3a75fa1019fbde93c67


        C++ API parity: SELU (pytorch#27434)

Summary: Adds `SELU` functional and module support for the C++ API. Issue: pytorch#25883 Pull Request resolved: pytorch#27434 Differential Revision: D17782762 Pulled By: yf225 fbshipit-source-id: 96c7ce84b9baf9e219a63e631929b8997ba6f3f0


        Add C++ torch::nn::functional::affine_grid (pytorch#27263)

Summary: Adds`torch::nn::functional::affine_grid` functional support for the C++ API. Issue: pytorch#25883, pytorch#27196 Reviewer: yf225 Pull Request resolved: pytorch#27263 Differential Revision: D17802350 Pulled By: yf225 fbshipit-source-id: e823ee53da4a4cc6a1650d2dfc09b0ef6a74e249


        C++ API: torch::nn::LogSoftmax (pytorch#27462)

Summary: Add torch::nn::LogSoftmax module and functional support for the C++ API. Related Issue: pytorch#25883 Reviewer: yf225 Pull Request resolved: pytorch#27462 Differential Revision: D17867121 Pulled By: yf225 fbshipit-source-id: dae8ac981c1c6ccdef013cd2d886ad4a043f6243


        C++ API: torch::nn::Softmin (pytorch#27459)

Summary: Add torch::nn::Softmin module and functional support for the C++ API. Related Issue: pytorch#25883 Reviewer: yf225 Pull Request resolved: pytorch#27459 Differential Revision: D17892852 Pulled By: yf225 fbshipit-source-id: db15b06e8ad33947e7d65995df700f5e90c3b6a8


        C++ API: torch::nn::Softmax2d (pytorch#27509)

Summary: Add torch::nn::Softmax2d module support for the C++ API. Softmax2d only supports module in Python API, so this PR adds only module support as well. This PR is WIP because it uses the function in pytorch#27446 . After pytorch#27446 is merged, I will remove WIP. Related Issue: pytorch#25883 Reviewer: yf225 Pull Request resolved: pytorch#27509 Differential Revision: D17899715 Pulled By: yf225 fbshipit-source-id: bd891bc995f5a92bf4f5405f8bf07d1bd5de2479


        C++ API parity: Unfold (pytorch#27809)

Summary: Adds `unfold` functional and module support for the C++ API. Issue: pytorch#25883 Reviewer: yf225 Pull Request resolved: pytorch#27809 Differential Revision: D17901792 Pulled By: yf225 fbshipit-source-id: ff58a1866bf240f37ebc589463c60593b8931f51


        Multi-Label Soft Margin loss (pytorch#27669)

Summary: In accordance with pytorch#25883, I added the `MultiLabelSoftMarginLoss` module and `multilabel_soft_margin_loss` functional. It looks like there isn't a C++ ATen implementation of `multilabel_soft_margin_loss`, so I translated the python version, which does not rely on a C/C++ backend either. Pull Request resolved: pytorch#27669 Differential Revision: D17907608 Pulled By: yf225 fbshipit-source-id: ccb02951e009973c2adbe604593ce929f10c39eb


        Soft Margin loss (pytorch#27660)

Summary: In accordance with pytorch#25883, I added the `SoftMarginLoss` module and `soft_margin_loss` functional. Pull Request resolved: pytorch#27660 Differential Revision: D17958325 Pulled By: yf225 fbshipit-source-id: c14422765e6e1fdabf6c9687080e6d5ff490d300


        Multi Label Margin loss (pytorch#27659)

Summary: In accordance with pytorch#25883, I added the `MultiLabelMarginLoss` module and `multilabel_margin_loss` functional. Pull Request resolved: pytorch#27659 Differential Revision: D17931905 Pulled By: yf225 fbshipit-source-id: 3642f75c79843dda55ac38de9f6f970f3e237847


        PixelShuffle module and functional (pytorch#28140)

Summary: Added `PixelShuffle` module and functional pytorch#25883 Pull Request resolved: pytorch#28140 Differential Revision: D18008474 Pulled By: yf225 fbshipit-source-id: f482495bb56998701c79a61ef065a121bf5a5154


        C++ API: torch::nn::LPPool1d (pytorch#27800)

Summary: Add torch::nn::LPPool1d module and functional support for the C++ API. Related Issue: pytorch#25883 Reviewer: yf225 Pull Request resolved: pytorch#27800 Differential Revision: D18045040 Pulled By: yf225 fbshipit-source-id: e61fefe9efec3423f7a93dd1e946f3e380122927


        Update C++ torch::nn parity table (pytorch#28419)

Summary: This PR updates `test/cpp_api_parity/parity-tracker.md` to reflect changes in pytorch#25883. Pull Request resolved: pytorch#28419 Differential Revision: D18061479 Pulled By: yf225 fbshipit-source-id: dbdc2e44e835f6125a42cf11e59723ef61903cff


        C++ parity, grid_sample functional (pytorch#28354)

Summary: pytorch#25883 I put grid_sample in vision.h with affine grid. I have a question in string argument(interpolation mode, padding mode) I reuse torch::native::detail::GridSamplerInterpolation in GridSampler.h instead of using string. It follows the way that uses reduction enum in loss functions. I am not sure this is right. yf225 Pull Request resolved: pytorch#28354 Differential Revision: D18109333 Pulled By: yf225 fbshipit-source-id: 1bf972b671b107464f73b937bbe0de76fb259fbf

yf225 added module: cpp module: nn labels Sep 9, 2019

yf225 changed the title ~~[Contributors Welcome] Python/C++ API Parity: torch.nn modules~~ Python/C++ API Parity: torch.nn modules Sep 10, 2019

rohan-varma mentioned this issue Sep 11, 2019

[pytorch][wip] Add clip_grad_norm_ to c++ api #25981

Closed

rohan-varma mentioned this issue Sep 11, 2019

Add clip_grad_norm_ to c++ api (#25981) #25982

Closed

rohan-varma mentioned this issue Sep 12, 2019

[c++ api] Add clip_grad_norm_ to c++ api #26140

Closed

fmassa added the triaged label Sep 13, 2019

This was referenced Jan 18, 2020

C++/Python API Parity: add pad_sequence #32386

Closed

C++/Python API Parity: add pad_sequence #32387

Open

yf225 mentioned this issue Feb 12, 2020

Python/C++ API parity master tracking task #33227

Open

7 of 24 tasks complete

pytorch / pytorch

Python/C++ API Parity: torch.nn modules and functional #25883

Python/C++ API Parity: torch.nn modules and functional #25883

yf225 commented Sep 9, 2019 •

edited

This comment has been minimized.

yf225 commented Nov 26, 2019

This comment has been minimized.

RithvikSharma commented Dec 30, 2019

This comment has been minimized.

Suyash458 commented Jan 14, 2020

This comment has been minimized.

yf225 commented Jan 17, 2020 •

edited

This comment has been minimized.

yf225 commented Jan 17, 2020 •

edited

This comment has been minimized.

pandeykartikey commented Jan 30, 2020

pytorch / pytorch

Join GitHub today

Python/C++ API Parity: torch.nn modules and functional #25883

Python/C++ API Parity: torch.nn modules and functional #25883

Comments

yf225 commented Sep 9, 2019 • edited

Containers

Convolution layers

Pooling layers

Padding layers

Non-linear activations (weighted sum, nonlinearity)

Non-linear activations (other)

Normalization layers

Recurrent layers

Transformer layers

Linear layers

Dropout layers

Sparse layers

Distance functions

Loss functions

Vision layers

Utilities

torch.nn.functional

Implementation Notes:

How do I run tests?

This comment has been minimized.

yf225 commented Nov 26, 2019

This comment has been minimized.

RithvikSharma commented Dec 30, 2019

This comment has been minimized.

Suyash458 commented Jan 14, 2020

This comment has been minimized.

yf225 commented Jan 17, 2020 • edited

This comment has been minimized.

yf225 commented Jan 17, 2020 • edited

This comment has been minimized.

pandeykartikey commented Jan 30, 2020

yf225 commented Sep 9, 2019 •

edited

yf225 commented Jan 17, 2020 •

edited

yf225 commented Jan 17, 2020 •

edited