Achieving API parity between our Python and C++ frontend is crucial for broadening the number of language communities PyTorch can address. By the PyTorch 1.5 release, we will provide the following improvements to the PyTorch C++ frontend:

Python/C++ API Parity:

• torch.nn modules and functional (tracking issue: #25883). Items that will be added in 1.5:
  • RNN
  • LSTM
  • GRU
  • RNNCell
  • LSTMCell
  • GRUCell
  • PackedSequence (#33652)
  • pack_padded_sequence (#33652)
  • pad_packed_sequence (#33652)
  • pad_sequence (#33652)
  • pack_sequence (#33652)
• torch.optim optimizers (tracking issue: #28440) (Ask Vincent to prioritize the rest of optimizers)
• tensor multi-dim indexing API
  • #32841
  • #30426
  • #30427
• C++ tensor autograd API (tracking issue: #25874). Remaining items:
  • grad_fn (#28287)
  • register_hook (#28287)
  • retain_grad (#33349)
  • _base (#33316)

Bug fixes:

• Allow skipping default arguments in module's forward method when module is used in Sequential (PR: #33027)
• ModuleList compile error: error: 'begin' was not declared in this scope (Issue: #32414)
• C++ nn::FractionalMaxPool2d/3d output_ratio option is integer, should be double (Issue: #33240, PR: #33304)

Docs:

• For each torch::nn layer / functional, document how to use *Options to specify options
• Fix torch::Tensor doc generation (Issue: #25845)

Tutorials:

• How to use C++ tensor multi-dim indexing
  • Maybe a table to show the translation between two languages
• Autograd in C++
  • Basic autograd operations (https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html, https://pytorch.org/tutorials/beginner/examples_autograd/two_layer_net_autograd.html?highlight=autograd)
  • How to use C++ custom autograd function (https://pytorch.org/docs/stable/notes/extending.html#extending-torch-autograd)
  • How to use at::Tensor::register_hook
  • How to compute higher-order gradients in C++ (e.g. issue: #18173)

cc @yf225 @gchanan