[P] An elegant and strong PyTorch Trainer

For lightweight use, pytorch-lightning is too heavy, and its source code will be very difficult for beginners to read, at least for me.

As we know, for a deep learning engineer, a powerful trainer is a sharp weapon. When reproducing the SOTA papers, you don't have to write a lot of template code every time and can pay more attention to the model implementation itself.

I opened source some works (AAAI 21 SeqNet, ICCV 21 MAED, etc) and earned more than 500 stars. After referring to some popular projects (detectron2, pytorch-image-models, and mmcv), based on my personal development experience, I developed a SIMPLE enough, GENERIC enough, and STRONG enough PyTorch Trainer: core-pytorch-utils, also named CPU. CPU covers most details in the process of training a deep neural network, including:

• Auto logging to console and tensorboard.
• Auto checkpointing.
• Argument parser which can load a YAML configuration file.
• Make ALL PyTorch LR scheduler supporting warmup.
• Support distributed training.
• Support Automatically Mixed Precision (AMP) training.

I try to keep the project code as simple and readable as possible. So the code comments are very detailed and everyone can understand them. What's more, a good document is also available: CPU document

For deep learning green hands, you can learn how to:

• write a standard and clean training loop.
• use AMP to speed up your training.
• save checkpoint, and resume from it.
• perform more smooth, and readable logging.
• use the popular visualization library: tensorboard.

For old hands, we can talk about whether the structure of CPU is elegant and reasonable.

I have thought a lot about this framework, combining the advantages of several popular frameworks and discarding their shortcomings. Welcome to use it!