How to Fine Tuning Hugging Faces Transformers in Python for NLP Classification

Learn how to fine-tune the transformer models  Roberta, Electra, BERT, T5, and Alberta in Python in this free beginner intro to fine-tuning with hugging faces with a simple NLP classification problem. In this free Python educational lesson, we will use transformers' Auto Model class to easily import each different model with similar Python code.  This pairs nicely with the Auto Tokenizer which helps ensure our texts are tokenized in the appropriate format for each model without having to import a separate tokenizer or model function.  

Using the Auto Model and Auto Tokenizer makes setting up experiments in with transforms so much faster and easier.

Here we fine-tune these various sequence classification models on a New Articles classification problem with 4 classes.  To show the effect of fine tuning with will look at the model's accuracy and confusion matrix first without fine tuning and then again after we have completed fine tuning.  In these free Python lessons, we will be training these models for only 3 epochs and it's amazing the effect of such a short training time on the final accuracy. 

A Challenging part of fine-tuning a hugging faces model with the size of these models makes them hard to even fit in your GPU's memory for speedy training.  DistilBert and Electra are smaller models and easier to train on a normal GPU but I had to use Google Colabratory's most expensive GPU, the A100 GPU, just to have a large enough memory to hold the model for training.   Even using the best GPU training some off these transformers for only 3 epochs took many hours.  In total to fine tune 6 models for the Free educational Python Project had a compute cost of $10 USD, generously donated by DataSimple.education.

Join us and learn how to fine-tune the powerful transformers available in the Hugging Faces library in the free educational Python class.