Explore CrossEncoders

[thefirebanks]

As specified in this issue: huggingface/sentence-transformers#350 (comment)

The authors of SBERT recommend using CrossEncoders for sentence classification.

[thefirebanks]

Explore CrossEncoders

[thefirebanks]

Relevant links:

• CrossEncoder folders: https://github.com/UKPLab/sentence-transformers/tree/master/sentence_transformers/cross_encoder
• CrossEncoder usage: https://github.com/UKPLab/sentence-transformers/blob/master/examples/applications/cross-encoder/cross-encoder_usage.py
• Main threads explaining how/why use CrossEncoders for our example:
  • https://www.sbert.net/examples/applications/cross-encoder/README.html
  • Fine Tuning Models huggingface/sentence-transformers#350
• Pretrained CrossEncoders: https://www.sbert.net/docs/pretrained_cross-encoders.html
• CrossEncoders and fine-tuning?: Using cross-encoders: fine-tuning required? huggingface/sentence-transformers#615
• Retrieve and rerank pipeline: https://www.sbert.net/examples/applications/retrieve_rerank/README.html
• Advantages of SBERT over BERT:
  • Questions about the inference process huggingface/sentence-transformers#354
  • Why are just the embeddings used for inference and not the finetuned model itself? huggingface/sentence-transformers#540
• Augmented SBERT:
  • Paper: https://arxiv.org/pdf/2010.08240.pdf
  • Clarifications on steps: Augmented SBERT training huggingface/sentence-transformers#553
• Optional: Batch hard triplet loss: https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/other/training_batch_hard_trec.py