Document Classifier
The Document Classifier Node is a transformer based classification model used to create predictions that are attached to documents as metadata. For example, by using a sentiment model, you can label each document as being either positive or negative in sentiment. Thanks to tight integration with the Hugging Face model hub, you can supply a model name to load it.
| Position in a Pipeline | After the PreProcessor in an indexing Pipeline or after a Retriever in a query Pipeline for classification on the fly |
| Input | Documents |
| Output | Documents |
| Classes | TransformersDocumentClassifier |
Note: The Document Classifier is different from the Query Classifier. While the Query Classifier categorizes incoming queries in order to route them to different parts of the pipeline, the Document Classifier is used to create classification labels that can be attached to retrieved documents as metadata. The Query Classifier is a decision Node while the Document Classifier is not.
Usage
To initialize the Node:
from haystack.nodes import TransformersDocumentClassifier
doc_classifier_model = 'bhadresh-savani/distilbert-base-uncased-emotion'doc_classifier = TransformersDocumentClassifier(model_name_or_path=doc_classifier_model)
Alternatively, if you can't find a classification model that has been pre-trained for your exact classification task, you can use zero-shot classification with a custom list of labels and a Natural language Inference (NLI) model as follows:
doc_classifier_model = 'cross-encoder/nli-distilroberta-base'doc_classifier = TransformersDocumentClassifier( model_name_or_path=doc_classifier_model, task="zero-shot-classification", labels=["negative", "positive"]
To use it in a Pipeline:
pipeline = Pipeline()pipeline.add_node(component=retriever, name="Retriever", inputs=["Query"])pipeline.add_node(component=doc_classifier, name='DocClassifier', inputs=['Retriever'])
It can also be run in isolation:
documents = doc_classifier.predict( documents = [doc1, doc2, doc3, ...]):