AuDoLab.subclasses.tf_idf¶
- class AuDoLab.subclasses.tf_idf.Tf_idf[source]¶
Bases:
object- static tfidf(data, papers, data_column='lemma', papers_column='lemma', features=None, ngrams=2)[source]¶
Creates tf-idf objects for one-class SVM classification.
The tf-idf scores are calculated over a joint corpus, however the target data and the out-of-domain training data are stored in seperate, as the one-class SVM is only trained on the tf-idf scores of the out-of-domain training data.
- Args:
data (DataFrame): preprocessed target documents
papers (DataFrame): preprocessed out-of-domain training data
- data_colum (String): name of columnin target dataframe where
lemmatized documents are stored. Defaults to ‘lemma’
- papers_colum (String): name of column in out-of-domain training
dataframe where lemmatized documents are stored. Defaults to ‘lemma’
- ngrams (int, optional): whether ngram are formed.
Defaults to 2.
- features (int, optional): number of max features.
Defaults to 8000.
- Returns:
- data and papers: tfidf object data for target data and
out-of-domain training data