Training a Sentiment Classifier with Naive Bayes and Logistic Regression in Python

This notebook demonstrates a compact sentiment-analysis workflow using a small synthetic text dataset. It compares multinomial naive Bayes and logistic regression with TF-IDF feature extraction, a fixed stratified split, and a final holdout evaluation on untouched examples.

What the notebook evaluates

The dataset contains short text examples labeled as negative, neutral, or positive. The workflow follows the standard text-classification pattern:

1. split the data before fitting,
2. convert text to TF-IDF features,
3. train both models on the training rows only,
4. compare macro F1 on the untouched holdout.

The code also saves the figure assets used in the article so the narrative and results stay synchronized.

Data and preprocessing

The notebook creates a tiny sentiment fixture with phrases such as “excellent service”, “support was unhelpful”, and “the package arrived today”. It uses a 70/30 stratified split so each class is represented in both partitions.

train_df, test_df = train_test_split(
    data,
    test_size=0.30,
    stratify=data["sentiment"],
    random_state=RANDOM_SEED,
)

The text-length distribution is also inspected before modeling to check whether the classes are distorted by unusually short or long phrases.

Model training

Both models use the same TF-IDF setup and are fitted only on training data:

logistic_pipeline = Pipeline([
    ("tfidf", TfidfVectorizer(ngram_range=(1, 2), min_df=1)),
    ("classifier", LogisticRegression(max_iter=1000, random_state=RANDOM_SEED)),
])

bayes_pipeline = Pipeline([
    ("tfidf", TfidfVectorizer(ngram_range=(1, 2), min_df=1)),
    ("classifier", MultinomialNB(alpha=0.5)),
])

The same evaluation helper is used for both models, and the holdout predictions are scored with macro F1 so each class is weighted equally.

Verified holdout results

The notebook was executed and the result was checked from the model output. On the untouched holdout of 27 examples (9 per class), both classifiers achieved a macro F1 of 1.00.

This is the exact verified result:

• Logistic regression: macro F1 = 1.00
• Multinomial naive Bayes: macro F1 = 1.00

Interpretation

For this synthetic example, both algorithms separate all three classes perfectly. That is useful for illustrating the workflow, but it is not evidence that the same performance will hold on noisy real-world product reviews, slang-heavy social posts, or multilingual customer feedback.

This example is best understood as a beginner-friendly demonstration of text preprocessing, vectorization, holdout evaluation, and model comparison.

Limitations

This is a small synthetic fixture designed for teaching, not a production sentiment model. Real-world sentiment data often includes slang, misspellings, sarcasm, negation, mixed-language text, and class imbalance. A deployment-ready pipeline would also need larger labeled data, error analysis, and validation on a real review or customer-support corpus.

The code is intentionally simple and deterministic so the notebooks, figures, and article text remain reproducible and easy to follow.