Customer-Support Chatbot on a 4 GB GPU

The problem

Build a customer-support assistant that answers common queries in a natural conversational register — on the hardware actually available, which was a single RTX 3050 with 4 GB of VRAM.

That constraint decides the whole project. Most of the interesting conversational models of 2025 do not fit in 4 GB at any batch size. The question stopped being "which model is best" and became "what is the best model that fits, and how much can I get out of it."

Approach

Model choice

DialoGPT-small — a 117M-parameter GPT-2 variant already pretrained on conversational data. Small enough to fine-tune in the memory available, and already carrying the shape of dialogue rather than plain prose, so the fine-tuning budget went toward the support domain instead of teaching the model to hold a conversation at all.

Building the corpus

The source was the Twitter Customer Support dataset (TWCS) — over 3 million tweets of real exchanges between customers and company support accounts.

Raw tweets are not training data. TWCS is a flat table of tweets with metadata — timestamps, author IDs, tweet IDs, reply references — and no explicit notion of a conversation. Getting usable pairs out of it took a specific sequence:

1. Drop empty text. Filter to tweets with a non-empty text field.
2. Restore chronology. Parse created_at into a datetime and sort, because support threads develop over hours or days and the raw ordering does not reflect that.
3. Separate the speakers. The inbound flag marks whether a tweet came from a customer (True) or a support agent (False).
4. Rebuild the exchange. Match each customer tweet to the agent tweet that replied to it via response_tweet_id, giving a customer message as the prompt and the agent's reply as the response.
5. Discard the incomplete. Any pair whose response was missing from the dataset, or whose thread was broken, was thrown out rather than padded or guessed.

That produced roughly 20,000 single-turn prompt–response pairs, all English, from a single brand — Sprint's support account.

I profiled the corpus by token length and vocabulary before training, and the profile turned out to matter. Support agents on Twitter are not trying to solve the problem in the tweet — they are trying to move the customer into a private channel. "Please DM us" and its variants are among the most frequent constructions in the responses, alongside "please" and "help". That is a property of the medium, not of the dataset, and it puts a hard ceiling on what a model trained on it can learn to do. It learns the register of support. It does not learn to resolve anything, because the training data mostly does not resolve anything either.

Training under the ceiling

• Mixed precision (fp16) — roughly halves activation memory
• Batch size 2, with a maximum sequence length of 64 tokens — the two levers that actually made it fit
• AdamW, Hugging Face Trainer API
• 1–2 epochs, at roughly 25–30 minutes per epoch on the RTX 3050

Training loss converged from 4.6 to 3.3 with no signs of overfitting, memory use stayed flat, and there were no gradient problems. For a dataset and model of this size, converging quickly is expected rather than impressive — worth saying so.

What it couldn't do

I profiled and documented the model's limitations, and I would rather lead with them than bury them.

No memory past one turn. The corpus is single-turn pairs, so the model has no conversational state. Ask a follow-up that depends on what you just said and it will not connect them. This rules out exactly the case support exists for — a problem that takes several messages to resolve.

Single-brand bias. The training data came from one company's support account. The model's register, vocabulary, and its assumptions about what a "product" even is are shaped by that one brand. Deployed anywhere else it would sound subtly wrong, and would confidently reproduce another company's policies.

No personalisation. Every query is handled in isolation. There is no user history, so the model cannot adapt to who it is talking to — which is table stakes in current support tooling.

No factual grounding. Nothing connects the model to a knowledge base or product catalogue. It generates plausible support language, which means it will state something false with exactly the same fluency as something true. For production use this is the blocker, and it is not fixable by more fine-tuning.

What a production version would need

Three changes, in order of how much they matter:

Retrieval, not memorisation. The factual-grounding problem is the one that makes this a prototype. A production version retrieves from real product documentation and conditions the response on what it found, so the model is choosing how to say something rather than inventing what to say.

Multi-turn context. Training on reconstructed threads rather than single-turn pairs, with a context window that carries the conversation so far. This is a data problem more than a modelling one — the threads are in TWCS, I just did not use them.

A confidence threshold and a handoff. A support bot that does not know when it is out of its depth is worse than no bot. Anything below a confidence threshold, or anything touching billing, accounts or complaints, goes to a human with the transcript attached.

Why this project is worth reading

The loss curve is unremarkable. What the project demonstrates is working inside a hard constraint and being honest about the result: a 117M model trained in 4 GB is a real engineering exercise, and knowing precisely why the output is not deployable is a more useful skill than getting a bigger number on a bigger GPU.

Stack

PyTorch · Hugging Face Transformers (Trainer API) · DialoGPT-small · Python