If you’ve used ChatGPT for any serious tasks in your work - such as automating scripts, refactoring code, summarizing logs, or simply asking it to explain why your Docker container won’t shut down - you’ve probably run into its limitations at least once. If your session suddenly ended in the middle of a response or you received a somewhat confusing “too many requests” error, we’ll address this issue in this article.
It’s important to understand that this isn’t a bug in the strict sense, since ChatGPT and the OpenAI API do have real technical limitations - for example, regarding messages, tokens, context size, request speed, and specific tools. Let’s now break down how these limits work, why they exist, and how to deal with them if you’re using ChatGPT not just for fun, but for specific work tasks.
Why does ChatGPT have limits at all?
Without getting too deep into it, any limits are necessary to protect the service from abuse, manage load, and maintain stable performance for all its users. OpenAI is no exception, explicitly stating that even plans with virtually unlimited access are still “subject to abuse guardrails”, meaning they are subject to protective restrictions against all forms of abuse.
Just imagine if every developer on the planet started pasting full React application code, Kubernetes clusters, and log dumps into the input field hundreds of times an hour. No infrastructure could handle that, and sooner or later it would hit a wall with computational resources.
And of course, the limits are also tied to the economics of running the models, since every token requires computation, and every long response, file analysis, or processing of a large context demands resources. Therefore, limits are a perfectly legitimate and appropriate part of the service’s architecture, which needs to generate revenue somehow.
Message limits: “Too many requests, please try again later”
ChatGPT does indeed have usage limits, and they depend on the plan, the selected model, the type of task, and the service’s current operating conditions. OpenAI explicitly states that free plan users have a limited number of messages within a specific time window, while paid plans receive higher limits. For some models and plans, OpenAI publishes specific values, but these values are subject to change, so they should not be taken as a permanent technical guarantee.
For example, for the free plan, OpenAI specifies limited use of GPT-5.5 within a five-hour window.
For Plus and Go, the official documentation specifies up to 160 messages with GPT-5.5 every 3 hours, after which the chat may switch to the mini version of the model until the limit resets. Paid plans naturally have higher limits, but this still doesn’t mean there are no restrictions at all.
It’s not always clear where to check the actual message limit at any given moment. So users usually only find out about it after they’ve exceeded it, and this often happens at the worst possible time - for example, after you’ve pasted a large Bash script, a long log, or a configuration snippet and hit Enter.
If you use the OpenAI API, the situation is slightly more predictable. There, limits are described via rate limits: requests per minute, requests per day, tokens per minute, tokens per day, and images per minute. These limits depend on the organization, project, model, and your access level.
How these limits work in practice
| Usage environment | Limit type | What happens when exceeded |
|---|---|---|
| ChatGPT web | Messages per time window | A limit notification appears or the selected model becomes temporarily unavailable |
| ChatGPT tools | Separate limits for files, images, data analysis and other tools | The tool becomes temporarily unavailable |
| OpenAI API | RPM, RPD, TPM, TPD, IPM | The API returns a rate limit error, usually 429 |
| OpenAI API billing/usage | Budget and access tier limits | Requests may be rejected until the limit is increased or the plan is changed |
In other words, even if the interface seems almost limitless, there are always restrictions. It’s just that in ChatGPT, they are more often perceived as a “message limit”, while in the API, they are seen as formal limits on requests and tokens.
Prompt size
The hard limit on prompts depends on the specific model and usage environment. It is no longer accurate to rely on the old “GPT-4 32K” formula, because context windows vary significantly across modern OpenAI models. The official API documentation for individual models specifies large context windows and, separately, the maximum response size. For example, the OpenAI API documentation for GPT-5.4 specifies a 1M context window and a maximum output of 128K tokens, whereas for GPT-4.1, the context is approximately 1M tokens.
But a large context does not mean that it is always wise to send everything in a single request. For example, it would be a bad idea to paste the entire Linux man page, the full nginx.conf, a journalctldump, several Kubernetes YAML manifests, and expect a perfect response. Even if the request technically fits within the context, the quality of the analysis of such a “mishmash” can be significantly reduced due to noise.
It is important to understand that the context includes more than just your last request. It may also include system instructions, the conversation history, developer messages, tool data, and the model’s future response. Therefore, the term “context limit” refers not just to the size of a single content field.
You should select a specific model based on the current OpenAI API documentation, because model names, context sizes, and maximum output change over time.
How to estimate the request size before sending
But there is a perfectly viable workaround - counting tokens before sending. For this, OpenAI recommends using the token counting API, because local estimates via tiktoken do not always account for images, files, tool schemas, and model-specific behavior.
For example, for plain text, you can use a rough estimate where 1 token often corresponds to several text characters. Code, JSON, YAML, and logs can consume tokens less predictably, and while local estimates are certainly useful, they don’t always match the API’s actual count exactly.
Therefore, here’s a Python example using the tiktoken toolto help estimate plain text:
import tiktoken
enc = tiktoken.get_encoding("cl100k_base")
text = "your input here"
tokens = len(enc.encode(text))
print(tokens)
But if you're building a production tool, it's better to rely on the official API, especially when your request includes files, images, tools, or complex schemas. Here is a simple curl example of how to count tokens via the API without generating a response tailored to your specific task:
curl https://api.openai.com/v1/responses \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-5.4",
"input": [
{
"role": "user",
"content": [
{ "type": "text", "text": "Here is my Docker Compose file..." }
]
}
],
"max_output_tokens": 0
}'
In response, you will receive a usageblock, for example:
{
"usage": {
"input_tokens": 523,
"output_tokens": 0,
"total_tokens": 523
}
}
This is the actual token count taking into account the model and the request structure.
Token consumption
It’s worth reiterating that tokens are consumed not only by your request but also by the model’s response. That is, if the input context is already very large, there is less and less space left for the response. This is precisely why a long response may be truncated or end up shorter than you expected.
In the model API, the context window and maximum output size are specified separately. This is an important distinction, because even if a model supports a large context, it doesn’t mean it will always be able to generate an infinitely long response. Unfortunately, the maximum output is also limited.
If you’re using the API, be sure to limit the response length with output parameters and design your big data processing logic in advance. If you’re using the CLI tool, split large requests into parts and include preliminary summarization and storage of intermediate results.
Estimated token consumption depends heavily on the data type. For example, 100 lines of Bash code typically result in an average load on the context. Docker Compose files also fall in the medium-to-high consumption range, but here it all depends on their size and structure. journalctl logs most often result in fairly high token consumption, especially if they contain long stack traces. Kubernetes YAML files can also result in either moderate or high consumption, depending on the number of resources and their nesting. It’s worth noting that JSON API responses often consume more tokens than they appear to at first glance due to their structure and repetitive keys. Of course, these estimates aren’t exact figures, as their main purpose is to help you understand which data types inflate the context quota the fastest.
Rate Limits by IP, Session, or API Key
If you use any tool based on the OpenAI API, you need to be aware of rate limits. In the official OpenAI API documentation, limits are measured in RPM (requests per minute), RPD (requests per day), TPM (tokens per minute), TPD (tokens per day), and IPM (images per minute). A limit can be reached based on any of these parameters, depending on which one is exhausted first.
This means you can hit the limit not only based on the number of requests but also on the number of tokens. For example, a few very large requests can exhaust the TPM limit much faster than hundreds of small requests.
However, you shouldn’t try to circumvent the limits by changing keys, accounts, or IP addresses, as this may violate the terms of service. Additionally, the ChatGPT web interface and the OpenAI API have different limits; for example, if the limit is hit in the web version of ChatGPT, this doesn’t necessarily mean the API key is also limited, and vice versa.
Common mistakes when working with rate limits
Users often make mistakes that seem trivial at first glance but ultimately lead to limits being exhausted quickly. Among the most common mistakes are the following:
- Aggressive repeated requests without a delay;
- Using a single API key for a large number of users;
- Lack of a task queue;
- Lack of caching for repeated responses;
- Sending large logs without prior filtering;
- Resending the same request after a 429 error;
Even the simplest approach using retries with gradually increasing delays can help. Well, in production code, it’s better to use exponential backoff with jitter so that multiple processes don’t repeat requests simultaneously.
Real-world examples of request optimization
If you’re an experienced user or use ChatGPT as part of your DevOps workflow, it’s best to design its operation with limits in mind from the start. For example, when working with large logs, you shouldn’t send the full journalctl or the entire application log for the day. First, try to narrow the sample as much as possible, for example like this:
journalctl -u nginx --since “1 hour ago” --no-pager
Or like this:
tail -n 200 /var/log/nginx/error.log
And if the log is large, first filter out the specific errors:
grep -i “error” /var/log/nginx/error.log | tail -n 100
This way, the model receives less noise and performs the task more quickly, such as identifying the problem.
When working with code, of course, you shouldn’t paste the entire project; instead, it’s better to provide:
- the problematic file;
- the error message;
- the command that caused the error;
- the expected behavior;
- the actual behavior.
This is a common principle for any refactoring - it’s always better to proceed in parts, fixing one module, one service, one controller, or one Dockerfile at a time.
And while there are countless such examples, we can try to summarize them into a short list of general best practices for working with ChatGPT to avoid hitting limits or at least significantly improve your productivity:
- Check the request size before sending;
- Try to break large documents into logical parts;
- Be sure to cache frequently used prompts and responses;
- Don’t send the entire chat history if it’s not needed - work with short blocks;
- After receiving a 429 error, be sure to introduce a delay rather than retrying immediately;
- Clean up noise - old logs, irrelevant YAML blocks, repetitive stack traces, etc.;
Frequently Asked Questions (FAQ)
Why does ChatGPT cut off the response?
This is usually due to an output limit, a context limit, current restrictions of the selected model, or interface-specific features. The API for each model specifies a maximum output length, so long responses are also limited.
What should I do if I get a 429 error?
Error 429 usually means you’ve exceeded the rate limit. You need to reduce the frequency of requests, shorten the size of requests, add retries with backoff, and check the project or organization limits in the OpenAI API.
Can the limits be increased?
For the API, limits depend on the organization, project, model, and access level. In ChatGPT, limits depend on the plan and the selected model. Paid plans usually offer more capabilities, but some protective limits still remain.
How many tokens fit in a single message?
There is no single answer. The context size depends on the model and the usage environment. For the API, you need to check the current OpenAI models page, which specifies the context window and max output.
Why does the same query sometimes work and sometimes not?
The reason may be the current limit, the size of the conversation history, the selected model, tool activity, or rate limits. In ChatGPT and the API, these limits work differently.
Can I send large files to ChatGPT?
Yes, but there are separate limits for files. The official OpenAI documentation specifies limits on the size of uploaded files, the number of files, and separate caps for users and organizations. These limits differ from the standard text message limit.
Conclusion
ChatGPT’s limits aren’t technical quirks designed to make users’ lives miserable; they’re part of the system’s normal operation. They’re necessary for load control, abuse prevention, and managing computational resources. And if you understand these limits in advance, working with the system becomes much easier, resulting in fewer unexpected errors, fewer incomplete responses, and less frustration.
Treat ChatGPT as a powerful tool with real limits, not as an infinite oracle. Respect the token budget, don’t send unnecessary context, don’t spam it with repetitive requests, and don’t expect the model to parse your entire codebase, entire DevOps pipeline, and all logs from the past week in a single go.
And don’t even think about feeding it your entire Kubernetes cluster configuration without filtering it first - it won’t end well.
And if you need a VPS server for your project or to test the OpenAI API, then you’ve come to the right place:)
