Handling Telegram Bot Updates Concurrently

Learn how to efficiently manage Telegram bot updates when using webhooks with ASP.NET Core and the Telegram.Bot library. Whether you're developing a simple bot or a complex application, this guide will help you optimize your bot's performance and responsiveness using modern .NET techniques.

Handling Telegram Bot Updates Concurrently
Before we dive in, it's important to note that this post assumes you have a working knowledge of C# and have previously created a functional Telegram bot using the Telegram.Bot library or a similar tool.

Finding Out You Have a Problem

As you likely know, when coding a Telegram bot, there are two primary methods for receiving updates:

  1. Long Polling: The client actively requests updates from the server
  2. Webhooks: Telegram pushes updates to the client when they occur

One might assume that using webhooks with ASP.NET Core would naturally take advantage of its built-in concurrency features, the idea is simple: receive an update, process it asynchronously, await any long-running operations like database queries or HTTP requests, and in the meantime, start handling other updates concurrently.

At least, that was my assumption.

And since my bots were always lightweight and responsive, I never had a reason to question it—there were no noticeable delays or lagging.

That is, until a couple of weeks ago when I integrated a self-hosted LLM into one of my bots. Suddenly, while waiting for the LLM to generate a response, I noticed that nothing else was happening in the background. Since I'm self-hosting these models, responses are naturally slower, and this revealed an unexpected reality: my bot was handling updates fully sequentially.

After some debugging and extensive searching online, I was relieved to find that my understanding of ASP.NET Core's inner workings, and async programming in .NET, was correct. The issue wasn't with my code but with how Telegram handles updates.

It turns out that Telegram sends updates via webhooks sequentially.

What does this mean? Essentially, until your bot acknowledges the current request from Telegram by responding with an HTTP Status 200, no new updates will be sent to it.

How Can We Work Around This?

Well, as always there are many options available, so let's have a look.

We could use hacky shortcuts

One of the most suggested solutions is to wrap the handler in a fire-and-forget contraption, or sometimes to use Task.Run() with the same purpose:



The code above seems like it would work, but try fetching a service from the DI container down the line and you'll find out that some scoped services have already been disposed... please, don´t.

We could do... nothing?

Maybe you don't really need to change anything. Seriously. Even with a couple hundred different behaviors coded into your telegram bot, it is most likely not a necessity, even with a fair amount of users. KISS.

No, but I need to do something!

Fair enough, let's get started.

One of Many Solutions

Keep in mind that solutions are plenty, but on this article we will focus on delegating the processing of updates to a background worker via Channels. Channels are amazing—they're easy to use, yet powerful and configurable. Plus, they're thread-safe (yay!).

Other alternatives include the usage of ConcurrentQueue<T> or something similar.

Starting Point

Let's consider the following example, a basic implementation of a bot using Telegram.Bot and ASP.NET Core Minimal APIs:



Lines 1 to 16 cover the basic configuration steps, ensuring that all necessary classes, particularly an instance of TelegramBotClient, are properly set up in the DI container.

Line 21 is where things get more interesting—we're specifying the URL for our webhook to Telegram.

Lines 24 to 32 describe the webhook endpoint, which, before responding, performs an "expensive operation".

Lines 33 to 37 contain the "expensive operation," which, in this case, is simply a placeholder that takes 5 seconds to complete.

Given the code above, we can expect that even if we send several messages in quick succession, the output will still appear as follows:

dotnet run
Update received!
Hard work done!
Update received!
Hard work done!
Update received!
Hard work done!
  

Let's Improve it!

The improved process should (theoretically) work as follows:

  1. When Telegram sends an update to our webhook endpoint, we quickly enqueue the update into the Channel.
  2. We immediately send a response back to Telegram, this ensures minimal latency and keeps the interaction swift.
  3. Meanwhile, in the background, a dedicated service reads updates from the channel and processes them asynchronously.

This approach allows us to handle updates efficiently without blocking the main thread, enabling the system to manage high volumes of incoming updates smoothly.

First, we will create a new background service. Notice the Channel<Update> injected by the DI container in line 8:



Don't forget to add the Channel<Update> to the container (lines 18 to 19):



For the sake of simplicity we will directly inject the Channel into the container, but keep in mind that on a real project you may benefit of declaring a specific class or service that contains the Channel itself.

Let's implement Task ExecuteAsync(CancellationToken stoppingToken), just in case, let's add some more logging too:



Okay, now that we have that in place, our updates should be ready to be handled concurrently, let's try:

dotnet run
Update received!
Update received!
Starting hard work...
Update received!
Hard work done!
Starting hard work...
Hard work done!
Starting hard work...
Hard work done!

Hmm, what’s going on here?

We’re definitely separating the handling of updates and the expensive operations, so why are the updates still being handled sequentially?

To simplify, we currently have one thread handling the webhook and another processing the updates, both running concurrently. However, the bottleneck is that we only have a single worker thread.

To resolve this issue, we just need to make a small change: add more worker threads. How? It’s simple — just spin up additional instances of our UpdateHandlerHostedService.

Heads up! Avoid adding too many worker threads. Experiment to find the optimal number, but remember that peak performance is often achieved with fewer threads than your CPU's total count. This is especially true when factoring in other services and processes that may be competing for CPU resources simultaneously.

As of right now, we can’t use AddHostedService<T>() to add multiple instances of our background service. For more details, check out this issue. The proposed solution, by @davidfowl himself, is to use as many AddSingleton() as needed instead. So, let’s go ahead and implement that (lines 19 to 21):



Let's check the output this time:

dotnet run
Update received! 272348172
Starting hard work... 272348172
Update received! 272348173
Starting hard work... 272348173
Update received! 272348174
Starting hard work... 272348174
Update received! 272348175
Hard work done! 272348172
Starting hard work... 272348175
Hard work done! 272348173
Hard work done! 272348174
Hard work done! 272348175

Huzzah! It works!

Wrapping Up

In this post, we've explored the challenges of handling Telegram bot updates using webhooks with ASP.NET Core, especially when facing tasks that can take time to complete. We've started by identifying the problem of sequential update processing and worked our way through potential solutions. We've discussed why a simple fire-and-forget approach might not be suitable, and then moved on to a more reliable method using Channels to delegate work to background services.

We saw how using a single worker thread still led to sequential processing and solved this by introducing multiple worker threads, allowing the bot to handle updates concurrently and more efficiently.

This is just the beginning of creating scalable and responsive Telegram bots. There’s a lot of room for further improvement, such as optimizing the number of worker threads or exploring other concurrent data structures.

Don't forget to check out the source code for the complete example.

If you have any suggestions, spot any errors, or simply want to share your thoughts, feel free to let me know. Your feedback is always welcome as I continue to refine and enhance my approach.

Thank you for reading Hardcoded ❤️.


GitHub - elementh/hardcoded at entry/handling-telegram-bot-updates-concurrently
Contribute to elementh/hardcoded development by creating an account on GitHub.

Source code for all the code in this entry.

Channels - .NET
Learn the official synchronization data structures in System.Threading.Channels for producers and consumers with .NET.

I highly recommend checking out the documentation; it's brief but very informative.

Background tasks with hosted services in ASP.NET Core
Learn how to implement background tasks with hosted services in ASP.NET Core.

Again, highly recommended reading material on Background Services.

Introduction - A guide to Telegram.Bot library

Documentation for the Telegram.Bot library.