Библиотека libevent для асинхронного неблокирующего ввода/вывода
В этом цикле статей рассматривается библиотека libevent, предназначенная для обработки оповещений о событиях и организации асинхронного ввода/вывода.
В первой статье описывается общая структура библиотеки.
Вторая статья посвящена основам использования libevent: наибольшее внимание уделено обработке событий.
В третьей статьеречь идёт о механизмах буферизации ввода/вывода. Тема четвёртой, заключительной статьи — применение libevent в сетевых приложениях.
Лучше бы про Pi-calculi и настоящее параллельное программирование писали, чем про этот убогий асинхрон.
Ну, лично меня заинтересовало.
Вопрос вот в чём: вики говорит, что
ОС Linux, *BSD, Mac OS X, Solaris и Windows.
А как же поддержка всяких AIX, HP-UX, QNX и прочего?
Это не параллельное программирование. Это кросс-платформенная обёртка всяких там poll.
>Лучше бы про Pi-calculi и настоящее параллельное программирование писали, чем про этот убогий асинхрон.
Никто не мешает сочетать одно с другим. 😉
Закопать немедленно
Однажды, год назад где-то, мы решили ее попробовать в реальном проекте. Нужен был сервер, работающий под РедХат. Довольно быстро посыпались проблемы. Мы писали одну заплатку за другой. Любимый перл в этом чуде был exit(1) при небольших и совсем не фатальных проблемах. Это мы тоже зашили, переписав error handler и загрузив его вместо их стандартного. Но нас добила поддержка Windows. Libevent в Windows использует.. select, и про 10K problem и в ус не дует. К этому моменту обьем кода нашей обертки уже начала быть сравнима с обьемом libevent. Короче, выбросили libebent и написали с нуля. Кода там немного, на порядок меньше чем в libebent, производительность не хуже, и работает одинаково быстро как под Linux, так и под Windows. Я не хочу обсуждать версию 1.4 против 2.0. Но те баги, что мы увидели в 1.4, которая была в каждом дистре — они неприемлимы даже в версии 0.1.
Re: Закопать немедленно
Какой механизм использовали в Windows?
Да, версия до 2.0 это жесть, но в 2.0 исправлены многие косяки, мы сейчас на реальном проекте её используем, а именно используем http-клиент и сервер оттуда. Работает все как часы, нагрузку держит.
Под Windows, сообщения Winsock направляются невидимому окну через WSAAsyncSelect. Сообщения окна слушаются в отдельной нити.
Re: Закопать немедленно
С уважением, Евгений.
> Лучше бы про Pi-calculi и настоящее параллельное программирование писали, чем про этот убогий асинхрон.
Почитав комментарии, начинаешь понимать людей, которые серверные приложения пишут на Qt 🙂 Хотя, конечно, избыточнооость.
Re: Закопать немедленно
Используется в node.js, и довольно успешно.
Re: Закопать немедленно
Ага. Если не считать жесточайших тормозов и утечек.
Мы успешно используем libevent в реальном проекте (версию 1.4). У libevent достаточно хорошо продуманная архитектура и простой код. У них также адекватные мэйнтейнеры, которые принимают патчи и активно общаются в рассылке. Если у вас возникли проблемы с libevent, и если вы даже написали патч, то почему бы не зарепортить баг?
. попробовать в реальном проекте. Нужен был сервер, работающий под РедХат. Довольно быстро посыпались проблемы.
Как показывает опыт, в подавляющем числе случаев ошибки, связанные с libevent, возникают из-за ошибок основной программы или из-за неправильного использования libevent.
Мы писали одну заплатку за другой.
Не верю. Мы при очень интенсивном использовании libevent нашли только пару багов (исправления уже давно в коде libevent).
Это уже какой-то другой реальный проект? Ведь речь шла о РедХате.
> Libevent в Windows использует.. select, и про 10K problem и в ус не дует.
Не знаю, как в 1.4, но в 2.0 (цитирую документацию):
A bufferevent that uses the Windows IOCP interface to send and receive data to an underlying stream socket. (Windows only; experimental.)
Источник
What is libevent in linux
libevent is aEvent triggerofNetwork library, Suitable for windows, linux, bsd and other platforms, internal useselect、epoll、kqueueWaitSystem callManagement event mechanism. The well-known distributed caching software memcached is also libevent based, and libevent can be used across platforms, and according to statistics published on the libevent official website, it seems to have extraordinary performance.
Libevent is an event-based network library. To put it in a popular way, for exampleClientConnected toServerAn event belonging to a connection will be processed when the event is triggered.
2.1 Create event_base
event_base iseventone ofset. Event_base stores the event you are monitoring whether it is ready. Generally, one event_base is for one thread, and multiple event_bases need to be opened in the case of multiple threads.
event_base is mainly used to manage and implement the event monitoring loop.
Under normal circumstances, directly new event_base can meet most of the needs. If you need to configure parameters, you can refer to the libevent official website.
2.2 View the IO model
In the IO multiplexing model (IO model article), there are many ways for us to choose, but these models are under different platforms: select poll epoll kqueue devpoll evport win32
When we create an event_base, libevent willautomaticChoose for usThe fastest IO multiplexing model, Linux generally uses the epoll model.
The following method is mainly used to obtain the name of the IO model.
2.3 Destroy event_base
2.4 event loop event loop
As mentioned above, event_base is a collection of events, and we can also register event events to this collection. When event monitoring is needed, we need to loop this event_base.
The following function is very important and will beInternal continuous loop monitoring of registered events。
Return value: 0 means successful exit -1 means there is error message.
You can also use this method:
The event_base_loop method is more flexible than the event_base_dispatch method.
EVLOOP_ONCE: Block until there is an active event, and then exit after executing the active event callback.
EVLOOP_NONBLOCK: No blocking, check which event is ready, call the one with the highest priority, and then exit.
0: If the parameter is filled with 0, thenOnly when the event comes inWillCall the callback function of an event, More commonly used
When the event loop stops:
There is no event in event_base
Call event_base_loopbreak(), then the event loop will stop
Call event_base_loopexit(), then the event loop will stop
Program error, abnormal exit
Two exit methods:
The difference between the two methods:
event_base_loopexit(base, NULL) If the callback function is currently being called for multiple active events, it will not exit immediately, but will wait until the callback functions of all active events are executed before exiting the event loop
event_base_loopbreak(base) If the callback function is currently being called for multiple active events, the callback function currently being called will be executed, and then the event loop will immediately exit without processing other active events.
2.5 example of event_base
event_base is a collection of events, responsible for the cycle of events, and the destruction of the collection. Event is the basic unit in event_base: event.
Let’s take a simple example to understand events. For example, when our socket is used for network development, we will use the accept method to block monitoring whether there is a client socket connection. If the client connects, a thread will be created for the server to interact with the client. , And the server will continue to block waiting for the next client socket to connect. The client connecting to the server is actually an event.
3.1 Create an event
fd: file descriptor.
what: Various conditions that the event cares about.
cb: callback function.
arg: User-defined data, which can be passed to the callback function.
Libevent is event-based, which means that the current event will only be triggered when the event arrives. E.g:
fd file descriptor is ready to be written or readable
fd is ready to be written and readable immediately.
User triggered event
What parameter event various conditions:
3.2 Release event_free
Really release the event memory.
event_del clears the memory of the event. This method does not release memory in the true sense.
When the function turns the event into a non-pending and non-activing state.
3.3 Register event
This method will be usedRegister events with event_base。
Parameters: ev is the event pointer; tv is the time pointer. When tv = NULL, there is no timeout period.
The function returns: 0 means success -1 means failure.
Examples of tv time structure:
event_newEvery timeAllocate memory on the heap. In some scenarios, it is not necessary to allocate memory on the heap every time. At this time, we can use the event_assign method.
For an event that has been initialized or is pending, first call event_del() and then call event_assign(). This event can be reused at this time.
3.5 Signal event
Signal events can also process events on signals. The usage is similar to event_new. It just deals with signals.
3.6 event details
Each event event needs to be initialized and generated by event_new. The event generated by event_new is the memory allocated on the heap.
When an event is registered to event_base through event_add, the event is in pending (waiting state). When an event comes in, the event will be activated in the active state, and the related callback function will be called.
persistent If EV_PERSIST is selected for the what parameter in event_new, it is a persistent type. After the persistent type calls the play callback function, it will continue to be in the pending state and will continue to wait for the event to come in. In most cases, persistent events are selected.
Instead of a persistent type of event, it will become an initialized state after calling it once. At this time, you need to call event_add to continue registering the event to event_base before it can be used.
4 Socket example
- The socket must be set to non-blocking mode, otherwise it will be blocked there and affect the operation of the entire program
- The events we created first are mainly usedMonitor client connection. When the client has a socket connected to the server, the callback function do_accept will be executed; when idle, this event will be in a pending waiting state, waiting for a new connection to come in, and then continue after the new connection comes in carried out.
- In the do_accept event we created a new event, the callback function of this event is do_read. Mainly usedLoop to monitor the data uploaded by the client. The do_read method will always be executed in a loop, and the client data will be processed.
The above socket example is estimated that after testing, everyone will have a lot of questions:
The do_read method will always be looped as an event
When the client connection is disconnected, the do_read method is still looping, without knowing that the client has disconnected the socket connection.
Need to solve various sticking and unpacking (related sticky and unpacking articles)
If we want to solve this problem, we may do a lot of work to maintain the connection status of these sockets and read the status. And Libevent’s Bufferevent helped us solve these problems.
Bufferevent is mainly used to manage and schedule IO events; Evbuffer (described in the next section) is mainly used to buffer network IO data.
Bufferevent currently supports the TCP protocol and does not know the UDP protocol. We also only talk about the use of Bufferevent under the TCP protocol.
Let’s take a look at the following interface first (and then combine the following example of improving the socket, do it yourself to experiment):
5.1 Create Bufferevent API
fd: file descriptor. If it is a socket method, the socket needs to be set to non-blocking mode.
options: behavior options, the following is the content of behavior options
BEV_OPT_CLOSE_ON_FREE: When the bufferevent is released and the bottom layer is closed (socket is closed, etc.), this option is generally used
BEV_OPT_THREADSAFE: Automatically allocate locks for bufferevent, so that it can be used safely in a multithreaded environment
BEV_OPT_DEFER_CALLBACKS: When this flag is set, bufferevent will delay all its callbacks (refer to the delayed callback mentioned earlier)
BEV_OPT_UNLOCK_CALLBACKS: If bufferevent is set to be thread-safe, the bufferevent lock will be held when the user-provided callback is called. If this option is set, Libevent will release the bufferevent lock when calling your callback
Источник
