Linux hard soft limit

Ulimit, Soft Limits and Hard Limits in Linux

ulimit is admin access required Linux shell command which is used to see, set, or limit the resource usage of the current user. It is used to return the number of open file descriptors for each process. It is also used to set restrictions on the resources used by a process.

Syntax:

To check the ulimit value use the following command:

Working with ulimit commands:

1. To display maximum users process or for showing maximum user process limit for the logged-in user.

2. For showing the maximum file size a user can have.

3. For showing maximum memory size for the current user.

4. For showing maximum memory size limit.

What are Soft limits and Hard limits in Linux?

The soft limits are the limits which are allocated for actual processing of application or users while the Hard limits are nothing but an upper bound to the values of soft limits. Hence,

Working with Hard and Soft limit values:

1. For displaying the Hard limit. Hard limits are a restriction to the maximum value of soft limits

2. For displaying Soft Limit. The soft limits are the limits that are there for processing.

3. To change Soft Limit values:

Note: Replace with the value you want to set for the soft limit and also remember size can not exceed the Hard Limit!

4. Displaying current Values for opened files

Источник

Soft limit vs hard limit

I ran into a problem of :

I know that nproc is the problem Some suggested to increase the soft limit of nproc while other suggested the hard limit.

Which should I increase? Isn’t the soft limit is there just to warn the user and the hard limit is the one that really limits eventually?

2 Answers 2

It’s actually other way around.

The soft limit’s value(s) is actually implemented i.e. in use, you can increase the limit upto the relevant hard limit’s value(s) (assuming you are not super user or do not have CAP_SYS_RESOURCE capability).

Think of the hard and the soft limits as mandatory and discretionary limits, resp. The hard limit is imposed by the system (through suitable configuration, e.g. limits.conf(5) ) and can be increased only by the superuser (i.e. root), while the user can gauge soft limits at his discretion within the range of the according hard limits.

From the getrlimits(2) manual page:

The soft limit is the value that the kernel enforces for the corresponding resource. The hard limit acts as a ceiling for the soft limit: an unprivileged process may set only its soft limit to a value in the range from 0 up to the hard limit, and (irreversibly) lower its hard limit.

Thus, if you hit a resource limit, then check whether you can increase the according soft limit; if not, then the hard limit needs to be increased.

Response to Comment

There is no such thing as only the hard limit. Limits are set by setrlimit(2) , which refers to struct rlimit . This in turn has members for soft and hard limit ( rlim_cur and rlim_max , resp.). If a soft limit is not explicitly defined, then it defaults to some value; most likely the according hard limit, but this is up to the process that sets the limits. ( setrlimits(2) rejects rlim_cur exceeding rlim_max , so RLIM_INFINITY is typically not a valid default for rlim_cur .)

Источник

Soft limit vs Hard limit?

Can anyone explain in layman’s terms what the difference between soft and hard limit is?

Should I set my soft and hard limit to be the same? Or should soft be significantly lower? Does the system benefit either way?

3 Answers 3

The hard limit is the ceiling for the soft limit. The soft limit is what is actually enforced for a session or process. This allows the administrator (or user) to set the hard limit to the maximum usage they wish to allow. Other users and processes can then use the soft limit to self-limit their resource usage to even lower levels if they so desire.

Users who violate a soft limit quota get an e-mail indicating that they have a few days’ grace period before the penalties kick in. Users who cross the hard limit threshold get no such grace period. The penalty differs depending on the particular quota, but usually nearly every command will fail such that the user will want to come into compliance quickly.

User resource limits dictate the amount of resources that can be used for a particular session. The resources that can be controled are:

It is important to note that these settings are per-session. This means that they are only effective for the time that the user is logged in (and for any processes that they run during that period). They are not global settings. In other words, they are only active for the duration of the session and the settings are not cumulative. For example, if you set the maximum number of processes to 11, the user may only have 11 processes running per session. They are not limited to 11 total processes on the machine as they may initiate another session. Each of the settings are per process settings during the session, with the exception of the maximum number of processes.

There are two types of limits that can be set for each property listed above, a hard and soft limit.

Источник

Change Soft/Hard limits and file descriptor limits on Amazon Linux 2(systemd)

Motivation

When I migrate Amazon Linux to Amazon Linux2 , I investigate how to change file descriptor limits and number of process per user on Linux server working with systemd . This post is technical memo for myself.

I try the following procedure after booting the official Amazon Linux 2 AMI as it is.

Soft limits and Hard limits

Let me review what Soft limits and Hard limits are.

Linux limits the number of process per user. There are two kinds of limits named Soft limits and Hard limits. Soft limits mean number of the process for the current user, and Hard limits mean upper limit of Soft limits that can be changed by the user.

Setting for Linux login user

Edit limits.conf

The configurations in /etc/security/limits.conf are applied to Linux login user only login with pluggable authentication module(PAM). The pam_limits module in Ansible edits this file.

Overriding limits.conf

A part of comments in /etc/security/limits.conf explains the priority of settings as bellow.

Also note that configuration files in /etc/security/limits.d directory, which are read in alphabetical order, override the settings in this file in case the domain is the same or more specific.

There is a default Soft limits settings in /etc/security/limits.d/20-nproc.conf on Amazon Linux 2.

Check current configuration

ulimit -a command lists available system resources for current user. ulimit command sets Hard limits with -H option, and Soft limits with -S option.

Configures daemon

Now limits the system resources for daemon process. The daemon process in systemd uses the configuration in /etc/security/limits.conf , but /etc/systemd/system.conf and /etc/systemd/user.conf,/etc/systemd/ /override.conf .

Change overall default settings

To change default settings for process controlled by systemd, edit /etc/systemd/system.conf . For instance, changing the file descriptor limits and upper limits of the process is as bellow.

Settings per daemon

When system administrators give services in systemd appropriate system resources, they creates /etc/systemd/system/(service name).service and add [Service] block.

Changing only system resource despite of /etc/systemd/system/（service name）.service file already exists, override with creating /etc/systemd/system/（service name）.service.d/override.conf .

In addition, the setting of override.conf takes precedence over (service name) .service .

Changing the number of file descriptor limits is as below.

Now restart daemon.

Check the configuration.

The file descriptor (Max open files) is 40000.

Conclusion

It is available to …

• Change Soft limits for current login user
• Change Hard limits for current login user
• Change default settings for daemon process managed by systemd

Источник

системные настройки высоконагруженного сервера

Настройки Linux по-умолчанию не годятся для высоких нагрузок. Под высокими нагрузками я понимаю от 10000 запросов в секунду. В данной статье рассмотрим два «переключателя», покрутив которые мы добьемся от сервера устойчивости и отзывчивости при высоких нагрузках. Эти переключатели: limits.conf и sysctl.conf

limits.conf

Это конфигурационный файл для pam_limits.so модуля. Он определяет ulimit лимиты для пользователей и групп. В Linux есть системные вызовы: getrlimit() и setrlimit() для получения и установления лимитов на системные ресурсы. Конфигурация по-умолчанию лежит в /etc/security/limits.conf. Также присутствует возможность добавлять отдельные конфиги для приложений в /etc/security/limits.d. Для того, чтобы сервер держал большую нагрузку, нужно увеличить некоторые лимиты. Посмотрим, какие лимиты у нас стоят по-умолчанию. Запустим под рутом.

Обратим внимание на open files, max user processes и max locked memory. Это стандартные ограничения, которые нужно убрать, если хотим держаться под нагрузкой. Перед изменениями хочу предупредить, что если железо недостаточно мощное, то ваш сервер может подвергнуться атаке типа fork bomb, так что аккуратно раздавайте права на сервере.

Приведем /etc/security/limits.conf к такому виду.

В файле мы устанавливаем мягкий и жесткий лимиты на количество запущенных процессов, открытых файлов (читай портов) для всех пользователей и неограниченный лок памяти для рута.

Хочу поподробнее рассказать, почему для открытых файлов было выбрано ограничение в 1048576. Это волшебное число захардкожено в ядре, чтоб поставить больше, нужно пересобирать ядро. Более развернуто на эту тему отвечают на stackoverflow и вот здесь.

Изменения вступят в силу или после перезагрузки или после нового логина или создания сессии ssh. Проверить, включается ли модуль pam_limits можно в файлах /etc/pam.d Для Debian есть статья в wiki на эту тему: https://wiki.debian.org/Limits

sysctl.conf

В /etc/sysctl.conf настраиваются параметры ядра, модулей и других подсистем. По сути, можно вручную менять значения псевдо-фс /proc, но такие изменения не сохранятся после перезагрузки, поэтому будем сразу вносить изменения в этот конфиг файл.

Для пользователей systemd этот файл уже не играет роли. Если вы вдруг используете systemd, вам нужно править файлы в /etc/sysctl.d/. Подробнее читайте на http://www.freedesktop.org/software/systemd/man/sysctl.d.html

Вот пример моего sysctl.conf для высоконагруженной системы.

Подробно про все опции, можно прочитать в

Или, например, здесь. А тут написано как люди выдерживают миллион pps в секунду и приведены примеры используемых sysctl.conf

После изменения sysctl.conf применим наши правки.

После перезагрузки все изменения будут применяться автоматически.

Заключение

Все вышеописанное применялось мной на работе для высоконагруженного сервера, который перед выкаткой в прод тестировался замечательным Yandex танком и держал нагрузку порядка 20000 rps. Стоит учесть, что сам по себе Linux не обеспечит вам стойкости под нагрузкой, если ваш софт ломается при 100 rps. Тут уже вас может спасти балансировка нагрузки. Смотрите в сторону HAProxy, LVS, Keepalived. Удачного прохождения высоких нагрузок!

Источник