resource — Resource usage information¶

This module provides basic mechanisms for measuring and controlling system resources utilized by a program.

Symbolic constants are used to specify particular system resources and to request usage information about either the current process or its children.

An OSError is raised on syscall failure.

exception resource. error В¶

A deprecated alias of OSError .

Changed in version 3.3: Following PEP 3151, this class was made an alias of OSError .

Resource Limits¶

Resources usage can be limited using the setrlimit() function described below. Each resource is controlled by a pair of limits: a soft limit and a hard limit. The soft limit is the current limit, and may be lowered or raised by a process over time. The soft limit can never exceed the hard limit. The hard limit can be lowered to any value greater than the soft limit, but not raised. (Only processes with the effective UID of the super-user can raise a hard limit.)

The specific resources that can be limited are system dependent. They are described in the getrlimit(2) man page. The resources listed below are supported when the underlying operating system supports them; resources which cannot be checked or controlled by the operating system are not defined in this module for those platforms.

Constant used to represent the limit for an unlimited resource.

resource. getrlimit ( resource ) В¶

Returns a tuple (soft, hard) with the current soft and hard limits of resource. Raises ValueError if an invalid resource is specified, or error if the underlying system call fails unexpectedly.

resource. setrlimit ( resource, limits ) В¶

Sets new limits of consumption of resource. The limits argument must be a tuple (soft, hard) of two integers describing the new limits. A value of RLIM_INFINITY can be used to request a limit that is unlimited.

Raises ValueError if an invalid resource is specified, if the new soft limit exceeds the hard limit, or if a process tries to raise its hard limit. Specifying a limit of RLIM_INFINITY when the hard or system limit for that resource is not unlimited will result in a ValueError . A process with the effective UID of super-user can request any valid limit value, including unlimited, but ValueError will still be raised if the requested limit exceeds the system imposed limit.

setrlimit may also raise error if the underlying system call fails.

VxWorks only supports setting RLIMIT_NOFILE .

Raises an auditing event resource.setrlimit with arguments resource , limits .

resource. prlimit ( pid, resource [ , limits ] ) В¶

Combines setrlimit() and getrlimit() in one function and supports to get and set the resources limits of an arbitrary process. If pid is 0, then the call applies to the current process. resource and limits have the same meaning as in setrlimit() , except that limits is optional.

When limits is not given the function returns the resource limit of the process pid. When limits is given the resource limit of the process is set and the former resource limit is returned.

Raises ProcessLookupError when pid can’t be found and PermissionError when the user doesn’t have CAP_SYS_RESOURCE for the process.

Raises an auditing event resource.prlimit with arguments pid , resource , limits .

Availability : Linux 2.6.36 or later with glibc 2.13 or later.

New in version 3.4.

These symbols define resources whose consumption can be controlled using the setrlimit() and getrlimit() functions described below. The values of these symbols are exactly the constants used by C programs.

The Unix man page for getrlimit(2) lists the available resources. Note that not all systems use the same symbol or same value to denote the same resource. This module does not attempt to mask platform differences — symbols not defined for a platform will not be available from this module on that platform.

The maximum size (in bytes) of a core file that the current process can create. This may result in the creation of a partial core file if a larger core would be required to contain the entire process image.

The maximum amount of processor time (in seconds) that a process can use. If this limit is exceeded, a SIGXCPU signal is sent to the process. (See the signal module documentation for information about how to catch this signal and do something useful, e.g. flush open files to disk.)

The maximum size of a file which the process may create.

The maximum size (in bytes) of the process’s heap.

The maximum size (in bytes) of the call stack for the current process. This only affects the stack of the main thread in a multi-threaded process.

The maximum resident set size that should be made available to the process.

The maximum number of processes the current process may create.

The maximum number of open file descriptors for the current process.

The maximum address space which may be locked in memory.

The largest area of mapped memory which the process may occupy.

The maximum area (in bytes) of address space which may be taken by the process.

The number of bytes that can be allocated for POSIX message queues.

New in version 3.4.

The ceiling for the process’s nice level (calculated as 20 — rlim_cur).

New in version 3.4.

The ceiling of the real-time priority.

New in version 3.4.

The time limit (in microseconds) on CPU time that a process can spend under real-time scheduling without making a blocking syscall.

New in version 3.4.

The number of signals which the process may queue.

New in version 3.4.

The maximum size (in bytes) of socket buffer usage for this user. This limits the amount of network memory, and hence the amount of mbufs, that this user may hold at any time.

Availability : FreeBSD 9 or later.

New in version 3.4.

The maximum size (in bytes) of the swap space that may be reserved or used by all of this user id’s processes. This limit is enforced only if bit 1 of the vm.overcommit sysctl is set. Please see tuning(7) for a complete description of this sysctl.

Availability : FreeBSD 9 or later.

New in version 3.4.

The maximum number of pseudo-terminals created by this user id.

Availability : FreeBSD 9 or later.

New in version 3.4.

Resource Usage¶

These functions are used to retrieve resource usage information:

resource. getrusage ( who ) В¶

This function returns an object that describes the resources consumed by either the current process or its children, as specified by the who parameter. The who parameter should be specified using one of the RUSAGE_* constants described below.

A simple example:

The fields of the return value each describe how a particular system resource has been used, e.g. amount of time spent running is user mode or number of times the process was swapped out of main memory. Some values are dependent on the clock tick internal, e.g. the amount of memory the process is using.

For backward compatibility, the return value is also accessible as a tuple of 16 elements.

The fields ru_utime and ru_stime of the return value are floating point values representing the amount of time spent executing in user mode and the amount of time spent executing in system mode, respectively. The remaining values are integers. Consult the getrusage(2) man page for detailed information about these values. A brief summary is presented here:

python-resources 0.3

pip install python-resources Copy PIP instructions

Released: Mar 22, 2013

A fixture lifecycle management library for your tests

Navigation

Project links

Statistics

View statistics for this project via Libraries.io, or by using our public dataset on Google BigQuery

License: BSD

Maintainers

Project description

Why do we need this library

We are not satisfied with classical xUnit way of setup and teardown. We prefer concise approach of py.test over the verbosity of standard unittest.

We found ourselves copying and pasting the same boilerplate code from one test to another or creating extensive structure of test class hierarchy.

py.test fixtures, injected in test functions as parameter names, is different approach for fixture management. It’s neither worse nor better, but we found it to be not as flexible as we need.

Some questions, that we wanted to solve often, looked like:

• I have a py.test fixture which creates a new user with default set of properties. Is there a way I can create a user with different name by the same fixture?
• Is there a way to create two users in one test case with the same fixture?
• Is there an easy recipe to create a user first, and then, say, a todo item for this particular user in another, separate, fixture?

Sure enough, we can handle or work around all these issues somehow with xUnit setups and teardowns or py.test fixtures, but we wanted something more flexible, easy and convenient to use. That’s why we created resources library.

How do we use it

First, we define functions which we call “resource makers”. These makers are responsible for creating and destroying resources. It’s like setup and teardown in one callable.

The flow is simple: we create, we yield, we destroy.

We get a number of resource makers, and we group them into modules, like tests/resources_core.py , tests/resources_users.py , etc.

Then, in a test file, where we plan to use resources, we import the same global object, load resource modules we need, and activate them in tests.

This is where a little bit of magic happens. Once you define and register the resource maker with name foo, a context manager foo_ctx is created for your convenience. This context manager creates a new resource instance with the corresponding maker function, and destroys the object the way you defined, once the code flow abandons a wrapping “with”-context.

When it shines

At this point and maybe not so exciting. Yeah, everyone can write the code like this, the difference is that we actually did it :-). We also have a bunch of nifty features making the whole stuff more interesting.

Feature 1. Customizeable resources

Contexts are better than py.test fixtures, because they are customizeable. Provide everything you need to context manager, and it will be passed to resource maker function as an arguments.

Feature 2. Global object scope and dependent resources

We need to have access to resources at different stages of our tests: to get access to object’s properties and methods, to initiate another, dependent fixture instance, and finally to tear down everything.

As soon as you enter the context with resources.foo_ctx() a variable resources.foo will be created and will be available from everywhere, including your test function, and other resource makers.

The latter fact is especially important, because it’s the way we manage dependent resources. Yet we need some conventions, which resource is created first, and so on.

We agreed that we create user resource first, and todo item afterwards, and created a new resource maker, taking advantage of this convention.

We use it like this:

By the way, if you are still stuck with python2.6, several context managers in the same “with” expression aren’t available for you yet. Use contextlib.nested to avoid deep indentation.

Feature 3. Several resources of the same class, and tuneable resource names

Sometimes we need to create a couple of resources of the same type, instead of just one instance. It’s not a problem, if you don’t want to use global namespace to get access to them. Otherwise you must create a unique identifier for every resource.

Actually, it’s trivial. All you should do is provide a special _name attribute to context manager constructor. This attribute won’t be passed to your resource maker function.

Feature 4. Function decorators

Context manager can work as a decorator too. When we use it like this, an extra argument will be passed to the function.

We should say that usually it works, but to make it work along with py.test which performs deep introspection of function signatures, we made in with some “dirty hacks” inside, and you may find out that in some cases the chain of decorators dies with a misleading exception. We’d recommend to use context managers instead of decorators, wherever possible.

Feature 5. Resource managers

Yes, we do use setup and teardown methods too. If every function in your test suite uses the same set of resources, it would be counterproductive to write the same chain of decorators or context managers over and over again.

In this case we use another concept: resource managers. Every resource maker foo creates the resources.foo_mgr instance, having start and stop methods. The start method accepts all arguments which the foo_ctx function does, including special _name argument. The stop method has only one optional _name argument, and is used to destroy previously created instance.

Here is a py.test example

Feature 6. Built-in console and debugger

Sometimes it’s nice to take a look on what’s going on within test function and get access at some point to python console or debugger.

Usually you probably do something like

Or, if you need to get shell and have IPython installed

As it happens often, we added to resources two functions, launching either debugger or python console inside your test function.

If you install IPython and ipdb ( pip install IPython ipdb ), you get more friendly versions of consoles, otherwise resources fall back to built-in python console and debugger.

Launch py.test with -s switch to be able to fall into interactive console.

It’s especially cool that resources object is autocomplete-friendly and it works well in IPython

Feature 7. Globally accessible storage of constants

This feature is not something unique to resources module. Pretty much every object can act this way, but it is handy to have a convention about the way you store your test-related constants.

It may work like this.

And then, in the test file.

Conclusion

The resources library works for us in py.test environment. We don’t see any reasons why it shouldn’t work the same way with nose or classic unitttests. It works for python versions 2.6, 2.7 and 3.3.

Please bear in mind that the library is not thread safe, as we are happy with single threaded tests at this time.

And after all… Seven extra features to improve your test suites for free! What are you waiting for? It’s already improved the quailty of our lives in Doist Inc, and we do hope it will do the same for your projects.