- Linux Applications Debugging Techniques/Core files
- Contents
- Prerequisites [ edit | edit source ]
- Where is my core? [ edit | edit source ]
- Dumping core from outside the program [ edit | edit source ]
- Dumping core from within the program [ edit | edit source ]
- Shared libraries [ edit | edit source ]
- Source Code [ edit | edit source ]
- analyze-cores [ edit | edit source ]
- Canned user-defined commands [ edit | edit source ]
- analyze-pid [ edit | edit source ]
- Thread Local Storage [ edit | edit source ]
- Выполняю установку, настройку, сопровождение серверов. Для уточнения деталей используйте форму обратной связи
- Core File In unix
- 1 Answer 1
- What is a core dump file in Linux? What information does it provide?
- 2 Answers 2
Linux Applications Debugging Techniques/Core files
A core dump is a snapshot of the memory of the program, processor registers including program counter and stack pointer and other OS and memory management information, taken at a certain point in time. As such, they are invaluable for capturing the state of rare occurring races and abnormal conditions.
What is more, such rarities will be found usually on heavily used production or QA machines where gdb is not available, nor is access easy to the machine. Worse, the heaviest users are usually the biggest clients (moneywise. ). As such, it is important to get as much forensic data as available, and plan for it.
One can force a core dump from within the program or from outside at chosen moments. What a core cannot tell is how the application ended up in that state: the core is no replacement for a good log. Verbose logs and core files go hand in glove.
Contents
Prerequisites [ edit | edit source ]
For a process to be able to dump core, a few prerequisites have to be met:
- the set core size limit should permit it (see the man page for ulimit). E.g.: ulimit -c unlimited. It can also be set from within the program.
- the process to dump core should have write permissions to the folder where the core is to be dumped to (usually the current working directory of the process)
Where is my core? [ edit | edit source ]
Usually the core is dumped in the current working directory of the process. But the OS can be configured otherwise:
Dumping core from outside the program [ edit | edit source ]
One possibility is with gdb, if available. This will let the program running:
Another possibility is to signal the process. This will terminate it, assuming the signal is not caught by a custom signal handler:
Dumping core from within the program [ edit | edit source ]
Again, there are two possibilities: dump core and terminate the program or dump and continue:
Note: use dump_core_and_continue() with care: in a multi-threaded program, the forked child will have only a clone of the parent thread that called fork() [Butenhof Ch5; re: threads & fork]. This has number of implications, in particular with respect to mutexes, but the particular point here is that the core that the child will dump will contain information only for one thread. If you need to dump a core with all threads without aborting the process, try to use the google core dumper library, even if it has not been maintained for years.
Shared libraries [ edit | edit source ]
To obtain a good call stack, it is important that the gdb loads the same libraries that were loaded by the program that generated the core dump. If the machine we are analyzing the core has different libraries (or has them in different places) from the machine the core was dumped, then copy over the libraries to the analyzing machine, in a way that mirrors the dump machine. For instance:
At the gdb prompt:
Source Code [ edit | edit source ]
To point the debugger to the source files:
analyze-cores [ edit | edit source ]
Here is a script that will generate a basic report per core file. Useful the days when cores are raining on you:
An alternative worth exploring is btparser.
Canned user-defined commands [ edit | edit source ]
Same reporting functionality can be canned for gdb:
analyze-pid [ edit | edit source ]
A script that will generate a basic report and a core file for a running process:
Thread Local Storage [ edit | edit source ]
TLS data is rather difficult to access with gdb in the core files, and __tls_get_addr() cannot be called.
Источник
Выполняю установку, настройку, сопровождение серверов. Для уточнения деталей используйте форму обратной связи
При падении приложения оно создаёт core-файл, который можно проанализировать. За создание core файлов отвечает утилита coreadm. С её помощью можно указать, где будут лежать core-файлы, для каких типов создавать core-файлы и т.д.
Важно понимать, что core dump файлы системы и процессов отличаются, и их нужно анализировать по разному.
coreadm
Данная команда введённая без параметров выведет текущие настройки:
Для того, что бы начать использовать coreadm установим путь, где будут храниться наши core-файлы:
#coreadm -g /var/cores/core.%f.%p
Далее выберем то, что хотим что бы попадало в core-файл:
#coreadm -G content
Возможные значения для content (оставлю без перевода, что бы не терялся смысл):
— ctf: CTF type information sections for loaded object files
— data: Writable private file mappings — dism: DISM mappings
— heap: Process heap
— ism: ISM mappings
— rodata: Read-only private file mappings
— shanon: Anonymous shared mappings
— shfile: Shared mappings that are backed by files
— shm: System V shared memory
— stack: Process stack
— symtab: Symbol table sections for loaded object files
— text: Readable and executable private file mappings
Либо просто указать all что бы попадало всё.
Есть ещё опции -e/-d которые означают включить/выключить определённые параметры.
Напомню, что coreadm управляется через SMF сервис svc:/system/coreadm:default. Так же можно отметить, что все изменения сохраняются в файле /etc/coreadm.conf
Правда частенько системные администраторы отключают создание core-файлов для процессов
coreadm -d process
так как они могут генерить большие объёмы данных.
При отлавливании багов лучше всего включать создание всех
Отдельно хочется отметить команду
которая позволит регистрировать в логах появление core-файлов
dumpadm
Данная команда управляет параметрами системного crash dump’a. Управляется она тоже через SMF — svc:/system/dumpadm:default и работает через устройство /dev/dump. Команда без параметров выведет текущие настройки:
savecore/gcore
Эта утилита служит для сохранения core dump файлов. Ею очень удобно сбрасывать в дамп текущее состояние системы:
# savecore -L
dumping to /dev/zvol/dsk/rpool/dump, offset 65536, content: kernel
0:03 100% done
100% done: 128815 pages dumped, dump succeeded
savecore: System dump time: Fri Apr 27 19:46:16 2012
savecore: Saving compressed system crash dump in /var/crash/vmdump.1
savecore: Decompress the crash dump with
‘savecore -vf /var/crash/vmdump.1’
# gcore $$
gcore: core.2770 dumped
Для того, чтобы проанализировать дамп, можно восстановить его в привычном виде:
# savecore -vf /var/crash/vmdump.1
savecore: System dump time: Fri Apr 27 19:46:16 2012
savecore: saving system crash dump in /var/crash/
Constructing namelist /var/crash/unix.1
Constructing corefile /var/crash/vmcore.1
0:12 100% done: 128815 of 128815 pages saved
3481 (2%) zero pages were not written
0:12 dump decompress is done
Анализ core dump-файлов.
Для анализа можно использовать отладчик mdb. Вот небольшой пример использования
# /usr/bin/mdb -k unix.0
Loading modules: [ unix krtld genunix ip nfs ipc ptm ]
> ::status
debugging crash dump /dev/mem (64-bit) from ozlo
operating system: 5.10 Generic sun4v
> ::system
set ufs_ninode=0x9c40 [0t40000]
set ncsize=0x4e20 [0t20000]
set pt_cnt=0x400 [0t1024]
Очень хорошо анализ core dump файлов описаны здесь
http://solaristhings.blogspot.com/2006/08/dont-be-afraid-of-mdb.html
http://kristof.willen.be/node/1100
http://www.c0t0d0s0.org/archives/4391-Less-known-Solaris-Features-About-crashes-and-cores.html
http://www.cuddletech.com/blog/pivot/entry.php?id=965
http://www.cuddletech.com/blog/pivot/entry.php?id=966
http://eteck.blogspot.com/2012/04/solaris-crash-dump-anylysis.html
Анализ core файлов.
Небольшой пример анализа core файла:
$ ./a.out
Segmentation Fault(coredump)
$ /usr/proc/bin/pstack ./core
core ‘./core’ of 19305: ./a.out
000108c4 main (1, ffbef5cc, ffbef5d4, 20800, 0, 0) + 1c
00010880 _start (0, 0, 0, 0, 0, 0) + b8
Очень удобно анализировать через gdb, но он не входит в базовую систему и его придётся доставлять отдельно.
#gdb /usr/php54/sbin/php-fpm ./core.php-fpm.23922
.
Reading symbols from /lib/amd64/libelf.so.1. done.
Loaded symbols for /lib/64/libelf.so.1
Reading symbols from /usr/lib/security/amd64/pkcs11_kernel.so.1. done.
Loaded symbols for /usr/lib/security/64/pkcs11_kernel.so.1
Reading symbols from /lib/amd64/ld.so.1. done.
Loaded symbols for /lib/amd64/ld.so.1
Core was generated by `/usr/php54/sbin/php-fpm -y /usr/php54/etc/php-fpm.conf’.
Program terminated with signal 11, Segmentation fault.
[New process 89458 ]
#0 0x000000000093f2eb in zend_fetch_var_address_helper_SPEC_CONST_VAR ()
(gdb) bt
#0 0x000000000093f2eb in zend_fetch_var_address_helper_SPEC_CONST_VAR ()
#1 0x0000000000938c81 in execute ()
#2 0xfffffd7f923747b7 in xdebug_execute () from /usr/php54/lib/php/extensions/no-debug-non-zts-20100525/xdebug.so
#3 0x00000000009394c7 in zend_do_fcall_common_helper_SPEC ()
#4 0x0000000000938c81 in execute ()
#5 0xfffffd7f923747b7 in xdebug_execute () from /usr/php54/lib/php/extensions/no-debug-non-zts-20100525/xdebug.so
#6 0x00000000008fa118 in zend_call_function ()
#7 0x00000000009222b1 in zend_call_method ()
#8 0x00000000007d0672 in zif_spl_autoload_call ()
#9 0x00000000008fa080 in zend_call_function ()
#10 0x00000000008fa8d4 in zend_lookup_class_ex ()
#11 0x000000000091170f in zend_is_callable_check_class ()
#12 0x0000000000912811 in zend_is_callable_ex ()
#13 0x00000000008f9605 in zend_call_function ()
#14 0x00000000008f93f3 in call_user_function_ex ()
#15 0x00000000008f92e7 in call_user_function ()
#16 0x00000000007c16d4 in ps_call_handler ()
#17 0x00000000007c1c0a in ps_write_user ()
#18 0x00000000007b8edf in php_session_save_current_state ()
#19 0x00000000007be9e8 in zm_deactivate_session ()
#20 0x0000000000910c1a in zend_deactivate_modules ()
#21 0x0000000000898073 in php_request_shutdown ()
#22 0x00000000009859b1 in main ()
(gdb) where
#0 0x000000000093f2eb in zend_fetch_var_address_helper_SPEC_CONST_VAR ()
#1 0x0000000000938c81 in execute ()
#2 0xfffffd7f923747b7 in xdebug_execute () from /usr/php54/lib/php/extensions/no-debug-non-zts-20100525/xdebug.so
#3 0x00000000009394c7 in zend_do_fcall_common_helper_SPEC ()
#4 0x0000000000938c81 in execute ()
#5 0xfffffd7f923747b7 in xdebug_execute () from /usr/php54/lib/php/extensions/no-debug-non-zts-20100525/xdebug.so
#6 0x00000000008fa118 in zend_call_function ()
#7 0x00000000009222b1 in zend_call_method ()
#8 0x00000000007d0672 in zif_spl_autoload_call ()
#9 0x00000000008fa080 in zend_call_function ()
#10 0x00000000008fa8d4 in zend_lookup_class_ex ()
#11 0x000000000091170f in zend_is_callable_check_class ()
#12 0x0000000000912811 in zend_is_callable_ex ()
#13 0x00000000008f9605 in zend_call_function ()
#14 0x00000000008f93f3 in call_user_function_ex ()
#15 0x00000000008f92e7 in call_user_function ()
#16 0x00000000007c16d4 in ps_call_handler ()
#17 0x00000000007c1c0a in ps_write_user ()
#18 0x00000000007b8edf in php_session_save_current_state ()
#19 0x00000000007be9e8 in zm_deactivate_session ()
#20 0x0000000000910c1a in zend_deactivate_modules ()
#21 0x0000000000898073 in php_request_shutdown ()
#22 0x00000000009859b1 in main ()
Очень удобно так же запустить программу через gdb:
Источник
Core File In unix
I have some doubts about core file.
1) what is core file. 2) where it is located. 3) what is the use of core fie. 4) How to generate core files while compiling c programs. 5) what is the difference between core file and core dump.
1 Answer 1
It is a file that contains a core dump. Certain abnormal program terminations such as executing an illegal instruction cause a core dump to be generated by default.
Usually it is a file called core in whatever was the current directory of the process that caused it to be generated, but in some operation systems like Linux it is possible to reconfigure this so that it is saved to a different file, or so that it is piped to a program. For Linux, looks in /proc/sys/kernel/core_pattern to find the name of the file in which core dumps will be placed.
3) what is the use of core fie.
A core dump can be loaded into a debugger like gdb. This enables examining the state of the program just as it was when it got terminated. It can come in handy for the developer to determine why the program crashed.
4) How to generate core files while compiling c programs.
That’s on odd question. I guess the answer is that in order to generate a core dump while compiling C programs oyu’d have to cause the compiler to crash! For example you could find and exploit a bug in the compiler! Or you could just send it a SIGQUIT by pressing Ctrl ^ which by default forcibly terminates programs with a core dump.
5) what is the difference between core file and core dump.
I guess you could define it as: a core file is a file that contains a core dump. But generally I think that most people would use the terms interchangeably.
Источник
What is a core dump file in Linux? What information does it provide?
What is a core dump file in linux? What all information does it provide?
2 Answers 2
It’s basically the process address space in use (from the mm_struct structure which contains all the virtual memory areas), and any other supporting information *a , at the time it crashed.
For example, let’s say you try to dereference a NULL pointer and receive a SEGV signal, causing you to exit. As part of that process, the operating system tries to write your information to a file for later post-mortem analysis.
You can load the core file into a debugger along with the executable file (for symbols and other debugging information, for example) and poke around to try and discover what caused the problem.
*a : in kernel version 2.6.38, fs/exec.c/do_coredump() is the one responsible for core dumps and you can see that it’s passed the signal number, exit code and registers. It in turn passes the signal number and registers to a binary-format-specific (ELF, a.out, etc) dumper.
The ELF dumper is fs/binfmt_elf.c/elf_core_dump() and you can see that it outputs non-memory-based information, like thread details, in fs/binfmt_elf.c/fill_note_info() , then returns to output the process space.
If a program terminates abnormally, the status of the program at the point of abnormal termination should be recorded for further analysis. and this status is recorded in core dump file.
In a multiuser and multitasking environment, accessing resources which doesn’t belong to you is not acceptable. If a process-A tries to access system resources which belongs to process-B, Its a violation. At this point of time, the operating system kills the process and stores the process status into a file. And this file is called core dump file. There are many reasons for core dump. I just explained one of the possibilities for core dump. Usually it will be because of SIGSEGV (segmentation fault) and SIGBUS(Bus error).
The core dump file contains details of where the abnormal termination happened, process stack, symbol table etc.
There are many tools available to debug the coredumps. gdb dbx objdump mdb
Compiler options are present to make the debugging process easier. while compilation giving these flags (-g usually) will result in leaving extra information in symbol table of object files, which helps debuggers (gdb/dbx) to easily access the symbols(symbolic references).
Источник