Linux exit code 137

Tasks fail with non-zero exit (137) #27238

Comments

saithala commented Oct 9, 2016

Description

Im trying to launch a service with Docker engine in swarm mode, but the service fails after a few seconds with the error task: non-zero exit (137). How to debug this issue and where can i find additional logs? docker logs -f containerName does not have any errors. so was not sure how I could debug this.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version :

Output of docker info :

The text was updated successfully, but these errors were encountered:

pacoxu commented Oct 9, 2016

you may try to run ‘docker log stopped_container_id’

You can run «docker ps -a» to get the container id. Or you can ‘docker ps your_service’ and ‘docker inspect task_id’ => Get the container id and get its log.

saithala commented Oct 9, 2016

Thanks for the answer. Unfortunately none of these have any error message. No clue on what’s going on here

saithala commented Oct 9, 2016 •

I tried looking at the docker daemon logs and although I dont know if its related, just putting it out here so that docker members can predict what might be wrong..
dockerd: time=»2016-10-09T12:35:51.065725323-04:00″ level=error msg=»fatal task error» error=»network my-network not found» module=taskmanager task.id=ahym0290e69o51oz3jkhouhfl

After I saw this error, i ran the command docker network ls and I did see the custom overlay network (that I created) does exist. Not sure what is going on here.. Please help..

After some hours of debugging, i tried recreating the custom overlay network , the error message in the daemon logs is just
level=error msg=»fatal task error» error=»task: non-zero exit (137)» module=taskmanager task.id=0yrw0n0jmmqqdno5y4cdtnudv

How do I know what caused the task to return non zero exit code ?

thaJeztah commented Oct 10, 2016

@saithala what is running in that container? Can you provide steps to reproduce? Do you get this error with any image, or just that image?

saithala commented Oct 10, 2016

Thanks for your response @thaJeztah .I researched and found out that exit code 137 means out of memory. I was trying to launch services and wanted to limit the amount of memory allocated by using —limit-memory flag. Could you please let me know the usage of that flag? For Ex. if I want to limit memory to 2G, should it be —limit-memory 2Gb ?

thaJeztah commented Oct 10, 2016

@saithala afaik, currently that flag accepts values in bytes so just a numeric value (no GB , B , etc)

saithala commented Oct 10, 2016 •

Thanks @thaJeztah . The documentation says —limit-memory value [ Limit Memory (default 0 B) ]
So I thought it takes B/Mb/Gb as part of the value. I think the use of Gb was causing the service to not start. Its helpful if we have some samples of the usage of each flag.
Closing this

Источник

What is Docker container exit code 137? #21083

Comments

synergiator commented Mar 10, 2016

Is there an official exit code list?

The text was updated successfully, but these errors were encountered:

GordonTheTurtle commented Mar 10, 2016

If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.

If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.

For more information about reporting issues, see CONTRIBUTING.md.

You don’t have to include this information if this is a feature request

(This is an automated, informational response)

BUG REPORT INFORMATION

Use the commands below to provide key information from your environment:

docker version :
docker info :

Provide additional environment details (AWS, VirtualBox, physical, etc.):

List the steps to reproduce the issue:
1.
2.
3.

Describe the results you received:

Describe the results you expected:

Provide additional info you think is important:

Источник

Container crashes with code 137 when given high load #22211

Comments

Omnipresent commented Apr 21, 2016 •

I have a docker container running python Flask REST service. The REST service takes a URL of an image as input, does processing on it using OpenCV, and returns the result. When I test the container with sending 15 to 20 requests, it eventually crashes with error code 137

Output of docker version :

Output of docker info :

Additional environment details (AWS, VirtualBox, physical, etc.):
VirtualBox on OSx

Steps to reproduce the issue:

1. Send 15 to 20 requests to the server
2. constantly keep running ps -aux inside the container to monitor the metrics. Also monitor the statistics with docker stats

Describe the results you received:
Eventually the container crashes:

Crash monitoring with ps -aux

Crash monitoring with docker stats . I’m showing the last few stats before the container crashes.

Describe the results you expected:
I expected the container to not crash.

Additional information you deem important (e.g. issue happens only occasionally):
I believe this happens every time the %MEM is between 86% to 87% for an extended period of time.

The text was updated successfully, but these errors were encountered:

phemmer commented Apr 21, 2016

Sounds like you’re likely getting hit by the OOM killer. Run dmesg and look for messages from the kernel telling you it’s killing your stuff.

Exit code 137 means your process was sent a SIGKILL. http://tldp.org/LDP/abs/html/exitcodes.html

Omnipresent commented Apr 21, 2016

I can see the following in the output

Out of memory: Kill process 10183 (python) score 873 or sacrifice child
Killed process 10183 (python) total-vm:3026880kB, anon-rss:868740kB, file-rss:0kB
docker0: port 1(vethbd301f9) entered disabled state
veth99072dc: renamed from eth0
docker0: port 1(vethbd301f9) entered disabled state

So it does look like am OOM killer. But my question is, how do I solve this? I’ve tried giving a memory limit while running the container but that doesn’t help either.

FWIW and in-case this is not considered a docker issue, I’ve opened a question on SO about this http://stackoverflow.com/questions/36759132/why-does-docker-crash-on-high-memory-usage

cpuguy83 commented Apr 21, 2016

Just looks like you’re running out of memory on the machine.
Is it normal for your python process to consume a gig of memory under such a load?

thaJeztah commented Apr 21, 2016

Setting a memory-limit only prevents the kernel from randomly killing other processes (e.g. the docker daemon), but doesn’t prevent your container from running out of memory. If the process in your container requires more memory, increase the amount of memory for the Virtual Machine (or add swap), or tweak / change the application to consume less memory.

Note that this is not different from running the application outside of a docker container; if the system is running out of memory, the kernel will kill processes. With Docker, those processes just happen to be in a container.

I’m closing this issue, because I don’t think there’s a bug here, but feel free to continue the discussion

Источник

Docker Container exited with code 137

When attempting to fire up a bunch of docker containers using docker-compose on a Mac, one of the containers was randomly exiting with the exit code 137. The message would look something like this:

My first reaction was to try to quickly run docker exec -it container-id /bin/bash and then take a look at the logs, but before I was quick enough to do that I decided to look into exit code 137.

As it turns out this code is commonly associated with Docker for Mac not having enough RAM allocated to it. More specifically the Linux OOM (out of memory) Killer, kicked in and terminated the process.

So it is an easy fix, go to the Docker Menu and select Preferences then the Advanced tab and increase the Memory.

My docker memory was set to 2.0 GB (perhaps the default), I increased it to 4.0 GB and problem solved.

You may also be able to get around this problem by setting the —memory limit when starting a container, or if you are using docker compose, you can do something like this in your docker-compose.yml file:

Like this? Follow me ↯

Docker Container exited with code 137 was first published on January 18, 2018.

Comments

You saved my day with your solution.
I was trying to run druid in docker and received this «container_name exited with code 137» many times. With your solution, the problem went away.

Thanks a lot. Worked like a charm.

I have this mysterious problem for days. With the same conf other VM run well. Turns out it’s a RAM problem. Solved after resizing from 2 to 4 GB.

Thanks for sharing, first hit in google and on the spot.

After latest docker update on mac the settings were reset causing this issue for me.

Источник

error Command failed with exit code 137 #1177

Comments

mashirozx commented Dec 2, 2017

Here when install it, it shows:

On Ubuntu 17.04, with: nodejs v8.9.1; yarn 1.3.2; npm 5.5.1

Sorry, I’m new to Node, and the official site had a bad connection in my country, so I need to install StackEdit locally. Thanks,

The text was updated successfully, but these errors were encountered:

mashirozx commented Dec 2, 2017

P.S. Someone said it’s caused by the memory limit of the machine 🙁
Is that true? My server has 1G RAM, is it enough? 😢

benweet commented Dec 21, 2017

That’s right, this is provoked by linux’s oom killer. Try to increase the ram or the swap.

thinsoldier commented Sep 21, 2018 •

How much should it be increased to if 1GB is not enough?

mashirozx commented Sep 22, 2018

@thinsoldier I tried a 4 GB machine, still not enough :-/

thinsoldier commented Sep 22, 2018 •

@mashirozx I encountered the error while running the new vue-ui which uses some kind of server to serve its gui and then webpack dev server to serve my project.

At 512 megs the dev server failed to start, even with the vue-ui gui turned off.

With yarn instead of npm at 512 megs both servers started but if I navigated within the vue-ui gui site the dev server would crash.

At 1GB I would see errors mentioning memory in the terminal of the dev server but it wouldn’t crash.

It seems that 2gb is enough for me but I think the switch from npm to yarn might have something to do with it.

Источник