Unable to find removal id 66304 linux

linux spec user: Unable to find user xxx #22323

Comments

edannenberg commented Apr 26, 2016 •

Since upgrading to Docker 1.11.0 the following won’t work anymore:

You also get some funny behavior when using a non root user that exists in the image’s /etc/passwd :

Depending on your system the above might actually work fine or output some other random user name, on my host /etc/passwd has no entry for uid 37.

Apparently —user is now set before the /etc/passwd file of the host is mounted in the container. Can this functionality be restored in a future release? If not, mounting the host’s /etc/passwd should not be allowed as it will just lead to problems down the road.

The use case for this is mainly for local development, where, for example, you would like to run a css preprocesser watching your source files and want to avoid any file permission head aches.

The text was updated successfully, but these errors were encountered:

aaronlevin commented Apr 26, 2016 •

I am also experiencing this:

GordonTheTurtle commented Apr 26, 2016

The best way to get notified of updates is to use the Subscribe button on this page.

Please don’t use «+1» or «I have this too» comments on issues. We automatically
collect those comments to keep the thread short.

The people listed below have upvoted this issue by leaving a +1 comment:

thaJeztah commented Apr 26, 2016

Why not use the numeric ID of the user, instead of a user-name; doing so, you don’t have to bind-mount /etc/passwd , because that file is only needed to convert the username to its uid/gid inside the container.

the only thing you’ll miss is having a «nice» name visible inside the container, but I suspect the reason you’re doing this is for setting the right permissions, and not to show a user’s name.

edannenberg commented Apr 26, 2016

Interesting, I actually tried that, but with docker-compose only, which complained with a schema validation error as it expects user: to be a string. So I assumed it actually had to be a user name. Turns out putting the uid into quotes will do the trick.

@thaJeztah can we a expect a fix for this? I certainly can live without one for now but still it’s somewhat ugly and might lead to unexpected behavior with some applications.

thaJeztah commented Apr 26, 2016

@edannenberg it would be good to know the reason this changed, it’s possible this change is actually related to creating volumes with the right permissions (i.e., if I start a container as user 123, and create a volume, then the volume should be owned by 123). If that is the reason, then I’m not sure we’d be able to revert to the old behavior

HackToday commented Apr 27, 2016

It seems docker master not have such issue,
also 1.11.1 not include such change in oci opencontainers/runc@3cfff67

HackToday commented Apr 27, 2016

thaJeztah commented Apr 27, 2016

@HackToday yeah, daemon is present by default in busybox, so in that case, it’s using /etc/passwd from the busybox image, not the bind-mounted /etc/passwd file

justincormack commented Apr 28, 2016 •

The opencontainers specification requires that the user id passed to runc in Linux is a number, and resolving user names is left to the layers above, see https://github.com/opencontainers/runtime-spec/blob/master/config.md#linux-user

Therefore since we use runc the name has to be derived before the mounts take place, so bind mounting /etc/passwd will be too late. Although this used to work, I think that it was just because the ordering happened to be like that, and I don’t think it can really be seen as a regression, and I don’t think it is sensible to fix it, as numeric ids are what the kernel really uses. (Despite the fact it has been a convenient use case!)

thaJeztah commented Apr 28, 2016

Thanks for looking into that @justincormack. Given that information, this is not an easy fix, and not something worth changing. I’ll close this.

frncmx commented May 6, 2016 •

@thaJeztah the docker run -it —rm —user $(id -u) busybox solution works. However, there is still 1 use case for /etc/passwd , if you don’t mind I describe it below.

If you need to use SSH protocol inside a container (e.g., git clone), you will need the password file.

So, I’ve modified our build environment to something like:
docker run —rm -it -v $SSH_AUTH_SOCK:/ssh-agent -e SSH_AUTH_SOCK=/ssh-agent -u $UID -v /etc/passwd:/etc/passwd:ro

That way I’ve my ssh-agent inside the container and my user is known by the contianer, which is a must for SSH.

varunagrawal commented Feb 2, 2018

The docs over at https://docs.docker.com/engine/reference/run/#user should be updated to reflect that only UIDs work and usernames don’t.

kingdonb commented Sep 14, 2018 •

They still haven’t been updated (the docs, mentioned in the previous comment.) But is that really the answer? Maybe we are talking about two different bugs. I think the original reporter was describing an (persistent) issue with bind mounts, but some of us are experiencing an intermittent issue that is not related to bind mounts.

I have a k8s cluster with three nodes, and right now only one of the nodes appears to refuse «docker exec -u» with a username rather than UID (I was mistaken, it’s not per-node. it appears to be a per-container issue). The thread at https://forums.docker.com/t/unable-to-find-user-root-no-matching-entries-in-passwd-file/26545 has more details about what I’m experiencing (and I think it might be a different bug than described here). but I haven’t bind-mounted passwd and the same container launched on a different AKS node simply accepts the USER (username) or exec -u with a username from the passwd file.

I was able to exec into one of these containers refusing username parameter today by using the UID=0 trick (—user 0) and I can see the passwd file is there, and both root and the other user are certainly listed. Is no matching entries in passwd file. a different issue than the one reported here? Certainly you can still use a username in the USER [param] directive in your Dockerfile, right? Well, even that is not working, as my Readiness check which as far as I know should simply inherit the USER directive from the Dockerfile, also reports no matching entries in passwd file.

I will follow up and report a new issue if I can find a way to repro this, but I’m afraid I’m not using the docker runtime directly, and I’m honestly not sure if it’s what powers my AKS cluster at all (my AKS appears to be on Docker v1.13.1). This seems like a long-standing bug though (and it’s probably not the specific issue this report was about, sorry for hijacking.)

Источник

Troubleshooting

Here we provide help for some of the problems that DVC user might stumble upon.

Failed to pull data from the cloud

Users may encounter errors when running dvc pull and dvc fetch , like WARNING: Cache ‘xxxx’ not found. or ERROR: failed to pull data from the cloud . The most common cause is changes pushed to Git without the corresponding data being uploaded to the DVC remote. Make sure to dvc push from the original project , and try again.

Too many open files error

A known problem some users run into with the dvc pull , dvc fetch and dvc push commands is [Errno 24] Too many open files (most common for S3 remotes on macOS). The more —jobs specified, the more file descriptors need to be open on the host file system for each download thread, and the limit may be reached, causing this error.

To solve this, it’s often possible to increase the open file descriptors limit, with ulimit on UNIX-like system (for example ulimit -n 1024 ), or increasing Handles limit on Windows. Otherwise, please try using a lower JOBS value.

Unable to find credentials

Make sure that you have your AWS credentials setup either through usual AWS configuration or by setting access_key_id and secret_access_key with dvc remote modify .

Unable to connect

Make sure you are online and able to access your AWS S3 endpoint, or endpointurl if explicitly set with dvc remote modify .

Bucket does not exist

Make sure your bucket exists in the correct region and/or endpointurl (see dvc remote modify ).

Unable to detect cache type

Unable to detect supported link types, as the cache directory doesn’t exist. It is usually created automatically by DVC commands that need it, but you can create it manually (e.g. mkdir .dvc/cache ) to enable this check.

Unable to acquire lock

You may encounter an error message saying Unable to acquire lock if you have another DVC process running in the project. If that is not the case, it usually means that DVC was terminated abruptly and manually removing the lock file in .dvc/tmp/lock should resolve the issue.

If the issue still persists then it may be the case that you are running DVC on some network filesystem like NFS, Lustre, etc. If so, the solution is to enable core.hardlink_lock which can be done by running following command:

Cannot add files in symlinked directory

DVC only supports symlinked files as valid targets for dvc add . If the target path is a directory symlink, or if the target path contains any intermediate directory symlinks, dvc add will fail.

No possible cache types

You may encounter this error if DVC cannot find a valid file link type to use when linking data files from cache into your workspace. To resolve the issue, you may need to reconfigure DVC to use alternative link types which are supported on your machine.

After reconfiguring cache types, you can re-link data files in your workspace using:

HTTP Git authentication is not supported

Experiment sharing commands accept a git_remote argument. In order to access the Git remote, you may need to authenticate for write ( dvc exp push ) or read ( dvc exp list , dvc exp pull ) permissions.

DVC does not currently support authentication with Git credentials. This means that unless the Git server allows unauthenticated HTTP write/read, you should use an SSH Git URL when listing, pulling or pushing experiments.

Источник

Solution to bug ‘Error, unable to determine correct filename for 64bit linux’ #5672

Comments

LeydenJar commented Jul 29, 2020 •

Expected Behavior

Ok, so, I’m just getting started with InstaPy, but I’m supposing that when you start a session, we check either the geckodriver we have is the latest version or not, and if it is not the case then the expected behavior is to download the new one and start it.

Current Behavior

Turns out that currently, with geckodriver 0.27, you get this error on linux 64 bit when trying to start an InstaPy session.

‘Error, unable to determine correct filename for 64bit linux’

Possible Solution (optional)

The reason is that the code in the webdriverdownloader dependency, wich gets the list of possible geckodriver versions to download and then filters for the ones that match your os and if it is 32 or 64 bit. Then it expects to have only one left in the list, wich it downloads. Turns out there are 2 now, with 2 extensions(something like ‘.tar.gz’ and ‘.tar.gz.asc’). So the solution is to remove one of them so that we don’t fall on the exception.
The one we want to remove is the one that ends on .asc cause apparently the program is expecting the other one

You can do that by changing a single line, i just made a pull request here where you can check it out: leonidessaguisagjr/webdriverdownloader#12

After that all works fine.

The text was updated successfully, but these errors were encountered:

operador777 commented Jul 29, 2020 •

Thanks man, tested and working.

gixxerfrank commented Jul 30, 2020

I am having the same issue running on Digital Ocean and am unsure of how exactly to apply this fix

LeydenJar commented Jul 30, 2020 •

@gixxerfrank So what you should do is go to your virtual environment or wherever your pip is installing your project dependencies and then find the folder webdriverdownloader. The path may be something like this:

Then, inside this folder you have to edit the file ‘webdriverdownloader.py’

Arround the line 320 there’s a line like this:
filename = [name for name in filenames if os_name + bitness in name ]

What you want to do is change it to something like this:

filename = [name for name in filenames if os_name + bitness in name and name[-3:] != ‘asc’ ]

filename = [name for name in filenames if os_name + bitness in name and name[-7:] == ‘.tar.gz’ ]

elissonnog commented Jul 30, 2020

Works well, thanks a lot

colinmasters commented Jul 31, 2020

Nice man thanks!

gixxerfrank commented Jul 31, 2020

@gixxerfrank So what you should do is go to your virtual environment or wherever your pip is installing your project dependencies and then find the folder webdriverdownloader. The path may be something like this:

Then, inside this folder you have to edit the file ‘webdriverdownloader.py’

Arround the line 320 there’s a line like this:
filename = [name for name in filenames if os_name + bitness in name ]

What you want to do is change it to something like this:

filename = [name for name in filenames if os_name + bitness in name and name[-3:] != ‘asc’ ]

filename = [name for name in filenames if os_name + bitness in name and name[-7:] == ‘.tar.gz’ ]

Thanks so much, it works great!

samuellmiller commented Aug 8, 2020

I am having the same issue but when I deploy it in Heroku. How should I do?

LeydenJar commented Aug 14, 2020

@samuellmiller The problem is in the webdriverdownloader module. So, I’m not sure on how Heroku works but you must have a way to connect to the server or something. What you have to do then is find out where pip is installing stuff(for example running ‘pip show webdriverdownloader’ and looking at location) then you go and do the edits that I’ve mentioned.

ansango commented Aug 15, 2020 •

filename = [name for name in filenames if os_name + bitness in name and name[-3:] != ‘asc’ ]

filename = [name for name in filenames if os_name + bitness in name and name[-7:] == ‘.tar.gz’ ]

aramodi commented Aug 15, 2020

Thank you very much!

lgv-0 commented Aug 16, 2020

Verified on Ubuntu 16.04.6 LTS Server, thanks!

kmessilj commented Sep 6, 2020

I am having the same issue but when I deploy it in Heroku. How should I do?

Did you find any solution for heroku? I solved the issue locally but unable to solve it on heroku. I’m able to make changes, but dynos does not make permanent changes. And hence it is just for time being.

Courxge commented Sep 27, 2020

i cannot find the webdriver folder.. i am very inexperienced with coding and attempted to follow the steps but i cannot find this folder anywhere.. I’ve located the InstaPy folder but inside of that are assets, db, and logs, none of which lead to webdriver

Courxge commented Sep 27, 2020

i cannot find the webdriver folder.. i am very inexperienced with coding and attempted to follow the steps but i cannot find this folder anywhere.. I’ve located the InstaPy folder but inside of that are assets, db, and logs, none of which lead to webdriver

i am using ubuntu btw

thyarles commented Sep 28, 2020 •

It was fixed on version 0.9.0. Just update your version with

thursd4y commented Oct 27, 2020 •

When starting instapy manually on my raspberry everything works fine, but doing it via cronjob does not work.

I just tried both versions of the proposed script. Both then yield the new following error:
Traceback (most recent call last):
File «/home/pi/-/interact_plant.py», line 39, in
session = InstaPy(username=insta_username, password=insta_password, headless_browser=True)
File «/home/pi/.local/lib/python3.7/site-packages/instapy/instapy.py», line 333, in init
self.logger,
File «/home/pi/.local/lib/python3.7/site-packages/instapy/browser.py», line 130, in set_selenium_local_session
options=firefox_options,
File «/home/pi/.local/lib/python3.7/site-packages/selenium/webdriver/firefox/webdriver.py», line 164, in init
self.service.start()
File «/home/pi/.local/lib/python3.7/site-packages/selenium/webdriver/common/service.py», line 76, in start
stdin=PIPE)
File «/usr/lib/python3.7/subprocess.py», line 775, in init
restore_signals, start_new_session)
File «/usr/lib/python3.7/subprocess.py», line 1522, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 8] Exec format error: ‘/home/pi/InstaPy/assets/geckodriver’

mikegrep commented Nov 4, 2020

——> Installing requirements with pip
! error fetching custom buildpack https://github.com/LeydenJar/webdriverdownloader/tree/fix_linux_64
! Push failed

Issue when deploy in heroku

saribricka commented Mar 15, 2021

I tried @LeydenJar solution but it isn’t working

RomjanHossain commented May 5, 2021 •

@saribricka make sure you did it right.

goto line 320 of webdriverdownloader.py (

/.local/lib/python3.9/site-packages/webdriverdownloader/webdriverdownloader.py )

edit this line filename = [name for name in filenames if os_name in name] to filename = [name for name in filenames if os_name + bitness in name and name[-3:]!=’asc’]

Done.

I was having the same issue on Manjaro Linux.

another solution is: create a virtual environment and run instapy on that.

ITzMirko commented May 6, 2021

$ python3 /home/kts/Scaricati/gianscript.py
InstaPy Version: 0.6.13
.. .. .. .. .. .. .. ..
Workspace in use: «/home/kts/InstaPy»
Error, unable to determine correct filename for 64bit linux
Traceback (most recent call last):
File «/home/kts/Scaricati/gianscript.py», line 39, in
session = InstaPy(username=insta_username,
File «/home/kts/.local/lib/python3.8/site-packages/instapy/instapy.py», line 325, in init
self.browser, err_msg = set_selenium_local_session(
File «/home/kts/.local/lib/python3.8/site-packages/instapy/browser.py», line 122, in set_selenium_local_session
driver_path = geckodriver_path or get_geckodriver()
File «/home/kts/.local/lib/python3.8/site-packages/instapy/browser.py», line 38, in get_geckodriver
sym_path = gdd.download_and_install()[1]
File «/home/kts/.local/lib/python3.8/site-packages/webdriverdownloader/webdriverdownloader.py», line 174, in download_and_install
filename_with_path = self.download(version,
File «/home/kts/.local/lib/python3.8/site-packages/webdriverdownloader/webdriverdownloader.py», line 129, in download
download_url = self.get_download_url(version, os_name=os_name, bitness=bitness)
File «/home/kts/.local/lib/python3.8/site-packages/webdriverdownloader/webdriverdownloader.py», line 324, in get_download_url
raise RuntimeError(info_message)
RuntimeError: Error, unable to determine correct filename for 64bit linux

RomjanHossain commented May 7, 2021

read previous comments bruh @ITzMirko

trankov commented May 28, 2021

No, guys, now the changing of line 320 as it shown above doesn’t work.
It seems Mozilla renewed filenames again (last release was Apr 29th 2021).
Everything says that problem will appear again and again.

Let me explain quickly how to fix it every time.

Now let’s see at webdriverdownloader.py
In lines 313, 314 it tries to find out the correct filename for download:

(The json_data[‘assets’] contains list of dicts <'name', 'browser_download_url'>, then the script extracts to filename items which contains OS name).
F.e. on my MacOS 11.3.1 filenames are:

And then filename :

After this we comes to our beloved string 320:

Here in my case os_name + bitness == ‘macos64’ , and, of course, it cannot find it. The filename == [] and line 321 raises an Error.
Of course, we can just hardcode the right name of file:

Or, we can add more common condition which is checking whether list item contains both ‘mac’ and ’64’ :

Now it can find and download the binary. But there is another problem, it’s a file for M1 CPU 🙂 My Mac is Intel.

Anyway, now you know how and where to fix this issue. I will not explain the solution for my precise platform*, because we need the common recipe for everyone. Unfortunately, it’s a handwork. Fortunately, we know what to do. Just change the condition at the end of the line 320 for your own (also I hope my solution will work correctly for all except of Intel Mac users).

*Okay, I just changed the condition to: os_name in name and ‘aarch’ not in name and got the successfull result.

Источник