Escaping backslash in windows paths passed to unix programs

I am trying to escape backslashes in cygwin, but it seems almost impossible I have tried a lot of things, but none work right..

More specifically I need to get a command to receive input in bash without losing the backslash characters. every time I try to pass an argument, the backslashes always disappear, destroying my input. And I don’t know how I can tell it to just leave the backslashes on the input alone.

I have tried the following but neither seems work

By the way, I want to be able to type C:\Users\Ted\Music\Enigma without quotes. One of those aliases works when using quotes.

6 Answers 6

Please read the section titled QUOTING in bash(1) ( man bash ).

In your first example,

the shell interprets (and then removes) backslashes within double quotes, so the first echo command only sees the string C:UsersTedDocumentsUnixScripts .

In your second example,

the single quotes correctly protect the backslashes, as you can see by running just the first echo command, without the | xargs echo .

In all of your remaining examples,

since the argument ( C:\Users\Ted\Music\Enigma ) is unquoted, once again the shell interprets and removes the backslashes before the command ( cyg0 , cyg1 , . ) ever sees them. And.

I want to be able to type C:\Users\Ted\Music\Enigma without quotes.

Sorry, it can’t be done. If you don’t put the backslashes in single quotes, the shell will interpret each one as a quoting character for the character that follows it, then remove the backslash.

You have a few options:

Use single quotes on the command line, to protect the backslashes:

Quote each backslash character separately, by doubling it, e.g.

Use forward slashes instead:

Option 3 could really be your best solution. Unfortunately the backslash character is just a very special character for all Unix shells, and this causes a lot of quoting hassles when dealing with DOS/Windows paths. But what’s not widely known is that DOS/Windows is also perfectly happy to use the forward slash ( / ) as its path separator. Try it.

fl_filename_absolute mishandles double-backslash on Windows #141

Comments

fire-eggs commented Sep 17, 2020

Windows 10 / FLTK 1.4 / Built locally with Microsoft Visual Studio 2019

I’ve been having problems with fl_filename_absolute on Windows. Here is a sample:

Stepping into the fl_filename_absolute() call, I find that any invocation of isdirsep inside Fl_WinAPI_System_Driver.cxx is using the version of isdirsep in filename_absolute.cxx. The latter does not accept the double-backslash.

My fix was to declare each instance of isdirsep to be static . Then Fl_WinAPI_System_Driver::filename_absolute would use the correct version of isdirsep .

As of this writing, I have not yet tested the change on Linux.

The text was updated successfully, but these errors were encountered:

fire-eggs commented Sep 17, 2020 •

I have just confirmed that my change also fixes another problem I’ve been having:

Before my fix, the results would be: E:/temp/E:\\temp\\log\\log.txt .

Albrecht-S commented Sep 18, 2020

My fix was to declare each instance of isdirsep to be static. Then Fl_WinAPI_System_Driver::filename_absolute would use the correct version of isdirsep.

Thanks for the report and the proposed fix. Can you please post your patch here as a git diff (or diff -u ) file? TIA

fire-eggs commented Sep 18, 2020

Please find below the two git diff files. I hope I’ve done it right — let me know if there are any questions/concerns.

I took the liberty of including the changes for both this issue (#141) and my next issue (#142). Added comments to clarify.

fire-eggs commented Sep 18, 2020

Here’s my test program.

erco77 commented Sep 19, 2020 •

Hmm, I could swear I ran into this stuff once before in a bug report, and did a deep dive into the parsing of pathnames down deep into the linux system calls, and I think I went into the windows stuff too. Ended up rewriting the code, but it wasn’t released because of some unfinished business, can’t remember what.

Lemme sniff around, it might be in an unresolved STR or some such.. it wasn’t that long ago.

erco77 commented Sep 19, 2020

In this case I was working on the OPPOSITE function, fl_filename_relative(), which had similar issues.
Built a pretty good test suite at the time too, similar to your tests OP.
The basis of my work went down the linux realpath(3) rabbit hole. Looks like I /only/ addressed unix, and saw Windows as a whole other can of worms because of UNCs and drive maps (X:). I know I put a bit of work into that too, but not sure if I kept it.

erco77 commented Sep 19, 2020

Of interest, the realpath(1) function would do what this issue wants.
If you do something like (bash script here):

..the result is the proper «cleaned up» absolute path, with unncessary «.» and «..»s removed.

In unix we have realpath(3), and under the Windows API there’s the GetFullPathNameA() function.
So between those and some of the code I proposed in STR 3441, there might be a way to clean all this up.
Might be a lotta work, or might be a little. Either way a lot of testing is needed, cause there’s a lot of side cases; in unix symlinks and multiple slashes, in windows uncs, drive maps, fronts vs backs, symlinks (MKLINK /?), junctions, and lord knows what else.

Beware of dragons in the OS functions; in the case of linux, realpath(3) doesn’t work with fictional pathnames, but realpath(1) can with the -m flag, so there’s some smarts in there to avoid realpath(3), doing some pathname parsing itself.

Albrecht-S commented Sep 19, 2020

. under the Windows API there’s the GetFullPathNameA() function.

@erco77 Just commenting on this tiny part: please keep in mind that we need to use GetFullPathNameW() , i.e. the «wide character» version with path names converted from UTF-8 to «Windows wide chars» and vice versa. This applies to all other file handling functions as well.

@fire-eggs Thanks for the patches and the demo program.

. git diff files. I hope I’ve done it right .

I believe the patches can be applied (didn’t test though). Hint: if you edit directly in the Git working directory you can just run git diff or (if you want to restrict the diff to only certain files) git diff —
.

fire-eggs commented Sep 19, 2020

My full test program also exercises the function using a network share, but I thought including it would complicate things.

An additional thought, since this topic has opened up further . Any feeling about having fl_absolute_filename consistently return Unix-style path separators on Windows? Right now, the function may return a mix of path separators (depending on the input path).

erco77 commented Sep 19, 2020 •

Regarding returning unix style paths, either make it an option flag that perhaps defaults to unix style, or offer a separate method to change from one to the other. Some users might need the backslash version, as some windows browsers can’t deal with frontslash paths.

fire-eggs commented Sep 19, 2020

Some users might need the backslash version, as some windows browsers can’t deal with frontslash paths.

Then there’s another bug, as the existing version consistently returns the current working directory portion of the output using Unix-style separators.

erco77 commented Sep 19, 2020 •

Not a bug, since the docs don’t say one way or the other what style it returns.
However, I think the caller might want to choose.

My thinking is we could provide a flag that forces it one way or the other, and document that. Perhaps for backwards compatibility (where the style of paths used as input affects what is returned), it can be a three state flag, with the default being the current behavior, and the other two states being backs or fronts.

Trouble with that though is then ALL fltk functions dealing with paths should provide such an option, which might be bad/confusing.

So supplying a function that converts an input path to fronts or backs might be good. Though that’s kind of trivial these days with the std::string stuff to just replace /’s with \\’s and vice versa.

In which case we could document the current behavior (possible mix of slash types), or force unix style. Not sure what the right call is in that case. Should probably choose one and document what it returns.

Curious what Albrecht thinks..

Albrecht-S commented Sep 19, 2020

First thought: if it ain’t broken, don’t fix it. Did anybody complain? Well, that’s only a first thought.

Not a bug, since the docs don’t say one way or the other what style it returns.

However, I think the caller might want to choose.

Why? Serious. I think there are two points:

1. Many of the Windows API functions allow both front and back slashes. If not, please correct me.
2. If this would have been an issue in the past we’d have seen more «complaints» by users.

I’m always for fixing bugs, and if it’s possible, make code and behavior consistent. But if we don’t know what the correct behavior is, what can we do?

I think we should try to avoid inconsistencies (and bugs for sure) as far as possible but leave everything else as-is unless it’s really easy to fix. I don’t think that adding a user option to decide what behavior they want is not «really easy», particularly if this would have to be added to «ALL fltk functions dealing with paths» as you wrote above.

OTOH, if there’s a volunteer and the API is simple and hopefully backwards compatible and (what else) . I wouldn’t object but I also wouldn’t (be able to) contribute much.

Albrecht-S commented Sep 19, 2020

So supplying a function that converts an input path to fronts or backs might be good.

That’s simple, see for instance test/demo.cxx: fix_path() (only one direction). And if it’s so simple users can do it themselves, don’t they? But .
👍 if this helps to avoid a lot of work (with new bug potential) in FLTK and an additional option (API) to make it work.

erco77 commented Sep 19, 2020 •

Sure; there’s two main places in windows where front slashes don’t work:

I actually run into this enough, that in my commercial application I provide the user with two ‘Copy’ options for popup menus on input and text widgets to provide copy/paste:

..This allows the user to take a highlighted path from execution logs, and copy/paste them into DOS windows as arguments to either DOS (e.g. DIR and COPY commands) and Windows file browsers, or Unix shells. DOS tools only can handle backslashes because they use fronts as command line separators; e.g. COPY/XCOPY/MORE/TYPE/DIR, etc etc.

While the WIN32 API allows both, higher level software (DOS, native folder browsers) overloads fronts for other things, creating a conflict, and those situations might be relevant to an app’s needs.

Arguably we should provide a way for fronts backs for these and possibly other reasons, as users might not have std::string available (those embedded platforms that don’t support STL/templates), and just because there’s more than one possibility of return.

At very least we should /probably/ be consistent about what we return and document it, as returning a mix of slashes would I think be unexpected.

fire-eggs commented Sep 20, 2020

If this would have been an issue in the past we’d have seen more «complaints» by users.

Digging into history, I see the slashes behavior is the same as FLTK 1.0. So yeah, if no-one has raised the question all this time, I’d say it’s probably a non-issue IMO.

The original issue here (#141) was introduced in 1.4 during the «drivers» split. My next issue (#142) would technically belong in the «no one has complained up until now so it probably shouldn’t be changed» category.

re: Unix style slashes not working with Native Windows File choosers

Interesting, I’ll have to try that!

erco77 commented Sep 20, 2020 •

It’s so trivial to replace fronts backs that no one bothers to report it, but it probably gets through beta testing unnoticed, eventually a customer notices a problem, the programmer provides a way to normalize the paths and releases a fix.

Being undocumented, it’s a pitfall waiting to happen. If we at least document it, it could save the programmer a fix cycle.

Documenting might be as simple as «Under Microsoft Windows, slashes in the path returned may have fronts or backs or a mix of both, depending on circumstances», or we can force the slashes as fronts or backs (not sure which is best) and documenting that. We should probably be clear one way or the other.

Ran into a similar issue (#122) recently with the native file chooser. For that, the input path determined the returned slash type, and if there wasn’t enough info from the input path, native format slashes were returned (backs).

How to get rid of double backslash in python windows file path string? [duplicate]

I have a dictionary:

I iterate over it:

But I get this error:

It can’t find my file. Why does it think there are double backslashes in file path?

5 Answers 5

The double backslash is not wrong, python represents it way that to the user. In each double backslash \\ , the first one escapes the second to imply an actual backslash. If a = r’raw s\tring’ and b = ‘raw s\\tring’ (no ‘r’ and explicit double slash) then they are both represented as ‘raw s\\tring’ .

For clarification, when you print the string, you’d see it as it would get used, like in a path — with just one backslash:

And in this printed string case, the \t doesn’t imply a tab, it’s a backslash \ followed by the letter ‘t’.

Otherwise, a string with no ‘r’ prefix and a single backslash would escape the character after it, making it evaluate the ‘t’ following it == tab:

So in the PDF path+name:

Double backslashes are due to r , raw string:

It is used because the \ might escape some of the characters.

It doesn’t. Double backslash is just the way of the computer of saying backslash. Yes, I know this sounds weird, but think of it this way — in order to represent special characters, backslash was chosen as an escaping character (e.g. \n means newline, and not the backslash character followed by the n character). But what happens if you actually want to print (or use) a backslash (possibly followed by more characters), but you don’t want the computer to treat it as an escaping character? In that case we escape the backslash itself, meaning we use a double backslash so the computer will understand it’s a single backslash.

It’s done automatically in your case because of the r you added before the string.

save yourself from getting a headache you can use other slashes as well. if you know what I saying. the opposite looking slashes.

you’re using now PDF = ‘C:\Users\user\Desktop\File_%s.pdf’ %item

PDF = ‘C:/Users/user/Desktop/File_%s.pdf’ %item

** it won’t be treated as a escaping character .