How to: Save and open files with encoding

You can save files with specific character encoding to support bidirectional languages. You can also specify an encoding when opening a file, so that Visual Studio displays the file correctly.

To save a file with encoding

From the File menu, choose Save File As, and then click the drop-down button next to the Save button.

The Advanced Save Options dialog box is displayed.

Under Encoding, select the encoding to use for the file.

Optionally, under Line endings, select the format for end-of-line characters.

This option is useful if you intend to exchange the file with users of a different operating system.

If you want to work with a file that you know is encoded in a specific way, you can tell Visual Studio to use that encoding when opening the file. The method you use depends on whether the file is part of your project.

If you want to save the project file with encoding, the Save File As option is not enabled until you unload the project.

To open an encoded file that is part of a project

In Solution Explorer, right-click the file and choose Open With.

In the Open With dialog box, choose the editor to open the file with.

Many Visual Studio editors, such as the forms editor, will auto-detect the encoding and open the file appropriately. If you choose an editor that allows you to choose an encoding, the Encoding dialog box is displayed.

In the Encoding dialog box, select the encoding that the editor should use.

To open an encoded file that is not part of a project

On the File menu, point to Open, choose File or File From Web, and then select the file to open.

Click the drop-down button next to the Open button and choose Open With.

Follow Steps 2 and 3 from the preceding procedure.

Change encoding on a per file or per extension basis

I’m using Microsoft Visual Studio Express 2012 for Web. It seems that every file which I open with it gets encoded into UTF-8. For most files which are going to be web-facing, that’s fine. However, I have files in my projects that are specifically for build purposes (e.g., .bat files), which must be encoded in ANSI.

Are there any configuration settings in VS to either designate on a per file or a per extension basis the encoding? Or, if not specify the encoding, at least disable the auto-conversion to UTF-8?

3 Answers 3

Open the problematic file in Visual Studio and.

• On the File menu, click Advanced Save Options .
• In the Encoding dropdown, select Unicode (UTF-8 … or the encoding you require.
• Click OK .

An option to handle the encoding of all files of a given extension on a per open basis can be configured in the Options dialog. See MSDN page on Options, Text Editor, File Extension.

Navigate to Tools > Options > Text Editor > File Extension.

For the bat extension, I selected Source Code (Text) Editor with Encoding. The with Encoding part means that the user will be given options as to what encoding to use when opening the file. The default in this mode is Auto-detect, which preserves the ANSI encoding, if that is what the file already uses. Otherwise, one can explicitly designate it for the individual file.

Unfortunately, it doesn’t seem to remember the setting last used when opening a file, and will thus prompt for an encoding setting every time a file is opened.

Change files’ encoding recursively on Windows?

Does anybody know a tool, preferably for the Explorer context menu, to recursively change the encoding of files in a project from ISO-8859-1 to UTF-8 and other encodings? Freeware or not too expensive would be great.

Edit: Thanks for the answers, +1 for all of then. But I would really like to be able to just right click a folder and say «convert all .php files to UTF-8». 🙂 Further suggestions are appreciated, starting a bounty.

5 Answers 5

You could easily achieve something like this using Windows PowerShell. If you got the content for a file you could pipe this to the Out-File cmdlet specifying UTF8 as the encoding.

Try something like:

I don’t know about from the context menu, but notepad++ allows you to change file encodings and it has a macro option. so you could automate the process

If you import a test.reg file having the following contain

After this you will receive the menu item «convert all .php files to UTF-8» in the context menu of explorer on every directory. After the choosing of the item the batch program C:\TEMP\t.cmd will be started with «php» string as the first parameter and the quoted directory name as the second parameter (of cause the first parameter «php» you can skip if it is not needed). The file t.cmd like

can be used to prove that all this work.

So you can decode the *.php files with any tool which you prefer. For example you can use Windows PowerShell (see the answer of Alan).

If you want that the extension like PHP will be asked additionally you can write a small program which display the corresponding input dialog and then start the Windows PowerShell script.

How do I correct the character encoding of a file?

I have an ANSI encoded text file that should not have been encoded as ANSI as there were accented characters that ANSI does not support. I would rather work with UTF-8.

Can the data be decoded correctly or is it lost in transcoding?

What tools could I use?

Here is a sample of what I have:

I can tell from context (café should be café) that these should be these two characters:

12 Answers 12

EDIT: A simple possibility to eliminate before getting into more complicated solutions: have you tried setting the character set to utf8 in the text editor in which you’re reading the file? This could just be a case of somebody sending you a utf8 file that you’re reading in an editor set to say cp1252.

Just taking the two examples, this is a case of utf8 being read through the lens of a single-byte encoding, likely one of iso-8859-1, iso-8859-15, or cp1252. If you can post examples of other problem characters, it should be possible to narrow that down more.

As visual inspection of the characters can be misleading, you’ll also need to look at the underlying bytes: the § you see on screen might be either 0xa7 or 0xc2a7, and that will determine the kind of character set conversion you have to do.

Can you assume that all of your data has been distorted in exactly the same way — that it’s come from the same source and gone through the same sequence of transformations, so that for example there isn’t a single é in your text, it’s always ç? If so, the problem can be solved with a sequence of character set conversions. If you can be more specific about the environment you’re in and the database you’re using, somebody here can probably tell you how to perform the appropriate conversion.

Otherwise, if the problem characters are only occurring in some places in your data, you’ll have to take it instance by instance, based on assumptions along the lines of «no author intended to put ç in their text, so whenever you see it, replace by ç». The latter option is more risky, firstly because those assumptions about the intentions of the authors might be wrong, secondly because you’ll have to spot every problem character yourself, which might be impossible if there’s too much text to visually inspect or if it’s written in a language or writing system that’s foreign to you.

Get encoding of a file in Windows

This isn’t really a programming question, is there a command line or Windows tool (Windows 7) to get the current encoding of a text file? Sure I can write a little C# app but I wanted to know if there is something already built in?

13 Answers 13

Open up your file using regular old vanilla Notepad that comes with Windows.
It will show you the encoding of the file when you click «Save As. «.
It’ll look like this:

Whatever the default-selected encoding is, that is what your current encoding is for the file.
If it is UTF-8, you can change it to ANSI and click save to change the encoding (or visa-versa).

I realize there are many different types of encoding, but this was all I needed when I was informed our export files were in UTF-8 and they required ANSI. It was a onetime export, so Notepad fit the bill for me.

FYI: From my understanding I think «Unicode» (as listed in Notepad) is a misnomer for UTF-16.
More here on Notepad’s «Unicode» option: Windows 7 — UTF-8 and Unicdoe

If you have «git» or «Cygwin» on your Windows Machine, then go to the folder where your file is present and execute the command:

This will give you the encoding details of all the files in that folder.

The (Linux) command-line tool ‘file’ is available on Windows via GnuWin32:

If you have git installed, it’s located in C:\Program Files\git\usr\bin.

Another tool that I found useful: https://archive.codeplex.com/?p=encodingchecker EXE can be found here

Here’s my take how to detect the Unicode family of text encodings via BOM. The accuracy of this method is low, as this method only works on text files (specifically Unicode files), and defaults to ascii when no BOM is present (like most text editors, the default would be UTF8 if you want to match the HTTP/web ecosystem).