If special characters look broken after a find and replace, such as café turning into café, or apostrophes becoming ’, the file was read in one text encoding and written back in another. The replace itself was fine; the tool (or script) guessed the encoding wrong. You can usually repair the files by reinterpreting them with the right encoding, and you can prevent it by using a method that detects and preserves each file's encoding, or by stating the encoding explicitly in scripts.
This guide explains what the symptoms mean, how to fix files that are already damaged, and how to stop it happening on the next run.
What the symptoms tell you
| You see | Instead of | Most likely cause |
|---|---|---|
| café, über | café, über | UTF-8 text read as ANSI (Windows-1252), then saved |
| ’, “ | Curly quotes | Same: UTF-8 read as ANSI |
| caf� (a diamond with a question mark) | café | ANSI text read as UTF-8; the byte was invalid and replaced |
| caf? | café | Saved in an encoding that cannot hold the character |
|  at the top of a page | Nothing | A UTF-8 byte order mark (BOM) shown as text |
| Spaces between every letter, or Chinese-looking text | Normal text | UTF-16 read as single-byte text, or the reverse |
The first two are by far the most common. One accented character in UTF-8 takes two or three bytes. Read as ANSI, each byte becomes its own character, which produces those familiar à and â sequences.
Why find and replace changes encodings
- Windows PowerShell 5.1 defaults.
Get-Contentreads files without a BOM as ANSI, andSet-Contentwrites ANSI unless you pass-Encoding. A UTF-8 file without a BOM goes in as UTF-8 bytes and comes out as mangled ANSI. - Guessing. A UTF-8 file without a BOM that happens to contain few or no non-ASCII characters looks identical to ANSI. Editors and tools have to guess, and sometimes guess wrong, especially on short files.
- Normalizing on save. Some tools save everything in one default encoding, regardless of what the file was before.
- The replacement text itself. If your search or replacement contains an accented character and the tool's input was in a different encoding, the inserted text can be wrong even if the rest of the file is fine.
Step 1: find out what the file is now
- Notepad (Windows 10 and 11): the status bar at the bottom right shows UTF-8, UTF-8 with BOM, UTF-16 LE, UTF-16 BE or ANSI.
- VS Code: the status bar shows the encoding. Click it for Reopen with Encoding and Save with Encoding.
- Notepad++: the Encoding menu has a dot next to the current encoding, and the status bar shows it too.
Step 2: repair the damaged files
Best option: restore and redo
If you have a backup or version control, restore the files and run the replace again with the encoding handled correctly. This is cleaner than repairing, and guaranteed to be right.
Case A: the file shows é but is otherwise intact
This is UTF-8 text that was decoded as ANSI and then saved again as UTF-8 ("double encoded"). In Notepad++ you can usually reverse it:
- Open the file. The garbled characters show as é and so on.
- Choose Encoding → Convert to ANSI. This turns each garbled character back into its original single byte.
- Choose Encoding → UTF-8 (the plain "Encode in" option, not "Convert to"). This reinterprets those bytes as UTF-8, and café reappears.
- Save. Check a few characters before moving on to the next file.
The difference matters: Convert to rewrites the bytes into a new encoding, while choosing an encoding without "Convert" only changes how the existing bytes are read.
Case B: the file is ANSI but should be UTF-8, and looks right in Notepad
Nothing is broken, only the encoding is different from what a web server or program expects. Open the file, then save it as UTF-8: in Notepad use Save As and pick UTF-8 in the Encoding box; in VS Code use Save with Encoding; in Notepad++ use Encoding → Convert to UTF-8.
Case C: question marks or replacement diamonds
The original bytes were discarded when the file was saved. There is nothing to reinterpret. Restore from a backup, or fix the characters by hand with targeted replacements, such as caf? → café.
Case D: a visible BOM or "headers already sent" in PHP
The file gained a UTF-8 byte order mark. Re-save it as UTF-8 without BOM. In Notepad that option is labeled plain "UTF-8"; in Notepad++ choose Encoding → Convert to UTF-8 (not "UTF-8-BOM").
Step 3: stop it happening again
In PowerShell
Always state the encoding on read and write:
$t = Get-Content $f -Raw -Encoding UTF8; Set-Content $f $n -NoNewline -Encoding UTF8
In Windows PowerShell 5.1 that writes a BOM. To write UTF-8 without one, use [System.IO.File]::WriteAllText($f, $n), or use PowerShell 7, where -Encoding utf8NoBOM is available and UTF-8 without BOM is the default. Files that are genuinely ANSI should be read and written with -Encoding Default in 5.1. Our PowerShell replace guide has the complete commands.
In editors
In VS Code, consider turning on files.autoGuessEncoding if you work with mixed files, and set files.encoding to what most of your files use. Check the status bar on a sample file before a big replace.
With a batch tool
Bulk Replace detects each file's encoding (UTF-8, ANSI or UTF-16) and writes it back the same way, so a folder with a mix of encodings comes out exactly as it went in, apart from your replacements. If you do want to standardize, you can optionally convert every file to a single encoding as part of the run. Bulk Replace backs up every file first, so if something looks wrong, one click restores the originals.
Quick prevention checklist
- Check the encoding of a few sample files before the run.
- Use a method that preserves encodings, or state them explicitly.
- Include an accented word in your test files and check it after a trial run.
- Keep a backup so a bad run is a minor setback, not a repair job.
For background on the encodings themselves, see UTF-8 vs ANSI vs UTF-16. If your problem is that a search finds nothing in some files, encoding may be the reason; see find and replace not finding text.
Frequently asked questions
Why do accented letters turn into odd symbols?
The file was read in one encoding and written in another, often UTF-8 read as ANSI. Each byte of a multi-byte character is then shown as a separate wrong character.
How do I check a file's encoding on Windows?
Open it in Notepad and look at the status bar, which shows UTF-8, UTF-8 with BOM, UTF-16 LE or ANSI. VS Code and Notepad++ also show the encoding.
Can I fix a file that was saved in the wrong encoding?
Often yes, if you reopen it with the encoding it was actually written in and save it again correctly. If characters were replaced with question marks, the original data is lost and you need a backup.
