Reading a CSV file that is not UTF-8
Excel on Windows does not write UTF-8. Pick the encoding instead of getting replacement characters.
Install#
dart pub add csv_plusimport 'package:csv_plus/csv_plus.dart';The symptom#
A name comes back with a replacement character instead of an accent, or a price shows a stray symbol where the euro sign should be. That file is not UTF-8. The Excel CSV export on Western European Windows writes Windows-1252, a single byte encoding, and reading those bytes as UTF-8 either throws or substitutes.
Pick the encoding#
Pass a CsvCharset to any of the byte decoders:
import 'package:csv_plus/csv_plus.dart';
final rows = const CsvCodec().decodeBytes(
bytes,
charset: CsvCharset.windows1252,
);Three encodings are built in, with no extra dependency:
CsvCharset.utf8, the default. A leading byte order mark is stripped, and a malformed byte is replaced rather than thrown so one bad byte does not lose the file.CsvCharset.latin1, also called ISO-8859-1. Every byte maps to the code point of the same value.CsvCharset.windows1252, the usual Excel output on Windows. It matches Latin-1 except in the 32 slots from 0x80 to 0x9F, which hold the euro sign, curly quotes and dashes instead of control codes.
Which one do I need#
If the file came out of Excel on Windows, try windows1252 first. It is a superset of Latin-1 in practice, so it reads correctly in both cases and is the safer default of the two. Reach for latin1 only when you know the source is strictly ISO-8859-1 and you want the control codes preserved.
The byte order mark#
A UTF-8 byte order mark is stripped whichever encoding you choose, so it never gets glued onto your first column name. That is the bug behind a header that will not match: the key is not name but an invisible prefix followed by name.
When writing, set addBom so Excel opens the result as UTF-8 rather than guessing:
const codec = CsvCodec(CsvConfig(addBom: true));
final bytes = codec.encodeToBytes(rows);Writing a non UTF-8 file#
Output is always UTF-8. If a downstream tool truly requires a single byte encoding, encode to a string first and convert it yourself with the latin1 encoder from dart:convert.