Western Latin character sets (computing)

Several binary representations of character sets for common Western European languages are compared in this article. These encodings were designed for representation of Italian, Spanish, Portuguese, French, German, Dutch, English, Danish, Swedish, Norwegian, and Icelandic, which use the Latin alphabet, a few additional letters and ones with precomposed diacritics, some punctuation, and various symbols (including some Greek letters). Although they're called "Western European" many of these languages are spoken all over the world. Also, these character sets happen to support many other languages such as Malay, Swahili, and Classical Latin.

Summary

The ISO-8859 series of 8-bit character sets encodes all Latin character sets used in Europe, albeit that the same code points have multiple uses that caused some difficulty. The arrival of Unicode, with a unique code point for every glyph, resolved these issues.

• ISO/IEC 8859-1 or Latin-1 is the most used and also defines the first 256 codes in Unicode
• ISO/IEC 8859-15 modifies ISO-8859-1 to fully support Estonian, Finnish and French and add the euro sign.
• Windows-1252 is a superset of ISO-8859-1 that includes the characters from ISO-8859-15 and popular punctuation such as curved quotation marks. It is common that web page tools for Windows use Windows-1252 but label the web page as using ISO-8859-1, this has been addressed in HTM 5, which mandates that pages labeled as ISO-8859-1 must be interpreted as Windows-1252.
• IBM CP437, being intended for English only, has very little in the way of accented letters but has far more graphics characters than the others and also some Greek characters that are useful as technical symbols.
• IBM CP850 has all the printable characters that ISO-8859-1 has (albeit arranged differently) and still manages to have enough graphics characters to build a usable text-mode user interface.
• IBM CP858 differs from CP850 only by one character — a dotless i (ı), rarely used outside Turkey, was replaced by euro currency sign (€).^[1]
• IBM CP859 contains all the printable characters that ISO-8859-15 has, so unlike CP850 it supports the € and French.
• IBM code pages 037, 500, and 1047 are EBCDIC encodings that include all of the ISO-8859-1 characters.
• The Mac OS Roman character set (often referred to as MacRoman and known by the IANA as simply MACINTOSH) has most, but not all, of the same characters as ISO-8859-1 but in a very different arrangement; and it also adds many technical and mathematical characters (though it lacks the important ×) and more diacritics. Older Macintosh web browsers were known to munge the few characters that were in ISO-8859-1 but not their native Macintosh character set when editing text from Web sites. Conversely, in Web material prepared on an older Macintosh, many characters were displayed incorrectly when read by other operating systems.
• The euro sign post-dates these (ISO-8859) specifications: conflicting ways to retrofit it led to significant difficulty until Unicode became more generally adopted.

Notes

• The mappings for the IBM code pages are from the Unicode site supplied by Microsoft. Refer to the Unicode Consortium's document on the differences between IBM's and Microsoft's mappings for these code pages.
• The old PC code pages actually defined printable characters for the control code ranges. While these could not be used when printing text through DOS, as they would be trapped before reaching the screen, they could be used by applications that used screen memory directly.
• Position F0_HEX was used in the Macintosh character sets for the Apple logo. The Apple logo was not accepted into Unicode due to its trademarked nature, and so Apple mapped it to a code point (U+F8FF) in the private use area. Therefore, it may not display correctly in the table.
• In Windows-1252, positions 81, 8D, 8F, 90, and 9D are unused according to the WINDOWS mapping tables on the Unicode site. However, the Windows API MultiByteToWideChar maps these to the corresponding C1 control codes; so does the "best fit" mapping which describes "windows code page behavior".

History

The earlier seven-bit U.S. ASCII encoding has characters sufficient to properly represent only English, Latin, and Swahili. It is missing some letters and letter-diacritic combinations used in other Latin-alphabet languages. However, since there was no other choice on most U.S.-supplied computer platforms, ASCII was unavoidable in most of the non-English-speaking world (seven-bit encoding was necessitated by the limitations of early computing networks). There was the ISO 646 group of encodings which replaced some of the symbols in ASCII with local characters, but space was very limited, and some of the symbols replaced were quite common in things like programming languages.

Although seven-bit communication was the norm, most computers internally used eight-bit bytes, and they mostly put some form of characters in the 128 higher byte positions. In the early days most of these were system specific, but gradually a few standards were settled on.

In recent years, as storage and memory costs fall, the issues associated with multiple meanings of a given eight-bit code (there are seven ISO-Latin code sets alone) have ceased to be justified. All major operating systems have moved to Unicode as their main internal representation. However Windows does not support Unicode using their 8-bit character interfaces (by supporting UTF-8 in standard interfaces such as fopen), so many applications continue to be restricted to these legacy character sets.

The euro sign

The coming of the euro and its euro sign introduced significant pressure to support the euro sign (€), and most 8-bit character sets had to be adapted in some way.

• Apple with MacRoman and Sun Microsystems with Solaris OS simply replaced the generic currency sign (¤). This caused significant difficulty because organisations had found other uses for it, such as the company logo.
• ISO introduced a further variant of ISO 8859, ISO 8859-15, which replaced the generic currency sign with the euro sign as well as making some other replacements of symbols with letters with diacritics. ISO 8859-15 never received widespread adoption.
• Windows-1252 placed the euro sign in a gap (position 80_hex) in the existing C1 control codes.

All of these issues have been resolved as operating systems have been upgraded to support Unicode as standard, which encodes the euro sign at U+20AC (decimal 8364).

Comparison table

Code points U+0000 to U+007F are not shown in this table currently, as they are directly mapped in all character sets listed here. The ASCII coding standard defines the original specification for the mapping of the first 0-127 characters.

The table is arranged by Unicode code point. Character sets are referred to here by their IANA names in upper case.

In addition, Macintosh assigns the Apple logo ⟨⟩ (Mac OS Roman: F0) to U+F8FF in the Private Use Area.

Notes

References

Character encodings
Character sets
Early telecommunications	ASCII ISO/IEC 646 ISO/IEC 6937 T.61 BCDIC Baudot code Morse code (Telegraph code Wabun code) Special telegraphy codes: Non-Latin, Chinese, Cyrillic
ISO/IEC 8859	-1 -2 -3 -4 -5 -6 -7 -8 -9 -10 -11 -12 -13 -14 -15 -16
Bibliographic use	ANSEL ISO 5426 / 5426-2 / 5427 / 5428 / 6438 / 6861 / 6862 / 10585 / 10586 / 10754 / 11822 MARC-8
National standards	ArmSCII CNS 11643 GOST 10859 GB 18030 HKSCS ISCII JIS X 0201 JIS X 0208 JIS X 0212 JIS X 0213 KOI-7 KPS 9566 KS X 1001 PASCII SI 960 TIS-620 TSCII VISCII YUSCII
EUC	CN JP KR TW
ISO/IEC 2022	CN JP KR CCCII
MacOS code pages ("scripts")	Arabic Mac OS Celtic CentEuro ChineseSimp / EUC-CN ChineseTrad / Big5 Croatian Cyrillic Devanagari Dingbats Farsi Gaelic Greek Gujarati Gurmukhi Hebrew Iceland Japanese / ShiftJIS Korean / EUC-KR Roman Romanian Symbol Thai / TIS-620 Turkish Ukrainian
DOS code pages	100 111 112 113 151 161 162 163 164 165 220 300 301 437 449 620 667 668 708 709 710 711 720 737 770 771 772 773 774 775 776 777 778 790 808 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 872 874 876 877 878 881 882 883 884 885 891 895 897 898 899 900 903 904 906 907 909 910 911 925 926 927 928 929 932 934 936 938 941 942 943 944 946 947 948 949 950/1370 951 966 991 1004 1034 1039 1040 1041 1042 1043 1044 1046 1086 1088 1090 1092 1093 1098 1108 1109 1114 1115 1116 1117 1118 1119 1125 1126 1127 1131 1139 1161 1162 1167 1168 1351 1361 1362 1363 1372 1373 1374 1375 1380 1381 1385 1386 1391 1392 1393 1394 17248 Kamenický Mazovia CWI-2 KOI8 MIK Iran System
IBM AIX code pages	367 371 806 813 819 895 896 901 902 912 913 914 915 916 919 920 921 922 923 952 953 954 955 956 957 958 959 960 961 962 963 964 965 970 971 1006 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1029 1036 1089 1111 1124 1129 1133 1163 1350 1382 1383
IBM Apple MacIntosh Emulations	1275 1280 1281 1282 1283 1284 1285 1286
IBM Adobe Emulations	1038 1276 1277
IBM DEC Emulations	1021 1023 1100 1101 1102 1103 1104 1105 1106 1107 1287 1288
IBM HP Emulations	1050 1051 1052 1053 1054 1055 1056 1057 1058
Windows code pages	874/1162 (TIS-620) 932/943 (Shift JIS) 936/1386 (GBK) 950/1370 (Big5) 949/1363 (EUC-KR) 1169 1174 1200 (UTF-16LE) 1201 (UTF-16BE) 1250 1251 1252 1253 1254 1255 1256 1257 1258 1261 1270 54936 (GB18030)
EBCDIC code pages	1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 29 30 31 32 33 34 35 36 37/1140 38 39 40 251 252 254 256 257 258 259 260 264 273/1141 274 275 276 277/1142 278/1143 279 280/1144 281 282 283 284/1145 285/1146 286 287 288 289 290 293 297/1147 298 310 320 321 322 330 351 352 353 355 357 358 359 360 361 363 382 383 384 385 386 387 388 389 390 391 392 393 394 395 410 420/16804 421 423 424/12712 425 435 500/1148 803 829 833 834 835 836 837 838/1160 839 870/1153 871/1149 875/9067 880 881 882 883 884 885 886 887 888 889 890 892 893 905 918 924 930/1390 931 933/1364 935/1388 937/1371 939/1399 1001 1002 1003 1005 1007 1024 1025/1154 1026/1155 1027 1028 1030 1031 1032 1033 1037 1047 1068 1069 1070 1071 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1087 1091 1097 1110 1112/1156 1113 1122/1157 1123/1158 1130/1164 1132 1136 1137 1150 1151 1152 1159 1165 1166 1278 1303 1364 1376 1377 JEF KEIS
Platform specific	ATASCII Atari ST BICS CDC CPC DEC Radix-50 DMCS/NRCS ELWRO-Junior FIELDATA GEM GEOS GSM 03.38 HP Roman Extension HP Roman-8 HP Roman-9 HP calculators LICS LMBCS NEC APC NeXT PETSCII TI calculators WISCII XCCS ZX80 ZX81 ZX Spectrum
Unicode / ISO/IEC 10646	UTF-1 UTF-7 UTF-8 UTF-16 (UTF-16LE/UTF-16BE) / UCS-2 UTF-32 (UTF-32LE/UTF-32BE) / UCS-4 UTF-EBCDIC GB 18030 BOCU-1 CESU-8 SCSU
Miscellaneous code pages	ABICOMP APL Cork HZ JOHAB TRON UTF-5 UTF-6 WTF-8
Related topics	Control character (C0 C1) CCSID Character encodings in HTML Charset detection Han unification Hardware ISO 6429/IEC 6429/ANSI X3.64 Mojibake