This \\ is a backslash, and these are \{braces\}.
}
+If you want to write your input file in a character set other than
+ASCII, you can do so by using the \c{\\cfg\{input-charset\}}
+command. See \k{input-config} for details of this.
+
\H{input-inline} Simple \i{inline formatting commands}
Halibut formatting commands all begin with a backslash, followed by
and in every output format Halibut generates, it will choose the
best quote characters available to it in that format.
-You can still use ordinary ASCII \i{double quotes} if you prefer; or
-you could even use the \c{\\u} command (see \k{input-unicode}) to
-generate \i{Unicode matched quotes} (single or double) and fall back
-to the normal ASCII one if they aren't available. But I recommend
-using the built-in \c{\\q} command in most cases, because it's
-simple and does the best it can everywhere.
+You can still use the ordinary quote characters of your choice if
+you prefer; or you could even use the \c{\\u} command (see
+\k{input-unicode}) to generate \i{Unicode matched quotes} (single or
+double) in a way which will automatically fall back to the normal
+ASCII one if they aren't available. But I recommend using the
+built-in \c{\\q} command in most cases, because it's simple and does
+the best it can everywhere.
(Note that if you're using the \c{\\c} or \c{\\cw} commands to
display literal computer code, you probably \e{will} want to use
\S{input-unicode} \c{\\u}: Specifying arbitrary \i{Unicode}
characters
-When Halibut is finished, it should have full Unicode support. You
-should be able to specify any (reasonably well known) \i{character
-set} for your input document, and Halibut should convert it all to
-Unicode as it reads it in. Similarly, you should be able to specify
-the character set you want for each output format and have all the
-conversion done automatically.
-
-Currently, none of this is actually supported. Input text files are
-assumed to be in \i{ISO 8859-1}, and each output format has its own
-non-configurable character set (although the HTML output can use the
-\c{Ӓ} mechanism to output any Unicode character it likes).
+Halibut has extensive support for Unicode and character set
+conversion. You can specify any (reasonably well known) \i{character
+set} for your input document, and Halibut will convert it all to
+Unicode as it reads it in. See \k{input-config} for more details of
+this.
If you need to specify a Unicode character in your input document
-which is not supported by the input character set, you can use the
-\i\c{\\u} command to do this. \c{\\u} expects to be followed by a
-sequence of hex digits; so that \c{\\u0041}, for example, denotes
-the Unicode character \cw{0x0041}, which is the capital letter A.
+which is not supported by the input character set you have chosen,
+you can use the \i\c{\\u} command to do this. \c{\\u} expects to be
+followed by a sequence of hex digits; so that \c{\\u0041}, for
+example, denotes the Unicode character \cw{0x0041}, which is the
+capital letter A.
If a Unicode character specified in this way is not supported in a
particular \e{output} format, you probably don't just want it to be
\dd Exactly like \c{chapter}, but changes the name given to
appendices.
+\dt \I\cw{\\cfg\{input-charset\}}\cw{\\cfg\{input-charset\}\{}\e{character set name}\cw{\}}
+
+\dd This tells Halibut what \i{character set} you are writing your
+input file in. By default, it is assumed to be US-ASCII (meaning
+\e{only} plain \i{ASCII}, with no accented characters at all).
+
+\lcont{
+
+You can specify any well-known name for any supported character set.
+For example, \c{iso-8859-1}, \c{iso8859-1} and \c{iso_8859-1} are
+all recognised, \c{GB2312} and \c{EUC-CN} both work, and so on.
+
+This directive takes effect immediately after the \c{\\cfg} command.
+All text after that in the file is expected to be in the new
+character set. You can even change character set several times
+within a file if you really want to.
+
+When Halibut reads the input file, everything you type will be
+converted into \i{Unicode} from the character set you specify here,
+will be processed as Unicode by Halibut internally, and will be
+written to the various output formats in whatever character sets
+they deem appropriate.
+
+}
+
In addition to these configuration commands, there are also
configuration commands provided by each individual output format.
These configuration commands are discussed along with each output
freetree234(macros);
}
+static void input_configure(input *in, paragraph *cfg) {
+ assert(cfg->type == para_Config);
+
+ if (!ustricmp(cfg->keyword, L"input-charset")) {
+ char *csname = utoa_dup(uadv(cfg->keyword));
+ in->charset = charset_from_localenc(csname);
+ sfree(csname);
+ }
+}
+
/*
* Can return EOF
*/
return c;
}
else if (in->currfp) {
- int c = getc(in->currfp);
- if (c == EOF) {
- fclose(in->currfp);
- in->currfp = NULL;
- }
- /* Track line numbers, for error reporting */
- if (pos)
- *pos = in->pos;
- if (in->reportcols) {
- switch (c) {
- case '\t':
- in->pos.col = 1 + (in->pos.col + TAB_STOP-1) % TAB_STOP;
- break;
- case '\n':
- in->pos.col = 1;
- in->pos.line++;
- break;
- default:
- in->pos.col++;
- break;
+ while (in->wcpos >= in->nwc) {
+
+ int c = getc(in->currfp);
+
+ if (c == EOF) {
+ fclose(in->currfp);
+ in->currfp = NULL;
+ return EOF;
+ }
+ /* Track line numbers, for error reporting */
+ if (pos)
+ *pos = in->pos;
+ if (in->reportcols) {
+ switch (c) {
+ case '\t':
+ in->pos.col = 1 + (in->pos.col + TAB_STOP-1) % TAB_STOP;
+ break;
+ case '\n':
+ in->pos.col = 1;
+ in->pos.line++;
+ break;
+ default:
+ in->pos.col++;
+ break;
+ }
+ } else {
+ in->pos.col = -1;
+ if (c == '\n')
+ in->pos.line++;
+ }
+
+ /*
+ * Do input character set translation, so that we return
+ * Unicode.
+ */
+ {
+ char buf[1];
+ char const *p;
+ int inlen;
+
+ buf[0] = (char)c;
+ p = buf;
+ inlen = 1;
+
+ in->nwc = charset_to_unicode(&p, &inlen,
+ in->wc, lenof(in->wc),
+ in->charset, &in->csstate,
+ NULL, 0);
+ assert(p == buf+1 && inlen == 0);
+
+ in->wcpos = 0;
}
- } else {
- in->pos.col = -1;
- if (c == '\n')
- in->pos.line++;
}
- /* FIXME: do input charmap translation. We should be returning
- * Unicode here. */
- return c;
+
+ return in->wc[in->wcpos++];
+
} else
return EOF;
}
already = TRUE;/* inhibit get_token at top of loop */
prev_para_type = par.type;
addpara(par, ret);
+
+ if (par.type == para_Config) {
+ input_configure(in, &par);
+ }
continue; /* next paragraph */
}
}
in->currfp = fopen(in->filenames[in->currindex], "r");
if (in->currfp) {
setpos(in, in->filenames[in->currindex]);
+ in->charset = in->defcharset;
+ in->csstate = charset_init_state;
read_file(&hptr, in, idx);
}
in->currindex++;
\define{eur} \u20AC{EUR }
-\versionid $Id: test.but,v 1.26 2004/04/09 18:47:33 simon Exp $
+\versionid $Id: test.but,v 1.27 2004/04/19 17:09:49 simon Exp $
\C{ch\\ap} First chapter title; for similar wrapping reasons this
chapter title will be ludicrously long. I wonder how much more
A slightly more difficult macro: \eur\.2500.
+Test of input character set switching.
+
+\b 8859 character in ASCII mode: expect nothing useful. [coöpt]
+
+\cfg{input-charset}{ISO-8859-1}
+
+\b 8859 character in 8859 mode: expect the right thing. [coöpt]
+
+\cfg{input-charset}{UTF-8}
+
+\b 8859 character in UTF-8 mode: expect the wrong thing. [coöpt]
+
+\b UTF-8 sequence in UTF-8 mode: expect the right thing again. [coöpt]
+
+\cfg{input-charset}{ASCII}
+
+Back to ASCII again.
+
Oh, while I'm here: some special characters. The \\, \{ and \}
characters, to be precise. And their code equivalents, \c{\\},
\i\c{\{}, \c{\}}.
~mdw / sgt / halibut / commitdiff