> For the complete documentation index, see [llms.txt](https://www.lisppad.app/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://www.lisppad.app/libraries/lispkit/lispkit-sxml-xml.md).

# (lispkit sxml xml)

Library `(lispkit sxml xml)` implements a validating, namespace-aware XML parser following the [XML 1.0 Recommendation](http://www.w3.org/TR/1998/REC-xml-19980210.html) as well as the [XML Namespaces Recommendation](http://www.w3.org/TR/REC-xml-names). The implementation is an adaptation of Oleg Kiselyov's *SSAX* parsing framework (Simple SAX, or "Static SAX"), a purely functional variant of a SAX-style, streaming parser.

Most clients only need the high-level conversion procedure `xml->sxml`, which reads XML from a port and returns the corresponding SXML tree (see library `(lispkit sxml)` for details on the SXML representation). The remaining procedures exported by this library expose the lower-level building blocks of the SSAX framework and are intended for programmers who want to implement a custom XML parser, e.g. one that produces a different in-memory representation than SXML, that performs validation against a DTD/schema, or that only processes a subset of a (potentially large) XML document while streaming through it.

## Parsing XML into SXML

**(xml->sxml)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">\
**(xml->sxml&#x20;*****port*****)**\
**(xml->sxml&#x20;*****port namespace-prefixes*****)**

Parses the XML document available from input port *port* and returns its content as an SXML tree, wrapped in a `*TOP*` element. If *port* is not provided, `xml->sxml` reads from the current input port (as defined by `current-input-port` of library `(lispkit port)`). After `xml->sxml` returns, the read position of *port* is right after the root element of the document.

*namespace-prefixes* is a list of `(user-prefix . uri-string)` pairs, where *user-prefix* is a symbol chosen by the caller and *uri-string* is a string identifying an XML namespace. For every element or attribute name that is qualified by a namespace listed in *namespace-prefixes*, the corresponding *user-prefix* is used to build a prefixed symbol `user-prefix:localname` in the resulting SXML tree, irrespective of the actual namespace prefix used in the source document. If *namespace-prefixes* is provided and not empty, the top-level `*TOP*` element carries a `*NAMESPACES*` attribute listing the requested namespace mappings. The default for *namespace-prefixes* is the empty list.

A `<!DOCTYPE ...>` declaration, if present, is skipped (together with an internal DTD subset, if any) and a warning is printed via `ssax-warn`; the DTD itself is not used for validation. Processing instructions are collected into `(*PROCESSING-INSTRUCTIONS* target body)` nodes; comments are discarded.

```scheme
(import (lispkit sxml xml) (lispkit port))

(call-with-port (open-input-string "<book id=\"1\"><title>SXML in a Nutshell</title></book>")
  (lambda (port) (xml->sxml port)))
  ⇒ (*TOP* (book (@ (id "1")) (title "SXML in a Nutshell")))

(call-with-port (open-input-string
                  "<root xmlns:h=\"http://www.w3.org/HTML\"><h:p>Hi</h:p></root>")
  (lambda (port) (xml->sxml port '((html . "http://www.w3.org/HTML")))))
⇒ (*TOP* (@ (*NAMESPACES* (html "http://www.w3.org/HTML")))
         (root (html:p "Hi")))
```

## SSAX parsing framework

The remaining procedures of this library implement the low-level, extensible SSAX parsing framework. They share the following terminology and data representations:

* **UNRES-NAME**: an unresolved element, attribute, or processing-instruction name as it appears literally in an XML document. A simple name (an `NCName`) is represented as a Scheme symbol; a qualified name (a `QName`, e.g. `ns:local`) is represented as a pair `(prefix . localpart)` of symbols.
* **RES-NAME**: a namespace-resolved version of an UNRES-NAME. A name qualified by a non-empty namespace URI is represented as a pair `(uri-symbol . localpart)`; an unqualified name remains a plain symbol.
* **NAMESPACES**: a list describing the namespace declarations currently in effect. Each element has one of the forms `(prefix uri-symbol . uri-symbol)`, `(prefix user-prefix . uri-symbol)`, `(*DEFAULT* user-prefix . uri-symbol)`, or `(*DEFAULT* #f . #f)` (the latter un-declaring the default namespace). If several elements describe the same prefix, the one closest to the beginning of the list is in effect.
* **ATTLIST**: an ordered collection of `(name . value)` pairs (where `name` is a RES-NAME or UNRES-NAME, and `value` is a string), sorted by name.
* **STR-HANDLER**: a procedure of three arguments, *string1*, *string2*, and *seed*, returning a new seed. It handles a chunk of character data *string1*, immediately followed by another (typically short) chunk *string2*.
* **ENTITIES**: an association list of `(name . body)` pairs describing declared general entities, where *name* is the symbol under which the entity was declared and *body* is either a string, or (for an external entity) a thunk returning an input port from which the entity's replacement text can be read.
* **xml-token**: a record describing a markup token (a start-tag, end-tag, processing instruction, declaration, or named entity reference), consisting of a *kind* and a *head*. See `xml-token-kind` and `xml-token-head` below.

**(xml-token-kind&#x20;*****token*****)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">

Returns the kind of markup token *token*, one of the symbols `START`, `END`, `PI`, `DECL`, `COMMENT`, `CDSECT`, or `ENTITY-REF`.

**(xml-token-head&#x20;*****token*****)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">

Returns the head of markup token *token*. For a `START` or `END` token, this is the (UNRES-NAME) tag name; for a `PI` token, the PI target; for a `DECL` token, the declaration keyword (e.g. `DOCTYPE`); for an `ENTITY-REF` token, the entity name. For `COMMENT` and `CDSECT` tokens, the head is `#f`.

For example, `<p>` is tokenized as kind `START` and head `p`; `</p>` as kind `END` and head `p`; `<!DOCTYPE ...>` as kind `DECL` and head `DOCTYPE`; `<?xml version="1.0"?>` as kind `PI` and head `xml`; `&my-ent;` as kind `ENTITY-REF` and head `my-ent`.

**(ssax-warn&#x20;*****port msg other-msg ...*****)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">

Writes a warning to the current error port (as defined by `current-error-port` of library `(lispkit port)`), concatenating *msg* and all *other-msg* arguments. *port* is currently unused by the default implementation but is passed for consistency with other SSAX procedures that may report errors relative to the position within an input port.

**(ssax-scan-misc&#x20;*****port*****)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">

Scans past a sequence of comments and whitespace in the prolog or epilog of an XML document (the `Misc*` production of the XML grammar), reading from input port *port*. Returns either the eof object, or the xml-token describing the next processing instruction, declaration, or start tag that was encountered (comments are silently skipped and not reported).

**(ssax-read-char-data&#x20;*****port expect-eof? str-handler seed*****)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">

Reads the character content of an XML element (the `content` production of the XML grammar) from *port*, invoking STR-HANDLER *str-handler* for every chunk of character data encountered, threading the seed value *seed* through successive invocations. CDATA sections and character references are expanded and passed to *str-handler* inline; comments are silently disregarded. If *expect-eof?* is `#t`, encountering the end of *port* is not treated as an error (this is used while reading a parsed entity).

`ssax-read-char-data` stops reading as soon as it encounters a start tag, an end tag, the beginning of a processing instruction, a named entity reference, or (if *expect-eof?* is `#t`) the end of the input. It returns two values: the final seed (the result of the last invocation of *str-handler*, or the original *seed* if *str-handler* was never invoked), and either the eof object, or an xml-token describing what interrupted the character data (which the caller is responsible for handling further).

**(ssax-read-attributes&#x20;*****port entities*****)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">

Reads a sequence of `name="value"` attribute declarations from *port*, using ENTITIES *entities* to resolve named entity references occurring within attribute values, and returns the corresponding ATTLIST of `(UNRES-NAME . value)` pairs. The current position of *port* must be at the first character of the first attribute name (or at the character immediately following the last attribute, e.g. `>` or `/`, if there are no more attributes).

**(ssax-read-external-id&#x20;*****port*****)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">

Reads an `ExternalID` production (`SYSTEM SystemLiteral` or `PUBLIC PubidLiteral SystemLiteral`) from *port* and returns the `SystemLiteral` part as a string; a `PubidLiteral`, if present, is skipped. The current position of *port* must be at the `S` or `P` character starting the `SYSTEM` or `PUBLIC` keyword.

**(ssax-resolve-name&#x20;*****port unres-name namespaces apply-default-ns?*****)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">

Converts UNRES-NAME *unres-name* into a RES-NAME using the NAMESPACES declarations *namespaces* currently in effect. If *apply-default-ns?* is `#t`, an unqualified name is resolved against the default namespace (if any is declared in *namespaces*); this should be `#f` for attribute names, since the default namespace does not apply to attributes. The `xml` prefix is always resolved to the pre-declared `http://www.w3.org/XML/1998/namespace` namespace. *port* is used only for error reporting when a used namespace prefix has not been declared.

**(make-ssax-pi-parser&#x20;*****handlers*****)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">

Creates and returns a procedure `(port target seed)` that parses and processes a single processing instruction (PI). *handlers* is an association list of `(pi-tag . pi-handler)` pairs, where *pi-tag* is a symbol denoting a PI target and *pi-handler* is a procedure `(port pi-tag seed)`. When invoked, *pi-handler* is expected to read the remainder of the PI, up to and including the terminating `?>`, and to return a new seed. The special *pi-tag* `*DEFAULT*` may be used to handle PIs for which no specific handler is registered; if no `*DEFAULT*` handler is given, unhandled PIs are skipped (with a warning printed via `ssax-warn`).

**(make-ssax-elem-parser&#x20;*****new-level-seed finish-element char-data-handler pi-handlers*****)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">

Creates and returns a procedure `(start-tag-head port elems entities namespaces preserve-ws? seed)` that parses and processes a single element, including all of its attributes, character data, and child elements. The returned procedure must be invoked right after the start-tag head has been read; it is typically used to parse the root element of a document.

* *new-level-seed* is a procedure `(elem-gi attributes namespaces expected-content seed)` that computes the seed passed to the handlers processing the content of the element about to be parsed (RES-NAME *elem-gi*).
* *finish-element* is a procedure `(elem-gi attributes namespaces parent-seed seed)`, invoked once parsing of the element is finished; *seed* is the result of the last content handler invocation (or of *new-level-seed*, if the element was empty), and *parent-seed* is the seed that was passed to *new-level-seed*. The procedure computes the seed that becomes the overall result of the element parser.
* *char-data-handler* is a STR-HANDLER used to process character content of the element.
* *pi-handlers* has the same shape as the *handlers* argument of `make-ssax-pi-parser`.

**(make-ssax-parser&#x20;*****tag val ...*****)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">

Creates and returns a full XML document parser, a procedure `(port seed)` that parses the document prolog and then delegates to an element parser (as created by `make-ssax-elem-parser`) to process the root element and the remainder of the document. The generated parser can act as a SAX parser, a DOM parser (such as `xml->sxml`), or as a specialized parser, depending on the supplied handlers.

`make-ssax-parser` takes a property list of *tag*/*val* pairs. The following tags are recognized (all other tags are rejected):

* `NEW-LEVEL-SEED` (required) — see `my-new-level-seed` of `make-ssax-elem-parser`.
* `FINISH-ELEMENT` (required) — see `my-finish-element` of `make-ssax-elem-parser`.
* `CHAR-DATA-HANDLER` (required) — a STR-HANDLER, see `my-char-data-handler` of `make-ssax-elem-parser`.
* `PROCESSING-INSTRUCTIONS` — an association list as expected by `make-ssax-pi-parser`. Defaults to `'()`.
* `DOCTYPE` — a procedure `(port docname systemid internal-subset? seed)`, invoked when a `<!DOCTYPE ...>` declaration is encountered. If *internal-subset?* is `#t`, the current position of *port* is right after the `[` that begins the internal DTD subset; the handler is responsible for reading past the internal subset before returning. The handler must return four values: `elems entities namespaces seed`, where *elems* describes the declared elements (or `#f` to disable validation). The default handler skips the internal subset, if any, prints a warning via `ssax-warn`, and returns `(values #f '() '() seed)`.
* `UNDECL-ROOT` — a procedure `(elem-gi seed)`, invoked with the UNRES-NAME of the root element when the document does not contain a `DOCTYPE` declaration. Like the `DOCTYPE` handler, it must return four values: `elems entities namespaces seed`. The default handler returns `(values #f '() '() seed)`.
* `DECL-ROOT` — a procedure `(elem-gi seed)`, invoked with the UNRES-NAME of the root element when the document does contain a `DOCTYPE` declaration. It returns a new seed. The default handler is the identity function.

```scheme
(import (lispkit sxml xml) (lispkit port))

(define count-elements
  (make-ssax-parser
    'NEW-LEVEL-SEED
    (lambda (elem-gi attributes namespaces expected-content seed) (+ seed 1))
    'FINISH-ELEMENT
    (lambda (elem-gi attributes namespaces parent-seed seed) seed)
    'CHAR-DATA-HANDLER
    (lambda (string1 string2 seed) seed)))

(call-with-port (open-input-string "<a><b/><c><d/></c></a>")
  (lambda (port) (count-elements port 0)))
⇒ 4
```
