> For the complete documentation index, see [llms.txt](https://www.lisppad.app/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://www.lisppad.app/libraries/lispkit/lispkit-sxml-html.md).

# (lispkit sxml html)

Library `(lispkit sxml html)` implements a permissive, streaming HTML parser. Unlike the strict XML parser of library `(lispkit sxml xml)`, this parser is designed to cope with the malformed and inconsistent markup that is commonplace in real-world HTML documents: it inserts "virtual" start and end tags as needed to keep the parsed document a properly nested tree, aiming for behavior comparable to how common web browsers parse HTML.

The parser is implemented as a tree-folding parser (in the style of `foldts`) with an interface modeled after the SSAX parser of library `(lispkit sxml xml)`: a low-level, callback-based procedure `make-html-parser` for building custom parsers, plus two ready-to-use convenience procedures, `html->sxml` and `html-strip`, built on top of it.

**(make-html-parser&#x20;*****key val ...*****)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">

Returns a new HTML parser, a procedure `(seed [port])`, that parses the HTML document from input port *port* (or, if *port* is a string, an input port opened on that string), invoking the given callbacks while folding over the document tree. If *port* is not provided, the parser reads from the current input port (as defined by `current-input-port` of library `(lispkit port)`). *seed* is the initial seed value, threaded through the callbacks and returned as the final result once the whole document (or fragment) has been parsed.

`make-html-parser` accepts the following keyword arguments; all of them are optional:

* `start:` A procedure `(tag attrs seed virtual?)`, invoked when a start-tag is encountered. *tag* is the (lower-cased) tag name symbol, *attrs* is the list of `(name value)` attribute pairs, *seed* is the current seed, and *virtual?* is `#t` if this start-tag was inserted by the parser to fix up the document tree (rather than being present literally in the source). The procedure returns a new seed to be used while parsing the children of the element. The default handler returns *seed* unchanged.
* `end:` A procedure `(tag attrs parent-seed seed virtual?)`, invoked when an end-tag is encountered (including virtual ones inserted for unclosed elements). *attrs* are the attributes of the corresponding start-tag, *parent-seed* is the seed that was passed to the `start:` handler, *seed* is the seed resulting from parsing the element's children, and *virtual?* is `#t` if the end-tag was inserted by the parser. The procedure returns the seed to be used for further sibling elements. The default handler returns *seed* unchanged.
* `text:` A procedure `(text seed)`, invoked with a chunk of entity-decoded character content. May be called multiple times between a start-tag and its end-tag; results need to be accumulated by the handler itself. The default handler returns *seed* unchanged.
* `comment:` A procedure `(text seed)`, invoked with the content of an HTML comment. The default handler returns *seed* unchanged.
* `decl:` A procedure `(name attrs seed)`, invoked for a declaration such as `<!DOCTYPE ...>`. The default handler returns *seed* unchanged.
* `process:` A procedure `(list seed)`, invoked for a processing instruction, where *list* is a two-element list of the (lower-cased) PI target symbol (or `#f`) and the PI body string. The default handler returns *seed* unchanged.
* `entity:` A procedure `(name-or-num seed)` that converts a named or numeric character entity (given either as a string name or an integer code point) into a string. The default handler looks *name-or-num* up in *entities* and falls back to re-emitting the original `&name;` notation if the entity is unknown.
* `entities:` An association list of `(name . string)` pairs used to resolve named entity references (in addition to the numeric `&#NNNN;` / `&#xHHHH;` forms, which are always understood). Defaults to a small built-in table covering `amp`, `quot`, `lt`, `gt`, `apos`, and `nbsp`.
* `tag-levels:`, `unnestables:`, `bodyless:`, `literals:`, `terminators:` Advanced parameters controlling the error-recovery heuristics used to cope with malformed markup, namely the relative nesting levels of tags (used to decide when to force-close an open tag), tags that may not be nested within themselves (such as `p`, `li`, `td`, `tr`), tags that never have a body or a closing tag (such as `img`, `hr`, `br`), tags whose content is read verbatim up to a matching closing tag rather than being parsed as markup (such as `script`, `xmp`), and tags whose content extends to the end of the document (such as `plaintext`). These rarely need to be customized.

```scheme
(define (html-strip . o)
  (call-with-output-string
    (lambda (out)
      (let ((parse
              (make-html-parser
                'start: (lambda (tag attrs seed virtual?) seed)
                'end:   (lambda (tag attrs parent-seed seed virtual?) seed)
                'text:  (lambda (text seed) (display text out)))))
        (apply parse (cons #f #f) o)))))
```

**(html->sxml)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">\
**(html->sxml&#x20;*****port*****)**

Parses the HTML document available from input port *port* (or a string, in which case an input port is opened on it) and returns its content as an SXML tree, wrapped in a `*TOP*` element (see library `(lispkit sxml)` for details on the SXML representation). If *port* is not provided, `html->sxml` reads from the current input port. Declarations, processing instructions, and comments are represented, respectively, as `(*DECL* name attr ...)`, `(*PI* target body)`, and `(*COMMENT* text)` nodes.

```scheme
(import (lispkit sxml html) (lispkit port))

(call-with-port (open-input-string "<ul><li>one</li><li>two</li></ul>")
  (lambda (port) (html->sxml port)))
  ⇒ (*TOP* (ul (li "one") (li "two")))

;; A missing closing tag for <b> is automatically fixed up:
(call-with-port (open-input-string "<p>Hello, <b>World</p>")
  (lambda (port) (html->sxml port)))
⇒ (*TOP* (p "Hello, " (b "World")))
```

**(html-strip)** <img src="https://1467949168-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fna2foeoaXHYkSD3fhs0t%2Fuploads%2Fgit-blob-d20368c588cfbb523beb2fae4f8be0f8ef011884%2Fproc.png?alt=media" alt="" data-size="line">\
**(html-strip&#x20;*****port*****)**

Parses the HTML document available from input port *port* (or a string) and returns a string containing all of its text content, with every tag removed. No whitespace reduction or other post-processing of the extracted text is performed. If *port* is not provided, `html-strip` reads from the current input port.

```scheme
(html-strip "<p>5 &lt; 7 &amp; 7 &gt; 5</p>")
⇒ "5 < 7 & 7 > 5"
```
