Functional HTML scraping and rewriting with CSS in OCaml. http://aantron.github.io/lambdasoup

Anton Bachin b266158194 Remove Bisect_ppx development pin 2 weeks ago
.github d22c9b99ab Try GitHub Sponsors 2 months ago
docs 18e8fddab2 Remove some compatibility shims 8 months ago
src 34091652fe Bump to 0.7.0 1 month ago
test 203dfa9c3a Tweaks 1 month ago
.gitignore c4ffd2fb89 .gitignore _opam/ for opam 2 1 year ago
.travis.yml 5d9bddbbd8 Travis: upgrade opam to 2.0.6 1 month ago
LICENSE.md 7b95128631 Wrap text in LICENSE.md 3 months ago
Makefile 9d8864c91f Upgrade Bisect_ppx 7 months ago
README.md 34091652fe Bump to 0.7.0 1 month ago
dune-project 7a69a597d8 Convert to Dune 1 year ago
lambdasoup.opam b266158194 Remove Bisect_ppx development pin 2 weeks ago

README.md

Lambda Soup   version 0.7.0 Travis status Coverage

Lambda Soup is a functional HTML scraping and manipulation library for OCaml aimed at being easy to use.

Lambda Soup usage example

Lambda Soup is simple. It provides a set of elementary traversals for getting from node to node, familiar functional combinators such as filter, map, and fold, and support for all CSS selectors that still make sense when not running in a browser (and a few obvious extensions on top of that).

Here is a trivial self-contained example:

(parse "<p class='Hello'>World!</p>") $ ".Hello" |> R.leaf_text;;
- : string = "World!"

And, a mutation:

let soup = parse "<p class='Hello'>World!</p>" in
wrap (soup $ ".Hello" |> R.child) (create_element "strong");
soup |> to_string;;
- : string = "<p class=\"Hello\"><strong>World!</strong></p>"

For some more examples, see the Lambda Soup postprocessor that runs on Lambda Soup's own documentation after it is generated by ocamldoc.

The library is tested thoroughly.

Lambda Soup is based on Markup.ml. As a consequence, it resolves entity references, detects character encodings automatically, and converts everything to UTF-8. And, you can use Lambda Soup on XML, by parsing the XML with Markup.ml and feeding the signals to Lambda Soup.


Installing

opam install lambdasoup


Starting from scratch

To use Lambda Soup interactively as in the GIF at the top of this README, you need to have done something like this:

your-package-manager install ocaml opam
opam init
eval `opam config env`          # Or restart your shell
opam install lambdasoup

and make sure your ~/.ocamlinit file looks something like this:

let () =
  try Topdirs.dir_directory (Sys.getenv "OCAML_TOPLEVEL_PATH")
  with Not_found -> ()
;;

#use "topfind";;

Then, run ocaml -short-paths to start the top-level, and scrape away!


Depending

Lambda Soup uses semantic versioning, but is currently in 0.x.x. For now, the minor version number will be incremented on breaking changes. So, to give yourself a chance to review the changelog before your code breaks, put the following constraint on Lambda Soup: lambdasoup {< "0.7.0"}.


Documentation

Lambda Soup's interface consists of one module Soup, whose signature is documented here.


Developing

See CONTRIBUTING. All feedback is welcome – open an issue on GitHub, or send me an email at antonbachin@yahoo.com. If you find yourself repeatedly writing the same helper on top of Lambda Soup's functions, perhaps we should add it to Lambda Soup.