# Created 2021-04-29 Thu 09:07
#+TITLE: The Library of Babel
#+AUTHOR: Org-mode People


* Introduction

The Library of Babel is an extensible collection of ready-made and
easily-shortcut-callable source-code blocks for handling common tasks.
Org-babel comes pre-populated with the source-code blocks located in
this file.  It is possible to add source-code blocks from any org-mode
file to the library by calling =(org-babel-lob-ingest
"path/to/file.org")=.

This file is included in worg mainly less for viewing through the web
interface, and more for contribution through the worg git repository.
If you have code snippets that you think others may find useful please
add them to this file and [[file:./worg-git.org::#contribute-to-worg][contribute them]] to worg.

The raw Org-mode text of this file can be downloaded at
[[https://orgmode.org/worg/library-of-babel.org][library-of-babel.org]]

* Simple

A collection of simple utility functions:

#+name: echo
#+begin_src emacs-lisp :var input="echo'd"
  input
#+end_src

* File I/O

** Reading and writing files

Read the contents of the file at =file=.  The =:results vector= and
=:results scalar= header arguments can be used to read the contents of
file as either a table or a string.

#+name: read
#+begin_src emacs-lisp :var file="" :var format=""
  (if (string= format "csv")
      (with-temp-buffer
        (org-table-import (expand-file-name file) nil)
        (org-table-to-lisp))
    (with-temp-buffer
      (insert-file-contents (expand-file-name file))
      (buffer-string)))
#+end_src

Write =data= to a file at =file=.  If =data= is a list, then write it
as a table in traditional Org-mode table syntax.

#+name: write
#+begin_src emacs-lisp :var data="" :var file="" :var ext='()
  (flet ((echo (r) (if (stringp r) r (format "%S" r))))
    (with-temp-file file
      (case (and (listp data)
                 (or ext (intern (file-name-extension file))))
        ('tsv (insert (orgtbl-to-tsv data '(:fmt echo))))
        ('csv (insert (orgtbl-to-csv data '(:fmt echo))))
        (t    (org-babel-insert-result data)))))
  nil
#+end_src

** Remote files

*** json

Read local or remote file in [[http://www.json.org/][json]] format into emacs-lisp objects.

#+name: json
#+begin_src emacs-lisp :var file='() :var url='()
  (require 'json)
  (cond
   (file
    (org-babel-with-temp-filebuffer file
      (goto-char (point-min))
      (json-read)))
   (url
    (require 'w3m)
    (with-temp-buffer
      (w3m-retrieve url)
      (goto-char (point-min))
      (json-read))))
#+end_src

*** Google docs

The following code blocks make use of the [[http://code.google.com/p/googlecl/][googlecl]] Google command line
tool.  This tool provides functionality for accessing Google services
from the command line, and the following code blocks use /googlecl/
for reading from and writing to Google docs with Org-mode code blocks.

**** Read a document from Google docs

The =google= command seems to be throwing "Moved Temporarily" errors
when trying to download textual documents, but this is working fine
for spreadsheets.

#+name: gdoc-read
#+begin_src emacs-lisp :var title="example" :var format="csv"
  (let* ((file (concat title "." format))
         (cmd (format "google docs get --format %S --title %S" format title)))
    (message cmd) (message (shell-command-to-string cmd))
    (prog1 (if (string= format "csv")
               (with-temp-buffer
                 (org-table-import (shell-quote-argument file) '(4))
                 (org-table-to-lisp))
             (with-temp-buffer
               (insert-file-contents (shell-quote-argument file))
               (buffer-string)))
      (delete-file file)))
#+end_src

For example, a line like the following can be used to read the
contents of a spreadsheet named =num-cells= into a table.
: #+call: gdoc-read(title="num-cells"")


A line like the following can be used to read the contents of a
document as a string.

: #+call: gdoc-read(title="loremi", :format "txt")

**** Write a document to a Google docs

Write =data= to a google document named =title=.  If =data= is tabular
it will be saved to a spreadsheet, otherwise it will be saved as a
normal document.

#+name: gdoc-write
#+begin_src emacs-lisp :var title="babel-upload" :var data=fibs(n=10) :results silent
  (let* ((format (if (listp data) "csv" "txt"))
         (tmp-file (make-temp-file "org-babel-google-doc" nil (concat "." format)))
         (cmd (format "google docs upload --title %S %S" title tmp-file)))
    (with-temp-file tmp-file
      (insert
       (if (listp data)
           (orgtbl-to-csv
            data '(:fmt (lambda (el) (if (stringp el) el (format "%S" el)))))
         (if (stringp data) data (format "%S" data)))))
    (message cmd)
    (prog1 (shell-command-to-string cmd) (delete-file tmp-file)))
#+end_src

example usage
: #+name: fibs
: #+begin_src emacs-lisp :var n=8
:   (flet ((fib (m) (if (< m 2) 1 (+ (fib (- m 1)) (fib (- m 2))))))
:     (mapcar (lambda (el) (list el (fib el))) (number-sequence 0 (- n 1))))
: #+end_src
:
: #+call: gdoc-write(title="fibs", data=fibs(n=10))

* Plotting code

** R

Plot column 2 (y axis) against column 1 (x axis). Columns 3 and
beyond, if present, are ignored.

Running this code will create a file =Rplots.pdf= in the current working directory.

#+name: R-plot
#+begin_src R :var data=R-plot-example-data
  plot(data)
#+end_src

#+name: R-plot-example-data
| 1 |  2 |
| 2 |  4 |
| 3 |  9 |
| 4 | 16 |
| 5 | 25 |

#+call: R-plot(data=R-plot-example-data)

** Gnuplot

* Org reference

** Headline references

#+name: headline
#+begin_src emacs-lisp :var headline=top :var file='()
  (save-excursion
    (when file (get-file-buffer file))
    (org-open-link-from-string (org-make-link-string headline))
    (save-restriction
      (org-narrow-to-subtree)
      (buffer-string)))
#+end_src

#+call: headline(headline="headline references")

* Tables

** LaTeX Table Export

*** booktabs

This source block can be used to wrap a table in the latex =booktabs=
environment. The source block adds a =toprule= and =bottomrule= (so
don't use =hline= at the top or bottom of the table).  The =hline=
after the header is replaced with a =midrule=.

Note that this function bypasses the Org-mode LaTeX exporter and calls
=orgtbl-to-generic= to create the output table.  This means that the
entries in the table are not translated from Org-mode to LaTeX.

It takes the following arguments -- all but the first two are
optional.

| arg   | description                                |
|-------+--------------------------------------------|
| table | a reference to the table                   |
| align | alignment string                           |
| env   | optional environment, default to "tabular" |
| width | optional width specification string        |

#+name: booktabs
#+begin_src emacs-lisp :var table='((:head) hline (:body)) :var align='() :var env="tabular" :var width='() :noweb yes :results latex
  (flet ((to-tab (tab)
                 (orgtbl-to-generic
                  (mapcar (lambda (lis)
                            (if (listp lis)
                                (mapcar (lambda (el)
                                          (if (stringp el)
                                              el
                                            (format "%S" el))) lis)
                              lis)) tab)
                  (list :lend " \\\\" :sep " & " :hline "\\hline"))))
    (org-fill-template
     "
  \\begin{%env}%width%align
  \\toprule
  %table
  \\bottomrule
  \\end{%env}\n"
     (list
      (cons "env"       (or env "table"))
      (cons "width"     (if width (format "{%s}" width) ""))
      (cons "align"     (if align (format "{%s}" align) ""))
      (cons "table"
            ;; only use \midrule if it looks like there are column headers
            (if (equal 'hline (second table))
                (concat (to-tab (list (first table)))
                        "\n\\midrule\n"
                        (to-tab (cddr table)))
              (to-tab table))))))
#+end_src

*** longtable

This block can be used to wrap a table in the latex =longtable=
environment, it takes the following arguments -- all but the first two
are optional.

| arg       | description                                                 |
|-----------+-------------------------------------------------------------|
| table     | a reference to the table                                    |
| align     | optional alignment string                                   |
| width     | optional width specification string                         |
| hline     | the string to use as hline separator, defaults to "\\hline" |
| head      | optional "head" string                                      |
| firsthead | optional "firsthead" string                                 |
| foot      | optional "foot" string                                      |
| lastfoot  | optional "lastfoot" string                                  |

#+name: longtable
#+begin_src emacs-lisp :var table='((:table)) :var align='() :var width='() :var hline="\\hline" :var firsthead='() :var head='() :var foot='() :var lastfoot='() :noweb yes :results latex
  (org-fill-template
   "
  \\begin{longtable}%width%align
  %firsthead
  %head
  %foot
  %lastfoot

  %table
  \\end{longtable}\n"
   (list
    (cons "width"     (if width (format "{%s}" width) ""))
    (cons "align"     (if align (format "{%s}" align) ""))
    (cons "firsthead" (if firsthead (concat firsthead "\n\\endfirsthead\n") ""))
    (cons "head"      (if head (concat head "\n\\endhead\n") ""))
    (cons "foot"      (if foot (concat foot "\n\\endfoot\n") ""))
    (cons "lastfoot"  (if lastfoot (concat lastfoot "\n\\endlastfoot\n") ""))
    (cons "table" (orgtbl-to-generic
                   (mapcar (lambda (lis)
                             (if (listp lis)
                                 (mapcar (lambda (el)
                                           (if (stringp el)
                                               el
                                             (format "%S" el))) lis)
                               lis)) table)
                   (list :lend " \\\\" :sep " & " :hline hline)))))
#+end_src

*** booktabs-notes

This source block builds on [[booktabs]].  It accepts two additional
arguments, both of which are optional.

#+name: arguments
| arg    | description                                          |
|--------+------------------------------------------------------|
| notes  | an org-mode table with footnotes                     |
| lspace | if non-nil, insert =addlinespace= after =bottomrule= |

An example footnote to the =arguments= table specifies the column
span. Note the use of LaTeX, rather than Org-mode, markup.

#+name: arguments-notes
| \multicolumn{2}{l}{This is a footnote to the \emph{arguments} table.} |

#+name: booktabs-notes
#+begin_src emacs-lisp :var table='((:head) hline (:body)) :var notes='() :var align='() :var env="tabular" :var width='() :var lspace='() :noweb yes :results latex
  (flet ((to-tab (tab)
                 (orgtbl-to-generic
                  (mapcar (lambda (lis)
                            (if (listp lis)
                                (mapcar (lambda (el)
                                          (if (stringp el)
                                              el
                                            (format "%S" el))) lis)
                              lis)) tab)
                  (list :lend " \\\\" :sep " & " :hline "\\hline"))))
    (org-fill-template
     "
    \\begin{%env}%width%align
    \\toprule
    %table
    \\bottomrule%spacer
    %notes
    \\end{%env}\n"
     (list
      (cons "env"       (or env "table"))
      (cons "width"     (if width (format "{%s}" width) ""))
      (cons "align"     (if align (format "{%s}" align) ""))
      (cons "spacer"    (if lspace "\\addlinespace" ""))
      (cons "table"
            ;; only use \midrule if it looks like there are column headers
            (if (equal 'hline (second table))
                (concat (to-tab (list (first table)))
                        "\n\\midrule\n"
                        (to-tab (cddr table)))
              (to-tab table)))
      (cons "notes" (if notes (to-tab notes) ""))
      )))
#+end_src

** Elegant lisp for transposing a matrix

#+name: transpose-example
| 1 | 2 | 3 |
| 4 | 5 | 6 |

#+name: transpose
#+begin_src emacs-lisp :var table=transpose-example
  (apply #'mapcar* #'list table)
#+end_src

#+name:
| 1 | 4 |
| 2 | 5 |
| 3 | 6 |

** Convert every element of a table to a string

#+name: hetero-table
| 1 | 2 | 3 |
| a | b | c |

#+name: all-to-string
#+begin_src emacs-lisp :var tbl='()
  (defun all-to-string (tbl)
    (if (listp tbl)
        (mapcar #'all-to-string tbl)
      (if (stringp tbl)
          tbl
        (format "%s" tbl))))
  (all-to-string tbl)
#+end_src

#+begin_src emacs-lisp :var tbl=hetero-table
  (mapcar (lambda (row) (mapcar (lambda (cell) (stringp cell)) row)) tbl)
#+end_src

#+name:
| nil | nil | nil |
| t   | t   | t   |

#+begin_src emacs-lisp :var tbl=all-to-string(hetero-table)
  (mapcar (lambda (row) (mapcar (lambda (cell) (stringp cell)) row)) tbl)
#+end_src

#+name:
| t | t | t |
| t | t | t |

* Misc

** File-specific Version Control logging
This function will attempt to retrieve the entire commit log for the
file associated with the current buffer and insert this log into the
export. The function uses the Emacs VC commands to interface to the
local version control system, but has only been tested to work with
Git. 'limit' is currently unsupported.

#+name: vc-log
#+header: :var limit=-1
#+header: :var buf=(buffer-name (current-buffer))
#+begin_src emacs-lisp
  ;; Most of this code is copied from vc.el vc-print-log
  (require 'vc)
  (when (vc-find-backend-function
         (vc-backend (buffer-file-name (get-buffer buf))) 'print-log)
    (let ((limit -1)
          (vc-fileset nil)
          (backend nil)
          (files nil))
      (with-current-buffer (get-buffer buf)
        (setq vc-fileset (vc-deduce-fileset t)) ; FIXME: Why t? --Stef
        (setq backend (car vc-fileset))
        (setq files (cadr vc-fileset)))
      (with-temp-buffer
        (let ((status (vc-call-backend
                       backend 'print-log files (current-buffer))))
          (when (and (processp status)   ; Make sure status is a process
                     (= 0 (process-exit-status status))) ; which has not terminated
            (while (not (eq 'exit (process-status status)))
              (sit-for 1 t)))
          (buffer-string)))))
#+end_src

** Trivial python code blocks

#+name: python-identity
#+begin_src python :var a=1
  a
#+end_src

#+name: python-add
#+begin_src python :var a=1 :var b=2
  a + b
#+end_src

** Arithmetic

#+name: lob-add
#+begin_src emacs-lisp :var a=0 :var b=0
  (+ a b)
#+end_src

#+name: lob-minus
#+begin_src emacs-lisp :var a=0 :var b=0
  (- a b)
#+end_src

#+name: lob-times
#+begin_src emacs-lisp :var a=0 :var b=0
  (* a b)
#+end_src

#+name: lob-div
#+begin_src emacs-lisp :var a=0 :var b=0
  (/ a b)
#+end_src

* GANTT Charts

The =elispgantt= source block was sent to the mailing list by Eric
Fraga.  It was modified slightly by Tom Dye.

#+name: elispgantt
#+begin_src emacs-lisp :var table=gantttest
  (let ((dates "")
        (entries (nthcdr 2 table))
        (milestones "")
        (nmilestones 0)
        (ntasks 0)
        (projecttime 0)
        (tasks "")
        (xlength 1))
    (message "Initial: %s\n" table)
    (message "Entries: %s\n" entries)
    (while entries
      (let ((entry (first entries)))
        (if (listp entry)
            (let ((id (first entry))
                  (type (nth 1 entry))
                  (label (nth 2 entry))
                  (task (nth 3 entry))
                  (dependencies (nth 4 entry))
                  (start (nth 5 entry))
                  (duration (nth 6 entry))
                  (end (nth 7 entry))
                  (alignment (nth 8 entry)))
              (if (> start projecttime) (setq projecttime start))
              (if (string= type "task")
                  (let ((end (+ start duration))
                        (textposition (+ start (/ duration 2)))
                        (flush ""))
                    (if (string= alignment "left")
                        (progn
                          (setq textposition start)
                          (setq flush "[left]"))
                      (if (string= alignment "right")
                          (progn
                            (setq textposition end)
                            (setq flush "[right]"))))
                    (setq tasks
                          (format "%s  \\gantttask{%s}{%s}{%d}{%d}{%d}{%s}\n"
                                  tasks label task start end textposition flush))
                    (setq ntasks (+ 1 ntasks))
                    (if (> end projecttime)
                        (setq projecttime end)))
                (if (string= type "milestone")
                    (progn
                      (setq milestones
                            (format
                             "%s  \\ganttmilestone{$\\begin{array}{c}\\mbox{%s}\\\\ \\mbox{%s}\\end{array}$}{%d}\n"
                             milestones label task start))
                      (setq nmilestones (+ 1 nmilestones)))
                  (if (string= type "date")
                      (setq dates (format "%s  \\ganttdateline{%s}{%d}\n"
                                          dates label start))
                    (message "Ignoring entry with type %s\n" type)))))
          (message "Ignoring non-list entry %s\n" entry)) ; end if list entry
        (setq entries (cdr entries))))  ; end while entries left
    (format "\\pgfdeclarelayer{background}
  \\pgfdeclarelayer{foreground}
  \\pgfsetlayers{background,foreground}
  \\renewcommand{\\ganttprojecttime}{%d}
  \\renewcommand{\\ganttntasks}{%d}
  \\noindent
  \\begin{tikzpicture}[y=-0.75cm,x=0.75\\textwidth]
    \\begin{pgfonlayer}{background}
      \\draw[very thin, red!10!white] (0,1+\\ganttntasks) grid [ystep=0.75cm,xstep=1/\\ganttprojecttime] (1,0);
      \\draw[\\ganttdatelinecolour] (0,0) -- (1,0);
      \\draw[\\ganttdatelinecolour] (0,1+\\ganttntasks) -- (1,1+\\ganttntasks);
    \\end{pgfonlayer}
  %s
  %s
  %s
  \\end{tikzpicture}" projecttime ntasks tasks milestones dates))
#+end_src

* Available languages

** From Org's core

| Language   | Identifier | Language       | Identifier |
|------------+------------+----------------+------------|
| Asymptote  | asymptote  | Awk            | awk        |
| Emacs Calc | calc       | C              | C          |
| C++        | C++        | Clojure        | clojure    |
| CSS        | css        | ditaa          | ditaa      |
| Graphviz   | dot        | Emacs Lisp     | emacs-lisp |
| gnuplot    | gnuplot    | Haskell        | haskell    |
| Javascript | js         | LaTeX          | latex      |
| Ledger     | ledger     | Lisp           | lisp       |
| Lilypond   | lilypond   | MATLAB         | matlab     |
| Mscgen     | mscgen     | Objective Caml | ocaml      |
| Octave     | octave     | Org-mode       | org        |
|            |            | Perl           | perl       |
| Plantuml   | plantuml   | Python         | python     |
| R          | R          | Ruby           | ruby       |
| Sass       | sass       | Scheme         | scheme     |
| GNU Screen | screen     | shell          | sh         |
| SQL        | sql        | SQLite         | sqlite     |

** From Org's contrib/babel/langs

- ob-oz.el, by Torsten Anders and Eric Schulte
- ob-fomus.el, by Torsten Anders

* Default result for block execution

When a source code block has no =:result= parameter, the default
behavior is to use the /functional value/ of the execution as the
result.  However, some languages use an option to deviate from
this default behavior.  Below is a list of such options:

| Org Babel file | Default result | Option                                     |
|----------------+----------------+--------------------------------------------|
| ob-shell.el    | output         | org-babel-shell-results-defaults-to-output |

* Python
** Collect RSS Feeds
   #+header: :results output drawer
   #+name: collect-rss-feeds
   #+begin_src python
     #!/usr/bin/env python3
     # -*- coding: utf-8; mode: python -*-
     import logging
     import xml.etree.ElementTree as ET

     import feedparser
     import requests


     logging.basicConfig(filename='collect_rss_feeds.log', level=logging.ERROR)


     def fake_feed(status_code):
         feed = ET.Element("feed", xmlns="http://www.w3.org/2005/Atom")
         ET.SubElement(feed, "title").text = "Fake Feed"

         entry = ET.SubElement(feed, "entry")
         ET.SubElement(entry, "title").text = f"{status_code} Feed error"
         ET.SubElement(entry, "link", href="http://example.org/2003/12/13/atom03")

         return ET.tostring(feed, encoding='utf-8')


     def req_feed(url):
         headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}
         try:
             r = requests.get(url, headers=headers)
             logging.info(r.raise_for_status())
         except requests.exceptions.ConnectionError as err:
             logging.exception(f'{url} - {r.status_code} - {err}\n')
             return fake_feed(r.status_code)
         except requests.HTTPError as err:
             logging.exception(f'{url} - {r.status_code} - {err}\n')
             return fake_feed(r.status_code)
         else:
             logging.info('Success!')
             return r.text


     def main():
         entry_str = '- [[{0}][{1}]]'
         feeds = {}
         for item in data:
             feed_name = item[0].upper()
             feed_url = item[1]
             parse_feed = feedparser.parse(req_feed(feed_url))
             entries = parse_feed.entries[:15]
             feeds[feed_name] = '\n'.join([entry_str.format(entry.link, entry.title)
                                           for entry in entries])

         for k,v in feeds.items():
             print(f'\n{k}\n{v}\n')


     if __name__ == '__main__':
         main()
  #+end_src

** Print orgtbl
   #+name: pd2org
   #+begin_src python :var df="df" :exports none :results value raw
     from tabulate import tabulate
     return f"return tabulate({df}, headers={df}.columns, tablefmt='orgtbl')"
   #+end_src
** Contributing to git
   #+name: git-contributing
   #+begin_src python :results output drawer :exports results
     import os
     from git import Repo

     repo = Repo(os.path.abspath('.'))

     def org_list(l):
         return [f"\n- {i}\n" for i in l]

     def unique(l):
         return sorted(set(l))

     def authors(commits):
         return [f"{i.author.name} <{i.author.email}>" for i in commits]

     def commits(repo):
         return list(repo.iter_commits("master"))

     print(*org_list(unique(authors(commits(repo)))))
   #+end_src

** Show gitlab issues
   #+name: gitlab-issues
   #+begin_src python
     import requests
     import feedparser


     def show_issues(URL):
         issues = []
         states = ['&label_name[]=Doing',
                   '&label_name[]=Ready',
                   '&label_name[]=In Progress']
         for state in states:
             full_url = URL + state
             response = requests.get(full_url)
             feed = feedparser.parse(response.text)
             for entry in feed.entries:
                 _id = entry['id'].split('/')[-1]
                 _url = entry['id']
                 _title = entry['title']
                 issues.append(f"[[{_url}][{_title} ({_id})]]")
         for issue in issues:
             print(issue)
   #+end_src