#+TITLE: lisp-babel # #+DATE: <2013-08-31 Sat> #+AUTHOR: Derek Feichtinger #+EMAIL: derek.feichtinger@psi.ch #+OPTIONS: ':nil *:t -:t ::t <:t H:3 \n:nil ^:t arch:headline #+OPTIONS: author:t c:nil creator:comment d:(not LOGBOOK) date:t e:t #+OPTIONS: email:nil f:t inline:t num:t p:nil pri:nil stat:t tags:t #+OPTIONS: tasks:t tex:t timestamp:t toc:t todo:t |:t # #+CREATOR: Emacs 24.3.1 (Org mode 8.0.7) #+DESCRIPTION: #+EXCLUDE_TAGS: noexport #+KEYWORDS: #+LANGUAGE: en #+SELECT_TAGS: export # By default I do not want that source code blocks are evaluated on export. Usually # I want to evaluate them interactively and retain the original results. #+PROPERTY: header-args :eval never-export # #+SETUPFILE: ~/.emacs.d/git-submod/org-html-themes.git/setup/theme-readtheorg.setup # #+SETUPFILE: ~/.emacs.d/git-submod/org-html-themes.git/setup/theme-bigblow.setup * How to use this document You should look at this document in its Org mode source form. The PDF rendering is useful to see the results of some of the export options, but the syntax of the source block is only seen in the source text. * Version information #+BEGIN_SRC emacs-lisp :results output :exports both (princ (concat (format "Emacs version: %s\n" (emacs-version)) (format "org version: %s\n" (org-version)))) #+END_SRC #+RESULTS: : Emacs version: GNU Emacs 24.5.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.10.8) : of 2015-05-04 on dflt1w : org version: 8.3.2 * using a table as input for a src block ** simple example We first create a table from a lisp *list of lists*. Each inner list will form a row in the resulting table. I already insert a header row with the names of three columns. A separator line can be obtained by putting the =hline= symbol into the resulting list. #+NAME: make-table1 #+BEGIN_SRC emacs-lisp :results value :exports both (cons '(col1 col2 col3) (cons 'hline (loop for i from 5 to 15 collect `(,i ,(* i 5) "")))) #+END_SRC #+NAME: table1 #+RESULTS: make-table1 | col1 | col2 | col3 | |------+------+------| | 5 | 25 | | | 6 | 30 | | | 7 | 35 | | | 8 | 40 | | | 9 | 45 | | | 10 | 50 | | | 11 | 55 | | | 12 | 60 | | | 13 | 65 | | | 14 | 70 | | | 15 | 75 | | We now can fill the third column by passing the table into the next source block. We force babel to treat the first row as table header by using the *:colnames yes* header argument. This also causes the result table to contain the headers (as long as the new table has the same number of columns as the original table) Here I also demonstrate the use of the *-n* option that will export the code with line numbers. #+BEGIN_SRC emacs-lisp -n :results value :var tbl=table1 :colnames yes :exports both (let (result) (dolist (row tbl result) (setf (nth 2 row) (* 2 (nth 1 row))) (setq result (cons row result))) (reverse result)) #+END_SRC #+RESULTS: | col1 | col2 | col3 | |------+------+------| | 5 | 25 | 50 | | 6 | 30 | 60 | | 7 | 35 | 70 | | 8 | 40 | 80 | | 9 | 45 | 90 | | 10 | 50 | 100 | | 11 | 55 | 110 | | 12 | 60 | 120 | | 13 | 65 | 130 | | 14 | 70 | 140 | | 15 | 75 | 150 | ** passing a sub-range It is possible to specify a sub-range for the table that is handed over through =:var=. But currently it does not work well with the =:colnames yes= option, as the following example shows. #+BEGIN_SRC emacs-lisp -n :results value :var tbl=table1[4:8] :colnames yes :exports both (let (result) (dolist (row tbl result) (setf (nth 2 row) (* 2 (nth 1 row))) (setq result (cons row result))) (reverse result)) #+END_SRC #+RESULTS: | 7 | 35 | | |----+----+-----| | 8 | 40 | 80 | | 9 | 45 | 90 | | 10 | 50 | 100 | | 11 | 55 | 110 | ** Investigating how tables are passed to the src block #+NAME: tableCheckConv | col1 | col2 | col3 | |------+------+-------------| | 10 | str | two strings | | 20.5 | str2 | 2 strings | #+BEGIN_SRC emacs-lisp :results output :var tbl=tableCheckConv :colnames yes :exports both (pp tbl) #+END_SRC #+RESULTS: : ((10 "str" "two strings") : (20.5 "str2" "2 strings")) Note that the =raw value= output of the source block does not yield the same. It loses the string quotes of the single entries! #+BEGIN_SRC emacs-lisp :results raw value :var tbl=tableCheckConv :colnames yes :exports both tbl #+END_SRC #+RESULTS: ((10 str two strings) (20 str2 2 strings)) * TODO using a list as input for a source block #+NAME: mylist - item1 - item2 1. subitem2.1 2. subitem2.2 - item3 - subitem3.1 - item4 Let's look at the resulting structure that is passed to a source block #+BEGIN_SRC elisp :results output :var lst=mylist (pp lst) #+END_SRC #+RESULTS: : (("item1") : ("item2" : (ordered : ("subitem2.1") : ("subitem2.2"))) : ("item3" : (unordered : ("subitem3.1"))) : ("item4")) This is different from the current entry in the Manual ([[info:org#var][info:org#var]]), where it is said that only top level items are passed along. But this complete passing along of the structure opens nice and interesting ways of using lists. Need to ask whether this interface will remain stable. One important point to clarify is, why is the returned structure not exactly the result of what the Org function =org-list-to-lisp= returns? Comparing with that output we see that the current handing over of a list by the =:var= argument is losing the outermost layer of information that describes whether the top level list is of the ordered, unordered, ... type. #+BEGIN_SRC elisp :results output :var lname="mylist" (pp (save-excursion (goto-char (point-min)) (unless (search-forward-regexp (concat "#\\\+NAME: .*" lname) nil t) (error "No list of this name found: %s" lname)) (forward-line 1) (org-list-to-lisp))) #+END_SRC #+RESULTS: #+begin_example (unordered ("item1") ("item2" (ordered ("subitem2.1") ("subitem2.2"))) ("item3" (unordered ("subitem3.1"))) ("item4")) #+end_example * calling source blocks as a function ** Chaining source block execution I *can have another piece of code implicitly called* by using its name as an input variable in another code block. So, I could directly fill the third column of our initial example table without ever having to print out that table table. We can just pass into the next function a variable =tbl= and the name of the initial code block =make-table1=. #+BEGIN_SRC emacs-lisp :results value :var tbl=make-table1 :colnames yes (let (result) (dolist (row tbl result) (setf (nth 2 row) (* 2 (nth 1 row))) (setq result (cons row result))) (reverse result)) #+END_SRC #+RESULTS: | col1 | col2 | col3 | |------+------+------| | 5 | 25 | 50 | | 6 | 30 | 60 | | 7 | 35 | 70 | | 8 | 40 | 80 | | 9 | 45 | 90 | | 10 | 50 | 100 | | 11 | 55 | 110 | | 12 | 60 | 120 | | 13 | 65 | 130 | | 14 | 70 | 140 | | 15 | 75 | 150 | ** simple call syntax using CALL We first define a function in a named code block called =mydouble=. The variable x will be passed in by defining a header argument =:var x= #+NAME: mydouble #+header: :var x=2 #+BEGIN_SRC emacs-lisp :results silent :exports code (* 2 x) #+END_SRC Now we can call this babel function by using the code block's name =mydouble= from any place in the document. For example: #+CALL: mydouble(x=5) #+RESULTS: : 10 Another example where we pass in two variables x and y. #+NAME: mydivide #+header: :var x=2 y=3 #+BEGIN_SRC emacs-lisp :results silent :exports code (/ x y) #+END_SRC Note that *you can/must pass additional header arguments* to the call. The ones added at the end influence the final result (e.g. putting it into a drawer), while the ones added in [] are evaluated in the context of the original definition (e.g whether to capture the output or return a value). #+CALL: mydivide(12,3) :results value #+RESULTS: : 4 Another alternative calling syntax #+CALL: mydivide(y=2,x=10) #+RESULTS: : 5 ** calling a source block function from elisp This function is also used for table formulas #+BEGIN_SRC elisp (org-sbe mydivide (x 10) (y 3)) #+END_SRC #+RESULTS: : 3 #+NAME: srcRepeatStrings #+BEGIN_SRC elisp :results output :var s1="hello" s2="world" (princ (concat s1 " " s2)) #+END_SRC #+RESULTS: srcRepeatStrings : hello world #+BEGIN_SRC elisp :results output (princ (org-sbe srcRepeatStrings (s1 $"hello") (s2 $"world"))) #+END_SRC #+RESULTS: : : (s1 (quote "hello")) : : (s2 (quote "world")) : : (results (quote "hello world")) : hello world #+BEGIN_SRC elisp :results output (org-babel-execute-src-block nil '("emacs-lisp" "results" ((:var . "results=mydivide[](x=30, y=5)"))) '((:results . "silent"))) #+END_SRC #+RESULTS: : : (x (quote 30)) : : (y (quote 5)) : : (results (quote 6)) #+BEGIN_SRC elisp :results value :results value (org-babel-execute-src-block nil '("emacs-lisp" "(princ (format \"%s haahahaa\n\"results))" ((:var . "results=mydivide[](x=30, y=5)"))) '((:results . "silent"))) #+END_SRC #+RESULTS: : 6 haahahaa ** Naming an output table produced by a CALL If the called function produces an output table that one wants to use in subsequent function calls or in table formulas (using the =remote= keyword) one can give the CALL a name utilizing the syntax used for other org elements: #+NAME: new-make-table1 #+CALL: make-table1() #+RESULTS: new-make-table1 | col1 | col2 | col3 | |------+------+------| | 5 | 25 | | | 6 | 30 | | | 7 | 35 | | | 8 | 40 | | | 9 | 45 | | | 10 | 50 | | | 11 | 55 | | | 12 | 60 | | | 13 | 65 | | | 14 | 70 | | | 15 | 75 | | * Inline src calls Basic inline source calls: - src_emacs-lisp[:var tbl=table1]{(nth 0 (nth (- (length tbl) 1) tbl))} {{{results(=15=)}}} - src_emacs-lisp[:results value]{(org-table-get-remote-range "table1" "@>$1" )} {{{results(=15=)}}} Note, that the result gets wrapped into an ORG MACRO syntax using three curly brackets. This allows org to replace the results of the evaluation if the inline call is executed multiple times. If one uses the =:results raw= option, the results are placed "as is" into the buffer, so multiple executions will lead to multiple insertions of the result. src_emacs-lisp[:results raw]{(org-table-get-remote-range "table1" "@>$1" )} 15 Inline source code is only supposed to create one-line results. If you write code that generates multiple result lines, an error is raised: /Inline error: multiline result cannot be used/ src_emacs-lisp[:results value]{(princ "hahha\nyesyesyes" )} * Defining buffer wide variables for src blocks One can use a verbatim block like this. I define a named block =myvar= and I pass it into the variable s of the following code block. This lends itself to some nice ideas of inserting test in form of templates with some custom variable replacement #+NAME: myvar : world #+BEGIN_SRC emacs-lisp :var s=myvar :exports both (concat "hello " s) #+END_SRC #+RESULTS: : hello world * Using a :post function for post-formatting and executing generated tables Often I produce multiple tables from a source block (e.g. printing several pandas data frames). These tables do not get aligned in the org document after the execution of the code block (even though they will get aligned upon exporting the document). Also, I may want to have table calculations using =#+TBLFM= lines executed, instead of manually having to execute them in the resulting tables. The following function can be used in a :post argument for getting all tables in the output aligned and their TBLFM instructions executed, as shown further below #+NAME: srcPostAlignTables #+header: :var text="|5|22222|\n|0||\n|12|45|\n|---\n|||\n#+TBLFM:@>$1=vsum(@1..@-1)\n\n|1|22222|\n|0||\n|12|45|\n" #+BEGIN_SRC emacs-lisp :results value :exports both (with-temp-buffer (erase-buffer) (cl-assert text nil "PostAlignTables received nil instead of text ") (insert text) (beginning-of-buffer) (org-mode) (while (search-forward-regexp org-table-any-line-regexp nil t) (org-table-align) (org-table-recalculate 'iterate) (goto-char (org-table-end))) (buffer-string)) #+END_SRC #+RESULTS: srcPostAlignTables #+begin_example | 5 | 22222 | | 0 | | | 12 | 45 | |----+-------| | 17 | | #+TBLFM:@>$1=vsum(@1..@-1) | 1 | 22222 | | 0 | | | 12 | 45 | #+end_example Example without using the =:post= function: #+BEGIN_SRC emacs-lisp :results output drawer :exports both (princ (concat "#+CAPTION: Test1\n" "|A|B|C|\n" "|---\n" "|1|20|300|\n" "|200|30|4|\n" "|---\n" "||||\n" "#+TBLFM: @>$1..@>$3=vsum(@I..@II)\n" "\n#+CAPTION: Test2\n" "|A|B|C|\n" "|---\n" "|1|20|300|\n" "|200|30|4|\n" )) #+END_SRC #+RESULTS: :RESULTS: #+CAPTION: Test1 |A|B|C| |--- |1|20|300| |200|30|4| |--- |||| #+TBLFM: @>$1..@>$3=vsum(@I..@II) #+CAPTION: Test2 |A|B|C| |--- |1|20|300| |200|30|4| :END: The same example with the =:post= function: #+BEGIN_SRC emacs-lisp :results output drawer :post srcPostAlignTables(*this*) :exports both (princ (concat "#+CAPTION: Test1\n" "|A|B|C|\n" "|---\n" "|1|20|300|\n" "|200|30|4|\n" "|---\n" "||||\n" "#+TBLFM: @>$1..@>$3=vsum(@I..@II)\n" "\n#+CAPTION: Test2\n" "|A|B|C|\n" "|---\n" "|1|20|300|\n" "|200|30|4|\n" )) #+END_SRC #+RESULTS: :RESULTS: #+CAPTION: Test1 | A | B | C | |-----+----+-----| | 1 | 20 | 300 | | 200 | 30 | 4 | |-----+----+-----| | 201 | 50 | 304 | #+TBLFM: @>$1..@>$3=vsum(@I..@II) #+CAPTION: Test2 | A | B | C | |-----+----+-----| | 1 | 20 | 300 | | 200 | 30 | 4 | :END: * Library of babel The "Library of Babel" feature provides a kind of primitive function library system for org files. It allows running source blocks that have been added to it in every org file. The library is implemented as an association list with the source block names as the keys. It is stored in the =org-babel-library-of-babel= variable. Execute the following source block to load the source code blocks of the test file =lib-of-babel-test.org= into the library of babel. #+BEGIN_SRC elisp :exports code' (org-babel-lob-ingest "./lib-of-babel-test.org") #+END_SRC #+RESULTS: : 1 For example, the post table alignment function of the last section is a useful generic function. I renamed it in the =lib-of-babel-test.org= file to =srcPostAlignTablesLIB= to demonstrate that it indeed is the definition from that file. I can call the function like any normally defined named source code block which produces: #+CALL: srcPostAlignTablesLIB() #+RESULTS: #+begin_example | 5 | 22222 | | 0 | | | 12 | 45 | |----+-------| | 17 | | ,#+TBLFM:@>$1=vsum(@1..@-1) | 1 | 22222 | | 0 | | | 12 | 45 | #+end_example But more interesting for this example, I can also use it in the =:post= block: #+header: :var text="|A|22222|\n|B||\n|C|45|\n|---\n|||\n#+TBLFM:@>$2=vsum(@1..@-1)\n\n|X|3|\n|Y|4|\n|Z|5|\n|---\n|||\n#+TBLFM:@>$2=vsum(@1..@-1)\n" #+BEGIN_SRC elisp :results output raw drawer :post srcPostAlignTablesLIB(*this*) :exports both (princ text) #+END_SRC #+RESULTS: :RESULTS: | A | 22222 | | B | | | C | 45 | |---+-------| | | 22267 | #+TBLFM:@>$2=vsum(@1..@-1) | X | 3 | | Y | 4 | | Z | 5 | |---+----| | | 12 | #+TBLFM:@>$2=vsum(@1..@-1) :END: Note: Originally, I thought I could have the babel library as a local variable by executing the =org-babel-lob-ingest= on a file local variable in the local variable section of the file (using first make-local-variable and the using the ingest). But it turns out that during the ingest the buffer associated with the sourced file is active, so the local variable in this buffer remains unset. This is regrettable, because this means that the library of babel is always global. One could set the =org-babel-library-of-babel= variable directly to the final value instead of using the ingest function, but this would break the abstraction. * COMMENT org babel settings Local variables: org-confirm-babel-evaluate: nil End: