marcuskammer/emacs.d
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
emacs.d/clones/lisp/www.cliki.net/TutorialDebuggingDeadLock.html
Marcus Kammer 7e48bb96c8
Update clones
2022-10-07 15:47:14 +02:00

162 lines
No EOL
22 KiB
HTML
Raw Blame History

<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>CLiki: TutorialDebuggingDeadLock</title>
<link rel="alternate" type="application/atom+xml" title="ATOM feed of edits to current article"
href="https://www.cliki.net/site/feed/article.atom?title=TutorialDebuggingDeadLock">
<link rel="stylesheet" href="static/css/style.css">
<link rel="stylesheet" href="static/css/colorize.css">
</head>
<body>
<span class="hidden">CLiki - TutorialDebuggingDeadLock</span>
<div id="content"><div id="content-area"><div id="article-title">TutorialDebuggingDeadLock</div><div id="article">Debugging a dead-lock in a program using <tt>bordeaux-threads</tt> in <tt>ccl</tt>.<p>(Original in org-mode format from: <a href="https://gist.github.com/informatimago/5efcbb643a86df73b85dd7a11d83a93b">gist bt-ccl-debug.org</a>)<p>We have a program with three threads:<p><pre>
cl-user&gt; (list-threads)
1) #&lt;process Telnet REPL Client #1 DOWN LAYER(22) [semaphore wait] #x302002A4903D&gt;
2) #&lt;process Telnet REPL Client #1(21) [semaphore wait] #x302002A469FD&gt;
3) #&lt;process Telnet REPL Server(20) [Active] #x30200291EB5D&gt;
4) …
</pre><p>The server thread listens to connections and forks client threads for
accepted connections.<p>The client thread forks a down layer thread that loops reading bytes
from the client socket, and forwarding them up the protocol layers, up
to a buffer in a <tt>TELNET-STREAM</tt> Gray stream.<p>The client thread then goes on into a REPL loop using the
<tt>TELNET-STREAM</tt> Gray stream as <tt>*TERMINAL-IO*</tt>. Writing back to
<tt>*TERMINAL-IO*</tt> goes down to the client socket in this client thread.<p>Unfortunately, when sending a byte to the upper layer, the down layer
thread hangs waiting for the stream-lock. Who has locked this stream?<p>Neither <tt>ccl</tt> nor <tt>bordeaux-threads</tt> are very helpful in debugging that…<p><h2>Recording the thread and function that holds an lock</h2><p>What we'd want, is to know what thread are holding a lock. So we will
implement a macro shadowing <tt>BT:WITH-LOCK-HELD</tt>, to record that
information into a weak hash-table. Happily, <tt>ccl</tt> has native weak
hash-tables so we don't have to use
<tt>com.informatimago.clext.closer-weak</tt>.<p><div class="code"><span class="nonparen">
#+<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/any_and.html" class="symbol">and</a> ccl debug-condition-variables</span>)</span>
<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_defpackage.html" class="symbol"><i><span class="symbol">defpackage</span></i></a> <span class="string">"COM.INFORMATIMAGO.BORDEAUX-THREAD.PATCH"</span>
<span class="paren2">(<span class="nonparen"><span class="keyword">:use</span> <span class="string">"COMMON-LISP"</span> <span class="string">"BORDEAUX-THREADS"</span></span>)</span>
<span class="paren2">(<span class="nonparen"><span class="keyword">:shadow</span> <span class="string">"MAKE-CONDITION-VARIABLE"</span> <span class="string">"WITH-LOCK-HELD"</span></span>)</span>
<span class="paren2">(<span class="nonparen"><span class="keyword">:export</span> <span class="string">"MAKE-CONDITION-VARIABLE"</span> <span class="string">"WITH-LOCK-HELD"</span></span>)</span>
<span class="paren2">(<span class="nonparen"><span class="keyword">:documentation</span> <span class="string">"Implements MAKE-CONDITION-VARIABLE on ccl to print the name,
and WITH-LOCK-HELD to record the locking thread."</span></span>)</span></span>)</span>
<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_defpackage.html" class="symbol"><i><span class="symbol">defpackage</span></i></a> <span class="string">"COM.INFORMATIMAGO.CLEXT.TELNET.STREAM"</span>
<span class="paren2">(<span class="nonparen"><span class="keyword">:use</span> <span class="string">"COMMON-LISP"</span> <span class="string">"BORDEAUX-THREADS"</span> …</span>)</span>
#<a href="https://www.cliki.net/site/HyperSpec/Body/any_pl.html" class="symbol">+</a><span class="paren2">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/any_and.html" class="symbol">and</a> ccl debug-condition-variables</span>)</span>
<span class="paren2">(<span class="nonparen"><span class="keyword">:shadowing-import-from</span> <span class="string">"COM.INFORMATIMAGO.BORDEAUX-THREAD.PATCH"</span>
<span class="string">"MAKE-CONDITION-VARIABLE"</span> <span class="string">"WITH-LOCK-HELD"</span></span>)</span>
<span class="paren2">(<span class="nonparen"><span class="keyword">:export</span> <span class="string">"TELNET-STREAM"</span> …</span>)</span></span>)</span>
<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_defpackage.html" class="symbol"><i><span class="symbol">defpackage</span></i></a> <span class="string">"COM.INFORMATIMAGO.CLEXT.TELNET.REPL"</span>
<span class="paren2">(<span class="nonparen"><span class="keyword">:use</span> <span class="string">"COMMON-LISP"</span> <span class="string">"BORDEAUX-THREADS"</span> …</span>)</span>
#<a href="https://www.cliki.net/site/HyperSpec/Body/any_pl.html" class="symbol">+</a><span class="paren2">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/any_and.html" class="symbol">and</a> ccl debug-condition-variables</span>)</span>
<span class="paren2">(<span class="nonparen"><span class="keyword">:shadowing-import-from</span> <span class="string">"COM.INFORMATIMAGO.BORDEAUX-THREAD.PATCH"</span>
<span class="string">"MAKE-CONDITION-VARIABLE"</span> <span class="string">"WITH-LOCK-HELD"</span></span>)</span>
<span class="paren2">(<span class="nonparen"><span class="keyword">:export</span> <span class="string">"START-REPL-SERVER"</span> …</span>)</span></span>)</span>
</span></div><p>
In addition to recording the current thread, we also get the name of
the caller function from <tt>ccl:backtrace-as-list</tt>.<p>We use a <tt>PRINT-OBJECT :AROUND</tt> method to print the locking threads
when it's available<p><div class="code"><span class="nonparen">
<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_in-package.html" class="symbol">in-package</a> <span class="string">"COM.INFORMATIMAGO.BORDEAUX-THREAD.PATCH"</span></span>)</span>
<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_defparametercm_defvar.html" class="symbol"><i><span class="symbol">defvar</span></i></a> <span class="special">*status*</span> <span class="paren2">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/fun_make-hash-table.html" class="symbol">make-hash-table</a> <span class="keyword">:weak</span> <span class="keyword">:key</span> <span class="keyword">:test</span> '<a href="https://www.cliki.net/site/HyperSpec/Body/fun_eq.html" class="symbol">eq</a></span>)</span></span>)</span>
<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_defun.html" class="symbol"><i><span class="symbol">defun</span></i></a> caller <span class="paren2">(<span class="nonparen"></span>)</span> <span class="paren2">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/acc_firstcm_s_inthcm_tenth.html" class="symbol">third</a> <span class="paren3">(<span class="nonparen">ccl:backtrace-as-list</span>)</span></span>)</span></span>)</span>
<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_defmacro.html" class="symbol"><i><span class="symbol">defmacro</span></i></a> <i><span class="symbol">with-lock-held</span></i> <span class="paren2">(<span class="nonparen"><span class="paren3">(<span class="nonparen">place</span>)</span> &amp;body body</span>)</span>
<span class="paren2">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/speope_letcm_letst.html" class="symbol"><i><span class="symbol">let</span></i></a> <span class="paren3">(<span class="nonparen"><span class="paren4">(<span class="nonparen">vlock <span class="paren5">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/fun_gensym.html" class="symbol">gensym</a></span>)</span></span>)</span></span>)</span>
`<span class="paren3">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/speope_letcm_letst.html" class="symbol"><i><span class="symbol">let</span></i></a> <span class="paren4">(<span class="nonparen"><span class="paren5">(<span class="nonparen">,vlock ,place</span>)</span></span>)</span>
<span class="paren4">(<span class="nonparen"><i><span class="symbol">ccl:with-lock-grabbed</span></i> <span class="paren5">(<span class="nonparen">,vlock</span>)</span>
<span class="paren5">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_push.html" class="symbol">push</a> <span class="paren6">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/any_list.html" class="symbol">list</a> <span class="keyword">:locking</span> <span class="paren1">(<span class="nonparen">caller</span>)</span> <span class="paren1">(<span class="nonparen">bt:current-thread</span>)</span></span>)</span> <span class="paren6">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/acc_gethash.html" class="symbol">gethash</a> ,vlock <span class="special">*status*</span> '<span class="paren1">(<span class="nonparen"></span>)</span></span>)</span></span>)</span>
<span class="paren5">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/speope_unwind-protect.html" class="symbol"><i><span class="symbol">unwind-protect</span></i></a>
<span class="paren6">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/speope_progn.html" class="symbol"><i><span class="symbol">progn</span></i></a> ,@body</span>)</span>
<span class="paren6">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_pop.html" class="symbol">pop</a> <span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/acc_gethash.html" class="symbol">gethash</a> ,vlock <span class="special">*status*</span> '<span class="paren2">(<span class="nonparen"></span>)</span></span>)</span></span>)</span></span>)</span></span>)</span></span>)</span></span>)</span></span>)</span>
<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_defmethod.html" class="symbol"><i><span class="symbol">defmethod</span></i></a> <a href="https://www.cliki.net/site/HyperSpec/Body/stagenfun_print-object.html" class="symbol">print-object</a> <span class="keyword">:around</span> <span class="paren2">(<span class="nonparen"><span class="paren3">(<span class="nonparen">lock ccl::recursive-lock</span>)</span> <a href="https://www.cliki.net/site/HyperSpec/Body/syscla_stream.html" class="symbol">stream</a></span>)</span>
<span class="paren2">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/speope_letcm_letst.html" class="symbol"><i><span class="symbol">let</span></i></a> <span class="paren3">(<span class="nonparen"><span class="paren4">(<span class="nonparen">status <span class="paren5">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/acc_gethash.html" class="symbol">gethash</a> lock <span class="special">*status*</span></span>)</span></span>)</span></span>)</span>
<span class="paren3">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/speope_if.html" class="symbol"><i><span class="symbol">if</span></i></a> status
<span class="paren4">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_print-unr_dable-object.html" class="symbol">print-unreadable-object</a> <span class="paren5">(<span class="nonparen">lock <a href="https://www.cliki.net/site/HyperSpec/Body/syscla_stream.html" class="symbol">stream</a> <span class="keyword">:identity</span> <a href="https://www.cliki.net/site/HyperSpec/Body/any_t.html" class="symbol">t</a> <span class="keyword">:type</span> <a href="https://www.cliki.net/site/HyperSpec/Body/any_t.html" class="symbol">t</a></span>)</span>
<span class="paren5">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/fun_format.html" class="symbol">format</a> <a href="https://www.cliki.net/site/HyperSpec/Body/syscla_stream.html" class="symbol">stream</a> <span class="string">"~S :status ~S"</span> <span class="paren6">(<span class="nonparen">ccl:lock-name lock</span>)</span> status</span>)</span></span>)</span>
<span class="paren4">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/locfun_call-next-method.html" class="symbol">call-next-method</a></span>)</span></span>)</span></span>)</span></span>)</span>
</span></div><p>Then when the dead-lock occurs, we can have a look at the status of
the various locks, and notice immediately our culprit stream lock that
is held by not once but TWICE by the same thread! <tt>ccl</tt> only has
recursive locks, and <tt>bt:with-lock-held</tt> uses the native locking
mechanism, which is a recursive lock. But now, it is clear what
functions are involved in this double locking and the solution will be
obvious: split <tt>input-buffer-fetch-octet</tt> into an inner function that
assumes the lock is already held, and use this in %stream-read-char.
Problem solved.<p><pre>
(map nil &#039;print (com.informatimago.common-lisp.cesarum.utility:hash-table-entries *status*))
(#&lt;recursive-lock &quot;down-layer&quot; [ptr @ #x605080] #x3020028D45FD&gt;)
(#&lt;recursive-lock &quot;Telnet REPL Server Lock&quot; [ptr @ #x10D880] #x30200279729D&gt;)
(#&lt;recursive-lock &quot;telnet-stream&quot; :status ((:locking
(funcall &quot;#&lt;STANDARD-METHOD COM.INFORMATIMAGO.CLEXT.TELNET.STREAM::INPUT-BUFFER-FETCH-OCTET (COM.INFORMATIMAGO.CLEXT.TELNET.STREAM:TELNET-STREAM T)&gt;&quot; &quot;#&lt;TELNET-STREAM #x3020028D75CD&gt;&quot; &quot;NIL&quot;)
#&lt;process Telnet REPL Client #1(21) [semaphore wait] #x3020028D192D&gt;)
(:locking
(com.informatimago.clext.telnet.stream::%stream-read-char &quot;#&lt;TELNET-STREAM #x3020028D75CD&gt;&quot; &quot;NIL&quot;)
#&lt;process Telnet REPL Client #1(21) [semaphore wait] #x3020028D192D&gt;)) #x3020028D74BD&gt;
(:locking
(funcall &quot;#&lt;STANDARD-METHOD COM.INFORMATIMAGO.CLEXT.TELNET.STREAM::INPUT-BUFFER-FETCH-OCTET (COM.INFORMATIMAGO.CLEXT.TELNET.STREAM:TELNET-STREAM T)&gt;&quot; &quot;#&lt;TELNET-STREAM #x3020028D75CD&gt;&quot; &quot;NIL&quot;)
#&lt;process Telnet REPL Client #1(21) [semaphore wait] #x3020028D192D&gt;)
(:locking
(com.informatimago.clext.telnet.stream::%stream-read-char &quot;#&lt;TELNET-STREAM #x3020028D75CD&gt;&quot; &quot;NIL&quot;)
#&lt;process Telnet REPL Client #1(21) [semaphore wait] #x3020028D192D&gt;)) nil
</pre><p><h2>Naming Condition Variables</h2><p>In addition, <tt>ccl</tt> condition-variables are not named;
<tt>bordeaux-threads</tt> ignores the name parameter. So we shadow
<tt>make-condition-variable</tt> and record the name of the condition
variables in a weak hash-table, and add a <tt>print-object :around</tt>
method to print this name when available. This is very convenient
when *inspecting* threads, to see on what condition variable they're
actually waiting.<p><div class="code"><span class="nonparen">
<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_in-package.html" class="symbol">in-package</a> <span class="string">"COM.INFORMATIMAGO.BORDEAUX-THREAD.PATCH"</span></span>)</span>
<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_defparametercm_defvar.html" class="symbol"><i><span class="symbol">defvar</span></i></a> <span class="special">*names*</span> <span class="paren2">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/fun_make-hash-table.html" class="symbol">make-hash-table</a> <span class="keyword">:weak</span> <span class="keyword">:key</span> <span class="keyword">:test</span> '<a href="https://www.cliki.net/site/HyperSpec/Body/fun_eq.html" class="symbol">eq</a></span>)</span></span>)</span>
<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_defun.html" class="symbol"><i><span class="symbol">defun</span></i></a> make-condition-variable <span class="paren2">(<span class="nonparen">&amp;key name</span>)</span>
<span class="paren2">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/speope_letcm_letst.html" class="symbol"><i><span class="symbol">let</span></i></a> <span class="paren3">(<span class="nonparen"><span class="paren4">(<span class="nonparen">semaphore <span class="paren5">(<span class="nonparen">ccl:make-semaphore</span>)</span></span>)</span></span>)</span>
<span class="paren3">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/any_setf.html" class="symbol">setf</a> <span class="paren4">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/acc_gethash.html" class="symbol">gethash</a> semaphore <span class="special">*names*</span></span>)</span> name</span>)</span>
semaphore</span>)</span></span>)</span>
<span class="paren1">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_defmethod.html" class="symbol"><i><span class="symbol">defmethod</span></i></a> <a href="https://www.cliki.net/site/HyperSpec/Body/stagenfun_print-object.html" class="symbol">print-object</a> <span class="keyword">:around</span> <span class="paren2">(<span class="nonparen"><span class="paren3">(<span class="nonparen">semaphore ccl:semaphore</span>)</span> <a href="https://www.cliki.net/site/HyperSpec/Body/syscla_stream.html" class="symbol">stream</a></span>)</span>
<span class="paren2">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/speope_letcm_letst.html" class="symbol"><i><span class="symbol">let</span></i></a> <span class="paren3">(<span class="nonparen"><span class="paren4">(<span class="nonparen">name <span class="paren5">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/acc_gethash.html" class="symbol">gethash</a> semaphore <span class="special">*names*</span></span>)</span></span>)</span></span>)</span>
<span class="paren3">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/speope_if.html" class="symbol"><i><span class="symbol">if</span></i></a> name
<span class="paren4">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/mac_print-unr_dable-object.html" class="symbol">print-unreadable-object</a> <span class="paren5">(<span class="nonparen">semaphore <a href="https://www.cliki.net/site/HyperSpec/Body/syscla_stream.html" class="symbol">stream</a> <span class="keyword">:identity</span> <a href="https://www.cliki.net/site/HyperSpec/Body/any_t.html" class="symbol">t</a> <span class="keyword">:type</span> <a href="https://www.cliki.net/site/HyperSpec/Body/any_t.html" class="symbol">t</a></span>)</span>
<span class="paren5">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/fun_format.html" class="symbol">format</a> <a href="https://www.cliki.net/site/HyperSpec/Body/syscla_stream.html" class="symbol">stream</a> <span class="string">":NAME ~S"</span> name</span>)</span></span>)</span>
<span class="paren4">(<span class="nonparen"><a href="https://www.cliki.net/site/HyperSpec/Body/locfun_call-next-method.html" class="symbol">call-next-method</a></span>)</span></span>)</span></span>)</span></span>)</span>
</span></div><p><hr>
Categories: <a href="Online&#32;Tutorial.html" class="category">Online Tutorial</a> <a href="ccl.html" class="category">ccl</a> <a href="Debugging.html" class="category">Debugging</a> <a href="Thread.html" class="category">Thread</a>
Keywords: debug debugger debugging tutorial ccl bordeaux-thread thread</div></div>
<div id="footer" class="buttonbar"><ul><li><a href="TutorialDebuggingDeadLock.html">Current version</a></li>
<li><a href="https://www.cliki.net/site/history?article=TutorialDebuggingDeadLock">History</a></li>
<li><a href="https://www.cliki.net/site/backlinks?article=TutorialDebuggingDeadLock">Backlinks</a></li><li><a href="https://www.cliki.net/site/edit-article?title=TutorialDebuggingDeadLock&amp;from-revision=3830592997">Edit</a></li><li><a href="https://www.cliki.net/site/edit-article?create=t">Create</a></li></ul></div>
</div>
<div id="header-buttons" class="buttonbar">
<ul>
<li><a href="https://www.cliki.net/">Home</a></li>
<li><a href="https://www.cliki.net/site/recent-changes">Recent Changes</a></li>
<li><a href="CLiki.html">About</a></li>
<li><a href="Text&#32;Formatting.html">Text Formatting</a></li>
<li><a href="https://www.cliki.net/site/tools">Tools</a></li>
</ul>
<div id="search">
<form action="https://www.cliki.net/site/search">
<label for="search_query" class="hidden">Search CLiki</label>
<input type="text" name="query" id="search_query" value="" />
<input type="submit" value="search" />
</form>
</div>
</div>
<div id="pageheader">
<div id="header">
<span id="logo">CLiki</span>
<span id="slogan">the common lisp wiki</span>
<div id="login"><form method="post" action="https://www.cliki.net/site/login">
<label for="login_name" class="hidden">Account name</label>
<input type="text" name="name" id="login_name" class="login_input" />
<label for= "login_password" class="hidden">Password</label>
<input type="password" name="password" id="login_password" class="login_input" />
<input type="submit" name="login" value="login" id="login_submit" /><br />
<div id="register"><a href="https://www.cliki.net/site/register">register</a></div>
<input type="submit" name="reset-pw" value="reset password" id="reset_pw" />
</form>
</div>
</div>
</div>
</body></html>
Reference in a new issue View git blame Copy permalink