0a: a thunk, a propositional variable, a \nMay the *force* be with you. Let us embrace null-arity. There exists a more straightforward interpretation as well.
http://0a.io/
Fri, 05 Feb 2016 13:50:38 +0800Fri, 05 Feb 2016 13:50:38 +0800Jekyll v3.0.2End of Chapter 1<p>I used to be afraid of waking up one day and realising that everything I had experienced was a construct of the mind and the reality that I knew of did not exist, and that I was the equivalence of a deaf, blind, and anosmic person with the inability to sense anything - resembling that of a function which takes in no argument, in a sense an I/O of <a href="https://en.wikipedia.org/wiki/0A">0-arity</a> - in the actual reality wherein, as such, I have no way of knowing or interacting with the outside world. So I mentally constructed the current reality and deluded myself into thinking that it is real, and that I am a member of the most dominant species on a gigantic sphere, and that I have friends and first-degree relatives I can count on, and that things make sense here, and that it is natural for there to be consistency in the way stuff behave in this universe, which, comes to think about it, is bloody strange.</p>
<p>I used to be afraid.</p>
<p>But I no longer do. Perhaps the past never existed and I was never once afraid: all there is is the presence: the concept about the flow of time is nothing more than a mental device engendering the expectation regarding the arrival of “the future” which is too a construct of the mind in the frozen presence that never once moved. Or perhaps I’ve constructed this immediate experience only to get myself to realise that perhaps the discussion of whether something is real or exists is meaningless and that existence itself does not quite exist in the actual reality the way as depicted in any ontological thinking. Or perhaps after all these years of existence, I’m beginning to take peace in the idea that I have always been alone and that the world never existed. It has all along been me, and nothing. Very minimalistic. Very elegant. And by <em>me</em> I don’t mean the person who is doing all this thinking of course. Nor am I referring to the conscious mind which is basically a product of a certain subset of neurons in the brain. I’m not exactly sure what I am referring to. It doesn’t quite matter actually.</p>
<p>Oh, my dear readers, what I’d like to tell you is that I’ve changed quite a lot since I started writing on 0a.io last September. I am now ready for a new chapter in life. I’m thinking about doing some substantial reconstruction for this site of mine. I would be altering the way things are organized, adding new features, improving the GUI, and building it more functionally this time. The plan in mind: <a href="https://github.com/jaspervdj/hakyll">Hakyll</a> + <a href="https://github.com/rackt/redux">Redux</a> + <a href="https://github.com/dekujs/deku">Deku</a>.</p>
<p>This is the end of Chapter 1.</p>
Sat, 28 Nov 2015 00:00:00 +0800
http://0a.io/end-of-chapter-1/
http://0a.io/end-of-chapter-1/NDTM vs DTM<p>From a set-theorical point of view, non-deterministic Turing machines (NDTM) and deterministic Turing machines (DTM) are pretty similar objects. Actually, all DTMs are NDTMs: all functions are relations, and the only difference between these two objects lies in that one is defined in terms of a function, while the other is defined in terms of a relation. To be more precise, a DTM is represented by some 3-tuple <script type="math/tex">(\Sigma, Q, f)</script> where <script type="math/tex">f</script> is a function mapping from <script type="math/tex">Q \times \Sigma^k</script> into <script type="math/tex">Q \times \Sigma^{k-1} \times \{-1,0,1\}^k</script> (for some <script type="math/tex">k \geq 2</script>), while a NDTM is represented by some 3-tuple <script type="math/tex">(\Sigma, Q, r)</script> where <script type="math/tex">r</script> is a relation between <script type="math/tex">Q \times \Sigma^k</script> and <script type="math/tex">Q \times \Sigma^{k-1} \times \{-1,0,1\}^k</script>.</p>
<p>Consider the following variant of the P vs NP question: for every finite subset of any language decidable by a NDTM in a polynomial number of state transitions proportional to the size of the input, does there exist a DTM that too decides this subset of the language in a polynomial number of state transitions? Unlike the P vs NP question, this is asking about a finite subset of a language, rather than the language itself which has a cardinality of <script type="math/tex">\aleph_0</script>, and is to be answered in the affirmative.</p>
<p>For any finite subset of some language decidable by a polynomial-time NDTM, there always exists a polynomial-time DTM that decides on this subset of the language, where its set of states is the superset of the Cartesian product of this subset of the language and the set of states of the NDTM, and its transition function is a superset of the union of the transition functions of DTMs each deciding only a singleton subset of the subset (with its set of states <script type="math/tex">Q</script> changed to <script type="math/tex">\{ (S,q) : q \in Q \}</script> where <script type="math/tex">S</script> is the input string that encodes the element in the singleton), to be constructed based on how the NDTM would halt on the element. It decides the subset of the language by first reading the input string and, in a few state transitions, arriving at the corresponding initial state of the DTM that decides a singleton whose element is encoded in the input, or a rejecting state if none of the DTMs it is composed of accepts the input.</p>
<p>The idea is that so long as it is about deciding a finite subset of a language, we can always derive a DTM from a NDTM theoretically.</p>
<p>Now consider another variant of the question about the existence of a DTM whose set of states has a cardinality less than or equal to <script type="math/tex">\aleph_0</script>, rather than a DTM with a finite set of states (as how it is defined in Turing’s model). Similar to how there always exists a DTM for deciding a finite subset of a language as demonstrated above, there always exists an infinite-states DTM (so long as the axioms of our set theory allow it) for deciding a language in polynomial time, for every language decidable by a polynomial-time NDTM. And similar to the corresponding finite-states DTM for a subset of the language, the construction of such infinite-states DTM relies on the “knowledge” about how the NDTM would decide each element in the language. Therefore, once again, there is no practical way for constructing such DTM if we do not have access to the NDTM at the first place.</p>
<p>So here is a question: “<em>for every language decidable by such infinite-states DTM, does there exist a finite-states DTM that too decides it in polynomial-time?</em>” and that is one way of viewing the P vs NP question.</p>
Sun, 08 Nov 2015 00:00:00 +0800
http://0a.io/nondeterministic-Turing-machines-vs-deterministic-Turing-machines/
http://0a.io/nondeterministic-Turing-machines-vs-deterministic-Turing-machines/to \aleph 1 and beyond<p>Today’s lecture was on code abstraction. In one part Professor Danvy gave an example on how recursion can be viewed from a more mathematical point of view through demonstrating the $\mathbb{N}$-bijective nature of some set of procedures. Here is the idea:</p>
<figure class="highlight"><pre><code class="language-scheme" data-lang="scheme"><span class="p">(</span><span class="k">define</span> <span class="nv">f0</span>
<span class="p">(</span><span class="k">lambda</span> <span class="p">(</span><span class="nf">a</span><span class="p">)</span>
<span class="p">(</span><span class="nf">errorf</span> <span class="ss">'f</span> <span class="s">"is not defined for ~s"</span> <span class="nv">a</span><span class="p">)))</span>
<span class="p">(</span><span class="k">define</span> <span class="nv">f1</span>
<span class="p">(</span><span class="k">lambda</span> <span class="p">(</span><span class="nf">a</span><span class="p">)</span>
<span class="p">(</span><span class="k">if</span> <span class="p">(</span><span class="nb">zero?</span> <span class="nv">a</span><span class="p">)</span>
<span class="mi">1</span>
<span class="p">(</span><span class="nb">*</span> <span class="nv">a</span> <span class="p">(</span><span class="nf">f0</span> <span class="p">(</span><span class="nb">-</span> <span class="nv">a</span> <span class="mi">1</span><span class="p">))))))</span>
<span class="p">(</span><span class="k">define</span> <span class="nv">f2</span>
<span class="p">(</span><span class="k">lambda</span> <span class="p">(</span><span class="nf">a</span><span class="p">)</span>
<span class="p">(</span><span class="k">if</span> <span class="p">(</span><span class="nb">zero?</span> <span class="nv">a</span><span class="p">)</span>
<span class="mi">1</span>
<span class="p">(</span><span class="nb">*</span> <span class="nv">a</span> <span class="p">(</span><span class="nf">f1</span> <span class="p">(</span><span class="nb">-</span> <span class="nv">a</span> <span class="mi">1</span><span class="p">))))))</span>
<span class="p">(</span><span class="k">define</span> <span class="nv">f3</span>
<span class="p">(</span><span class="k">lambda</span> <span class="p">(</span><span class="nf">a</span><span class="p">)</span>
<span class="p">(</span><span class="k">if</span> <span class="p">(</span><span class="nb">zero?</span> <span class="nv">a</span><span class="p">)</span>
<span class="mi">1</span>
<span class="p">(</span><span class="nb">*</span> <span class="nv">a</span> <span class="p">(</span><span class="nf">f2</span> <span class="p">(</span><span class="nb">-</span> <span class="nv">a</span> <span class="mi">1</span><span class="p">))))))</span></code></pre></figure>
<p>as more procedures are enumerated in this a manner, we would be able to do factorial on a larger subset of the natural numbers. There isn’t really recursion going on since there is no self-referencing. And this can continue to infinity. But as anyone with a common sense would know, this goes as far as $\aleph_0$ by virtue to the fact that it is bijective to $\mathbb{N}$.</p>
<p>So a curious question arises: let <script type="math/tex">F_{\mathbb{N}}</script> be some set of procedures that can be enumerated like above, is it possible to construct a set of procedures, <script type="math/tex">F_{\mathbb{R}}</script>, similar to <script type="math/tex">F_{\mathbb{N}}</script> (the notation of it being similar to <script type="math/tex">F_{\mathbb{N}}</script> is rather tricky here: let’s just say it is similar in that it is related to recursion, or that it is a generalization of <script type="math/tex">F_{\mathbb{N}}</script>), that has a cardinality equal or greater than <script type="math/tex">\aleph_1</script>? Since such set of procedures cannot be enumerated as George Cantor had demonstrated more than a century ago (O! the whirligig of time), it would need to be constructed in a different manner.</p>
<p>The same question can be asked regarding Turing machine: can we formulate a model of computation, similar to that of Turing’s (or more of a generalization of it), such that there exists a single object, U, that can (similar to how every Turing machine can be simulated by a Universal Turing machine) simulate every object in this model, wherein every $r \in \mathbb{R}$ can be used to encode a distinct object? Perhaps in such model, there exists some object that can solve a subset of the Halting problem previously unsolvable (i.e. those which halt in a set of steps with a cardinality less than $\aleph_1$ (and now we have successfully drawn the <a href="http://plato.stanford.edu/entries/continuum-hypothesis/">continuum hypothesis</a> into the discussion)).</p>
Tue, 29 Sep 2015 00:00:00 +0800
http://0a.io/to-aleph-1-and-beyond/
http://0a.io/to-aleph-1-and-beyond/Attending univ lecture live for the first time<p>For anthropic machines of my age, I have gone through a good number of papers and undergrad & grad texts, (some of which I’m still struggling to decipher) - I am fortunate to have a sister currently in NUS, whose student ID can be used to gain access to databases of publishing firms around the world - and sat through a good number of video lectures - once again, I am fortunate to have been born into the era of Open Course Ware, a point in time in human history when it is possible to learn everything taught in undergrad classes by yourself without the necessity of spending a tremendous sum of money. Nonetheless, I have never physically attended a univ lecture before, until yesterday. It was a lecture on computer vision and pattern recognition, or to be more precise, that was the name of the module, CS4243, and one of the topics in CS that I am least bit familiar with and not especially passionate about. (There are pretty few modules with first digit n ≥4 whose lecture is on Monday so I ended up taking that). The professor was a man in his forties. He was not particularly charismatic but neither was he a bore. He did a great job making the lecture as interesting as possible, and it was really fun.</p>
<p>I enjoyed it.</p>
<p>As an anthropic machine, what it “feels like” in one’s mind plays a significant role in one’s overall evaluation of a certain experience. Somehow to the mind there is a remarkable distinction between watching a video recording of a lecture, and being in the lecture room attending the lecture in person, despite the fact that as far as the content of the lecture is concerned, they are just different mediums presenting the same information for the process of knowledge acquisition. As much as I hate to admit it, in the end of the day the way our minds works is hugely dominated by the “feeling” aspect of our lives, no matter how rational we may want to believe we are. But how can we give a more formal definition for these things that we label such ambiguously as “feelings” which may turn out to play a rather fundamental role in the process of thought composition?</p>
Tue, 08 Sep 2015 00:00:00 +0800
http://0a.io/attending-univ-lecture-live-for-the-first-time/
http://0a.io/attending-univ-lecture-live-for-the-first-time/Some thoughts I had while working on the conjecture<p>At a young age I thought that solving algebraic equations on paper was lame. I much prefer working things out mentally and simply put down the value for $x$ in the second step.</p>
<script type="math/tex; mode=display">% <![CDATA[
\begin{align} \frac{\frac{(x^5 + 20x)^2}{log_2(x)} - (\sqrt{\frac{x}{4}}+13x)}{159773} &= (3124x- 23901)(x^2-190) \\ x &= 16 \end{align} %]]></script>
<p>It is elegant and it made a socially awkward kid feel competent. So I gradually developed the inclination to do everything in my mind. There is something magical about the moment when you arrive at the final answer, the moment when you can let your mind be free as you no longer bear the responsibility to cache the data in your random-access memory. But we are <a href="https://www.youtube.com/watch?v=PXYeARRyDWk">humans after all</a> so we don’t really have RAM and when things get really complicated, we would have to resort to writing the working down. For example, there is no way I can solve the equation above without pen and paper. It is too complex. (Interestingly, even Wolfram Alpha can’t quite solve it, likely a consequence of <a href="https://en.wikipedia.org/wiki/Abel%E2%80%93Ruffini_theorem">Abel–Ruffini theorem</a>: the best it can do is to <a href="http://www.wolframalpha.com/input/?i=solve+x+in+%28%28%28x%5E5+%2B+20x%29%5E2%29%2F%28log_2%28x%29%29+-+%28%28x%2F4%29%5E%281%2F2%29%2B13x%29%29%2F159773+%3D+%283124x-+23901%29%28x%5E2-190%29+">use some approximation algorithm</a>. Perhaps the most efficient approach to obtain a solution for $x$ is through trial and error - checking integers that are some powers of 2 and for which $\sqrt{\frac{x}{4}}$ returns a whole number - but that requires one to have access to the information that $x$ has an integer solution.)</p>
<p>As humans we can retrieve stored information more efficiently when we receive certain sensory inputs (often in the form of EM radiation or air vibration) that we associated the information with previously. Suppose a person is given the task to compute the product of two random 4 digit numbers. Let’s say the person is not well trained in the art of multiplication, it would be a lot more difficult for her or him to do it mentally than doing it on a piece of scrap paper. Even having the two 4 digit numbers displayed in front of her or him as she or he works on the problem would ease things out slightly, comparing to merely hearing the numbers for once and relying on memories.</p>
<p>We have already been accustomed to interpreting those Arabic symbols as numbers, a concept in which morphisms can happen. So when we do the morphism (or transformation of abstract entities according to pre-defined rules preserving internal structure) in our mind, it is much easier to recall the information that would be necessary at the next phase by looking at the symbols we have written, than to assign some part of the brain to keep track of the information. It is not that the later cannot be done when the amount of information reaches a certain threshold: <a href="https://en.wikipedia.org/wiki/Kim_Peek">Kim Peek</a> had clearly demonstrated that such threshold does not quite exist. The human brain is a powerful computational device. It’d be more logical to conjecture that in general it tends not to do so as a trade-off to achieve better performance in other areas (e.g. abstract thinking) due to its limited capacity. That is why I believe the ability to forget plays a much more important role than the ability to memorise.</p>
<p>There are clearly evolutionary advantages in relying on sensory stimuli to retrieve stored information rather than employing other mechanism to be able to retrieve them on a whim. Perhaps this is how the notion of semantics arises at a higher abstraction layer.</p>
Tue, 25 Aug 2015 00:00:00 +0800
http://0a.io/while-working-on-the-conjecture/
http://0a.io/while-working-on-the-conjecture/a simple proof for a basic operational equivalence regarding multisets<p>Among my favorite Youtube channels is <a href="https://www.youtube.com/user/njwildberger/">that of Prof N J Wildberger</a>, on which he uploads explanatory videos on different topics in mathematics (along with lectures he gave in UNSW). The <a href="https://youtu.be/alMiaIbxNV4">latest video is on multisets</a>, and it happens to be related to what I am currently working with. So I figured I would compose a short proof for the interesting equivalence he pointed out at the 11th minute mark.</p>
<h3 id="theorem">Theorem:</h3>
<p>For any multisets A and B</p>
<script type="math/tex; mode=display">(A \cup B) + (A \cap B) = A + B</script>
<h3 id="proof">Proof:</h3>
<p>Firstly, let $a_k$ denote some element in $A$ or $B$ and let’s establish that</p>
<script type="math/tex; mode=display">a_i = a_j \Leftrightarrow i = j</script>
<p>Now let $n_i$ denote the number of $a_i$ occurring in $(A \cup B)$, <br />$p_i$ denotes the number of $a_i$ occurring in $(A \cap B)$, and <br />$v_i$ to denotes the number of $a_i$ occurring in $(A + B)$.</p>
<p>The equivalence theorem above can be rewritten as:</p>
<script type="math/tex; mode=display">n_i + p_i = v_i \quad \forall i \in I, \text{ where } \\ \bigcup_{i \in I} a_i \text{ is the set of distinct elements in } A \text{ and } B</script>
<p>According to the definitions of union and intersection between multisets:</p>
<script type="math/tex; mode=display">m_{A \cup B}(x) = max( m_A(x), m_B(x)) \\ m_{A \cap B}(x) = max( m_A(x), m_B(x)) \\ \text{where }m_V(x) \text{ denotes the number of element }x \text{ in } V</script>
<p>we can further rewrite the formula as</p>
<script type="math/tex; mode=display">\forall x \in X \text{ where } X \text{ is the set of distinct elements in } A \text{ and } B \\ max( m_A(x), m_B(x)) + min( m_A(x), m_B(x)) = m_A(x) + m_B(x) \\</script>
<p>and that is equivalent as stating</p>
<script type="math/tex; mode=display">max(a,b) + min(a,b) = a + b \quad \forall a,b \in \mathbb{N}</script>
<p>which is trivially true.</p>
Mon, 24 Aug 2015 00:00:00 +0800
http://0a.io/a-simple-proof-for-a-basic-operational-equivalence-regarding-multisets/
http://0a.io/a-simple-proof-for-a-basic-operational-equivalence-regarding-multisets/Don't underestimate the pigs and their Great Firewall of China<p>I stumbled upon <a href="https://news.ycombinator.com/item?id=10101653">this comment on HN</a> today:</p>
<blockquote>
<p>I was visiting China recently (my first time there). I thought bypassing The Great Firewall was going to be as simple as an “ssh -D” SOCKS setup, or a “ssh -w” tunnel. Oh boy, I was wrong. If you try this, or even a basic OpenVPN setup, you will quickly find out your VPN works fine for about 5 minutes, but then latency increases to 5sec, 10sec, 30sec(!), and then everything times out. After some research I read online the government does deep packet analysis and uses machine learning to find heuristics to guess which TCP connection or UDP stream is likely being used as a VPN. When they think there is a high probability a VPN is detected, they simply start dropping all the packets.
Encryption is not enough. You need to disguise your VPN traffic to make it look like standard HTTPS sessions (since they don’t block HTTPS). For example in a traditional HTTPS session, if the client browser downloads, say, a 500kB image over HTTPS, it will send periodical empty TCP ACK packets as it receives the data. But when using a VPN that encrypts data at the IP layer, these empty ACK packets will be encrypted, so The Great Firewall will see the client sending small ~80-120 bytes encrypted packets, and will count this as one more sign that this might be a VPN.
That’s why people in China have to use VPN tools that most westerners have never heard of: obfsproxy, ShadowVPN, SoftEther, gohop, etc. All these tools try to obfuscate and hide VPNs. I have a lot of respect for all these Chinese hackers like clowwindy who try to escape censorship, as it takes more technical prowess than you think to design a VPN that works in China.</p>
</blockquote>
<p>Everything makes sense now. No wonder I kept getting into some strange time-out with VPN services like VyprVPN, strongVPN, etc, when I was in China one and a half year ago. At first I thought it was because the ISP randomly blocked packages going to overseas servers but apparently it was a lot more sophisticated than that. The Chinese Government is paying vigilant attention to Internet censorship. (Perhaps one reason being that tension is building up in Hong Kong?) Initially I got the impression that it is all just a form of protectionism (you know, that “micro-blog” thingy would not have won against the made-in-harvard-I-am-the-CEO-bitch thingy had it not been the Big Brother’s help). But the pigs are indeed serious about social control as well.</p>
<p>And yup, recently the Chinese polices just contacted the author of <a href="https://github.com/shadowsocks">a popular open-source tool to circumvent the Great Fire Wall of China</a> and <a href="https://news.ycombinator.com/item?id=10101469">politely requested her or him</a> to stop working on the project and delete the source code on Github. So the author cleaned up the repo <a href="https://github.com/shadowsocks/shadowsocks">/shadowsocks/shadowsocks</a>, although for the repos <a href="https://github.com/shadowsocks/shadowsocks-windows">/shadowsocks/shadowsocks-windows</a> and <a href="https://github.com/shadowsocks/shadowsocks-iOS">/shadowsocks/shadowsocks-iOS</a> she or he simply created a new branch <code class="highlighter-rouge">rm</code> and switched to it, so you can still grab the source of latest versions in the <code class="highlighter-rouge">master</code> branch.</p>
Sun, 23 Aug 2015 00:00:00 +0800
http://0a.io/the-great-firewall-of-China/
http://0a.io/the-great-firewall-of-China/A strange little thing about the human conscious awareness about the flow of time<p><a href="https://www.google.com.sg/search?q=Ex+Machina"><em>Ex Machina</em></a>, the greatest motion picture ever produced revolving around the themes of artificial intelligence and human psychology, and one of the very few movies that I actually enjoyed, was just a few buttons away, in the <a href="https://plex.tv/">Plex</a> collection of a friend whom I currently stay with.</p>
<p>This afternoon while I was eating my cereal I turned on the LCD and let the movie played in the background as I sat lazily on the canapé, enjoying my breakfast in the most leisurely manner possible. It then suddenly hit me that the plot certainly felt as if it was progressing faster than it felt the first time I watched the movie.</p>
<p>In the blink of an eye the story was already in stage X, that was how it felt like.</p>
<p>Further introspection suggested that this phenomenon is not limited to just movies, but that there is clearly a discrepancy between our awareness about the flow of time when we re-watched a video for the second time, and that of when we watched it for the first time.</p>
<p>Is it a consequence of how consistent the neural networks are responding to the ever-changing visual stimuli in according to our intuitive expectation of how they would respond since we have already experienced beforehand what we are experiencing?</p>
<p>Or perhaps is it because we are prompted to compute what visual stimuli we should expect to receive next, as a response to the ever-changing visual stimuli, and the efficiency of the computation, together with the accuracy of the result, give us a feeling, which, compared to what we previously felt the first time when we watched the video, made us feel like things are going so much smoother and quicker this time?</p>
<p>It is undeniable that memories affect the way one experiences time. But in the 21st century it remains an open question at to how the brain perceives time.</p>
<p>Related: <a href="https://en.wikipedia.org/wiki/Chronostasis">chronostasis</a></p>
Sun, 09 Aug 2015 00:00:00 +0800
http://0a.io/a-strange-little-thing-about-human-conscious-awareness-about-the-flow-of-time/
http://0a.io/a-strange-little-thing-about-human-conscious-awareness-about-the-flow-of-time/A reasonable model to describe reasons, simplified<p>In June I started the <em>Daily Logs of Arch’s</em> in hopes that I’d be disciplined enough to write on a daily basis. A few days after its inception I <a href="http://0a.io/halt1/">took a long break from it</a> and eventually decided to abandon the whole idea. Surely taking blogging as a hobby and putting down words every day sounds wondrous. But from experience I’m aware that I’m far from being a good writer. I struggle to write. Often, after I have something written, it would have to be edited myriad times before I consider it presentable. If life is an <a href="https://en.wikipedia.org/wiki/Optimization_problem">optimization problem</a> and finding a solution constitutes of how one lives her or his life, let my life be an instance of it, and an efficient algorithm to an <em>optimal solution</em> in this case certainly would not consist of attempting to master the art of writing. I’d be better off trying to become a mathematician cum computer scientist (which is what I’m pursuing at the moment) since I am the type of anthropic machines that are good at deconstructing thoughts but can hardly remember anything which I have little interest in, and despite the sometimes confusing terminology and not-so-reader-friendly texts that I’ve now and then encountered, I’m glad that CS and maths are becoming an enormous part of my daily life.</p>
<p>When I don’t see good reasons in doing something, I tend not to do it. That being said, I agree I don’t always behave in such manner, and the same goes to other anthropic machines. Furthermore, even when there are good reasons to do something, we may not do it. Our actions are not entirely reasons-driven.</p>
<p>Let’s define a mental representation to be a mental construct that demonstrates some relationship between different mental representations. We refer to a mental representation as a <em>reason</em> when a consequential relationship is established between it and another mental representation (where the former is the “cause” while the later is the “result”).</p>
<p>Essentially a <em>reason</em> is just a description to account for the ‘why’ of some mental relationship.</p>
<p>In the case of actions, <em>reasons</em> are not the direct cause of them.</p>
<p>Neurotransmissions occurring at a much lower abstraction layer, as a result of the overall relations between mental representations, are what is truly responsible for our actions. That is to say, according to this model, the attributive relationships gradually established between <em>blogging on a daily basis</em> and <em>the states of being time consuming, and not important</em>, together with some other mental relations, are the true causes of the favoring of doing something else over blogging consecutively each day, and eventually the writing of this short essay you are reading.</p>
<p>No matter how good our mind is at simulating a part of reality and describing things, there is always a gap between the state of mind described by the mind, and the actual state of mind which comprises every mental process including its attempt to describe itself and its attempt to describe how it attempts to describe itself. When it comes to our own actions, there is always a discrepancy between the <em>reasons</em> we come up with and the actual causes.</p>
<p>Nonetheless, <em>reasons</em> appear to have a huge impact on the way mental relationships are constructed, transformed and demolished. They evoke emotions, draw connections and implications (connective and implicational relationships between mental representations), and most importantly, bring forth the notion of logic and make us rational creatures. The ability to reason, or in other words, the ability to construct consequential relationships, is crucial in the development of an intelligent entity. It is reasonably the essence of intelligence.</p>
Thu, 06 Aug 2015 00:00:00 +0800
http://0a.io/on-reason/
http://0a.io/on-reason/The Boolean Satisfiability Problem [S<small>AT</small>] and S<small>AT</small> solvers in 5 mins* (or more)<div class="row d_shortcuts">This is a quick sum-up on (and a beginner guide to) the Boolean Satisfiability Problem.</div>
<p><small><sup>*</sup>In 5 minutes if one skips the introduction and the words in <i>the Blocks of Grey Text</i>.<br />
<a id="blockquoteToggle">Click here to hide the introduction and all <i> Blocks of Grey Text</i>.</a>
<br />
Word count includes only words <u>not</u> in <i>the Blocks of Grey Text</i>.</small></p>
<div id="introduction">
<h3 id="small0th-minutesmall-introduction-small-classwordcount386nbspwordssmall"><small>0th minute</small>: Introduction <small class="wordCount">382 words</small></h3>
<p>In CS, when we talk about a <strong>problem</strong>, we are referring to a set of questions, where each question is an instance of the problem. “Is 7 a prime number?” and “Is 2513414231 a prime number?” are both instances of a problem that can be expressed as: “Is n a prime number?”</p>
<p>Problems can be classified based on the time or space complexity of the algorithms used to compute an answer for every instance of the problem. Among the most easy-to-understand <strong>NP-complete</strong> problems is the <strong>Boolean Satisfiability Problem</strong> (aka S<small>ATISFIABILITY</small>, or S<small>AT</small>). </p>
<p>The <strong>Boolean Satisfiability Problem</strong> is also the first problem proven<sup>{2}</sup> to be NP-complete. A problem is <b>NP-complete</b> if it belongs to the set (or <a href="https://en.wikipedia.org/wiki/Class_(set_theory)">“class”</a> if you prefer) of the hardest problems in <strong>NP</strong> - hardest in the sense that every problem ever exists in NP can be <a href="https://en.wikipedia.org/wiki/Polynomial-time_reduction">reduced</a> to them. (Thus being able to solve a NP-complete problem is equivalent to being able to solve every problem in NP). </p>
<p class="text-center"><img style="max-width:400px" src="/assets/img/SAT.jpg" /></p>
<blockquote>
<p><strong>NP</strong> is the set of decision problems for which there exists a non-deterministic Turing machine that would output a value 1, or 0, after at most O(n<sup>k</sup>) steps where k is some constant and n is the complexity of the input. We often refer to such number of steps (or running time) as polynomial, hence Non-deterministic Polynomial time, or NP. </p>
<p>A Turing machine is basically a mathematical object that formalizes that idea of algorithm (<a href="http://www.cs.virginia.edu/~robins/Turing_Paper_1936.pdf">in Turing model’s of computation</a>). A non-deterministic Turing machine (NDTM) is, in simple words, a theoretical Turing machine that can, in a sense, engage in all possible computations while taking the same time/step as a deterministic Turing machine. (<a href="http://www.encyclopediaofmath.org/index.php/Probabilistic_Turing_machine">Note: nondeterministic doesn’t mean probabilistic.</a>) <b>One way I like to think about a non-deterministic Turing machine is that it can only exist in a universe where time is a 2 dimensional plane.</b> A deterministic Turing machine (DTM) on the other hand computes in one-dimension time (and is defined in terms of a <strong>transition function</strong>, unlike NDTM which relies on a <strong>transition relation</strong>).</p>
<p>Besides the Boolean Satisfiability Problem, the prime factorization problem, “Given $X$ and $N$, where $1< X < N$, is there a $d$ where $X < d < N$ and $d$ a prime factor of $N$?”, is also a problem in <strong>NP</strong>.</p>
<p>Decision problems<sup>{1}</sup> are problems concerning only with the "trueness" of some statement. That is to say that algorithms for solving decision problems would output either <code>True</code> or <code>False</code> (often in binary value). The problem above is a decision problem, and so is the problem “Is X a prime number?”</p>
<p>I would be diving deeper into computational complexity in an upcoming essay. If you are interested in theoretical CS, I recommend you checking these out:</p>
<ol>
<li>
<p><a href="http://www.amazon.com/gp/product/0521424267/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=0521424267&linkCode=as2&tag=0aarhe-20&linkId=PHEETUYVXM6OG77J">Computational Complexity: A Modern Approach</a> (Book)</p>
</li>
<li>
<p><a href="http://www.amazon.com/gp/product/052188473X/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=052188473X&linkCode=as2&tag=0aarhe-20&linkId=PDEIBZRCW7SZRZHB">Computational Complexity: A Conceptual Perspective</a> (Book)</p>
</li>
<li>
<p><a href="http://www.amazon.com/gp/product/0716710455/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=0716710455&linkCode=as2&tag=0aarhe-20&linkId=SAE3OYM4X44YGLQS">Computers and Intractability: A Guide to the Theory of NP-Completeness</a> (Book)</p>
</li>
<li>
<p><a href="https://complexityzoo.uwaterloo.ca/">Complexity Zoo</a> (Website)</p>
</li>
<li>
<p><a href="https://rjlipton.wordpress.com/">Gödel’s Lost Letter and P=NP</a> (Website)</p>
</li>
</ol>
</blockquote>
<p>You can just think of <b>NP</b> as the set of problems whose solution can be <b>verified</b> by an efficient algorithm. And that is different from <b>P</b>, the set of problems whose solution can be <b>found</b> by an efficient algorithm. We can easily prove that each problem in P is also in NP, but we are not sure whether it is true that each problem in NP is too in P. </p>
<script type="math/tex; mode=display"> (P \subseteq NP) \text{ Duh! } \\ (NP \subseteq P) \text{ ???? }</script>
<p>Take the prime factorization problem for example. At the moment the most efficient algorithm <a href="http://www.math.vt.edu/people/brown/doc/briggs_gnfs_thesis.pdf">(for integers larger than 100 digits)</a> we can implement into pre-existing computers<sup>{3}</sup> has a sub-exponential running time. But we can’t say for certainty that the it is not in P. Many brave men and women have tried but failed to come up with a polynomial time algorithm for every instance of the problem. It may simply be the case that we, members of the Human species, are not intelligent enough to design such algorithm. (Or it may be that $NP \not= P$ and the prime factorization problem is simply not in P, in which case we have yet to prove it.)</p>
<p>If we ever come up with an algorithm that can solve all Boolean Satisfiability Problem in polynomial time, it would mean that $P = NP$ since any problem in NP can be reduced into the Boolean Satisfiability Problem. <a href="https://news.ycombinator.com/item?id=2121727">(Here’s a courageous attempt.)</a></p>
<blockquote>
<p>statement<sup>{1}</sup>: More formally, a <strong>decision problem</strong> is defined as a set of strings, $L$ composed by an alphabet $\Sigma$, where there exists a Turing Machine TM,</p>
<script type="math/tex; mode=display"> x \in L \Leftrightarrow TM(x) = 1</script>
<p>So, by the formal definition, <strong>the set of prime numbers is a decision problem</strong>. And this decision problem is known as PRIMES. In 2002, it was <a href="https://www.cs.auckland.ac.nz/~msta039/primality_v6.pdf">proven that PRIMES is in P</a>.</p>
<p>Here $L$ is often referred to as <a href="https://en.wikipedia.org/wiki/Formal_language">a language over $\Sigma$</a>.</p>
<p><sup>{2}</sup>: First proven by Cook in his 1971 paper, <a href="https://www.cs.toronto.edu/~sacook/homepage/1971.pdf"><em>The complexity of theorem-proving procedure</em></a>. It was later known as the <a href="https://en.wikipedia.org/wiki/Cook%E2%80%93Levin_theorem">Cook–Levin theorem</a>.</p>
<p><sup>{3}</sup>: By “pre-existing computerese I mean classical ones that don’t rely on quantum mechanics (for computation). The most efficient factorization algorithm we know of in the 21st century is the one described in <a href="http://arxiv.org/pdf/quant-ph/9508027v2.pdf">Shor’s 1995 paper</a>, a super-polynomial time algorithm that makes use of <a href="https://en.wikipedia.org/wiki/Quantum_Fourier_transform">quantum Fourier transform</a>. The sad (or <a href="https://edwardsnowden.com/2014/01/04/penetrating-hard-targets-and-owning-the-net/">good</a>) news is that there yet exists a quantum computer powerful enough to have <a href="https://en.wikipedia.org/wiki/RSA_problem">any practical uses of the algorithm</a>.</p>
</blockquote>
</div>
<h3 id="small1st-minutesmall-what-is-the-boolean-satisfiability-problem-small-classwordcount334nbspwordssmall"><small>1st minute</small>: What is the Boolean Satisfiability Problem? <small class="wordCount">334 words</small></h3>
<p>The <strong>Boolean Satisfiability Problem</strong> (or S<small>AT</small>) is the problem of determining if a proposition statement<sup>*</sup> is <strong>satisfiable</strong>. A propositional statement is satisfiable when it is possible to assign some true-false values for the variables in the statement such that the statement yields <code class="highlighter-rouge">True</code>. Otherwise the statement is unsatisfiable.</p>
<p>A propositional statement is simply a string made up of variables, brackets and these 3 symbols:</p>
<div class="row d_shortcuts text-center">
$\neg$ with the meaning <i>not</i>.<br /> $\neg True = False $<br />
$\land$ with the meaning <i>and</i>.<br /> $ False \land True = False $<br />
$\lor$ with the meaning <i>or</i>.<br /> $ False \lor True = True $ <br />
</div>
<p>These symbols are also known as logical connectives. They’re arranged in the order of precedence above. Here is an example of a propositional statement:</p>
<script type="math/tex; mode=display">d \lor (a \land b \land (c \lor d \land \neg a))</script>
<p>And here is an instance of The Boolean Satisfiability Problem:</p>
<script type="math/tex; mode=display">\text{Is }d \lor (a \land b \land (c \lor d \land \neg a)) \text{ satisfiable?}</script>
<p>The answer is yes, it is satisfiable. One solution:</p>
<div class="row d_shortcuts">
$a$ assigned: Any value<br />
$b$ assigned: Any value<br />
$c$ assigned: Any value<br />
$d$ assigned: True<br />
</div>
<p>Meanwhile, the answer to the question</p>
<script type="math/tex; mode=display">\text{Is } a \land \neg a \land b \land c \land d \text{ satisfiable?}</script>
<p>is no, it is unsatisfiable. That is because there is no value we can assign to $a$ such that $a \land \neg a$ would yield <code class="highlighter-rouge">True</code>, and therefore it is impossible for the entire statement to yield <code class="highlighter-rouge">True</code>.</p>
<blockquote>
<p><sup>*</sup>: More technically, it is known as a “formula” or “wff” (“well-formed formula”). “Well-formed” in the sense that it is syntactically correct such that it can be interpreted in our context. “Formula” in the sense that it is something formed by putting symbols together.</p>
<script type="math/tex; mode=display">wff \subset statement</script>
<p>”$\land\land\land A \land \lor B B B$” for example is a statement but not a formula since it is not well-formed.</p>
<p>p.s. In order to know how the symbol works here we rely on its association with the meaning of some lexical item in English, like the word “not” in the case of $\neg$, and our intuitive understanding of how “not” functions. We need to keep in mind that the symbols by themselves have no absolute meaning: some are even interpreted differently in different systems of logic (<a href="http://ncatlab.org/nlab/show/classical+logic">classical logic</a> vs <a href="http://ncatlab.org/nlab/show/intuitionistic+logic">intuitionistic logic</a> for example). But that is outside of the scope of this essay, and so for the moment let’s just take it as that these symbols indeed mean what the words mean.</p>
<p>p.s. Often in proposition logic, you would encounter these symbols as well:</p>
<div class="row d_shortcuts text-center"> $\Rightarrow$ with the meaning <i>imply (if .. then ..) </i>. [$\Leftarrow$ for different direction]<br /> $ (False \Rightarrow True) = True $<sup>{4}</sup><br />
$\Leftrightarrow$ with the meaning <i>if and only if (two-way implication)</i>.<br />$ (False \Leftrightarrow True) = False $ </div>
<p>They are omitted because statements that use these symbols can be rewritten using the basic Boolean operations: $\lor$, $\land$, and $\neg$.</p>
<script type="math/tex; mode=display">a \Rightarrow b \text{ can be rewritten as } \neg a \lor b \\ \text{without changing its satisfiability}</script>
<p>Since our main concern is satisfiability here, we’d like to be minimalist, and exclude symbols from our alphabet that are not necessary.</p>
<p>p.s. In model theory, we can view the assignment of booleans values to the variables in a statement as <strong>an interpretation</strong> of the statement, and we write</p>
<script type="math/tex; mode=display">I \models F</script>
<p>where $I$ is the interpretation and $F$ is the statement (or more technically, formula<sup>{1}</sup>) and we say $I$ is a model of $F$. With this in mind, we can say that two formulas $F_1$ and $F_2$ are equivalent as long as each model of A is a model of B, and vice versa.</p>
<script type="math/tex; mode=display">F_1 \equiv F_2 \Leftrightarrow (\forall I (I \models F_1) \Leftrightarrow (I \models F_2))</script>
</blockquote>
<p>S<small>AT</small> is a member of problems known as the <strong>Constraint Satisfaction Problems (or CSPs)</strong>. CSP is basically a generalization of S<small>AT</small>, where the followings are both to be defined:</p>
<ol>
<li>
<p>the values a variable can be assigned to (often referred to as <em>domain</em>)</p>
</li>
<li>
<p>the constraints on what makes a satisfiable statement</p>
</li>
</ol>
<p>Another member of CSPs is the <a href="http://www.eecs.berkeley.edu/~sseshia/pubdir/SMT-BookChapter.pdf"><strong>Satisfiability Modulo Theories Problem (or SMT)</strong></a>, where, instead of proposition statements (like what we are dealing with in S<small>AT</small>), it is the problem of determining the satisfiability of first-order logic statements, and whether a first-order logic statement is satisfiable depends on the theory of our choice.</p>
<p>Let’s say the theory of our choice is linear arithmetic, we would say that the statement below is satisfiable if variables can take in any value in $\mathbb{R}$, the set of real numbers.</p>
<script type="math/tex; mode=display">2a + 4b = 7</script>
<p>It is, however, unsatisfiable if we define $\mathbb{Z}$, the set of integers, to be the domain instead.</p>
<p class="text-center"> ● ● ● </p>
<h3 id="extra-small-classwordcountskip-this-if-you-want-213nbspwordssmall">Extra <small class="wordCount">(skip this if you want) 213 words</small></h3>
<p>When a satisfiable statement always yields <code class="highlighter-rouge">True</code> no matter what the value the variables take (every set of true-false assignments to the variables is a solution), we call it a <strong>tautology</strong>. Here is one:</p>
<script type="math/tex; mode=display">a \lor \neg a</script>
<p>Interestingly, the problem of determining if some statement is not a tautology can be easily reduced into the Boolean Satisfiability Problem.</p>
<p>Since a tautology always yields <code class="highlighter-rouge">True</code>, negating a tautology would give us a statement that always yields <code class="highlighter-rouge">False</code>, and that, my friend, is an unsatisfiable statement. The problem of determining if some statement, V, is not a tautology is thus equivalent to the problem of determining if the negation of the statement, V, is satisfiable. And that means the “Not-Tautology problem” is also in NP.</p>
<p>On the other hand, the Boolean Unsatisfiability Problem is in what’s known as <b>co-NP</b>, and so is the problem of determining if a statement is a tautology.</p>
<p><strong>co-NP</strong> can be thought of as the set of problems whose non-solution can be verified efficiently using a Non-Deterministic Turing Machine. At the moment “NP = co-NP?” remains an open question.</p>
<blockquote>
<p>Formally, a co-NP problem is a language<sup>*</sup> over some alphabet Σ, for which there exists Non-Deterministic Turing Machine that would output 0 in polynomial time, after taking in any string from the language as input.</p>
<p>Note that that is different from a definition of a NP problem, a language<sup>*</sup> for which there exists exists Non-Deterministic Turing Machine that would output 1 in polynomial time, after taking in any string from the language as input.</p>
<p>* : If you are confused by the usage of the word “language”, read {1} (in the <em>block of Grey Text</em>) in 0th minute: Introduction.</p>
<p>All problems in P are in (NP $\cap$ co-NP). Among other known problems in (NP $\cap$ co-NP) is the prime factorization problem.</p>
</blockquote>
<p class="text-center"> ● ● ● </p>
<p>From the 2nd minute on we’d be</p>
<ol>
<li>
<p>using the word <strong>formula</strong> (instead of “statement”) [For the reason, see <sup>{1}</sup> in 1st minute: What is the Boolean Satisfiability Problem? if you haven’t]</p>
</li>
<li>
<p>using <strong>S<small>AT</small></strong> as the abbreviation for the Boolean Satisfiability Problem</p>
</li>
</ol>
<h3 id="small2nd-minutesmall-classifying-ssmallatsmall-small-classwordcount354nbspwordssmall"><small>2nd minute:</small> Classifying S<small>AT</small> <small class="wordCount">354 words</small></h3>
<p>We can classify instances of S<small>AT</small> based on what form the formulas are in.</p>
<p><strong>3S<small>AT</small></strong> is the instances of S<small>AT</small> where the formulas are in the form</p>
<script type="math/tex; mode=display">\bigwedge_{j \in J} ( a_j \lor b_j \lor c_j )</script>
<p>where $a_j,b_j,c_j$ are called <strong>literals</strong>. A literal is either an atomic formula or its negation. One example:</p>
<script type="math/tex; mode=display">( x \lor \neg y \lor \neg x ) \land ( x \lor \neg x \lor z )</script>
<blockquote>
<p>An atomic formula is a formula that has no deeper structure: such as “$a$” in “$a \land b$”.</p>
<p>$\bigwedge$ works the same as a loop. We can arrive at the formula above by taking $J = \{ 1,2 \} \text{ and }a_1 = a_2 = x, b_1 = \neg y, b_2 = c_1 = \neg x, c_2 = z $.</p>
<script type="math/tex; mode=display">% <![CDATA[
\begin{align} &\bigwedge_{j \in J} ( a_j \lor b_j \lor c_j ) \\ =& ( a_1 \lor b_1 \lor c_1 ) \land ( a_2 \lor b_2 \lor c_2 ) \\ =& ( x \lor \neg y \lor \neg x ) \land ( x \lor \neg x \lor z ) \\ \end{align} %]]></script>
</blockquote>
<p>This is also known as the <b>Conjunctive<sup>{1}</sup> Normal Form (CNF)</b>. To be more precise, the form above is called <b>3CNF</b> (because it has 3 literals in each clause<sup>{2}</sup>). <b>kCNF</b> is the type of CNF with k number of literals in each clause.</p>
<script type="math/tex; mode=display">% <![CDATA[
\begin{align} \bigwedge_{j \in J} ( a_j \lor b_j \lor c_j ) &\text{ is 3CNF} \\ \bigwedge_{j \in J} ( a_j \lor b_j) &\text{ is 2CNF} \end{align} %]]></script>
<p>More generally, any formula is in CNF as long as it’s of the form:</p>
<script type="math/tex; mode=display">\bigwedge \bigvee x \text{ is CNF}</script>
<p>Clauses in a general CNF formula need not contain the same number of literals. Examples:</p>
<script type="math/tex; mode=display">( a \lor b) \land ( c \lor d ) \land (a \lor c \lor b \lor \neg b) \land (\neg a \lor c \lor d) \\ ( a \lor b ) \land c</script>
<p>Instances of S<small>AT</small> on statements in <strong><em>k</em>CNF</strong> are called <strong><em>k</em>S<small>AT</small></strong>, where <em>k</em> is a natural number. Great thing about 3S<small>AT</small> is that all instances of S<small>AT</small> are <a href="http://web.mit.edu/~neboat/www/6.046-fa09/rec8.pdf">reducible</a> into 3S<small>AT</small>. And that means 3S<small>AT</small> is also NP-complete. <strong>2S<small>AT</small></strong>, on the other hand, is S<small>AT</small> in 2CNF and it is not NP-complete.</p>
<p>2S<small>AT</small> is in P (and <a href="https://en.wikipedia.org/wiki/NL-complete">NL-complete</a>) and not all instances of S<small>AT</small> can be reduced into 2S<small>AT</small> (or else we can conclude that P = NP).</p>
<p>Lastly, we shall talk about <strong>HORN-S<small>AT</small></strong>, instances of S<small>AT</small> with Horn formula. When each clause contains at most 1 positive literal, we call it a <strong>Horn<sup>{3}</sup> formula</strong>. Here are some examples:</p>
<script type="math/tex; mode=display">( a \lor \neg b) \land ( c \lor \neg d ) \land (\neg a \lor \neg c \lor \neg b) \land \neg x \\ x \land y \land z \land (\neg d \lor \neg x \lor \neg i \lor \neg y) \\ ( a \lor \neg b ) \land c</script>
<div class="text-center">
<img style="max-width:300px" src="/assets/img/such horn.jpg" />
</div>
<p>Not only is Horn-S<small>AT</small> in P (just like 2S<small>AT</small>), it is also <a href="https://en.wikipedia.org/wiki/P-complete">P-complete</a>, and <a href="http://www.sciencedirect.com/science/article/pii/0743106684900141">solvable in linear time</a>. (<a href="http://cs.stackexchange.com/questions/45002/is-horn-sat-in-lin-if-so-why-is-that-not-an-indication-that-p-lin">That doesn’t mean P = LIN though.</a>)</p>
<blockquote>
<p><sup>{1}</sup> Conjunctive since it’s formed by $\land$. It yields <code class="highlighter-rouge">False</code> if one of its clauses yields <code class="highlighter-rouge">False</code>.</p>
<p><sup>{2}</sup>: Formally a <strong>clause</strong> is defined to be a <strong>disjunction</strong> of literal:</p>
<p>Dejunctive since it’s formed by $\lor$. It yields <code class="highlighter-rouge">True</code> if one of its literals yields <code class="highlighter-rouge">True</code>.</p>
<p><sup>{3}</sup> Initially in propositional logic, a Horn clause is in the form</p>
<script type="math/tex; mode=display">(\bigwedge_{a \in A} a) \Rightarrow b</script>
<p>In S<small>AT</small>, since we don’t use the $\Rightarrow$ symbol, the above is re-written as:</p>
<script type="math/tex; mode=display">b \lor (\bigvee_{a \in A} \neg a )</script>
<p>or in the case that $b$ is an absolute <code class="highlighter-rouge">True</code>, we have $(\bigwedge_{a \in A} a) \Rightarrow True$, which gives us</p>
<script type="math/tex; mode=display">\bigvee_{a \in A} \neg a</script>
<p>or in the case that $\bigvee_{a \in A}$ is an absolute <code class="highlighter-rouge">True</code>, we have $True \Rightarrow b$, giving us</p>
<script type="math/tex; mode=display">b</script>
</blockquote>
<p>This pretty much sums up generally how instances of S<small>AT</small> are classified. If you are interested in classifying S<small>AT</small> in a more ‘hard core’ manner (viewing S<small>AT</small> itself as a classification problem), <a href="http://www.cs.ucc.ie/~osullb/pubs/classification.pdf">check out this 2008 paper by David Devlin and Barry O’Sullivan</a>.</p>
<p>Perhaps you’ve also heard of problems with names like <a href="https://en.wikipedia.org/wiki/MAX-3SAT">MAXS<small>AT</small></a> and <a href="https://en.wikipedia.org/wiki/Sharp-SAT">#S<small>AT</small></a>. Instances of these problems are not really instances of S<small>AT</small>; it’d be more accurate to say that they are variations of S<small>AT</small>. MAXS<small>AT</small> is the problem of finding a boolean assignment satisfying most clauses in a CNF formula, and #S<small>AT</small> is the problem of counting the number of satisfying assignments.</p>
<h3 id="small3rd-minute-small-ssmallatsmall-solvers---general-overview-small-classwordcount337nbspwordssmall"><small>3rd minute :</small> S<small>AT</small> Solvers - General Overview <small class="wordCount">337 words</small></h3>
<p>There are many known algorithms capable of solving some (but not all) instances of S<small>AT</small> pretty efficiently. We often refer to them as <b>S<small>AT</small> solvers</b>. In general they can be classified into <strong>complete</strong> or <strong>incomplete</strong>.</p>
<p>Complete S<small>AT</small> solvers would always return either <code class="highlighter-rouge">satisfiable</code> or <code class="highlighter-rouge">unsatisfiable</code>. Incomplete S<small>AT</small> solvers, on the other hand, are less reliable - they may be unable to prove unsatisfiability (when it’s impossible for the formula to yield <code class="highlighter-rouge">True</code>), or unable to find a solution when the formula is satisfiable - but perhaps a lot more useful in scenarios when the instances of S<small>AT</small> cannot be solved by complete algorithms in reasonable time.</p>
<blockquote>
<p>To make thing simpler S<small>AT</small> solvers normally take in formulas in CNF form encoded in <a href="http://www.satcompetition.org/2009/format-benchmarks2009.html">DIMACS</a> format as inputs. Suppose we have a formula that isn’t in CNF form, we would first <a href="http://stackoverflow.com/a/11582917/2041954">convert it into CNF form</a> before inputting it into a S<small>AT</small> solver.</p>
</blockquote>
<p>To compare the performance between different S<small>AT</small> solvers, we give them a large set of formulas and see how well each performs (e.g. by the total time taken, or number of formulas solved in n-minutes, or using <a href="http://link.springer.com/chapter/10.1007%2F978-3-642-21581-0_25#page-1">a methodology called <em>careful ranking</em></a>). The set of formulas is often referred to as a benchmark instance. A solver previously lost to another solver may do better if different type of benchmark instances are used.</p>
<p>In <a href="http://satcompetition.org/2014/index.shtml">the S<small>AT</small> 2014 competition</a>, <a href="http://fmv.jku.at/lingeling/">Lingeling</a> beat all the other solvers when evaluated using application-type benchmark instances, but lose to <a href="http://www.labri.fr/perso/lsimon/glucose/">glucose</a> when hard-combinatorial-type are used.</p>
<blockquote>
<p>Due to the NP-complete nature of S<small>AT</small>, we can use S<small>AT</small> solvers to solve real world problems in areas like circuit design (for <a href="http://www.cise.ufl.edu/~zhuang/doc/TR-05.pdf">Combinational Equivalence Checking</a>), and artificial intelligence (for <a href="http://users.ics.aalto.fi/rintanen/jussi/papers/Rintanen11aaai.pdf">Automated Planning and Scheduling</a>). Application-type benchmark instances consist of formulas from real world problems.</p>
</blockquote>
<p>Many modern S<small>AT</small> solvers are based on the original <strong>DPLL</strong> algorithm designed in the 60’s. In summary, DPLL is a complete algorithm that works by assigning a variable some boolean value, checks if this breaks the overall satisfiability: if it doesn’t break, pick a new variable, and check again; if it breaks, alters the assigned value and check again (if both True-False values are tried, go back to the previous variable, alter it and check). Repeatedly doing so until we arrive at either one of these two ends:</p>
<ol>
<li>
<p>All variables are assigned to some boolean value and it does not break the overall satisfiability, and hence the formula is satisfiable.</p>
</li>
<li>
<p>No matter what boolean values a set of variables are assigned to, it would break the overall satisfiability, and hence the formula is unsatisfiable.</p>
</li>
</ol>
<h3 id="small4th-and-5th-minutes-small-ssmallatsmall-solvers---dpll-and-more-small-classwordcount613nbspwordssmall"><small>4th and 5th minutes :</small> S<small>AT</small> Solvers - DPLL and More <small class="wordCount">613 words</small></h3>
<p>You may have already imagined, DPLL is basically a <a href="https://www.youtube.com/watch?v=bkROCj-BTWE">depth-first search</a>, and that means at its heart it uses <strong>backtracking</strong>, a brute-force-like technique that, rather than checking every single possible solution one by one (which is what’s known as brute-force), it checks possible solutions by compositing part by part in a combinatorial fashion [e.g. building a solution variable by variable as described above], discarding sets of solutions that are deemed invalid by parts.</p>
<blockquote>
<p>In short, <strong>a backtracking algorithm is very similar to a classical brute-force algorithm, except that it is slightly smarter and thus more efficient</strong>. Here is an example. Imagine you are given 9 numbers</p>
<script type="math/tex; mode=display">4015,1635,8744,1236,3241,123,6126,142,13421</script>
<p>and your task is to find if there is a set of $n$ numbers adding up to a prime number $x$ where $n > 3$ and $x < 14000$. One example of an algorithm that uses backtracking would firstly</p>
<ol>
<li>
<p>form a set of 3 numbers (e.g. first 3 numbers $4015,1635,8744$)</p>
</li>
<li>
<p>get its sum (in this case $4015+1635+8744=14394$ )</p>
</li>
<li>
<p>check if it makes sense to go on adding more numbers into this set of 3 numbers.</p>
</li>
</ol>
<p>In our case $14394>14000$, so it makes no sense to go on and we abandon the last number, 8744, in our set, makes a new set of 3 numbers $4015,1635,1236$, and do the same thing to check if it makes sense to go on (in this case $4015+1635+1236=6886$). Since it did not exceed the limit of 14000, we continue to</p>
<ol>
<li>
<p>add a new number into the the set</p>
</li>
<li>
<p>get its sum (in this case it didn’t exceed 14000. We can continue)</p>
</li>
<li>
<p>now that $n>3$ (we have 4 numbers), we check if it is prime: if not we add more numbers to it (repeat step 1)</p>
</li>
</ol>
<p>once we exceed the limit we abandon the last number just like before (and hence <em>backtracking</em>)… Keep doing so until we have</p>
<ol>
<li>
<p>either tried every sensible combination (sum < 14000), but there is no prime</p>
</li>
<li>
<p>or we arrive at a solution, a set of numbers that meets our criteria.</p>
</li>
</ol>
<p>Read up on <a href="https://en.wikipedia.org/wiki/Eight_queens_puzzle">the Eight queens puzzle</a> and its backtracking algorithm and you’d have a complete grasp of the concept (if you haven’t).</p>
</blockquote>
<p>The other 2 techniques classical DPLL uses are</p>
<ol>
<li>
<p><strong>pure literal elimination</strong>: The idea is simple. If a literal appears only as positive or negative (but not both, and hence we call it <strong>pure</strong>) in the formula,</p>
<ol>
<li>
<p>we assign a value to the literal for it to yield <code class="highlighter-rouge">True</code>.</p>
</li>
<li>
<p>and delete all the clauses which contain it in the formula.</p>
</li>
</ol>
<blockquote>
<p>This is because since it is pure, we can just make it yield <code class="highlighter-rouge">True</code> and it won’t have any other consequence to the overall satisfiable.</p>
<script type="math/tex; mode=display">(a \lor b \lor \neg c) \land (a \lor \neg b) \land ( b \lor \neg c)</script>
<p>In the formula above, $a$ and $c$ are both pure, so we can give $a = True$ and $c = False$, and delete all the clauses that contain them. After that we’d end up with no clauses. And that means the formula is satisfiable with</p>
<div class="row d_shortcuts">
$a$: True<br />
$b$: Any value<br />
$c$: False<br />
</div>
</blockquote>
</li>
<li>
<p><strong>unit propagation</strong>: It is obvious that for clauses made up of 1 literal, that 1 literal has to evaluate <code class="highlighter-rouge">True</code> for the formula to yield <code class="highlighter-rouge">True</code>. Therefore</p>
<ol>
<li>
<p>we assign a value for the literal to yield <code class="highlighter-rouge">True</code></p>
</li>
<li>
<p>delete clauses that contains the literal, and instances of it in opposite polarity (the opposite of positive is negative, vice versa).</p>
</li>
</ol>
<blockquote>
<p>Take this CNF for example:</p>
<script type="math/tex; mode=display">a \land (a \lor x \lor y) \land ( \neg a \lor \neg b) \land (c \lor b ) \land (d \lor \neg b)</script>
<p>After 1st round of unit propagation, we would have $a = True$ and, after reduce the original forumla to:</p>
<script type="math/tex; mode=display">\neg b \land (c \lor b ) \land (d \lor \neg b)</script>
<p>Great! We can do a 2nd round of unit propagation with $\neg b$: make $b = False$ and reduce it into:</p>
<script type="math/tex; mode=display">c</script>
<p>And here we have it, the original formula is satisfiable with</p>
<div class="row d_shortcuts">
$a$: True<br />
$b$: False<br />
$c$: False<br />
$d$: Any value<br />
</div>
<p class="text-center"> ● ● ● </p>
<p>When we end up in a situation where there is an <strong>empty clause</strong> (a clause with 0 literals), it indicates that there is a set of variables no matter what value we assign them, we can’t satisfy the formula. And thus it is unsatisfiable.</p>
<p>Here is an example where after one round of unit propagation, we’d have an empty clause, indicating that no matter what values we assign $a$ and $b$, it won’t satisfy the formula.</p>
<script type="math/tex; mode=display">a \land b \land (\neg b \lor \neg a) \land (c \lor b \lor d) \land (d \lor a)</script>
</blockquote>
</li>
</ol>
<p>More technically, we would say that unit propagation and pure literal elimination are used at each <strong>decision level</strong>. We start off at decision level 0, and the decision level increases by 1 each time as we assign a boolean value to a variable.</p>
<blockquote>
<p>We often call such variable “assigned variable”. And that is different from “implied variables”, variables that get their values from pure literal elimination and unit propagation.</p>
</blockquote>
<p>Here is a simple implementation of classical DPLL in Haskell:</p>
<figure class="highlight"><pre><code class="language-haskell" data-lang="haskell"><span class="kr">import</span> <span class="nn">Data.List</span>
<span class="kr">data</span> <span class="kt">Satisfiability</span> <span class="o">=</span> <span class="kt">SAT</span> <span class="o">|</span> <span class="kt">UNSAT</span> <span class="kr">deriving</span> <span class="p">(</span><span class="kt">Eq</span><span class="p">,</span> <span class="kt">Show</span><span class="p">)</span>
<span class="kr">type</span> <span class="kt">Literal</span> <span class="o">=</span> <span class="kt">Int</span>
<span class="kr">type</span> <span class="kt">Variable</span> <span class="o">=</span> <span class="kt">Int</span>
<span class="kr">type</span> <span class="kt">Clause</span> <span class="o">=</span> <span class="p">[</span><span class="kt">Literal</span><span class="p">]</span>
<span class="kr">type</span> <span class="kt">VariableAssignement</span> <span class="o">=</span> <span class="p">(</span><span class="kt">Int</span><span class="p">,</span><span class="kt">Bool</span><span class="p">)</span>
<span class="c1">--extract variable from literal</span>
<span class="n">exv</span> <span class="o">::</span> <span class="kt">Literal</span> <span class="o">-></span> <span class="kt">Variable</span>
<span class="n">exv</span> <span class="n">l</span>
<span class="o">|</span> <span class="n">l</span> <span class="o">></span> <span class="mi">0</span> <span class="o">=</span> <span class="n">l</span>
<span class="o">|</span> <span class="n">l</span> <span class="o"><</span> <span class="mi">0</span> <span class="o">=</span> <span class="o">-</span><span class="n">l</span>
<span class="n">exvWithValue</span><span class="o">::</span> <span class="kt">Literal</span> <span class="o">-></span> <span class="kt">VariableAssignement</span>
<span class="n">exvWithValue</span> <span class="n">l</span>
<span class="o">|</span> <span class="n">l</span> <span class="o">></span> <span class="mi">0</span> <span class="o">=</span> <span class="p">(</span><span class="n">l</span><span class="p">,</span><span class="kt">True</span><span class="p">)</span>
<span class="o">|</span> <span class="n">l</span> <span class="o"><</span> <span class="mi">0</span> <span class="o">=</span> <span class="p">(</span><span class="o">-</span><span class="n">l</span><span class="p">,</span><span class="kt">False</span><span class="p">)</span>
<span class="n">assignTrueFor</span> <span class="o">::</span> <span class="p">[</span><span class="kt">Clause</span><span class="p">]</span> <span class="o">-></span> <span class="kt">Variable</span> <span class="o">-></span> <span class="p">[</span><span class="kt">Clause</span><span class="p">]</span>
<span class="p">(</span><span class="n">assignTrueFor</span><span class="p">)</span> <span class="n">cnf</span> <span class="n">literal</span> <span class="o">=</span> <span class="p">[</span> <span class="p">(</span><span class="o">-</span> <span class="n">literal</span><span class="p">)</span> <span class="p">`</span><span class="n">delete</span><span class="p">`</span> <span class="n">clause</span> <span class="o">|</span> <span class="n">clause</span> <span class="o"><-</span> <span class="n">cnf</span><span class="p">,</span> <span class="p">(</span><span class="n">not</span> <span class="o">$</span> <span class="n">literal</span> <span class="p">`</span><span class="n">elem</span><span class="p">`</span> <span class="n">clause</span><span class="p">)]</span>
<span class="n">hasConflict</span> <span class="o">=</span> <span class="p">(</span><span class="n">elem</span><span class="p">)</span> <span class="kt">[]</span>
<span class="n">dpll</span> <span class="o">::</span> <span class="p">([</span><span class="kt">Variable</span><span class="p">],</span> <span class="p">[</span><span class="kt">Clause</span><span class="p">])</span> <span class="o">-></span> <span class="p">[</span><span class="kt">VariableAssignement</span><span class="p">]</span> <span class="o">-></span><span class="p">(</span><span class="kt">Satisfiability</span><span class="p">,[</span><span class="kt">VariableAssignement</span><span class="p">])</span>
<span class="n">dpll</span> <span class="p">(</span><span class="n">vss</span><span class="o">@</span><span class="p">(</span><span class="n">x</span><span class="o">:</span><span class="n">vs</span><span class="p">),</span><span class="n">cnf</span><span class="p">)</span> <span class="n">as</span>
<span class="o">|</span> <span class="n">hasConflict</span> <span class="n">cnf</span> <span class="o">=</span> <span class="p">(</span><span class="kt">UNSAT</span><span class="p">,</span><span class="kt">[]</span><span class="p">)</span>
<span class="o">|</span> <span class="kr">let</span> <span class="p">(</span><span class="n">result</span><span class="p">,</span><span class="n">list</span><span class="p">)</span> <span class="o">=</span> <span class="n">enterDecisionLevelWAL</span> <span class="n">x</span><span class="p">,</span>
<span class="n">result</span> <span class="o">==</span> <span class="kt">SAT</span> <span class="o">=</span> <span class="p">(</span><span class="n">result</span><span class="p">,</span><span class="n">list</span><span class="p">)</span>
<span class="o">|</span> <span class="n">otherwise</span> <span class="o">=</span> <span class="n">enterDecisionLevelWAL</span> <span class="p">(</span><span class="o">-</span><span class="n">x</span><span class="p">)</span>
<span class="c1">-- enterDecisionLevelWAL: enter Decision Level With Assigned Literal</span>
<span class="kr">where</span> <span class="n">enterDecisionLevelWAL</span> <span class="n">theVariable</span> <span class="o">=</span> <span class="n">do_up_and_ple</span> <span class="p">(</span><span class="n">vs</span><span class="p">,</span> <span class="p">(</span><span class="n">cnf</span> <span class="p">`</span><span class="n">assignTrueFor</span><span class="p">`</span> <span class="n">theVariable</span><span class="p">))</span> <span class="p">(</span><span class="n">exvWithValue</span> <span class="n">theVariable</span><span class="o">:</span><span class="n">as</span><span class="p">)</span>
<span class="n">dpll</span> <span class="p">(</span><span class="kt">[]</span><span class="p">,</span><span class="kr">_</span><span class="p">)</span> <span class="n">as</span> <span class="o">=</span> <span class="p">(</span><span class="kt">SAT</span><span class="p">,</span><span class="n">as</span><span class="p">)</span>
<span class="c1">-- do_up_and_ple: do unit propagation && pure literal elimination</span>
<span class="n">do_up_and_ple</span> <span class="o">::</span> <span class="p">([</span><span class="kt">Variable</span><span class="p">],</span> <span class="p">[</span><span class="kt">Clause</span><span class="p">])</span> <span class="o">-></span> <span class="p">[</span><span class="kt">VariableAssignement</span><span class="p">]</span> <span class="o">-></span><span class="p">(</span><span class="kt">Satisfiability</span><span class="p">,[</span><span class="kt">VariableAssignement</span><span class="p">])</span>
<span class="n">do_up_and_ple</span> <span class="p">(</span><span class="n">vs</span><span class="p">,</span><span class="n">cnf</span><span class="p">)</span> <span class="n">as</span> <span class="o">=</span> <span class="n">dpll</span> <span class="p">(</span><span class="n">vs'</span><span class="p">,</span><span class="n">cnf'</span><span class="p">)</span> <span class="n">as'</span>
<span class="kr">where</span>
<span class="p">((</span><span class="n">vs'</span><span class="p">,</span><span class="n">cnf'</span><span class="p">),</span><span class="n">as'</span><span class="p">)</span> <span class="o">=</span> <span class="n">up_and_ple</span> <span class="p">((</span><span class="n">vs</span><span class="p">,</span><span class="n">cnf</span><span class="p">),</span><span class="n">as</span><span class="p">)</span>
<span class="n">up_and_ple</span> <span class="n">x</span> <span class="o">=</span> <span class="n">check_if_ple_gets_same_result</span> <span class="p">(</span><span class="n">ple</span> <span class="n">x'</span><span class="p">)</span> <span class="n">x'</span>
<span class="kr">where</span> <span class="n">x'</span> <span class="o">=</span> <span class="p">(</span><span class="n">up</span> <span class="p">(</span><span class="n">ple</span> <span class="n">x</span><span class="p">))</span>
<span class="n">check_if_ple_gets_same_result</span> <span class="n">x</span> <span class="n">previous</span>
<span class="o">|</span> <span class="n">x</span> <span class="o">==</span> <span class="n">previous</span> <span class="o">=</span> <span class="n">x</span>
<span class="o">|</span> <span class="n">otherwise</span> <span class="o">=</span> <span class="n">up_and_ple</span> <span class="n">x</span>
<span class="c1">-- pure literal elimination</span>
<span class="n">ple</span> <span class="p">((</span><span class="n">vs</span><span class="p">,</span><span class="n">cnf</span><span class="p">),</span><span class="n">as</span><span class="p">)</span>
<span class="o">|</span> <span class="n">length</span> <span class="n">pls</span> <span class="o">==</span> <span class="mi">0</span> <span class="o">=</span> <span class="p">((</span><span class="n">vs</span><span class="p">,</span><span class="n">cnf</span><span class="p">),</span><span class="n">as</span><span class="p">)</span>
<span class="o">|</span> <span class="n">otherwise</span> <span class="o">=</span> <span class="n">up</span> <span class="p">((</span><span class="n">vs'</span><span class="p">,</span><span class="n">cnf'</span><span class="p">),</span><span class="n">as'</span><span class="p">)</span>
<span class="kr">where</span>
<span class="n">cnf'</span> <span class="o">=</span> <span class="n">foldl</span> <span class="p">(</span><span class="n">assignTrueFor</span><span class="p">)</span> <span class="n">cnf</span> <span class="n">pls</span>
<span class="n">vs'</span> <span class="o">=</span> <span class="n">vs</span> <span class="o">\\</span> <span class="p">(</span><span class="n">fmap</span> <span class="n">exv</span> <span class="n">pls</span><span class="p">)</span>
<span class="n">as'</span> <span class="o">=</span> <span class="n">as</span> <span class="o">++</span> <span class="p">(</span><span class="n">fmap</span> <span class="n">exvWithValue</span> <span class="n">pls</span><span class="p">)</span>
<span class="n">pls</span> <span class="o">=</span> <span class="n">nubBy</span> <span class="p">(</span><span class="o">==</span><span class="p">)</span> <span class="o">$</span> <span class="n">find_pure_literals</span> <span class="n">literals</span> <span class="kt">[]</span>
<span class="n">literals</span> <span class="o">=</span> <span class="n">concat</span> <span class="n">cnf</span>
<span class="n">find_pure_literals</span> <span class="p">(</span><span class="n">x</span><span class="o">:</span><span class="n">xs</span><span class="p">)</span> <span class="n">o</span> <span class="o">=</span> <span class="n">find_pure_literals</span> <span class="n">xs</span> <span class="n">o'</span>
<span class="kr">where</span>
<span class="n">o'</span>
<span class="o">|</span> <span class="p">(</span><span class="n">x</span> <span class="p">`</span><span class="n">elem</span><span class="p">`</span> <span class="n">literals</span><span class="p">)</span> <span class="o">&&</span> <span class="p">(</span> <span class="o">-</span><span class="n">x</span> <span class="p">`</span><span class="n">elem</span><span class="p">`</span> <span class="n">literals</span><span class="p">)</span> <span class="o">=</span> <span class="n">o</span>
<span class="o">|</span> <span class="p">(</span><span class="o">-</span><span class="n">x</span> <span class="p">`</span><span class="n">elem</span><span class="p">`</span> <span class="n">literals</span><span class="p">)</span> <span class="o">=</span> <span class="o">-</span><span class="n">x</span><span class="o">:</span><span class="n">o</span>
<span class="o">|</span> <span class="n">otherwise</span> <span class="o">=</span> <span class="n">x</span><span class="o">:</span><span class="n">o</span>
<span class="n">find_pure_literals</span> <span class="kt">[]</span> <span class="n">o</span> <span class="o">=</span> <span class="n">o</span>
<span class="c1">-- unit propagation</span>
<span class="n">up</span> <span class="p">((</span><span class="n">vs</span><span class="p">,</span><span class="n">cnf</span><span class="p">),</span><span class="n">as</span><span class="p">)</span>
<span class="o">|</span> <span class="n">length</span> <span class="n">ucs</span> <span class="o">==</span> <span class="mi">0</span> <span class="o">=</span> <span class="p">((</span><span class="n">vs</span><span class="p">,</span><span class="n">cnf</span><span class="p">),</span><span class="n">as</span><span class="p">)</span>
<span class="o">|</span> <span class="n">otherwise</span> <span class="o">=</span> <span class="n">up</span> <span class="p">((</span><span class="n">vs'</span><span class="p">,</span><span class="n">cnf'</span><span class="p">),</span><span class="n">as'</span><span class="p">)</span>
<span class="kr">where</span>
<span class="n">cnf'</span> <span class="o">=</span> <span class="n">foldl</span> <span class="p">(</span><span class="n">assignTrueFor</span><span class="p">)</span> <span class="n">cnf</span> <span class="n">ucs</span>
<span class="n">as'</span> <span class="o">=</span> <span class="n">as</span> <span class="o">++</span> <span class="p">(</span><span class="n">fmap</span> <span class="n">exvWithValue</span> <span class="n">ucs</span><span class="p">)</span>
<span class="n">vs'</span> <span class="o">=</span> <span class="n">vs</span> <span class="o">\\</span> <span class="p">(</span><span class="n">fmap</span> <span class="n">exv</span> <span class="n">ucs</span><span class="p">)</span>
<span class="n">ucs</span> <span class="o">=</span> <span class="p">[</span> <span class="n">x</span> <span class="o">|</span> <span class="p">(</span><span class="n">x</span><span class="o">:</span><span class="n">xs</span><span class="p">)</span> <span class="o"><-</span> <span class="n">cnf</span><span class="p">,</span> <span class="n">xs</span> <span class="o">==</span> <span class="kt">[]</span><span class="p">]</span>
<span class="n">dpllStart</span> <span class="o">::</span> <span class="p">([</span><span class="kt">Variable</span><span class="p">],</span> <span class="p">[</span><span class="kt">Clause</span><span class="p">])</span> <span class="o">-></span><span class="p">(</span><span class="kt">Satisfiability</span><span class="p">,[</span><span class="kt">VariableAssignement</span><span class="p">])</span>
<span class="n">dpllStart</span> <span class="p">(</span><span class="n">vs</span><span class="p">,</span><span class="n">cnf</span><span class="p">)</span> <span class="o">=</span> <span class="n">do_up_and_ple</span> <span class="p">(</span><span class="n">vs</span><span class="p">,</span> <span class="n">cnf</span><span class="p">)</span> <span class="kt">[]</span></code></pre></figure>
<p>To run it, simply invoke <code class="highlighter-rouge">dpllStart</code> with a parameter in the form <code class="highlighter-rouge">([Int], [[Int]])</code></p>
<script type="math/tex; mode=display">(a \lor b) \land (a \lor c ) \land (c \lor -d) \\ \text{would be ([1,2,3],[[1,2],[1,3],[3,-4]])}</script>
<p>and it would return either <code class="highlighter-rouge">UNSAT</code> with an empty list or <code class="highlighter-rouge">SAT</code> with a list of boolean assignments.</p>
<blockquote>
<p>I’ve integrated a parser for parsing CNF formula in <a href="http://www.satcompetition.org/2009/format-benchmarks2009.html">DIMACS format</a> into the implementation above.</p>
<p><a href="https://github.com/0a-/Haskell-DPLL-SAT-Solver">It’s available on GitHub.</a></p>
</blockquote>
<p>Below is DPLL in pseudo-code with JavaScript-like syntax. (I have omitted the array that stores variable assignments.)</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="kd">function</span> <span class="nx">dpll</span><span class="p">(</span><span class="nx">formula</span><span class="p">){</span>
<span class="k">if</span><span class="p">(</span><span class="nx">formula</span><span class="p">.</span><span class="nx">has</span><span class="p">(</span><span class="nx">empty_clause</span><span class="p">)){</span>
<span class="k">return</span> <span class="s2">"UNSAT"</span><span class="p">;</span>
<span class="p">}</span><span class="k">else</span> <span class="k">if</span><span class="p">(</span><span class="nx">formula</span><span class="p">.</span><span class="nx">variables</span><span class="p">.</span><span class="nx">length</span> <span class="o">===</span> <span class="mi">0</span><span class="p">){</span>
<span class="k">return</span> <span class="s2">"SAT"</span><span class="p">;</span>
<span class="p">}</span>
<span class="kd">var</span> <span class="nx">v</span> <span class="o">=</span> <span class="nx">formula</span><span class="p">.</span><span class="nx">variables</span><span class="p">[</span><span class="mi">0</span><span class="p">];</span>
<span class="nx">formula</span><span class="p">.</span><span class="nx">variables</span><span class="p">.</span><span class="nx">shift</span><span class="p">();</span>
<span class="kd">var</span> <span class="nx">new_formula</span> <span class="o">=</span> <span class="nx">formula</span><span class="p">;</span>
<span class="nx">new_formula</span><span class="p">.</span><span class="nx">assignVariable</span><span class="p">(</span><span class="nx">v</span><span class="p">,</span> <span class="nx">TRUE</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">result</span> <span class="o">=</span> <span class="nx">doUPandPLE</span><span class="p">(</span><span class="nx">new_formula</span><span class="p">);</span>
<span class="k">if</span><span class="p">(</span><span class="nx">result</span> <span class="o">===</span> <span class="s2">"UNSAT"</span><span class="p">){</span>
<span class="nx">formula</span><span class="p">.</span><span class="nx">assignVariable</span><span class="p">(</span><span class="nx">v</span><span class="p">,</span> <span class="nx">FALSE</span><span class="p">);</span>
<span class="nx">result</span> <span class="o">=</span> <span class="nx">do_up_and_ple</span><span class="p">(</span><span class="nx">formula</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nx">result</span><span class="p">;</span>
<span class="p">}</span>
<span class="kd">function</span> <span class="nx">doUPandPLE</span><span class="p">(</span><span class="nx">f</span><span class="p">){</span>
<span class="c1">//LiteralElmination & UnitPropagation would remove elements in f.variables</span>
<span class="c1">//as they make changes to the formula</span>
<span class="k">do</span><span class="p">{</span>
<span class="nx">f</span><span class="p">.</span><span class="nx">repeatedlyDo</span><span class="p">(</span><span class="s2">"PureLiteralElmination"</span><span class="p">);</span>
<span class="nx">f</span><span class="p">.</span><span class="nx">repeatedlyDo</span><span class="p">(</span><span class="s2">"UnitPropagation"</span><span class="p">);</span>
<span class="p">}</span><span class="k">while</span><span class="p">(</span><span class="nx">f</span><span class="p">.</span><span class="nx">ifDoPureLiteralElminationCanMakeChanges</span><span class="p">());</span>
<span class="k">return</span> <span class="nx">dpll</span><span class="p">(</span><span class="nx">f</span><span class="p">);</span>
<span class="p">}</span></code></pre></figure>
<p class="text-center"> ● ● ● </p>
<p>To improve the performance of a S<small>AT</small> solver, we can use a <strong>heuristic function</strong> to determine which variable to assign a boolean value to in each decision (rather than following a fixed order). We often refer to it as a <strong>decision heuristic</strong>. One example is <strong>VSIDS</strong> introduced in <a href="http://www.princeton.edu/~chaff/software.html">CHAFF</a>, a S<small>AT</small> solver that won the 2006’s S<small>AT</small> competition.</p>
<blockquote>
<p>We call something <strong>heuristic</strong> often when there is not much rigorous maths behind it so it does not guaranteed to be optimal or perfect but it works well after some tests and that is the main reason why it is being implemented & used. Usually, a heuristic function (or algorithm) is characterized by being simple, quick and effective.</p>
</blockquote>
<p>The idea of VSIDS is simple: a variable is chosen for each decision according a ranking of these variables that changes over time. Each variable is ranked by its “activity score”, a floating point variable that would plus some constant, $C$, every time the variable shows up in a newly generated clause through a process called “conflict clause learning”. Every now and then, activity scores would be divided by another constant, $N$, hence shrinking over time, and “decaying” in the sense. For that reason, it is called VSIDS: Variable State Independent Decaying Sum.</p>
<blockquote>
<p>To learn more about VSIDS you can check out <a href="http://arxiv.org/pdf/1506.08905v1.pdf">Understanding VSIDS Branching Heuristics
in Conflict-Driven Clause-Learning S<small>AT</small> Solvers</a> and <a href="https://classes.soe.ucsc.edu/cmps217/Fall07/Project/jaeheon/final_paper/final_paper/input-dist-subm.pdf">The Effect of VSIDS on S<small>AT</small> Solver Performance</a>.</p>
<p>Conflict clause learning is a technique that generates an “conflict clause” by using an <a href="https://en.wikipedia.org/wiki/Implication_graph">implication graph</a> that examines how conflict arises after unit propagation. Conflict happens when all literals in a clause are forced to yield <code class="highlighter-rouge">False</code> (which is the case when we have an empty clause, if our algorithm deletes False literal). Thus the process above is often referred to as conflict analysis. If you are interested in learning more about it, check out <a href="http://www.msoos.org/2011/05/understanding-implication-graphs/">this great article (with pictures)</a> by <a href="https://github.com/msoos">msoos</a> from <a href="http://www.msoos.org/">Wondering of a S<small>AT</small> geek</a>. Wanna dive deeper? You’d certainly find these papers useful:</p>
<ol>
<li>
<p><a href="https://www.cs.rochester.edu/u/kautz/papers/learnIjcai.pdf">Understanding the Power of Clause Learning</a></p>
</li>
<li>
<p><a href="http://www.cs.princeton.edu/courses/archive/fall13/cos402/readings/SAT_learning_clauses.pdf">Clause Learning in S<small>AT</small></a></p>
</li>
<li>
<p><a href="https://www.jair.org/media/1410/live-1410-2304-jair.pdf">Towards Understanding and Harnessing the Potential of Clause Learning</a></p>
</li>
<li>
<p><a href="http://www.princeton.edu/~chaff/publication/iccad2001_final.pdf">Efficient Conflict Driven Learning in a Boolean Satisfiability Solver</a>.</p>
</li>
</ol>
<p>S<small>AT</small> solvers that use such technique are also known as <a href="https://en.wikipedia.org/wiki/Conflict-Driven_Clause_Learning">Conflict-Driven Clause-Learning (CDCL) S<small>AT</small> Solvers</a>.</p>
</blockquote>
<p>Other ways of improving performance of DPLL-based algorithms include</p>
<ol>
<li>
<p>adding conflict clauses (generated by conflict analysis as described above) into the formula to prevent from getting into the same conflict again.</p>
</li>
<li>
<p>implementing unit propagation by using a technique that watches 2 literals at a time, instead of keeping track of all literals, in which case <a href="http://cs.stackexchange.com/q/44924/11573">you’d need to sacrifice pure literal elimination</a>. To learn more read <a href="http://www.princeton.edu/~chaff/publication/DAC2001v56.pdf">this 2001 paper</a>.</p>
</li>
<li>
<p>Other than using heuristic for decision as described above, we can use heuristic for backtracking. And that is actually what’s done in many CDCL S<small>AT</small> Solvers.</p>
<blockquote>
<p>Backtracking like the one in classical DPLL is called chronological backtracking. Chronological in the sense that we we simply go back to the previous decision level. In heuristic backtracking, we go back to a decision level (or backtrack point) based on a <strong>heuristic function</strong>. The principal behind it is that we should learn from conflicts and avoid making the mistake of trying out all different variables assignments down a decision level only to find out that all the assignment at that decision level would only lead to unsatisfiability. That is to say that we would rely on the heuristic to help us in figuring out if the boolean assignment at some previous decision level may be a bad pick, and that we should backtrack all the way up.</p>
</blockquote>
</li>
<li>
<p>implementing some <em>restart</em> policies, which force the solver to backtrack to decision level 0 (or some other level) when some condition is met. This is to prevent the solver from getting “stuck” at some part of the <a href="http://wiki.lesswrong.com/wiki/Search_space">search space</a> for a long time. To learn more about it, check out <a href="http://users.cecs.anu.edu.au/~jinbo/07-ijcai-restarts.pdf">The Effect of Restarts on the Efficiency of Clause Learning</a> and <a href="ftp://ftp.research.microsoft.com/pub/ejh/drestart.pdf">Dynamic Restart Policies</a>.</p>
</li>
</ol>
<p>and many and many more.</p>
<h3 id="end-note---further-reading">End Note - Further reading:</h3>
<ol>
<li>
<p><a href="http://cacm.acm.org/magazines/2014/3/172516-boolean-satisfiability/fulltext">Boolean Satisfiability: Theory and Engineering (2014)</a> <small class="wordCount">755 words</small></p>
</li>
<li>
<p><a href="http://cstheory.stackexchange.com/a/6891/30600">Why is 2S<small>AT</small> in P? (a CS.SE post)</a> <small class="wordCount">300+ words</small></p>
</li>
<li>
<p><a href="http://www.princeton.edu/~chaff/publication/cade_cav_2002.pdf">The Quest for Efficient Boolean Satisfiability Solvers (2002)</a> <small class="wordCount">~9k words</small></p>
</li>
<li>
<p><a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.489.8416&rep=rep1&type=pdf">The S<small>AT</small> Phase Transition (1994)</a> <small class="wordCount">~5k words</small></p>
</li>
<li>
<p><a href="http://www.cs.cornell.edu/gomes/papers/satsolvers-kr-handbook.pdf">Satisfiability Solvers</a> <small class="wordCount">a chapter from <a href="http://www.amazon.com/gp/product/0444522115/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=0444522115&linkCode=as2&tag=0aarhe-20&linkId=UZ2R6YMF7HQ6MZSN">Handbook of Knowledge Representation</a></small></p>
</li>
<li>
<p><a href="http://www.cs.toronto.edu/~fbacchus/csc2512/Lectures/2013Readings/Skallah_Empirical_Study_SAT_Solvers.pdf">Empirical Study of the Anatomy of Modern Sat Solvers</a> <small class="wordCount">~5k words</small></p>
</li>
<li>
<p><a href="http://www.amazon.com/gp/product/1586039296/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=1586039296&linkCode=as2&tag=0aarhe-20&linkId=4SB427BUQ5LKHOFJ">Handbook of Satisfiability</a> <small class="wordCount">a book</small></p>
</li>
<li>
<p><a href="http://www.cs.toronto.edu/~chechik/courses03/csc2108/algorithms-for-satisfiability.pdf">Algorithms for the Satisfiability Problem: A Survey (1996)</a> <small class="wordCount">~60k words</small></p>
</li>
<li>
<p><a href="http://www.amazon.com/gp/product/3319092839/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=3319092839&linkCode=as2&tag=0aarhe-20&linkId=ZKNKEH4TF3DC6R7R">Theory and Applications of Satisfiability Testing - S<small>AT</small> 2014</a> <small class="wordCount">a book</small> <small>Also available on <a href="http://www.springer.com/us/book/9783319092836">springer</a>.</small></p>
</li>
</ol>
<script>
(function(){
var bqt = document.getElementById("blockquoteToggle");
var bqHidden = false;
var x = document.getElementsByTagName("blockquote"),
xx = document.getElementById("introduction");
bqt.addEventListener("click",function(){
if(!bqHidden){
bqt.innerHTML = "Click here to show the introduction and all <i> Blocks of Grey Text</i>."
for(var i = 0; i < x.length; i++){
x[i].style.display = "none";
}
xx.style.display = "none";
}else{
bqt.innerHTML= "Click here to hide the introduction and all <i> Blocks of Grey Text</i>."
for(var i = 0; i < x.length; i++){
x[i].style.display = "block";
}
xx.style.display = "block";
}
bqHidden = !bqHidden;
});
})();
</script>
Thu, 06 Aug 2015 00:00:00 +0800
http://0a.io/boolean-satisfiability-problem-or-sat-in-5-minutes/
http://0a.io/boolean-satisfiability-problem-or-sat-in-5-minutes/