Adding draft of huffman encoding chapter.

Author: leios

chapters/data_compression/data_compression.md

@@ -142,4 +142,4 @@ L_2 &= 0.1\times 3 + 0.2 \times 3 + 0.3 \times 2 + 0.4 \times 1 = 1.9
 $$
 
 Here, it's clear that $$L_2 < L_1$$, and thus the second set of codewords compresses our data more than the first. 
-This measure can be used as a direct test of certain simple data compression techniques, notably those created by Shannon, Fano, and Huffman, which will be covered soon!
+This measure can be used as a direct test of certain simple data compression techniques, notably those created by Shannon, Fano, and [Huffman](huffman/huffman.md), which will be covered soon!

chapters/data_compression/huffman/code/julia/huffman.jl

@@ -0,0 +1,112 @@
+# This is for the PriorityQueue
+using DataStructures
+
+struct Leaf
+    weight::Int64
+    key::Char
+end
+
+struct Branch
+    right::Union{Leaf, Branch}
+    left::Union{Leaf, Branch}
+    weight::Int64
+end
+
+const Node = Union{Leaf, Branch}
+isbranch(branch::Branch) = true
+isbranch(other::T) where {T} = false
+
+function codebook_recurse(leaf::Leaf, code::String, 
+                          dict::Dict{Char,String})
+    dict[leaf.key] = code
+end
+
+function codebook_recurse(branch::Branch, code::String, 
+                          dict::Dict{Char,String})
+    codebook_recurse(branch.left, string(code, "1"), dict)
+    codebook_recurse(branch.right, string(code, "0"), dict)
+end
+
+# This will depth-first search through the tree 
+# to create bitstrings for each character.
+# Note: Any depth-first search method will work
+# This outputs encoding Dict to be used for encoding
+function create_codebook(n::Node)
+    codebook = Dict{Char,String}()
+    codebook_recurse(n, "", codebook)
+    return codebook
+end
+
+# This outputs huffman tree to generate dictionary for encoding
+function create_tree(phrase::String)
+
+    # creating weights
+    weights = PriorityQueue()
+    for i in phrase
+        temp_string = string(i)
+        if (haskey(weights, temp_string))
+            weights[temp_string] += 1
+        else
+            weights[temp_string] = 1
+        end
+    end
+
+    # Creating all nodes to iterate through
+    nodes = PriorityQueue{Node, Int64}()
+    while(length(weights) > 0)
+        weight = peek(weights)[2]
+        key = dequeue!(weights)[1]
+        temp_node = Leaf(weight, key)
+        enqueue!(nodes, temp_node, weight)
+    end
+
+    while(length(nodes) > 1)
+        node1 = dequeue!(nodes)
+        node2 = dequeue!(nodes)
+        temp_node = Branch(node1, node2, node1.weight + node2.weight)
+        enqueue!(nodes, temp_node, temp_node.weight)
+    end
+
+    huffman_tree = dequeue!(nodes)
+    return huffman_tree
+
+end
+
+function encode(codebook::Dict{Char, String}, phrase::String)
+    final_bitstring = ""
+    for i in phrase
+        final_bitstring = final_bitstring * codebook[i]
+    end
+
+    return final_bitstring
+end
+
+function decode(huffman_tree::Node, bitstring::String)
+    current = huffman_tree
+    final_string = ""
+    for i in bitstring
+        if (i == '1')
+            current = current.left
+        else
+            current = current.right
+        end
+        if (!isbranch(current))
+            final_string = final_string * string(current.key)
+            current = huffman_tree
+        end
+    end
+
+    return final_string
+end
+
+function two_pass_huffman(phrase::String)
+    huffman_tree = create_tree(phrase)
+    codebook = create_codebook(huffman_tree)
+    println(codebook)
+    bitstring = encode(codebook, phrase)
+    final_string = decode(huffman_tree, bitstring)
+    println(bitstring)
+    println(final_string)
+end
+
+two_pass_huffman("bibbity bobbity")

chapters/data_compression/huffman/code/rust/huffman.rs

@@ -0,0 +1,152 @@
+extern crate itertools;
+
+use std::cmp::{Ord, Ordering, PartialOrd};
+use std::collections::{BinaryHeap, HashMap};
+
+use itertools::Itertools;
+
+#[derive(Debug)]
+enum HuffmanTree {
+    Branch {
+        count: i32,
+        left: Box<HuffmanTree>,
+        right: Box<HuffmanTree>,
+    },
+    Leaf {
+        count: i32,
+        value: char,
+    },
+}
+
+impl PartialEq for HuffmanTree {
+    fn eq(&self, other: &Self) -> bool {
+        self.count() == other.count()
+    }
+}
+
+impl Eq for HuffmanTree {}
+
+impl PartialOrd for HuffmanTree {
+    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
+        other.count().partial_cmp(&self.count())
+    }
+}
+
+impl Ord for HuffmanTree {
+    fn cmp(&self, other: &Self) -> Ordering {
+        other.count().cmp(&self.count())
+    }
+}
+
+#[derive(Debug)]
+struct Codebook {
+    codebook: HashMap<char, String>,
+    tree: HuffmanTree,
+}
+
+impl HuffmanTree {
+    pub fn from(input: &str) -> Self {
+        let counts = input.chars().fold(HashMap::new(), |mut map, c| {
+            *map.entry(c).or_insert(0) += 1;
+            map
+        });
+        let mut queue = counts
+            .iter()
+            .map(|(&value, &count)| HuffmanTree::Leaf { value, count })
+            .collect::<BinaryHeap<HuffmanTree>>();
+
+        while queue.len() > 1 {
+            let left = queue.pop().unwrap();
+            let right = queue.pop().unwrap();
+            queue.push(HuffmanTree::Branch {
+                count: left.count() + right.count(),
+                left: Box::new(left),
+                right: Box::new(right),
+            })
+        }
+
+        queue.pop().expect("The Huffman tree has to have a root")
+    }
+
+    pub fn count(&self) -> i32 {
+        match *self {
+            HuffmanTree::Branch { count, .. } => count,
+            HuffmanTree::Leaf { count, .. } => count,
+        }
+    }
+
+    pub fn make_codebook(self) -> Codebook {
+        let mut codebook = HashMap::new();
+        self.dfs(String::from(""), &mut codebook);
+        Codebook {
+            codebook,
+            tree: self,
+        }
+    }
+
+    pub fn decode(&self, input: &str) -> String {
+        let mut result = String::from("");
+        let mut start = 0;
+        while !input[start..].is_empty() {
+            start += self.decode_dfs(&input[start..], &mut result);
+        }
+        result
+    }
+
+    fn decode_dfs(&self, input: &str, result: &mut String) -> usize {
+        let current = input.chars().next();
+        match *self {
+            HuffmanTree::Branch { ref left, .. } if current == Some('0') => {
+                1 + left.decode_dfs(&input[1..], result)
+            }
+            HuffmanTree::Branch { ref right, .. } if current == Some('1') => {
+                1 + right.decode_dfs(&input[1..], result)
+            }
+            HuffmanTree::Leaf { value, .. } => {
+                result.push(value);
+                0
+            }
+            _ => panic!("Unexpected end of input"),
+        }
+    }
+
+    fn dfs(&self, code: String, codebook: &mut HashMap<char, String>) {
+        match *self {
+            HuffmanTree::Branch {
+                ref left,
+                ref right,
+                ..
+            } => {
+                left.dfs(code.clone() + "0", codebook);
+                right.dfs(code.clone() + "1", codebook);
+            }
+            HuffmanTree::Leaf { value, .. } => {
+                codebook.insert(value, code);
+            }
+        }
+    }
+}
+
+impl Codebook {
+    fn encode(&self, input: &str) -> String {
+        input.chars().map(|c| &self.codebook[&c]).join("")
+    }
+
+    fn decode(&self, input: &str) -> String {
+        self.tree.decode(input)
+    }
+}
+
+fn main() {
+    let input = "hello, world";
+
+    let tree = HuffmanTree::from(input);
+    let codebook = tree.make_codebook();
+    let encoded = codebook.encode(input);
+    let decoded = codebook.decode(&encoded);
+
+    // Uncomment this line if you want to see the codebook/tree
+    // println!("{:#?}", codebook);
+    println!("{}", encoded);
+    println!("{}", decoded);
+}

chapters/data_compression/huffman/huffman.md

@@ -24,9 +24,65 @@ $$
 # Huffman Encoding
 
 If there were ever a data compression method to take the world by storm, it would be Huffman encoding. 
-In fact, this was the method that got me into methods to begin with.
+In fact, this was the method that got me into computational methods to begin with.
 I distinctly remember sitting in my data compression class and talking about the great information theorist Claude Shannon and Robert Fano, when suddenly my professor introduced a new kid to the mix: David Huffman.
-He managed to rip the heart out of the methods described by leaders of the field and create a data compression method that was easier to understand and implement, while also providing more robust results.
+He managed to rip the heart out of the methods described by leaders of the field and create a data compression method that was easier to understand and implement, while also providing more robust results, and apparently this was all done for a school project!
+
 It was in that moment, I knew I would never amount to anything.
+I have since accepted that fact and moved on.
+
+Huffman encoding follows from the problem described in the [Data Compression](../data_compression.md) section.
+We have a string that we want to encode into bits.
+Huffman encoding ensures that our encoded bitstring is as small as possible without losing any information.
+Because it is both lossless and gaurantees the smallest possible bitlength, it outright replaces both Shannon and Shannon-Fano encoding in most cases, which is a little weird because the method was devised while Huffman was taking a course from Fano, himself!
+
+The idea is somewhat straightforward in principle, but a little difficult to code in practice. 
+By creating a binary tree of the input alphabet, every branch can be provided a unique bit representation simply by assigning a binary value to each child and reading to a character in a leaf node if starting from the root node. 
+
+So now the question is: how do we create a binary tree?
+Well, here we build it from the bottom up like so:
+
+1. Order all characters according to the frequency they appear in the input bitstring, with the most frequent character at the top of the list. Be sure to keep track of the frequencies, too!
+2. Add the smallest two values together to create a new node with a new frequency.
+3. Keep doing step 2 until the tree is complete.
+4. Read the tree backwards from the root node and concatenate the final bitstring codeword. Keep all codewords and put them into your final set of codewords (sometimes called a codebook)
+5. Encode your phrase with the codebook.
+
+And that's it. 
+Here's an image of what this might look like for the phrase `bibbity_bobbity`:
+
+<p align="center">
+    <img src="res/huffman_tree.png" width="500" height="500" />
+</p>
+
+This will create a codebook that looks like this:
+
+| Character | Bit Representation |
+| --------- | ------------------ |
+| _b_       | 0                  |
+| _i_       | 100                |
+| _t_       | 101                |
+| _y_       | 110                |
+| _o_       | 1110               |
+| ___       | 1111               |
+
+and `bibbity bobbity` becomes `01000010010111011110111000100101110`.
+As mentioned this uses the minimum number of bits possible for encoding. 
+The fact that this algorithm is both conceptually simple and provably useful is rather extraordinary to me and is why Huffman encoding will always hold a special place in my heart.
+
+# Example Code
+In code, this can be a little tricky. It requires a method to continually sort the nodes as you add more and more nodes to the system.
+The most straightforward way to do this in some languages is with a priority queue, but depending on the language, this might be more or less appropriate.
+In addition, to read the tree backwards, some sort of [Depth First Search](../../tree_traversal/tree_traversal.md) needs to be implemented. 
+Whether you use a stack or straight-up recursion also depends on the language, but the recursive method is a little easier to understand in most cases.
+
+{% method %}
+{% sample lang="jl" %}
+### Julia
+[import, lang:"julia"](code/julia/huffman.jl)
+{% sample lang="rs" %}
+### Rust
+[import, lang:"rust"](code/rust/huffman.rs)
+{% endmethod %}
+
 
-Anyway, I'll be updating this chapter on Huffman encoding soon with all the information you could want / need, so stay tuned!