adding first visualizations and rough draft for chapters

Author: leios

contents/convolutions/1d/1d.md

@@ -1,29 +1,118 @@
-# Convolutions on images
+# Convolutions on Images
 
-Unlike the section on [one-dimensional convolutions](../1d/1d.md), for this section, we will no longer be focusing on signals, but instead images.
-For the purposes of this chapter, an image will be an array filled with elements with some red, green, and blue value associated with it; however, for the code examples, greyscale images may be used where each array element is simply composed of some floating-point value.
+For this section, we will no longer be focusing on signals, but instead images, where an image is an array filled with elements of red, green, and blue values.
+That said, for the code examples, greyscale images may be used such that each array element is simply composed of some floating-point value.
 
-In this case, extending the one-dimensional convolution to two-dimensions is a relatively straightforward task, but indexing still requires some thought.
+The extension of one-dimensional convolutions to two dimensions requires a little thought about indexing and the like, but is ultimately the same operation.
+Here is an animation of a convolution for a two-dimensional image:
 
-ADD CODE AND VIDEO
+ADD ANIMATION
 
-# Common filters
+In this case, we convolved the image with a 3x3 square filter.
+In code, using simple boundaries, it would look like this:
 
-For image processing, there are quite a few relatively common filters to use for various tasks.
-For this section, we will cover 2 of them: Gaussian and Sobel.
+ADD CODE
+
+It is worth highlighting common filters used for convolutions of images.
+In particular, we will further discuss the Gaussian filter introduced in the section on [one-dimensional convolutions](../1d/1d.md), and then introduce another set of kernels known as Sobel operators, which are used for naive edge detection or image derivatives.
 
 ## The Gaussian kernel
 
 The Gaussian kernel serves as an effective *blurring* operation for images.
 Like we showed with the triangle filter in the section on [one-dimensional convolutions](../1d/1d.md), the Gaussian kernel effectively adds a small amount of the neighboring elements to each pixel in an image.
 
+As a reminder, the foruma for any Gaussian distribution is
+
+$$
+g(x,y) = \frac{1}{2\pi\sigma^2}e^{-\frac{x^2+y^2}{2\sigma^2}},
+$$
+
+where $$\sigma$$ is the standard deviation and is a measure of the width of the Gaussian.
+A larger $$\sigma$$ means a larger Gaussian; however, Remember that the Gaussian must fit onto the filter!
+As a general rule of thumb, the larger the filter and standard deviation, the more "smeared" the final convolution will be.
+
+At this stage, it is important to write some code, so we will generate a simple function to return a Gaussian kernel with a specified standard deviation and filter size.
+As a note, all the kernels will be scaled at the end by the sum of all internal elements.
+This simply ensures that the output of the convolution does not have an obnoxious scale factor associated with it.
+
+ADD CODE
+
+
+Below are a few kernels generated with the above code along with their application to a ___ image.
+
+Add 1 + 2x4 image with original image going into each lane and then showing the convolution result for small, large, and somewhat in-between.
+
+In (a), we show a 3x3 kernel with a standard deviation of 1.
+In (b), we show a 10x10 kernel with a standard deviation of 5.
+In (c), we show a 3x3 kernel with a standard deviation of 5.
+Finally, in (d), we show a 10x10 kernel with a standard deviation of 1.
+
+We see that (a) and (d) are somewhat similar, (b) has the most blurred boundary, and (c) is partially blurred.
+
+There is a bit more to this, but I think this is a good place to wrap up the discussion on the Gaussian blurring operation before moving onto a slightly more complex convolutional method: the Sobel operator.
 
 ## The Sobel operator
 
 The Sobel operator effectively performs a gradient operation on an image by highlighting areas where a large change has been made and can be considered a naive edge detector.
-It is also the first non-trivial example of convolution.
 That is to say that the $$n$$-dimensional Sobel operator is composed of $$n$$ separate gradient convolutions that are then combined together into one, output array.
-
+Again, for the purposes of this chapter, we will stick to two dimensions, which will be composed of two separate gradients along the $$x$$ and $$y$$ directions.
+Each gradient will be created by convolving our image with their corresponding Sobel operator:
+
+$$
+\begin{align}
+S_x &= \left(\begin{bmatrix}
+1 \\
+2 \\
+1 \\
+\end{bmatrix} \otimes [1~0~-1]
+\right) = \begin{bmatrix}
+1 & 0 & -1 \\
+2 & 0 & -2 \\
+1 & 0 & -1 \\
+\end{bmatrix}\\
+
+S_y &= \left(
+\begin{bmatrix}
+1 \\
+0 \\
+-1 \\
+\end{bmatrix} \otimes [1~2~1]
+\right) = \begin{bmatrix}
+1 & 2 & 1 \\
+0 & 0 & 0 \\
+-1 & -2 & -1 \\
+\end{bmatrix}.
+\end{align}
+$$
+
+The gradients can then be found with a convolution, such that:
+
+$$
+\begin{align}
+G_x &= S_x*A \\
+G_y &= S_y*A.
+\end{align}
+$$
+
+Finally, these gradients can be summed in quadrature to find the total Sobel operator or image gradient:
+
+$$
+G_{\text{total}} = \sqrt{G_x^2 + G_y^2}
+$$
+
+At this point, it does not likely make sense, so let us now show what it does in practice:
+
+ADD IMAGE
+
+Here, we see that the edges in the image are highlighted.
+This is why the Sobel operators are also known as naive edge detectors and are integral components to many more sophisticated edge detection methods like one proposed by Canny [CITE].
+
+For now, it is a good idea to show what this operation might look like in code:
+
+ADD CODE
+
+With that, I believe we are ate a good place to stop discussions on two-dimensional convolutions.
+We will definitely return to this topic in the future as new algorithms require more information.
 
 <script>
 MathJax.Hub.Queue(["Typeset",MathJax.Hub]);

contents/convolutions/2d/2d.md

@@ -1,38 +1,38 @@
 # Convolutional Theorem
 
-Important note: this particular section will be expanded upon after the Fourier transform and fast Fourier transform chapters have been revised.
+Important note: this particular section will be expanded upon after the Fourier transform and fast Fourier transform (FFT) chapters have been revised.
 
 Now, let me tell you about a bit of computational magic:
 
 **Convolutions can be performed with Fourier Transforms!**
 
-That is crazy, but it's also incredibly hard to explain, so let me do my best.
+This is crazy, but it is also incredibly hard to explain, so let me do my best.
 As described in the chapter on [Fourier Transforms](../cooley_tukey/cooley_tukey.md), Fourier Transforms allow programmers to move from real space to frequency space.
-When we transform a wave into frequency space, we see a single peak in frequency space related to the frequency of that wave.
+When we transform a wave into frequency space, we can see a single peak in frequency space related to the frequency of that wave.
 No matter what function we send into a Fourier Transform, the frequency-space image can be interpreted as a series of different waves with a specified frequency.
+Each of these waves is parameterized by another $$e^{2\pi i k n / N}$$ term, where $$k$$ is the element's value in the frequency domain, $$n$$ is its value in the time domain, and $$N$$ is the overall length of the signal.
+In this way, each wave can be seen as a complex exponential.
 
 So here's the idea: if we take two functions $$f(x)$$ and $$g(x)$$ and move them to frequency space to be $$\hat f(\xi)$$ and $$\hat g(\xi)$$, we can then multiply those two functions and transform them back into a third function to blend the signals together.
 In this way, we will have a third function that relates the frequency-space images of the two input functions.
-This is precisely a convolution.
-
-This is because of something known as the *convolution theorem* which looks something like this:
+This is known as the *convolution theorem* which looks something like this:
 
 $$\mathcal{F}(f*g) = \mathcal{F}(f) \cdot \mathcal{F}(g)$$
 
 Where $$\mathcal{F}$$ denotes the Fourier Transform.
-Now, by using a Fast Fourier Transform (FFT) in code, this can take a standard convolution on two arrays of length $$n$$, which is an $$\mathcal{O}(n^2)$$ process, to $$\mathcal{O}(n\log(n))$$.
-This means that the convolution theorem is fundamental to creating fast convolutional methods for large inputs, assuming that both of the input signals are similar sizes.
-That said, it is debatable whether the convolution theorem will be faster when the filter size is small.
-Also: depending on the language used, we might need to read in a separate library for FFT's.
+
+At first, this might not seem particularly intuitive, but remember that frequency space is essentially a set of exponentials.
+As mentioned in the section about [convolutions in one dimension](../1d/1d.md), multiplication in base 10 space is also a convolution, therefore the convolutional theorem extends this concept into multiplication with *any* set of exponentials, not just base 10.
+Obviously, this description is still lacking a bit of explanation, but I promise we will add more when revising the Fourier transform sections!
+
+By using a Fast Fourier Transform (FFT) in code, this can take a standard convolution on two arrays of length $$n$$, which is an $$\mathcal{O}(n^2)$$ process, to $$\mathcal{O}(n\log(n))$$.
+This means that the convolution theorem is fundamental to creating fast convolutional methods for certain large inputs.
 
 {% method %}
 {% sample lang="jl" %}
-That said, Julia has an in-built fft routine, so the code for this method could not be simpler:
 [import:19-22, lang:"julia"](../code/julia/conv.jl)
-Where the `.*` operator is an element-wise multiplication.
 {% sample lang="hs" %}
 [import:11-14, lang:"haskell"](../code/haskell/convolution.hs)
-Where the `.*` operator is an element-wise multiplication.
 {% sample lang="c"%}
 [import:20-30, lang:"c"](../code/c/convolutions.c)
 {% sample lang="cpp"%}
@@ -42,9 +42,7 @@ Where the `.*` operator is an element-wise multiplication.
 {% endmethod %}
 
 This method also has the added advantage that it will *always output an array of the size of your signal*; however, if your signals are not of equal size, we need to pad the smaller signal with zeros.
-Also note that the Fourier Transform is a periodic or cyclical operation, so there are no real edges in this method, instead the arrays "wrap around" to the other side.
-For this reason, this convolution is often called a *cyclic convolution* instead of a *linear convolution* like above.
-Note that cyclic convolutions can definitely still be done without Fourier Transforms and we can do linear convolutions with Fourier Transforms, but it makes the code slightly more complicated than described above.
+Also note that the Fourier Transform is a periodic or cyclical operation, so there are no real edges in this method, instead the arrays "wrap around" to the other side, creating a cyclic convolution like we showed in the periodic bounday condition case for the [one-dimensional convolution](../1d/1d.md).
 
 <script>
 MathJax.Hub.Queue(["Typeset",MathJax.Hub]);

contents/convolutions/convolutional_theorem/convolutional_theorem.md

@@ -1,24 +1,29 @@
 # Convolutions
 To put it bluntly, convolutions can be confusing.
-Not only are they hard to describe, but if they are not used in practice, it's hard to understand why they woudl ever be needed.
+Some might even call them *convoluted*!
+(Get it? Because we are talking about *convolutions*? A wise man once told me that all good jokes need additional clarification.)
+
+Not only are convolutions hard to describe, but if they are not used in practice, it is hard to understand why they would ever be needed.
 I'm going to do what I can to describe them in an intuitive way; however, I may need to come back to this in the future.
-Let me know if there is anything here that is unclear, and I'll do what I can to clear it up.
+Let me know if there is anything here that is unclear, and I will do what I can to clear it up.
 
 As always, we should start at the start.
-If you take two functions $$f(x)$$ and $$g(x)$$, there are a number of ways you can combine them.
+If you take two functions $$f$$ and $$g$$, there are a number of ways you can combine them.
 All basic operations can do this (addition, subtraction, multiplication, and division), but there are also special operations that only work with functions and do not work on standard variables or numbers.
-For example, $$f \circ g$$ is a *composition* of the two functions, where you plug $$g(x)$$ into $$f(x)$$.
+For example, $$f \circ g$$ is a *composition* of the two functions, where you plug $$g(x)$$ into $$f$$.
 A convolution is another function-related operation, and is often notated with a star ($$*$$) operator, where
 
-$$f(x)*g(x)=c(x)$$
+$$
+f*g=c
+$$
 
-provides a third function $$c(x)$$ that is a blended version of $$f(x)$$ and $$g(x)$$.
-This concept is known as a *convolution*, and as a rather important side-note: there is an incredibly similar operator known as a *correlation* which will be discussed in the near future.
-Now we are left with a rather vague question: How do we *blend* functions?
+provides a third function $$c$$ that is a blended version of $$f$$ and $$g$$.
+This concept is known as a *convolution*, and as a rather important side-note: there is an incredibly similar operation known as a *correlation* which will be discussed in the near future.
+Now we are left with a rather vague question: how do we *blend* functions?
 
-To answer this question, we will need to show off a few simple graphics or animations in the [Convolutions in 1D](1d/1d.md) section before discussing the mathematical definition.
-We will then move on to the typical application of convolutions to images in the [Convolutions of images](2d/2d.md) section, where we will discuss two important filters: the Gaussian kernel and the Sobel operator.
-As a note: convolutions can be extended to $n$-dimensions, but after seeing how it is extended to two dimensions, it should be possible for the reader to extend it to three dimensions and beyond if that is needed, so we will not be covering that in great detail here unless is is useful for another algorithm.
+To answer this question, we will need to show off a few simple graphics or animations in the [Convolutions in 1D](1d/1d.md) section while also discussing the mathematical definition.
+We will then move on to the most stereotypical application of convolutions to images in the [Convolutions of Images](2d/2d.md) section, where we will discuss two important filters: the Gaussian kernel and the Sobel operator.
+As a note: convolutions can be extended to $$n$$-dimensions, but after seeing how they are extended to two dimensions, it should be possible for the reader to extend it to three dimensions and beyond if that is needed, so we will not cover that in great detail here unless is is useful for another algorithm.
 In addition, we will be touching on a rather difficult but powerful topic with the [Convolutional Theorem](convolutional_theorem/convolutional_therem.md) section where convolutions can be computed by using [Fourier transforms](../Cooley_tukey/cooley_tukey.md).
 
 <script>