You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: contents/convolutions/1d/1d.md
+36-33Lines changed: 36 additions & 33 deletions
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
# Convolutions in 1D
2
-
As mentioned in the [introductory section for convolutions](../convolutions.md), convolutions are methods that allow mathematicians to "blend" two seemingly unrelated functions; however, this definition is not very rigorouos, so it might be better to think of a convolution as a method to apply a filter to a signal or image.
2
+
As mentioned in the [introductory section for convolutions](../convolutions.md), convolutions are methods that allow mathematicians to "blend" two seemingly unrelated functions; however, this definition is not very rigorous, so it might be better to think of a convolution as a method to apply a filter to a signal or image.
3
3
This, of course, brings up more questions: what is a filter? What is a signal? How is this all related to images?
4
4
5
5
For this, we will start with some predefined signal.
@@ -15,15 +15,15 @@ Both of these are shown below:
15
15
16
16
So now we have a signal and a filter.
17
17
How do we apply the filter to the signal?
18
-
The easiest way to do this would be to iterate through every point in the signal and blend it with neighboring elements, where each neighboring element is weighted based on the filter value.
18
+
The easiest way to do this is to iterate through every point in the signal and blend it with neighboring elements, where each neighboring element is weighted based on the filter value.
19
19
So in the case where the triangle wave is only 3 elements (`[0.5, 1, 0.5]`), the output at each point would be
where $$C$$ is the output value, $$A$$ is the input array (a signal or image), and $$n$$ is an iterable element through that signal.
26
-
In this way, the "application of a filter," is simply a multiplication of the triangle wave centered around each point in the image followed by in integral or sum of the output.
26
+
In this way, the "application of a filter," is simply a multiplication of the triangle wave centered around each point of the input array, followed by in integral or sum of the output.
27
27
In some sense, this means we will shift the filter, then multiply and sum every step.
28
28
This can be seen in the following animation:
29
29
@@ -62,8 +62,7 @@ As an important note, if we were to extend the convolution into continuous space
Note that in this case, $$x$$ and $$\xi$$ are not necessarily spatial elements.
66
-
The interpretation is the same as before.
65
+
Note that in this case, $$x$$ and $$\xi$$ are not necessarily spatial elements, but the interpretation is otherwise the same as before.
67
66
68
67
At this stage, the mathematics and code might still be a little opaque, so it is a good idea to play around a bit and think about how this operation might be used in practice with a few different filters.
69
68
@@ -80,10 +79,10 @@ Your browser does not support the video tag.
80
79
</div>
81
80
82
81
In practice, the convolutional output here is very similar to the triangle wave we showed before.
83
-
The final convolved image lookes a lot like the square, except that its boundaries have been smoothed out or "blurred."
82
+
The final convolved image looks a lot like the square, except that its boundaries have been smoothed out or "blurred."
84
83
In practice whenever a Gaussian filter is used, it will always blur the other convolved signal, which is why a convolution with a Gaussian is also called a *blurring operation*.
85
-
This operation is used very often when dealing with two-dimensional images, and we will discuss common kernels found in the wild in the next section on [convolutions of images](../2d/2d.md).
86
-
Still, it is interesting to see the blurring operation in action by convolving a random distribution with a larger gaussian filter:
84
+
This operation is used very often when dealing with two-dimensional images, and we will discuss common kernels found in the wild in [the next section](../2d/2d.md).
85
+
Still, it is interesting to see the blurring operation in action by convolving a random distribution with a larger Gaussian filter:
87
86
88
87
<divstyle="text-align:center">
89
88
<videostyle="width:90%"controlsloop>
@@ -92,7 +91,7 @@ Your browser does not support the video tag.
92
91
</video>
93
92
</div>
94
93
95
-
In this animation, the final convolution is so blurred that it seems only slightly related to the random input signal.
94
+
In this animation, the final convolution is so blurred that it does not seem related to the random input signal at all!
96
95
In fact, this animation seems to average much more when compared to the previous Gaussian and the triangle wave animations.
97
96
This is because the Gaussian is wider than the previous to filters.
98
97
In general, the wider the filter, the stronger the blurring effect.
@@ -110,8 +109,9 @@ Your browser does not support the video tag.
110
109
111
110
As one might expect, the output is a blurrier Gaussian, which is essentially just wider.
112
111
As an interesting note here, the green area inside this visualization does not properly line up with the overlap of the two arrays.
112
+
Don't worry!
113
113
This is exactly what should happen!
114
-
Remember that the convolution requires a *multiplication* of the signal and filter, which is only the overlap of the two arrays when the signal is a square wave.
114
+
Remember that the convolution requires a *multiplication* of the signal and filter, which was the same as the overlap when the signal was a square wave; however, in the case of two distinct signals, we should expect the multiplied output to look somewhat distinct.
115
115
116
116
Let us extend this concept to one final example of a square wave convolved with a triangular, sawtooth function that looks like this:
117
117
@@ -135,9 +135,8 @@ Similarly, there is a negatively accelerating slope when the sawtooth function l
135
135
136
136
## Dealing with boundaries
137
137
138
-
In all of the animations, we have shown the filter constantly reappearing on the left edge of the screen.
139
-
That said, this is not always the best thing to do at the boundaries.
140
-
In fact, these boundary conditions are somewhat non-trivial to code, so we will start with relatively simple boundary conditions that were actually introduced in the previous code example.
138
+
In all of the animations, we have shown the filter constantly reappearing on the left edge of the screen, which is not always the best thing to do at the boundaries.
139
+
In fact, these boundary conditions are somewhat non-trivial to code, so for this section, we will start with relatively simple boundary conditions that were introduced in the previous code example.
141
140
142
141
### Simple boundaries
143
142
@@ -168,10 +167,10 @@ Your browser does not support the video tag.
168
167
169
168
Similar to the case without boundary conditions, this convolution needs to "ramp up," but not "ramp down."
170
169
This is because the convolution output no longer extends past the bounds of the original signal.
171
-
More than that, the convolution does not go all the way to 0.
170
+
More than that, the convolution does not go all the way to 0 on the right side.
172
171
This means that we are actually ignoring a rather important part of the convolution!
173
172
174
-
This is 100% true; however, if the signal is large and the filter is small (as is the case with most of image processing), we do not really need to care that much about the bits of the convolution we missed.
173
+
This is 100% true; however, if the signal is large and the filter is small (as is the case with most of image processing), we do not really care that much about the bits of the convolution we missed.
175
174
For this reason, simple bounds are used frequently when performing convolutions on an image.
176
175
177
176
In the previous code snippet, we were able to perform both a bounded and unbounded convolution.
@@ -197,13 +196,16 @@ On the other hand, the bounded call would set the output array size to simply be
Now let us talk a bit more in detail about the boundaries in the animations!
199
+
As another note, the point we are computing for the convolution in the previous animation seems to be at the very front of the Gaussian to match the code; however, it is possible to compute the convolution at the center of the filter by giving the iterations through `j` an offset.
200
+
201
+
I think this is a good place to stop discussions on the simple boundary conditions.
202
+
Now let us talk a bit more in detail about the case where we want to filter to continuously reappear every loop.
203
+
That case is known as the "periodic boundary condition."
202
204
203
205
### Periodic boundary conditions
204
206
205
-
Another relatively simple boundary condition is the periodic boundary condition.
206
-
With this condition, the filter will wrap itself around to the other end of the signal whenever it hits a boundary.
207
+
Though periodic boundary conditions are more complicated that those mentioned in the previous section, they are still *relatively* straightforward to implement.
208
+
With these conditions, the filter will wrap itself around to the other end of the signal whenever it hits a boundary.
207
209
In this way, the signal is periodic, with an identical copy of itself acting as left and right neighbors.
208
210
Those neighbors then have other neighbors, and those then have more neighbors, creating a sea of signals extending to infinity and beyond in both directions.
209
211
For us, this means that when the filter leaves one edge of the domain, it simply appears on the other, opposite edge.
@@ -225,8 +227,8 @@ In code, this typically amounts to using some form of modulus operation, as show
As a final note before continuing: dealing with boundaries is tricky business and can dramatically change the behaviour of the output convolution.
229
-
For this reason, it is important to think about what types of boundaries will work best for what you, the programmer, actually needs.
230
+
As a final note before continuing: dealing with boundaries is tricky business and can dramatically change the behavior of the output convolution.
231
+
For this reason, it is important to think about what types of boundaries will work best for what you, the programmer, actually need.
230
232
The selection of boundary conditions will be a common trope for a large portion of computer graphics and physics algorithms where researchers often need to present and simulate data on an array of some sort.
231
233
232
234
## Multiplication as a convolution
@@ -250,49 +252,50 @@ So let us use a similar notation to perform the convolution, by reversing the se
For these operations, any blank space should be considered a $$0$$.
296
299
In the end, we will have a new set of numbers:
297
300
298
301
$$
@@ -325,8 +328,8 @@ This will be discussed in further detail when we talk about the Schonhage-Strass
325
328
326
329
## Example Code
327
330
328
-
For the full code, we have used the convolution to generate a few files for the full convolution and the periodic and simple boundary conditions discussed in this chapter.
329
-
At a test case, we have chosen to use a random distribution for the input signal and a gaussian filter.
331
+
For the full code, we have used the convolution to generate a few files for the full convolution, along with the periodic and simple boundary conditions discussed in this chapter.
332
+
At a test case, we have chosen to use a random distribution for the input signal and a Gaussian filter.
This code is very similar to what we have shown in previous sections; however, it essentially requires 4 iterable dimensions because we need to iterate through each element of the output domain *and* the filter.
27
-
Because the indexing is slightly non-trivial, we felt it was worth writing a separate chapter for two-dimensional convolutions.
26
+
This code is very similar to what we have shown in previous sections; however, it essentially requires 4 iterable dimensions because we need to iterate through each axis of the output domain *and* the filter.
27
+
Because this indexing is slightly non-trivial, we felt it was worth writing a separate chapter for two-dimensional convolutions.
28
28
29
29
It is worth highlighting common filters used for convolutions of images.
30
-
In particular, we will further discuss the Gaussian filter introduced in the section on [one-dimensional convolutions](../1d/1d.md), and then introduce another set of kernels known as Sobel operators, which are used for naive edge detection or image derivatives.
30
+
In particular, we will further discuss the Gaussian filter introduced in the section on [one-dimensional convolutions](../1d/1d.md), and then introduce another set of kernels known as Sobel operators, which are used for naïve edge detection or image derivatives.
where $$\sigma$$ is the standard deviation and is a measure of the width of the Gaussian.
44
-
A larger $$\sigma$$ means a larger Gaussian; however, Remember that the Gaussian must fit onto the filter!
45
-
As a note, some definitions of $$\sigma$$ allow users to have a separate deviation in $$x$$ and $$y$$, but for the purposes of this chapter, we will assume they are the same.
44
+
A larger $$\sigma$$ means a larger Gaussian; however, remember that the Gaussian must fit onto the filter, otherwise it will be cut off!
45
+
Some definitions of $$\sigma$$ allow users to have a separate deviation in $$x$$ and $$y$$, but for the purposes of this chapter, we will assume they are the same.
46
46
As a general rule of thumb, the larger the filter and standard deviation, the more "smeared" the final convolution will be.
47
47
48
48
At this stage, it is important to write some code, so we will generate a simple function to return a Gaussian kernel with a specified standard deviation and filter size.
@@ -52,18 +52,19 @@ At this stage, it is important to write some code, so we will generate a simple
Though it is entirely possible to create a Gaussian kernel whose standard deviation is independent on the kernel size, we have decided to keep them the same for this chapter.
55
+
Though it is entirely possible to create a Gaussian kernel whose standard deviation is independent on the kernel size, we have decided to enforce a relation between the two in this chapter.
56
+
As always, we encourage you to play with the code and create your own Gaussian kernels any way you want!
56
57
As a note, all the kernels will be scaled at the end by the sum of all internal elements.
57
-
This simply ensures that the output of the convolution does not have an obnoxious scale factor associated with it.
58
+
This ensures that the output of the convolution does not have an obnoxious scale factor associated with it.
58
59
59
-
Below are a few kernels generated with the above code along with their application to a black and white heart image.
60
+
Below are a few images generated by applying a kernel generated with the code above to a black and white image of a circle.
In (a), we show the original image, which is just a black circle at the center of a $$50\times 50$$ grid.
67
+
In (a), we show the original image, which is just a white circle at the center of a $$50\times 50$$ grid.
67
68
In (b), we show the image after convolution with a $$3\times 3$$ kernel.
68
69
In (c), we show the image after convolution with a $$20\times 20$$ kernel.
69
70
Here, we see that (c) is significantly fuzzier than (b), which is a direct consequence of the kernel size.
@@ -73,8 +74,8 @@ There is a lot more that we could talk about, but I think this is a good place t
73
74
## The Sobel operator
74
75
75
76
The Sobel operator effectively performs a gradient operation on an image by highlighting areas where a large change has been made.
76
-
In essence, this means that this operation can be considered to be a naïve edge detector.
77
-
That is to say that the $$n$$-dimensional Sobel operator is composed of $$n$$ separate gradient convolutions (one for each dimension) that are then combined together into one, output array.
77
+
In essence, this means that this operation can be thought of as a naïve edge detector.
78
+
That is to say that the $$n$$-dimensional Sobel operator is composed of $$n$$ separate gradient convolutions (one for each dimension) that are then combined together into a final output array.
78
79
Again, for the purposes of this chapter, we will stick to two dimensions, which will be composed of two separate gradients along the $$x$$ and $$y$$ directions.
79
80
Each gradient will be created by convolving our image with their corresponding Sobel operator:
80
81
@@ -127,11 +128,11 @@ So let us now show what it does in practice:
In this diagram, we start with the circle image on the right, and then use the $$S_x$$ and $$S_y$$ operators on them to find the gradients along $$x$$ and $$y$$ before summing them in quadrature to get the final image gradient.
131
-
Here, we see that the edges in the image are highlighted, showing the outline of our circle image.
131
+
In this diagram, we start with the circle image on the right, and then convolve them with the $$S_x$$ and $$S_y$$ operators to find the gradients along $$x$$ and $$y$$ before summing them in quadrature to get the final image gradient.
132
+
Here, we see that the edges of our input image have been highlighted, showing outline of our circle.
132
133
This is why the Sobel operator is also known as naïve edge detection and is an integral component to many more sophisticated edge detection methods like one proposed by Canny {{ "canny1986computational" | cite }}.
133
134
134
-
In code, applying the Sobel operator involves first finding the operator in $$x$$ and $$y$$ and then applying them with a traditional convolution:
135
+
In code, the Sobel operator involves the same operations: first finding the operator in $$x$$ and $$y$$ and then applying them with a traditional convolution:
0 commit comments