This example is great, hope you don't mind if I spell it out how your impulse weight f(d) and impulse response functions g(d) are functions over d (each associated with 10^d):

  f(0) = 1      g(0) = 3      f*g(0) = 3 
  f(1) = 0      g(1) = 2      f*g(1) = 2 
  f(2) = 1                    f*g(2) = 3 
  f(3) = 0                    f*g(3) = 2
  f(4) = 0                    f*g(4) = 0
  f(5) = 1                    f*g(5) = 3 
                              f*g(6) = 2

You can see how when there is an impulse weight "x" in digit space entry d, the impulse response "3x" comes at d, and the impulse response "2x" comes at d+1. Note how f and g can be anything and it convolution (f * g) is still their multiplication (except for the obvious rub when f*g(d) is greater than 9, so you need to add a rule about that).

Woha

This is cool. It literally is the same, if you use base-infinity digits (not binary, or decimal, but infinity-ary?).

How do you multiply two polynomials with a single variable?

A polynomial with a single variable is more or less what you would mean by "use base-infinity digits" in the first place.

Roughly, in base 10, the number represented by the sequence of digits d_0, d_1, ..., d_n is d_0 + d_1 * 10 + d_2 * 10^2 + ... + d_n * 10^n.

In "base infinity", the "number" represented by the sequence of digits d_0, d_1, ..., d_n is the polynomial d_0 + d_1 * x + d_2 * x^2 + ... + d_n * x^n.

Multiplying these "numbers" (polynomials) is like decimal multiplication, except the digits can be arbitrarily high and you do no carrying at all.

> This is cool

I don't agree the example has anything to do with convolution. The example is explained with plain old primary school multiplication.

100101 * 23 = (100000+100+1)*23

         = 2300000+2300+23

There isn't any fantastic property, only cherry-picked number which works as decimal left-shifts.

Convolution is not a number. Convolution is an operator that outputs a function.

(sum_i ai x^i)(sum_j bj x^j) when you expand that, the coefficient of x^k is the sum of aibj such that i+j=k which is discrete convolution.

Set x=10 and you're dealing with decimal representation of numbers.

> Set x=10 and (...)

...and you no longer have a function, only the image of a function at a specific point of its domain.

Convolutions are functions, not functions evaluated at a specific, and cherry-picked point.

The parents point is that when you do the primary school multiplication algorithm, you are actually performing a discrete convolution.

You really aren't. That assertion is like commenting that elementary arithmetics is discrete signal analysis because 1+1=2, which happens to match the amplitude of a resonance of a normalized vibration mode.

It isn't. It's a cherry picked example. A broken clock which coincides with the current time.

> It isn't. It's a cherry picked example. A broken clock which coincides with the current time.

No. It really is. Look at the example.

Nobody can convince you if you are already set up against that idea, but look at what happens when you multiply two polyomials P(z)Q(z). You obtain a polynomial whose coefficients are the convolution of those of P and Q. Now, a decimal number is "just" a polynomial evaluated on z=10. And a vibrating string is "just" a polynomial evaluated at z=exp(it). The algebra of convolutions and products is exactly the same in all three cases.

The only cherry picking is that the numbers need to be small enough that the decimal digit does not overflow (4 times 2 is fine but 4 times 3 is not).

A more 'convolved' example:

121 * 23 = 2783

If you use hex instead it also works for larger digits like 0x42 * 0x31 = 0xCA2 = [12, 10, 2]

So, if you convert to a digit of a large enough base you can do any multiplication with just a convolution and no carry.

> The only cherry picking is that the numbers need to be small enough that the decimal digit does not overflow

Aren't you restating my point?

> So, if you convert to a digit of a large enough base you can do any multiplication with just a convolution and no carry.

There is no convolution at all. There's only a cherry-picked example of how plain old multiplication feels similar to a sliding dot product, which for some reason some people confuse with convolution. This is not helpful, and only adds to the confusion already expressed.

You’re right that the analogy between elementary-school-multiplication and convolution only works for multiplication without carrying. That still leaves a large amount of possible multiplication instances, for the term “cherry picking” to apply here.

You’re wrong to suggest that elementary-school-multiplication and (discrete) convolution are so dissimilar that the analogy is so unhelpful that it only adds to everyone’s confusion. Maybe to yours, but not to everyone’s.

If I understand your objection correctly (other than the already conceded fact that the analogy does not work in case of multiplication with carrying), you already have internalized that convolution is an operator on 2 functions, resulting in a function. And since you have already internalized that, the analogy with any operator on two natural numbers resulting in a natural number confuses you — although I assume you can see the similarity in the sliding operation.

For people who have not internalized what a convolution is, the visualisationg of the sliding operation, and the analogy with elementary-school-multiplication is the helpful bit.

To clarify why the operations are really similar: consider a natural number as a function that takes a natural number as argument, and returns the digit (between 0 and 9 inclusive) for that decimal position.

So f = 2 031 is considered to be: n → f(n) = if n = 3 then return 2 else if n = 1 then return 3 else if n = 0 then return 1 else return 0 (end if).

And g = 320 is considered to be: n → g(n) = if n = 2 then return 3 else if n = 1 then return 2 else return 0 (end if).

What is then the discrete convolution f ∗ g?

It’s f ∗ g = n → (f ∗ g)(n) = f(3) × g(n – 3) + f(1) × g(n – 1) + f(0) × g(n)

= n → if n = 5 then return 6 else if n = 4 then return 4 else if n = 3 or n = 2 then return 9 else n = 1 then return 2 else return 0 (end if).

This is how we consider 649 920 = 2 031 × 320.

> You’re wrong to suggest that elementary-school-multiplication and (discrete) convolution are so dissimilar (...)

They are way more than dissimilar, they are radically different concepts altogether.

Convolution is a function, or an operator that outputs a function given two input functions that share the same domain.

Plain old algebra over real numbers is nothing of the sort.

You're trying to compare a function, with its domain and codomain, with an operation between scalars that results in a scalar. It's way more than apples to oranges. It's apples to orange tree plantations.

The standard multiplication algorithm is an algorithm for taking base-b representations of two numbers and outputting the base-b representation of the product of those numbers. A base-b representation of a number is a sequence of digits. A sequence is a function.

This is not a large conceptual chasm. It is boilerplate for actually talking about decimal (or binary or hex or whatever) representations of numbers. Here is one version of that boilerplate, spelled out:

Think of a base-10 representation of a natural number as a function with domain N (the set of natural numbers) and codomain {0, ..., 9}, where f(0) is the ones digit, f(1) is the tens digit, f(2) is the hundreds digit and so on. (This function will be finitely supported, i.e. all but finitely many inputs to this function will give output 0.)

If f and g are the representations of two numbers n and m, then one can say the following about the representation of their product n * m:

  (1) Extend the codomain of f and g, to N, i.e. think of them as functions N -> N instead of functions N -> {0, ..., 9}.
  (2) Compute the convolution of those two functions, giving you another (still finitely supported) function h: N -> N. Usually at this point h will have values that are larger than 10.
  (3) Do all the carrying (e.g. repeatedly take the first n for which h(n) > 10, and subtract 10 from h(n) and add 1 to h(n+1)). Now all the values of h are in {0, ..., 9}, so you can think of h as a function N -> {0, ..., 9}.

The resulting function h is the base-10 representation of the product of the two numbers that f and g represent.

The first few Google results for "sliding dot product" basically describe it as a convolution with the domain of one of the functions reversed. (Note as far as whether or not to reverse one of the functions, multiplication in base-b agrees with "convolution" and not with "sliding dot product".)

Elsewhere in the thread you seem to be suggesting that convolution is a way to convert between a frequency domain and a time domain, which suggests that you are confusing convolution with Fourier transforms. There's some nice relationships between these things, so they are often discussed together, but even then a single convolution happens either entirely in the time domain or entirely in the frequency domain, not as a way of going from one of those domains to the other. E.g. the pointwise product (in the frequency domain) of the Fourier transforms of two functions is the Fourier transform of the convolution (in the time domain) of those two functions.

There's also a chance you are focused on the difference between functions with a discrete domain and functions with a continuous domain. The term "convolution" is often applied to both.

Ref reply below - https://news.ycombinator.com/item?id=25195046

> Convolution is mainly and primarily used as a convenient way to covert signals between time and frequency domain

Btw that isn't true. That operation is called a "transform" as in "Fourier transform" and "z-transform". The property of those transforms is that element-wise multiplication in one domain becomes convolution in the transformed domain. Element-wise multiplication in "time" domain is convolution in "frequency" domain and element-wise multiplication in "frequency" domain is convolution in "time" domain.

Ref comment https://news.ycombinator.com/item?id=25195046 about this.

Further to that, if you replace x with complex variable z^-1, you get what is usually called the z-transform. Set z to a specific complex root of unity exp(i2pi/N) and you have your discrete FT.

One of the algorithms for multiplying large arbitrary precision numbers uses multiplication of the discrete transform of the digit sequences (in a different base) iirc.

Same thing.

Correction: exp(i2πm/N) .. which gives you Fourier coefficients at m.