This is super cool! I would love to see the specific mapping of keys that were used which were said to be inspired by the accordion. Even though that's a way less interesting detail than the way that the spoon picks up the distances and the "bit banging" used to achieve 8-bit precision on the modulation from two 4-bit connections.
Sounds pretty swell!
I wonder if the spoon controller could be adapted to send modulation parameters to arbitrary instruments via a midi port. I would buy a spoon modulator if it was reasonably priced. It would be a great add-on to a piano style keyboard without pitch bend or mod wheel etc
Safe to say it's the same as qwertuoso, muscle memory isn't that easy to remap, even if you're Linus Åkesson. I forget if it's a type B or type C, he mentions it on his site somewhere.
Tried my personal benchmark on the gpt-oss:20b: What is the second mode of Phyrgian Dominant?
My first impression is that this model thinks for a _long_ time. It proposes ideas and then says, "no wait, it's actually..." and then starts the same process again. It will go in loops examining different ideas as it struggles to understand the basic process for calculating notes. It seems to struggle with the septatonic note -> Set notation (semitone positions), as many humans do. As I write this it's been going at about 3tok/s for about 25 minutes. If it finishes while I type this up I will post the final answer.
I did glance at its thinking output just now and I noticed this excerpt where it finally got really close to the answer, giving the right name (despite using the wrong numbers in the set notation, which should be: 0,3,4,6,7,9,10:
Check "Lydian #2": 0,2,3,5,7,9,10. Not ours.
The correct answers as given by my music theory tool [0], which uses traditional algorithms, in terms of names would be: Mela Kosalam, Lydian ♯2, Raga Kuksumakaram/Kusumakaram, Bycrian.
Its notes are: 1 ♯2 3 ♯4 5 6 7
I find looking up lesser known changes and asking for a mode is a good experiment. First I can see if an LLM has developed a way to reason about numbers geometrically as is the case with music.
And by posting about it, I can test how fast AIs might memorize the answer from a random comment on the internet, as I can just use a different change if I find that this post was eventually regurgitated.
After letting ollama run for a while, I'm post what it was thinking about in case anybody's interested. [1]
Also copilot.microsoft.com's wrong answer: [2], and chatgpt.com [3]
I do think that there may be an issue where I did it wrong because after trying the new ollama gui I noticed it's using a context length of 4k tokens, which it might be blowing way past. Another test might be to try the question with a higher context length, but at the same time, it seems like if this question can't be figured out in less time than that, that it will never have enough time...
And, I agree an interval is essentially a distance. Distance between three points makes no sense as they might very well lay outside of one straight line. Even they are on the same line.. are we measuring the distance between each distance?
It's ambiguous what that might even mean, but the original poster might think of a collection of intervals which is 0 or more notes with intervals relative to a given root.
For example if you think in integers (pitch set notation):
That particular algorithm doesn't care whether the instruments are guitar or otherwise. There are other algorithms in vamp that would deal with individual notes. But in terms of separating tracks, vamp doesn't do that. There are some new ML-based solutions for this though. So you could separate them and run vamp on those outputs.
But to get the chords I don't think you need to worry about that.
To add to this, it’s vanishingly seldom that you would have a finished track where multiple parts of the ensemble are playing different chords, rather than each part making up the same overall chord structure.
What a cool thread! I like how you put the specifics of your workflow and especially details of the commands you used! Particularly with the vamp commands, because as you say, they are somewhat inscrutably named/documented.
I started dabbling with vamp as well a couple years ago, but lost track of the project as my goals started ballooning. Although the code is still sitting (somewhere), waiting to be resuscitated.
I have had an idea for many years of the utility of having chord analysis further built out such that a functional chart can be made from it. With vamp most of/all the ingredients are there. I think that's probably what chordify.com does, but they clearly haven't solved segmentation or time to musical time, as their charts are terrible. I don't think they are using chordino, and whatever they do use is actually worse.
I got as far as creating a python script which would convert audio files in a directory into different midi files, to start to collect the necessary data to construct a chart.
For your use case, you'd probably just need to quantize the chords to the nearest beat, so you could maybe use:
vamp-aubio_aubiotempo_beats, or
vamp-plugins_qm-barbeattracker_bars
and then combine those values with the actual time values that you are getting from chordino.
I'd love to talk about this more, as this is a seemingly niche area. I've only heard about this rarely if at all, so I was happy to read this!
> I think that's probably what chordify.com does [...] I don't think they are using chordino
I think they were initially using the Chordino chroma features (NNLS-Chroma) but a different chord language model "front end". Their page at https://chordify.net/pages/technology-algorithm-explained/ seems to imply they've since switched to a deep learning model (not surprisingly)
I really appreciate the opinion that using args and kwargs is "bad".. It always annoys me when you get them as your parameters.. even worse when you go to declaration and the declaration also contains unlabeled parameters. A lot of wrapper libraries seem to do this. I try to just name every parameter I can. It's so much easier to use this code now that we are using autocomplete in the IDE.
I really like the link you provided and have watched it before!
But - I do want to say that C == green is not arbitrary at all. It is consistent with my calculations, which are consistent with Newton's calculations. Usually I see the colours being assigned to notes as wrong.. but this C == green is consistent with mapping light, using octave equivalence, given by the following:
f_prime = f * 2 ** (i / 12)
# Where,
# f' is the derived f, (in this case Green {5.66 × 10^14 Hz})
# f is the reference f (in this case 261.63 Hz)
# i is the interval in semitones
Let's say that C == 261.63 Hz, and that Green == 5.66 × 10^14 Hz. Using the preceding formula we can make a small (python) program to check whether C == Green.
light_range_min = 400 * 10 ** 9 # Hz
light_range_max = 790 * 10 ** 9 # Hz
C = 261.63 # Hz
octave = 0
for octave in range(100): # we are just using a high number here
f_prime = C * 2 ** (12 * octave / 12)
if f_prime >= light_range_min:
octave = octave
break
print(f"C in the range of light has f == {f_prime}, which is {f_prime / 10 ** 9} THz. We had to go {octave} octaves up to arrive there")
# outputs: C in the range of light has f == 561846146826.24, which is 561.84614682624 THz. We had to go 31 octaves up to arrive there
We can look up colour charts like [0] or [1] and find that this frequency is in fact associated with the colour green.
Exactly! It's all made visual and interactive here https://chromatone.center/theory/interplay/spectrum/ - you can even slide the tuning of A4 by dragging left-to-right the last graph - it will show the distribution on 44th octave notes along the visual spectrum
It could be related but I also want to weigh in here and say this. Hypoglycemia can occur with no relation to the other side of diabetic symptoms, i.e, hyperglycemia. In other words there are people who suffer from hypoglycemia without ever getting high blood sugar, and so they are not "diabetic" which would mean you can have issues from both directions.
As a piano/keyboard player, a lot of musicality is possible on a keyboard. It is possible to learn to modify the technique to better utilise the velocity available to a particular keybed, weighted, or non-weighted. When playing keyboards you are working within a subset of the potential dynamics available to a piano. Though expressivity is lessened, there is still a huge palette once you learn to use less total force and less differentiation in force (dynamics).
I know I can play with high musicality on almost any keyboards with velocity, because I was blessed to have learned to use bad instruments. But, it doesn't compare to the depth of the sound generated by all the moving parts and interactions happening in a real piano. Not only the sounds, but also the sheer weight of the keys.
Most* keyboards/vsts are just triggering a (pitch-shifted, looped) sample at a given note and then doing that for n notes and that doing an additive sum.
That is definitely not what occurs in a piano though. There you have the 3-dimensionality of the physical world, like the way waves are traveling through distance and shape. When sounds' harmonics interact, resonant nodes in overtone sequences can trigger each other to resonance, which can trigger other resonances throughout the tone. Maybe you know the feeling of depressing the sustain/damper pedal while sitting in front of it and giving the instrument a smack (or holding down the keys you are not playing and doing it). Or running your nail or a pick over all the low notes with sustain.. like you are in a cave.
In midi/digital, there's the fact that dynamic is usually gonna be 8-bit. Just because midi did that and it made sense at the time, other keyboards and VSTs mostly follow suit. I'm surprised this gets generally passed over. Obviously there's more than 128 strengths of note in real life.
But all that said I think it's possible to learn keyboards/music theory/songs/playing on a non-weighted keyboard, but false to say that digital/non-weighted is equivalent to acoustic piano. But you only really need that for really dynamic music like Jazz, Classical, Instrumental et al. But it feels so very wrong to play that kind of music on bad keyboards.
* Roland V-Piano, and PianoTeq, as well as many I'm unaware of do in fact use physical/acoustic modeling as opposed to triggering samples, but it has not been predominant even among high-end digital instruments
And then you can see that the bitrate is 2-20x better. I personally get bothered by anything under CD quality (44.1kHz 16-bit) so all these platforms are basically unlistenable for me.
Sounds pretty swell!
I wonder if the spoon controller could be adapted to send modulation parameters to arbitrary instruments via a midi port. I would buy a spoon modulator if it was reasonably priced. It would be a great add-on to a piano style keyboard without pitch bend or mod wheel etc
reply