Rule-based Improvisation Instruction systems?

[bako]

Christian,

Thanks for your very thorough response.

But I think you - and Reg - have it backwards actually. Typical guitar players! ;-)

Straight or backwards for me is not so much the question. More like this:

If I pretend that a viewpoint is valid, what actions are likely to result.
If a chromatic neighbor note is simply a melodic movement, the short story ends there.
If it can be understood as part of an approach chord, then there is a family of possible movements to be made,
or if better taste prevails, refrain from making in excess.

On thinking harmonically or melodically:

One could easily inflict either style of analysis/understanding on pretty much anything.

A harmonic perspective can open many interesting doors but each one leads to a natural conclusion,
which in the hands of a master improviser is still plenty of source material to work from.

A melodic perspective is in many ways freer. I have always enjoyed groups without a chordal element,
interwoven melody style.

The idea of chord tones on strong beats seems to me to be a linear approach of stating chord movements
using single note melodies. In essence, harmonic melody.

[christianm77]

Christian,

Thanks for your very thorough response.



Straight or backwards for me is not so much the question. More like this:

If I pretend that a viewpoint is valid, what actions are likely to result.
If a chromatic neighbor note is simply a melodic movement, the short story ends there.
If it can be understood as part of an approach chord, then there is a family of possible movements to be made,
or if better taste prevails, refrain from making in excess.

On thinking harmonically or melodically:

One could easily inflict either style of analysis/understanding on pretty much anything.

A harmonic perspective can open many interesting doors but each one leads to a natural conclusion,
which in the hands of a master improviser is still plenty of source material to work from.

A melodic perspective is in many ways freer. I have always enjoyed groups without a chordal element,
interwoven melody style.

The idea of chord tones on strong beats seems to me to be a linear approach of stating chord movements
using single note melodies. In essence, harmonic melody.

'There are no chords, only movement' - Barry Harris, after Coleman Hawkins

BTW IMO one of the clunkiest things about standard jazz education is studying things from the root of chords on a lead sheet. It makes the movement a lot harder to understand.... There's not really any need to analyse the harmony in a line in this way. You just have to play a good melodic line that has movement and resolution in it.

[benjaminjoe]

I looked very carefully but nobody mentioned it. HAL galper's forward motion has quite a few rules of thumb regarding musical rhythm. A sort of system if you like

Inviato dal mio GT-I9060I utilizzando Tapatalk

[zirenius]

I looked very carefully but nobody mentioned it. HAL galper's forward motion has quite a few rules of thumb regarding musical rhythm. A sort of system if you like

Inviato dal mio GT-I9060I utilizzando Tapatalk

You are right - forgot about it, even though I actually own the book. Thank you!!

[benjaminjoe]

You are right - forgot about it, even though I actually own the book. Thank you!!

Have you had a look at Bob Keller program? It's astounding! If you set it to improvise using neural network instead of grammar and run it through something like garritan jazz band it sounds like a real not so talented but realistic human improviser, it actually fooled my girlfriend.

Inviato dal mio GT-I9060I utilizzando Tapatalk

[omphalopsychos]

Wow how did I miss this thread. I have a few things to say.

1) If you want to propose a Chomskian approach to machine-generate music, your understanding is decades behind the current technology.

2) This has already been done, but we don't tell professional musicians because they tend to get upset.

I've programmed rule based etudes for myself, but it turns out that programming rule-based music is kind of boring.

Frankly, it's no longer a difficult problem in computer science. It's so trivial that some researchers have decided to host a website that generates an endless playlist of computer-composed music. Here: The Endless Traditional Music Session

Here are the caveats: with something like traditional irish music, the order of complexity is lower than with jazz. The form is less varied, the harmonic progression is less varied, and the choice of melodic material given the harmonic context is less varied. In theory, this could be applied to jazz, but the only issue is the quantity of transcribed and digitized data. If someone gives me a proper database of digital jazz transcriptions and wants to pay me a research stipend, I'd be happy to show you what current techniques can produce. You'd be surprised.

What did I take away from working on this? More than becoming fascinated with the algorithmic challenges (they're quite straightforward btw), I came to think more about my own playing and the question of creativity. Can you get a computer to generate music that sounds realistic and like *already existing music*? Yeah, no problem. But that same system will have trouble creating new music.

If you have any interest, I gave a very boring and dry lecture on this back in March of this year.

[omphalopsychos]

Oh and for the record. Grammatical/syntactical approaches are out. Markov chains are out. The current fad is Recurrent Neural Networks because they're actually effective. The only issue is that they're useless from a theoretical/musicological perspective. You can't really peer into the representation directly, so it doesn't help with conceptual knowledge. However, they can definitely write better pieces of music than most undergrads.

[christianm77]

HAL galper's forward motion....

Damn I'm slow. Open the pod bay doors already.

[docbop]

So weird this thread and last night a had a long conversation with an old friend and computer business partner. She has a degree in Classical Piano Performance and never stopped play even though her career path ended up being in marketing and designing training programs. A few months ago she decided to she wanted to learn to play Jazz. She got a fake book and signed up for a couple classes. Her first classes are almost over and first time we've spoken since she started.

So she said classes are what she was hoping for they are teaching enough Rules. Lucky we were on the phone so she could see my head drop. She things she said sounded like typical improv 101 topic, but she wanted more Rules on when and why to do the things they were talking about. I tell he she should focus on learning tunes, she says everyone keeps telling me that, I want rules not tunes. It sounded like her teacher was covering the typical stuff, but not a enough (all together now) Rules. This woman is one of the smartest people I've ever met we have worked on many projects together and consulted on big projects and has done very well in the business world in creative areas of Marketing and software design. So I found it odd the she probably like many can't just let go and play, be given a concept and just experiment with it. I just left her with the lesson Peter Bernstein and other tell people work on a tune, play it so many times you're bored with it and start trying some of those concepts to make it more interesting.

So how many here need Rules in order to work on improv, that the Rules need to come first before just playing?

[omphalopsychos]

So how many here need Rules in order to work on improv, that the Rules need to come first before just playing?

I totally agree. Using a rule based approach is suboptimal from an algorithmic perspective from a musical perspective. It creates a load of programming overhead and results in enormous loss of important data (this is where the soul dies). It also imposes aesthetic constraints on the music.

If you're into the sound of rules, then go ahead and program music that sounds like rules. By all means.

[docbop]

I totally agree. Using a rule based approach is suboptimal from an algorithmic perspective from a musical perspective. It creates a load of programming overhead and results in enormous loss of important data (this is where the soul dies). It also imposes aesthetic constraints on the music.

If you're into the sound of rules, then go ahead and program music that sounds like rules. By all means.

What gets me is some let it inhibit truly letting go and finding their own voice.

[CamillusUSA]

This is a similar question as posed by those considering the Schillinger system of automatic composition in the mid-20th century. Systems are not bad for generating "material" but beyond that (even considering how advanced AI has become) I don't have too much trust in it.

Schillinger System - Wikipedia

[zirenius]

Oh and for the record. Grammatical/syntactical approaches are out. Markov chains are out. The current fad is Recurrent Neural Networks because they're actually effective. The only issue is that they're useless from a theoretical/musicological perspective. You can't really peer into the representation directly, so it doesn't help with conceptual knowledge. However, they can definitely write better pieces of music than most undergrads.

Thank you very much for contributing. I am replying to this post of yours to keep the quote short. I had a rather busy work weekend on the practical side of dealing with music, but I hope to get some time in listening to your lecture tomorrow and also checking out the aforementioned system.

All I can say now is that I appreciate your standpoint on this matter and that I would actually ask your permission to contact you again about your point on this after listening to your lecture. The things you are saying "for the record" about the state of the (academic) art here are highly interesting and I was not fully aware of them. (which is likely to my lack of broad overview on the subject matter aka. my ignorance ) However, I will need to brush up on the state of my reading in this field a bit, so I don't ask stupid questions. Can I get back to you later this month?

[omphalopsychos]

Thank you very much for contributing. I am replying to this post of yours to keep the quote short. I had a rather busy work weekend on the practical side of dealing with music, but I hope to get some time in listening to your lecture tomorrow and also checking out the aforementioned system.

All I can say now is that I appreciate your standpoint on this matter and that I would actually ask your permission to contact you again about your point on this after listening to your lecture. The things you are saying "for the record" about the state of the (academic) art here are highly interesting and I was not fully aware of them. (which is likely to my lack of broad overview on the subject matter aka. my ignorance ) However, I will need to brush up on the state of my reading in this field a bit, so I don't ask stupid questions. Can I get back to you later this month?

Please do. Feel free to PM me and I'll try to respond. While I have a moment, here's a quick bibliography/videography/blog-graphy to check out.

David Cope

He's a processor Emeritus at UC Santa Cruz. He did a lot of great work using systems similar to what you approached. He's written programs to generate music in the style of different composers, and I think they're remarkably impressive. My only criticism is that the approach isn't immediately generalizable. For each new "style", it requires a deep knowledge of theory and the compositional characteristics the program is intended to emulate. But have a listen and let your jaw drop.

This, for example, was composed by his program. The performance, of course, is by human beings.

Here's a lecture of his worth checking out.

I recommend checking out his books.

Neural Networks

I really think this where the possibilities are today. The advantage of this approach is that you really don't have to explicitly program anything. I guess fitting markov chains is similar, but I don't think markov chains are able to represent flexible patterns and musical form very well.

If you don't know anything about Neural Networks, I strongly recommend this lecture series.

The types of networks that are particularly effective for representing music data are called Recurrent Neural Networks. They basically function like regular networks but also processing the time dimension of observations.

I recommend this as the best online explanation of Recurrent Networks Understanding LSTM Networks -- colah's blog

This paper was written by some grad students at Stanford a couple of years ago. They applied neural networks to boring generic pop music, but their results were pretty interesting too.

https://cs224d.stanford.edu/reports/NayebiAran.pdf

Here's a clip.

And since you were looking for source code, here is theirs: https://github.com/MattVitelli/GRUV. Their repo is not actively maintained, so you may find some bugs. I started with theirs but eventually wrote my own from scratch. Unfortunately, my workspace was saved on an AWS server that ultimately was decommissioned, so I lost all my work.

Regardless the process is simple:

1) Load an mp3 file as an array of data points (waveform).
2) Apply the Fourier Transform to extract spectral component evolution.
3) Create an LSTM network (using Keras). Pass data from step 2 through the network.
4) Use the trained model to generate new samples. Voila.

The majority of the work is in tuning the network.

I explicitly wanted to work with the waveform because I wanted to be able to represent timbre and texture in addition to just harmony and rhythm. However, you can apply the same methods listed above to something like MIDI or XML, but the data processing is a bit different. One caveat about working on this. If you are working with raw audio data rather than MIDI/XML, there's no way you will be able to train the model on your own personal computer. You will need to use (rent) server usage from AWS or Google Cloud or some other cloud computing provider. It's not too expensive, but you may sink a few hundred bucks into R&D. Try to get some funding if you go down that route.

Oh, and I almost forgot. Read John Glover's blog. http://www.johnglover.net/blog/gener...with-rnns.html

[Robertkoa]

Paint by numbers...

Might work for some.

Problem is usually the really great Players or merely excellent ones in the *Gènre don't usually use the same ' Rules' especially when it comes to note choices.

* Because the Guy writing the 'Rules' - that's HIS interpretation which may get you ' in the Ballpark' but not actually cover it or enable you to be creative within it .

So my advice is :

'Use the "Rules" to get you in the Ballpark , then screw the Rules .'

Obviously - the more novice one is in any field the more imitation or preparation one must do .

So the Computer could beat me in Chess but I can program much better Techno in 5 minutes with my preferred Hardware Workstation ?
As could most people who can do Midi Programming
on a Pro Level ..

Or am I missing something?
The Bach Piece sounded good but I ' d find two part inventions more revealing I think... but the Techno does not sound that good and should be much easier ..

Why the disparity ?

[zirenius]

Sorry, I have been a little quiet about this topic here despite all the great input so far.

The thesis is now done and handed in and received a not so slight shift in focus. I ended up being a little more careful and focussed on the analytic potential of computer based modelling regarding rule-like structure in jazz reharmonisation.

I am still very much tired after working on it for so long, but I do see now that the original question could have been phrased a little more carefully and should probably be asked in a different way.

To those who offered to stay in touch with me about these topics, I am looking forward to corresponding about it soon, just after I got a little more sleep.

[christianm77]

Well done on getting it in! Sleep...

[princeplanet]

Firstly, congrats to zerenius on his mission accomplished. In 1981 I was one of 20 students in the world who were combining our Lisp coding talents in order to prove that AI could indeed imitate a given musical style. In our case, it took a whole year to teach the computer to compose in the style of Handel's Water Music. This involved reams of "rules"....

Sure we all agreed that the computer made choices that a composer probably wouldn't, but the goal was to put AI to the test, not to compose music

Fast forward to 2018 and, as some have shown in this thread, AI is capable of way more than merely trotting out convincing jazz lines in a specified context. In fact, rule generated lines can sound convincing using way less sophisticated algorithms than are currently known. OK, depends on my definition of "convincing", right? Well, this reminds me of a challenge I put to a top tier Jazz pianist several years ago. I asked him to give me a recording of a blues, an RC and a standard that he'd made where he was happy with at least one chorus of his soloing. These were transcribed and re written as a midi file, where nothing more than the note choices were of interest (obviously not "feel", or any other nuance). Then I selected various auto soloists from BIAB (Peter Gannon's program in which I assume he was the author of the basic algorithm?). I can't remember which (i picked the good sounding ones as most were rather poor), but they were prepared in order to be compared with the pianist's files.

I played them to both Jazz educated listeners and general music lovers to see if they had a preference for one solo over another, against the same BIAB backing track. Yes I know, this hardly resembles real performance, but remember, we are assessing nothing but the note choices made by man vs those made by machine. If the human makes more musical sounding choices, then it will be obvious to everyone, right?

Well, you guessed right, the machine won, easily. I was surprised, as were the unsuspecting listeners, although my pianist friend remained adamant that the machine spewed out nonsensical lines ...

The point of interest for me was that an algorithm composed of motifs, chord tone targeting, various devices and bits of common vocabulary could sound convincing. In retrospect, perhaps I should not have been so surprised, the algorithm was probably based on the playing of great players, coded well enough that most of us here would fail to sound more impressive than it could. Certainly there were the odd clunky moments, but nothing that a few tweaks to the algorithm couldn't fix, you'd think.

Anyway, the whole man vs machine thing is old, just as is the "what really constitutes true improvisation" question. I could care less about it all, and for the same reason we all do, i.e., what we really care about is hearing human expression and story telling in performances. It's why I can't appreciate drum machine driven / loop laden music, or live DJ's over live instrumentalists. AI is now threatening most occupations, and people should be worried, very worried! But one occupation will continue to thrive right through the AI revolution, Art!

So we should be safe. I mean, you'd wanna hope that people never will be happy to prefer the art of machines, given they have no emotions to express, or stories to tell..... or do they? ....

[christianm77]

Oh god that’s depressing lol

[Robertkoa]

Also what was the sound output of the two ' instruments ?

Was it reduced to Piano notes or what ?

I can't see how the Computer would match Rhythms
of an excellent Soloist on Rhythmic Material..

I assume the live comparison was solely reduced to note choices ...right ?

[omphalopsychos]

Also what was the sound output of the two ' instruments ?

Was it reduced to Piano notes or what ?

I can't see how the Computer would match Rhythms
of an excellent Soloist on Rhythmic Material..

I assume the live comparison was solely reduced to note choices ...right ?

No, we computers can do rhythm as well.


Here's a blog post written by a friend of mine about a project he worked on to generate rhythm using neural networks. It's not jazz; it's techno. I'm not the biggest techno fan, but I was very impressed by it. We all admit that it's imperfect. A lot of the imperfection has to do with current technological limits, which will be less of an issue as data and computation become even cheaper.

One element of his conclusion that I really appreciated was that even though it was imperfect, the output was useful as a compositional aid by producing raw primitives that he could assemble as he wanted. In a sense I find that very freeing as a musician. I think a lot of musicians feel very challenged by the thought of computers writing music for them. In my opinion, this is a way to think at a more abstract level as a composer, allowing you to iterate very quickly on new material. Personally I see this as an opportunity for music to become more creative by assigning some of the more mechanical aspects to machines.

[omphalopsychos]

An additional comment, since I anticipate the response "Oh that's techno, it's easy. Jazz is much harder. A computer can't do that."


I respond to that: "You're right. Jazz is much harder. A computer can do it, but you're right it's harder." The beauty of the algorithms used for modeling this stuff is that it scales wonderfully to more complex data. The challenge, and the real reason you don't see too much output in the mainstream media, is that there isn't that much transcribed jazz material to use as source data to train these models. With techno, it's naturally easier because most techno composers are already writing it in a digital format. Therefore enormous databases exist of material. Moreover, techno is more repetitive and less varied and compositionally/"orchestrationally" much simpler than jazz. (That's not meant to come off as pejorative, I love those aspects of techno). Therefore in order to train a jazz model that writes convincing music, you need much much more data than you need to generate a techno drum loop.

Some of the research I worked on was to bypass this issue by relying on raw audio data rather than transcribed material. Unsurprisingly it's much harder. It's not inexpensive enough to do it at home on your laptop, but I think it can eventually be done. Let's just say the state of the field is still experimental and to appreciate the output of some of this stuff requires very understanding and patient ears.

[princeplanet]

Also what was the sound output of the two ' instruments ?

Was it reduced to Piano notes or what ?

I can't see how the Computer would match Rhythms
of an excellent Soloist on Rhythmic Material..

I assume the live comparison was solely reduced to note choices ...right ?

If the question is directed at my post, then I'll say the comparison was for midi produced piano (not even a real piano sample). And yes, there was a rhythmic component to the comparison as well. Many people here know of the BIAB soloist feature. I figure it's just a few dozen cells, motifs and lines per each chord group transposed around for each chord pitch and linked together so one chord's lines resolves nicely to a nearby note of the next chord.

Thomas Owen's Parker dissertation breaks down Bird's playing to essentially the same thing - naturally without the "spontaneous" bits . With the BIAB soloists (if you pick a good one), the thing can last 3 or 4 choruses before it's clunkiness, or repetitiveness becomes very annoying. But for a chorus or 2 it can sound disconcertingly convincing...

[sunnysideup]

Interesting thread. And congrats on the thesis.

Sure you can apply rule-based programming to music. The rules of older western music have been explicit for a long time, and jazz increasingly so. So many different sets of rules to chose from!

Programming a Parkeresque impro shouldn't be impossible, given his few improv choruses for any given tune, and the now explicit rules for constructing bop solos. But it would only be Parkeresque, not Parker. It wouldn't get drunk, yet.

AI is in its infancy and accelerating rapidly.

But I'm assuming it's not ready to take on mid-late period Coltrane, yet.

But what about listening and response, something so basic to jazz and blues in a small improv combo.

How's AI dong on that so far?

[Tal_175]

I have a Phd in classical AI. But I would say such a task could be accomplished to a much greater degree of success with a machine learning approach. Human brain is mostly a statistical machine after all. It's impossible to achieve good results in performance oriented tasks with rule or logic based approaches. That is any domain with a system of complex, non linear interactions and with probabilistic rather than logical definitions. (eg. How tall is "tall"? Certain heights are more likely to be considered tall by most people in a given context then not, as opposed to there being an exact rule or definition)
In performance oriented tasks, human brain uses logic and rules to help train the "statistical (or intuitive) brain" rather than directly using the logic and rules for performance. Logic and rules are too slow for humans to process but that's not the issue for computers. The issue is logic and rules do not constitute an expressive enough language to capture what's needed for performance in complex domains. That's why the AI world mostly abandoned the rules and logic in favor of big data and statistical methods.
But it would be interesting to see how far a rule based system can go in this domain.