While wandering the interwebs one day, I was intrigued to find that a common distinction between "classical" and "popular" music is that the former is notation-based and the latter is improvised. Yes, it is true that classical music is notation-based - that's why so much of it has survived over the centuries. Most of the earliest examples of Western classical music are based on Gregorian chant or plainchant associated with the liturgy of the Roman Catholic Church. From medieval time to late Renaissance, all church music was based on these chants. Even if the entire chant was not reproduced in a work, a fragment of the chant was required to be there. This music has survived through the centuries because of the industrious monks who copied and copied and copied.
However, most of the instrumental music of the same time period is lost to us since it was considered secular (so the monks weren't copying it) and it was improvised. It took longer for instrumental music to be notated, and it was done in gradual steps. Instruments were normally used to accompany vocalists. The first notation for instruments was called a "basso continuo" or ground bass accompaniment. The composer would notate the bass line (to be played by cello or an instrument of a similar range) and write numbers underneath to indicate the interval above this bass line that the other instruments were to play. Instruments were not specifically indicated; it was basically whoever showed up with whatever instrument.
It wasn't really until the Baroque period that instrumental music was precisely notated. Because of technology, the instruments were better (especially winds and brass), music could be printed (no more monks required), the instrumentalists became better at playing their instruments and instrumental music (without vocals) became an important means of expression.
But standardized notation and printing did not mean the end of improvisation. Mozart was famously renowned for improvising works during concerts. Many of these may have been lost to us if he neglected to write them down afterward. During the Classical period, composers developed a new form of instrumental music, the solo concerto - normally a 3-movement work with a solo instrumentalist accompanied by orchestra. A feature of the solo concerto is the cadenza. Near the end of the first movement, the orchestra sits back and relaxes while the soloist is expected to improvise a virtuosic display of his/her talents incorporating and embellishing the themes of the preceding movement. (Think contemporary guitar riff).
During the following Romantic period came the rise of the professional instrumental performer, who was not necessarily a composer. So cadenzas were then written out in full by the composer. No improvisation required.
The 20th century brought many divergent styles to classical music, aleatory music being one of them [see previous blog post for more explanation]. One way of thinking of aleatory (chance) music is that it is a sort of controlled improvisation. The composer provides the performer with options. The choice of option might be up to the performer or something more random, such as rolling dice. The options can involve pitch, rhythm, dynamics, tempo, etc., basically every musical element resulting in a different performance each time.
Now, back to popular music and improvisation. I don't think there is more improvisation in popular music than classical. To me the main difference is the means of replication. Classical music is primarily notation-based, whereas pop, folk and indigenous music is aurally replicated. Much is learned and passed on by aural repetition, repetition, repetition. Popular music is not dependent upon notation. Sir Paul McCartney cannot read a note of music, but he has "written" some enduring pop songs. Were they improvised in the heat of the moment while on stage? No, they were carefully rehearsed over and over, just without written music, using ears instead of eyes. In fact, with the exception of free-style rap, the Grateful Dead or the occasional guitar riff or drum solo, there is very little true improvisation in popular music of today. Yes, classical performers can be rigid about performing every note exactly as written. Popular music thrives on different "arrangements" of the music while keeping the original music recognizable, but is this really improvisation?
Just as the invention of printing was of paramount importance for classical music, modern technology in terms of recording techniques has done the same for popular music as a means of preserving the music, this time in an aural way that ensures its preservation.