diff --git a/README.md b/README.md index 8ca0ae5..a55d96f 100644 --- a/README.md +++ b/README.md @@ -11,6 +11,8 @@ Also included is conversion to a __Melody__ type which is suitable for use in a The conversion of a Gleitzman note name (e.g. Ab4) to a MIDI pitch uses middle C = C4. +For more information, see the [guide](https://github.com/newlandsvalley/purescript-soundfonts/blob/master/docs/GUIDE.md). + ### Acknowledgements The design in very heavily influenced by danigb's JavaScript soundfont project: [soundfont-player](https://github.com/danigb/soundfont-player) and in fact initial versions of this library simply wrapped his. However, this version minimises the amount of native JavaScript which is still necessary in order to wrap Web-Audio functions. diff --git a/docs/GUIDE.md b/docs/GUIDE.md new file mode 100644 index 0000000..d0bfa33 --- /dev/null +++ b/docs/GUIDE.md @@ -0,0 +1,177 @@ +# Purescript SoundFonts Guide + +The idea of ```soundfonts``` is to allow music to be played in the browser in the most basic way possble. It allows you to play a note of a specified pitch and volume for a given duration on a particular MIDI instrument. From this simple starting point, more complicated melodies may be built up. + +The [MIDI 1.0 Specification](https://www.midi.org/specifications/midi1-specifications) enumerates a set of 128 different instruments. If you were to sample one of these instruments over the range of notes that it can play and then digitise the result, you would produce a ```soundfont```. Luckily, this has been done already in Benjamin Gleitzman's [collection](https://github.com/gleitz/midi-js-soundfonts). + +When you use ```soundfonts``` you start by downloading the data for your chosen instruments, which may be from the original Gleitzman site or a different one if you prefer to host it yourself. This data is then converted into a set of web-audio [AudioBuffers](https://developer.mozilla.org/en-US/docs/Web/API/AudioBuffer) - one for each note. This buffer set is then associated with the instrument name (enumerated in [purescript-midi](https://github.com/newlandsvalley/purescript-midi)) and as a result an ```Instrument``` type is produced. These Instruments are saved in an array, + +```soundfonts``` adopts the MIDI definition of a note's [pitch](https://newt.phys.unsw.edu.au/jw/notes.html) and volume (a number between 0 and 1) however it differs from MIDI in that rather than using a ```NoteOn``` followed by a ```NoteOff``` message, it uses a duration (in seconds): + +```purs +type MidiNote = + { channel :: Int -- the MIDI channel + , id :: Int -- the MIDI pitch number + , timeOffset :: Number -- the time delay in seconds before the note is played + , duration :: Number -- the duration of the note + , gain :: Number -- the volume (between 0 and 1) + } +``` + +Here the ```channel``` in which the note plays is synonymous with the index into the Instrument array. Note that the note is scheduled to play after the ```timeOffset``` has elapsed. + +## Dependencies + +Nowadays browsers insist that ```web-audio``` cannot be used until the user has made a gesture on the page (such as hitting a ```play``` button). The minimal set of dependencies required to use ```soundfonts```, using ```web-``` functions to do this is as follows: + +```purs +dependencies = + [ "aff" + , "console" + , "effect" + , "exceptions" + , "maybe" + , "midi" + , "newtype" + , "prelude" + , "soundfonts" + , "unsafe-coerce" + , "web-dom" + , "web-events" + , "web-html" + ] +``` + +## Playing a Note + +The following example plays the note ```A``` on an acoustic grand piano with no delay for a duration of a second. + +```purs +module Main where + +import Prelude +import Effect (Effect) +import Effect.Class (liftEffect) +import Effect.Aff (Aff, Fiber, launchAff, delay) +import Effect.Exception (throw) +import Effect.Console (log) +import Data.Time.Duration (Milliseconds(..)) +import Data.Maybe (Maybe(..)) +import Data.Newtype (wrap) +import Audio.SoundFont (Instrument, MidiNote, loadRemoteSoundFonts, midiNote, playNote, playNotes) +import Data.Midi.Instrument (InstrumentName(..)) +import Web.DOM.ParentNode (querySelector) +import Web.Event.EventTarget (EventTarget, addEventListener, eventListener) +import Web.HTML (window) +import Web.HTML.HTMLDocument (toParentNode) +import Web.HTML.Window (document) +import Unsafe.Coerce (unsafeCoerce) + +main :: Effect Unit +main = do + -- a user gesture is required before the browser is allowed to use web-audio + doc <- map toParentNode (window >>= document) + play <- querySelector (wrap "#play") doc + case play of + Just e -> do + el <- eventListener \_ -> playExample + addEventListener (wrap "click") el false (unsafeCoerce e :: EventTarget) + Nothing -> throw "No 'play' button" + pure unit + +playExample :: Effect (Fiber Unit) +playExample = launchAff $ do + _ <- liftEffect $ log "loading soundfonts" + instruments <- loadRemoteSoundFonts [AcousticGrandPiano] + playNotesExample instruments + +playNotesExample :: Array Instrument -> Aff Unit +playNotesExample instruments = do + _ <- liftEffect $ log "paying note sample A" + duration_ <- liftEffect $ playNote instruments noteSampleA + pure unit + +noteSampleA :: MidiNote +noteSampleA = midiNote 0 57 0.0 1.0 1.0 +``` + +## Playing Chords + +To play a chord, you need to supply an array of ```MidiNote``` at the appropriate pitches where each note is set to play at an identical ```timeOffset```. Firstly we need to descibe the chord: + +```purs +noteSampleA :: MidiNote +noteSampleA = midiNote 0 57 0.0 1.0 1.0 + +noteSampleCAt :: Number -> MidiNote +noteSampleCAt offset = midiNote 0 60 offset 1.0 1.0 + +noteSampleEAt :: Number -> MidiNote +noteSampleEAt offset = midiNote 0 64 offset 1.0 1.0 + +chord :: Array MidiNote +chord = + [ noteSampleA + , noteSampleCAt 0.0 + , noteSampleEAt 0.0 + ] +``` + +and replace ```playNotesExample``` with this which uses ```playNotes``` to play the note array: + +```purs +playNotesExample :: Array Instrument -> Aff Unit +playNotesExample instruments = do + _ <- liftEffect $ log "paying a chord" + duration_ <- liftEffect $ playNotes instruments chord + pure unit +``` + +## Playing Note Sequences + +If you want to play a sequence of notes, then you have to start each note at the correct time offset. For example, to play the notes of the chord above, but in a legato sequence, you can use this: + +```purs +legato :: Array MidiNote +legato = + [ noteSampleA + , noteSampleCAt 1.0 + , noteSampleEAt 2.0 + ] +``` + +Notice that each succeding note starts at the accumulated time offset of the note sequence that has preceded it. You again supply this array to ```playNotes```: + +```purs +playNotesExample :: Array Instrument -> Aff Unit +playNotesExample instruments = do + _ <- liftEffect $ log "paying legato" + duration_ <- liftEffect $ playNotes instruments legato + pure unit +``` +## More Complex Melodies + +It quickly becomes cumbersome to describe an entire tune as a simple ```MidiNote``` array. It is much more convenient to break it up into phrases. For this reason, the ```Melody``` module defines the ```MidiNote``` array that we have already seen as a ```MidiPhrase``` and an array of such phrases as a ```Melody```. Each MidiPhrase has time offsets relative to the start of the phrase (usually zero for the first note). + +Remember that when you play through ```web-audio``` the sound is generated asynchronously. This means that if you need to play an entire melody and have it paced properly, you need to invoke a delay for the duration of each phrase after it has played in order that the next phrase starts at the correct time. The ```Melody``` module defines two functions that handle the pacing in this way - ```playPhrase``` and ```playMelody```. + +We can thus invoke the ```legato``` example using ```playPhrase``` instead of ```playNotes```. First, import from ```Melody```: + +```purs +import Audio.SoundFont.Melody (Melody, playPhrase, playMelody) +``` + +and then use: + +```purs + duration_ <- playPhrase instruments legato +``` + +Note that ```playPhrase``` runs directly in ```Aff``` and not ```Effect``` because it needs access to Aff's ```delay``` function. Nevertheless, the result is identical, apart from the fact that the main thread of execution is suspended until the playback is complete. + +Finally, if we also import ```Data.Unfoldable (replicate)```, we can play a melody of the repeated basic three note phrase: + +```purs + duration_ <- playMelody instruments (replicate 3 legato) +``` + \ No newline at end of file diff --git a/example/src/Main.purs b/example/src/Main.purs index 660700f..ccd7b9d 100644 --- a/example/src/Main.purs +++ b/example/src/Main.purs @@ -12,6 +12,7 @@ import Data.Unfoldable (replicate) import Audio.SoundFont (Instrument , MidiNote , loadRemoteSoundFonts + , midiNote , playNote , playNotes) import Audio.SoundFont.Melody (playMelody) @@ -23,27 +24,23 @@ import Web.HTML.HTMLDocument (toParentNode) import Web.HTML.Window (document) import Unsafe.Coerce (unsafeCoerce) -note :: Int -> Int -> Number -> Number -> Number -> MidiNote -note channel id timeOffset duration gain = - { channel : channel, id : id, timeOffset : timeOffset, duration : duration, gain : gain } - noteSampleA :: MidiNote -noteSampleA = note 0 57 0.0 0.5 1.0 +noteSampleA = midiNote 0 57 0.0 0.5 1.0 noteSampleC :: MidiNote -noteSampleC = note 0 60 0.0 0.5 1.0 +noteSampleC = midiNote 0 60 0.0 0.5 1.0 noteSampleE :: MidiNote -noteSampleE = note 0 64 0.0 0.5 1.0 +noteSampleE = midiNote 0 64 0.0 0.5 1.0 notesSample :: Int -> Array MidiNote notesSample channel = - [ note channel 60 1.0 0.5 1.0 - , note channel 62 1.5 0.5 1.0 - , note channel 64 2.0 0.5 1.0 - , note channel 65 2.5 0.5 1.0 - , note channel 67 3.0 1.5 1.0 - , note channel 71 3.0 1.5 1.0 + [ midiNote channel 60 1.0 0.5 1.0 + , midiNote channel 62 1.5 0.5 1.0 + , midiNote channel 64 2.0 0.5 1.0 + , midiNote channel 65 2.5 0.5 1.0 + , midiNote channel 67 3.0 1.5 1.0 + , midiNote channel 71 3.0 1.5 1.0 ] main :: Effect Unit diff --git a/src/Audio/SoundFont.purs b/src/Audio/SoundFont.purs index 69c6998..b8351ae 100644 --- a/src/Audio/SoundFont.purs +++ b/src/Audio/SoundFont.purs @@ -1,5 +1,6 @@ module Audio.SoundFont - ( AudioBuffer + ( module Exports + , AudioBuffer , Instrument , InstrumentChannels , MidiNote @@ -10,8 +11,11 @@ module Audio.SoundFont , logLoadResource , loadInstrument , loadInstruments + , loadInstrumentUsingProvider + , loadInstrumentsUsingProvider , loadRemoteSoundFonts , loadPianoSoundFont + , midiNote , playNote , playNotes , instrumentChannels @@ -20,7 +24,8 @@ module Audio.SoundFont import Affjax.Web (defaultRequest, request) import Affjax.ResponseFormat as ResponseFormat import Audio.SoundFont.Decoder (midiJsToNoteMap, debugNoteIds) -import Audio.SoundFont.Gleitz (RecordingFormat(..), SoundFontType(..), gleitzUrl) +import Audio.SoundFont.Gleitz (RecordingFormat(..), gleitzUrl) +import Audio.SoundFont.Gleitz (SoundFontType(..)) as Exports import Control.Parallel (parallel, sequential) import Data.Array (index, last, mapWithIndex) import Data.ArrayBuffer.Types (Uint8Array) @@ -86,7 +91,7 @@ foreign import isWebAudioEnabled foreign import setNoteRing :: Number -> Effect Unit --- | load a bunch of soundfonts from the Gleitzmann server +-- | load a bunch of soundfonts from the Gleitzmann server using the MusyngKite provider loadRemoteSoundFonts :: Array InstrumentName -> Aff (Array Instrument) @@ -94,22 +99,36 @@ loadRemoteSoundFonts = loadInstruments Nothing -- | load the piano soundfont from a relative directory on the local server +-- | using the MusyngKite provider loadPianoSoundFont :: String -> Aff Instrument loadPianoSoundFont localDir = loadInstrument (Just localDir) AcousticGrandPiano --- | load a single instrument SoundFont + +-- | load a single instrument SoundFont using the ```MusyngKite``` provider -- | The options are to load the soundfont from: --- | Benjamin Gleitzman's server (default) +-- | Benjamin Gleitzman's server (default of Nothing) -- | A directory from the local server if this is supplied - loadInstrument :: Maybe String -> InstrumentName -> Aff Instrument -loadInstrument maybeLocalDir instrumentName = do +loadInstrument maybeLocalDir instrumentName = + loadInstrumentUsingProvider maybeLocalDir Exports.MusyngKite instrumentName + +-- | load a single instrument SoundFont +-- | The options are to load the soundfont from: +-- | Benjamin Gleitzman's server (default: Nothing) or +-- | A directory from the local server if this is supplied +-- | and to use your choice of SoundFont provider +loadInstrumentUsingProvider + :: Maybe String + -> Exports.SoundFontType + -> InstrumentName + -> Aff Instrument +loadInstrumentUsingProvider maybeLocalDir provider instrumentName = do recordingFormat <- liftEffect prefferedRecordingFormat let url = @@ -117,7 +136,7 @@ loadInstrument maybeLocalDir instrumentName = do Just localDir -> localUrl instrumentName localDir recordingFormat _ -> - gleitzUrl instrumentName MusyngKite recordingFormat + gleitzUrl instrumentName provider recordingFormat res <- request $ defaultRequest { url = url, method = Left GET, responseFormat = ResponseFormat.string } @@ -132,15 +151,34 @@ loadInstrument maybeLocalDir instrumentName = do font <- traverse decodeAudioBuffer noteMap pure (Tuple instrumentName font) --- | load a bunch of instrument SoundFonts (in parallel) +-- | load a bunch of instrument SoundFonts (in parallel) using the MusyngKite provider. -- | again with options to load either locally or remotely -- | from Benjamin Gleitzman's server loadInstruments :: Maybe String -> Array InstrumentName + -> Aff (Array Instrument) +loadInstruments maybeLocalDir instrumentNames = + loadInstrumentsUsingProvider maybeLocalDir Exports.MusyngKite instrumentNames + +-- | load a bunch of instrument SoundFonts (in parallel) +-- | with options to load either locally or remotely +-- | from Benjamin Gleitzman's server +-- | and to use your choice of SoundFont provider +loadInstrumentsUsingProvider + :: Maybe String + -> Exports.SoundFontType + -> Array InstrumentName -> Aff (Array Instrument) -loadInstruments maybeLocalDir instrumentNames = - sequential $ traverse (\name -> parallel (loadInstrument maybeLocalDir name)) instrumentNames +loadInstrumentsUsingProvider maybeLocalDir provider instrumentNames = + sequential $ traverse + (\name -> parallel (loadInstrumentUsingProvider maybeLocalDir provider name)) instrumentNames + + +-- | Construct a MidiNote +midiNote :: Int -> Int -> Number -> Number -> Number -> MidiNote +midiNote channel id timeOffset duration gain = + { channel, id, timeOffset, duration, gain } foreign import decodeAudioBufferImpl :: Uint8Array -> EffectFnAff AudioBuffer @@ -204,7 +242,7 @@ logLoadResource -> Effect (Fiber Unit) logLoadResource instrument = let - url = gleitzUrl instrument MusyngKite OGG + url = gleitzUrl instrument Exports.MusyngKite OGG in launchAff $ do res <- request $ defaultRequest