Follow

Follow is a program that displays a musical score and shows a cursor in the score that follows what you play. The program listens to a midi interface to read the notes that you play. When you have no midi interface you can use the build in keyboard on the screen. You can also use an audio-pitch recognizer that outputs a midi stream.

You can select a staff that you want to play and the program will play all other staves while following you on the selected staff. So you hear your self accompanied.

The program can also play an audio file (that is synchronized to the score) and it checks the notes you play to the actual notes in the score. The cursor is colored red if your notes are wrong or too late. Follow can import timing data from abcweb to synchronize any audio file to the score.

Apart from displaying a cursor in the score, the program also measures the timing of your performance and makes a timing plot at the end with some statistical information.

Follow can also be used in a concert setup, where multiple devices show different parts of the same score. A master device determines the tempo and all other (slave) devices follow. The master can either use the play back feature or actually follow a midi instrument played by the director (or solist). (see the synchronization feature of Follow)

The Timing Bar

	The metronome symbol flashes red when the metronome is running. When you click this symbol playback starts with the metronome on (flash + click sound). When you click this symbol while the metronome is running it stops the metronome. With the input field to the right of the metronome you can change the score tempo.
Performance Bar	The bar displays your timing errors (see follow mode). The number in the bar indicates the difference in milliseconds between your note and the score time. A negative number means that you played to early, a positive number means you played too late. The colored bar represents the average absolute error of your last 10 notes. The organge and red color warns you when the average becomes too much. When the life bar mode is selected the bar displays a health precentage (starting with 100 and decreasing with your timing errors, see further life bar).
Start	Starts play back (in the selected follow mode). Normally a count-in is displayed before play back really starts. The count-in can be disabled with URL option "nocnt". The computer can also play the score with the internal synthesizer or on the connected MIDI device (see Output port). The notes you play yourself are always echoed on the selected midi channel unless you select the mute own staff option. When the internal sythesizer is selected in combination with "mute own staff", the computer plays all staves, except the one you play your self.
◉	Microphone symbol. Its color is brown when the microphone is on. Dark-grey when the microphone is off. The symbol is clickable and toggles the microphone on/off. It is a shortcut for the use microphone checkbox in the microphone dialog.
Volume Slider	Sets the volume of the internal synthesizer. Useful to balance the synthesizer and the audio player if an audio file is used.
	When an audio file is present, you can enable/disable sound, by clicking this icon.
Score	Here you can see how well you are playing. The score value is updated while you are playing and increases until 100% when you make no errors and play all notes at the right time. The score value is computed as follows: score = (correct notes - 0.5 * wrong notes - 0.5 * late notes) / total number of notes The factor 0.5 for wrong notes and late notes can be changed with a URL option.
Resizing	Dragging (touch-hold-move) downwards in the timing bar area resizes the bar.

The on-screen Keyboard

Touch/Click area

The on-screen keyboard (see menu to enable) has a sensitve area above the keys, stretching along the whole keyboard. The height of this area is the same as the green triangles you see at both ends of the keyboard.
When you click/touch-and-hold in this area it lights up green and you can resize or shift the keyboard. Dragging upwards enlarges the keys (makes them longer) and dragging downwards makes them shorter. Dragging sidewards shifts the keyboard to the left or right.

Green triangles

A click/touch-and-hold on the right triangle makes the keys broader in repeated steps as long as you hold. The left triangle does the same but makes the keys smaller.
A normal click/touch in one of the green triangles shifts the keyboard precisely one octave in that direction.

The Menu

Score file	Browse the local file system and select a score file. The file can either contain MusicXML or ABC notation. The selected file is opened and the score rendered on the screen. In addition you can also select an audio file (.mp3 or .ogg) or a preload file.
Use Dropbox	Change the score file button into the Dropbox Chooser button to browse files in your dropbox.
Microphone	The Settings button opens the microphone dialog.
Output port	This selection control can be used to activate the internal synthesizer or to select a midi output port. A midi output port is only available when the browser supports midi and the computer has a midi interface (connected to a midi instrument or sound module). When the option "No sound" is selected the internal synthesizer is disabled and no output is sent to MIDI. The audio player can still produce sound when an audio file is loaded. The default selection is "Internal synth".
Midi echo	When playing a midi instrument the notes from the instrument are echoed back on the midi ouput port. This may not be desirable when the instrument already produces sound from itself. In that case the midi echo can be switched off. Notes from the accompaniment are always sent to the output port.
Follow mode	The aim this program is to display a cursor in the score that follows what you play on a midi instrument. There are several options that determine how the notes you play are followed and checked. The cursor always gets one of the following colors: BLUE means you did not play anything, GREEN means you played the right note(s) at the right time, RED means you played one or more wrong notes. Adaptive tempo. In this mode the tempo is no contraint on your performance. The tempo just follows your playing speed. The program only checks if you correctly play the notes selected by the cursor. If so, it moves the cursor to the next position in the score. Constant tempo with statistics. In this mode you have to play exactly on the beats of the metronome. The metronome starts automatically when you play the central C and starts with a count-in. After the count-in finishes you have to play the notes selected by the cursor in the tempo of the metronome. Your timing accuracy is indicated in the performance bar at the top left of the window. When the average timing error of your last 10 notes exceeds 100 millisec. your performance is considered a failure and a dialog is presented with some statistics followed by a timing plot. When you reach the end of the score you get the same information but your performance is considered successful. Constant tempo with life bar This mode is identical to the previous one, only the performance bar now indicates a health percentage. You start with a full bar (100 points) and with every bad timing (> 35 millisecs) you loose 5 points. However you gain 5 points with a good timing (< 35 millisecs). Adaptive tempo (with timeout) This mode is identical to the first mode. There is no contraint on your tempo. However, the tempo you enter in the tempo box is now considered a minimum tempo at which play back will continue. You can play faster than this tempo, but not slower. When you play slower (or play the wrong notes or do not play at all) the program will continue to move the cursor and play the accompaniment at the minimum tempo. You can catch on at any time and start to play faster again. Adaptive tempo (minimum tempo) This mode is identical to the first mode, but the tempo never drops below the initial value of the tempo. You can speed up without limit, but you can only slow down until the initial tempo is reached. Playback (no check) The cursor continues even if you do not play any notes yourself. If you do play notes they are checked against the current cursor position and the score is computed. So even in playback mode you can attain a 100% score if you play all notes at the right time. Constant (no check) When you play a wrong note the cursor stops. Once you play the right note, the cursor resumes moving. When you do not play at all the cursor also stops. If there is a synchronized audio file playing it also stops when the cursor stops and also resumes at the correct position once you play the right note(s).
Check mode	Notes you play are followed in the score. Which notes you have to play is determined by one of the three check modes: Top notes of selected staff: Only the top most notes of the selected staff have to be played. When the staff has more voices or layers the top note of each of them must be played. In fact, 'top' only refers to chords. All notes of selected staff: You have to play all notes (of all voices) in a staff. All notes of grand staff: You have to play all notes (of all voices) in the selected grand staff. A grand staff is selected by clicking in the topmost staff of the grand staff. The staff below is then added to make a grand staff. All notes of all staves: In this mode all staves are taken into account. You have to play all notes in all staves of the score.
Mute own staff	This option mutes the notes of the selected staff during play back. Normally all staves are played. With this option marked, you hear all other staves but not your own staff (the one you follow/play). You should select this option when your (MIDI) instrument is already producing sound.
Next note delay	After you play the selected notes the cursor moves to the next time position in the score. When this option is selected (default) the cursor does not move immediately, but waits for half the time span between the current note and the next one (with the current tempo). This little delay feels more naturaly when you play in tempo. When you do not play in tempo it is better to switch this option off. Especially when you play slowly or irregularly.
Metronome	When this option is enabled the metronome continues ticking after the count-in has finished. Otherwise the metronome stops after the count-in. This option is automatically selected when you start the score by clicking on the metronome icon.
Play one note before start	When the count-on is finished the computer starts playing the accompaniment one note before the cursor. So you do not start playing immeriately after the count-in, but the accompaniment does, one note before the selection, where you have to fall in.
Show Timing	This button shows a timing plot with statistical information of your performance. It is only available if you play one of the constant tempo modes. In the adaptive tempo modes to timing data is stored. It is used to adapt the tempo.
Keyboard	Shows a keyboard where you can play the required notes. It is useful if you have no midi keyboard available. The keys can be moved and scaled by dragging (horizontally and vertially) in a small area just above the keyboard. There are also two octave shift buttons, that shift the keys precisely one octave. When you press-and-hold on these buttons, the keyboard is resized (scaled) in horizontal direction.
Transparency	This makes the keyboard transparent and it overlays the score. May be useful for small devices.
Show notes	With this option enabled you see the notes you must play highlighted on the keyboard with a red background. (The notes selected by the cursor are shown as depressed on the keyboard).
Extract staff	When the Single is selected the current staff will be extracted for exclusive display, while all other staves are hidden. The hidden staves still play when needed (in follow mode of the extracted staff or in general play back mode) When Grand is selected the current staff becomes the top staff of a grand staff that is subsequently selected for exclusive display.
Two lines	With this option enabled only two lines are shown: the line with the cursor and the next one. This eases score reading, as all other lines are not needed. Each line either shows a system or a staff, depending on the Extract staff option.
Save Preload	Pops up the save dialog where you can save a preload file. This file contains the score and most of the settings you entered either with the menu or with url parameters.
Page width	Uses the given width in cm to render the score. It should be larger than 5 and smaller than 50 cm. The original page formatting of the score is erased and an optimal layout is computed for the given page width. Also the (abbreviated) instrument names are erased from all but the first staff to save space. If the real screen/window width is larger than the given number the score will seem enlarged. The page width can also be set to a larger value than in the original score, which results in longer lines with more bars on a line. Entering a number smaller than 5 erases the setting and resets the score to the original page layout.
Bluetooth LE	Opens a dialog window that shows Bluetooth LE MIDI devices (keyboards, sensors). You can pair with one or more of those devices. After paring, the received MIDI messages are processed normally by follow. When the Midi echo checkbox is on, the Bluetooth MIDI messages are also echoed on the normal MIDI output port.
Full Screen	Switches the browser to full screen display (if supported).
Side Scroll	Displays the score as one long (horizontal) system. The cursor stays in the middle of the window while the score scrolls horizontally.
Hide Cursor	Makes the cursor invisible.
Help	Shows this file.

Microphone dialog

Use Mic.	Asks permission to use the microphone. When the microphone is working, a pitch detector converts the audio signal into notes, which are followed in the score. Only monophone melodies can be followed. To get access to the microphone the program uses the WebRTC API which is, at the time of writing, only available in Chrome and FireFox on the PC and Android platforms.
Gain	The gain slider sets the volume level of the microphone. Decibel values of the currently received signal are shown by the organge volume bar (dB). The target sound should be stronger than the level indicated by the blue line in the volume bar. This level corresponds to the Min. Level slider (see below). The peaks of the current microphone signal are shown in the graph by a grey wave line.
Sensitivity	The sensitivity slider sets the height of the dashed blue horizontal line in the graph. When the green wave line drops below the dashed blue line a note is recognized. Higher values of the sensitivity result in more (or earlier) recognition of pitches. Values above 5 are not practical, because even without sound notes will be detected (in the background noise). The optimal value is between 1 and 4. The green wave line represents the Average Squared Mean Difference Function (ASMDF) of the microphone signal. This function approaches zero when the input signal is periodical and the x-coordinate equals the period. The x-coordinate of the recognized period is shown with a small red vertical line in the graph.
Min. Level	The minimum level is a treshhold for the microphone signal. Only signal levels above this threshold are considered for pitch detection. The value should be between -40 dB and 0 dB. The current value of the minimum level is shown in the microphone level bar by a vertical blue line.
Cents bar	The cents bar displays the deviation of the recognized pitch from the closest musical note. Notes are supposed to have the twelve-tone equal temperament. Your instrument should be well tuned for the pitch detector to produce notes consistent with the score.
Bass guitar	Normally the lowest note that can be detected is an F#3. When the bass checkbox is marked, the lowest possible note becomes an F#2. However, this doubles the internal sample buffer and makes detection four times slower. Therefore it should only be used when detection of such low notes is really needed.

URL parameters

A link to follow.html can have several URL-parameters, which are explained in the table below. An example of a link with some parameters is:

http://wim.vree.org/js/follow.html?d:75mnyv0xsmye3mq/gadma_2.xml&syn&tmp=120&keyb&ksh&play

It plays a MusicXML file from Dropbox on the internal synthesizer and shows the played notes in the score and on the keyboard.

score file	The score file has to be a URL poining to a score file with exension .xml or .abc, for instance: http://some.domain.org/follow.html?path/to/score_file.xml The file can also reside in your dropbox, where it has to be shared publicly. In this case you use the dropbox identifier prefixed with d: For example, dropbox tells me that a shared file in my dropbox has this link: https://www.dropbox.com/s/75mnyv0xsmye3mq/gadma_2.xml?dl=0 The relevant part of this link is 75mnyv0xsmye3mq/gadma_2.xml, which you use as follows: http://wim.vree.org/js/follow.html?d:75mnyv0xsmye3mq/gadma_2.xml Note the link is prefixed with d: Try it out by clicking the link.
audio file	The file name can also be a URL pointing to an audio file (.mp3 or .ogg). You can have both a score file and an audio file. For example: http://wim.vree.org/js/follow.html?../js3/test/bwv99k.abc&../svgParse/bwv99_msc.mp3 You can also use the dropbox syntax for the audio file.
preload file	The third possibility for a file name is a preload file (see save). For instance: http://wim.vree.org/js/follow.html?d:0q8p1zvfjn9ijw2/pre_ave.js You do not need any further URL parameters as all settings are included in the preload file.
stf=	With the stf=staff_number parameter you specify which staff should be played by you. The notes in this staff are highlighted during your play. The first staff has number 1.
extr	Sets the Extract staff option to Single. Can be used in combination with stf= to show only a single part from an multi part score.
extrg	Sets the Extract staff option to Grand. Can be used in combination with stf= to show only a single grand staff from an multi part score. The Check mode remains unchanged. If you want all notes in the grand staff highlighted you have to add the URL option mod=3.
2ln	Sets the Two lines option.
pw=	The pw=xx parameter sets the page width to xx cm. (see Page width)
fmd=	The fmd=follow_mode chooses one of the four follow modes. The value of follow_mode is 1, 2, 3, 4 or 5. (see menu)
mod=	With the mod=check_mode parameter you can choose one of the three check modes. The value of check_mode is 1, 2, 3 or 4. (see menu)
tmp=	The tmp=tempo sets the play back tempo.
nodel	The Next note delay checkbox is unmarked. After (correctly) playing the highlighted notes the cursor jumps immediately to the next notes, without delay. Normally there is a dealy of half the (played) note length.
metro	The metro parameter marks the metronome checkbox in the menu, so that the metronome continues when the count-in terminates. Normally the metronome stops ticking after the count-in.
keyb	When the keyb parameter is present the keyboard will be shown.
ksh	The ksh parameter marks the show notes checkbox, so that notes highlighed by the cursor are also highlighted on the keyboard.
opa=	This parameter sets the opacity of the keyboard. opa should be in the range 1-9, where opa=1 is opaque and opa=9 is maximal transparent.
nosnd	The nosnd parameter disables the internal synthesizer and MIDI output. It is the same as selecting the option "no sound" in the menu (Output port selector)
nomenu	Hides the menu.
nobar	Hides the tempo field and the performance bar.
nocur	Hides the cursor.
line=	Sets the position of the dotted line. line=1 is at the top and line=100 is at the bottom of the page.
ip	When this parameter is present the device tries to connect to a Web Socket server at port 8091. The ip address of this server is taken from the host part of the URL. The required socket server and a web server at port 8090 are both started when you run the program abcSrv.js with nodejs. When connecion is successful the menu button becomes green. Positioning of the cursor is now syncrhonized with a master device (see parameter ipm).
ipm	When present this parameter makes the device the master of a group after connecting to a Web Socket server at port 8091. When connecion is successful the menu button becomes red. The master sends its cursor position (when it changes) to all slaves. The slaves synchronize their cursor movement to the master.
hrz=	Switches to horizontal scrolling (side scroll mode) when hrz > 0. (the exact value does not matter)
scl=	Sets the scale of the score when in side scroll mode (0 < scale < 2.0)
shrnk=	Sets the note spacing when in side scroll mode. Note spacing is maximal when shrnk = 0. Higher values make the score shrink (it is the ABC %%maxshrink parameter)
vol=	Sets the volume of the internal synthesizer (slider in the timing bar).
pnlerr=	Sets the penalty factor for wrong notes in the score calculation (between 0 and 1, default 0,5).
pnltim=	Sets the penalty factor for late (or early) notes in the score calculation (between 0 and 1, default 0,5). A note is considered too late (or too early) if it is played more than 100 milliseconds too late (or too early).
nosnd	Switch off the internal synthesizer. (output port selection shows "no sound")
nocnt	Do not show and play the count-in before play back is started (start button pressed, central C played or metronome icon clicked)
mute	Enables (marks) the "Mute own staff" option in the menu.
nosm	Disable smooth scrolling.
ctrls	Show the controls of the audio player. Normally the audio player is hidden. Only works when an audio file is present.
scal	Show the score calulation in the timing bar. Without this option only the score value is shown, but not the calculation.
tmng	Also include (and show) the timing penalty in the score calulation. Without this option no timing penalty is subtracted from the score.
lncur	Show a line cursor at the current real time. In addition to the normal note cursor this line cursor shows playback time continuously, linearly moving from note to note.
oneclr	Only use the blue color for the line cursor (one color). Without this option the line cusor receives the same color as the note cursor (i.e. green if you play the right note, red if you play the wrong note).

Synchronizing multiple devices

Multiple devices can follow the same score in precise synchronization. Each device can focus (extract) a different part of the score. In such a group of devices there is one master who determines the tempo, either by just using the play back feature or by actually playing on a midi device. The other devices are slaves and just follow the master. To achieve synchronization Follow uses a special server program called AbcSrv.js.

AbcSrv.js is included in the distribution zip file. It is a javascript program that implements a simple http server and a special Web Socket server. The normal http server runs on port 8090 and the web socket server on port 8091.
To run abcSrv.js you need to have Node installed. Node is a javascript engine with unrestricted access to the computer's (networking) capabilities.
When you have Node installed you can start abcSrv.js in a terminal window (command line prompt) with:

# nodejs abcSrv.js

The command for Node (nodejs in the example) may be different on your platform (for instance, just node). When started abcSrv.js prints the following output in the terminal window:

websocket server listening on port: 8091, address: 192.168.xx.xx
http server listening in port: 8090, adress: 192.168.xx.xx
web server root directory: /path/to/where/abcSrv.js/is/located/

Now you can check that the server works by just typing 192.168.xx.xx:8090 in a browser on another device connected to the same (WIFI) network. It should display a listing of files, one of which is follow.html. Then you can open follow.html as slave or as master, using:

http://192.168.xx.xx:8090/follow.html?some_preload.js&ip   (for all slave devices)
http://192.168.xx.xx:8090/follow.html?some_preload.js&mstr (for the master device)

In stead of a preload file, as in the example above, you can also just use a score file, or even no file at all and open a score file later with the menu button.

When a device succesfully connects to abcSrv.js the menu button becomes green. For the master device the menu button becomes red. This gives some feedback to indicate that all is workiing well and that henceforth all play/pause/jump actions will be issued by the master of the group. The slave devices cannot initiate any actions, they only follow the master.

You should have all files of the distribution (including abcSrv.js) extracted to one directory, which becomes the server root directory. All preload files you make or score files you want to use should also be in this directory.

On a normal (i.e. not jailbroken) iPad, abcSrv.js has been tested using the app Touch Code Pro from iTunes. This app can load and run abcSrv.js, turning the iPad into a simple http server and websocket server. While Touch Code Pro is running you can still use the browser to also open Follow, so you do not loose a device.