Dist files?

[chopin2256]

Hi,

Can you publish the distribution files here? You have them published here, however the files were compiled with webpack in development mode (since the map files are required).

I am evaluating solutions for my product Music Jotter, and I'd like to see if your neat solution here can work. Is it possible to run note on and note off events? I have to dig in a little more.

Thanks!

[chopin2256]

Also, fyi, the reason why I can't use your npm package is because of this error:

Could not load source '\node_modules\js-synthesizer\src\main\ISequencerEventData.ts': Unable to retrieve source content.

I'd prefer to use npm due to the typings. I can try adding the file manually to see if this fixes the problem, but I run automated solutions on my server with npm updates.

[jet2jet]

js-synthesizer package in npm includes JS files (ES modules) in dist/lib directory and if you use webpack, import * as JSSynth from 'js-synthesizer' will read JS files in dist/lib (since package.json has the field module with the value dist/lib/index.js).
Other bundlers are not tested, so some problems may occur.

If you can, please provide a reproduction for the problem.

[chopin2256]

Hi, thank you for your response.

This is actually the error I get with npm:

Could not load source '\node_modules\js-synthesizer\src\main\ISequencerEventData.ts': Unable to retrieve source content.

Given the error above, I just wonder if the npm installer is not downloading the correct file. I have decided that I am going to use your solution with my application since it does work otherwise, and offers a lot of the control I need. I might have to download your project later and test it locally if we can't figure out the npm issue.

[jet2jet]

I created the repository https://github.com/jet2jet/js-synthesizer-test-webpack for the test.
This repository should work (at lease it worked in my environment) so please use and confirm this to check the problem.

[chopin2256]

I was able to successfully run and launch this project, with the browser outputting [object Object]. Here's a copy of the log information. Please note, I upgraded my node to the latest version:

[2024-03-13T12:34:29.722Z] "GET /" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"
(node:28976) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
(Use node --trace-deprecation ... to show where the warning was created)
[2024-03-13T12:34:29.776Z] "GET /node_modules/js-synthesizer/externals/libfluidsynth-2.3.0.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"
[2024-03-13T12:34:29.779Z] "GET /dist/test-webpack.js" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"
[2024-03-13T12:34:29.811Z] "GET /favicon.ico" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:123.0) Gecko/20100101 Firefox/123.0"
[2024-03-13T12:34:29.813Z] "GET /favicon.ico" Error (404): "Not found"

[chopin2256]

Hi,

How can I contact you? I'm going to be using this library in my product Music Jotter since I am able to get it to work. However, I have some questions, especially regarding efficiency. Some question would be:

• Would I have to load a soundfont each time when playing the midi? If so, can the soundfont be cached somehow?
• Can we connect this synth via a virtual midi port, and hook it up to Kontakt? I'm pretty sure the answer is yes, but would love to have your input around this.

Looking forward to hearing from you.

[jet2jet]

• Would I have to load a soundfont each time when playing the midi? If so, can the soundfont be cached somehow?

No, it's OK to load Soundfont only once (until calling Synthesizer.unloadSFont).

• Can we connect this synth via a virtual midi port, and hook it up to Kontakt? I'm pretty sure the answer is yes, but would love to have your input around this.

I don't know Kontakt well, but I think it is necessary to pass MIDI messages to WebMIDI API, which is different from WebAudio. js-synthesizer only passes rendered data to WebAudio, but if you want to pick MIDI messages, Synthesizer.hookPlayerMIDIEvents can be used.

[chopin2256]

A bit confused with your documentation. I'm assuming this bit of code must be called each time in order to play the midi. Wouldn't each call load the soundfont? This is the problem I am having, high memory usage. As of now, I have to keep calling this block of code in order to get the play function to work with seek. I feel like I am missing something though.

// Prepare the AudioContext instance
var context = new AudioContext();
var synth = new JSSynth.Synthesizer();
synth.init(context.sampleRate);

// Create AudioNode (ScriptProcessorNode) to output audio data
var node = synth.createAudioNode(context, 8192); // 8192 is the frame count of buffer
node.connect(context.destination);

// Load your SoundFont data (sfontBuffer: ArrayBuffer)
synth.loadSFont(sfontBuffer).then(function () {
    // Load your SMF file data (smfBuffer: ArrayBuffer)
    return synth.addSMFDataToPlayer(smfBuffer);
}).then(function () {
    // Play the loaded SMF data
    return synth.playPlayer();
}).then(function () {
    // Wait for finishing playing
    return synth.waitForPlayerStopped();
}).then(function () {
    // Wait for all voices stopped
    return synth.waitForVoicesStopped();
}).then(function () {
    // Releases the synthesizer
    synth.close();
}, function (err) {
    console.log('Failed:', err);
    // Releases the synthesizer
    synth.close();
});

[chopin2256]

I was able to fix the memory problem by following your code example here. Actually, it seems to work really well now and I am not even noticing too much latency. I'll show you the product in action once I am ready to make my next YouTube video.

What are your plans for upkeep with this module? I know that down the line I will need to do more synth work once I start making more consistent sales with Music Jotter. I'll be doing work with virtual midi ports as well.

[jet2jet]

Wouldn't each call load the soundfont?

Sorry for the unkind code, but Soundfont is only needed to load once. That sample code was simply intended to play a MIDI file once.

What are your plans for upkeep with this module?

Currently there are no plans to update the module, but if necessary (such as bug fixes or valuable feature requests), I'll work for the update.

[chopin2256]

Can your module connect to an available midi port? I'm still trying to see if fluidsynth allows connection via a midi port (in this case, a virtual midi port, such as LoopMidi). Instead of using the loaded soundfont to playback via the synth, I'm trying to see if I can use an external sampler for this (Kontakt is an example, but any external sampler would be similar).

If this is not possible I may have to use WebMidi for this (if that is even possible, still researching). However, if fluidsynth can handle this, I'd rather just use this one piece of technology.

[jet2jet]

Can your module connect to an available midi port?

No, Emscripten, used to build wasm version of fluidsynth, does not have ability to send messages to a MIDI port, so wasm version of fluidsynth cannot use the MIDI port. Therefore, you must use WebMIDI manually.

[chopin2256]

Thanks! I figured as much. I also figured out how to use WebMIDI to connect to an external synth such as Kontakt. I'll still be using your module for soundfont integration.

[chopin2256]

Hi,

I have possibly one more question. I'm unsure how to send a midiNoteOn event with a specific instrument. And when I send a midiNoteOn event, can I load a new instance of synth? Would I have to set PlayPlayer at all?

For example, once I initialize the synth, I may do something like this:
synth.midiNoteOn(0, 60, 64)

And if I am using a GM soundfont, instrument 41 would be a violin. How would I play a noteOn event with a violin?

[jet2jet]

How would I play a noteOn event with a violin?

For MIDI, it is necessary to send Program Change message to change the instrument. So, you need to call synth.midiProgramChange(channel, 41).

• If you want to use the another bank (rather than 0), use synth.midiBankSelect(channel, bank).
• If you want to use the instrument in the Soundfont and multiple Soundfonts are loaded, use synth.midiProgramSelect(channel, sfontId, bank, instrument) (this is an original feature of fluidsynth).

[chopin2256]

Bingo, that got it to work. Thank you.

When I do my next YouTube video about playback, I will let you know. You might be interested in seeing how my product works, and makes use of this module. I also will demonstrate how Music Jotter works with Kontakt. But if you are generally interested in music composition, feel free to subscribe and spread the word of my small music channel.

[chopin2256]

Also, do you have any tips on how to keep memory usage low? I'm stress testing right now, and memory usage keeps climbing as I add more notes to the score for playback. I do not seem to have this problem with external daws since memory management is handled by the daw.

I don't know if synth.close() must be called each time the player stops (to clean up memory), but I can't get that function to work correctly. When the player stops, and I call close, the notes get stuck.

Sorry, another followup question: is there a way to increase the polyphony with this API?

[chopin2256]

BTW, this is the error I get when I go add notes (stress testing).

fluidsynth: warning: Ringbuffer full, try increasing synth.polyphony!

When I enter notes (and modify notes), I run a series of midiNoteOn events (paired with midiNoteOff events). But this doesn't seem to work well for note entry. The playback still seems to work fine. I have a new synth just for note entry. And another synth object just for playback.

Not sure if I am missing something. When I connect to a soundfont manager such as Coolsoft VirtualMidiSynth, there are no issues. I'm not sure if FluidSynth just has issues with memory management, or if there are settings that can't be accessed via this API.

One solution would be to disallow (turn off) sending midiNoteOn events when placing notes. Not ideal, but I'm not sure how else to overcome the memory problems.

[jet2jet]

is there a way to increase the polyphony with this API?

The second parameter of synth.init is SynthesizerSettings, which has polyphony field. To increase the polyphony, please specify SynthesizerSettings.
(Currently there is no method to change the setting after initialization.)

fluidsynth: warning: Ringbuffer full, try increasing synth.polyphony!

I'm not sure, but it seems that flushing data in voices does not work properly. This may occur when synth.render is not called enough times, so my advice would be to make sure that synth.render is called enough times.

[chopin2256]

Ah, I ignored the render function in your documentation because I wasn't sure what it was or how to use it. Do I simply call synth.render() in the play loop, like this?

synth.hookPlayerMIDIEvents(function (synth, type, event) { 
     synth.render()
})

Also from your docs: render(outBuffer: AudioBuffer | Float32Array[]): void;

What exactly is outBuffer? How do I retrieve that value?

[chopin2256]

Assuming the output buffer is defined here:

synth.createAudioNode(ac, 2048);

Also assuming render gets called in the hookPlayerMIDIEvents loop like this:

synth.render(2048)

If my assumptions are correct, I am getting this error now: Cannot use 'in' operator to search for 'numberOfChannels' in 2048

Edit: Ok figured out what output buffer is. It's this:

outputBuffer = ac.createBuffer(2, 2048, ac.sampleRate);

Only thing left is to figure out where I put render(outputBuffer) in the code. If I place it in the loop, the midi file plays twice as fast! So I am guessing that is not the right placement.

[chopin2256]

Ok I figured out that putting (render) when the player stops is the best way. And running "render" after each note off event should do the trick (upon executing note events). Unfortunately, there is skipping when I stress test (place notes at a fast pace rapid fire).

[jet2jet]

When using synth.createAudioNode, the audio node can be used as follows:

  const node = synth.createAudioNode(ac, 2048); // 2048 is optional
  ac.destination.connect(node);
  // after this, `synth.render` is called automatically (from 'audioprocess' event of audio node)

Note that createAudioNode uses ScriptProcessorNode, which is deprecated now. If you can, AudioWorkletNodeSynthesizer is nice to use (please see README With AudioWorklet section).
AudioWorkletNodeSynthesizer is also rendered automatically when the result of AudioWorkletNodeSynthesizer.createAudioNode is connected to the destination of AudioContext.

[chopin2256]

If I use AudioWorklet, I need access to the midi events.

I got everything to work except I have no idea how to access AudioWorkletGlobalScope. I had to give up for today, but if you can point me into the right direction on how to access this global object, this should be the last thing I need.

[chopin2256]

I found what the issue is. I've been getting this error all along but I've just been ignoring it. However this is why the AudioWorkletGlobalScope isn't being registered:

Uncaught ReferenceError ReferenceError: AudioWorkletGlobalScope is not defined

I load these scripts in the header of my html main template:

    <script src="/mj/jslibs/synth/libfluidsynth-2.3.0.js"></script>
    <script src="/mj/jslibs/synth/js-synthesizer.min.js"></script>
    <script src="/mj/jslibs/synth/js-synthesizer.worklet.js"></script>

And each time I run my project, I get that reference error. I believe if we could pinpoint why there is a reference error, then I can finally access AudioWorkletGlobalScope

[jet2jet]

js-synthesizer.worklet.js can only be used in Audio Worklet. It cannot be loaded via <script> (and is not necessary).

[chopin2256]

Thanks! I am still unable to access AudioWorkletGlobalScope.

I can implement the audioworklet technique for midiNoteOn and midiNoteOff events, but I am unable to implement the technique with playback (since I have to capture midi events during playback). If we are unable to figure this out, I will have to reverse engineer the code one of these days to see whats going on. The global variable is for some reason not being registered correctly.

[jet2jet]

I am still unable to access AudioWorkletGlobalScope.

AudioWorkletGlobalScope can be used only in scripts loaded to Audio Worklet. If you want to add codes working in Audio Worklet, you need to use ac.audioWorklet.addModule.