Audio API

[mateli]

Design and implement a audio API.

Playing sound over the network or using a sound card has the shared problem that a high level API is needed. As such controlling the playback of sound is a bit tricky because setting the current tone or frequency to play adds a lot of latency. Instead controlling what sound file to play and probing player status is more efficient. And this is how many widely used sound API:s work.

This is a plan to implement this in multiple steps:

1. Implement enough of FujiNet NOS in the emulator to have a shared filesystem. This is important because shuffling data from file storage to the sound API would be inefficient. To be clear the emulator should EMULATE FujiNet NOS not run it inside the 6502 CPU, like DOSBox emulates dos rather than running it inside the emulator.

2. Use the emulator to create a reference implementation of a NOS sound API. The control of playback has to be good enough for games and not just a music player frontend. This should be as hardware agnostic as possible. Creating and using a variant of the API should work as well on the C64 as on Atari.

3. Create an SDK for adding audio playback to applications and games.

4. Create real FujiNet implementation of this API, with server software that can play audio on any sufficient device.

5. Extend emulator capacity to utilize an actual FujiNet for audio playback. This is probably not possible in a browser but on the desktop it should be fully doable.

[mateli]

To clarify what all of above would bring to the project.

First of all there will be no need to implement any sound hardware emulation. Instead existing sound libraries could be wrapped and sandboxed to only play files withing for example a folder or zip-file where the "NOS filesystem" resides. Of course hardware emulation is still something nice to have but with this API it is not a requirement.

Sound on par with modern operating systems. Having a sound API for playing audio means that a large number of existing and not yet existing audio formats can be played.

An application for this would be a very portable audio player that can run on any machine to which the NOS has been supported.

This can be used to create games where sound work pretty much the same way that it does in a modern game engine. You point the API to a file and tell it to play it. You also get feedback so that you know when it starts playing.

For an emulator an API like this means rapid development and good performance. The system calls to start and control playbacks has a bit overhead compared to for example playing sound in a native game but the main operation here is to play the actual sound and that is handled by a modern native and optimized sound library. On platforms where it is supported playback may be handled by hardware which libraries like ffmpeg will know how to do.

In fact playing sound using this API from inside the VM would add no significant performance cost compared to playing sound from within a native game. In fact it may be faster because the emulator can switch to new and better backend libraries where a native game is stuck to the one it was compiled with.