Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FOSS music trackers packaging effort #81815

Open
20 of 86 tasks
fgaz opened this issue Mar 5, 2020 · 29 comments
Open
20 of 86 tasks

FOSS music trackers packaging effort #81815

fgaz opened this issue Mar 5, 2020 · 29 comments
Assignees
Labels
0.kind: packaging request Request for a new package to be added 2.status: work-in-progress This PR isn't done 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems

Comments

@fgaz
Copy link
Member

fgaz commented Mar 5, 2020

I recently took an interest in the world of tracker music and discovered that there are a lot of FOSS trackers, but that packages for them are scattered across distros (Debian has quite a lot though).

So i decided to package all of them for NixOS :D

This issue tracks myour packaging effort. Suggestions are welcome!

In no particolar order (name - {Repology, Homepage, pull request #} - additional details):

Related stuff which we could maybe package

There are definitely historical, long unmaintained, no longer used trackers in this list that probably belong to a NUR. Still, it'd be nice to have as much as possible in nixpkgs.

Resources
@OPNA2608
Copy link
Contributor

OPNA2608 commented Mar 5, 2020

  • bambootracker

I have been sitting on a derivation for BambooTracker since the big Qt infrastructure awhile ago, been meaning to open a PR for it once the ever-approaching 0.4 release is out. I can open a PR for the current state later today, you can give it a look if you want.

  • 0cc-famitracker

FT and any fork thereof is Windows-only, though some forks improve Wine support by alot. I don't think packaging Wine-based applications is something we do in nixpkgs.
In any case, you should give j0CC-FamiTracker a look instead for your personal use.

  • schismtracker - was already in nixpkgs, needs a .desktop file

I still have an update PR outstanding for schismtracker, can look into adding a .desktop file while I'm at it. #76725

Something that's missing from your list would be CheeseCutter, which is stuck waiting for GDC to get packaged (#69144 needs to get merged first) - I tried building it with the other Dlang compilers already in nixpkgs but it relies on GDC's support of legacy syntax to compile.

@deliciouslytyped
Copy link
Contributor

Tangentially related, if anyone is interested in this issue, they may also be interested in the various JACK related issues.

@fgaz
Copy link
Member Author

fgaz commented Mar 5, 2020

@OPNA2608 thanks for the interest! I added your prs and cheesecutter to the list

FT and any fork thereof is Windows-only

oops, I thought I filtered out all of those, thanks

@OPNA2608
Copy link
Contributor

OPNA2608 commented Apr 8, 2020

  • deflemask?

While it's a very popular chiptune tracker, DefleMask is closed-source and far from FOSS-happy. See the VICE emulator's Hall of Shame section on DefleMask & reSID. The latest distributed non-reSID binary could be patched & wrapped but I get abit of a stomach ache thinking about it.

@fgaz
Copy link
Member Author

fgaz commented Apr 13, 2020

That wasn't intended, thanks again for catching it! I removed it and added a bunch more from other repos and some nice resources I found (1 2 3 4).

37 trackers! That's a lot!

@OPNA2608
Copy link
Contributor

  • bambootracker

Merged! Can finally be ticked off the list. 😄


  • snestracker { - H -}

Quite new, and I'm talking to the dev about what can be done to improve packaging. It does some awful things:

  • Compiling, including and unconditionally expecting unrar at runtime
  • In-tree tools with unclear/conflicting licensing
  • Some path detecting problems making it necessary for the binaries to reside in $out/share/snestracker/

Once these problems are solved, I plan to open a PR.

@fgaz
Copy link
Member Author

fgaz commented Jul 31, 2020

Great! I Updated the list

Compiling, including and unconditionally expecting unrar at runtime

ow. that's unfree... what does it need it for?
The project looks cool though

@OPNA2608
Copy link
Contributor

OPNA2608 commented Jul 31, 2020

Really only for convenience, apparently packs of SPC files (SNES music code) are usually in the RAR format. I have open issues for those problems:

@fgaz
Copy link
Member Author

fgaz commented Aug 19, 2020

@OPNA2608 should I also ping you in my prs?

@OPNA2608
Copy link
Contributor

Feel free to do so!

@OPNA2608
Copy link
Contributor

OPNA2608 commented Sep 8, 2020

Can you do me a favour and put

[ ] Fix macOS build

under CheeseCutter? I have been meaning to tackle that for awhile but never got around to test it properly and keep forgetting about it.

There's an updated commit on the repo that introduces Darwin-related compilation fixes and the SDL loading module requires some patching on our side to handle our non-standard path straight to the dylib, as opposed to an SDL.framework in the Library.

I'll hopefully remember it and make a PR later this week.

@fgaz
Copy link
Member Author

fgaz commented Sep 8, 2020

Done!
I wonder if there's a way to let you edit the OP too

@OPNA2608
Copy link
Contributor

OPNA2608 commented Sep 8, 2020

I don't think there is.

For reference, a failed darwin evaluation on Hydra: https://hydra.nixos.org/build/126193708

@OPNA2608
Copy link
Contributor

OPNA2608 commented Oct 14, 2020

Talked to the developer abit to fix some installation problems over at yuxshao/ptcollab#3, will make a PR after testing the online part with some friends.

@fgaz
Copy link
Member Author

fgaz commented Oct 23, 2020

Added to the list 👍

@fgaz
Copy link
Member Author

fgaz commented Jan 2, 2021

I found another great resource: https://wiki.thingsandstuff.org/Tracker

@dukzcry
Copy link

dukzcry commented Feb 2, 2021

@OPNA2608
I really liked BambooTracker for these features:

  1. Modern interface
  2. MIDI keyboard support and jam mode
  3. FM synthesis

What other trackers with same features would you recommend? 3rd point is not necessary, since its quite niche

@fgaz
Copy link
Member Author

fgaz commented Feb 2, 2021

@dukzcry if I remember correctly, of those packaged soundtracker has the modern interface and midi support, and klystrack has all three but in limited form

@fgaz
Copy link
Member Author

fgaz commented Feb 17, 2021

I merged all resources and there are ~60 entries now... There's a lot of cool stuff in there! (and also a fair amount of abandoned projects)

@OPNA2608
Copy link
Contributor

  • exotracker {- H -}

I remembered this was still open. I hacked together a Darwin build, derivation needs cleanup & some more lib.optional* usage to not break Linux building. Has some problems like missing graphics on my end that may be a problem with my system, or with the packaging. But at least it builds & doesn't crash, so it's an improvement I guess? 😅.

Screenshot_macOS_2021-04-17_22:32:20

Don't have the time rn to flip between machines and fix everything up, but I'll review a PR if you clean it up & CC me.

Clickme for patch
@@ -3,6 +3,7
 , fetchFromGitHub
 , pkg-config
 , makeWrapper
+, xcbuildHook
 , SDL
 , SDL_image
 , SDL_ttf
@@ -24,34 +25,63 stdenv.mkDerivation rec {
   nativeBuildInputs = [
     pkg-config
     makeWrapper
-  ];
+  ]
+  ++ lib.optional stdenv.hostPlatform.isDarwin xcbuildHook;
 
   buildInputs = [
     SDL
     SDL_image
     SDL_ttf
-    gtk2
-    glib
+    # gtk2
+    # glib
   ];
 
-  makeFlags = [
-    "-C sdl"
-    "-f Makefile.linux"
-    PREFIX=$(out)"
-  ];
+  # makeFlags = [
+  #   "-C sdl"
+  #   "-f Makefile.linux"
+  #   PREFIX=$(out)"
+  # ];
 
   # TODO: try to exclude gtk and glib from darwin builds
   NIX_CFLAGS_COMPILE = [
-    "-I${SDL}/include/SDL"
+    "-I${SDL.dev}/include/SDL"
     "-I${SDL_image}/include/SDL"
     "-I${SDL_ttf}/include/SDL"
-    "-I${gtk2.dev}/include/gtk-2.0"
-    "-I${glib.dev}/include/glib-2.0"
+    # "-I${gtk2.dev}/include/gtk-2.0"
+    # "-I${glib.dev}/include/glib-2.0"
   ];
 
+  preBuild = ''
+    cd sdl/osx/xcode
+    # xcode project expects SDL Frameworks for linking, we only have dylibs
+    substituteInPlace Hivelytracker.xcodeproj/project.pbxproj \
+      --replace 'lastKnownFileType = wrapper.framework; name = SDL.framework; path = /Library/Frameworks/SDL.framework' 'lastKnownFileType = "compiled.mach-o.dylib"; name = libSDL.dylib; path = ${SDL}/lib/libSDL.dylib' \
+      --replace 'lastKnownFileType = wrapper.framework; name = SDL_image.framework; path = /Library/Frameworks/SDL_image.framework' 'lastKnownFileType = "compiled.mach-o.dylib"; name = libSDL_image.dylib; path = ${SDL_image}/lib/libSDL_image.dylib' \
+      --replace 'lastKnownFileType = wrapper.framework; name = SDL_ttf.framework; path = /Library/Frameworks/SDL_ttf.framework' 'lastKnownFileType = "compiled.mach-o.dylib"; name = libSDL_ttf.dylib; path = ${SDL_ttf}/lib/libSDL_ttf.dylib'
+    substituteInPlace ../../../hvl2wav/Makefile \
+      --replace 'gcc' '${stdenv.cc.targetPrefix}cc'
+  '';
+
   # Also build the hvl2wav tool
   postBuild = ''
-    make -C hvl2wav
+    make -C ../../../hvl2wav
+  '';
+
+  installPhase = lib.optionalString stdenv.hostPlatform.isDarwin ''
+    runHook preInstall
+    install -Dm755 Products/Release/Hivelytracker.app/Contents/MacOS/Hivelytracker $out/lib/Hivelytracker
+    cp -R Products/Release/Hivelytracker.app/Contents/Resources/* $out/lib/hivelytracker
+    # Crashes if fonts are not in ttf/ subdirectory
+    mkdir $out/lib/hivelytracker/ttf
+    mv $out/lib/hivelytracker/{*.,}ttf
+    mkdir $out/bin
+    runHook postInstall
   '';
 
   postInstall = ''
@@ -59,11 +89,11 @@ stdenv.mkDerivation rec {
     # Ideally we should patch the sources, but the program can't open
     # files passed as arguments anyway, so this works well enough until the
     # issue is fixed.
-    wrapProgram $out/bin/hivelytracker \
-      --run "cd $out/share/hivelytracker"
+    makeWrapper $out/lib/hivelytracker/Hivelytracker $out/bin/hivelytracker \
+      --run "cd $out/lib/hivelytracker"
 
     # Also install the hvl2wav tool
-    install -Dm755 hvl2wav/hvl2wav $out/bin/hvl2wav
+    install -Dm755 ../../../hvl2wav/hvl2wav $out/bin/hvl2wav
   '';
 
   meta = with lib; {
@@ -83,7 +113,6 @@ stdenv.mkDerivation rec {
     license = licenses.bsd3;
     platforms = platforms.all;
     maintainers = with maintainers; [ fgaz ];
-    broken = stdenv.isDarwin; # TODO: try to use xcbuild
   };
 }

FT and any fork thereof is Windows-only

oops, I thought I filtered out all of those, thanks

Actually, nesicide includes a version of FamiTracker that's cross-platform, via an MFC-over-Qt compatibility layer. It has some problems but it can launch and produce some sound. I haven't tried packaging the whole IDE yet though.

grafik

@OPNA2608
Copy link
Contributor

I picked afew projects from the list that I've heard of before / seemed interesting and hacked-together overlays:

Chibitracker

Requires some patching to build for x86_64-linux, builds fine out-of-the-box for i686-linux (via pkgsi686Linux.chibitracker).

package-chibitracker.txt

grafik

i686 versions works mostly fine, only crashes when exiting. x86_64 can't load a module most of the time and crashes with a bus error. It's an interesting little project but it seems abandoned and abit broken, not sure if it makes sense to include this one.

AdlibTracker2

x86_64 support requires replacing x86-exclusive inline assembly with equivalent x86_64 assembly… 😕

package-adlibtracker2.txt

Compiles on x86_64-linux but crashes immediately. This may be "normal" however because I heard that the AUR's package (which installs the prebuilt Linux version) also crashes for some people in a similar fashion. Not sure what's up with that.

  Sample buffer size: 1024 samples (requested 2048)
  Sampling rate: 49716 Hz (requested 49716)
Reading configuration file ... not found!
An unhandled exception occurred at $00000000004B200E:
EAccessViolation: Access violation
  $00000000004B200E

i686-linux doesn't build because i686 fpc fails to build for me. The project might still be active.

Arkostracker2

The project is still binary-only for now but I tried to package it regardless.

package-arkostracker2.txt

It "builds" but fails on me when it tries to draw via libGL. Maybe this works on an actual NixOS machine.

bt1cn@pcclu01:/dev/shm$ nixGL ./result/bin/ArkosTracker2 
./result/bin/ArkosTracker2() [0x7c8ce7]
./result/bin/ArkosTracker2() [0x70c009]
./result/bin/ArkosTracker2() [0x7bab0d]
/nix/store/sbbifs2ykc05inws26203h0xwcadnf0l-glibc-2.32-46/lib/libpthread.so.0(+0x13700) [0x7f17fd936700]
/nix/store/hpnazqkp4zbm2pkp1xj4whmld2zhwv0r-libglvnd-1.3.3/lib/libGLX.so.0(+0xa415) [0x7f17fd30b415]
/nix/store/hpnazqkp4zbm2pkp1xj4whmld2zhwv0r-libglvnd-1.3.3/lib/libGLX.so.0(glXChooseVisual+0x13) [0x7f17fd304633]
./result/bin/ArkosTracker2() [0xa67d16]
./result/bin/ArkosTracker2() [0x9620d7]
./result/bin/ArkosTracker2() [0x9a5147]
./result/bin/ArkosTracker2() [0x6c4f53]
./result/bin/ArkosTracker2() [0x70cf1b]
./result/bin/ArkosTracker2() [0x85984a]
./result/bin/ArkosTracker2() [0x4e8b1b]
/nix/store/sbbifs2ykc05inws26203h0xwcadnf0l-glibc-2.32-46/lib/libc.so.6(__libc_start_main+0xed) [0x7f17fd3c9ded]
./result/bin/ArkosTracker2() [0x4eace9]
/home/bt1cn/.nix-profile/bin/nixGL: line 6:  8477 Killed                  "$@"

Last release was on March 21, 2021 so it's very much in development still.

psycle

The qpsycle project isn't available on their SVN repository anymore. There is a new cpsycle rewrite that's in active development but it only has a build script for Windows. psycle-core etc are still available and buildable on Linux, albeit with some fixes.

package-psycle.txt

This is only a command-line-only player however, and so far I've been unable to get any sound out of it. Psycle modules embed the names of the plugins they need… with a DLL extension. I don't know if that affects the plugin loading. I saw in strace that it also tries to find a config.xml at some point. ALSA also doesn't seem to work & causes an abortion.

1tracker

I gave this a quick attempt. I think it's trying to test something by writing to the nix store at runtime - which obviously fails - so it instantly errors. I'll try to look into this more. I think it's still active.

package-1tracker.txt

(Excuse the monitor photo, any button press will close the window)
IMG_20210622_134625


ZorroTracker is a very new, WIP and active project (since there's some minor financial backing behind it) but it's Electron-based. I have no clue how to build that from source and gave up on it.

@fgaz
Copy link
Member Author

fgaz commented Jun 23, 2021

Wow, nice job!

AdlibTracker2

I also tried building it but gave up shortly after. Does the cross32 makefile help?


1tracker

I tried stracing it and it looks like it's missing some files: $out/lib/1tracker/engines/{test.1te,list,hash}.

If you add test.1te (even just by copying one of the other engines) it starts, but fails if you try to select a different engine.

If you write the list of engines in the list file, the selection works and you can make some sounds. I have yet to figure out the specific format though. I tried one filename per line and only the first entry seems to work.

I have no idea what the hash file does.

EDIT:
By looking at the readme, it seems like those files are caches, which explains the error
EDIT2:
yup. when running it from a writable directory, I get "updating engines"... and apparently better graphics too?!?
EDIT3:
okay, the cache files are a biit more complicated than I thought:
list.txt
hash.txt

EDIT4, screenshots:

In the store:
2021-06-23-135844_1366x768_scrot

In the writable src directory:
2021-06-23-135556_1366x768_scrot
2021-06-23-135601_1366x768_scrot

the different graphics are probabaly caused by 1tracker.cfg

@OPNA2608
Copy link
Contributor

OPNA2608 commented Jun 23, 2021

okay, the cache files are a biit more complicated than I thought:

.1te files seem to be C/C++ source code, with a function void Info(void) that prints the title & about stuff from list (i.e. ZX|1BIT|AntEater etc). Maybe we could write a bash script that scans and collects the engines' filename, title and about texts to build the list file.

hash looks to be all the file names, truncated / left-aligned and padded to 8 characters & appended, with a \0-terminator.

Launching & changing engines with in-store version

grafik
grafik
grafik

  installPhase = ''
    runHook preInstall

    mkdir -p $out/{bin,lib/1tracker}

    mv colors/ docs/ engines/ examples/ fonts/ 1tracker $out/lib/1tracker/
    ln -s $out/{lib/1tracker,bin}/1tracker

    # Write list and hash files for the tracker
    pushd $out/lib/1tracker/engines
    for engine in *; do
      echo $engine
      # Ignore non-engines for list, but not for hash
      if [[ "$engine" == *.1te ]]; then
        echo $engine >> list
        grep 'SetTitle' $engine | cut -d'"' -f2 >> list
        grep 'SetAbout' $engine | cut -d'"' -f2 | sed -e 's/\\n/\n/g' >> list
        echo "###" >> list
      fi  
      printf '%-8s' ''${engine:0:8} >> hash
    done
    printf '\0' >> hash
    ln -s ./{anteater,test}.1te
    popd

    runHook postInstall
  ''; 

I think we can fix the cfg path problem by patching in SDL_GetPrefPath.

@fgaz
Copy link
Member Author

fgaz commented Jun 23, 2021

.1te files seem to be C/C++ source code

angelscript actually! (which looks very nice, I might use it for some of my own projects...)
so we could use the angelscript package in nixpkgs to run them

@fgaz
Copy link
Member Author

fgaz commented Jun 23, 2021

Though the best long-term solution is probably to ask the author to use XDG for cache and configuration (and maybe to pick up additional engines). Should we contact them?

@OPNA2608
Copy link
Contributor

angelscript actually! (which looks very nice, I might use it for some of my own projects...)
so we could use the angelscript package in nixpkgs to run them

I think devendoring angelscript would be a good idea if it's feasible, yeah.

Should we contact them?

We can certainly ask! 😄


I patched in SDL_GetPrefPath and wrote a wrapper that copies the example cfg to the appropriate place if it's missing. It should work "from the store" now. LMK what you think.

package-1tracker.txt

@fgaz
Copy link
Member Author

fgaz commented Jan 24, 2024

https://github.com/djdiskmachine/LittleGPTracker is a LGPT fork with a more active maintainer. I may switch the package to it if djdiskmachine/LittleGPTracker#71 is resolved.

P.S.: wow, 2 years since the last tracker addition! Time flies! I need to finish that tiatracker pr...

@ScottFreeCode
Copy link

If I wanted to get QTFamitracker (or any other Linux-compatible Famitracker fork, if there are any) in, how involved is that likely to be? I've never packaged a QT app for Nix before, but I've done a little bit of C building in Nix and used to use C and C++ for whole projects back in the day (Nix can't be more of a slog than CMake's multiple steps and interminable build time).

@fgaz
Copy link
Member Author

fgaz commented Feb 23, 2024

@ScottFreeCode Qt isn't too bad, see the relevant section in the manual. qmake or cmake is used automatically if you put it in nativeBuildInputs.

@fgaz fgaz added the 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems label Jul 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: packaging request Request for a new package to be added 2.status: work-in-progress This PR isn't done 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems
Projects
None yet
Development

No branches or pull requests

5 participants