LANraragi v.0.6.7.EX -- No-Fun Naysayers

This is a quick hotfix to 0.6.7!
The Docker images will be tagged as 0.6.7.EX, but the Windows Installer will remain at 0.6.7. Therefore, you might have to uninstall the Windows version manually in order to be able to update.

Super quick Changelog

• Invalidate search cache when uploading archives from the web uploader since the Background Worker doesn't pick up the slack in that case
• Fix /api/page to not require an API key in No-Fun mode.
• Fix my Github actions workflows so hopefully this time I won't have to upload the MSI installer myself dagnabit dammit

LANraragi v.0.6.7 -- Crystal Japan

“Japan” sounds unlike anything else from Scary Monsters, too—it’s far more in line with earlier ambient pieces like “Moss Garden.”

This is a more laid-back release from the craziness of October -- Bunch of bugfixes and a good ol' revamp to WSL.

New Windows Installer

Biggest feature in this release is a revamp of how Windows/WSL builds are made, bringing in a quick 'n dirty installer made with WixSharp.
Upgrades from the old script installers should be painless, with no need to run the uninstallation script.
Feedback is most welcome! I can probably improve a bit the stray cmd window that pops at the end to register WSL.

Next up

I probably won't make another release for 2019 (dumb bugfixes notwithstanding), so uh, merry christmas! hohoho sounds in the distance
For the next year I'm probably slowing down a bit on development; I feel the software is in a pretty good place right now feature and speed-wise, and I have a few other projects I'd like to attend to.

I still have some big jobs lined up for 2020/0.7.0 (cough categories cough plugins 3.0), which I'll probably slowly get done throughout the year alongside misc issues.

Christmas Changelog

• (#144) (#196) Full revamp of the Windows builds using a WiX Installer

  🎅 This is the big one! The Windows builds have moved to a Windows Server runner in order to dynamically build the C# bootstrapper and Setup. I've also cleaned up the bootstrapper a bit, which should hopefully fix some issues people were having.

• (#232) Beef up the eze plugin

  🤶 eze-backed archives can now have their title and source directly referenced from the json file. This is a plugin setting, so don't forget to check it beforehand if you're interested!

• Remove workaround for Librera in OPDS API

  📚 Librera wasn't fond of application/x-cbz mimetypes, so I had added a workaround for it with a second button/mimetype. I fixed the issue myself and the matching version was released, so I can now remove said workaround.

• Add a few environment variables to help deployment

  ⚙ Namely, LRR_TEMP_DIRECTORY, LRR_LOG_DIRECTORY and LRR_FORCE_DEBUG. I added those for Homebrew and WSL respectively, but there's nothing stopping you from using them on your own.

• Add Console.app support to the macOS version

  🖥 Logs now make use of the aforementioned env variable to print themselves in a location macOS' Console.app can pick up. It's a bit overkill since the webapp has its own log view, but it's nice to have.

• Extracting an archive no longer returns direct links

  🖼 Due to the now-possible temp folder override, I could no longer simply spit out direct links to /public and call it a day. I've added instead an (undocumented) API that properly serves files from extracted archives.
  🧨 This shouldn't break existing API clients. (Thanks for notifying me when it could have!)

• Use the Background Worker's dynamic filemap to prune old IDs when performing searches

  🎄 This is a very niche issue that would happen if you modified the contents of an archive without changing its filepath, leading to duplicates showing up in the index.
  🌟 I've added some smarts to the Search Engine so it communicates with the Background Worker to know the exact ID of a file on the filesystem, avoiding this issue.

• Save to disk a bit more aggressively for Docker/WSL setups

  💾 The default redis.conf bundled in Docker images used default saving parameters, which are fine for caching but not so much when you made exotic design choices like me and use Redis as a main database.
  😏 Saving now occurs every second if at least one key changed.

• Bugfixes to server-side search

  🌟 Added a few more cache busters in places I forgot about (namely when removing the new flag on an archive)
  ✨ Some regex characters weren't being escaped properly.
  🌠 I was expecting the new search to be an endless source of bugs and hell but it's been...pretty smooth so far!

• (#234) Fix the hypnotoad.pid bug again (WSL edition)

  🎃 Nightmare before Christmas! This was fixed in 0.6.6 on other builds.

• (#218) Fix a bug where the background worker would go astray under Hypnotoad

  🐸 I swear to god if I see this frog one more time in my bug list

• Other miscellaneous junk

  🎁 Fix crash if provided an empty id in batch tagging
  🎁 Refactor tests so they don't need a Redis server to run anymore
  🎁 Don't reset search field when toggling new
  🎁 Fix bug where waiting for file to be openable would stay stuck if file was deleted

LANraragi v.0.6.6 -- Scary Monsters (and Super Searches)

It’s Bowie’s most aggressive straight-up rock track since “Diamond Dogs.”

This is actually...a big release! It's terrifying! More than 100 commits of pure mind-bending horror!

Happy Halloween!

Hacktoberfest Has Ended

As you can see from the commit list, this was a huge October! A big thank-you to everyone who contributed.
I received the LRR stickers(Pi Zero for scale) and proceeded to plaster them on my dev machine:

Everyone who filled in the form will receive 'em by post in the oncoming weeks.
Since I have some left, I'm leaving the form open for future would-be contributors. ✌

Introducing Search 2.0

🛸 You've entered the LANraragi Twilight Zone! Where search is actually good and the JSON cache is no more! 🛸

Search used to be made on the client side, leading to your web browser having to load the entire archive list, preprocessed by the server into the JSON cache. Said cache took a while to build, and said building happened frequently if you edited your metadata a lot.

This has always been a bit of a band-aid to speed up the old code that was building the index, which I've proceeded to tear off and replace by a proper search system, backed by the search API.

You can now search the archive index using the same modifiers you'd use on E-Hentai. And it's fast! that's the scariest part!

Homebrew Support! (Preview)

Steve Jobs has risen from the grave and he's out for my manga collection!
He's bringing in casks from Oktoberfest...except they're filled with blood! 🍻🍻🩸🩸🩸

Thanks to user Bl4Cc4t giving me some motivation, I've remixed LRR a bit to allow for macOS machines to run it again. This allowed the creation of a homebrew package, allowing macOS(and Linux) machines to easily install LRR with a simple brew install lanraragi.

The Homebrew releases are a bit wet behind the ears, but you can start using them now! 0.6.7 will bring them to a fully stable state.

Please check the Documentation for more detail.

OPDS Catalog Support

🧙‍♂️ The people have clamored for external reader support... So I obliged, by wrangling ancient dark XML magics to raise a software army from the depths of the Play Store! 🧟‍♀️🧟‍♂️🧟‍♀️🧟‍♂️🧟‍♀️🧟‍♂️

OPDS stands not for Overtly Powerful Demonic Summoning, but for Open Publication Distribution System. This standard is used mostly by ebook stores to provide catalogs that can be easily parsed by reader software.

A fair bit of readers also support OPDS for zips and rars however, and this is where I come in.
Using the new OPDS API, you can now consume your LANraragi archives on a bevy of external readers on Android, iOS and Windows.

Please check the Documentation for more detail.

Spooky Changelog

• (#114) More terrifying QoL fixes

  🎃 Clicking on a gallery's tag now fills the search box with namespace:tag instead of just tag.
  🩸 Overflow text on the batch tagging checklist is now hidden with ellipses.

• (#80) (#120) (#154) The JSON cache has been thoroughly murdered

  👻 Search API is now used to power the archive index as a whole.
  ⚗ The index is now much more reactive to database changes and will no longer show you outdated data.

• (#140) Fix the infamous Shinobu crashes in Docker for good

  ⚰ This issue actually contained multiple bugs linked to Hypnotoad and Redis.

• (#151) Add a metadata plugin for Koromo copy

  👁 Koromo is a korean downloader with support for a bunch of sources. I wasn't planning on doing this one at all but someone did, so we're all good!

• (#158) Restore macOS compatibility by introducing a fallback file watcher

  🦇 inotify will still be used in WSL/Linux, so this change should not introduce any performance regressions.
  💉 The inotify implementation itself has changed however, so if you see any weird issues like uploaded archives not appearing until a restart, make sure to report it!

• (#218) Fix Shinobu not reporting its status properly post-restart

  👿 Hypnotoad is slowly growing to be my new worst nemesis.

• (#221) Introduce support for Homebrew packages/casks

  🧛‍♀️ The cask will hopefully move to the main homebrew tap once it's been stabilized to a fixed release!
  🧛‍♂️ I'd like to add support for macOS's Console.app log viewer in 0.6.7 as well. Might as well use OS-specific goodies when we can.

• Introduce more unit tests

  💀 Knowing my own reticence to unit tests, I wonder if I haven't been manipulated by some form of eldritch entity to pull that one off. Hopefully the trend will stay!

• Chop up the Docker image to bits

  🔪 The image size has been reduced after being bloated up a bit by multi-architecture builds, falling back to 60 MBs!
  🧠 The surgery went well and shouldn't hamper direct updates.

LANraragi v.0.6.5 -- Art Decade

"Art Decade" is the most Eno-esque track on Low, as Eno assembled much of the piece while Bowie was away from the studio on legal business (though Bowie has sole songwriting credit*).

This release is almost all content contributed by others! I didn't think I'd be able to say that one day.

Hacktoberfest Incoming

Hacktoberfest 2019 is now on, so I've taken some time to highlight a few issues in the Volunteers milestone for would-be participants.
https://github.com/Difegue/LANraragi/issues?q=is%3Aissue+is%3Aopen+label%3Ahacktoberfest

I expected and got 0 contributions last year, but now that this project has picked up a bit more steam, maybe I'll get one.
Making a PR even for updating the documentation counts, so do keep that in mind if you want to get another open source shirt you'll never wear or something.

(You can also just open a new repo and make 4 PRs in it changing the readme and it'll count but you didn't hear that from me)

Full Changelog

• [EXPERIMENTAL] (#80) Introducing the Search API

  🔍 This API features an advanced searching syntax similar to the one from E-H.
  ⚗ Search is new and probably prone to errors or overall slowness for this release. The goal is to eventually use it to power the main page's search, getting rid of the JSON cache alongside it.

• Fix issues in batch tagging caused by the websocket connection dying

  ⏲ I'd naively removed the timeout increase in the previous release with the new messaging model, but since some plugins do take a bit longer than the default timeout, I've bumped it back up to 80 seconds. Should be enough.

• Fix various issues with tag autocompletion

  💿 Notably the fact that favorite tags totally weren't working well with it.

• (#165) Using .RAR archives now outputs a warning

  🤐 .rar support in libarchive is severely lacking and while it should work okayish for RAR2/RAR3, you might get weird errors at times like parsing filters unsupported or blank images. I didn't want to axe .rar support entirely though, so for now the app will simply warn you when opening one.
  🚫 The RAR5 format is entirely unsupported by libarchive and will now output a meaningful error and if you try opening one in LRR.

• (#198) The Docker container now automatically creates a volume if the content folder isn't specified

  💣 Doing this beforehand would simply crash the container.

• (#194) Update the systemd service example file to use the LRR_NETWORK environment variable

• (#172/#182) Document a few caveats/pitfalls with Docker SSL and Veracrypt on Windows

  👁‍🗨 This stuff certainly wouldn't be caught if not for the chivalrous testers making github issues. Thanks!

• Stop searching for expunged galleries with the E-H plugin to avoid false positives

• The "Series" column now accepts "Parody" tags again.

LANraragi v.0.6.4 -- Speed of Life

As structured as “Life” is, there’s still a sense of flow and improvisation under it all, from the various ways Davis plays his brief fills to how the synthesizer line begins to break out of its established patterns in the final chorus repeat.

Just a quick bugfix release with a changelog that rapidly spiraled out of control. Simply classic.
Tune in next release as I hopefully (maybe) get rid of the JSON cache system.

Full Changelog

• (#80) Searching will now show dynamic tag suggestions based on what's in your database

  🧠 Thanks to the power of the tag cloud API, your most used tags are now reused to give you suggestions when searching.
  🥇 The more archives use a tag, the higher it appears in the suggestions. So get taggin'!

• Implement logfile rotation

  💾 Logfiles will now be moved to a compressed backup once they go past 2MB. This allows for quicker loading of the logs view while still allowing you to go as far back as you want if you really need to.

• Fix a few issues caused by the increased multithreading in Hypnotoad

  🐸 All that speed caught up with me. If you had any issues with thumbnails or the Background Worker, this is certainly a release for you.

• (#153) Switch batch tagging model from a big json to one message-per-archive

  ⚙ This fixes batch tagging not working if you tried to batch over 6000 archives at once.

• (#166) Add buttons to wipe the database and remove unused data

  🧽 'cause sometimes you need to start over. You can find both options in the settings page.

• Other QoL Fixes

  • Add number of tags added by AutoPlugin to the Web Upload screen
  • Version themes/javascript to avoid the web interface being broken on new versions until you Ctrl-F5
  • Don't show tags that only appear once in the tag cloud

LANraragi v.0.6.3 -- Neuköln

On “Neuköln,” though, Bowie wanted to go avant-garde, but the limits of his technique and of his allotted time compromise the power of his playing—it’s startling but has no real depth.

Also known as 0.6.0-BETA.3! I'll be switching up the version naming in the next release to 6.0.0 outright, since at the rate I'm going I don't think I'll ever make it to 1.0.0 here.
So yeah, have a simpler version name for now.

I thought this release would be small, but looking at the full changelog certainly made me reconsider! I got a sizeable amount of fixes from external contributors this time around, which is very much appreciated.

Speaking of contributors, I'd like to mention here that I opened up an official chat room thing, so that you don't have to write your woes in a random 4chan thread I'll likely miss out on. 👀

Also hey, I'm getting a Win10 Desktop Client! It's not super ultra polished yet, but it's certainly worth checking out.

Full Changelog

• (#161) Shipped Docker images are now compatible with ARM devices

  🥧 I can't believe there's no raspberry emoji yet
  🐋 Raspberry Pi/ODroid/whatever users can now pull the latest and nightly images, just like x86 PC users! This admittedly took me a bit too long to make happen but you can thank GitHub Actions v2.

• Thumbnail resolution doubled to 550px wide

  📷 It's retina time! This doesn't apply to previously generated thumbnails, so if you want to HD-fy everything I recommend you delete the thumb folder in your content folder.

• (#114) Index Enhancements

  🖱 The Index now features a context menu for various functions such as editing metadata, deleting a file, downloading it, etc. Just right-click on an archive to try it out!
  🏷 Tag display in thumbnail mode has been slightly modified to show a preview of the tags like in list view.
  🖼 Landscape covers are now shown cropped in thumbnail mode so they have the same height as portrait covers.
  👑 Per-browser reading progress is now shown in the Index if you have bookmarking enabled in reader options.
  🖌 Various CSS enhancements have been applied to make for a more pleasant browsing experience.

• (#114) Reader Fixes

  🖲 When quickly flipping through pages in the reader, the image won't select itself anymore.
  📶 The reader now won't load all images in an archive until you open up the overview.

• Switch Mojolicious server implementation to Hypnotoad

  🐸 This should make the server a bit snappier under heavy load scenarios.
  🐞 The dev-server still uses Morbo due to its auto-reload capabilities.

• API Changes

  💽 The /api/backup endpoint can now be used to generate backup JSONs without using the web UI. You can also use the npm run backup-db command if you don't fancy APIs.
  🛑 You can now stop the Shinobu background worker without restarting it using the endpoint /api/stop_shinobu. This'll be API only -- I don't plan on adding a button for it in the web interface.
  🧹 /api/clear_new now allows you to clear New status for a single archive by passing an ID parameter.
  😶 API methods also now return proper HTTP error codes. Sorry about that.

• E-Hentai plugin now fetches category and original url

  💲 Support for the star cookie has also been added if you used the power of money to go through the recent exhentai woes.

• Other bugfixes and improvements

  • Move version number and title to package.json entirely and stop relying on lrr.conf
  • Version checking for update nagging in Index is now smart
  • The isnew field in the database now uses true/false instead of block/none
  • (#173) Take into account plugin Result when tallying up errors in Auto-Plugin
  • (#145) Fix unicode folders inside archives not extracting properly
  • (#149) Properly use WSS in batch tagging if we're in a SSL context
  • (#141) Fix JSON escaping in backup/restore
  • (#140) Better error check in web uploads
  • (#148) Listen to less inotify events with background worker
  • (#164) Handle null IDs caused by upload errors properly

LANraragi v.0.6.0 Beta 2 -- Moss Garden

“Moss Garden” came about through a similar clash of intentions, with Bowie playing chaos (having drawn a card that read “destroy everything”), Eno, order (“change nothing and continue with immaculate consistency”).

Welcome back for round 2 of the 0.6 betas! This is mostly bugfixes as expected, but you might appreciate the bigger new things.

Windows builds are back! (#131)

While I've given up on providing a source port, WSL has proven itself to be a solid alternative to the dumb virtual machines of Docker/Vagrant Windows users were subjected to.
As such, I'm introducing new builds for Windows users, featuring a small friendly GUI and a wrapped WSL distro. You can find it in the Assets section right at the bottom of this huge Release post. 👇

You can read the newly written documentation for more details. I don't think it's ever been easier to install the software on Windows, so hopefully more people who want to try it are now able to.
(If you want a more detailed technical breakdown, I am currently writing one here!)

Plugins 2.0 have arrived

Typed arguments! More convenient settings! Custom icons!
I've massively overhauled the plugin system behind the scenes to provide better usage and error reporting, alongside less code and bugs.
This doesn't really bring anything new for end-users at the moment; I'd planned to open up support for generic "script" plugins that could basically do other stuff besides adding metadata to a file, but decided against it as the feature was too messy and didn't really bring much to the table.

About the user survey

I've received way more replies to the survey that I thought I would, which makes me feel pretty happy, not gonna lie. 50+ users isn't Hydrus tier yet, but it's pretty good!

I plan to provide a detailed breakdown of the survey when it ends in a blogpost, so please look forward to it.

Next Up

I'll spend Beta 3 on overhauling the Web Interface, alongside probably providing extended support for the new WSL wrapper GUI. And maybe a little extra if time allows it.

Full Changelog

• (#102) Plugin system 2.0 has landed

  🧩 Plugin arguments are now typed, with 3 possible types: integers, strings and booleans.
  🖼 The plugin setting page has been updated to take advantage of these types, and display a small base64 icon if present in the plugin metadata.
  💣 I've also went ahead and got rid of the obscure [Object object] errors when running plugins.

• More Plugin changes

  🐼 I've removed login/password access to ExHentai, since it often gets blocked by CAPTCHA requests. The cookie id/hash combo is used by all other programs who parse E-H, so no hard feelings I hope.
  🎁 As a way to make up, the E-H plugin now adds the matching gallery URL to the tags it fetches, under the source namespace. If you want to, it can also change the saved title to the one it finds.
  🚗 I've fully swapped the Chaika plugin to use its API for everything, which should give you better results. It can also do the title thing now if you want to.

• (#123) Rolled back Vagrant to use Virtualbox shared folders

  🗃 SMB seemed like a good idea at the time but it's a bit too faulty for most people.
  ⚠ Due to the new WSL GUI wrapper, I'm sunsetting support for Vagrant as an installation method. It's still perfectly usable and documented, but I'd rather not maintain a billion different ways to install LRR.

• (#117) Rename extracted folders to absolutely safe encoded variants to avoid breakage with poorly encoded archives with subfolders

  🈚 I've accepted that I probably won't ever be completely done with filename encoding issues. True suffering. I never modify archives, but messing with the extracted folders is alright in my book since those are all going to temporary folders.

• (#124/#125) Fix Docker builds modifying permissions of uploaded files

  🐳 Talking about never modifying archives, yeah. I really didn't mean it tho

• (#129) Various fixes following Beta 1's decision to ditch filename encodings

  🉑 This was a good move overall but there was some leftover code to change, leading to some bugs.

• (#114) Remove namespaces in shortened list view and colorize some frequently-seen namespaces

• (#118) Add series to tags that aren't counted in find_untagged_archives (mostly used in Batch tagging)

• (#121) Use full, unparsed filename for incorrectly parsed titles instead of "Please edit metadata"

• (#122) Add a more careful inotify watch to the temp folder to ensure it really follows the max size setting.

• (#115/#127) Fix partial file extraction for plugin API if the file was unicode or in a subfolder of the archive

• (#128) Add a bit more defaults to the tag blacklist

• Fix weird edge case for chaika archives that have an archive registered but 404'd on their matching gallery

LANraragi v.0.6.0 Beta 1 -- Sense of Doubt

Hey there! I'm making 0.6.0 a slightly more involved release than usual.

0.5.9-EX had some real issues going on, so I want to make the next release as stable as it can be, while still adding some more stuff on top.

As such, I'm chopping up 0.6 into three betas:

• Beta 1 "Sense of Doubt", what you're getting now
• Beta 2 "Moss Garden", featuring Plugin System 2.0
• Beta 3 "Neuköln", featuring...a surprise!

You can expect about a month between each beta, but this isn't a hard deadline.

User Survey ongoing

Before moving on to the changelog, I'd also like to gather some info from all users as to what they'd like to see in future releases.
I don't put metrics/telemetry of any kind in the software itself, so I have to rely on old-timey methods such as polls. If you have 2 minutes to spare(it's just checking 4 boxes), it'd be much appreciated !

(Big!)Changelog

• (#82) Uncheck archives as they get processed in batch tagging sessions

  ❎ This should make resuming a batch session easier in case it crashes -- successfully processed archives will be unchecked from the list.
  🧈 I also went ahead and added a progress bar to batch sessions to make them easier to follow.

• (#95) Increase connection timeout to 300 for batch tagging

  ⏲ This should fix issues people are having with batch tagging sessions dying if you set a timeout too long.

• (#99) Make Docker container step down from root

  🐳 Just some extra security for you people who care. If you encounter permission issues after updating make sure to check the Docker documentation. Windows and Vagrant users shouldn't be affected.

• (#106) Switch default autotagger namespace to series:

  🤖 Series tags inferred from the filename will no longer be assigned to parody:. Just an old leftover.

• (#106) Add magazine: to potential tags that can show up in Series

  📓 This is a fallback namespace and will be overridden if a tag with parody: or series: is present.

• (#100) Move Tag Cloud calculation and untagged archive filter to dedicated API entrypoints

  ⚙ Some more API sugar for both you 'n me, pardner. I plan to eventually use the tag cloud API to provide tag suggestions when doing a search. And so can you, if you're the one person making a LRR app! (thanks a lot)
  🔮 The untagged filter is more for internal use, but I opened it up anyways in case someone finds a use for it. Make sure to check the API documentation!

• Switch Vagrant folder mapping solution from VirtualBox shared folders to SMB shares

  💾 I went over the whole Vagrant method recently, which I hadn't done for well, a year and a half. SMB shares are much more stable and easier to set up than VBox folders, but it comes at the expense of dropping Linux support for Vagrant.
  🐧 I think Linux users have better solutions anyway, so I hope I'm not doing too much harm here. Documentation has been updated as well!

-BUGFIX LINE STARTS HERE-

• (#101) Ensure the temp dir is created before extracting thumbnails

• (#107) Fix index CSS in chrome

• (#110) Fix files added through in-app upload not being processed properly by Shinobu

• Fix E-Hentai plugin searches with no results leading to a plugin crash

LANraragi v.0.5.9-EX -- Panda Purgatory

This release fixes the E-H(due to the layout changing again for Minimal+) and Chaika plugins... and that's about it.
Hopefully I won't have to do a 0.5.9-EX2 just because tenboro added more javascript.

I really need to make plugin sideloading better/easier to get to in order to avoid making those dumb releases.

LANraragi v.0.5.9 -- Loving the Panda

Believing the strangest things

Can you imagine a world where metadata providers actually have stable APIs and proper deprecation notices?
Yeah me neither.

This is a very speedy boi release with little testing, but as there haven't been any core changes since 0.5.8 it should be rather stable.

Changelog

• Fixed the E-Hentai plugin to account for the website redesign

  💥 f u c k y o u

• Fixed an error that could occur with the new Background Worker and new archive parsing

  🤯 This actually shows how resilient the new worker parsing is, since you can actually end up with broken data in the end JSON and I wasn't accounting at all for this kind of stuff before!

• (#92) Added SHA-1 reverse image search to the Chaika plugin

  🖼 This is a feature of the chaika API I forgot to implement in the first version, so here we go now.
  ⚒ I also fixed some issues with the text search that might lead to some archives being unsearchable.

• (#94) Improve Batch Tagging filter to pre-check galleries which only have artist/series/event tags

  🏷 This should allow users to quickly batch tag archives that have a few tags already in thanks to filename autoparsing.
  🤝 Thanks to @koma-cute for the PR!

• (#97) Timestamp plugin

  🕰 This is a simple plugin that adds a tag with a unix timestamp in the "date_added" namespace.
  🤝 Thanks to @Utazukin for the PR!