Non-uniquely decodable word lists and "" separator can lead to issues

[sts10]

Passphraseme allows users to not use a word separator with --sep ""; for example passphraseme -sw --sep="" might give: "givenopportunityionreportedrimslukeshowed".

Passphraseme also allows users to use the EFF fandom lists, which are not uniquely decodable. For example, "light", "saber" and "lightsaber" are all on the Star Wars list. This can be an issue if a user runs passphraseme -sw --sep="" 5, expecting a 5-word passphrase and all the entropy that entails (about 60 bits), but instead gets "helplessyachtdecemberlightsaber", which is effectively a 4-word passphrase, giving only about 48 bits of entropy (again, this is because "lightsaber" is a single word on the list).

The EFF long and short lists avoid this issue because they do not have prefix words: "We also ensured that no word is an exact prefix of any other word." If we removed all prefix words from the fandom lists, we'd remove "light", since it's a prefix word of "lightsaber", and would avoid the issue described above.

Is this a serious, real-world issue?

Honestly, I've never been confident about a mathematical procedure to, given a not uniquely decodable list, calculate the actual chance of one of these issues actually happening. The issue also assumes that the attacker knows the word list that you used to create your passphrase and is brute-forcing through all possible 1-word passphrases, 2-word passphrases, 3-word passphrases, 4-word passphrases (where they would guess "helplessyachtdecemberlightsaber"!), etc..

But I've always thought there was good reason for EFF to remove prefix words on the long and short non-fandom lists. (And for 1Password's passphrase generator to require word separators.) tbh I'm kind of disappointed EFF didn't do the same for the fandom lists.

Possible solutions for Passphraseme

1. Force users to use a separator of some kind (in other words, do not accept a separator of ""), as 1Password's passhrase generator does.
2. Capitalize the first letter of each word if --sep is set to "". These capitalized letters effectively act as word separators ("LightSaber" vs. "Lightsaber").
3. Make all word lists uniquely decodable, either by removing all prefix words or another method (I have a preferred method). This solution has the disadvantage of making combining word lists, as requested in [Feature Request] Is it possible to add option for generating passphrases using different built-in wordlists? #18, complicated, as combining 2 or more uniquely decodable word lists will very likely result in a list that is NOT uniquely decodable.