Adding a reverse function of avro.parse()

[TanimSk]

Details:
Add a reverse parser, for example:

output = avro.reverse_parse("ভালো")
print(output)

Desired output: valo

Reason:
Actually I am building a chatbot for processing Bangla language typed in English alphabets. Cause people nowadays chat in bangla using English alphabets for saving time and efforts. Moreover, adding this functionality will give this project more flexibility.

I highly appreciate that you guys are still maintaining this project.

[hitblast]

Hi there! This seems like a pretty legit idea to work on, so definitely will look into it. Also, if you have any thoughts about making this feature possible then don't hesitate to note it down here :D The code's open so hope that you will also have a look at it to brainstorm a little bit.

Last but not least, <3

[TanimSk]

glad to have a wave back!
I will surely look into the codes 🥂

[hitblast]

Alright 👊 If you're interested, then we can convert this Issue into a GitHub Discussion and then develop the feature together, since it'll definitely require some reverse engineering over the already-existing Avro Dictionary.

[baseplate-admin]

I will surely look into the codes 🥂

lemme know if you run into errors.

The approach i mentioned should work :D

@TanimSk if you want to we can work together on making this a better project ...

[baseplate-admin]

Looking over the codes I have a few questions

@hitblast,

Is it possible not to bundle dependencies such as click and rich and pyclip ?

Say i want to bundle it inside a django project. I dont have a hard dependency on either rich or click. Adding a CLI toolkit into a parser seems a bit useless no?

Suggestions :

• Release a separate package named maybe avro.py-cli
• Freeze the core so that it works with all the python versions

---

Also it seems that you are not using type-hints properly.. I have seen many cases of Any ( which can be strongly typed )

[baseplate-admin]

@TanimSk

Can you add tests with your code?

Also another recommendation

Use reverse_parse() as your function name

---

PR might be preferable in this case because that'd indicate your contribution to this package :D

If he commits directly he can also get his contribution badge

[TanimSk]

@baseplate-admin I will of course add tests. But for now, I wanna focus on the new avro dict construction. I need some time to figure it out.

[hitblast]

@TanimSk Okay so if everything's sorted out, I hope that you won't have much problem developing the new dictionary. Make sure to not override the key:value pairs used for the original parse() function or that could potentially break how avro can parse same texts even with multiple inputs (bk -> বক but bok is also বক). Best of luck 👍 and if you need any assistance, ping me.

If he commits directly he can also get his contribution badge

@baseplate-admin A PR might just be more visible in his activity feed.

[TanimSk]

@hitblast got it. I am working on separate files, so, don't worry about breaking up things ✌️

[hitblast]

@TanimSk Alright! Looks like you're pretty cheered up there. Make sure to sync your fork before you continue to develop on your feature request.

[hitblast]

@TanimSk Let us know about your progress once you're free :P

[TanimSk]

@hitblast Actually I have bunch of projects in this month. I'll work on avro as soon as I get free, btw do you use any social media? I would love to connect with you. You can mail me if you dont wanna make your account public :D

[hitblast]

Ah being public is something I won't hate though. My mail address is given in my GitHub bio, and other social links include my Discord account. You could apparently use Discord to contact me if you're familiar!

[hitblast]

@baseplate-admin @TanimSk Well, I have launched avro.py 0.1.5 ahead of the release of the feature Tanim's currently working on in order to force compatibility with avro.py-cli. I hope we'll have a version 0.1.6 as soon as he's done prototyping and we're done doing fault-checks!

[TanimSk]

okay, so we can do put a key named exceptions

"data": {
        "patterns": [
            {
                "find": "bhl",
                "replace": "ভ্ল"
                "reversed": "bhl"
            },
            ...
        ]
        "exceptions":
            {
                "ফেসবুক": "facebook",
                ...
            }
}

so, the workflow will be,

word = dict.get("ফেসবুক", None)
if not word == None:
   new_word =  iterate_through_dict()
   return new_word
else:
   return word

[baseplate-admin]

okay, so we can do put a key named exceptions

Yes but it should not be in the same JSON. We should split out json file.

In fact i am -1 on using JSON.

Lets just use a python dictionary. Its faster and doesnt need to tokenize stuff

word = dict.get("ফেসবুক")
if not word == None:
  new_word =  iterate_through_dict()
  return new_word
else:
  return word

This is better

word = dict.get("ফেসবুক")
if not word:
   word = iterate_through_dict()
return word

[TanimSk]

umm actually we ARE converting JSON to python dict, not using JSON

avro.py/avro/config.py

Lines 49 to 51 in 908c29b

	# Loads json data from avrodict.json.
	AVRO_DICT = json.load(codecs.open(AVRO_DICT_FILE, encoding='utf-8'))

type(AVRO_DICT)

returns <class 'dict'>

[baseplate-admin]

umm actually we ARE converting JSON to python dict, not using JSON

Thats what we shouldn’t do. If you do inspect on JSON module you will see how many calls it make under the hood.
Why not just use a python dictionary? Whats the benefit JSON is giving us ( JSON is not even a python standard | Marshall is )

Also loading json and converting it to dictionary on every call seems like a very inefficient design choice no?

[hitblast]

@TanimSk @baseplate-admin

• Based on both of your replies, we could hardcode the dictionary into a separate avrodict.py file. This will make the dictionary native to Python's interpreter, increasing performance while adding syntax highlighting to distinguish errors.

• I've come upon a much cleaner approach with the syntax:

import avro

some_random_english = 'bulbuli'

# Eng to Ban
banglified = avro.parse(some_random_english)

# Ban to Eng
reversed = avro.reverse(banglified)

# Singular imports make up for a cleaner code style :D

• I like the 'exceptions' idea, except I think that it should be added based on user's usage data. If you have used an Avro application (iAvro / Avro on Windows) before then you've most probably noticed that Avro saves a copy of the word you chose from Avro's suggestions list, then corrects them when you type them afterwards.

[hitblast]

@TanimSk Based on our previous conversation, should I start working on the "Pythonized" Avro Dictionary? This way you could straight up start working on the newer suggested patterns.

[TanimSk]

Yea sure @hitblast, lmk when you're done

[hitblast]

Alright, thanks for the confirmation from your side.

[TanimSk]

Hi there, @hitblast. Hopefully, all is going well.
I've updated my repo according to the basis of your and @baseplate-admin suggestions. Also have added some rules to enhance the output result :D
The current dictionary structure looks like this:

{
       ...
        [
             {
                "find": "A`",
                "replace": "া",
                "reverse": "a"
            },
            {
                "replace": "আ",
                "reverse": "a"
            },
            ....
       ],

        "exceptions":{
            "ফেসবুক": "Facebook",
            "গুগল": "Google",

        },

        "vowel": "aeiou",
        "consonant": "bcdfghjklmnpqrstvwxyz",
        "casesensitive": "oiudgjnrstyz",
        "number": "0123456789",
        
        # For reverse parsing
        "shorborno": "অআইঈউঊঊএঐওঔ",
        "kar": {'া', 'ি', 'ী', 'ু', 'ূ', 'ৃ', 'ে', 'ৈ', 'ো', 'ৌ'},
        "ignore": {"ঁ", "।", "?", ".", "-", ";"}
}

Some examples:

import avro
print(avro.reverse('তোমার বাসায় সবাই কেমন আছে?'))
print(avro.reverse('আমি যা বলি তাই করি'))
print(avro.reverse('আসসালামু আলাইকুম ভাইজান। কি খবর?'))
print(avro.reverse('তুমি কি খেতে ভালোবাসো?'))
print(avro.reverse("কাকতাড়ুয়া"))
print(avro.reverse("কাঁকন"))
print(avro.reverse("রাহিম, তোমাকে করিম ডাকছে। এখন কি রওনা দেবে?"))
print(avro.reverse('আমার সোনার বাংলা।'))
print(avro.reverse('গুগল'))
print(avro.reverse('ফেসবুক'))

output:

tomar basay sobai kemon ase?
ami za boli tai kori
assalamu alaikum bhaijan. ki khobor?
tumi ki khete bhalobaso?
kaktaruya
kakon
rahim, tomake korim dakse. ekhon ki rowna debe?
amar sonar bangla.
Google
Facebook

Hopefully, you'll sync up soon and have a look at my repo. If everything seems perfect, I will send a PR 🥂

[hitblast]

@baseplate-admin It's actually possible with GitHub Actions as well. I just gotta let the workflow have permissions to write content and remove the --check flag from the command.

[baseplate-admin]

While its possible to use github actions. Your minutes will quickly run out when you have a large number of PR/ a large number of repositories...

Pre-Commit CI offers unlimited minutes for any open source project :)

[hitblast]

@baseplate-admin Hello! Sorry I got busy during exams.

GitHub Actions also has unlimited runtime for public repositories. The limit kicks in at private repositories only.

[baseplate-admin]

Hi there,

I also got busy due to exam .

Now back to topic at hand.

I recommended https://pre-commit.ci for 3 reasons.

1. Its a lot faster than GitHub Actions.
2. Its interoperable with Github Actions.
3. Its cross language and open source :D

You dont need to completely go in one route or another. Going pre-commit means you can not only get a code checker but also you can run those using github actions.

[hitblast]

@baseplate-admin Alright, I will look into it.

[hitblast]

Well it's been long since the feature has been implemented and released, so I guess I'll close this discussion here.