Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update with changes to main #38

Open
wants to merge 134 commits into
base: 23-unit-test-exercise-test-framework
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
134 commits
Select commit Hold shift + click to select a range
c9f139e
moved gui scripts into new branch
Mast3rwaf1z Apr 20, 2022
e4dc918
got a lot of the gui ready to be used with stepper
Mast3rwaf1z Apr 20, 2022
65e22da
moved progress
Mast3rwaf1z Apr 20, 2022
3b75dc6
moved progress from other branch
Mast3rwaf1z Apr 20, 2022
a2cce40
the final code from today
Mast3rwaf1z Apr 20, 2022
f090b8c
resized posix elements
Mast3rwaf1z Apr 21, 2022
0183565
fixed end of script issue
Mast3rwaf1z Apr 26, 2022
cc112f6
fixed commit issue
Mast3rwaf1z Apr 26, 2022
303c9b0
added some data to devices in overlay
Mast3rwaf1z Apr 26, 2022
556e377
started displaying actual data in overlay
Mast3rwaf1z Apr 27, 2022
a932681
moved device object and added some fixes to window size
Mast3rwaf1z Apr 28, 2022
9812fd4
added table module and implemented it in gui
Mast3rwaf1z Apr 29, 2022
9c8daf5
started working on placing device representations correctly
Mast3rwaf1z May 1, 2022
b2023bb
finished placement in exercise_overlay
Mast3rwaf1z May 1, 2022
60e3176
redefined stuff
Mast3rwaf1z May 1, 2022
570724d
finished adding tables and tidying data printing
Mast3rwaf1z May 1, 2022
35f914b
Merge branch '22-gui-for-stepping' of github.com:petergjoel/Distribut…
Mast3rwaf1z May 1, 2022
ca2778f
added status label
Mast3rwaf1z May 1, 2022
5a58c02
merge stuff
Mast3rwaf1z May 2, 2022
906fc55
merge complete
Mast3rwaf1z May 2, 2022
621fa20
added new stuff in stepper
Mast3rwaf1z May 2, 2022
cb48444
fixed listener issue, added more illustrations to gui
Mast3rwaf1z May 9, 2022
22d4a18
fixed content field sizing issue
Mast3rwaf1z May 11, 2022
7b07b17
Merge branch '22-gui-for-stepping' of github.com:petergjoel/Distribut…
Mast3rwaf1z May 11, 2022
0ece677
fixed content field sizing issue
Mast3rwaf1z May 11, 2022
fc0e9a8
added receiving lines to gui and added controls page in gui
Mast3rwaf1z May 12, 2022
6835119
work from last time...
Mast3rwaf1z May 21, 2022
6d93e20
made a more general stepping emulator
Mast3rwaf1z May 21, 2022
0d9fd11
pick command draft
Mast3rwaf1z May 21, 2022
a679496
progress on pick command
Mast3rwaf1z May 21, 2022
49ddb43
changed default emulator to be sync, added a switch between async and…
Mast3rwaf1z Jun 18, 2022
7f26091
added installation section
Mast3rwaf1z Jun 18, 2022
820a195
added a last note about python version
Mast3rwaf1z Jun 18, 2022
1159bc0
added text about stepper and gui
Mast3rwaf1z Jun 18, 2022
be41d3d
added figure to show gui
Mast3rwaf1z Jun 18, 2022
0974fa1
worked on select command
Mast3rwaf1z Jun 20, 2022
2563e4c
finally found the issue with stepper
Mast3rwaf1z Jun 24, 2022
c239853
added a few comments, got pick working and renamed some stuff
Mast3rwaf1z Jun 25, 2022
21a1041
merged with GUI
Mast3rwaf1z Jul 2, 2022
3c4eb9d
finished creating a GUI version of the pick command, might wanna make…
Mast3rwaf1z Jul 3, 2022
05389e4
added gui queue command
Mast3rwaf1z Jul 4, 2022
e512da1
rewrote the GUI in PyQt6, need to test it in windows, so i'm taking d…
Mast3rwaf1z Jul 15, 2022
1bcb381
forgot lines, changed it to colors to show it better
Mast3rwaf1z Jul 15, 2022
529d7f5
merged with new PyQt gui
Mast3rwaf1z Jul 15, 2022
91ff359
updated readme
Mast3rwaf1z Jul 15, 2022
5d0b311
updated figure
Mast3rwaf1z Jul 15, 2022
4bba26b
updated controls
Mast3rwaf1z Jul 16, 2022
96eabe3
fixed the bug, it wasn't a bug really, it was some misplaced printing
Mast3rwaf1z Jul 19, 2022
8c472de
Update README.md
michelealbano Jul 25, 2022
83b6f83
Update README.md
michelealbano Jul 25, 2022
efe1461
added requirements.txt and updated readme
Mast3rwaf1z Jul 31, 2022
9e27726
added a combo box
Mast3rwaf1z Aug 4, 2022
bcddc92
made the pick command dequeue the chosen message correctly
Mast3rwaf1z Aug 6, 2022
8404be4
finished pick gui
Mast3rwaf1z Aug 6, 2022
4c44cf0
made improvements to pick command
Mast3rwaf1z Aug 6, 2022
220feb5
fixed a bug where the wrong devices would try to receive the ping packet
Mast3rwaf1z Aug 9, 2022
1843799
cleaned up the stepper and fixed the last few bugs
Mast3rwaf1z Aug 17, 2022
b5b2b16
cleaned up the stepping emulator and fixed the last few bugs, its bee…
Mast3rwaf1z Aug 17, 2022
b2ce30f
Merge pull request #27 from DEIS-Tools/22-gui-for-stepping
michelealbano Sep 7, 2022
1521939
Merge pull request #28 from DEIS-Tools/26-extend-readme
michelealbano Sep 7, 2022
3df5df0
fixed the pick bug, added coloring to the terminal output and impleme…
Mast3rwaf1z Sep 15, 2022
3b33e5f
fixed windows printing
Mast3rwaf1z Sep 16, 2022
a3dacfe
new printing and new runner layout
Mast3rwaf1z Sep 16, 2022
5394b1c
updated terminal interactivity to not trigger in the same way as the gui
Mast3rwaf1z Sep 16, 2022
e72d2cb
changed stepper to a barrier, no idea why i hadn't considered doing t…
Mast3rwaf1z Sep 16, 2022
3999af3
forgot to change shell to use the barrier, added a lock to avoid comm…
Mast3rwaf1z Sep 17, 2022
19db8ba
Fixed few typos
sdfg610 Sep 22, 2022
ff8e1b7
Merge pull request #32 from sdfg610/patch-1
michelealbano Sep 22, 2022
b8f44f2
Added a notice to "Exercise 2, Task 1"
sdfg610 Sep 23, 2022
c8044ec
Merge pull request #33 from sdfg610/patch-2
michelealbano Sep 23, 2022
56d4c9a
added a switch for cli and gui instead of running both simultaneously
Mast3rwaf1z Sep 27, 2022
5b2effd
changed some static variables to not be static, caused the gui to not…
Mast3rwaf1z Sep 27, 2022
fb10260
placed the buttons list in exercise overlay wrong when moving static …
Mast3rwaf1z Sep 27, 2022
e338a91
fixed an issue where the program hangs if run with --gui flag and not…
Mast3rwaf1z Sep 29, 2022
30c6219
Updated Exercise 5
sdfg610 Oct 11, 2022
99b89d2
Merge pull request #35 from sdfg610/patch-3
michelealbano Oct 11, 2022
7b0aa4e
Update README.md
sdfg610 Oct 18, 2022
b66e528
Updates on exercise 7
sdfg610 Oct 26, 2022
e51964b
Merge pull request #36 from sdfg610/patch-4
michelealbano Oct 27, 2022
e0bd9c4
Updated exercise descriptions. Updated exercise 11 code
sdfg610 Oct 28, 2022
80fe7df
Re-fixed my fix in exercise 11
sdfg610 Oct 28, 2022
cd2deba
Small cleanup in exercise 10 code
sdfg610 Oct 30, 2022
1667de5
Added code on miner to receive transactions from client
sdfg610 Oct 30, 2022
a73d245
Merge pull request #37 from sdfg610/patch-4
michelealbano Oct 31, 2022
c3986c2
Merge pull request #30 from DEIS-Tools/ExtensionGUI_update
michelealbano Oct 31, 2022
87b90f5
Many small, assorted updates to exercise 8 code
sdfg610 Nov 1, 2022
3320c2f
Merge pull request #40 from sdfg610/patch-4
michelealbano Nov 1, 2022
37ace76
Update README.md
michelealbano Nov 1, 2022
9c7e24f
Small fix in exercise 8
sdfg610 Nov 3, 2022
e03a57d
Merge pull request #41 from sdfg610/patch-4
michelealbano Nov 3, 2022
58ad2d5
Updates to exercise 9 descriptions and code
sdfg610 Nov 7, 2022
43dd888
Send "Done" to master, not client
sdfg610 Nov 7, 2022
142d1eb
Added constants for client_index and master_index for readability
sdfg610 Nov 7, 2022
29be634
Merge pull request #42 from sdfg610/patch-4
michelealbano Nov 8, 2022
97f164a
Few minor updates to exercise 11
sdfg610 Nov 14, 2022
90018c1
Updates in ex. 11 code. Fixed default algorithm for ex. 11
sdfg610 Nov 14, 2022
81b804c
Additional guiding/helping type-hints
sdfg610 Nov 14, 2022
92312d3
Merge pull request #43 from sdfg610/patch-4
michelealbano Nov 16, 2022
9688182
Small additional update to Exercise 11 code
sdfg610 Nov 17, 2022
4abd1bf
Updates to Exercise 12 description and code
sdfg610 Nov 21, 2022
d73c4db
Small fix to my updates
sdfg610 Nov 21, 2022
4d8a82f
Merge pull request #44 from sdfg610/patch-4
michelealbano Nov 22, 2022
c5a4f1b
Update README.md
t-lohse Sep 7, 2023
1755553
Merge pull request #45 from t-lohse/patch-1
michelealbano Sep 7, 2023
1fd2d9f
micheles fix for shell.start() bug
Chri692u Sep 7, 2023
e708239
Docs setup
Chri692u Sep 7, 2023
754cafe
Documentation generation + index file
Chri692u Sep 7, 2023
1fad675
Update .gitignore
Chri692u Sep 7, 2023
f93511e
Create conda environment
andreasalstrup Sep 7, 2023
0063594
Update README.md
andreasalstrup Sep 7, 2023
e87fc57
Merge pull request #47 from Chri692u/main
michelealbano Sep 8, 2023
b114d8f
Merge branch 'DEIS-Tools:main' into main
andreasalstrup Sep 8, 2023
8863186
Update README.md
michelealbano Sep 14, 2023
284caa4
Formatted with black and fixed ruff linting errors.
Laursen79 Nov 2, 2023
62e8444
Now runs in python 3.12
Laursen79 Nov 2, 2023
46fa108
Converted getter methods to properties. Fixed ruff linting errors.
Laursen79 Nov 2, 2023
8cfdfa3
Updated README.md
Laursen79 Nov 2, 2023
0bd974c
refactored exercise_runner_overlay (#1)
MJHC Nov 2, 2023
d7f445d
Converted methods to properties, formatted using black and fixed ruff…
Laursen79 Nov 23, 2023
a0566b0
Merge pull request #48 from andreasalstrup/main
michelealbano Oct 14, 2024
b75983c
Merge branch 'main' into py312
michelealbano Oct 14, 2024
00a44ac
Merge pull request #49 from Laursen79/py312
michelealbano Oct 14, 2024
95c4ad5
Update README.md
michelealbano Oct 24, 2024
04c7334
Update README.md
michelealbano Nov 5, 2024
67d16cd
Added and improved comments for exercise 11
Casper-NS Nov 8, 2024
a88867c
Merge pull request #50 from Casper-NS/exercise11-update
michelealbano Nov 8, 2024
1386b80
Proposed a new exercise for the 12. lecture
Casper-NS Nov 12, 2024
055e741
simplified the exercise to only simulate a node failure
Casper-NS Nov 13, 2024
911959a
Merge pull request #51 from Casper-NS/update-exercise-12
michelealbano Nov 13, 2024
86f2060
Added a new exercise for lecture 8
Casper-NS Nov 26, 2024
adb830e
Merge pull request #52 from Casper-NS/exercise8-update
michelealbano Nov 27, 2024
8ffb74b
Update README.md
michelealbano Nov 28, 2024
d66806a
Update README.md
michelealbano Nov 28, 2024
a30cd88
Update README.md
michelealbano Nov 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -69,9 +69,6 @@ instance/
# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/
Expand Down Expand Up @@ -139,4 +136,4 @@ dmypy.json
cython_debug/

# notes, remove at merge
.notes/
.notes/
231 changes: 171 additions & 60 deletions README.md

Large diffs are not rendered by default.

14 changes: 14 additions & 0 deletions conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import os
import sys

sys.path.insert(0, os.path.abspath("."))

extensions = [
"sphinx.ext.autodoc",
]

autodoc_modules = {
"Device": "./emulators/Device.py",
"Medium": "./emulators/Medium.py",
"MessageStub": "./emulators/MessageStub.py",
}
27 changes: 27 additions & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
DistributedExercisesAAU docs
=============================

.. toctree::
:maxdepth: 2
:caption: Contents:

Device
------

.. automodule:: emulators.Device
:members:
:undoc-members:

Medium
------

.. automodule:: emulators.Medium
:members:
:undoc-members:

MessageStub
-----------

.. automodule:: emulators.MessageStub
:members:
:undoc-members:
71 changes: 46 additions & 25 deletions emulators/AsyncEmulator.py
Original file line number Diff line number Diff line change
@@ -1,74 +1,95 @@
import copy
import random
import threading
import time
from threading import Lock
from typing import Optional
from os import name

from emulators.EmulatorStub import EmulatorStub
from emulators.MessageStub import MessageStub

if name == "posix":
RESET = "\u001B[0m"
CYAN = "\u001B[36m"
GREEN = "\u001B[32m"
else:
RESET = ""
CYAN = ""
GREEN = ""

class AsyncEmulator(EmulatorStub):

class AsyncEmulator(EmulatorStub):
def __init__(self, number_of_devices: int, kind):
super().__init__(number_of_devices, kind)
self._terminated = 0
self._messages = {}
self._messages: dict[int, list[MessageStub]] = {}
self._messages_sent = 0

def run(self):
self._progress.acquire()
self._start_threads()
self._progress.release()

# make sure the round_lock is locked initially
while True:
time.sleep(0.1)
self._progress.acquire()
# check if everyone terminated
if self.all_terminated():
if self.all_terminated:
break
self._progress.release()
for t in self._threads:
t.join()
return

def queue(self, message: MessageStub):
self._progress.acquire()
def queue(self, message: MessageStub, stepper=False):
if not stepper:
self._progress.acquire()
self._messages_sent += 1
print(f'\tSend {message}')
print(f"\r\t{GREEN}Send{RESET} {message}")
if message.destination not in self._messages:
self._messages[message.destination] = []
self._messages[message.destination].append(copy.deepcopy(message)) # avoid accidental memory sharing
random.shuffle(self._messages[message.destination]) # shuffle to emulate changes in order
time.sleep(random.uniform(0.01, 0.1)) # try to obfuscate delays and emulate network delays
self._progress.release()
self._messages[message.destination].append(
copy.deepcopy(message)
) # avoid accidental memory sharing
random.shuffle(
self._messages[message.destination]
) # shuffle to emulate changes in order
time.sleep(
random.uniform(0.01, 0.1)
) # try to obfuscate delays and emulate network delays
if not stepper:
self._progress.release()

def dequeue(self, index: int) -> Optional[MessageStub]:
self._progress.acquire()
def dequeue(self, index: int, stepper=False) -> Optional[MessageStub]:
if not stepper:
self._progress.acquire()
if index not in self._messages:
self._progress.release()
if not stepper:
self._progress.release()
return None
elif len(self._messages[index]) == 0:
self._progress.release()
if not stepper:
self._progress.release()
return None
else:
m = self._messages[index].pop()
print(f'\tRecieve {m}')
self._progress.release()
print(f"\r\t{GREEN}Recieve{RESET} {m}")
if not stepper:
self._progress.release()
return m

def done(self, index: int):
time.sleep(random.uniform(0.01, 0.1)) # try to obfuscate delays and emulate network delays
time.sleep(
random.uniform(0.01, 0.1)
) # try to obfuscate delays and emulate network delays
return


def print_statistics(self):
print(f'\tTotal {self._messages_sent} messages')
print(f'\tAverage {self._messages_sent/len(self._devices)} messages/device')
print(f"\t{GREEN}Total{RESET} {self._messages_sent} messages")
print(
f"\t{GREEN}Average{RESET} {self._messages_sent/len(self._devices)} messages/device"
)

def terminated(self, index:int):
def terminated(self, index: int):
self._progress.acquire()
self._terminated += 1
self._progress.release()
self._progress.release()
57 changes: 52 additions & 5 deletions emulators/Device.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,24 +5,68 @@


class Device:
"""
Base class representing a device in a simulation.

Args:
index (int): The unique identifier for the device.
number_of_devices (int): The total number of devices in the simulation.
medium (Medium): The communication medium used by the devices.

Attributes:
_id (int): The unique identifier for the device.
_medium (Medium): The communication medium used by the device.
_number_of_devices (int): The total number of devices in the simulation.
_finished (bool): A flag indicating if the device has finished its task.
"""

def __init__(self, index: int, number_of_devices: int, medium: Medium):
self._id = index
self._medium = medium
self._number_of_devices = number_of_devices
self._finished = False

def run(self):
"""
Abstract method representing the main functionality of the device.
"""

raise NotImplementedError("You have to implement a run-method!")

def print_result(self):
"""
Abstract method for the result printer
"""
raise NotImplementedError("You have to implement a result printer!")

@property
def index(self):
"""
The unique identifier for the device.

Returns:
int: The unique identifier of the device.
"""
return self._id

@property
def number_of_devices(self):
"""
Get the total number of devices in the simulation.

Returns:
int: The total number of devices.
"""
return self._number_of_devices

@property
def medium(self):
"""
Get the communication medium used by the device.

Returns:
Medium: The communication medium.
"""
return self._medium


Expand All @@ -34,9 +78,12 @@ def __init__(self, index: int, number_of_devices: int, medium: Medium):
super().__init__(index, number_of_devices, medium)
self._has_work = False

@property
def has_work(self) -> bool:
# The random call emulates that a concurrent process asked for the
self._has_work = self._has_work or random.randint(0, self.number_of_devices()) == self.index()
self._has_work = (
self._has_work or random.randint(0, self.number_of_devices) == self.index
)
return self._has_work

def do_work(self):
Expand All @@ -45,19 +92,19 @@ def do_work(self):
# might require continued interaction. The "working" thread would then notify our Requester class back
# when the mutex is done being used.
self._lock.acquire()
print(f'Device {self.index()} has started working')
print(f"Device {self.index} has started working")
self._concurrent_workers += 1
if self._concurrent_workers > 1:
self._lock.release()
raise Exception("More than one concurrent worker!")
self._lock.release()

assert (self.has_work())
assert self.has_work
amount_of_work = random.randint(1, 4)
for i in range(0, amount_of_work):
self.medium().wait_for_next_round()
self.medium.wait_for_next_round()
self._lock.acquire()
print(f'Device {self.index()} has ended working')
print(f"Device {self.index} has ended working")
self._concurrent_workers -= 1
self._lock.release()
self._has_work = False
26 changes: 14 additions & 12 deletions emulators/EmulatorStub.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,19 @@


class EmulatorStub:

def __init__(self, number_of_devices: int, kind):
self._nids = number_of_devices
self._devices = []
self._threads = []
self._media = []
self._progress = threading.Lock()

for index in self.ids():
for index in self.ids:
self._media.append(Medium(index, self))
self._devices.append(kind(index, number_of_devices, self._media[-1]))
self._threads.append(threading.Thread(target=self._run_thread, args=[index]))
self._threads.append(
threading.Thread(target=self._run_thread, args=[index])
)

def _run_thread(self, index: int):
self._devices[index].run()
Expand All @@ -30,14 +31,15 @@ def _run_thread(self, index: int):
def _start_threads(self):
cpy = self._threads.copy()
random.shuffle(cpy)
print('Starting Threads')
print("Starting Threads")
for thread in cpy:
thread.start()

@property
def all_terminated(self) -> bool:
return all([not self._threads[x].is_alive()
for x in self.ids()])
return all([not self._threads[x].is_alive() for x in self.ids])

@property
def ids(self):
return range(0, self._nids)

Expand All @@ -46,19 +48,19 @@ def print_result(self):
d.print_result()

def run(self):
raise NotImplementedError(f'Please contact the instructor')
raise NotImplementedError("Please contact the instructor")

def queue(self, message: MessageStub):
raise NotImplementedError(f'Please contact the instructor')
raise NotImplementedError("Please contact the instructor")

def dequeue(self, id) -> MessageStub:
raise NotImplementedError(f'Please contact the instructor')
raise NotImplementedError("Please contact the instructor")

def done(self, id):
raise NotImplementedError(f'Please contact the instructor')
raise NotImplementedError("Please contact the instructor")

def print_statistics(self):
raise NotImplementedError(f'Please contact the instructor')
raise NotImplementedError("Please contact the instructor")

def terminated(self, index: int):
raise NotImplementedError(f'Please contact the instructor')
raise NotImplementedError("Please contact the instructor")
Loading