Mouse Predictor and Shape Classifier

This project allows you to draw shapes with your mouse and then train AI/ML models for shape classification and/or mouse path prediction based on previous positions.

Table of Contents

• Getting Started
  • Requirements
  • Launching
• Start Simulation
  • Settings
    • Predictor Definition
    • Classifier Definition
  • Simulation
    • Shortcuts
    • Recommended for Training
  • Finishing
• Training
  • Settings
    • Data Files
    • Model Parameters
  • Training Process
• Data Viewer
• Manual Operations
  • Training Predictor
  • Training Classifier

Getting Started

Requirements

Install dependencies:

pip install numpy torch customtkinter pygame

Launching

Run launch.py for GUI. You should see the menu:

Start Simulation

Settings

Click Start Simulation. You should see:

You can select multiple Predictors and a single Classifier or none.

Predictor Definition

Predictor models predict the future path of the mouse based on previous positions.

Naming Convention: Laa_bb_ccR-ddR-eeR_desc_f.pth

• L: Indicator it's a Predictor model
• aa: Model input size (number of points (x, y) pairs)
• bb: Model output size (recommended to keep at 1 for recursion)
• cc, dd, ee: Sizes of hidden layers (set to 0 to skip)
• R: Activation function for layer (modifiable via command line)
• desc: Description of what the model is trained on
• f: Flag indicating if data is normalized (N for normalized, U for not)

Classifier Definition

Classifier models classify shapes and display the probability of each shape being currently drawn.

Naming Convention: classifier_aa_bb_ccR-ddR-eeR-desc_f.pth

• classifier: Indicator it's a Classifier model
• aa: Model input size (number of points (x, y) pairs)
• bb: Model output size (number of recognized shapes)
• cc, dd, ee: Sizes of hidden layers (set to 0 to skip)
• R: Activation function for layer (modifiable via command line)
• desc: Description of what the model is trained on
• f: Flag indicating if data is normalized (N for normalized, U for not)

Simulation

After pressing Start, you will see:

You can move your mouse to draw and adjust simulation settings via shortcuts.

Shortcuts

• O: Toggle current prediction
• P: Toggle ghost of last prediction
• N/M: Decrease/increase length of mouse path
• K/L: Decrease/increase length of prediction
• ,/.: Decrease/increase FPS (refresh rate)
• C: Toggle continuous update (update every frame if TRUE, only when mouse moves if FALSE)

Recommended for Training

• Choose a shape (circle, square, etc.) and stick to it.
• Choose a direction (clockwise/anticlockwise) and stick to it.
• The longer you do this, the more data is collected for training.

Finishing

Press ESC to save collected data (mouse positions) to a file for later training.

Training

Settings

Click Start Simulation. You should see:

Data Files

These are files saved from the simulation.

Important: Correct data naming is crucial for the classifier! The name should start with the classified object name followed by an underscore (_). For example: square_somethingsomething.txt. The classifier's output size is determined by the number of unique object names. For example, if you have square_1, square_2, circle_abc_123, circle_1, you will have two unique objects: "square" and "circle".

Model Parameters

• Training Type: Select if you want to train a Predictor Model or Classifier Model.
• Hidden Layers: Select sizes (number of nodes) in each hidden layer.
• Input Size: Number of points (so (x, y) pairs) taken as input.
• Output Size: (Predictor only) Number of points to be predicted by default. Tip: For a small amount of data, keep the output = 1.
• Description: A short description for the file name to track what data the model was trained on.
• Normalize: Toggle data normalization. Instead of inputs being 0 <= x <= 1000 (window size), they will be in the range 0.0 <= x <= 1.0.

Training Process

The training process will run in the console (as a Python process). You will be informed when it finishes:

The model can be used immediately in the simulation.

Data Viewer

The Data Viewer loads all data files and allows you to view them. It shows the number of points stored in each file. Use the left/right arrows to navigate through the files.

Manual Operations

Training Predictor

python train_predictor.py

Arguments:

• --sequence_length: Model input, number of points used for prediction (default: 20).
• --output_size: Number of points to predict (default: 1). Note: If the app displays 1 prediction and the model is trained for 5, it will not show up until you increase the number of displayed predictions.
• --hidden_layers: Configuration of hidden layers (default: "64ReLU-32ReLU").
• --desc: Short description for the model (default: "mix").
• --normalize: Normalize data coordinates to the 0.0-1.0 range (optional).

Example:

python train_predictor.py --sequence_length 20 --output_size 1 --hidden_layers "64ReLU-32ReLU" --desc mix --normalize

Default (Recommended) Command:

python train_predictor.py --sequence_length 20 --output_size 1 --hidden_layers "64ReLU-32ReLU" --desc mix

Training Classifier

python train_classifier.py

Arguments:

• --sequence_length: Model input, number of points used for prediction (default: 20).
• --hidden_layers: Configuration of hidden layers (default: "64ReLU-32ReLU").
• --desc: Short description for the model (default: "CirclesSquares").
• --normalize: Normalize data coordinates to the 0.0-1.0 range (optional).

Example:

python train_classifier.py --sequence_length 20 --hidden_layers "64ReLU-32ReLU" --desc CirclesSquares --normalize

Default (Recommended) Command:

python train_classifier.py --sequence_length 20 --hidden_layers "64ReLU-32ReLU" --desc CirclesSquares