update#1
Conversation
add real-time keypoint streaming and multi-camera support
Leaderboard
Refactored the game update logic to use a delta time (dt) parameter, ensuring physics and movement calculations are consistent across different frame rates. The main loop now calculates dt based on the time elapsed between frames, improving gameplay smoothness and reliability.
Fixed leaderboard and music together
|
@niiccoo2 is attempting to deploy a commit to the codeprotech's projects Team on Vercel. A member of the Team first needs to authorize it. |
There was a problem hiding this comment.
Pull request overview
This PR represents a major architectural restructuring that transitions the project from a Python Flask backend with vanilla JavaScript frontend to a SvelteKit-based application with Socket.IO for real-time communication. The changes introduce pose detection capabilities using MediaPipe alongside the existing hand tracking functionality, creating a more comprehensive body tracking game experience.
Key Changes:
- Migration to SvelteKit framework with complete project reorganization into svelte/ directory
- Addition of body pose detection using MediaPipe Pose alongside existing hand tracking
- Implementation of Socket.IO server for real-time communication between camera clients and game instances
- Introduction of deployment configurations (Vercel, Docker) and custom Node.js server
Reviewed changes
Copilot reviewed 28 out of 98 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| vercel.json | Added Vercel deployment config (contains incorrect Python configuration) |
| vite.config.ts | Deleted root Vite config (moved to svelte/ directory) |
| tf/recognision.py | Enhanced with pose detection, body keypoint extraction, and restructured output format |
| tf/live_recognision.py | Added MediaPipe Pose initialization |
| templates/*.html | Added new HTML templates for camera connection and streaming |
| svelte/ | Complete new SvelteKit application structure with game logic, Socket.IO integration, and multi-camera support |
| svelte/server.js | Custom Express server with Socket.IO for handling camera sessions and real-time data relay |
| svelte/package.json | Project dependencies including MediaPipe, Socket.IO, and SvelteKit adapters |
Comments suppressed due to low confidence (1)
tf/live_recognision.py:113
- The comment "# hand reco stuff" is too informal and vague. Consider using a more descriptive comment like "# Process hand recognition landmarks" to better explain the purpose of this code block.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "src": "main.py", | ||
| "use": "@vercel/python" |
There was a problem hiding this comment.
The vercel.json configuration specifies Python deployment with "@vercel/python" and references "main.py", but this repository contains TypeScript/JavaScript/SvelteKit code. This configuration appears incorrect for the actual project structure. The deployment should use a Node.js adapter instead.
|
|
||
| # ----- MediaPipe Hands ----- | ||
| mp_hands = mp.solutions.hands | ||
| mp_hands = mp.solutions.hands #type:ignore |
There was a problem hiding this comment.
The type ignore comment is added inline which is less clean. Consider placing the type ignore comment on a separate line above for better readability, or better yet, configure the type checker to properly recognize the mediapipe module.
| def get_body_keypoints(pose_landmarks): | ||
| if not pose_landmarks: | ||
| return None | ||
|
|
||
| lm = pose_landmarks.landmark | ||
|
|
||
| keypoints = { | ||
| "head": { | ||
| "x": lm[mp_pose.PoseLandmark.NOSE].x, | ||
| "y": lm[mp_pose.PoseLandmark.NOSE].y | ||
| }, | ||
| "left_shoulder": { | ||
| "x": lm[mp_pose.PoseLandmark.LEFT_SHOULDER].x, | ||
| "y": lm[mp_pose.PoseLandmark.LEFT_SHOULDER].y | ||
| }, | ||
| "right_shoulder": { | ||
| "x": lm[mp_pose.PoseLandmark.RIGHT_SHOULDER].x, | ||
| "y": lm[mp_pose.PoseLandmark.RIGHT_SHOULDER].y | ||
| }, | ||
| "left_elbow": { | ||
| "x": lm[mp_pose.PoseLandmark.LEFT_ELBOW].x, | ||
| "y": lm[mp_pose.PoseLandmark.LEFT_ELBOW].y | ||
| }, | ||
| "right_elbow": { | ||
| "x": lm[mp_pose.PoseLandmark.RIGHT_ELBOW].x, | ||
| "y": lm[mp_pose.PoseLandmark.RIGHT_ELBOW].y | ||
| } | ||
| } | ||
|
|
||
| return keypoints |
There was a problem hiding this comment.
The newly added function get_body_keypoints lacks a docstring. It should include documentation describing its parameters, return value, and purpose, especially since it's extracting specific pose landmarks from MediaPipe results.
|
|
||
| def main(frame): | ||
| frame = cv2.flip(frame, 1) | ||
| # frame = cv2.flip(frame, 1) |
There was a problem hiding this comment.
The commented-out frame flip code should either be removed if no longer needed or accompanied by a comment explaining why it's kept but disabled. Leaving commented code without explanation reduces maintainability.
| }); | ||
|
|
||
| socket.on('frame', ({ cameraId, blob }) => { | ||
| for (const [session_id, session] of sessions.entries()) { |
There was a problem hiding this comment.
Unused variable session_id.
| import qrcode as qr | ||
| import os as os | ||
| import time | ||
| from threading import Lock |
There was a problem hiding this comment.
Import of 'Lock' is not used.
| from threading import Lock |
| try: | ||
| q.get_nowait() # Drop oldest frame | ||
| q.put_nowait((blob, int(time.time() * 1000))) | ||
| except: |
There was a problem hiding this comment.
Except block directly handles BaseException.
| except: | |
| except (queue.Empty, queue.Full): |
| processed_any = True | ||
|
|
||
| socketio.sleep(0) | ||
| except queue.Empty: |
There was a problem hiding this comment.
'except' clause does nothing but pass and there is no explanatory comment.
| except queue.Empty: | |
| except queue.Empty: | |
| # No frame is currently queued for this session; skip to the next one. |
| try: | ||
| q.get_nowait() # Drop oldest frame | ||
| q.put_nowait((blob, int(time.time() * 1000))) | ||
| except: |
There was a problem hiding this comment.
'except' clause does nothing but pass and there is no explanatory comment.
No description provided.