Skip to content

Feature/establish bounding box #6423

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Oct 16, 2025
Merged

Feature/establish bounding box #6423

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
a160a69
merge
benjaminpkane Oct 7, 2025
a7601d4
Merge branch 'feat/human-annotation' into ha-sidebar
benjaminpkane Oct 7, 2025
9bbf820
sidebar work
benjaminpkane Oct 10, 2025
c8979c4
merge
benjaminpkane Oct 10, 2025
4fcf1af
fix bounding box
benjaminpkane Oct 10, 2025
5a977cb
keep image pixelated
Oct 10, 2025
a61547d
kinda sorta workin
Oct 13, 2025
e7220e6
establish bounds for new bounding box
Oct 14, 2025
738612d
merged in feat/human-annotation
Oct 14, 2025
cdd7e26
resolved merge conflicts
Oct 14, 2025
f38aff1
establish bounds when scaled
Oct 15, 2025
1656b3d
Merge branch 'feat/human-annotation' into feature/establish-bounding-box
Oct 15, 2025
b1cc2c2
realign with feat/human-annotation
Oct 15, 2025
136fbcb
ensure we have a label before editing
Oct 15, 2025
a046f95
Merge branch 'feat/human-annotation' into feature/establish-bounding-box
Oct 15, 2025
951e979
fix scrim when bounds off scene
Oct 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 11 additions & 12 deletions app/packages/core/src/components/Modal/Sidebar/Annotate/Edit/AnnotationSchema.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,21 +120,20 @@ const useSchema = () => {

const useHandleChanges = () => {
return useRecoilCallback(
({ snapshot }) =>
async (currentField: string, path: string, data) => {
const expanded = await snapshot.getPromise(expandPath(currentField));
const schema = await snapshot.getPromise(field(`${expanded}.${path}`));
({ snapshot }) => async (currentField: string, path: string, data) => {
const expanded = await snapshot.getPromise(expandPath(currentField));
const schema = await snapshot.getPromise(field(`${expanded}.${path}`));

if (schema?.ftype === FLOAT_FIELD) {
return Number.parseFloat(data);
}
if (schema?.ftype === FLOAT_FIELD) {
return Number.parseFloat(data);
}

if (schema?.ftype === INT_FIELD) {
return Number.parseInt(data);
}
if (schema?.ftype === INT_FIELD) {
return Number.parseInt(data);
}

return data;
},
return data;
},
[]
);
};
Expand Down
5 changes: 4 additions & 1 deletion app/packages/core/src/components/Modal/Sidebar/Annotate/useFocus.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,10 @@ export default function useFocus() {
const store = getDefaultStore();

scene?.on(LIGHTER_EVENTS.OVERLAY_SELECT, (event) => {
store.set(editing, store.get(labelMap)[event.detail.id]);
const label = store.get(labelMap)[event.detail.id];
if (label) {
store.set(editing, label);
}
});
}, [scene]);
}
27 changes: 13 additions & 14 deletions app/packages/core/src/components/Modal/Sidebar/Annotate/useLabels.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,20 +115,19 @@ export default function useLabels() {
const schemaMap = useAtomValue(schemas);
const addLabel = useAddAnnotationLabel();
const getFieldType = useRecoilCallback(
({ snapshot }) =>
async (path: string) => {
const loadable = await snapshot.getLoadable(field(path));
const type = loadable
.getValue()
?.embeddedDocType?.split(".")
.slice(-1)[0];

if (!type) {
throw new Error("no type");
}

return type as LabelType;
},
({ snapshot }) => async (path: string) => {
const loadable = await snapshot.getLoadable(field(path));
const type = loadable
.getValue()
?.embeddedDocType?.split(".")
.slice(-1)[0];

if (!type) {
throw new Error("no type");
}

return type as LabelType;
},
[]
);

Expand Down
19 changes: 11 additions & 8 deletions app/packages/lighter/src/core/Scene2D.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ import {
LighterEventDetail,
type LighterEvent,
} from "../event/EventBus";
import type { InteractionHandler } from "../interaction/InteractionManager";
import { BaseOverlay } from "../overlay/BaseOverlay";
import { InteractionManager } from "../interaction/InteractionManager";
import type { BaseOverlay } from "../overlay/BaseOverlay";
import { SelectionManager } from "../selection/SelectionManager";
import type { InteractionHandler } from "../interaction/InteractionManager";
import type { Selectable } from "../selection/Selectable";
import type { SelectionOptions } from "../selection/SelectionManager";
import { SelectionManager } from "../selection/SelectionManager";
import type {
CanonicalMedia,
CoordinateSystem,
Expand Down Expand Up @@ -1436,11 +1436,15 @@ export class Scene2D {
overlay: BaseOverlay & Spatial
): void {
const absoluteBounds = overlay.getAbsoluteBounds();
const relativeBounds =
this.coordinateSystem.absoluteToRelative(absoluteBounds);

// Update the overlays relative bounds
overlay.setRelativeBounds(relativeBounds);
if (BaseOverlay.validBounds(absoluteBounds)) {
const relativeBounds =
this.coordinateSystem.absoluteToRelative(absoluteBounds);

// Update the overlays relative bounds
overlay.setRelativeBounds(relativeBounds);
overlay.markCoordinateUpdateComplete();
}
}

/**
Expand All @@ -1462,7 +1466,6 @@ export class Scene2D {
for (const overlay of this.overlays.values()) {
if (TypeGuards.isSpatial(overlay) && overlay.needsCoordinateUpdate()) {
this.updateSpatialOverlayRelativeBounds(overlay);
overlay.markCoordinateUpdateComplete();

this.dispatch({
type: LIGHTER_EVENTS.OVERLAY_BOUNDS_CHANGED,
Expand Down
35 changes: 24 additions & 11 deletions app/packages/lighter/src/interaction/InteractionManager.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,15 @@ import type { Renderer2D } from "../renderer/Renderer2D";
import type { SelectionManager } from "../selection/SelectionManager";
import type { Point } from "../types";
import { InteractiveDetectionHandler } from "./InteractiveDetectionHandler";
import { BoundingBoxOverlay } from "../overlay/BoundingBoxOverlay";

/**
* Interface for objects that can handle interaction events.
*/
export interface InteractionHandler {
readonly id: string;
readonly cursor?: string;
overlay?: BoundingBoxOverlay;

/**
* Returns true if the handler is being dragged or resized.
Expand Down Expand Up @@ -221,7 +223,8 @@ export class InteractionManager {
const interactiveHandler = this.getInteractiveHandler();

if (interactiveHandler) {
handler = interactiveHandler;
handler = interactiveHandler.overlay || interactiveHandler;
this.selectionManager.select(handler.id);
} else {
handler = this.findHandlerAtPoint(point);
}
Expand Down Expand Up @@ -259,7 +262,7 @@ export class InteractionManager {
this.currentPixelCoordinates = point;

const interactiveHandler = this.getInteractiveHandler();
const handler = this.findMovingHandler() || this.findHandlerAtPoint(point);
let handler = this.findMovingHandler() || this.findHandlerAtPoint(point);

if (!interactiveHandler) {
// we don't want to handle hover in interactive mode
Expand All @@ -278,7 +281,9 @@ export class InteractionManager {
this.maintainAspectRatio
);
} else {
interactiveHandler.onMove?.(
handler = interactiveHandler.overlay || interactiveHandler;

handler.onMove?.(
point,
worldPoint,
event,
Expand Down Expand Up @@ -317,15 +322,29 @@ export class InteractionManager {
const point = this.getCanvasPoint(event);
const worldPoint = this.renderer.screenToWorld(point);
const scale = this.renderer.getScale();
const handler = this.findMovingHandler() || this.findHandlerAtPoint(point);
const now = Date.now();

let handler: InteractionHandler | undefined = undefined;

const interactiveHandler = this.getInteractiveHandler();

if (interactiveHandler) {
handler = interactiveHandler.overlay || interactiveHandler;
} else {
handler = this.findMovingHandler() || this.findHandlerAtPoint(point);
}

if (handler?.isMoving?.()) {
const startPosition = handler.getMoveStartPosition()!;

// Handle drag end
handler.onPointerUp?.(point, event, scale);

if (interactiveHandler?.overlay === handler) {
this.removeHandler(interactiveHandler);
this.addHandler(handler);
}

this.canvas.style.cursor =
handler.getCursor?.(worldPoint, scale) || this.canvas.style.cursor;

Expand Down Expand Up @@ -611,6 +630,7 @@ export class InteractionManager {
const candidates: InteractionHandler[] = [];
for (let i = this.handlers.length - 1; i >= 0; i--) {
const handler = this.handlers[i];

if (skipCanonicalMedia && handler.id === this.canonicalMediaId) {
continue;
}
Expand Down Expand Up @@ -648,13 +668,6 @@ export class InteractionManager {
y: event.clientY - rect.top,
};

// if there's interactive handler, convert to world coordinates
// todo: make this simpler
const interactiveHandler = this.getInteractiveHandler();
if (interactiveHandler) {
return this.renderer.screenToWorld(screenPoint);
}

return screenPoint;
}

Expand Down
9 changes: 8 additions & 1 deletion app/packages/lighter/src/overlay/BaseOverlay.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,14 @@ import { LIGHTER_EVENTS } from "../event/EventBus";
import type { InteractionHandler } from "../interaction/InteractionManager";
import type { Renderer2D } from "../renderer/Renderer2D";
import type { ResourceLoader } from "../resource/ResourceLoader";
import type { DrawStyle, Point, RawLookerLabel } from "../types";
import type { DrawStyle, Point, Rect } from "../types";

/**
* Base abstract class for all overlays.
*/
export abstract class BaseOverlay implements InteractionHandler {
readonly id: string;
readonly field: string;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Duplicate field declaration.

Line 18 declares readonly field: string but Line 33 still declares protected field: string. TypeScript will reject this duplicate declaration.

Apply this diff to remove the duplicate:

   protected eventBus?: EventBus;
   protected resourceLoader?: ResourceLoader;
   protected currentStyle?: DrawStyle;
-  protected field: string;
   protected label: RawLookerLabel;

Also applies to: 33-33

🤖 Prompt for AI Agents 
In app/packages/lighter/src/overlay/BaseOverlay.ts around lines 18 and 33,
there's a duplicate declaration of `field` (`readonly field: string` at line 18
and `protected field: string` at line 33); remove the duplicate at line 18 so
only the `protected field: string` declaration remains (or alternatively make
both consistent by replacing line 33 with `protected readonly field: string` if
immutability is intended), ensuring a single declaration to satisfy TypeScript.

readonly cursor?: string;

protected isHoveredState = false;
Expand All @@ -32,6 +33,12 @@ export abstract class BaseOverlay implements InteractionHandler {
protected field: string;
protected label: RawLookerLabel;

static validBounds(bounds: Rect): boolean {
return ["x", "y", "width", "height"].every(
(prop) => typeof bounds[prop] === "number" && bounds[prop] >= 0
);
}

constructor(id: string, field: string, label: RawLookerLabel) {
this.id = id;
this.field = field;
Expand Down
Loading
Loading