TextInput selection(cursor) coordinates

[perunt]

Summary

This proposal brings an advanced enhancement to the TextInput component in React Native, focusing on sophisticated tracking of text selection positions. The main feature is providing developers with specific coordinates (x and y) for the start and end points of text selection or cursor within a TextInput. These coordinates are relative to the TextInput component itself, rather than the entire screen.

To illustrate, when a user selects text within the TextInput field, the coordinates will indicate the position of the selection's start (top left corner of the selection) and end (bottom right corner of the selection) points, all within the boundaries of the TextInput field. This functionality is pivotal for implementing features like inline suggestions (emojis, text completions) which can be displayed right at the point of text selection or cursor placement, thereby significantly improving user interaction and experience within the app.

This addition makes it easier for developers to create more intuitive and context-sensitive interfaces. For instance, a suggestion box or editing tools could appear exactly at the location of text selection or cursor in the TextInput, enhancing the user’s interaction with text input fields by providing immediate, relevant options and tools based on their current focus area within the text.

Basic example

  type SelectionType = {
    start: number,
    end: number,
    selectionPosition: {
        start: {x: number, y: number},
        end: {x: number, y: number}
    }
  }

  <TextInput
    onSelectionChange={(e) => {
      const selection: SelectionType = e.nativeEvent.selection;
    }}
  />

250882266-b010d4e7-3777-43fb-956a-673b725c15e7.mp4

Motivation

The current implementation of TextInput in React Native lacks the ability
to measure the content of the selection or the cursor position. This limitation
hinders the development of features like context-aware suggestions or tools that
need precise cursor location (e.g., rich text editors, autocomplete functionalities).
By providing detailed coordinates for the cursor and selection, developers can
enhance user experience with dynamic and interactive text input features.

Detailed design

The proposed design includes:

1. Enhanced Selection Data: Updating the onSelectionChange event's data to include
   selectionPosition, which will contain the coordinates {x: number, y: number}
   for the start and end points of the text selection.

  selection: {
    start: number,
    end: number,
    selectionPosition: {
      start: { x: number, y: number },
      end: { x: number, y: number }
    }
  }

2. Type Definitions and Flow Updates: Adjusting type definitions in TextInput.d.ts
   and TextInput.flow.js to integrate the new data structure.

3. Native Implementation: Modifying native methods in iOS (RCTBaseTextInputView.m)
   and Android (ReactTextInputManager.java) to capture and transmit the selection position
   data.

4. Event Handling and Dispatch: Revising event handling to accommodate the new selection
   position data.

5. TextInput Component Adaptation: Updating the TextInput component to handle and utilize
   the new selection data format.

6. Testing and Examples: Providing updated examples and test cases to demonstrate the
   usage of the enhanced feature.

Drawbacks

Increased Complexity: The implementation adds complexity to the codebase at various
levels (JavaScript, iOS, Android).

Potential Performance Impact: Additional calculations for selection position might
affect performance, particularly in complex text input scenarios.

Alternatives

Cursor Position Tracking: Initially considered tracking only the cursor
position (cursorPosition), but this was deemed less universally beneficial compared
to selection position tracking.

External Libraries: Relying on third-party libraries for this functionality,
though this might not offer the same level of integration and performance efficiency.

Adoption strategy

The feature is designed to be backward compatible and non-breaking.
Existing applications can adopt the new feature without any required changes to existing code.
Documentation and examples will be provided for smooth adoption.

How we teach this

Documentation Updates: Revise the documentation to include this new feature,
supplemented with code examples.

Tutorial Creation: Develop tutorials showing practical implementations of the feature.

Community Engagement: Promote the feature within the community for feedback
and wider adoption.

Unresolved questions

The exact performance impact on different devices and under various usage conditions.
Ensuring seamless integration with popular third-party libraries that extend TextInput.

RFC

#730

[kyle-belle]

How is this seriously still not a thing. Someone added a basic version of this themselves to their custom react-native build more than 5 or 6 years ago https://medium.com/@p9alisand/how-to-get-textinput-cursor-position-in-react-native-0-49-1845aee40e9b and it seemed to be fairly simple as well.

performance impact could be minimized by having a boolean prop (like calculateCursorLayout) which someone would need to be provided

I REALLY, REALLY dont want to have to make an entire custom build just to get basic functionality like this.

[liveakshay]

Has this been incorporated yet?