Implementation in Video(React Native) #258
Description
Implementation Issue
So I have been trying to implement gl-react over videos in React Native(Expo CLI). I follwed this example .
Expected behavior
Video should play with gl-react layers.
Actual behavior
However I get this error(copying the files exactly as it is):
blob:https://web.whatsapp.com/ccfde4db-5bbe-467e-a749-14dbd05dfc7d
So I tried changing the code a bit. (only index.js)
// @flow
import React, { useRef, useEffect } from "react";
import { Shaders, GLSL, Node } from "gl-react";
import { Video } from 'expo-av';
import VideoPlayer from 'expo-video-player'
import { Surface } from "gl-react-expo";
import raf from "raf";
import videoMP4 from "./videoMP4.mp4";
export { videoMP4 };
export const VideoContext: React$Context<?HTMLVideoElement> = React.createContext();
import {Dimensions} from 'react-native'
const width = Dimensions.get('window').width;
const height = Dimensions.get('window').height;
// We implement a component <Video> that is like <video>
// but provides a onFrame hook so we can efficiently only render
// if when it effectively changes.
export const VideoPlay = ({ onFrame, ...rest }: { onFrame: number => void }) => {
const video = useRef();
useEffect(() => {
let handle;
let lastTime;
const loop = () => {
handle = raf(loop);
if (!video.current) return;
const currentTime = video.current.currentTime;
// Optimization that only call onFrame if time changes
if (currentTime !== lastTime) {
lastTime = currentTime;
onFrame(currentTime);
}
};
handle = raf(loop);
return () => raf.cancel(handle);
}, [onFrame]);
return (
<VideoContext.Provider value={video}>
<VideoPlayer
{...rest} ref={video}
videoProps={{
shouldPlay: true,
resizeMode: Video.RESIZE_MODE_CONTAIN,
source: require('./videoMP4.mp4'),
}}
height={height}
inFullscreen={true}
/>
</VideoContext.Provider>
);
};
// Our example will simply split R G B channels of the video.
const shaders = Shaders.create({
SplitColor: {
frag: GLSL`
precision highp float;
varying vec2 uv;
uniform sampler2D children;
void main () {
float y = uv.y * 3.0;
vec4 c = texture2D(children, vec2(uv.x, mod(y, 1.0)));
gl_FragColor = vec4(
c.r * step(2.0, y) * step(y, 3.0),
c.g * step(1.0, y) * step(y, 2.0),
c.b * step(0.0, y) * step(y, 1.0),
1.0);
}`
}
//^NB perf: in fragment shader paradigm, we want to avoid code branch (if / for)
// and prefer use of built-in functions and just giving the GPU some computating.
// step(a,b) is an alternative to do if(): returns 1.0 if a<b, 0.0 otherwise.
});
const SplitColor = ({ children }) => (
<Node shader={shaders.SplitColor} uniforms={{ children }} />
);
// We now uses <Video> in our GL graph.
// The texture we give to <SplitColor> is a (redraw)=><Video> function.
// redraw is passed to Video onFrame event and Node gets redraw each video frame.
export default VideoShow = () => (
<Surface style={{height: height, width: width}} pixelRatio={1}>
<SplitColor>
{redraw => (
<VideoPlay onFrame={redraw} autoPlay loop />
)}
</SplitColor>
</Surface>
);
Now I get a complete black screen as the layer over the video. Video plays for sure, since audio is coming, seems like the gl layer over it comes as black only. This warning comes >> Node#138(colorify#50), uniform children: child is not renderable. Got:, null