Parsing EarthCam

Downloading videos from EarthCam

To download frames and videos from Earthcam and other similar livestream cameras, you need to first acquire the source of the stream and then encode the video with FFmpeg.

You could do this using either the Mozilla Firefox® FVD browser extension or Wireshark®, as explained in the following wiki.

OPTION 1: Downloading Earthcam with Firefox FVD and FFmpeg

Step 1:

Install the Firefox browser (https://www.mozilla.org/en-US/firefox/new/). If you are using Ubuntu, it should already be installed.

Step 2:

Launch Firefox and browse to the page for the Firefox Flash Video Downloader (https://addons.mozilla.org/en-US/firefox/addon/flash-video-downloader/). This browser extension will allow you to acquire the source URL of many livestreams with two clicks.

Step 3:

Install the browser extension by clicking the "Add to Firefox" button and then "Install". Firefox will install the extension and display a confirmation message. You should see the FVD tool appear on the Firefox bar (indicated in the red box).

Step 4:

Navigate to EarthCam. (http://www.earthcam.com/) Select the camera that you wish to download videos from.

We will choose Times Square for this example. (http://www.earthcam.com/usa/newyork/timessquare/?cam=tsrobo1)

Step 5:

You should notice the FVD icon turn blue. Click it to call up the pull-down menu. Click "Copy to Clipboard" for the target stream. You have now acquired the URL for the stream playlist.

Step 6 (Downloading the Video):

Be sure you have installed FFmpeg as explained here.

Next, use the URL you fetched in Step 5 to encode the video as either an AVI video or as individual JPEG frames:

AVI video

Use the following command:

ffmpeg -i [SRC URL] -t [SECONDS OF VIDEO] [DESTINATION VIDEO FILE]

In this example, you can download 10 seconds of video from the Times Square camera:

ffmpeg -i http://video3.earthcam.com/fecnetwork/9974.flv/chunklist_w1220748217.m3u8 -t 10 ffmpegOnly.avi

JPEG frames

Use the following command:

ffmpeg -i [SRC URL] -t [SECONDS OF VIDEO] [DESTINATION IMAGE FILE]

In this example, you can download 2 seconds' worth of frames from the Times Square camera:

ffmpeg -i http://video3.earthcam.com/fecnetwork/9974.flv/chunklist_w1220748217.m3u8 -t 2 frame%0d.jpg

OPTION 2: Downloading Earthcam with Wireshark and FFmpeg

Step 1:

Download Wireshark (https://www.wireshark.org/) (Wireshark is the world’s foremost and widely-used network protocol analyzer. It lets you see what’s happening on your network at a microscopic level and is the de facto (and often de jure) standard across many commercial and non-profit enterprises, government agencies, and educational institutions.)

Step 2:

Navigate to EarthCam. (http://www.earthcam.com/) Now, choose the camera that you want to sniff the information from. Note: Make sure all other tabs are closed because it would be difficult to locate the data packet from the live stream if multiple tabs are opened.

We will choose Times Square for this example. (http://www.earthcam.com/usa/newyork/timessquare/?cam=tsrobo1)

Step 3:

Now, open WireShark on your computer and double click on the way you access your internet connection. This window on your right should appear after you double click your internet source.

Step 4:

Note: Make sure only the NYC camera tab is open on your browser. This is the step where we sniff the data source from EarthCam.

1. Click on the blue button in the top left corner of WireShark to start sniffing. (Indicated by an orange box in the photo below).
2. As soon as you click the blue button, refresh the NYC tab. Then once the page loads again, allow it to play for a while (3 secs) and hit pause. Repeat the play and pause three times.
3. Now, go to WireShark and click on the “stop” button to its right to stop sniffing.

Step 5:

We have now (“hopefully”) sniffed the source of the video. Time to analyze the data which we have obtained.

1. We are now looking for the RTMP protocol. This protocol is used for streaming audio, video and data over the Internet, between a Flash player and a server. To do that, type in “rtmpt” in the filter section of WireShark.
2. Then manually scroll down and look for “Handshake” in the info section.

Step 6:

So, till now we have obtained information about the packets which EarthCam obtains. In this step, we shall analyze those packets to identify the source of EarthCam!

1. Right click on the field that says “Handshake” -> Follow -> TCP Stream.
2. A dialogue box now opens with all sorts of information – an ASCII dump. (It should be very similar to the image below. Make sure Show and Save data is in ASCII format.)

3. Now in Find, search for “play”. Keep search till you can see links above play which are like the ones shown in the image below. Alternatively, one could also search for “tcUrl”.

The information shown in the image above is all we need to parse the camera data.

Explanation: (What each number in the above figure means)

1. Box 1: rtmp://video3.earthcam.com:1935/fecnetwork
   This is the link with the RTMP protocol.
2. Box 2: http://static.earthcamcdn.com/swf/streaming/stream_viewer_v3.swf?20170307170000 This is the link to the flash file.
3. Box 3: (Blue) http://www.earthcam.com/usa/newyork/timessquare/?cam=tsrobo1 This is the link to the Times Square page on EarthCam.
4. Box 4: 9974.flv This is the name of the flash file.

Step 7 (Playing live stream on VLC):

In this step, we will use the information obtained in Step 6 (Figure 8.) and display a live stream on VLC media player. If you just want to download video, skip this step and go to Step 8 directly.

Note:

1. Make sure to have rtmpdump installed. Command: sudo apt-get install rtmpdump.
2. Make sure to have vlc-nox installed. Command: sudo apt-get vlc-nox.

The command given below will display the live-stream directly on VLC media player.

rtmpdump -r "rtmp://video3.earthcam.com:1935/fecnetwork" -a "fecnetwork" -f "LNX 23,0,0,162" -W "http://static.earthcamcdn.com/swf/streaming/stream_viewer_v3.swf?20170307170000" -p "http://www.earthcam.com/usa/newyork/timessquare/?cam=tsrobo1" -y "9974.flv" --quiet | vlc –

Step 8: (Downloading video from EarthCam)

Note: Install ffmpeg from here (https://www.faqforge.com/linux/how-to-install-ffmpeg-on-ubuntu-14-04/)

To download the live stream. Type the following:

rtmpdump -r "rtmp://video3.earthcam.com:1935/fecnetwork" -a "fecnetwork" -f "LNX 23,0,0,162" -W "http://static.earthcamcdn.com/swf/streaming/stream_viewer_v3.swf?20170307170000" -p "http://www.earthcam.com/usa/newyork/timessquare/?cam=tsrobo1" -y "9974.flv" -R -o - | ffmpeg -i pipe:0 -t 10 9974.avi

This command uses ffmpeg to download a 10 second video from the given link. “-t 10” - specifies the time interval of the video, a different time (in seconds) can be used to download a longer video.

We can use the following command to download images:

rtmpdump -r "rtmp://video3.earthcam.com:1935/fecnetwork" -a "fecnetwork" -f "LNX 23,0,0,162" -W "http://static.earthcamcdn.com/swf/streaming/stream_viewer_v3.swf?20170307170000" -p "http://www.earthcam.com/usa/newyork/timessquare/?cam=tsrobo1" -y "9974.flv" -R -o - | ffmpeg -i pipe:0 image%0d.jpg

Removing the -t flag would allow us to download images from the live stream indefinitely.