XCSoar (.CUP) and Winpilot (.DAT) file processing now works #90

paynterf · 2024-12-21T14:01:39Z

XCSoar (.CUP) and Winpilot (.DAT) file processing now works properly. Unfortunately I couldn't figure out how to get the left/right/top/bottom bounds values to show up in the corresponding web page windows, so the map still isn't generated automatically.

As a workaround the calculated bounds are displayed in the 'error' window, and the user can manually transcribe the values to the bounds entry fields and then select the 'select map bounds by hand' option.

lordfolken · 2024-12-23T05:56:08Z

lib/xcsoar/mapgen/waypoints/seeyou_reader.py

+    wpnum = 0
+    for byteline in lines:
+        wpnum = wpnum + 1
+        line = byteline.decode('ISO-8859-2') #gfp 241210: added 'ISO-8859-2' decoding for correct cherrypy logging display


Shouldn't this be UTF-8?

I tried 'UTF-8' and AFAICT, it doesn't handle the accent marks in the Slovenia3 waypoint list. I added the 'Slovenia3.cup' and 'Slovenia3.dat' test files for your consideration (the .DAT file was generated from the .CUP file using a separate Python script to make sure they both contained the exact same coordinates)

I have no problem using UTF-8 vs ISO-8859-2, but some of the characters in Slovenia3.cup won't be displayed properly. I don't think this will affect the bounds calculation either way.

The files you provide contain corrupted data.
Slovenia3.cup: Lines 26, 94, 774, 776.

A CUP file should be utf-8. https://github.com/naviter/seeyou_file_formats/blob/main/CUP_file_format.md

These files should only be included if they serve an automated test case, and then they should be under test/data/cup/filename.

https://download.xcsoar.org/content/waypoint/country/SI-WPT-National-OpenAIP.cup if you need an example that works.

https://www.condorsoaring.com/forums/viewtopic.php?f=29&t=22737&p=191905#p191905 seems to be the file from slovenia3 in condor.

lordfolken · 2024-12-23T05:56:44Z

lib/xcsoar/mapgen/waypoints/seeyou_reader.py

+        # cherrypy.log('in for loop: wpnum = %s line = %s' %(wpnum, line))
+#        cherrypy.log(f'for loop row {wpnum}: {line}')
+
+        #check for blank lines or comments


Lots of debugging code. We don't really want that in the code base.

lordfolken

Congrats on figuring out the basic problem. :)

Can you please make a change that is as minimal as possible to fix the cup parsing issue and make a commit for it? No Test files or commented out debug code please, also no excessive logging which isn´t needed for operation.

paynterf · 2025-01-02T01:07:27Z

Let me get my Linux laptop back out and I'll do the changes you requested ;)

paynterf · 2025-01-02T01:31:20Z

Regarding the .CUP file encoding issue, I checked both the Slovenia3.cup and the Slovenia3.cup, and it appears the Slovenia3.cup file is corrupted - at least in my Landscapes/Slovenia3 folder. I did see that Slovenia3.cup is encoded in UTF-8 as you stated, but for some reason the Slovenia2.cup file is encoded as 'ANSI'. In any case I'll change back to UTF-8, but what to do about the corrupted lines in Slovenia3.cup? Can you check Slovenia3.cup on your end? Here is my line for 'Amzs Koa' (which isn't in the Slovenia2.cup file)

"Amzs Koâ”œÃ",Amzs Koì,4539.139N,01450.914E,470.0m,1,,,,

for more information, see https://pre-commit.ci

paynterf · 2025-01-02T03:10:03Z

commit 2abe1a4 (HEAD -> master)
Author: Frank Paynter [email protected]
Date: Wed Jan 1 21:47:37 2025 -0500

Changed decoding to UTF-8, removed logging code, removed test files

lordfolken · 2025-01-02T16:26:28Z

Regarding the .CUP file encoding issue, I checked both the Slovenia3.cup and the Slovenia3.cup, and it appears the Slovenia3.cup file is corrupted - at least in my Landscapes/Slovenia3 folder. I did see that Slovenia3.cup is encoded in UTF-8 as you stated, but for some reason the Slovenia2.cup file is encoded as 'ANSI'. In any case I'll change back to UTF-8, but what to do about the corrupted lines in Slovenia3.cup? Can you check Slovenia3.cup on your end? Here is my line for 'Amzs Koa' (which isn't in the Slovenia2.cup file)

"Amzs Koâ”œÃ",Amzs Koì,4539.139N,01450.914E,470.0m,1,,,,

I also have condor3 and i see the same issue in the Slovenia3.cup file. So its really corrupted.

lordfolken · 2025-01-02T16:35:17Z

Slovenia3.cup

Please remove this cup/dat files for now. While i understand your testing intent, we don't want that in the production code base.

lordfolken · 2025-01-02T16:36:14Z

lib/xcsoar/mapgen/server/server.py

+cherrypy.config.update(
+    {
+        "log.screen": True,  # Log to stdout
+        "log.error_file": "error.log",  # Log errors to a file
+        "log.access_file": "access.log",  # Log access to a file
+    }
+)
+


Normally this runs in a container. The Filesystem is usually readonly. Please remove this configuration.

lordfolken · 2025-01-02T16:36:46Z

lib/xcsoar/mapgen/server/views/index.html

@@ -57,7 +57,7 @@
      <div id="map"/>
      <em class="note">Draw the map bounds rectangle by clicking in the map.</em>
      <hr/>
-      <label class="left" for="left">Minimum Longitude:</label><input id="left" type="text" name="left"/><br/>
+      <label class="left" for="left">Minimum Longitude:</label><input id="left" type="text" name="left" value="43"/><br/>


why the default value=43?

lordfolken · 2025-01-02T16:37:02Z

lib/xcsoar/mapgen/waypoints/list.py

@@ -33,10 +34,13 @@ def extend(self, wp_list):
    def get_bounds(self, offset_distance=15.0):
        rc = GeoRect(180, -180, -90, 90)
        for wp in self.__list:
+            # cherrypy.log(f'In list.py: {wp.name}, lat: {wp.lat:.3f}, lon: {wp.lon:.3f}')


please remove.

lordfolken · 2025-01-02T16:37:14Z

lib/xcsoar/mapgen/waypoints/list.py

+        # cherrypy.log(f'In list.py - final rc: left {rc.left:.3f}, right: {rc.right:.3f}, top: {rc.top:.3f}, bot {rc.bottom:.3f}')
+


please remove.

lordfolken · 2025-01-02T16:38:39Z

lib/xcsoar/mapgen/waypoints/seeyou_reader.py

    return waypoint_list
+
+    return waypoint_list


I don't quite understand this double return statement?

lordfolken · 2025-01-02T16:41:59Z

lib/xcsoar/mapgen/waypoints/seeyou_reader.py

-        if first:
-            first = False
-            continue
+    header = "name,code,country,lat,lon,elev,style,rwdir,rwlen,freq,desc"


beware that the there is a new standard since 2022, that has some additional fields:
https://github.com/naviter/seeyou_file_formats/blob/main/CUP_file_format.md

Maybe xcsoar's parser can serve as a reference:
https://github.com/XCSoar/XCSoar/blob/master/src/Waypoint/WaypointReaderSeeYou.cpp

Frank Paynter added 3 commits December 17, 2024 09:58

Changes to allow *.CUP file parsing for geobounding

85ef006

Both .CUP and .DAT files work now

4679ad9

Added Slovenia .cup/.dat test files, commmented out logging code

0e1083a

lordfolken linked an issue Dec 23, 2024 that may be closed by this pull request

XCSoar Map Generator Error - cup file upload #17

Open

lordfolken reviewed Dec 23, 2024

View reviewed changes

lordfolken requested changes Dec 23, 2024

View reviewed changes

Changed decoding to UTF-8, removed logging code, removed test files

2abe1a4

paynterf force-pushed the master branch from ff5c3c1 to 2abe1a4 Compare January 2, 2025 03:09

[pre-commit.ci] auto fixes from pre-commit.com hooks

93a5170

for more information, see https://pre-commit.ci

lordfolken reviewed Jan 2, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

XCSoar (.CUP) and Winpilot (.DAT) file processing now works #90

XCSoar (.CUP) and Winpilot (.DAT) file processing now works #90

paynterf commented Dec 21, 2024

lordfolken Dec 23, 2024

paynterf Dec 24, 2024

lordfolken Dec 24, 2024

lordfolken Dec 24, 2024

lordfolken Dec 26, 2024

lordfolken Dec 23, 2024

lordfolken left a comment

paynterf commented Jan 2, 2025

paynterf commented Jan 2, 2025

paynterf commented Jan 2, 2025

lordfolken commented Jan 2, 2025

lordfolken Jan 2, 2025

lordfolken Jan 2, 2025

lordfolken Jan 2, 2025

lordfolken Jan 2, 2025

lordfolken Jan 2, 2025

lordfolken Jan 2, 2025

lordfolken Jan 2, 2025

		# cherrypy.log(f'In list.py - final rc: left {rc.left:.3f}, right: {rc.right:.3f}, top: {rc.top:.3f}, bot {rc.bottom:.3f}')

XCSoar (.CUP) and Winpilot (.DAT) file processing now works #90

Are you sure you want to change the base?

XCSoar (.CUP) and Winpilot (.DAT) file processing now works #90

Conversation

paynterf commented Dec 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lordfolken left a comment

Choose a reason for hiding this comment

paynterf commented Jan 2, 2025

paynterf commented Jan 2, 2025

paynterf commented Jan 2, 2025

lordfolken commented Jan 2, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment