Skip to content

Commit a4ac96c

Browse files
azimutazimut
committed
updates, make, gnuplot, ai smn dataset
1 parent ace2154 commit a4ac96c

File tree

7 files changed

+163
-47
lines changed

7 files changed

+163
-47
lines changed

‎ai/self.org‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,25 @@
9494
- https://en.wikipedia.org/wiki/Iris_flower_data_set
9595
- https://en.wikipedia.org/wiki/ImageNet
9696
- https://github.com/several27/FakeNewsCorpus
97+
- https://meteostat.net/es/station/87582?t=2022-01-01/2022-12-31
98+
- ezeiza https://meteostat.net/es/station/87576?t=2025-01-01/2025-09-10
99+
- SMN - https://www.smn.gob.ar/sites/default/files/catalog.xlsx
100+
- "csv" per day https://ssl.smn.gob.ar/dpd/descarga_opendata.php?file=observaciones/datohorario20230101.txt
101+
- https://ssl.smn.gob.ar/dpd/descarga_opendata.php?file=
102+
- https://ssl.smn.gob.ar/dpd/zipopendata.php?dato=
103+
- tiepre
104+
- regtemp
105+
- estaciones
106+
- observaciones/datohorario YYYYMMDD.txt
107+
- observaciones/obs YYYYMMDD.txt
108+
- radiacionsolar/radsolar YYYYMMDD.txt
109+
- pron5d/pron YYYYMMDD.txt
110+
- https://ssl.smn.gob.ar/dpd/observaciones/estadisticas.txt
111+
- json search https://ws1.smn.gob.ar/v1/georef/location/search?name=lomas
112+
- json forecast https://ws1.smn.gob.ar/v1/forecast/location/4856
113+
- needs JWT
114+
- obtain JWT https://www.smn.gob.ar/pronostico
115+
- headers={"Referer": "https://www.smn.gob.ar/"}
97116
- https://en.wikipedia.org/wiki/MNIST_database
98117
- Modified NIST (National Institute of Standards and Technology)
99118
- 28x28 grayscale

‎editor/ed.org‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
- 20 video [[https://www.youtube.com/watch?v=7pqaj050X7g][I Love ED on CP/M]]

‎languages/compiled/makefile/self.org‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -150,9 +150,19 @@ $(ab:.cpp=.o) # string replace
150150
| != | exec shell command and assign it |
151151
|-----+--------------------------------------------|
152152
*** target specific
153-
#+begin_src makefile
153+
154+
This defined a variable visible by Make.
155+
156+
#+begin_src makefile
154157
%.o: CFLAGS = -O
155-
#+end_src
158+
#+end_src
159+
160+
This defines a variable also (? visible by shell subcommands.
161+
162+
#+begin_src makefile
163+
%.o: export PAGER = more
164+
#+end_src
165+
156166
*** goal specific
157167
#+begin_src makefile
158168
ifeq (,$(filter debug,$(MAKECMDGOALS))
@@ -316,6 +326,7 @@ $(THUMBS): %.thumb.jpg: %.jpg
316326
- ~\t~ for indentation, NOT spaces
317327
- Will *keep* an incomplete target on failure, unless ~.DELETE_ON_ERROR:~
318328
- What is an ~intermediate~ target? And how to stop them from being deleted?
329+
- to do Process Substitution we need to use *bash -c*, escape *$* with *$* and *\*, one for make other for bash itself
319330

320331
* tools
321332
|------------+-------------------------------------------|

‎languages/interpreted/lisp/common-lisp/libraries.org‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -436,3 +436,9 @@ scene2d-layout.lisp
436436
screen-transition.lisp
437437
+select-box.lisp+
438438
tiled-map.lisp
439+
* CMN
440+
441+
- by Richard/Heinrich Taube
442+
- https://es.wikipedia.org/wiki/Common_Music
443+
- https://ccrma.stanford.edu/CCRMA/Courses/AlgoComp/cm/doc/cm.html
444+
- https://ccrma.stanford.edu/software/cmn/cmn/cmn.html

‎terminal/tools/parsers/self.org‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,18 @@
11
- tool: tui https://www.visidata.org/
2+
- vids https://www.youtube.com/watch?v=yhunJc8Nu4g&list=PLxu7QdBkC7drrAGfYzatPGVHIpv4Et46W&index=5
3+
- doc https://www.visidata.org/docs/
24
- doc https://jsvine.github.io/intro-to-visidata/
35
- graph https://www.visidata.org/docs/graph/
46
- csv, json, sqlite
57
- # number, % float, $ currency, @ date, ~ any, z~ remove
6-
- ! make key, - to hide
8+
- ! make key, - to hide, gv to unhide
79
- g. - to plot
8-
- Shift+C - Columns view
10+
- : - split command, asks for regex
11+
- M - melt aka unpivot
12+
- Shift+F - frequency sheet/view
13+
- Shift+O - options sheet/view
14+
- Shift+I - Summary sheet/view
15+
- Shift+C - Columns sheet/view
916
- ^ edit column name
1017

1118
- tool: web ui https://datasette.io/

‎terminal/tools/visualization/gnuplot/self.org‎

Lines changed: 114 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,9 @@
4242
- curve fitting https://www.youtube.com/watch?v=vHM_5AeKwXE
4343
- do for [i=1:96] { plot f(x,i*0.1) }
4444
- operators http://www.gnuplot.info/docs_4.2/node55.html
45+
- Colors
46+
- rgb "#0b0000"
47+
- rgb "white"
4548

4649
** functions
4750
- http://www.gnuplot.info/docs/loc1283.html
@@ -56,59 +59,111 @@
5659

5760
** commands
5861

59-
|-------+------------------------------------+-----------------------------|
60-
| <r> | | |
61-
| unset | OPTION | |
62-
| set | OPTION [VALUE] | |
63-
| with | (l)ines[*], dots[.], lines(points) | |
64-
| show | variables | |
65-
| stats | FILENAME name S | shows file field statistics |
66-
| | | later can be referenced |
67-
| | | eg: S_min_x |
68-
|-------+------------------------------------+-----------------------------|
62+
|-----------+------------------------------------+-----------------------------|
63+
| <r> | | |
64+
| multiplot | layout N,N | |
65+
| unset | OPTION | |
66+
| set | OPTION [VALUE] | |
67+
| with | (l)ines[*], dots[.], lines(points) | |
68+
| show | variables | |
69+
| stats | FILENAME name S | shows file field statistics |
70+
| | | later can be referenced |
71+
| | | eg: S_min_x |
72+
|-----------+------------------------------------+-----------------------------|
6973

7074
*** plot
7175

76+
#+begin_src gnuplot
77+
[re]plot INPUT [PLOT_A OPTS], INPUT [PLOT_B OPTS]
78+
#+end_src
79+
7280
|--------------+-------------------------|
7381
| <c> | |
7482
| INPUT | reads from... |
7583
|--------------+-------------------------|
7684
| "file.txt" | file contents |
7785
| "/dev/stdin" | standard input |
78-
| "< CMD" | CMD stdout |
86+
| "<CMD" | CMD stdout |
7987
| "" | re-use last given input |
8088
|--------------+-------------------------|
8189

82-
- [re]plot INPUT [PLOT_A OPTS], INPUT [PLOT_B OPTS]
83-
- every N - plots every N data points
84-
- for [n = 1 : 5]
85-
- widths = "30 15 5"
86-
for [width in widths]
87-
- using N:N+1:N+n
88-
- [[http://www.gnuplot.info/docs/loc9016.html][smooth]]
89-
- [[http://www.gnuplot.info/docs/loc9099.html][frequency]] - points with the same X-value are replaced with 1 point with Y-values sum
90-
- [bezier¦mcsplines]
91-
- title S (for legend box)
92-
- bins=N - total N number of bins in graph http://gnuplot.info/docs_6.1/loc9125.html
93-
- binvalue=[avg¦sum] - how to group each bin, default: sum
94-
- index N
95-
- reference block N of data (aka separated by 2 \n)
96-
- blocks index start at 0
97-
- with
98-
- histograms
99-
- filledcurves
100-
- fillstyle pattern N
101-
- linespoint
102-
- lines
103-
- [d]ash[t]ype N
104-
- [d]ash[t]ype S
105-
- [l]ine[s]tyle N
106-
- [l]ine[t]ype N
107-
- [l]ine[c]olor rgb "#0b0000"
108-
- [l]ine[c]olor rgb "green"
109-
- [l]ine[w]ith N
110-
- points
111-
- pointsize N
90+
- every N - plots every N data points
91+
- for [VAR = START : END]
92+
- for [LVAR in VAR] # widths = "30 15 5"
93+
- using N:N+1:N+n
94+
- [[http://www.gnuplot.info/docs/loc9016.html][smooth]]
95+
- [[http://www.gnuplot.info/docs/loc9099.html][frequency]] - points with the same X-value are replaced with 1 point with Y-values sum
96+
- [bezier¦mcsplines]
97+
- title S (for legend box)
98+
- bins=N - total N number of bins (NOT SIZE) in graph http://gnuplot.info/docs_6.1/loc9125.html
99+
- binvalue=[avg¦sum] - how to group each bin, default: sum
100+
- index N
101+
- reference block N of data (aka separated by 2 \n)
102+
- blocks index start at 0
103+
104+
- with
105+
- histograms
106+
- filledcurves
107+
- [f]ill[s]tyle pattern N
108+
- points
109+
- pointsize N
110+
- lines
111+
- [d]ash[t]ype N¦S
112+
- [l]ine[t]ype N
113+
- [l]ine[s]tyle N
114+
- [l]ine[w]idth N
115+
- [l]ine[c]olor COLOR
116+
- linespoint
117+
118+
*** fit
119+
120+
1) We come up with a function shape that "fits" our data.
121+
- the function has 2 types of variables
122+
- variables that depend on the input (eg: x)
123+
- variables that we want to find it's value and make "constants" (eg: a)
124+
2) We use =fit= command giving it:
125+
- the function in 1)
126+
- the input data
127+
- the fields *using*
128+
- the list of variables to find
129+
130+
#+begin_src gnuplot
131+
f(x) = a*x + b
132+
fit f(x) "" using 1:2 via a,b
133+
#+end_src
134+
135+
- https://www.youtube.com/watch?v=vHM_5AeKwXE
136+
137+
**** Example output
138+
139+
#+begin_src
140+
gnuplot> fit f(x) "<awk -vOFS=, '/EZEIZA/{ print $1,$2,$3 }' registro_temperatura365d_smn.txt" using 3:2 via a,b
141+
iter chisq delta/lim lambda a b
142+
0 4.3014300000e+04 0.00e+00 9.71e+00 1.000000e+00 1.000000e+00
143+
1 7.0509078061e+03 -5.10e+05 9.71e-01 1.315008e+00 6.153986e+00
144+
2 3.9723792475e+03 -7.75e+04 9.71e-02 9.075167e-01 1.243513e+01
145+
3 3.9719144040e+03 -1.17e+01 9.71e-03 9.024370e-01 1.251326e+01
146+
4 3.9719144040e+03 -1.81e-07 9.71e-04 9.024364e-01 1.251327e+01
147+
iter chisq delta/lim lambda a b
148+
149+
After 4 iterations the fit converged.
150+
final sum of squares of residuals : 3971.91
151+
rel. change during last iteration : -1.81067e-12
152+
153+
degrees of freedom (FIT_NDF) : 363
154+
rms of residuals (FIT_STDFIT) = sqrt(WSSR/ndf) : 3.30786
155+
variance of residuals (reduced chisquare) = WSSR/ndf : 10.9419
156+
157+
Final set of parameters Asymptotic Standard Error
158+
======================= ==========================
159+
a = 0.902436 +/- 0.02768 (3.067%)
160+
b = 12.5133 +/- 0.3791 (3.03%)
161+
162+
correlation matrix of the fit parameters:
163+
a b
164+
a 1.000
165+
b -0.890 1.000
166+
#+end_src
112167

113168
*** other
114169
- imagesc - matrix/heatmap - 2d contour
@@ -134,6 +189,16 @@
134189
| +reread+ | - | REMOVED in 6.0 +reread input+ |
135190
|------------+------------+-------------------------------------|
136191

192+
#+CAPTIONS: background
193+
#+begin_src gnuplot
194+
set object rectangle \
195+
from screen 0,0 to screen 1,1 \
196+
behind \
197+
fillcolor rgb '$background' \
198+
fillstyle solid \
199+
noborder
200+
#+end_src
201+
137202
*** general: file parsing
138203

139204
|-------------+-----------------+----------------------------------|
@@ -158,6 +223,7 @@
158223
| <c> | <c> | |
159224
|---------------+----------------+----------------------------------|
160225
| border | [N] | pick axis lines (takes a bitsum) |
226+
| | linecolor COL | |
161227
| grid | - | axis grid |
162228
| logscale | [x¦y] [N] | with step N |
163229
| autoscale | [[x¦y]fix] | prevents autoextend of scale |
@@ -170,7 +236,7 @@
170236
| | (S N[,S N]) | custom labels S at point N |
171237
| [xyz]tic | rotate by N | rotates tic labels |
172238
| [xyz][2]label | S | axis label |
173-
| | offset N | |
239+
| | offset N[,N] | |
174240
|---------------+----------------+----------------------------------|
175241
*** key
176242

@@ -204,7 +270,7 @@
204270

205271
#+CAPTION: gnuplot "test" command output
206272
#+ATTR_HTML: :width 500
207-
#+ATTR_ORG: :width 600
273+
#+ATTR_ORG: :width 800
208274
[[./test_page.png]]
209275

210276
*** term[inal]
@@ -251,6 +317,7 @@ http://gnuplot.info/demo/pm3dcolors.html
251317
* tools
252318

253319
- emacs: [[https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-gnuplot.html][Org-babel-gnuplot]]
320+
- color theming https://github.com/GideonWolfe/Gnuplot-Pywal
254321
- wrapper: plot realtime and stored data from the cli https://github.com/dkogan/feedgnuplot
255322
- config https://github.com/Gnuplotting/gnuplot-configs
256323
- pallettes
@@ -267,3 +334,7 @@ http://gnuplot.info/demo/pm3dcolors.html
267334
- no *%a* date format for input parsing
268335
- to read from stdin "-" might not work in plot (?
269336
- use /dev/stdin
337+
- you cannot plot fields from differents files/inputs against each other
338+
- on multiplot, sometimes a plot will silently fail due some formatting issue
339+
- try adding ";"
340+
- try adding "\n"

‎terminal/tools/visualization/gnuplot/videos.org‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
- Gnuplot Tutorial for Scientists https://www.youtube.com/playlist?list=PLIBLYHTGmm_4ZVHGoPMp0JRG3MPVmMpM1
12
- 24 gnuplot bootcamp https://www.youtube.com/watch?v=vu_LBbqI-oQ
23
- 4:47 end overview
34
- https://www.youtube.com/playlist?list=PLaYBfUc8SG7W4cTZprVWjbtwrottWuk8c

0 commit comments

Comments
 (0)