Skip to content

Commit ace2154

Browse files
azimutazimut
committed
split ai, updates (gnuplot, sqlite cli, other)
1 parent 0eb4070 commit ace2154

File tree

10 files changed

+921
-643
lines changed

10 files changed

+921
-643
lines changed

‎ai/books.org‎

Lines changed: 687 additions & 0 deletions
Large diffs are not rendered by default.

‎ai/nn.org‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@
6767

6868
- James L. McClelland, David E. Rumelhart
6969
- book about Parallel Distributed Processing
70-
- introduced ~hidden layers~
70+
- introduced ~Hidden Layers~
7171

7272
* Articles
7373

‎ai/self.org‎

Lines changed: 46 additions & 588 deletions
Large diffs are not rendered by default.

‎ai/videos.org‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
- Video: How to Create a Neural Network (and Train it to Identify Doodles)
2+
https://www.youtube.com/watch?v=hfMk-kjRv4c
3+
- MIT RES.LL-005 Mathematics of Big Data and Machine Learning, IAP 2020
4+
https://www.youtube.com/watch?v=t4K6lney7Zw
5+
- Video: Conal Elliott: Can Tensor Programming Be Liberated from the Fortran Data Paradigm?
6+
https://www.youtube.com/watch?v=oaIMMclGuog
7+
https://github.com/conal/talk-2021-can-tensor-programming-be-liberated
8+
- Vincent Warmerdam: Winning with Simple, even Linear, Models | PyData London 2018
9+
https://www.youtube.com/watch?v=68ABAU_V8qI
10+
- NARDOZ MARZO - Fairness en Machine Learning + Testing en desarrollo de software
11+
https://www.youtube.com/watch?v=rrwrornKhjM
12+
* courses
13+
- Deeplearning - Udacity - https://www.youtube.com/playlist?list=PLAwxTw4SYaPn_OWPFT9ulXLuQrImzHfOV
14+
- deep learning https://www.youtube.com/playlist?list=PLoROMvodv4rPOWA-omMM6STXaWW4FvJT8
15+
- Course: Neural Networks https://karpathy.ai/zero-to-hero.html
16+
- Neural Networks: Zero to Hero https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ
17+
- MIT 6.S191 Introduction to Deep Learning https://www.youtube.com/playlist?list=PLtBw6njQRU-rwp5__7C0oIVt26ZgjG9NI
18+
- MIT Self Driving Lectures https://www.youtube.com/playlist?list=PLrAXtmErZgOeY0lkVCIVafdGFOTi45amq
19+
- Course: Introduction to ML https://www.youtube.com/playlist?list=PLuh62Q4Sv7BUhFs1KpeVctFmOYpTTxj33
20+
- Course: intro for probability for cs https://www.youtube.com/playlist?list=PLoROMvodv4rOpr_A7B9SriE_iZmkanvUg
21+
- course: pragmatic data science https://www.youtube.com/playlist?list=PLjxbCynJ0Gd8k75-zdMcqcEbH90jfk9eg
22+
- Standford Machine Learning Andrew NG https://www.youtube.com/watch?v=jGwO_UgTS7I&list=PLoROMvodv4rMiGQp3WXShtMGgzqpfVfbU&index=1
23+
- Course: 2020 Cornell CS 5787: Applied Machine Learning.
24+
https://www.youtube.com/playlist?list=PL2UML_KCiC0UlY7iCQDSiGDMovaupqc83
25+
- NN https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi
26+
- Video: Information Thoery, Pattern Recognition, and Neural Networks
27+
https://www.youtube.com/playlist?list=PLruBu5BI5n4aFpG32iMbdWoRVAA-Vcso6
28+
- MIT http://introtodeeplearning.com/
29+
https://www.youtube.com/playlist?list=PLtBw6njQRU-rwp5__7C0oIVt26ZgjG9NI
30+
- (Welch Labs) Neural Networks Demystified https://www.youtube.com/playlist?list=PLiaHhY2iBX9hdHaRr6b7XevZtgZRa1PoU
31+
- [Coursera] Neural Networks for Machine Learning — Geoffrey Hinton
32+
https://www.youtube.com/playlist?list=PLoRl3Ht4JOcdU872GhiYWf6jwrk_SNhz9
33+
- CS50's Introduction to Artificial Intelligence with Python 2020
34+
https://www.youtube.com/playlist?list=PLhQjrBD2T382Nz7z1AEXmioc27axa19Kv
35+
- https://www.youtube.com/playlist?list=pll8olhzgyoq7bkvburthesalr7bonzbxs
36+
- channel https://www.youtube.com/@vlavrenko/playlists
37+
- neural networls and backpropagation https://www.youtube.com/watch?v=jzyz0eupybi&list=plbv09bd7ez_4bs9j3o8l_ztjqzon_3oqs
38+
* channels
39+
- Deep Learning Book Lectures https://www.youtube.com/c/AlenaKruchkova/videos
40+
- Course: zero to neural networks https://www.youtube.com/@datacorelinux/videos
41+
- https://www.youtube.com/user/mathematicalmonk/videos
42+
- Video Channel
43+
https://www.youtube.com/c/ShashankKalanithiData/videos
44+
- will kwan https://www.youtube.com/c/willkwan/videos
45+
- instituto de calculo secretaria https://www.youtube.com/channel/uczcbeanqeihofbrbdyx1nea
46+
- https://www.youtube.com/c/sirajraval/videos
47+
- https://www.youtube.com/c/yannickilcher/videos
48+
- https://www.youtube.com/user/ministeriodeciencia/videos
49+
- https://www.youtube.com/channel/ucfxnrdbm1yrv9j2mb8aiy4q
50+
* video: practical deep learning for coders (2020)
51+
https://www.youtube.com/playlist?list=plfyubjixbdtrl3fmb3gowhri8ieu6fhfm
52+
** lesson 1 https://www.youtube.com/watch?v=_quexshfsa0
53+
- neural networks have limited range of things it can do with 1 layer (by minsky research)
54+
- but with more layers the problem is solved (also by minsky)
55+
- people used just 2 layers which limited performance
56+
- usinge more layers makes it "deep" learning
57+
58+
* video: 2011 - machine learning for the web - hilary mason
59+
** 1 introduction
60+
- classification problems:
61+
- clustering of categorical data
62+
- named entity disambiguation: separate different entities (similar)
63+
- recommendation systems
64+
- special data (uses domain knowledge)
65+
- geographic
66+
- timeseries
67+
- approaches from:
68+
- having data
69+
- having a problem
70+
- have infrastructure
71+
- methodology:
72+
1) obtain
73+
2) scrub
74+
3) explore
75+
4) model
76+
5) interpret
77+
** 2 classifying web documents - the theory
78+
- problems harder to classify (write logic)
79+
- supervised learning
80+
- examples:
81+
- spam classification
82+
- language identification
83+
- face detection
84+
- book: "data source handbook"
85+
- nyt has <meta> tags with information about the article.
86+

‎languages/julia.org‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
- 24 video Get Started with Julia Programming | Full Course https://www.youtube.com/watch?v=lZskxdMpYfE
12
- Article: 18 An introduction to the Julia language, part 1 https://lwn.net/Articles/763626/
23
- Article: 18 An introduction to the Julia language, part 2 https://lwn.net/Articles/764001/
34
- Article: 20 The accelerating adoption of Julia https://lwn.net/Articles/834571/

‎languages/sql/sqlite/self.org‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,19 @@
3737
$ sqlite3 foo.db .dump
3838
#+end_src
3939

40+
** flag
41+
|-------------+---+-----------------|
42+
| -csv | | .mode |
43+
| -html | | .mode |
44+
| -column | | .mode |
45+
| -list | | .mode |
46+
| -line | | .mode |
47+
| -vfs | S | |
48+
| -[no]header | | .headers on/off |
49+
| -separator | S | .separator S |
50+
| -cmd | S | run command |
51+
| -bail | | exit on error |
52+
|-------------+---+-----------------|
4053
** metacomands
4154
|------------+-------------------+---------------------------------------------------|
4255
| | ARGUMENT | DESCRIPTION |
@@ -50,12 +63,13 @@
5063
| .timer | on/off | show runtime after queries |
5164
| .import | --csv foo.csv bar | import csv file into "bar" table |
5265
| .tables | - | list all tables |
53-
| .output | FILE | redirects output to FILE |
5466
| .load | EXNAME | loads an ~extension~ |
5567
| .open | DBFILE | opens database when non was already opened |
5668
| .save | DBFILE | write in-memory db into FILE |
5769
| .separator | "¦" | change the separator for ".mode list" |
5870
| .eqp | on/off/full | explain query planner |
71+
| .output | FILE | redirects output to FILE |
72+
| .output | stdout | resets output |
5973
|------------+-------------------+---------------------------------------------------|
6074
| .mode | | change output format |
6175
| | list | DEFAULT |

‎terminal/shells/sh.org‎

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -71,15 +71,13 @@
7171
until ssh [email protected]; do sleep 3; done
7272
#+end_src
7373

74-
|-----------+-----------+---------------|
75-
| <c> | <c> | <l> |
76-
| | cmd | lines |
77-
|-----------+-----------+---------------|
78-
| take | head -n 5 | first 5 |
79-
| drop | tail -n+5 | skips first 4 |
80-
| take-last | tail -n 5 | last 5 |
81-
| drop-last | head -n-5 | skips last 5 |
82-
|-----------+-----------+---------------|
74+
|---------------+------------+-----------|
75+
| <c> | <c> | <c> |
76+
| first 5 | head -n 5 | take |
77+
| skips first 4 | tail -n +5 | drop |
78+
| last 5 | tail -n 5 | take-last |
79+
| skips last 5 | head -n -5 | drop-last |
80+
|---------------+------------+-----------|
8381

8482
* gotchas
8583

‎terminal/tools/parsers/self.org‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,14 @@
1+
- tool: tui https://www.visidata.org/
2+
- doc https://jsvine.github.io/intro-to-visidata/
3+
- graph https://www.visidata.org/docs/graph/
4+
- csv, json, sqlite
5+
- # number, % float, $ currency, @ date, ~ any, z~ remove
6+
- ! make key, - to hide
7+
- g. - to plot
8+
- Shift+C - Columns view
9+
- ^ edit column name
10+
11+
- tool: web ui https://datasette.io/
112
- tool: fake data generator https://github.com/multiprocessio/fakegen
213
- tool: yaml/json/xml/csv/toml https://github.com/TomWright/dasel
314
- tool: yaml/json/xml/csv/toml https://github.com/mikefarah/yq

‎terminal/tools/visualization/gnuplot/self.org‎

Lines changed: 64 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@
5555
- putting 2 newlines one after another, separate different blocks/groups of data
5656

5757
** commands
58+
5859
|-------+------------------------------------+-----------------------------|
5960
| <r> | | |
6061
| unset | OPTION | |
@@ -65,18 +66,31 @@
6566
| | | later can be referenced |
6667
| | | eg: S_min_x |
6768
|-------+------------------------------------+-----------------------------|
69+
6870
*** plot
6971

70-
- [re]plot [PLOT_A OPTS], [PLOT_B OPTS]
72+
|--------------+-------------------------|
73+
| <c> | |
74+
| INPUT | reads from... |
75+
|--------------+-------------------------|
76+
| "file.txt" | file contents |
77+
| "/dev/stdin" | standard input |
78+
| "< CMD" | CMD stdout |
79+
| "" | re-use last given input |
80+
|--------------+-------------------------|
81+
82+
- [re]plot INPUT [PLOT_A OPTS], INPUT [PLOT_B OPTS]
83+
- every N - plots every N data points
7184
- for [n = 1 : 5]
7285
- widths = "30 15 5"
7386
for [width in widths]
74-
- "+" (previous filename)
75-
- "filename.txt"
76-
- /dev/stdin
7787
- using N:N+1:N+n
78-
- smooth [bezier¦mcsplines]
88+
- [[http://www.gnuplot.info/docs/loc9016.html][smooth]]
89+
- [[http://www.gnuplot.info/docs/loc9099.html][frequency]] - points with the same X-value are replaced with 1 point with Y-values sum
90+
- [bezier¦mcsplines]
7991
- title S (for legend box)
92+
- bins=N - total N number of bins in graph http://gnuplot.info/docs_6.1/loc9125.html
93+
- binvalue=[avg¦sum] - how to group each bin, default: sum
8094
- index N
8195
- reference block N of data (aka separated by 2 \n)
8296
- blocks index start at 0
@@ -95,6 +109,8 @@
95109
- [l]ine[w]ith N
96110
- points
97111
- pointsize N
112+
113+
*** other
98114
- imagesc - matrix/heatmap - 2d contour
99115
- splot - 3d surface
100116
- hist
@@ -104,34 +120,36 @@
104120
*** general
105121

106122
#+CAPTION: S ="string", N=number, F=float
107-
|------------+------------+-----------------------------------|
108-
| <c> | <c> | |
109-
| table | "FILENAME" | save plot to file, unset it after |
110-
| encoding | utf8 | |
111-
|------------+------------+-----------------------------------|
112-
| title | S | the |
113-
| scale | N | |
114-
| output | S | saves into filepath at S |
115-
| monochrome | _ | |
116-
|------------+------------+-----------------------------------|
117-
| pause | N | seconds to pause |
118-
| +reread+ | - | REMOVED in 6.0 +reread input+ |
119-
|------------+------------+-----------------------------------|
123+
|------------+------------+-------------------------------------|
124+
| <c> | <c> | |
125+
| table | "FILENAME" | save plot to file, unset it after |
126+
| encoding | utf8 | |
127+
|------------+------------+-------------------------------------|
128+
| title | S | the |
129+
| scale | N | |
130+
| output | S | saves into filepath, unset it after |
131+
| monochrome | _ | |
132+
|------------+------------+-------------------------------------|
133+
| pause | N | seconds to pause |
134+
| +reread+ | - | REMOVED in 6.0 +reread input+ |
135+
|------------+------------+-------------------------------------|
120136

121137
*** general: file parsing
122138

123-
|-----------+-----------------+----------------------------------|
124-
| <c> | <c> | |
125-
|-----------+-----------------+----------------------------------|
126-
| datafile | separator S | |
127-
| | separator tab | |
128-
| | separator comma | input separator (default: space) |
129-
|-----------+-----------------+----------------------------------|
130-
| samples | N | sampling frequency |
131-
| [xyz]data | time | tells ? axis is a date/time |
132-
| timefmt | FMT | INPUT format date |
133-
| format | [x¦y] FMT | OUTPUT format string |
134-
|-----------+-----------------+----------------------------------|
139+
|-------------+-----------------+----------------------------------|
140+
| <c> | <c> | |
141+
|-------------+-----------------+----------------------------------|
142+
| datafile | separator S | |
143+
| | separator tab | |
144+
| | separator comma | input separator (default: space) |
145+
|-------------+-----------------+----------------------------------|
146+
| decimalsign | S | "." or "," |
147+
| samples | N | sampling frequency |
148+
| [xyz]data | time | tells ? axis is a date/time |
149+
| timecolumn | N FMT | INPUT format date on column N |
150+
| timefmt | FMT | INPUT format date |
151+
| format | [x¦y] FMT | OUTPUT format string, eg: %.2f |
152+
|-------------+-----------------+----------------------------------|
135153

136154
*** general: axis styling
137155

@@ -150,6 +168,7 @@
150168
| | rotate | |
151169
| | START,STEP,END | |
152170
| | (S N[,S N]) | custom labels S at point N |
171+
| [xyz]tic | rotate by N | rotates tic labels |
153172
| [xyz][2]label | S | axis label |
154173
| | offset N | |
155174
|---------------+----------------+----------------------------------|
@@ -167,20 +186,21 @@
167186

168187
*** style
169188

170-
|-----------+-------------+----------------------------------|
171-
| <c> | <c> | |
172-
| fill | transparent | |
173-
| | solid | |
174-
| solid | F | |
175-
| noborder | _ | |
176-
| data | lines | sets default for *plot* with ... |
177-
| | histograms | |
178-
| line | N | can be later referenced on plot |
179-
| histogram | gap N | |
180-
| | cluster | |
181-
| | clustered | |
182-
| | rowstacked | |
183-
|-----------+-------------+----------------------------------|
189+
|-----------+---------------+----------------------------------|
190+
| <c> | <c> | |
191+
| fill | transparent | |
192+
| | solid | |
193+
| solid | F | |
194+
| noborder | _ | |
195+
| data | lines | sets default for *plot* with ... |
196+
| | histograms | |
197+
| line | N | can be later referenced on plot |
198+
| histogram | gap N | |
199+
| | cluster | |
200+
| | clustered | |
201+
| | rowstacked | |
202+
| | columnstacked | |
203+
|-----------+---------------+----------------------------------|
184204

185205
#+CAPTION: gnuplot "test" command output
186206
#+ATTR_HTML: :width 500

‎terminal/tools/visualization/graphviz.org‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,8 @@ one binary for each layout filter
1818

1919
* language
2020

21+
https://www.graphviz.org/doc/info/lang.html
22+
2123
** layout filters
2224

2325
https://graphviz.org/docs/layouts/
@@ -67,5 +69,6 @@ https://graphviz.org/docs/layouts/
6769

6870
* codebases
6971

72+
NN https://www.geeksforgeeks.org/deep-learning/how-to-visualize-a-neural-network-in-python-using-graphviz/
7073
https://graphviz.org/Gallery/undirected/root.html
7174
http://www.yifanhu.net/GALLERY/GRAPHS/

0 commit comments

Comments
 (0)