Scimax - The manual

\maketitle

Scimax is an Emacs starterkit designed for people interested in reproducible research and publishing. Scimax is just Emacs that has been configured extensively to make it act like we need it to for research documentation and publication. The notes below document some features we have built into scimax.

Note: The org-mode version of this manual is the preferred version for reading; it is the most functional version, and the most up to date. The info:scimax version is also useful though, and you might prefer that one.

Org-mode

scimax provides a lot of customization of org-mode. Here are some of the more important ones.

Markup shortcuts

scimax defines some shortcuts for org-mode markup. You can select text and type the shortcut to wrap the region in the markup, or if on a word, type the shortcut to wrap the word, or type the short cut to create the markup with the cursor inside it to keep typing.

Note these modifier keys:

S	shift
M	alt/meta
C	Ctrl
s	super
H	hyper

Notation like s-b means press super and b together.

super and Hyper aren’t common on all keyboards, and using them is platform dependent. One good resource for configuring them is http://ergoemacs.org/emacs/emacs_hyper_super_keys.html.

For Mac, I use:

(setq mac-right-command-modifier 'hyper)
(setq mac-right-option-modifier 'super)

For Windows you can use:

(setq w32-pass-lwindow-to-system nil)
(setq w32-lwindow-modifier 'super) ; Left Windows key

(setq w32-pass-rwindow-to-system nil)
(setq w32-rwindow-modifier 'super) ; Right Windows key

(setq w32-pass-apps-to-system nil)
(setq w32-apps-modifier 'hyper) ; Menu/App key

It was suggested here that you can also make some seldom used keys like <pause> and <scroll> act like these extra modifiers. Here is a snippet that will do that.

(define-key function-key-map (kbd "<pause>") 'event-apply-super-modifier)
(define-key function-key-map (kbd "<scroll>") 'event-apply-hyper-modifier)

This does not work for all key combination with the Windows key because Windows intercepts some keys. The intercepted keys depend on the Windows version:

Key	Windows 7	Windows 10
A	+	w
B	w	w
C
D	w	w
E	w	w
F	w	w
G	+	w
H		w
I	+	w
J	+
K	+	w
L	w	w
M	w	w
N	+	+
O	+
P	w	w
Q	+	w
R	w	w
S	+	w
T	w	w
U	w	w
V
W	+	w
X	+	w
Y	+	+
Z	+
0 … 9	w	w
+	w	w
-	+	+
_	+	+
.	+
,	+
#	+
*	+	+

+: Key is available for Emacs
w: Key is intercepted by Windows
empty: No reaction from Emacs or Windows

The following keyboard shortcuts are defined:

markup	keyshortcut	emacs-command
Bold	s-b	`org-bold-region-or-point’
italics	s-i	`org-italics-region-or-point’
`verbatim`	s-v	`org-verbatim-region-or-point’
`code`	s-c	`org-code-region-or-point’
underline	s-u	`org-underline-region-or-point’
~~strike~~	s-+	`org-strikethrough-region-or-point’
_subscript	s–	`org-subscript-region-or-point’
^superscript	s-=	`org-superscript-region-or-point’
$equation$	s-4	`org-latex-math-region-or-point’
$inline eqn$	C-u s-4
@@latex:snippet@@	C-u C-u 4
°	s-e	`ivy-insert-org-entity’

Use `ivy-insert-org-entity’ (s-e) for all accented characters.

Template expansions in org-mode

The following examples show some of the shortcuts defined in scimax for expansion. Put your cursor after the shortcut and press tab to expand them. These are done with yasnippets (defined in ./snippets/).

<ip


<p



<el


<sh


<ti
#+title:

<ok   org-keywords with completions
#+ARCHIVE:

lh
#+latex_header:

lhp   latex header with completion on usepackage
#+latex_header: \usepackage{12many}


lc
#+latex_class:

lco
#+latex_class_options:

ao
#+attr_org: :width 300

al
#+attr_latex: :placement [H] :width 300

ca
#+caption:

ncf
#+name: name
#+caption: caption
[[...]]

nct
#+name: name
#+caption: caption
|  |

<n
#+name:

o
#+options:

oo (org-mode options with completion
toc:

<<    noweb with completion
<<nil>>

Here are some convenient table expansions.

<t
|  |

<tt
|  |  |

<ttt
|  |  |  |


<tttt
|  |  |  |  |

<tn (enter number of columns)
|  |  |  |  |  |  |  |  |

There are some specific snippets to

emacs-lisp ./snippets/emacs-lisp-mode/
python ./snippets/python-mode/

and in org-mode:

./snippets/org-mode/
- ./snippets/org-mode/org-ref/ (bibliography, biliographystyle, cites and ref links)
- ./snippets/org-mode/src-blocks/ (src blocks)
- ./snippets/org-mode/blocks/ (quotes, verses, etc.)

Latex

For numbered LaTeX equations, scimax will make sure that the overlays have correct numbers on them.

Latex classes that scimax knows about

Here is a list of installed latex classes.

(mapcar 'car org-latex-classes)

Better image scaling

Image rescaling in org-mode/Emacs used to require an Emacs with imagemagick support compiled in. This is a problem on Windows. Scimax support image rescaling with an external imagemagick program called mogrify. You can rescale the appearance of an image in org-mode by adding an attribute like this above the figure.

where resize-option is one of:

N%	to scale the image by a percentage.
N	to set the width, keeping the aspect ratio constant.
xN	to set the height, keeping the aspect ratio constant.
NxM!	to set the width and height, ignoring the aspect ratio.

Note that you have to have a line like this in your init file for this to work. This is the default in scimax, and if you set it to t, then scaling will not work (since this variable when set to t says to use the actual image width).

(setq org-image-actual-width nil)

Here are some examples.

new speed commands

scimax defines these new speed commands that are active when the cursor is on the first character of a headline.

m	Mark the subtree
S	widen
k	kill the subtree
q	jump to a headline with avy
T	org-teleport (move headline)

The best way to see other speed commands is to put your cursor at the beginning of a headline and press ?. You can also run `org-speed-command-help’ to see a full list of speed commands.

Formatted copy and paste

`ox-clip-formatted-copy’ provides a way to copy org-mode with formatting so it can be pasted into other programs like MS Word and web browsers. It does this by copying the selected text to the HTML clipboard.

Writing tools

Spell-checking

scimax is configured with flyspell and flycheck on. When you misspell a word it will be underlined in red, and you will see a message in the minibuffer that tells you how to fix it (C-;). Type that, and you will be able to fix the word spelling without losing your place! Not only that, but the correction will be saved as an abbreviation so it will auto correct every time you make the mistake again! Note this only works for single word corrections (i.e. the correction cannot be a two word correction).

If the word is correct, you can type M-o s to save the word in your dictionary so you will not see it as misspelled again.

Note: you need to have hunspell installed for this to work. On Windows, this should get installed into the scimax root directory in the emacs-win directory. For Mac and Linux, you need to install it using your package manager.

Autoformat

Scimax will auto-format several things for you.

Ordinal numbers, e.g. 1st to 1^st.
Fractions: 1/4 to ¼
Some superscripts, e.g. cm2 to cm²
Transposed cAps, e.g. tHe to The

These are controlled by the following variables you can customize (they default to t):

`scimax-autoformat-ordinals’
`scimax-autoformat-fractions’
`scimax-autoformat-superscript’
`scimax-autoformat-transposed-caps’

To get the autoformatting you have to enable `scimax-autoformat-mode’. If you want it on all the time, add something like this to your init files:

(add-hook 'org-mode-hook 'scimax-autoformat-mode)

scimax also defines some abbreviations that you can toggle on and off. The abbreviations are defined in these variables:

Auto-capitalization of weekdays and months, e.g. Monday and June.
1. `scimax-month-abbreviations’
2. `scimax-weekday-abbreviations’
Contraction expansion: cant to can not and can’t to can not
1. `scimax-contraction-abbreviations’
Commonly transposed letters in words: teh to the
1. `scimax-transposition-abbreviations’
Some common chemicals like co2 to CO₂
1. `scimax-chemical-formula-abbreviations’
Some convenience symbols like degC to °C and Ang to Å, and some names like norskov to Nørskov.
1. `scimax-misc-abbreviations’

You should be able to undo any expansion with C-/. Alternatively you can prevent the expansion by typing C-q after the abbreviation before the next character is typed.

These are not on by default. You have to enable them with commands like the one below in your init file. Use +1 to enable the abbrevs, and -1 to disable the abbrevs that are defined in the variable.

(scimax-toggle-abbrevs 'scimax-month-abbreviations +1)

`scimax-toggle-abbrevs’ is an interactive command you can use to toggle the abbreviations on and off. If you run that command it will toggle the state of the abbrevs. With a single prefix arg it will turn them on, and with a double prefix arg turn them off.

You can see the abbreviations defined with this command elisp:edit-abbrevs.

Track changes

Scimax provides some support for track changes and edit marks in org-mode through the new ./scimax-editmarks.org project.

{>+Add this text.+<}{>-Delete this-<} {>~ @jk Here is a comment~<}

This is a {>$tpyo$<}

There are persistent highlights also.

{g>Green highlight<g} {b>blue highlight<b} {y>yellow<y} {p>pink<p}

There is a hydra menu bound to H-m that provides access to easily insert these markups.

There is limited support to convert these editmarks to LaTeX and html, and to generate these editmarks from git diffs or diffs with the copy on disk.

There is also an experimental feature for track-change mode.

Note: This library replaces several older attempts at this which are no longer maintained by me:

ov-highlight
org-editmarks.el
cm-mods.el

org-ref

org-ref is the answer to citations and bibliographies in scientific writing. Run `org-ref-help’ and read it.

Publishing (ox-manuscript)

The key-binding C-c C-e j should enter the ox-manuscript export menu. This process differs from the regular export process in a number of ways. It will detect if bibtex, makeindex, or makeglossary, and if minted is used, -shell-escape will automatically be used with pdflatex. The function `ox-manuscript-latex-pdf-process’ handles all of that.

Additionally, you can generate different outputs of an org-file:

`ox-manuscript-build-submission-manuscript-and-open’: creates a standalone tex file with embedded bibliography, and image extensions stripped, and the pdf file.
`ox-manuscript-make-submission-archive’: creates a directory containing all the files you normally need for submission.
`ox-manuscript-toggle-interactive-build’: if you are having trouble building a PDF, this will show you what happens at each step.

`ox-manuscript-nobibliography’

`ox-manuscript-texcount’: Estimate how many words are in your manuscript. For when you are limited to a fixed number of words.

Manuscript templates

We have templates prepared for the following manuscripts, proposals and documents.

(mapcar (lambda (x) (list (plist-get x :template))) (ox-manuscript-candidates))

`ox-manuscript-new-ivy’
`ox-manuscript-new-helm’

Bibliography management

Bibliographies in scimax are stored in bibtex files. A bibtex file is a plain text file containing bibtex entries. Each entry describes an item. Here is a typical example.

@article{kitchin-2015-examp,f
  author =	 {Kitchin, John R.},
  title =	 {Examples of Effective Data Sharing in Scientific Publishing},
  journal =	 {ACS Catalysis},
  volume =	 {5},
  number =	 {6},
  pages =	 {3894-3899},
  year =	 2015,
  doi =		 {10.1021/acscatal.5b00538},
  url =		 { http://dx.doi.org/10.1021/acscatal.5b00538 },
  keywords =	 {DESC0004031, early-career, orgmode, Data sharing },
  eprint =	 { http://dx.doi.org/10.1021/acscatal.5b00538 },
}

This entry identifies the item as an article, labels it with a key (kitchin-2015-examp), and describes the details of the item in a series of key = {value} lines. You can learn more about bibtex here http://www.bibtex.org.

If you know the DOI I recommend you use `doi-add-bibtex-entry’ as much as possible to add bibtex entries to your bibliography files. This will add properly formatted and cleaned entries and download the pdf if it knows how. You may also find `crossref-add-bibtex-entry’ useful if you have a freeform citation and want to search for it.

In a bibtex file, `org-ref-bibtex-hydra/body’ will give you a menu of options to do things on an entry including:

Search pubmed, WebOfScience (wos), wos-citing, wos-related, crossref, and google scholar using the DOI or title of the entry.
Clean the entry, replace non-ascii characters, sort the entry fields, or change the case of the title.
Update the entry or fields using the doi.
Open the notes, pdf, or url associated with the entry.
Email the entry to someone
Copy or cut the entry to paste somewhere
Copy a formatted bibliography entry
Add tags to an entry
Add a new entry

`org-ref-bibtex-file/body’ provides menu access to bibtex file functions:

`bibtex-validate’ - Checks if the file is syntactically valid, and for duplicate keys.
`bibtex-sort-buffer’
`bibtex-reformat’
`bibtex-count-entries’
`org-ref-build-full-bibliography’

Searching your bibliography files

I like `helm-bibtex’. You type C-SPC to mark multiple entries. Type TAB to see what actions you can perform on the entries.

See `org-ref-help’ for information on using org-ref to insert citations.

Bibtex entries

Here are the bibtex entry types and fields you should use. You can add extra fields like keywords, doi, url, notes, etc… to each entry. Usually they will be ignored by bibtex, but they are useful for you. Note that biblatex is considered a more powerful bibliography formatting tool, but we use bibtex because that is what most of the publishers we use support.

(loop for (type doc required crossref optional) in bibtex-BibTeX-entry-alist
      do
      (princ (format "\n*** %s (%s)\n" type doc))
      (princ "\n**** Required fields\n\n")
      (loop for field in required
	    do
	    (princ (format "- %s" (car field)))
	    (message "%s" field)
	    (if (>= (length field) 2)
		(princ (format " :: %s\n" (nth 1 field)))
	      (princ "\n")))

      (princ "\n**** Optional if Crossref present but otherwise required fields\n\n")
      (loop for field in crossref
	    do
	    (princ (format "- %s" (car field)))
	    (if (>= (length field) 2)
		(princ (format " :: %s\n" (nth 1 field)))
	      (princ "\n")))
      (princ "\n**** Optional fields\n\n")
      (loop for field in optional
	    do
	    (princ (format "- %s" (car field)))
	    (if (>= (length field) 2)
		(princ (format " :: %s\n" (nth 1 field)))
	      (princ "\n"))))

Article (Article in Journal)

Required fields

author

title: Title of the article (BibTeX converts it to lowercase)

Optional if Crossref present but otherwise required fields

journal
year

Optional fields

volume: Volume of the journal
number: Number of the journal (only allowed if entry contains volume)
pages: Pages in the journal

month
note

InProceedings (Article in Conference Proceedings)

Required fields

author

title: Title of the article in proceedings (BibTeX converts it to lowercase)

Optional if Crossref present but otherwise required fields

booktitle: Name of the conference proceedings

year

Optional fields

editor

volume: Volume of the conference proceedings in the series
number: Number of the conference proceedings in a small series (overwritten by volume)
series: Series in which the conference proceedings appeared
pages: Pages in the conference proceedings

month
address

organization: Sponsoring organization of the conference
publisher: Publishing company, its location

note

InCollection (Article in a Collection)

Required fields

author

title: Title of the article in book (BibTeX converts it to lowercase)
booktitle: Name of the book

Optional if Crossref present but otherwise required fields

publisher
year

Optional fields

editor

volume: Volume of the book in the series
number: Number of the book in a small series (overwritten by volume)
series: Series in which the book appeared
type: Word to use instead of “chapter”
chapter: Chapter in the book
pages: Pages in the book
edition: Edition of the book as a capitalized English word

month
address
note

InBook (Chapter or Pages in a Book)

Required fields

author: nil
editor: nil
title: Title of the book
chapter: Chapter in the book

Optional if Crossref present but otherwise required fields

publisher
year

Optional fields

volume: Volume of the book in the series
number: Number of the book in a small series (overwritten by volume)
series: Series in which the book appeared
type: Word to use instead of “chapter”

address

edition: Edition of the book as a capitalized English word

month

pages: Pages in the book

note

Proceedings (Conference Proceedings)

Required fields

title: Title of the conference proceedings

year

Optional if Crossref present but otherwise required fields

Optional fields

booktitle: Title of the proceedings for cross references

editor

volume: Volume of the conference proceedings in the series
number: Number of the conference proceedings in a small series (overwritten by volume)
series: Series in which the conference proceedings appeared

address
month

organization: Sponsoring organization of the conference
publisher: Publishing company, its location

note

Book (Book)

Required fields

author: nil
editor: nil
title: Title of the book

Optional if Crossref present but otherwise required fields

publisher
year

Optional fields

volume: Volume of the book in the series
number: Number of the book in a small series (overwritten by volume)
series: Series in which the book appeared

address

edition: Edition of the book as a capitalized English word

month
note

Booklet (Booklet (Bound, but no Publisher))

Required fields

title: Title of the booklet (BibTeX converts it to lowercase)

Optional if Crossref present but otherwise required fields

Optional fields

author

howpublished: The way in which the booklet was published

address
month
year
note

PhdThesis (PhD. Thesis)

Required fields

author

title: Title of the PhD. thesis
school: School where the PhD. thesis was written

year

Optional if Crossref present but otherwise required fields

Optional fields

type: Type of the PhD. thesis
address: Address of the school (if not part of field “school”) or country

month
note

MastersThesis (Master’s Thesis)

Required fields

author

title: Title of the master’s thesis (BibTeX converts it to lowercase)
school: School where the master’s thesis was written

year

Optional if Crossref present but otherwise required fields

Optional fields

type: Type of the master’s thesis (if other than “Master’s thesis”)
address: Address of the school (if not part of field “school”) or country

month
note

TechReport (Technical Report)

Required fields

author

title: Title of the technical report (BibTeX converts it to lowercase)
institution: Sponsoring institution of the report

year

Optional if Crossref present but otherwise required fields

Optional fields

type: Type of the report (if other than “technical report”)
number: Number of the technical report

address
month
note

Manual (Technical Manual)

Required fields

title: Title of the manual

Optional if Crossref present but otherwise required fields

Optional fields

author

organization: Publishing organization of the manual

address

edition: Edition of the manual as a capitalized English word

month
year
note

Unpublished (Unpublished)

Required fields

author

title: Title of the unpublished work (BibTeX converts it to lowercase)

note

Optional if Crossref present but otherwise required fields

Optional fields

month
year

Misc (Miscellaneous)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

author

title: Title of the work (BibTeX converts it to lowercase)
howpublished: The way in which the work was published

month
year
note

Bibtex hotkeys

Scimax provides hotkeys for when your cursor is on the @ at the beginning of an entry. Here are the currently defined hotkeys.

(require 'bibtex-hotkeys)
(loop for (key . desc) in bibtex-hotkeys collect (list key desc))

Email utilities

`email-buffer’: email the whole buffer
`email-region’: emails selected region
`email-heading’: email the current heading, including properties, deadlines, etc…
`email-heading-body’: email just the body of the current heading
`email-bibtex-entry’: email the bibtex entry at point

html mail

You can send html email from org-mode.

From an org-file run `org-mime’. You will be prompted for the scope to send, which is either the whole buffer, the heading you are in, or the selected region. Then you will select how to make the email: html will export the text to html and put it in an email, and htmlize will use the htmlize library to generate the html.

Alternatively, you may want to type org-mode directly in an email buffer. Use `org-mime-compose-mail’ to do that.

mail merge

scimax provides some tools to do a mail merge in org-mode. The idea is to run `mail-merge-make-headings’ with a template and data-source to generate a series of org-mode headings that are the messages. You can inspect these, edit them if needed, and then run `mail-merge’ to actually send them. Each heading will be tagged as sent, and marked DONE.

Contacts

scimax provides a emacs-lisp library to interface with a contact database written in org-mode.

First, add some org-files to the variable `contacts-files’. Any headline in these files that has an EMAIL property will be considered a contact. Then, you can search your database with `ivy-contacts’ or `helm-contacts’. There are a variety of actions to choose from ranging from inserting email addresses, copying properties, opening contact urls, etc…

See `contacts-help’ for more information.

Google

google-this is installed in scimax. You can use these commands to search for things from Emacs:

C-c / SPC `google-this-region’ C-c / a `google-this-ray’ C-c / c `google-this-translate-query-or-region’ C-c / e `google-this-error’ C-c / f `google-this-forecast’ C-c / g `google-this-lucky-search’ C-c / i `google-this-lucky-and-insert-url’ C-c / l `google-this-line’ C-c / m `google-maps’ C-c / n `google-this-noconfirm’ C-c / r `google-this-cpp-reference’ C-c / s `google-this-symbol’ C-c / t `google-this’ C-c / w `google-this-word’ C-c / <return> `google-this-search’

Magit

Magit is a front-end for git version control. There is a good manual here: info:magit#Top

Basic magit

Create a git repo

info:magit#Repository setup You can create a git repo with M-x magit-init. This will create a git-repo in the current directory.

(magit-init)

help:magit-init

Clone a repo

info:magit#Repository setup

M-x magit-clone

This will prompt you for a repo, which is either a url, or a path, and a path to clone it to.

help:magit-clone

Check the status of your repo

info:magit#Status buffer

Run M-x magit-status to see the status of your repo.

Press “g” in the window to refresh it.

press “n” (next) or “p” (previous) to navigate in this window.

help:magit-status

Stage a file

info:magit#Staging and unstaging

In the magit-status window, put your cursor on an unstaged file and press “s”.

If you press TAB on the file, it will expand to show the changes that are unstaged. Deletions show in red, and additions in green. The changes are in “hunks”.

You can unstage a file with “u”

Commit a file

info:magit#Initiating a commit

info:magit#Editing commit messages

In the magit-status window with some files that are staged, press “c”, review the options, and probably press “c” again. Enter a commit message and type “C-c C-c” to commit it, or “C-c C-k” to cancel it.

Diffs

info:magit#Diffing

From the magit-status window, press “d” then “d” to see what has changed.

See the log

info:magit#Logging

In the magit-status window press “l”, review the options, and press “l” again.

If you want to see only the commits that affected a file, in the magit-status window press “l” then “=f”, enter the filename, and then press “l” again.

Push

info:magit#Pushing

In the magit-status window press “P” then “p”.

Note that tags don’t normally get pushed, but there are options (“T” to push a tag, and “t” to push all tags).

Pull

info:magit#Pulling In the magit-status window press “F” then “p”.

Run a command-line git command manually

info:magit#Running Git manually In the magit-status window, type “!” to get the popup and choose what you want to do (e.g. where to run the command, etc… You do not need to type “git” in the command. Note you can also run a shell command from this interface.

Check the output of the git command

Press “$”

Keybindings

info:magit#Keystroke Index

Intermediate concepts

Checkout an older version of a file

Use M-x magit-checkout-file select the branch, or enter a revision, and then choose a file.

help:magit-checkout-file

help:magit-find-file help:magit-find-file-other-window

Search the commit messages for a pattern

In a magit-status window press “l =g” enter a pattern to grep for, and then press “l”.

Revert a commit

info:magit#Reverting

Got to the log, select the commit and type “V” then “V”.

Tag a version

info:magit#Tagging

press “t” in the magit-status window. You can then create a tag, annotate it, delete tags, and prune them.

Checkout an existing branch.

info:magit#The branch popup

In the magit-status window press “b” then “b” and choose the branch.

To checkout a new branch, in the magit-status window press “b” then “c”. Choose the branch to start from then a name for the new branch.

Merge two branches

info:magit#Merging

In the magit-status window press “m”, then “m” and select the branch to merge into the current one.

Resolving conflicts

info:magit#Resolving conflicts

You will probably also want to get familiar with info:ediff#Top.

On a file in a magit-status window, press “e” to enter the 3-window ediff view. The A window is the version at HEAD, the B window is what is in the index, and the C window is the current version.

Fetching

info:magit#Fetching

In the magit-status window press “f”.

Add a remote

info:magit#Remotes

M-x magit-remote-add then enter an alias, and the url.

Stashing

info:magit#Stashing

Press “z” in the magit-status window

Git blame

Advanced concepts

Resetting

info:magit#Resetting

Rebasing

info:magit#Rebasing

Interactve rebasing

Open the log, select the oldest commit you want to rebase on then press “r” and then “i”. Use M-p and M-n to move commits around. Press “s” on any commits you want to squash into the commit above it. C-c C-c will start the commands.

From the magit-status on unpushed commits, you can also press “r” to get the rebase popup.

Reword a commit message

“r w” allows you to reword the commit message.

Create patches

info:magit#Creating and sending patches

In magit-status window, press “W”

“W p” creates patches “W r” makes a pull request. This just creates an email with information in it. It is not a GitHUB request, and it is only useful if there is a public, external copy of the repo.

Cherry-picking

info:magit#Cherry picking

Press “A”

Apply patches

info:magit#Applying patches

Notes about commits

info:magit#Notes

Press “T” to attach a note.

A typical use of notes is to supplement a commit message without changing the commit itself. Notes can be shown by git log along with the original commit message. To distinguish these notes from the message stored in the commit object, the notes are indented like the message, after an unindented line saying “Notes (<refname>):” (or “Notes:” for refs/notes/commits).

Cherry-picking

info:magit#Cherry picking

Project management - projectile

https://github.com/bbatsov/projectile for project management. A project is basically a directory under version control, e.g. git.

Projectile makes it easy to jump to projects, find files in projects, search projects, etc…

C-c p p	Switch to a project
C-c p k	Kill project buffers

You can see all the key bindings with C-c p C-h.

While in a project, you may want to try:

`helm-projectile-grep’ or `counsel-git-grep’ to search all project files for a phrase.

Programming

Python

scimax is setup with `elpy-mode’.
`pydoc’ provides nice, hyperlinked documentation for python.

Asynchronous Python

You can run python blocks asynchronously with M-x `org-babel-async-execute:python’ with the cursor in a code block. This will allow you to keep typing, and show you a buffer with the progress of your code block. When it is done, the results will be inserted into the buffer where it belongs when the job is done. A temporary hash string will go in the results. That hash will be replaced when the calculation is done.

You can make this the default behavior by adding this to your init file:

(add-to-list 'org-ctrl-c-ctrl-c-hook 'org-babel-async-execute:python)

Jupyter/Ipython

If you like sessions in Python, the ob-ipython library is better than the default ob-python in org-mode.

Pygments doesn’t support ipython out of the box for some reason, which is a problem if you want to export your src block to LaTeX. scimax fixes this for you and automatically installs this if you don’t already have it.

ob-ipython allows you to use Ipython magic commands in your src blocks. Here is a protypical Ipython src block with a line magic.

%time print("hello world")
a = 6

And a block with cell magic.

%%timeit
7

Scimax enhancements to ob-ipython

We have made a few improvements to ob-ipython (see ./scimax-ipython.org for much more detail). Inline images are supported now similar to how they are supported in the Jupyter notebook. You specify the %matplotlib inline magic, and then plots will appear “inline” along with any output from your cell block. The plots are saved in a directory ipython-inline-images in filenames derived from and md5 hash of the image. You can have more than one image, and you also get the output from your block, similar to the way the Jupyter notebook behaves.

print('Hello world!')

%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

x = np.linspace(0, 20 * np.pi, 200)
y = np.exp(-0.1 * x) * np.sin(x)
plt.plot(x, y)
plt.xlabel('x')
plt.ylabel('y')
plt.title('Decaying sin wave')

# new plot
plt.figure()
y2 = np.exp(-0.1 * x) * np.cos(x)
plt.plot(x, y2)
plt.xlabel('x')
plt.ylabel('y')
plt.title('Decaying cosine')

It also works with :results set to value.

%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

x = np.linspace(0, 20 * np.pi, 200)
y = np.exp(-0.1 * x) * np.sin(x)
plt.plot(x, y)
plt.xlabel('x')
plt.ylabel('y')
plt.title('Decaying sin wave')

a = 5
b = 6
a + b

We also support HTML and Latex outputs like this. The results will contain an HTML or LaTeX block. As with the Jupyter notebook

from IPython.display import HTML, Latex

HTML('H<sub>2</sub>O')

and

Latex('H$_2$O')

As with the Jupyter notebook, only the last returned cell is rendered.

Asynchronous evaluation

We finally have asynchronous execution of ipython blocks. This allows you to run a block, and keep working in Emacs. A temporary string is inserted into the results which is replaced when the code is done running. This command will make a clickable link to interrupt the kernel, or cancel the cell running. This will put names on every src block if they are not already named.

import time
time.sleep(5)
#print(8)

%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np

x = np.linspace(0, 20 * np.pi, 200)
y1 = np.exp(-0.1 * x) * np.sin(x)
y2 = np.exp(-0.1 * x) * np.cos(x)
plt.plot(y1, y2)
plt.xlabel('x')
plt.ylabel('y')
plt.title('Decaying spiral')

<Figure size 432x288 with 1 Axes>

You can run all the blocks asynchronously with `org-babel-execute-ipython-buffer-async’. You can clear the queue `org-babel-async-ipython-clear-queue’.

Export org to ipynb

Finally, we can export org-mode files to Jupyter notebooks with the ox-ipynb library. If you load the library, there is a new export option in the export menu with the letter “n”. Or you can use `ox-ipynb-export-to-ipynb-file-and-open’.

Using other kernels - hy

Amazing. You can use other language kernels with ob-ipython.

scimax provides the jupyter-hy src block to run hylang in src blocks. The required :session and :kernel headers are automatically provided.

(print "hello world")
(import time)
(print (time.asctime))

Emacs-lisp

`lispy-mode’ is just amazing.
`scimax-org-eldoc’ will generate documentation on elisp libraries in an org-mode format. It is a library you have to require.

RSS feeds

It is a major challenge to keep up with the scientific literature. `elfeed’ is the package we use in scimax for that. It aggregates RSS feeds and provides a pretty easy way to consume them, capture them in to org-mode, search them, and do things with them. scimax preconfigures elfeed with some python, and emacs feeds, and you can easily add new feeds:

The scimax customizations of elfeed are not loaded by default. If you want them, you have to add this to your init files.:

(require 'scimax-elfeed)

(add-to-list 'elfeed-feeds "http://feeds.feedburner.com/acs/accacs")

or if you want entries from a feed to be automatically tagged, e.g. anything from Nature magazine could be tagged with nature:

(add-to-list 'elfeed-feeds '("http://feeds.nature.com/nchem/rss/current" nature))

Elfeed is configured to run every half hour after Emacs is started.

On an entry you can type:

e	email entry to someone
c	capture the entry to org-mode
d	add bibtex entry

In the list of entries you can type:

f,j	marks entry as read
o	open the entry
b	open browser to entry url
s	start a search query (see https://github.com/skeeto/elfeed#filter-syntax)

Miscellaneous scimax utilities

Open a bash window

`bash’ will open an external bash terminal in the current working directory.

Open Finder/Explorer

`explorer’ and `finder’ will open a Windows Explorer or Mac Finder window in the current working directory.

words

Try out `words-hydra/body’ on a selection or word. I bound it to \[words-hydra/body].

ore

This command: `ore’ tells you about the org-element your point is on. It gives some hints on commands you can use on the element, and provides a way for you to write your own notes.

org-show

This is a simple library to make simple presentations in org-mode. See `org-show-help’ for an example use.

org-db

`org-db’ is an org-mode database. When it is active every org-mode file you visit will be indexed into a sqlite database. In each file, each headline with its title, tags and properties are stored, and every link in each file is stored.

This becomes useful because you can then search all of your org-files and jump to different locations.

`org-db-headings’: Jump to a heading
`org-db-contacts’: jump to a heading with an email property
`org-db-files’: open a file in the db
`org-db-recent-files’: open a list of recent files from the db
`org-db-locations’: open a heading with an address property
`org-db-hashtags’: open a hashtag.
???

You can add a lot of files with `org-db-index’.

When active, every time you save an org-file it will be added to a queue to be indexed during idle time.

External Packages

These are external packages that are included in scimax and might be useful for you.

avy

avy lets you jump to things in Emacs. See https://github.com/abo-abo/avy

There are a lot of avy commands. Click this to see them: elisp:(apropos-command “^avy”)

We use a lot of them in Navigation - navy.

counsel

This is for completing stuff. See http://oremacs.com/2015/04/09/counsel-completion/

In particular, scimax sets these keybindings:

(“M-x” . counsel-M-x) (“C-x b” . ivy-switch-buffer) (“C-x C-f” . counsel-find-file) (“C-h f” . counsel-describe-function) (“C-h v” . counsel-describe-variable) (“C-h i” . counsel-info-lookup-symbol) (“H-c r” . ivy-resume) (“H-c l” . counsel-load-library) (“H-c g” . counsel-git-grep) (“H-c a” . counsel-ag) (“H-c p” . counsel-pt)

helm

Helm is another completion tool. See https://github.com/emacs-helm/helm

Type C-c h C-h to see the helm key bindings. There are so many good things in there!

swiper

This is a powerful search tool in Emacs. See https://github.com/abo-abo/swiper

C-s is bound to `counsel-grep-or-swiper’ for searching.

undo-tree

There are a few undo features:

C-/	undo the last action
C-x u	use the undo-tree (q to quit)

Scientific notebook

scimax provides a scientific notebook capability. Each “notebook” is actually collection of org-files in a “project”. A project is the set of files in a directory that is under git version control. Each project should have a master file (the default is README.org, but you can customize `nb-master-file’ to change this). The master file contains what ever you want, but typically it links to other documents in the project and provides an overview of the project.

You are basically free to structure the notebook however you want. You have all the freedom of org-mode at your fingers to document your work.

We leverage projectile for project management in the notebook. We use magit for version control.

Use `nb-new’ to create a new project. You will be prompted for a name, which must be a valid directory name. The directory will be created in `nb-notebook-directory’. Note that all git repos will be considered projects, so it is not necessary to use `nb-new’. It just automates a few things for you.

Use `nb-open’ to open a project. This will open the project to your master file. Previously visited projects are remembered by projectile and should be shown in an ivy completion minibuffer for selection.

Probably you will keep your projects separate from your agenda files, but you still would like to see what tasks the project has? Use `nb-agenda’ while in your project, and it will show you all the tasks in the org-files associated with the project.

Here are some other interesting commands you may want to use.

`counsel-git-grep’: grep for a string in the project
`projectile-find-file’: jump to a file in the project
`projectile-switch-to-buffer’: switch to a project buffer
`projectile-kill-buffers’: kill all the buffers associated with the project

These commands help you navigate to a headline.

`ivy-org-jump-to-heading’: in the current file
`ivy-org-jump-to-heading-in-directory’: in the current directory
`ivy-org-jump-to-project-headline’: in the project
`counsel-org-tag’: add/remove tags on a headline

You can manage the version control with Magit. There are also keyboard shortcuts for version control. Type C-x v C-h to see them.

`vc-next-action’ will do the next logical thing for vc, e.g. add or commit.
`vc-diff’ will show you what has changed in the buffer since the last commit.
`vc-print-log’ will show you the vc log.

scimax defines these additional key bindings:

C-x v p	git push
C-x v P	git pull

Archive the notebook with git

Sometimes you may want a zip file of your notebook. You can use the `nb-archive’ command to create a zip file of the current state of your notebook. This can be useful to back up versions of your project, send copies to collaborators who don’t use git, or to create archives for data sharing and supporting information.

Note that only files under version control are archived, and only the current committed state is archived. So, uncommitted files and changes will not be included.

ivy/counsel

scimax currently uses ivy extensively for completion, and enables `ivy-mode’ and `counsel-mode’. The default matching behavior is `ivy–regex-ignore-order’.

When you get an ivy minibuffer, start typing to select what you want, and when it is highlighted, press enter.

There are a couple of nuances though.

To eliminate matches use ! pattern, but you can only use one of these, and it comes last.

I have extended the `ivy-minibuffer-map’ to enable the following:

M-spc will show the actions, and spc to resume the selection process.
C-RET to perform the action and move to the next candidate, and C-u C-RET to perform the action and move to the previous candidate.
M-RET will perform the default action on every candidate from the current selection to the end of the candidates. C-u M-RET does that from the current candidate to the beginning.
s-RET quits the selection buffer with no action.
? shows you the keymap.

In the ivy selection buffer

M-spc	Show the actions
spc	resume
RET	Default action and exit
C-RET	Default action and move to next line
C-u C-RET	Default action and move to previous line
M-RET	Default action on each candidate from current to end
C-u M-RET	Default action on each candidate from current to beginning
C-u C-u M-RET	Default action on all candidates
s-RET	quit with no action
?	show keymap help

I use Hyper-c as a prefix map for these commands.

H-c r	`ivy-resume’
H-c l	`counsel-load-library’
H-c a	`counsel-ag’

counsel-find-file

Here are some additional actions we define for `counsel-find-file’.

a	attach to email
c	copy relative path
4	Open in new window
5/f	Open in new frame
C	Copy absolute path
d	open in dired
D	Delete file
e	open in external app
p	insert relative path
P	insert absolute path
l	relative org link
L	absolute org link
r	rename file
b	open in bash
F	open in Finder

scimax-hydras

Scimax uses a central hydra to launch a key-driven menu of commonly used commands.

(describe-function 'scimax/body)

So, instead of C-x C-f to open a file (which still works by the way) you can simply type: f12 ff. Or instead of C-x C-s to save the file, type f12 RET. Each hydra will show you a hint in the minibuffer on which keys do what.

Appendix

`scimax-help’ will open this document.

Emacs

Emacs is described as “self-documenting”, and it provides a lot of introspective ways to access documentation. A classic way to access help is via “info” pages. These are hyperlinked documents containing a lot of detail about Emacs and its libraries.

Emacs-manual: info:Emacs

n	goto next page
u	up a node
d	Main directory
?	list of keys and commands
q	quit

Emacs libraries: info:dir Emacs-lisp manual: info:elisp

org-mode manual: info:org

Getting help on Emacs functions

To get help on functions use: `counsel-describe-function’ and on variables use `counsel-describe-variable’.

If you do not know exactly what you are looking for try: `helm-apropos’. This covers commands, functions, variables, faces, classes, and some other things.

Getting help on system commands

If you have man pages installed (Linux/Mac for sure, maybe on windows) you can access them easily via Emacs. I like to read man pages in Emacs with `helm-man-woman’ although `man’ is also good.

Note while on a man page you can use these keys:

m Prompt to retrieve a new manpage. r Retrieve reference in SEE ALSO section. M-n Jump to next manpage in circular list. M-p Jump to previous manpage in circular list. n Jump to next manpage section. p Jump to previous manpage section. g Go to a manpage section. s Jumps to the SEE ALSO manpage section. q Deletes the manpage window, bury its buffer. k Deletes the manpage window, kill its buffer. ? Prints this help text.

scimax also defines a link for man pages.

Manpage for BSDTAR(1)

What happens if I press a key?

The `describe-key’ function will prompt you for a key sequence or you can click on a menu item to see what it does.

The `describe-mode’ function will provide a full list of all the keybindings in the current buffer.

In org-mode the `ore’ command will give you information about the org-element under your cursor, including some commands that you can use at the point. You can also customize this by adding your own notes (click on User documentation in the help buffer).

How do I learn shortcuts?

If you use the menus, they will often have the shortcut in the menu.

The help functions usually show you if a shortcut is defined for a command.

If you really want to define your own keys, see `define-key’ and `global-set-key’.

Make yourself some notes in org-mode using `emacs-keybinding-command-tooltip-mode’.

Using only parts of scimax

Suppose you have your own Emacs setup already and only want to use a few parts of scimax without loading the whole thing. No problem. Just add the scimax dir to your load-path, and require as you like.

For example, to just use the scimax ipython enhancements (jkitchin#191 (comment)), this might work:

(add-to-list 'load-path "path/to/scimax-dir")
(require 'ob-ipython)
(require 'scimax-org-babel-ipython-upstream)

Note that a lot of packages are installed in ./packages.el and you may have to inspect the source of the packages you are requiring to make sure all the dependencies get installed.

Export the info manual

(require 'ox-texinfo)
(org-texinfo-export-to-info)

;	avy-goto-char-2
‘	avy-goto-line
<	goto point-min
>	goto point-max

j	left
l	right
i	up
k	down

r	counsel-git-grep
o	helm-org-agenda-files-headings
a	swiper-all

Files

scimax.org

Latest commit

History

scimax.org

File metadata and controls

Scimax - The manual

Org-mode

Markup shortcuts

Template expansions in org-mode

Latex

Latex classes that scimax knows about

Better image scaling

new speed commands

Formatted copy and paste

Writing tools

Spell-checking

Autoformat

Track changes

org-ref

Publishing (ox-manuscript)

Manuscript templates

Bibliography management

Searching your bibliography files

Bibtex entries

Article (Article in Journal)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

InProceedings (Article in Conference Proceedings)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

InCollection (Article in a Collection)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

InBook (Chapter or Pages in a Book)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

Proceedings (Conference Proceedings)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

Book (Book)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

Booklet (Booklet (Bound, but no Publisher))

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

PhdThesis (PhD. Thesis)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

MastersThesis (Master’s Thesis)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

TechReport (Technical Report)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

Manual (Technical Manual)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

Unpublished (Unpublished)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

Misc (Miscellaneous)

Required fields

Optional if Crossref present but otherwise required fields

Optional fields

Bibtex hotkeys

Email utilities

html mail