infer-abstract.tex

%This paper shows how to
%generate recursive heterogeneous type annotations for
%programs that manipulate plain data.
%Our approach includes observing an instrumented running program,
%and using a novel algorithm to ``squash'' the observed structure
%of program values into named recursive types.
%
%We apply this approach to generate Typed Clojure annotations,
%and report on experience in using our tool to generate annotations for real-world
%Clojure programs, and enumerate the remaining changes needed to fully port
%them to Typed Clojure.

%%%% V1

%The untyped-typed porting process is costly, but tools to smooth
%this transition are scarce.
%To type check a dynamically-typed program
%with most optional type systems,
%type annotations must be added.
%This burden is sometimes large, and has put off real users 
%attempting to migrate to
%existing optional
%type systems. When not discouraged, programmers often
%annotate tens of thousands of lines of code without assistance.

%Programming languages that encourage programming with plain values over declared classes
%tend follow Alan Perlis' advise:
%``It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures.''
%As a consequence, a rich variety of idioms and 

%As a consequence of this flexibility, the central data structures in such languages
%are used in numerous, implicit 

% Recursive Type Annotations for Ad-Hoc Data Structures
% Semi-Automatic Type Annotations for Recursive Ad-Hoc Data Structures
% Tool-Assisted Type Annotations for Recursive Ad-Hoc Data Structures
% Annotating Recursive Ad-Hoc Data Structures for Optional Type Systems

% A Semi-Automatic Workflow for Type Annotating Ad-Hoc Data Structures

We present a semi-automated workflow for porting
untyped programs to annotation-driven optional type systems.
Unlike previous work, we infer useful types for
recursive heterogeneous entities that have ``ad-hoc''
representations as plain data structures like maps, vectors, and sequences.

Our workflow starts by using dynamic analysis to collect samples from program execution
via test suites or examples.
Then, initial type annotations are inferred by
combining observations across different parts of the program.
Finally, the programmer uses the type system
as a feedback loop to tweak the provided annotations until they
type check.

Since inferring perfect annotations is usually undecidable
and dynamic analysis is necessarily incomplete,
the key to our approach is generating close-enough annotations
that are easy to manipulate to their final form by following static type error
messages.
We explain our philosophy behind achieving this
%, including choosing compact, recursive types with recognizable names,
along with a formal model of the automated stages of our workflow,
featuring maps as the primary ``ad-hoc'' data representation.

We report on using our workflow to convert real untyped Clojure programs to type check with Typed Clojure,
which both feature extensive support for ad-hoc data representations.
First, we visually inspect the initial annotations for conformance to our philosophy.
Second, we quantify the kinds of manual changes needed to amend them.
Third, we verify the initial annotations are meaningfully underprecise by enforcing them at runtime.

We find that the kinds of changes needed are
usually straightforward operations on the initial annotations,
leading to a substantial reduction in the effort required to port such programs.

%Our only requirement of existing
%programs is that they be runnable, with a suite of tests or
%examples. Given a running program, we instrument the execution, record
%type information, summarize it, and annotate the existing program with
%the recovered types.

%We present an approach to lighten the load on programmers moving to
%optional types. 

%We apply our approach to Clojure, a dynamically typed
%language with a culture of unit testing as well as both an existing
%optional type system and a contract system. Given a component under
%consideration, we instrument the source and analyze the behavior of the
%program while running unit tests.
%Equipped with this information, we summarize it by generating compact
%type specifications for all the functions in the component, including
%well-named type definitions. Our tool can also automatically generate
%contracts using the Clojure spec tool. Since Clojure relies
%heavily on ad-hoc data structures in the Lisp tradition, we describe
%an algorithm for automatically inferring recursive structural types
%from data examples, a challenge not considered in prior work.

%Our approach, as must be the case for a testing-driven tool, is
%incomplete---programs may have too few unit tests, and untested
%execution paths can have differing type behavior. We therefore
%evaluate our tool by running it on real Clojure programs and then
%completing the porting to Typed Clojure. We find that while
%some changes are always needed, the generated types are
%valuable and the effort reduction is substantial.

%%%% OLD
%  The untyped-typed porting process is costly, but
%  tools to smooth this transition are scarce.
%  We isolate the process of writing static type
%  annotations for untyped top-level variables, often manual and tedious.
%  Programmers must first annotate their own variablr es.
%  Even then, annotate used libraries. u
%  Worse still, annotate variables in the macroexpansion of imported macros.
%
%  In this paper, we explore a
%  tool dyinfer to generate type annotations.
%  What makes a good annotation?
%  Annotations should be readable, compact,
%  and capture the essential structure of
%  the running program.
%  Annotations should help programmers
%  type check their programs
%  by capturing relevant usages in the current 
%  context.
%  We aim to generate mostly-good annotations
%  that require little engineering effort
%  to correct, of which most is driven by
%  static type errors.
%
%  Our tool instruments running programs
%  and summarises observed values, outputing type annotations.
%  We handle higher-order functions
%  and record-like constructs.
%  To improve readability and make annotations
%  more useful for type checking programs, we generate
%  recursive union types when appropriate.
%
%  We apply our algorithm to Clojure programs
%  to generate Typed Clojure annotations.
%  We show the resulting annotations are often
%  adequate to type check usages.