A Nano-Pattern Language for Java

Journal of Computer Languages 54 (2019) 100905

Contents lists available at ScienceDirect

Journal of Computer Languages journal homepage: www.editorialmanager.com/cola/default.aspx

A Nano-Pattern Language for Java

T

Yossi Gil, Ori Marcovitch, Matteo Orrú

⁎

Computer Science Department Technion—IIT Taub Building Haifa 3200003 Israel

ARTICLE INFO

ABSTRACT

Keywords: Software and its engineering General programming languages Social and professional topics History of programming languages Nano-patterns Java Programming languages Code reuse Cliches Idioms

Patterns, micro-patterns, and nano-patterns have many applications: program comprehension, code transformations, documentation aids, improving code robustness, etc. This work revisits the notion of nano-patterns—originally an obiter dictum of the work on micro-patterns. Nano-patterns here are taken as more general than their previous definition in the literature: predicates on short code snippets that represent some common and elementary programming missions such as “for each m ∈ M do...”, or, “use x (but if x is null, y is a substitute)”, which represent small and recurring programming idioms. With this generalization, we offer a taxonomized languageof nanos nano-patterns for Java. We also describe the process of pattern harvesting we used and the underlying rationale, including our proposed prevalence threshold criterion, which, by capitalizing on Hirsch’s famous h-index, makes a robust yard-stick of the pattern’s significance. An empirical survey of 78 Open Source Java projects indicates that the nano-patterns of our proposed language described here have a substantial prevalence in the code. About a third of the commands (executable statements) and half of the methods are instances of nano-patterns in the proposed language. Also, the language’s prevalence is typically higher than that of languages harvested in a project specific, automated machine learning process. Nano-patterns are implementation/language level details for most high level software engineering purposes. One contribution made by the present paper is in identifying the clutter made by the snippets, appreciating its presence, and imposing order on it. The language, the nano-patterns in it, and the contributed automatic tool for tracing nano-patterns in code may help to deal systematically with this low level, yet significant, portion of code.

1. Introduction 1.1. A Parable We start with the following illustrative example. A team of professionals is designing a snack vending machine. Mike, the mechanical engineer, is responsible for mechanical aspects including these subsystems of the vending machine: buttons, slots, the coin mechanics, motors, etc. To attach these modules, Mike needs nuts and bolts of many kinds. Mike can order (or make) custom-made nuts, bolts, and even threaded rods in any desired handedness, pitch and diameter. Mike is skilled at making these custom-made items. He is well-trained in appreciating the relative merits of the parameters that define them. He may even use custom-made nuts and bolts, for instance, for securing the safe and lock. Still, productivity dictates that most nuts and bolts should be ordered from some standard set of preferred combination of diameter and pitch, such as ISO 262 [1]. In choosing to use ISO 262 nuts and bolts, Mike reuses the design

⁎

decisions of the ISO committee. Mike’s reuse, however, is not limited to nuts and bolts. Even though he is capable of designing and producing mechanical modules, he prefers standard, off-the-shelf modules over custom-made ones. Mike notices that modules, both custom-made and off-the-shelf, are typically made of many, typically standard nuts and bolts. The same productivity considerations apply in the work of Elia, the electrical engineer. To be productive, Elia prefers standard modules: keyboards, modems, and chip-sets over custom-made modules, even though she is capable of making them herself. Electronic components, resistors, capacitors, transistors, etc., are Elia’s world equivalent of Mike’s nuts and bolts. If Elia needs, for example, transistors and resistors for connecting standard modules, she prefers to select these from standards [2], e.g., an NPN polarity BJT transistor (a 2N2222 or the like). Elia, just like Mike, notices that modules she buys, just as those she decides to make, tend also to use standard components. The equivalent of modules in the software world are frameworks and libraries. Sveta, the software engineer member of the team, uses an

Corresponding author. E-mail addresses: [email protected] (Y. Gil), [email protected] (O. Marcovitch), [email protected] (M. Orrú).

https://doi.org/10.1016/j.cola.2019.100905 Received 25 December 2018; Received in revised form 1 June 2019; Accepted 2 June 2019 Available online 20 June 2019 1045-926X/ © 2019 Elsevier Ltd. All rights reserved.

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

external communication library, device driver, GUI framework and operating system, despite the alternatives of producing or subcontracting a custom-made implementation. The parallels drawn in the scenario suggest that reuse occurs at different granularity levels: reuse at the “module” level is of elements of design that contain “components”. Modules are found in Mike’s, Elia’s and Sveta’s work. Transistors and resistors are Elia’s words for nuts and bolts: little things that are used in many ways and in different places in the system, either for connecting modules, or in the clockwork of the modules themselves. Yet, what are nuts and bolts in Sveta’s world?

which passed the Phase 2 are reported in Table 4.1. 1.3. The Prevalence Threshold We use a prevalence threshold to make the selection of candidates more objective. The threshold is meant to be a robust and objective filter of domain specific nanos, and other coincidences. A nano-pattern passes the prevalence threshold if it passes the reuse threshold in, at least, = 50% of the projects in a corpus C. We call prevalence score the fraction of projects for which a nano-pattern passes the reuse threshold. The nano passes the reuse threshold of a project p, if it is reused more times than the r most popular methods in p. The r most popular (i.e., most reused) methods in the project are determined in a manner similar to that of selecting the publications that contribute to the h-index [10] of a scholar: A project has a method (class) reuse index r if it has r methods (classes) that are reused r or more times. The prevalence threshold is parameterized by the corpus C, and the fraction ρ. Here we used = 0.5 and the baseline corpus as described in Sect. 5.

1.2. Nano-Patterns and Language We suggest that the nuts and bolts of the software world are the many little things that software engineers do repetitively: safeguarding against null, caching a returned value, delegating to a field or a parameter, catching an exception only to ignore it, executing a command depending on some conditions, a loop searching for an element in a given collection, etc. Any developer who has written a lot of code will be familiar with the feeling of déjávu when writing a fragment of code, the sense that one has written this fragment many times in the past, and will no doubt write it many times in the future. While the fragment may take only a matter of a few seconds to write, and even less than that to read, the frequency with which it is written means it represents a non-trivial effort over the life-time of developers and projects. We seek to better understand these little fragments of code, with the objective of reducing their number and the overhead they impose. In fact, many aspects of language evolution reflect the identification of patterns of such fragments and providing support in the language for them (e.g., in Java: generics, enhanced for, λ-expressions, in C # setters and getters, etc.). Let us call these “little things that software engineers do” nanopatterns, or, for short, nanos. As results from Appendix A we are not the first ones to use this term. However, to the best of our knowledge, the first appearance of the term nano-pattern was in a paper from the first author [3]. Other researchers, such as Batarseh [4] and Singer et al. [5] used the same terms since then. Additionally, a few scholars investigated the impact of nanos in the context of security [6] and quality [7] of software systems, whereas others worked on similar concepts such as stereotypes [8] and idioms [9]. A survey of the available catalogues of nanos (and the like) is reported in Sect. 3.1. In the present work, we are expanding the definition of nano-pattern furthermore. Technically, a nano-pattern is a predicate over short code snippets that tries to match code written in response to an elementary programming mission, e.g.,

for each m

M do. ..,

1.4. Evaluation Evaluation (see Sect. 5) was across three corpora of 26 Java Open Source Software (OSS) projects each. Results indicate that the language’s prevalence is substantial; e.g., about a quarter of all Abstract Syntax Tree (AST) nodes are characterized by a nano, just as about half of the iterative commands, and at least 40% of the conditionals. These values are above the prevalence of project specific idioms discovered automatically by Allamanis and Sutton [9]. The language was initially refined on a non-representative small training corpus (six projects in similar application domains). During this refinement, we neither examined the language’s effect on any other code nor checked prevalence values in other projects. The results were then submitted for empirical evaluation on a larger Baseline Corpus. This corpus was a readily available one, previously used in another research study (see, e.g., Gil and Lalouche [11] for size information and the rationale behind this particular selection). The baseline corpus comprises three projects from the training corpus (by coincidence) along with 23 other projects. A second layer of evaluation was made on two corpora assembled for this research:

• The Java Corpus, comprising the 26 top most starred , non-Android, 1

2

•

1.5. The Nano-Tracer

(1.1)

Each nano is detected using a tool we designed, called Nano-Tracer, which matches input code snippets with patterns. For each nano, the tracer applies a small nano-specific algorithm, which tries to refactor the code—i.e., apply one or more behavior-preserving code transformations, so that the input is brought in to the pattern form, e.g., (1.6) for the nano evaluateUnlessDefaultsTo. An example of the process performed by the Nano-Tracer is represented by Fig. 1.1, which represent the implementation of the predicate defaultsTo. This predicate replaces the following code snippet:

or, use x but , if x does not exist , use null or , some specificy instead”.

GitHub projects, with the exception of those already in the baseline corpus. The Android Corpus, comprising the 26 top most starred, Android, GitHub projects.

(1.2)

Each discovered nano is classified according to the taxonomy reported in Fig. 2.1. They are also described in terms of pseudo-code and, additionally, we provide a description of their meaning using both natural language and mathematical formalism (whenever necessary or useful) as reported in the fourth column of 6.2. In particular, 6.2 collects all the 82 initial candidates, and reports further information about the process of generation - i.e. it includes the source of inspiration represented by constructs already available in real programming languages, etc. At first, the candidates where selected and categorized through a brainstorming process, then they went through a process of subjective scrutiny (Phase 1) and finally, those which passed the Phase 1, went through the Phase 2, where they were tested with an objective prevalence threshold against a baseline corpus. The 38 nanos nanos

state = customer. getAddress(). getState() = = null ? california : customer. getAddress(). getState(); with: 1 2

2

At the time of writing. https://github.com

(1.3)

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

return *)));

where * is a wildcard capturing any number of tokens, may be prevalent, but it does not meet the second criterion we impose. It must fulfill or define a small and well-defined programming task. This is why evaluateUnlessDefaultsTo is a nano, while (1.10) is not. Other prevalent and purposeful nano-patterns are letIn, firstSuchThat, pojo.Atomic.Constructor, and holdsOrReturn. Outline. Context and applications are discussed in Sect. 3. Sect. 4 describes the language of nanos in greater detail. Sect. 5 details the empirical evidence for the language prevalence. Sect. 6 overviews the language’s evolution, from a list of candidates to a reasonably coherent language. The prevalence threshold and its ties to the h-index are the subject of Sect. 7. Related work and scientific background are given in Sect. 8. Sect. 9 discusses threats to validity, and Sect. 10 concludes the paper. Appendix A broadens the scientific background further, and refers to other more related work. Appendix B gives code snippets exemplifying the nanos, and Appendix C provides essential reproducibility information. Finally, Appendix D provides more details on the Java-like stenography we use occasionally in tables.

Fig. 1.1. Class for tracing the defaultsTo in Java code

state = de. fault(customer. getAddress(). getState()). to(california), (1.4) which is valid Java, assuming these definitions of de, faults( · ) and to( · ),

2. Language Overview Fig. 2.1 depicts our taxonomized catalog, or “language”6 of prevalent nanos. The 38 nanos of the language appear as leaves of the tree in Fig. 2.1. Some nanos have also sub-nanos. There are too many nanos and sub-nanos to describe each individually, but Tables 4.1 and C.5 below provide some hints. Example snippets from real code appear in Appendix B. Also, as in Snippet (1.6), nano names are often juxtapositions of the keywords in their pseudo-code. For example, the pseudocode command Let i ← e in C(v), or in a slightly more general form,

public interface de { interface To < T > {T to(T default); } static < T > To < T > fault(T expr){ return default > expr! = null ? expr: default; } (1.5)

}

This specific example is discussed in further details in Sect. 6. Currently, the Nano-Tracer is not a standalone tool, but is included in the SpartanRefactoring, an open source Eclipse plugin available in the Eclipse Marketplace3. Moreover, being an open source software, its repository is publicly available on GitHub4.

Let· e in C (·)

First · M , p (·).

(1.6)

The predicate behind evaluateUnlessDefaultsTo is designed to capture Java snippets such as (1.7)

and,

pos > 0 ? pos

1: 0

(1.8)

and,

if (b = =0) return null; return new Integer(a/b);

(2.2)

Taxa are shown in the hierarchy of Fig. 2.1 as internal nodes, rendered with shade opacity, with increasing granularity. Primary taxa correspond to syntactical elements. The terms composite and atomic, occurring as taxa in the secondary and tertiary levels, are interpreted in the context of the recursive language structure elements. For example, an if command is a composite of one or two sub- commands, while a method invocation command is atomic since it has no sub-commands. In a composite/atomic recursive hierarchy, we say that constructs such as “if”, “synchronized”, and, “{...}” (curly brackets) are composite constructors since they all are used by programmers to construct a composite command from sub-command(s). An Elaborator taxon is composed of nanos that are similar to synchronized composite constructors in two respects: a synchronized composite command has a (i) single sub-command, which it executes in a designated (ii) elaborated/protected execution environment. We use the term elaborator for nanos that do essentially the same thing in their respective syntactical taxon. For example, defaultArguments (i) makes one and only method invocation (ii) to a method in an (iii) elaborated execution environment that provides missing values for arguments for this function. The nano safeNavigation (also known as the “?.” operator in, e.g., Swift and C # ) is an Elaborator.Expression nano—it executes a (i) sub-expression in a (ii) null guarded execution environment. The similarity between elaborators and Java language constructs

Take as an example, the nano called evaluateUnlessDefaultsTo, which can be written in pseudo-code as5

(a instanceof A) ? ((A)a). f(): null

(2.1)

(where “.” denotes an anonymous free variable) is summarized in the nano name letIn. Similarly, the nano firstSuchThat is short for

1.6. An Example: The evaluateUnlessDefaultsTo Nano

[Evaluate] e unless b [default e ]

(1.10)

(1.9)

As we already said, not all little things are equal. Our interest lies with prevalent nanos: those that recur in substantial numbers across a variety of projects. The nano evaluateUnlessDefaultsTo, for instance, passed our prevalence threshold test. Prevalence is a necessary condition for a predicate to be considered a nano, but not sufficient. Code snippets such as 3

https://marketplace.eclipse.org/content/spartan-refactoring-0 https://github.com/SpartanRefactoring/Main 5 Henceforth, we use these conventions: C–command, e-expression, ppredicate, M-multitude (which might be an array, stream, list, set, collection, etc.), b—boolean expression, i—identifier, T for type, and, X—exception type. 4

6 The term “language”, endemic to the patterns literature, means roughly “useful catalog” or “domain-specific catalog”: see, e.g., the conference series “Pattern Languages of Programs—PLoP” [12] and other examples [13,14].

3

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

Fig. 2.1. Taxonomy of the language of Java nanos

such as “synchronized” and the “= =” operator is limited, since nanos try to identify a simple semantic action. Further, this action needs to be so simple that it is the clutter, not essence, of design. The nano executesUnless is an elaborator, but it is intentionally not as general as the Java if. The predicate behind executesUnless represents the frequent need to conduct an atomic command on (say) a variable, but only if a certain condition applies. To summarize the notions of pattern, instance, and the various taxa, consider these key points related to the safeNavigation nano:

• •

• The little (nano) thing that engineers do is null-guarding, e.g., trying

The distinction between Weak.Elaborator.Expression and Strong.Elaborator.Expression is described among other language details in Sect. 4.

to code

State state = customer. getAddress(). getState();

(2.3)

but they write instead,

State state = null; if (customer ! = null) if (customer. getAddress() ! = null); state = customer. getAddress(). getState();

•

that under-approximate patterns. In the case of this nano, the tracer has to demonstrate that there are some refactorings in the corpus, which lead to a form using the “?.” operator, but it may err on the false negative side. Note that the nano is in the Expression taxon even though its trace might be a command. The nano is in the Elaborator taxon, since it executes its subexpression, in an environment that guards against null. The guarding here is similar to the protected execution environment offered by try...catch.7

3. Context and Applications 3.1. Available Catalogs

(2.4)

In the scientific literature we find several examples of catalogs of patterns. Compared to the present work, these catalogs present several

to guard against the customer being null and getAddress() returning null. The nano is realized by a predicate on a code that identifies the Snippet (2.4) as (multiple) instance(s) of safeNavigation. This identification is also called tracing. Our interest lies in predicates

7 In fact, the protected execution environment that safeNavigation traces can be Java encoded by the protected execution environment of the try...catch block.

4

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

Table 3.1 Generative research of recurrence and patterns in code, The ✗symbol denotes an information not available in the corresponding paper. #

Code

Study

Patterns

Coveragea

Granularity

Lang.

Corpus Sizeb

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

G AB GM D1 HO1 HO2 D2 B S SHM SPVS APMM A1 AS1 W ABBDMS A2 L GMO

Gamma et al. [16] Abd-El-Hafiz and Basili [17] Gil and Maman [18] Dragan et al. [19] Høst and Østvold [20] Høst and Østvold [21] Dragan et al. [22] Batarseh [4] Singer et al. [5] Sutton et al. [23] Sridhara et al. [24] Andras et al. [8] Arnaoudova et al. [25] Allamanis and Sutton [9] Wang et al. [26] Allamanis et al. [27] Arnaoudova et al. [28] Lee et al. [29] Gil, Marcovitch and Orrú (The present paper)

23 8 27 11 10 17 15 15 12 8 5 17(34) 17 18 12 ✗ 17 67 38

✗ ✗ 75% 80-98% ✗ ✗ 30 32% ✗ 80% 85% 11—40% < 35% ✗ ≈ 30% 57, 9% ✗ 16,89% > 50% > 30%

Class+ loops Class Method Method Method Method Method Method ✗ Loop Statement Method Method, Attributes ✗ Loop-if Statements Loop Statements Method, Attributes Method Method, Field, Statement+, Expression

Any Pascal ✗ Java C++ Java Java C++ Java Java C++ Java Java ✗ ✗ Java C# ✗ ✗ Java

✗

a b

14 2 53 100 1 100 ✗ 1 1,000 3 ✗ 7,159 ✗ 6 10 78

Percentage of patterns found. See Catalogs section for further details. In terms of number of projects.

differences in terms of number of patterns, number of projects forming the studied datasets, prevalence or coverage, not to mention other relevant features mostly related to the discovery process. Table 3.1 lists, in a chronological order (from the oldest to the most recent, including one of the authors [15], covering about two decades), the most relevant research works that feature, besides other results, a catalog of patterns or the like. The large majority of these works do not deal explicitly with nanopatterns as we defined them here, i.e., they do not use the term nanopatterns, nano etc. On the other hand, they deal with constructs somehow related to nano-patterns such as idioms, stereotypes and properties of code constructs. For sake of simplicity, in the following we mention them using the common name for the patterns, providing a comment to underline the differences with nano-patterns whenever relevant. One of the most recent catalog is the one presented herein, Gil, Marcovich, Orrú (GMO): Information about the reproducibility is reported in Appendix C and specifically the tables Table C.1, Table C.2 and Table C.3, whereas aggregating statistics is reported in Table 5.1. The number of patterns identified is on average 18.72 (median 16). A meaningful exception is represented by the work of Andras et al. [8], which reports 17 stereotypes that combined amount to 34. It is worth noting the work of Lee et al. [29], which reports 67 properties that characterize Java methods. There are some cases in which there is no information about coverage or prevalence, e.g., in Gamma et al. [16] and Batarseh [4]. In other cases, the authors used a particular definition of prevalence that differs from the one used in the present work. Moreover, in some cases the prevalence analysis led to results that cannot be summarized with a number, e.g. Arnaoudova et al. [25] and Høst and Østvold [21]. Table 3.1 reports the cases in which a single number can be considered representative of the entire catalog, or at least represents an upper or lower bound. For example, in Lee et al. [29], the prevalence is generally higher than 50%, but it depends on the selected pattern. In general, there is high variability, with some work reporting prevalence values higher than 80% (Dragan et al. [19] or Singer et al. [5]) whereas others report values under 20% (Arnaoudova et al. [28]). Some works in our list report values around 30% (Dragan et al. [22], Andras et al. [8]). The majority of the works regards patterns at the method (or lower) level. Some of them deal specifically with loops, such as Wang et al. [26] and Allamanis et al. [27]. In Wang et al. [26], the study regards the number of loop-if structures (namely, loops that contain exactly one if statement) that were automatically associated with a high level

action. Frequencies depends on the action (12 in total) with “find” being the most frequent and “max/min” the less. Abd-El-Hafiz and Basili [17] defined a taxonomy based on three dimensions of loops (control computation, complexity of condition and complexity of body). Each dimension is split into two (not overlapping) classes. The number reported in Table 3.1 represents the number of possible combinations. The reported works differ also in the method used for the pattern discovery (manual or automatic). Some studies involved human intervention not only in the discovery process but also in the validation. An example can be seen when validating the outcomes of the automatic discovery algorithms. In Wang et al. [26], the authors relied on the opinion of 15 experienced developers to assess the relevance of the actions associated to the loop-if structures. They also differ in a property of the found patterns that we called orthogonality, which is related to the reciprocal in(ter)dependence of the discovered patterns. The vast majority of the research deals with ordinary software systems written in Java language. Exceptions are the work of Sutton et al. [23], which is focused specifically on C++ libraries, instead of regular projects, the work of Abd-El-Hafiz and Basili [17] on Pascal and that of Allamanis et al. [27] on C#. It is worth noting that, in general, the number of projects in the research that relies on manual discovery is low, while it increases when an automatic discovery process is involved. In Andras et al. [8], the research is carried out on a dataset of three projects, depending on the stereotype. In Lee et al. [29], the authors analyzed 10 projects, but none of them is specified. In Dragan et al. [19], the analysis is performed on a single project, HippoDraw, and there are variations depending on the stereotype and the perspective (classes or methods). Moreover, the information about the size of the dataset is never available. Some of the above studies were the first to introduce new concepts into the software engineering research. Such is the case of the idea of micro-patterns or linguistic anti-patterns (LAP). Arnaoudova et al. [25] and Arnaoudova et al. [28] defined the first catalog of linguistic nanopatterns. They determined the coverage through automatic search. In the former study, they found 126 LAP in 81 files in a sample of four projects with a precision of 72%. In the latter work, they analyzed six projects. The reported value is the average coverage computed on this dataset. Høst and Østvold [21] designed a phrase book of 364 phrases that constitute a restricted natural language, the Programmers English used by Java developers to write methods names. Prevalence information regards the phrases (not the method attributes) that involve more that 5

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

100 methods and appears in more than half of the programs that form their dataset. Sometimes, the information provided lacks important data regarding the reproducibility. For example, in Lee et al. [29], none of the 10 projects studied is specified. Moreover, there are some meaningful differences with our study and some of the others. Allamanis et al. [27] performed large-scale mining looking for loop idioms. They mined a large dataset, finding a number of meaningful loop idioms. They report some example but do not offer a specific catalog.

Also, nanos border on language and library engineering, and the study suggests applications for these areas. Nano-patterns are code abstractions at a fine level of granularity. For most high level software engineering purposes, they are implementation and language level details. Our results indicate that together, these accumulate to form clutter of a non-negligible size. The language, the taxonomy behind it, the nano-patterns in it, and our automatic tool for tracing nanos provide the means for managing and imposing order on this clutter. Accordingly, we see applications for nanos in contexts such as program visualization [44], code comprehension [33] and reverse engineering [34], clone and plagiarism detection [37], automatic documentation Wang et al. [39], etc. Our work follows the work in design patterns [45], micro-patterns, and the two previous publications on nano-patterns [4,5], in these respects:

3.2. Context Source code is not random text. As in natural language, it has regularities: words that occur more often than others, idioms, clichés, recurring phrases, patterns, etc.—see Gabel and Su [30] Tu et al. [31] and Hindle et al. [32]. A body of research in software engineering concentrates on identifying these regularities, and in their exploitation for desirable ends such as productivity, identification of potential bugs, program comprehension, etc. Nanos are similar to “idioms”, and other recurring, often nameless, pattern-like creatures found in code. Observe that the study of regularities of this sort tends to follow three main themes.

• Patterns are being manually generated, i.e., human expertise is employed for identifying commonalities. • Patterns are named, i.e., the mental effort of identifying the patterns •

(1) In the “opportunistic” research theme, regularity is sought to serve a specific end. For example, work on code completion [33] concentrates on the regularities that can make code completion more efficient. The history of research employing such regularities is long, starting from early work on using them for purposes such as reverse engineering [34], software design [35], education [36], program recognition [37], and automatic programming [38], continuing to contemporary work such as that of Sridhara et al. [24] who observed that certain recurring code patterns represent “high level abstract algorithmic actions” that can then be converted into natural language description. Regularities found by the first research theme often risk being ad hoc and non-reproducible. Regularities found for the sake of (say) identifying algorithmic steps (or code completion) may be useful for better CASE tools, but the regularities found in one work are not necessarily encapsulated, named, and organized sufficiently to be applicable in similar domains. Wang et al. [39] who considered the problem of automatic segmentation of method bodies—the locations in the code in which insertion of new line increases code readability—were able to identify a number of recurring patterns, e.g., syntactically-same, data-flow chain, and extended-SWIFT. Nevertheless, to date, the patterns found by Wang et al. have not found other applications. (2) In the “generative” research theme, effort is invested in producing reusable regularity, i.e., identifying regularity that persists over a wide domain or, more sophisticatedly, inventing techniques for discovering local regularity—the patterns and idioms peculiar to, for example, an individual software artifact. Examples include, e.g., the work of Singer et al. [5] on nano-patterns, and the seminal work of Hindle et al. [32] on “naturalness”. (3) The “applicative” research theme exploits the recurrences that were systematically found by generative studies for purposes similar to opportunistic study of regularity. Numerous examples exist such as the use of nano-patterns for fault detection [7], and in the context of security vulnerabilities [6], the links found between micro-patterns and class names [40], the use of micro-patterns to predict class lifetime [41], bug introducing changes [42], and source code search engines [43], not to mention the use of design patterns for program visualization [44].

culminates in suggesting names such as CHAINOFCOMMAND design pattern, the STATELESS micro-pattern and the LocalReader nano-pattern (of Singer et al.). Patterns are Java universal (rather than being artifact or project specific).

In contrast, regularities discovered by statistical algorithms [30,32] identify anonymous recurrences that may be idiosyncratic to a specific input. Also, our language is different from the previous work on nanopatterns (but similar to the work on design- and micro-patterns) in the following sense. Previous nano-patterns languages concentrated on finding essentially the orthogonal properties of methods: a method can be both a LocalReader and StraightLine (even though the events might be correlated). Design patterns, μ-patterns and the language presented here focus on a specific and often peculiar incidence of properties (see also Sect. 8.3 below). The literature is silent on the subjective process of the discovery of patterns such as syntactically-same or STATELESS, i.e., what was it that made the authors choose these patterns rather than others. The community may find our description (Sect. 6) of the harvesting process we applied and the rationale behind useful. 4. The Nano Language We now discuss the language and the taxonomy (inspired by the writings of Watt [46]) of Fig. 2.1. The primary taxa of the hierarchy divides the nanos by their syntactical role. Three taxa, Field, Constructor, and, Method are the syntactical elements that comprise Java’s classes and types. Other nano taxa are at the sub-method level—they occur in methods, constructors, and, initializers. Sub-method nanos include Command (a synonym for an executable statement, i.e., excluding non-executable statements such as variable declarations), and Expression, which syntactically occur in commands. The last primary taxon, Phrase, groups two nanos at the sub-expression level: myName, the pattern of a method calling itself, or another identically named, overloaded version, of itself, and, myArguments, which is a method to use a verbatim copy of its own argument list in calls to other methods. The secondary and tertiary divisions in the taxonomy refer to the role of the nano in the composite–atomic relationship found in these syntactical elements, e.g., secondary taxon Atomic.Command gathers atomic command nanos. This is to say, there are nanos that look like an atomic command. Nanos that capture code, which has the structure of sub-command(s) in a compound, are gathered in a secondary taxon, Composite.Command. These include:

The present work follows the generative theme. Specifically, we offer a set of patterns that occur at the method and sub-method granularity level. We expect future research to follow the applicative theme. 6

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

(1) The nano whenHoldsOn captures a recurring case in which a programmer applies a predicate to a series of values, taking action based on the first value that satisfies the predicate. Writing this nano in pseudo-code shows that it is a compound command assembled from sub-commands:

When p (·) of x , y, z: C1 of w: C2 otherwise C3

• • •

(4.1)

(2) The tertiary taxon Elaborator.Composite.Command is composed of nanos executing a single, non-block command in a protected environment, e.g., ignoring thrown exceptions. (3) The tertiary taxon Iterative.Composite.Command is composed of nanos that capture the small recurring task of making a “simple”, iteration over a collection, a sequence, an array, etc. By “simple” we mean that the predicates behind the nanos restrict the iteration body to be an atomic command, in a form suitable, for example, for adaption to a Java 8 stream API.

state = customer. getAddress(). getState() = = null ? california: customer. getAddress(). getState();

(4.2)

can be replaced by state = de. fault(customer. getAddress(). getState()). to(california),

(4.3)

which is valid Java, assuming these definitions of de, faults( · ) and to(·),

The Expression sub-hierarchy includes:

• The taxon Multitude.Expression composed of the nanos that re•

code language, and we use pseudo-code for describing the intent of some nanos. Some nanos, e.g., questionQuestion, occur as operators in other programming languages. Even though nanos are not Java Specification Requests (JSRs), it may be possible to extend the language to support shorter encoding of some nanos. In describing some nanos, we employed a “stenographic Java” notation, which should be self-explanatory, but interested readers may consult Appendix D for more information. Column (*) c in Table 4.1marks those nanos that can be encoded as a Java 8 library service, e.g., the occurrence of defaultsTo in the code snippet:

public interface de { interface To < T > {T to (T default); }

present recurring rudimentary operators (of languages other than Java) over non-scalar values. The taxon Elaborator.Expression, which includes rudimentary (nonJava) operators guarding from exceptional situations in the computation of their operand. The distinction between Weak.Elaborator.Expression and Strong.Elaborator.Expression is based on the permitted corrective action. Strong elaborators throw an exception, weak ones may not do so.

static < T > To < T > fault(T expr){return default >expr! = null? expr: default; } }

(4.4)

As noted, nanos border on language and library engineering. One potential application is a nano-library to encode nanos, with fluent API call chains and λ-expressions. Examining Table 4.1, we see this is possible for about half of the nanos, all located on the left -hand side of the table. (The current Java 8 stream library covers almost all nanos in the lower part of the table.) Nanos show a couple of themes of little things that engage programmers.

The distinction between elaborators and atomic nanos carries over also to methods and constructors. All nanos in Elaborator.Method are methods whose body is essentially delegation to another method. Each particular nano in the taxon makes its own specific demands on this method call. Similarly, all nanos in the Atomic.Method taxon are methods whose very simple body does not include any method calls, and are limited. Each nano in the taxon makes its own strict demands on the simplicity of this call-less body. The sub-hierarchy of Constructor is smaller, but similar, including Atomic.Constructor and Elaborator.Constructor. The metaphor is extended also to fields, distinguishing between Atomic.Field, which includes no field-access, and Elaborator.Field nanos, which elaborate this access. As indicated by the dashed line in Fig. 2.1, the taxon Elaborator.Field specializes Atomic.Method, i.e., methods that make no calls to other methods. In the case of Elaborator.Field, the restriction placed on this method is retrieve the field’s value (getter), set this field’s value (setter), or set the field’s value and then return this; (fluent-setter). Table 4.1 provides more details on each nano. The table is complemented by Table C.5 providing similar information on sub-nanos. We do not discuss sub-nanos further than this. Column (*) a in Table 4.1 traces nanos back to their origin, the nanocandidate(s) from which they evolved. The language evolution can be learned by comparing nanos with their legacy candidates (see Table 6.2). Column (*) b in Table 4.1, shows each nano’s prevalence in the baseline corpus. The score and filtering process are described in Sect. 6 and Sect. 7. The information in the subsequent columns in Table 4.1 highlight places in which the study of nanos abuts language design and library design:

• Defaulting and guarding: Nanos in the first block of the table are in-

• •

volved in making small corrections of small errors and exceptional values, substituting a missing value with some default, terminating the current method and returning if a parameter is null, or when a pre-condition is not met, etc. This theme shows in safeNavigation, but also in many other nanos. Simple operations on multitudes: As shown in the second table block. Pipes and sinks: As shown in the right-hand side of Table 4.1, there are a number of nanos in which methods and constructors are used for adjustment, adding parameters, delegating to a receiver, etc. We also see cases of “sinks”, methods that call no other methods and whose body is trivial, a standard constructor body, etc.

The themes may find an application in the evolution of new programming languages and the extension of existing ones. Language designers should consider substituting these “common nuts and bolts” with language built-in, dedicated syntax and less-traditional operators. In terms of our scenario, we can say that language mechanisms standardize nuts and bolts. Software libraries do the same standardization, but offer a different trade off: language syntax is more powerful than libraries, but libraries are easier to design and change. A certain balance between flexibility and expressiveness is found in macro processing [47] and syntax sugaring [48]. These techniques might also be used for standardization of nuts and bolts. In particular, macros and sugaring may deal better with the smaller theme of reference to self (the two nanos in Phrase), which is impossible to

• The name of the nano is often derived from its encoding in pseudo7

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

Table 4.1 Language of Java Nano-Patterns. (*)

a

(*)

b

(*)

c

Name Expression and Command Elaborators executeUnless

1

C2

100%

2

C5

100%

whenHoldsOn

3

E2

81%

defaultsTo

4

E3

96%

questionQuestion

5

E4

50%

safeNavigation

6 7 8

E5 E7 F1

92% 96% 100%

evaluateUnlessDefaultsTo safeCast throwOnFalse

9

F2

81%

throwOnNull

10

F4

88%

notNullRequired

11

F5

92%

notNullAssumed

12

F6

100%

holdsOrReturn

13 14

F7 M2

58% 100%

15

C4

73%

16

58%

aggregate

17

G2, G4, H2, H8 H4

73%

selectBy

18

H5, H6

96%

forEach

19

I4

50%

firstSuchThat

20 21

A1 A3

96% 100%

22

A2

77%

23 24 25 26

K1 K2 K4 K5

88% 100% 54% 58%

27 28 29

J3 J4 J5

85% 73% 100%

30 31

L5 M3

100% 100%

32

L1

96%

35

L4, L5

100%

33 34

L2 L3

85% 69%

ignoringExceptions letInNext Operations and Operators on Multitudes forFromTo

constantValue thrower defaultValue

Phrase myArguments myName me | recurse | overload Field const const Pi = 3.14159; setter getter fluentSetter vanillaCollection Map⟨K, T⟩= new HashMap⟨⟩(); Set⟨T⟩ = new HashSet⟨⟩(); Atomic Method

examiner letIn Elaborator Method (Single Call) factory defaultArguments adjuster

delegator Tf (A1 , ...,An )

Intent

d

[Execute] C unless b [Execute] C when b When p() of x, y, z: C1 of w: C2 otherwise C3 e1 default e2 e1 ?? e2 e1 default e2 else e3 e1 ?? e2 : e3 e?.f e?.m() v = [Evaluate] e unless b [default e′] T?(e) Throw X when b b ?! X Throw X when e nulls e !! X Return default when a nulls T f(safe F1, ..., safe Fn){...}; Return default when e nulls e ≠ null || return default; Return default when b b || return default; {...} ignoring X1, ..., Xn; let v ← e in C; For i = 0, ...,n 1 [do]... for (i: e1.. e2) C; // i = e1, ...,e2 1 for (i:.. e) C; // i = 0, 1, ...,e 1 for (i: e..) C; // i = e, ... (∞-loop) for (..) C; // ∞-loop (no loop index) Aggregate M [st p()] with A(, ) M.filter(...).reduce(...) Select i ∈ M s.t. p(i) {x ∈ M|p(x)} M/p() M.filter(...).collect(...) For i ∈ M [s.t. p(i)] [do]... for (..) p(): C M.forEach(...) M.filter(...).forEach(...) First M/p() M.filter(...).findFirst() T f(...) = {...g (#). ..} T f(...) = {...f(...)...} π ≡ 3.141593 protected ← private T i = e; public → protected T i =e; T i = e; public fluent List⟨T⟩ = new ArrayList⟨⟩();

T f(...) = c; return c; arguments (if any) are ignored T f(...) = throws X; Tf(...)= default; depending on T returns null, 0, false, or nothing (if T is void); arguments if any are ignored Tf(...) = e; Tf(...) = let v ← e in C; Tf(A1, ..., An) → new [( A1 , ...,An )]; T f (D0, A1 , D1, ...,Dn 1, An , Dn+ 1) ; Tf (A1 , ...,An )[ g ]; Tf(A1, ..., An) → g;

g (A1 , ...,An );

Tf(A1, ..., An) → ai: : g [(A1 , ...,An ) ]; Tf(A1, ..., An) → super [(A1 , ...,An ) ];

(continued on next page) 8

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

Table 4.1 (continued) 36 37

38 a b c d

100% 88%

N1 N2 N3

pojo super

81%

Constructor

T(A1, ..., An) → default; T(A1, ..., An) → super [(A1 , ...,An ) ]; T(A1, ..., An) → this [(A1 , ...,An ) ];

this

Candidate(s) from which this nano was created Prevalence score in the baseline corpus Can be emulated by lazy-evaluation, using lambda functions e.g., pseudo-code, Java 8 streams, suggestion for extending plain Java syntax, etc.

Table 5.1 Aggregating statistics of prevalence (in percent point) of the nanos language in three corpora of 26 OSS projects each. Control Flow Methods Commands

Expressions

Conditional

Nodes

Covered

Touched

Iterative

Expressions

Commands

26 4 26 24–28

29 5 29 27–33

54 9 53 50–57

31 7 31 28–36

69 9 68 61–75

81 13 87 69–91

41 9 40 37–46

32 24 6 5 32 22 28–37 20–26 Android Corpus (see Table C.3)

27 5 26 24–30

53 11 52 49–60

29 8 29 23–33

72 9 72 66–77

84 23 81 69–97

40 8 39 34–45

25 5 26 23–30

46 8 45 41–52

31 8 31 27–36

74 25 76 57–97

73 33 71 52–86

46 11 44 37–52

Baseline Corpus (see Table C.1) Mean σ Median Q–Q Mean σ Median Q–Q Mean σ Median Q–Q

33 7 34 29–38 Java Corpus (see Table C.2)

30 7 29 25–35

23 5 23 21–27

standardize using a library, and probably too drastic. Conversely, the themes are also material in domains such as visualization and clone elimination, and code comprehension, in which the clutter of nuts and bolts should be hidden, not eliminated. The effectiveness of an automatic document generator would be better if it could classify more portions of the code as boilerplate.

baseline corpus comprises 26 popular projects [11] selected from of GitHub’s TrendingWikiBookJavaIdiom repositories https://github.com/ trending?l=javasince=monthly augmented by the GitHub Java Corpus http://groups.inf.ed.ac.uk/cup/javaGithub/ list by Allamanis and Sutton [49]. Reproducibility and other information on projects constituting this corpus are listed in Table C.1. Overall, the corpus represents the work of about two thousand programmers and at least a century of overall development time. The two other corpora in Table 5.1 can be thought of as testing corpora since we did not use them for language evolution and pattern harvesting.

5. Language Prevalence Table 5.1 shows the prevalence (in percent points) of the nanos in three corpora. Prevalence is defined as the fraction of syntactical elements, expressions, statements, methods, etc., that are matched by one or more nanos. It is worth to note that in Table 5.1 we refer to nanos which proved to be prevalent, which means those whose reuse index r is equal or higher to the reuse index of a project methods p, and this results for a number of projects higher than = 50%. We refer to the fraction of projects for which this happens as prevalence score. Table 5.1 demonstrates that the instances of occurrences of the language (the nanos) are substantial. Nanos appear in about a third of all commands, more than a half of the iterative commands, 80% of the conditional commands that can be converted into conditional expressions, and 40% of the remaining conditional commands. About 55% of all methods are characterized completely by our language, while in about a third of all other methods, one or more nano is found. We note that prevalence of our subjectively selected language rivals that of patterns discovered by an automatic learning process [9]. This comparison is done in terms of AST node coverage. No data was available for the other measures we use. The first corpus in Table 5.1 is the baseline corpus. It is distinguished from the other two corpora because it took part in the language evolution, and can, therefore, be thought of as a training corpus. The

• The second corpus in the prevalence table (Table 5.1) consists of the •

26 top most starred Java projects in GitHub, excluding Android projects, and projects included in the baseline corpus. Reproducibility information can be found in Table C.2. The third corpus in the prevalence table (Table 5.1), consists of the 26 top most starred Java Android projects in GitHub, excluding projects included in the baseline corpus. Reproducibility information can be found in Table C.3.

The two test corpora were sampled at the time of writing. In contrast, the last update in the training corpus was 2014-8-21 (in project Hadoop-common). Examining Table 5.1, and comparing the respective prevalence values in the three corpora, we can see that the numbers tend to be lower in the test corpora than in the training corpus. Still, prevalence is high, regardless of corpus, measure of prevalence, and the summarizing statistic, and the differences, when present, are relatively small. We also ran a battery of Wilcoxon signed-rank tests for statistical significance. The results were unequivocal: The null hypothesis is that 9

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

Table 5.2 Results of the Kruskal-Wallis test, for the Baseline and Java corpora. The value of Chi squared function for x = 0.95 is 6.635 Category

Methods Control Flow

Conditional

Commands Expressions Nodes Covered Touched Iteratives Expressions Commands

Chi-squared

p-value

0.109 3.693 2.477 0.209 1.481 1.884 0.148 0.244

0.742 0.055 0.116 0.647 0.224 0.17 0.701 0.621

Table 5.3 Results of the Kruskal-Wallis test, for the Baseline and Android corpora. The value of Chi squared function for x = 0.99 is 6.635 Category

Methods Control Flow

the prevalence values (for a measure such as “conditional expression”) found in two corpora, one of which is a training corpus and the other a test corpus, are drawn from the same distribution. This seems to suggest that, even if we generated the taxonomy (and computed the prevalence) on the intentionally biased corpus of Table 6.1, the presence and prevalence on the nanos is significant also in the testing corpus. As it turns out, the null hypothesis was not rejected for all prevalence measures, and when it was, the rejection sometimes indicated that the testing corpora were better than the training corpus, or that the differences between the two testing corpora were as significant as the difference with the training corpus. Since we can not assume the normality of the distribution we applied a Kruskal-Wallis analysis. Tables 5.2 and 5.3 report the results for p value = 0.01. Under the null hypothesis the k samples can be treated as a single (combined) sample from one population [50], which in turn supports our conclusions about distribution of the nano-pattern prevalence over the three corpora. The null hypothesis is confirmed when p-value is higher than 0.01 and the value of the Chi-squared function is lower than the corresponding tabulated value (for P = 0.01) which is 6.635, which occurs for all the categories with the exception of the covered methods of Table 5.3.

Conditional

Commands Expressions Nodes Covered Touched Iteratives Expressions Commands

Chi-squared

p-value

2.774 3.763 5.574 10.14 0.052 1.574 2.959 1.935

0.096 0.052 0.018 0.001 0.819 0.21 0.085 0.164

Fig. 6.1. Class for tracing the defaultsTo in Java code

the library (4.4)). The tracer also features local name binding and rudimentary type inference, but not data flow or control flow analysis. Instead of these advanced analyses, prior to actual matching, the tracer makes a number of semantic preserving small code transformations, designed to minimize style variations and to bring the code into a shorter, more canonical, form. These transformations include inlining of temporary variables when possible, simplifying boolean expressions, removing redundant parenthesis, bringing loops to canonical form, converting if...else commands to conditional expressions, etc.

6. Language Evolution and Implementation This section reviews the evolution of our language. To the best of our knowledge, this is the first description in the literature of the subjective process of pattern harvesting. In the first phase of our language’s evolution, an extensive number of initial “candidates” underwent subjective evaluation and refinement. Our objective was to identify candidates that are

6.1. Phase 0: Candidate Collection We collected candidates following these three steps repeatedly:

• traceable (within the limitation of our nano-tracer), • likely to be prevalent, and • purposeful, e.g., appear frequently as familiar pseudo-code, or a de-

(1) Implement the predicates that approximate a new pattern idea on the tracer. (2) Examine the code snippets to refine the predicates. (3) Reflect whether to refine, defunct, and/or reorganize existing patterns and to raise new patterns ideas.

scription of a small programming activity.

Candidates that passed this subjective scrutiny phase went onto the second phase, which was an objective prevalence threshold test against the baseline corpus. The language was then taxonomized from candidates that either passed this test or could have been rephrased to pass it. Recall that nanos are predicates of code, designed to spot a small semantic action at a fine level of granularity. They are realized by our nano-tracer, a lightweight Java library wrapping Eclipse’s Java Development Tool (JDT), for matching and substitution on the abstract syntax tree. Upon identifying an instance of a nano in the code, the tracer can, for instance, inject a code comment representing the nano trace. Fig. 6.1, showing our predicate implementation of defaultsTo, demonstrates the capabilities (and limitations) of the tracer employed. The string literals in Fig. 6.1 show a small Domain Specific Language (DSL) [51,52] for structural match-and-replace. In fact, the code in the figure replaces code such as Snippet (4.1) with Snippet (4.3) (assuming

These steps were carried out against a (biased) corpus of 6 OSS projects (depicted in Table 6.1). Overall, this “preliminary” corpus is not small—spanning circa 300,000 lines of code. The manual collection of candidates was done by manual inspection of code samples drawn from the corpus. This sampling, however, was neither in any particular order nor gave extra attention to intra-corpus bias (i.e., sampling did not give equal weight to projects in the corpus). 6.2. Phase I: Candidate Pruning Table 6.2 depicts the 82 candidates thus collected. Every nano in the language of Fig. 2.1 evolved from one or more of these candidates, but not all surviving candidates became nanos. Table 6.2 is not intended to be exhaustive—we focus on the more promising patterns. Some obvious absences are some higher order 10

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

Table 6.1 Corpus of 6 OSS Java projects used for preliminary testing: Identification, reproducibility information and size metrics. Project

Version

Date

Hash

Lines (K)

Pa(K)

Tb(K)

Mc(K)

Atmosphere Commons-bcel Commons-lang Guava JUnit SpartanRefactoringd

2.1.1 BCEL_6_0 LANG_3_5 v20.0 r4.12 v2.8.0-beta

2014-04-12 2016-07-14 2016-10-20 2016-10-28 2014-12-04 2017-01-23

521350f 7b9e151 a9a3cef 32130d5 e87c6fc 10b19af

31 47 46 113 11 44

20 12 15 32 44 42

0.4 0.5 0.2 2.0 0.2 0.5

3 4 3 15 1 6

Total:

6 projects

292

165

3.8

32

a b c d

number of packages number of types (classes, interfaces, enums), in thousands number of methods, in thousands https://github.com/SpartanRefactoring/Main

functions (e.g., currying, composition, and folding), set operations (intersection, union, complement, etc.), zipping and streams of pairs, element swapping, and many more. These may be pursued in future research on identifying higher level operations of one language in the code of another. Candidate selection and categorization was carried out by brainstorming, trying to draw inspiration from experience in other languages and programming environments (and to a certain extent, from the nuts and bolts scenario). The intent column in Table 6.2 records the source of inspiration, including first order predicate logic, programming for the vi editor, experience in Eclipse and programming languages, from ancient Pascal to contemporary Swift. The rationale behind seeking this inspiration is that designers of languages and environments include in their design abstractions and mechanisms that help programmers in their recurring nano activities: Candidate C1 in the table represents the conjecture that the while…else control of Python would be found as a nano of Java. A search for nanos of classical Lisp processing was done through candidates I1 and I3. We were unable to trace these specific Python and Lisp nanos in our corpus. Nevertheless, other nanos such as const and forEachDo owe their roots to Pascal. There are too many candidates in the table to describe each individually. The division into 14 taxa enables a quick survey. As typical in the case of an intuitive free form, opportunistic search for recurrence, some candidates are similar, and divisioxxsxn into taxa is not absolute. It is also interesting to compare the taxa in Table 6.2 with the form they took in the taxonomy (see Fig. 2.1 and Table 4.1 above). Taxon A is about phrases and language features that do not correspond to the familiar syntactical constructs and mechanisms of the subsequent taxa. Taxon B represents the automatic conversions between scalar types, typical of the C language. Taxa C, D and E represent not so-usual control flow (and in particular—iteration) and scalar operators such as Elvis. Noticing that many of these operators tend to be used for managing exceptional situations or null values, taxon F was made the target of a broader search for candidates that deal with these situations. More operators and operations on sequences and other nonscalars are taxa G, H and I. A number of taxa represent our experience in examining the code of short methods.

commands and expressions, and are not as small as phrases. 6.3. Phase II: Nano Candidates Table 6.2 ’s legend summarizes the fate of the candidates. Nine candidates were abandoned at an initial stage, when the small number of instances detected in the preliminary study corpus did not justify (in our mind) further study. Candidates are described in the table by their “stenographic Java” comment syntax. The remaining candidates went to prevalence testing in the baseline corpus (Table C.1). A total 38 candidates were included in the language, and 10 candidates were merged into other nanos. The tested, yet rejected candidates, are listed in Table 6.3. The first block in the table enumerates 10 candidates whose prevalence was below = 50% in the baseline corpus. The two other blocks of the table show candidates that are in fact missed opportunities for a “simple” method call, i.e., a method not relying on inner classes, function arguments, or function return values. As shown in Table 6.3, prevalence of all but one of these method nanos was below the set threshold. The only exception is the “first” nano candidate. It was rejected for the sake of language uniformity. All nanos in the language are not missed opportunities for simple method abstraction. 7. The Prevalence Threshold Not all patterns can be encoded in Java, and nano tracing is useful, even without modifying the code at all (for purposes such as visualization, code summary and documentation, fault location, etc.). Still, in this section, we draw some parallels between nanos encodable as lambda expressions, to describe a rationale for our use of the “r-index”. Consider nano defaultsTo, and its implementation as a fluent API call chain de.fault( · ).to( · ) via the four-line definition (4.4). The nano can be abstracted and captured as a fluent chain but, the introduction of even the short definition (4.4), just like any other library code, has its maintenance costs. Is the capture worth the effort? In the language of this paper’s scenario, is this “reuse of a standard bolt” worth the trouble of standardization? The question is similar to the issue of eliminating code duplicates. Upon identification of duplication of a certain cardinality, an engineer should consider abstracting over the duplication, representing it in (say) a method, and then refactoring the duplication sites to invoke this method. This consideration must, however, take into account that reuse has its maintenance and other costs [53]. A conservative and greedy project management policy is adopting a personnel management guideline, “to avoid degrading, recruit above your grade”. To understand why, assume that reuse is a valuable end, measured at the project level by some agreed-upon “reuse grade”. The incurred cost of a reuse opportunity is to be weighed against this grade.

• Taxon J represents the many short methods that are the sink of the • •

control flow graph (stub) and whose body is a simple expression of a peculiar kind. Boilerplate methods (L) are those whose body is a single call to another method. Taxon N represents nanos of method constructors.

Taxon K is dedicated to nanos on fields; these nanos are often methods as well. Finally, taxon M is dedicated to nanos that combine 11

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

Table 6.2 Nano Candidates. #

Fate

A. Recurring phrase A1 A2 A3 A4 B. Convenience conversion and casts B1 B2 B3 B4 B5 C. Fancy control (and not so fancy) C1 C2 C3 C4 C5 D. Fancy iteration over... D1 D2 D3 D4 D5 D6 D7 E. Fancy operators E1 E2 E3 E4 E5 E6 E7 E8 F. Safe guarding F1 F2 F3 F4 F5 F6 F7 ✗ F8 F9 ✗ G. Boolean operators on multitudes G1 G2 G3 G4 G5 H. Operations on multitudes H1 H2 H3 H4 H5 H6 H7 H8 I. Operators on sequences I1 I2 I3 I4 J. Stub methods ✗ J1 J2 ✗ J3 J4 J5 J6 ✗ J7 ✗

Inspiration

Description / Programming task

Bash Pascal Java Java

Comma separated list of names of parameters to current method Constant definition—a public static final of primitive, boxed, or String type Method invoking an overloaded (different arity) version of itself Find the min/max of two numerical values

Java C C C C

Cast Cast Cast Cast Cast

Python Swift Swift Pascal ML

An else of a while loop C unless b: execute C, but not if b ≡ true C when b: wexecute command C, but not if b ≡ false for i:=1 to n do C: Execute command C for i = 1, ...,n Generalized switch defined by f() and a sequence of pairs 〈v1, C1〉, ..., 〈vn, cn〉: if f(vi) holds then execute Ci (variations: what to do in case there is more than one/no such i)

Snobol VIM Python Python Scala/SQL Mathematica XQuery

... ... ... ... ... ... ...

Icon Swift

Does the result of a numerical expression lie between two given values? e1??e2: if e1 ≠ null, then e1, else e2 e1 == null ? e2 : e3 e?. ( field/method): if e ≡ null then null else e. e unless b: if b holds then null, else the evaluation of e e when b: if b holds, evaluate and return e, else null Checking if an expression is of some type, then converting to it e1?:e2: if e1 auto-converts to boolean true then e1 else e2

Swift Swift Swift PHP Swift Swift Java maps Eiffel Eiffel C++10 C++11 Scala FOPL FOPL FOPL FOPL

multitude to boolean: true iff it is not empty boolean to integer: false → 0; true → 1 reference to boolean: null → false; !null → true reference to integer: null → 0; !null → 1 number to boolean: 0 to false, others values to true

the n + 1 locations that separate the elements of a sequence of length n the n 1 pairs of consecutive elements in a sequence of n elements the min(n, m) full pairs obtained by simultaneous iteration on sequences of length n and m the max(n, m) partial pairs obtained as in G3 the n × m pairs in the Cartesian product of two multitudes of size n and m values in multitudes contained in a given multitude values in a recursively defined data structure, e.g., binary tree

If expression evaluates to false, throw If expression evaluates to null, throw Associate some default value with a key in a map, unless the key is already present in this map Require, in beginning of a method, that an expression is not null, otherwise return Assume, inside a method, that an expression is not null, otherwise return Assume, inside a method, that an expression is not false, otherwise return Catch phrase: ignoring caught exception Catch phrase: re-throwing an exception Catch phrase: returning default value |S| ≡ ∃x ∈ S, ∃x ∈ S, ∀x ∈ S, ∀x ∈ S,

1: is multitude a singleton? p(x): exists a value which satisfies p()? ¬p(x): exists a value which does not satisfy p()? p(x): all values in a collection satisfy p()? ¬p(x): none of the values satisfies p()?

Scala

Compute cardinality Compute cardinality of subset of values in multitude Copy multitude Compute subset of values in multitude Apply command to values in a multitude Apply command on a subset of values in a multitude Select single value from a multitude if is a singleton (or some default value if it is not) Aggregate a multitude into one value using an aggregation function, optionally filtering out some values

Lisp12 Lisp Lisp13 SQL

Extract the first value in a sequence Extract the last value in a sequence Extract the rest of a sequence Find the first value in sequence that satisfies a given predicate

SQL Java SQL

Eclipse

Returning this Returning parameter Returning constant Throwing exception Returning default value/nothing Down-casting parameter Up-casting parameter

(continued on next page) 12

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

Table 6.2 (continued) #

Fate

Inspiration

J8 K. Field sublimation K1 K2 K3 ✗ K4 K5 ✗ K6 L. Boilerplate methods L1 L2 L3 L4 L5 L6 ✗ L7 M. Mixed expression M1 ✗ M2 M3 N. Constructor N1 N2 N3

Description / Programming task Determine whether a parameter is of certain type

C# C# Smalltalk Scala 14 FACTORY 15

C++ C++16 Java

Icon/C ML ML

Setting a field value Returning a field value Update field and return old value Setting a field value and returning this Lazy initialized field Field of a generic collection type initialized as an empty concrete type collection Returning new object Default arguments: calling another method with same arguments, and add some more (Gil and Lenz 2010) Calling another with same arguments adjusted Delegating to another method Delegating to super Returning boolean expression Method whose body consists of applying a similarly named method to each member of a collection Exhaust an iterator without doing anything with its values An Assignment followed by a command expression or return expression which uses the assigned value A method whose body is a single letIn whose body is command expression or return expression; A POJO constructor, taking arguments which are values for initializing all fields A constructor delegating all arguments to super A constructor delegating to this

Legend—Fate of candidates Description (n.) Description (n.) Made into a nano ( × 38) ✗ Rare ( × 10) Merged into another nano ( × 10) Function ( × 15) Abandoned ( × 9)

elimination) that do not increase the supposed reuse grade could be rejected immediately. The actual grade-increasing of remaining abstraction and reuse candidates is compared to their (typically unknown) cost. In using the grade increase as the sole criterion for evaluating nanos, we effectively assume that the cost of using captured nanos is relatively small (but still positive). In truth, this reuse cost (e.g., learning a new language feature or library of tricks) is amortized across the many projects in which a nano is used. It is tempting to define the grade of the project as the mean reuse level of its components. The long tail distribution of reuse level components, however, will prevent this grade from being effective. The average number of locations in which a method is called is often less than two. The recruit above your grade would mean that nanos occurring just twice should not be rejected. A similar issue lies with the median and other simple statistics. The h-index proved robust in grading scientific proliferation, which due to the long tail distribution of citations, resists statistics such as mean and median. The r-index adopts the idea as a robust grade of the reuse level of a project. Hirsch’s famous definition [10] can be rephrased as:

Table 6.3 Candidates phrased as nanos, but rejected later (*)a 1 2 3 4 5 6

(*)b

Rare Candidates F6 0% rethrowExceptionAs F7 19% returnOnException J1 19% returnThis J2 12% returnParameter J6, J7 15% caster K3 4% returnPrevious

7 K5 15% 8 L7 12% 0% 9 M2 Rare Functions 10 A4 0% 11 B2 23% 12 F3 0% 35% 13 G1 14 H1 0% 0% 15 H2 16 I2 4% Prevalent Function 17 I1 77% a b

Name

Description

lazy mapper exhaust

{…} rethrowing [ X1, |…|Xn ] as Xn+ 1 {…} returningOn [ X1 |…|Xn ] T f () = this; T f (…, a, …) = a; T1 f (T2) = caster; old v = e; (assign e to v, but return v’s old value) T f = e lazily; T1 f (T2) = mapWith g( · ); exhaust e

min/max asBit putIfAbsent isSingleton count copy last

min( · , · )/max( · , · ) as.bit(e) M.putIfAbsent(k, v) is.singleton(M) M.size() v = copy.of(M) v = last(M)

first

v = first(M)

The h-index of a scholar s is k if k is the maximal value that satisfies the statement: Scholar s has k publications that are all cited k or more times.

Candidate(s) from which this nano was created Prevalence score in the baseline corpus

(7.1)

The recruit above your grade policy offers two nice properties:

By this definition, if the h index is k, then s accumulated at least k2 citations. Similarly, the r-index of a software project p is k if k is the maximal value k that satisfies the statement:

• Globally, each new addition is justified by a specific increase in the •

overall grade. If no reused components are ever erased, the grade will increase indefinitely. A reuse opportunity that does not increase the grade can be rejected without considering its cost.

Project p has k distinct cases of code reuse, each of which recurs k or more times.

(7.2)

The above definition may be applied to reuse of classes, methods, and any other modular structure. We concentrate on reuse of methods.

The result is that candidates for abstraction (of nanos and duplicate 13

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

Table 7.1 Summary statistics of r-index values in the baseline corpus (Table C.4) Statistics

r

ri

rx

ri

Mean SD Median Quartiles Q1

46.7 25.5 41.5 26.5–60.8

40.7 22.8 37 23–51

24 12.1 21.5 14.2–29

16.7 13.3 13.5 5.8–24.2

Q3

A use instance of a method is a code location in which it is called. With the exception of the first use instance, all other use instances of a method are reuse instances. Thus, if the r-index is k, then p accumulated at least k2 distinct cases of code reuse. A nano used n times in p passes the reuse threshold of p if n ≥ k, where k is the value of the r-index of p. A nano passes the reuse threshold of a certain project p, if it is reused more times than the r most popular methods in p, where the r most popular methods in project p are defined with respect to the reuse index (abbreviated as the “r-index”) of p, i.e., (7.2). Suppose, for example, that a certain nano was detected 99 different times in project wildfly. To determine whether this nano passed the reuse threshold of this project, 99 should be compared to the r-index of wildfly. Consulting Table C.4 in the appendix, we see that the r-index of wildfly is 106, and therefore, the nano did not pass the reuse threshold of wildfly. The number of occurrences of the nano in each project has to be compared with the r-index of the particular project. To be prevalent, the nano has to pass the reuse threshold of 13 projects or more. Table 7.1 gives summary statistics of the distribution of r-index values in the baseline corpus. We distinguish between ri, the internal rindex in which only methods defined in project p are counted, and rx, the external r-index, in which only methods defined externally to p are counted (i.e., External libraries, API, etc.). When no subscripts are attached to r, the index does not discriminate between reused methods based on their definition’s location. The last three columns of Table 7.1 show the increase in values of the three indices over the monitored period in the evolution of the baseline corpus. As expected, all these grades increase as projects evolve. Notice that the ri grade is typically greater than the rx grade. Overlooking the slight bias of project-overriding methods defined in external code, this difference indicates that projects tend to reuse their own code rather than external code. A priori, nanos not intended to be project specific are to be compared with the lower rx grade, since they are not project specific. We, however, used the higher r threshold, while giving partial score to nanos that fail this threshold, but still met the ri or rx thresholds. In ending this section, we note that our technique for finding the core reuse (most popular methods) in an artifact may be effective in other cases where the underlying distribution is long tailed [54]. Such distributions are abundant in many software engineering contexts [55–59]. For example, to accumulate the set of main contributors to a software artifact, instead of considering the smallest set of authors whose cumulative contribution is 80% or some other arbitrary threshold, we might use the author index (a-index) of the artifact (defined as the largest a for which there are a authors of, at least, a classes) and only consider the authors whose contribution is no smaller than the a-index. It might be worthwhile to correlate the projects’ success (measured, e.g., by popularity) with their r-indices and a-indices, and see whether these are better predictors than plain counts of authorship and reuse. Similarly, in evaluating the eco-system [60] of package repositories, we can define the reuse index (r-index) of the repository based on the number of times a package is used by other packages in the repository. The reuse-core of the echo system is defined as the set of packages that contribute to the r-index of the repository.

rx

Δr

Δri

Δrx

20.2 18.8 15 7.8–19.5

18.2 16.8 13.5 8–19.2

7.9 9.2 6 2.2–9.5

8. Related Work 8.1. Design Patterns and Micro Patterns A “design pattern” is a specific composition of object oriented components (classes, methods, inheritance) and captures a recurring solution for a common design problem [16]. Interestingly, both practical and research oriented, design patterns have flourished ever since the original gang-of-four announced their discovery [61]. A “micro pattern” [18] is a predicate on OO types (classes) that captures a peculiar restrictive use of OO features, e.g., no fields, or just one field, no methods, strict restriction on constructors, inheritance, etc. Compare μpatterns to design patterns, noting that

• μ-patterns are prevalent. Empirical evidence is that around 75% of all • •

classes in Java projects can be categorized as one or more of 27 or so μ-patterns. Demands on design patterns are more modest. To be worthy, they are to be encountered in a handful of projects. Design patterns are limited in not being traceable, i.e., they are not automatically recognizable. See, for example, the classical van Emde Boas lament [62], the many attempts to formalize these [63–67], and to automatically detect them in source code [68–73]. Micro patterns, in contrast, aretraceable. Design patterns have an attached purpose. A worthy design pattern is one that addresses a specific problem. No such purpose is attached to μ-patterns: they do not resolve a specific design problem, but capture a programming technique frequently used by software engineers. It was argued [18] that prevalence is an indication that a μpattern is purposeful, i.e., that there was one (or more) design problem that drove engineers to abide by the pattern’s restrictive predicate. Alas, this indication only hints at the folklore behind these presumed purposes.

Judging nanos (Fig. 2.1) by these three criteria, we see that they are prevalent (Sect. 5) and traceable (by manner of construction of a nano tracer (Sect. 6). Nanos, just as μ-patterns, have no designated design purpose, but for a different reason. Nanos clutter the design and hide the essence among this clutter. It takes a short second to decipher Snippet (2.3) from Snippet (2.4), and to recognize that Snippet (1.9) is an instance of Snippet (1.6). Programmers are inevitably trained to eliminate clutter and focus on the essence when dealing with language specific, style or convention differences. The prevalence of nanos, however, indicates that these small abstract gaps accumulate over a substantial portion of the code. The purpose of a nano is the small programming task that it represents, and can often be learned from its name. This purpose is usually independent of the software in which the nano occurs. For this reason, in many contexts, the substantial portion of the code dedicated to nanos is noise. A contribution of this work is providing a better understanding of the noise. Potential applications of the language are in reducing it. 8.2. Automatic Discoverable Idioms Relevant here is the work of Allamanis and Sutton [9]. The authors’ notion of “idioms”, “a syntactic fragment that recurs frequently across 14

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

software projects and has a single semantic purpose”, is similar to that of nano-patterns. The difference is that nano-patterns are predicates, rather than a single fragment (subject to variations). Allamanis and Sutton applied machine learning techniques (probabilistic syntactical derivation and nonparametric Bayesian methods) for the first automatic discovery of idioms. Idioms thus discovered included cross-project idioms that represent important program concepts such as object creation, exception handling, etc. Applying the three criteria above, we have that idioms are (by definition) traceable and prevalent. Prevalence of idioms, ranging between 3% to 31%, depending on the training and the testing dataset and the fitting parameters, tends to be lower than the prevalence of nanos. For the “purpose” criterion, note that idioms are discovered, while nanos are sought. A priori, discovery does not imply semantic purpose. Think of a compression algorithm that excels in finding recurring sequences, which in most cases, are useless to humans. The authors retrospectively demonstrate semantic purposes such as object creation, exception handling, and resource management for some of their idioms. In general, purpose and usability are a greater issue when dealing with idioms and other recurrences generated by unsupervised machine learning processes.

Table 6.1, including the authors’ own code, and other code authors’ domain specific interest in language processing tools. Further, in honesty, the informal refinement process rarely visited projects such as JUnit, and hardly ever, Atmosphere, manual inspection was disorganized, etc. Consequently, we cannot be statistically confident that our nanos’ yield is exhaustive. There are two aspects to the bias towards the specific corpus of Table 6.1. First, manual examination of code snippets before and after nano-transformation was limited to those drawn from this corpus. Second, prevalence was computed only in projects from this corpus. The presumption was that prevalent nanos would maintain their prevalence in spite of the corpus being intentionally biased. 9.2. The Nano-Tracer There are two limitations to the process performed by the NanoTracer as described in Sect. 1. First, since program equivalence is intractable, the nano tracer can only give a lower bound on the number of occurrences of nanos in code. For this reason, all prevalence results here approximate from below the actual prevalence of nanos. Second, as with most code refactoring tools, there are corner cases, atypical of real code, in which refactoring fails. For example,

8.3. Previous Work on Nano-Patterns

int i = 0 , j = i + +*++i; i = j + +*++ i* i;

The term nano-patterns was first briefly mentioned in the literature by Gil and Maman [18]. Following the work of Høst and Østvold [21], Singer et al. [5] gathered a language of 17 nano-patterns, which are demonstrably prevalent (at the 80% level), traceable, and, purposeful. Evidence of the purposefulness of the nano-patterns found in Singer et al. is the list of their names: NoParams, NoReturn, Recursive, SameName, Leaf, ObjectCreator, FieldReader, FieldWriter, TypeManipulator, StraightLine, Looping, Exceptions, LocalReader, LocalWriter, ArrayCreator, ArrayReader, ArrayWriter. Singer et al.’s language and ours are unconnected: Their language is of properties of methods, whereas our nanos tend to relate to the method, command, expression and field levels17. Also, the nano-patterns of Singer et al.’s language are essentially orthogonal, while ours are peculiar. Mathematically, this difference means that the combined entropy of their language is greater than ours. As the authors show, this orthogonality makes it possible to define composite nano-patterns capturing correlations between properties. Similar to the work of Singer et al. is that of Batarseh [4], in which the author described a language consisting of 16 method properties, and Lee et al. [29], who identified 67 such attributes, organized in three taxa: method signature, method body, and ties of body and behavior. We argue that in contrast to languages having orthogonal properties, the nano language presented here is more effective in reducing clutter.

(9.1)

is likely to baffle most refactoring tools. Yet another typical failure point sof refactoring tools is tied to overloading: Converting

if (a) return f(b) else return f(c)

(9.2)

to

return f(a? b: c);

(9.3)

requires that function f is not overloaded with respect to the types of b and c. Our experience is that errors such as this are rare, and happen at most a handful of times in each project we inspected. Upon identification of a pattern, the tracer may be programmed to attach an appropriate comment to the code, refactor the captured instance to make it look more like the desired nano, or notify a userprogrammable listener of the match, including code location and the matched nano. 9.3. Missed Nanos and the Language More generally, a potential issue with methodology in this research is that of implicitly making the reaching (and imprecise) presumption that

“There is one universal prevalent and cohesive language

9. Threats to Validity and Critique

of Java nanos”.

9.1. Limited Domain

(9.4)

Our informal line of reasoning followed (9.4), making strong assumptions on the extent of universality and prevalence of :

Validity of our findings is limited to the domain of the empirical study: OSS Java projects. Generalization to other kinds of software and to other languages is a matter of conjecture. It is likely that nanos will be found in other programming languages such as C++, C #, Swift, and Go, but different language capabilities may drive programmers to prefer some nanos over others, use new nanos specific to a particular language, or abandon some nanos. The results here do not tell us whether non-OSS code uses the same nano language as OSS code. More generally, just as there are domain specific pattern languages, there might be domain specific nanos for, say, Android applications. The present work does not resolve this issue, but it makes a start by showing that the nanos found are prevalent in Android projects, as shown in Table 5.1. A potential threat to validity is in the use of a biased small corpus,

is prevalent and universal, then we (the authors) being Java 1. If programmers ourselves, must have seen many of its nanos. Therefore, even our disorganized brainstorming (Table 6.2) is likely to hit many (but perhaps not all) of the nanos in . 2. Similarly, the biased selection of the corpus for the initial evolution, and the subjective pruning (Table 6.2) are rationalized by the presumption that universality spans all Java code. Such universality should be able to sustain this bias. 3. Finally, pruning for prevalence on a somewhat dated baseline corpus (Table C.1) makes the presumption of time-universality, i.e., that and its prevalence do not change over time.

15

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

• Language design, library design, macros and syntax sugaring to re-

The contribution made here is much weaker than presumption (9.4). All we can claim is:

There exists a language

•

of Java nanos that is universal

(under some definition of universality ),

•

prevalent (under some definition of prevalence ) over OSS projects, and , cohesive (i. e . , tree organized ) ”. (9.5) Presumptions such as (9.4) are too vague to require proving, but the result (9.5) is an indication of the truth in the presumption. The threat to validity is the attempt to produce the result (9.5), the evolved lan, under estimates guage (i.e., inferior recall). The answer to this over two independently selected threat is in the high prevalence of large corpora, which is substantial (second and third sections of Table 5.1), and not too inferior than the prevalence of over the baseline corpora on which was evolved. Further research is needed to improve the prevalence values by specifically looking for new candidates in other corpora. Also, the nanotracer’s capabilities are limited: it uses minimal data flow analysis heuristics, it does not create a control flow graph, its analysis of the namespace is crude, etc. This weakness is a strength of each nano in . We know now that a light weight tracer is sufficient to show their prevalence. This weakness also means that may be incomplete. The prevalence of nano candidates such as iteration over pairs (D4 in Table 6.2) could be demonstrated with a more powerful tracer.

duce the clutter of low level details, by eliminating the nuts and bolts or making them less conspicuous. Visualization, reverse engineering, code summarization, and, refactoring, in which the challenge is to hide, not eliminate, the substantial cumulative nuts-and-bolts clutter that nanos represent. Unexpected code aspects that connect to recurrences, e.g., if the presence of nano-patterns is correlated with bug proneness, the easy-to-detect presence of nanos might be used as a predictor of the illusive bug proneness property.

A (probable) unicum in the literature is the description of our nano harvesting including candidate collection and pruning. We explained the rationale behind this process, and pointed out applications in other domains of the ideas behind the r-index and the h-index on which it is based. The subjective and sample bias in the harvest may have led us to miss some patterns. This bias, however, together with the prevalence tests strengthens the evidence of the universality of the language we found, at least for its particular domain of OSS written in Java. The taxonomy we describe evolved over the nanos we found. A crucial test of the validity of the taxonomy would be in applying it to the nanos we may have missed. We provide evidence regarding the substantial prevalence of nanopatterns in actual code. About half of the methods are instances of nanos; a third of all remaining methods have one nano or more. A third of the commands and a quarter of the expressions are nano instances. Further, prevalence of our subjectively selected language rivals that of patterns discovered by an automatic learning process. An additional contribution is our OSS nano-tracer.

10. Conclusion We introduced a universal taxonomized language of 38 nano-patterns for Java. A nano-pattern is a recurring solution to a common task, both of which are at what we call the nano-scale, i.e., occurring in method and smaller syntactical structures. Examples include safeguarding against null, adding default arguments, or simple stub implementation. Nanos are often the accumulated clutter of small implementation level code and are likened to nuts and bolts in the software world. Applications are:

Acknowlgedgments J. Singer directed our attention to a literature gap, arguing that making pattern languages should be more reproducible and that the authors should take the time to describe the harvesting process. This research was supported by the ISF (Israel Science Foundation), grant No. 1803/13 *

Appendix A. Other Related Work Vocabulary vs. Structure In Sect. 3 we divided this research into three themes: generative, applicative and opportunistic. Here we divide general research in the area into two: the study of the nomenclature—the vocabulary that programmers tend to choose, and the study of the recurrence in the syntactical structure of programs, as we did in the present research. Studies of nomenclature include the work of Linstead et al. [74], who analyzed the source code vocabulary, identifying trends specific to syntactic types such as classes and interfaces. Enslen et al. [75]’s algorithm for splitting identifiers into word sequences makes use of word frequencies in source code to optimize the split. Abebe et al. [76] investigated the evolution of vocabulary with time; see also the work of Newman et al. [77] and the bibliography there. Høst and Østvold [21] dealt with the problem of generating a semantics that captures our common interpretation of method names. They started from the definition of a traceable pattern and proposed a set of traceable attributes useful as building blocks to define nano-patterns. They explicitly claimed that they were not proposing nano-patterns. In a previous work, the same authors analyzed the implementation of methods taken from a corpus of Java applications, in order to determine which word to use as a method name [20]. Høst and Østvold [78] were also interested in vocabulary, but they matched it against structure. In noticing that method names reflect patterns in the code’s structure, the authors devised automatic tools to identify “naming bugs” and for suggesting more appropriate names for methods. Kashiwabara et al. [79] and Kashiwabara et al. [80] presented a technique to identify candidate verbs to be used as method names. There is room for similar research that links structure and vocabulary in the context of nanos, i.e., linking these with names, or conversely, eliminating them in generating method names. Andras et al. [8] and Moreno and Marcus [81] investigated the consistency between design and implementation of OO methods using a run-time analysis of their behavior. They compared the outcome of the run-time with their stereotype [8]. Stereotypes capture the intent of a method/class and are considered as an extension of the micro/nano-patterns concept [19]. This idea is often used in automatic summarization applications. Attempts to employ recurring structure for software engineering purposes can be traced back to the work on beacons [82,83], which are stereotypical (and hence recurring) and telling segments of code that are quickly captured by experienced programmers. Qiu et al. [84] gave empirical support to structure-based search for recurring patterns. In particular, the authors showed that the use of syntactical rules in actual programs follows a Zipf-like distribution (just as the use words in natural language) or conversely, that a small set of syntactic structures tends to govern entire projects. 16

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

Hypothesis-Based vs. Automatic Search The identification of repeating patterns, idioms, common vocabulary etc., is traditionally done in one of two ways: scholars’ hypothesis confirmed by automatic discovery and/or manual browsing and search [see [4,5,16,18]], and, a hypothesis-free, automatic search [e.g., see [9,13,23,69,74]]. The evolution of our language is hypothesis guided, but, unlike previous work, we also describe cases in which the hypothesis was rejected. Sect. 6 reported on 82 nano “candidates”, organized in 14 preliminary taxa. Some of these candidates matured to be nanos, others were rejected for not being sufficiently prevalent, or for being difficult to trace with our tools. Idioms vs. Patterns The more popular literature speaks of idioms. There are online catalogs of commendable idioms, e.g., [85–88], IDEs provide support for idiom definition and customization [89,90], and books [91–93] that try to teach style and idioms to beginners. In these, idioms are rarely defined, but they are often meant to be programming tips, mostly at the class and below levels. In the scientific literature on idioms, we find the work of Sutton et al. [23] who studied idioms in generic C++ libraries, finding a high degree (circa 85%) of “idiomization”, i.e., coverage by μ-patterns [18] (which can be thought of as idioms at the class level). Other studies of idioms can be found, for example, here [9,37,94,95]. Patterns on Iterative Structures Abd-El-Hafiz and Basili [17] present a classification of loops by complexity levels. Wang et al. [26] offered an automatic approach to identify the high level action implemented by a given loop. Also, related is the work of Allamanis et al. [27]. In general, our loop analysis tools are weaker than these. Taxonomy of nanos Mens et al. [96], proposed defining patterns with a declarative meta programming language (DML), which is a Prolog variant. This approach can be used to build development tools to support developers in their programming tasks. The authors did not provide a catalog, but illustrated the use of DML by applying it to some of the best practices of Smalltalk programmers, design patterns, such as visitor, heuristics and code smells such as duplicated code. Appendix B. Nano Snippets B1. Elaborators B1.1. executeUnless.Elaborator.Composite.Command.

if (!doubleIsDifferent(unexpected, actual, delta)) failEquals(message, Double. valueOf(actual));

(B.1)

(project: Junit, file: Assert.java) B1.2. whenHoldsOn.Composite.Command

if (binding instanceof ConstructorBindingImpl){ return ((ConstructorBindingImpl)binding). getInternalDependencies(); } else if (binding instanceof HasDependencies){ return ((HasDependencies) binding). getDependencies(); } else { return ImmutableSet. of(); }

(B.2)

(Guice, InjectorImpl.java) B1.3. defaultsTo.Weak.Elaborator.Expression (B.3)

e. getCause() = = null ? e : e. getCause() (Cucumber-Jvm, RhinoStepDefinition.java) B1.4. questionQuestion.Weak.Elaborator.Expression

keyValueWriterClass = = null ? new HadoopStoreWriter() : (KeyValueWriter)Utils. callConstructor(keyValueWriterClass)

(B.4)

(Voldemort, HadoopStoreBuilderReducer.java) B1.5. safeNavigation.Weak.Elaborator.Expression (B.5)

newSearcher ! = null && closeNewSearcher (ElasticSearch, InternalEngine.java) B1.6. evaluateUnlessDefaultsTo.Weak.Elaborator.Expression

this . markedWriterIndex < = decrement ? 0 : this . markedWriterIndex

decrement

(B.6)

(Netty, AbstractByteBuf.java) B1.7. safeCast.Weak.Elaborator.Expression (B.7)

(ThreadPoolExecutor)holder. executor (ElasticSearch, ThreadPool.java)

17

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

B1.8. throwOnFalse.Weak.Elaborator.Expression (B.8)

if (!buffer. isReadable()) throw new EOFException(); (Netty, ByteBufInputStream.java) B1.9. throwOnNull.Weak.Elaborator.Expression

(B.9) (Guice, InternalInjectorCreator.java) B1.10. notNullRequired.Atomic.Command

(B.10) (Junit, Categories.java.java) B1.11. notNullAssumed.Atomic.Command

(B.11)

(ElasticSearch, MapperQueryParser.java) B1.12. holdsOrReturn.Atomic.Command (B.12)

if (environment = =null) return; (Titan, BerkeleyJEStoreManager.java) B1.13. ignoringExceptions.Elaborator.Composite.Command

(B.13) (Guice, BytecodeGen.java) B1.14. letInNext.Elaborator.Composite.Command

String method Name = each. getMethodName(); return (methodName = = null ? Request. aClass($) : Request. method($, methodName)). getRunner();

(B.14)

(Junit, MaxCore.java) B2. Multitude B2.1. forFromTo.Iterative.Composite.Command (B.15) (ElasticSearch, ExceptionsHelper.java) B2.2. aggregate.Multitude.Expression

for (JRubyWorldDefinition w : worldDefinitions) currentWorld = w. execute(currentWorld);

(B.16)

(Cucumber-Jvm, JRubyBackend.java) B2.3. selectBy.Select.Multitude.Expression

for (Object a : args) jrubyArgs. add(a = = null ? null : ! (a instanceof DataTable) ? stepdef. getRuntime(). newString((String)a) : (B.17)

JavaEmbedUtils. javaToRuby(stepdef. getRuntime(), a)); (Cucumber-Jvm, JRubyBackend.java)

18

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

B2.4. forEach.Iterative.Composite.Command

for (String columnName : columnNames) let(()

> fieldNames. indexOf((new CamelCaseStringConverter()). map(columnName))). in(index > $[n + + ] = index = = 1 ? ”” : fieldValues. get(index));

(B.18)

(Cucumber-Jvm, ComplexTypeWriter.java) B2.5. firstSuchThat.Select.Multitude.Expression

for (int x = 0 ; x < output2. length; ++ x) if (output2. bytes[x + output2 . offset] = = payloadSep){ sepIndex = x; break; }

(B.19)

(ElasticSearch, XAnalyzingSuggester.java) B3. Phrase B3.1. myArguments.Phrase

@Override public boolean hasBankAccount(String bank, String name){ return this. BOSEconomy. isBankOwner(bank, name) this. BOSEconomy. isBankMember(bank, name); }

(B.20)

(Essentials, BOSE7.java) B3.2. myName.Phrase

protected String uuid(AtmosphereResource r) { return r. uuid(); }

(B.21)

(atmosphere, UUIDBroadcasterCache.java) B4. Field B4.1. const.Atomic.Field (B.22)

public static final String FACTORSYSP ROP = ts. timeout. factor”; (WildFly, TimeoutUtil.java) B4.2. setter.Elaborator.Field

public void setName(String name){ this. name = name; }

(B.23)

(WildFly, ManagedCreateLdapServer.java) B4.3. getter.Elaborator.Field

public String getProductName(){ return name; }

(B.24)

(WildFly, ProductConfig.java) B4.4. fluentSetter.Elaborator.Field

public Builder setNumericType(NumericType x){ this. numericType = x; return this; }

(B.25)

(ElasticSearch, PackedArrayIndexFieldData.java) B4.5. vanillaCollection.Atomic.Field

private final List < List < String > > arguments = new ArrayList < List < String > >(); (WildFly, Usage.java)

19

(B.26)

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

B5. Sink Method B5.1. constantValue.Atomic.Method

@Override protected String getOutputSuffix(){ return “< /div > ”; }

(B.27)

(Docx4j, SvgConversionContext.java) B5.2. thrower.Atomic.Method

public long time(){ throw new UnsupportedOperationException(); }

(B.28)

(ElasticSearch, VersionValue.java) B5.3. defaultValue.Atomic.Method

@Override public Permission getRequiredPermission(){ return null; }

(B.29)

(Hazelcast, AuthenticationRequest.java) B5.4. examiner.Atomic.Method

public boolean shouldReply(){ return !noreply; }

(B.30)

(Hazelcast, TouchCommand.java) B5.5. letIn.Atomic.Method

public static long addJitter( final long pause, final float jitter){ long $ = pause + jitter * (long)pause * (RANDOM. nextFloat() return $ > 0? $: 1 ; }

0. 5f); (B.31)

(Hbase, ConnectionUtils.java) B6. Delegating Method B6.1. factory.Elaborator.Method

protected AsyncSupportResolver createAsyncSupportResolver(){ return new DefaultAsyncSupportResolver(config); }

(B.32)

(Atmosphere, AtmosphereFramework.java) B6.2. defaultArguments.Elaborator.Method

public static void validate(String s, Type t){ validate(s, t, false); }

(B.33)

(Atmosphere, UriComponent.java) B6.3. adjuster.Elaborator.Method

public void trackSynchronously(FocusPoint x){ httpRequest. request(urlBuildingStrategy. buildURL(x)); }

(B.34)

(Atmosphere, JGoogleAnalyticsTracker.java)

20

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

B6.4. delegator.Elaborator.Method

@Override public Action doService(AtmosphereRequest r, AtmosphereResponse res) throws IOException, ServletException{ return super. service(r, res); }

(B.35)

(Atmosphere, Tomcat7BIOSupportWithWebSocket.java) B7. Constructor B7.1. pojo.Atomic.Constructor

private WriteResult(AtmosphereResource r, Object message){ this. r = r; this. message = message; }

(B.36)

(Atmosphere, JSR356WebSocket.java) B7.2. super.Elaborator.Constructor

public WebLogicServlet30WithWebSocket(AtmosphereConfig config){ super(config); }

(B.37)

(Atmosphere, WebLogicServlet30WithWebSocket.java) B7.3. this.Elaborator.Constructor

public AtmosphereServlet(boolean isFilter){ this(isFilter, true); }

(B.38)

(Atmosphere, AtmosphereServlet.java) Appendix C. Reproducibility Information For each project, the table reports r, ri, rx, the difference between the two and the differences between r, ri, rx, values in an initial version of the project and the latest one. Table C.1 Baseline corpus of 26 OSS Java projects: Identification, reproducibility information and work volume indicators. Project

First Version

Last Version

Last Hash

#Days

#Authors

Atmospherea CraftBukkit Cucumber-jvm Docx4j Elasticsearch Essentials Guavaa Guice Hadoop-common Hazelcast Hbase Hector Hibernate-orm Jclouds Jna Junita K-9 Lombok Mongo-java-driver Netty Openmrs-core RxJava Spring-framework Titan Voldemort Wildfly

10-04-30 11-01-01 11-06-27 12-05-12 11-10-31 11-03-19 11-04-15 07-12-19 11-08-25 09-07-21 12-05-26 10-12-05 09-07-07 09-04-28 11-06-22 07-12-07 08-10-28 09-10-14 09-01-08 11-12-28 10-08-16 13-01-23 10-10-25 13-01-04 01-01-01 10-06-08

14-04-28 14-04-23 14-07-22 14-07-04 14-06-20 14-04-27 14-02-25 14-07-01 14-08-21 14-07-05 14-01-30 14-05-28 14-07-02 14-04-25 14-07-07 14-05-03 14-05-04 14-07-01 14-06-16 14-01-28 14-06-18 14-04-25 14-01-28 14-04-17 14-04-28 14-04-22 Total:

557e1044 62ca8158 fd764318 8edaddfa 812972ab 229ff9f0 6fdaf506 76be88e8 0c648ba0 3c4bc794 c67682c8 f2fc542c b0a2ae9d f1a0370b a5942aaf 56a03468 95f33c38 3c4f6841 5565c46e 3061b154 05292d98 12723ef0 c5f908b1 55de01a3 fb3203f3 5a29c860 26 projects

1,459 1,208 1,120 783 963 1,134 1,047 2,386 1,092 1,809 613 1,270 1,821 1,823 1,110 2,338 2,014 1,721 1,984 762 1,401 456 1,190 468 4,865 1,413 38,250

62 156 93 19 129 67 12 17 69 65 25 95 150 100 46 91 81 22 75 72 119 47 53 17 56 194 1,932

a

Highlighted projects were included in the biased corpus (Table 6.1). 21

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

Table C.2 Reproducibility information for the Java corpus consisting of the 26 most starred Java projects on GitHub. Project

Files

LOC

Classes

Methods

Commit Id

Date

bazel clojure disruptor dropwizard druid dubbo fastjson gradle Hystrix java-design-patterns jedis kafka kotlin leakcanary metrics mockito mybatis-3 neo4j okhttp presto retrofit spark spring-boot storm webmagic zipkin

3,028 170 190 749 2,748 1,210 2,035 6,545 399 894 167 1,095 4,762 56 222 823 1,084 6,434 277 3,759 175 171 3,237 449 213 324

591,275 58,705 20,922 57,080 462,726 163,255 163,270 510,172 76,561 59,236 38,719 177,021 597,691 5,721 25,297 68,774 85,285 944,638 72,626 553,854 23,841 19,047 333,777 67,422 17,125 43,748

3,807 739 149 525 2,501 1,211 1,336 7,103 411 601 126 1,146 5,524 56 211 494 790 6,241 290 3,652 193 136 2,722 665 185 309

25,401 4,135 676 2,287 16,265 8,546 6,925 37,592 2,919 1,909 4,294 7,122 26,231 335 1,187 2,482 4,631 41,344 2,429 27,885 811 824 11,359 5,693 959 2,009

5dbb23b c0326d2 2314a27 e823137 78b8a57 27917f2 490213c 2f9dac0 c674545 cca4760 2c6ade7 5b682ba fa5728b f0bfbbd 6b6aefb fad7211 ed2cc17 8ae7eac 89621df 0c6c3b2 39fcdb4 9618b83 3a8be10 cdb116e a872a64 4624229

Fri Feb 24 17:14:35 2017 Fri Oct 28 22:24:47 2016 Mon Nov 21 20:16:47 2016 Fri Feb 24 19:55:30 2017 Sat Feb 25 11:30:30 2017 Wed Jun 15 12:09:37 2016 Tue Feb 21 05:04:12 2017 Sun Feb 26 08:06:12 2017 Fri Feb 10 21:05:15 2017 Sun Feb 12 00:33:30 2017 Wed Feb 22 09:43:01 2017 Sun Feb 26 04:44:32 2017 Fri Feb 24 17:58:25 2017 Sun Feb 26 10:12:58 2017 Fri Feb 24 20:05:34 2017 Sun Feb 26 01:25:29 2017 Sat Feb 25 04:04:18 2017 Fri Feb 24 23:34:00 2017 Wed Feb 15 05:53:53 2017 Sat Feb 25 03:40:27 2017 Fri Feb 24 19:51:33 2017 Thu Feb 23 10:58:28 2017 Fri Feb 24 18:46:04 2017 Fri Dec 13 20:53:13 2013 Sat Feb 25 16:46:29 2017 Wed Feb 15 16:45:00 2017

Table C.3 Reproducibility information for the Android corpus consisting of the 26 most starred Android projects on GitHub. Project

Files

LOC

Classes

Methods

Commit Id

Date

android-async-http Android-CleanArchitecture Android-Universal-Image-Loader androidannotations AndroidUtilCode butterknife EventBus fresco glide HomeMirror iosched libgdx lottie-android Material-Animations material-dialogs MaterialDesignLibrary MPAndroidChart PhotoView picasso plaid PocketHub RxAndroid SlidingMenu tinker ViewPagerIndicator weex

83 81 88 878 84 116 81 809 442 27 463 2,175 85 15 35 30 235 29 72 144 271 11 33 213 42 963

12,609 5,207 13,857 68,031 18,330 11,795 7,707 106,284 54,246 2,462 70,687 384,154 7,138 1,240 8,381 3,771 42,569 3,421 12,675 24,446 36,190 1,666 4,932 39,867 4,059 185,533

94 72 114 708 87 100 41 762 501 41 568 2,553 119 15 62 38 245 43 77 220 260 10 45 291 50 957

884 310 799 3,218 1,011 255 195 5,186 2,673 140 3,151 39,853 603 121 646 272 2,285 282 501 1,730 1,828 42 428 1,917 274 5,836

f53deef ae30300 4c72879 1e91a3b a2e6a93 581666a 7504c43 9bf8cf5 3bcea91 a71c860 9f38e2d 6302af8 09f459c 28e65b2 2205c69 7271050 0c919ab 6c227ee 0b38d19 abf8669 a4f3a6f eb27d75 4254fec 9d2b250 8cd549f 4733936

Sun Mar 20 15:55:25 2016 Mon Jan 9 00:30:18 2017 Tue Jan 26 13:29:19 2016 Sun Feb 26 16:56:35 2017 Mon Feb 27 13:33:55 2017 Tue Jan 24 18:57:50 2017 Wed Oct 19 13:33:50 2016 Fri Feb 24 22:04:35 2017 Wed Feb 15 18:32:15 2017 Fri May 20 21:54:56 2016 Tue Nov 15 22:13:20 2016 Mon Feb 27 02:18:33 2017 Sat Feb 25 04:17:52 2017 Sun May 15 20:30:33 2016 Tue Feb 21 17:50:45 2017 Mon Nov 14 23:06:39 2016 Wed Feb 22 17:29:33 2017 Sun Dec 18 02:13:52 2016 Fri Feb 24 17:18:20 2017 Thu Feb 23 15:34:32 2017 Mon Feb 27 10:54:37 2017 Mon Jan 9 20:59:49 2017 Sun Mar 9 23:30:35 2014 Tue Jan 17 05:47:08 2017 Wed Sep 12 19:08:57 2012 Mon Feb 27 07:32:23 2017

Table C.4 Values of the r-, ri- and rx- indices in the baseline corpus (Table C.1) at the end of the monitored period, and differences from initial values. Project

r

ri

rx

ri

Atmosphere CraftBukkit Cucumber Docx4j Elasticsearch

34 42 16 54 63

25 39 13 46 59

22 17 12 28 27

3 22 1 18 32

rx

Δr

Δri

Δrx

17 41 5 7 34

12 38 5 8 31

8 16 3 1 12

(continued on next page)

22

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

Table C.4 (continued) Project

r

ri

rx

ri

Essentials Guava Guice Hadoop-common Hazelcast Hbase Hector Hibernate-orm Jclouds Jna Junit K-9 Lombok Mongo-java-driver Netty Openmrs-core RxJava Spring-framework Titan Voldemort Wildfly

41 33 25 99 52 89 20 83 47 26 15 46 36 25 37 65 17 63 28 52 106

29 32 20 90 48 72 17 75 39 22 14 42 30 23 35 52 15 59 23 45 94

27 13 16 47 20 47 12 35 29 14 9 28 21 10 15 43 6 29 19 31 46

2 19 4 43 28 25 5 40 10 8 5 14 9 13 20 9 9 30 4 14 48

rx

Δr

Δri

Δrx

11 17 15 67 17 56 7 18 12 14 3 16 20 0 15 42 0 13 10 4 63

6 17 11 64 15 44 9 14 11 14 5 17 20 0 13 32 2 16 8 3 59

6 6 10 25 3 26 2 6 2 4 1 6 8 0 3 30 -6 -3 7 5 24

Table C.5 The 5 sub-nanos augmenting the language of Table 4.1.

1 s2 3 4 5 a b c d

(*)a

(*)b

(*)c

Name

Intentd

G2 G4 H2 H5 H7

12% 0% 0% 23% 19%

✓ ✓ ✓ ✓ ✓

anyHolds allHolds countIf forEachSuchThatDo aggregateWith

any M holds p( · ) ∃x ∈ M, p(x) M.anyMatch(…) all M holds p( · ) ∀x ∈ M, p(x) M.allMatch(…) count M [s.t. p( · )] #{x M|p (x )} M.filter(…).count() for M s.t. p( · ) do C ∀x ∈ M, p(x): C M.filter(…).forEach(…) AggregateM [st p( · )] with A( · , · ) M.filter(…).reduce(…)

Candidate(s) from which this nano was created Prevalence score in the baseline corpus Can be emulated by lazy-evaluation, using lambda functions e.g., pseudo-code, Java 8 streams, suggestion for extending plain Java syntax, etc.

Appendix D. Java Like Stenography We allow method declaration to include stenography extension for some of the nano-patterns: (1) A method may have a body even if ends with “;”, provided that the method declaration defines how the function return value (or body in case of void function) is implemented. (2) After the method header, one may add = e, which means that the function’s return value is computed by evaluating this expression. This specification replaces the function’s body. (3) After the method header, one may add → g, which implies that the function’s return value is computed by delegating to function g, which is declared elsewhere. (4) The formal arguments list may be interspersed with expressions, which depend on other real formal arguments. (5) A formal argument definition may be written as: Double pow(double d Double.valueOf(d), int n Integer.valueOf(n));

[7] A. Deo, B. Williams, Preliminary study on assessing software defects using nanopattern detection, 2th4 Int. Conf. Soft. Eng. & Data Eng., SEDE 2015, IEEE, NY, NY, 2015. [8] P. Andras, A. Pakhira, L. Moreno, A. Marcus, A measure to assess the behavior of method stereotypes in OO software, 2013 4th Int. Workshop on Emerging Trends in Software Metrics (WETSoM), IEEE, Washington, DC, USA, 2013, pp. 7–13, https:// doi.org/10.1109/WETSoM.2013.6619330. [9] M. Allamanis, C.A. Sutton, Mining idioms from source code, in: S. Cheung, A. Orso, M.D. Storey (Eds.), Proc. the 2nd 2 “ACM” Found. Soft. Eng., (FSE-22), ACM, Hong Kong, 2014, pp. 472–483, , https://doi.org/10.1145/2635868.2635901. [10] J.E. Hirsch, An index to quantify an individual’s scientific research output, Proc. National Academy of Sciences of the USA of America 102 (46) (2005) 16569–16572, https://doi.org/10.1073/pnas.0507655102. National Academy of Sciences [11] J. Gil, G. Lalouche, When do soft. complexity metrics mean nothing?—when examined out of context, J. Object Tech. 15 (1) (2016) 2:1–25, https://doi.org/10. 5381/jot.2016.15.5.a2. [12] PLoP, Pattern Lang. Programs (PLoP) Conf. Series, 2017.

References [1] ISO, Iso general purpose metric screw threads - selected sizes for screws, bolts & nuts, 1998. [2] JEDEC, JEDEC solid state Tech. association, 2017. [3] J. Gil, I. Maman, Micro patterns in Java code, OOPSLA ’05 Proc. the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications (2005) 97, https://doi.org/10.1145/1103845.1094819. [4] F. Batarseh, Java nano patterns: A set of reusable objects, Proc. 4th8 Ann. Southeast Regional Conf. ACM, ACM, NY/USA, 2010, pp. 60:1–60:4. [5] J. Singer, G. Brown, M. Luján, A. Pocock, P. Yiapanis, Fundamental nano-patterns to characterize and classify java methods, Elect. Notes Theor. Comp. Sci. 253 (7) (2010) 191–204, https://doi.org/10.1016/j.entcs.2010.08.042. Springer Publishing, New York City, NY, USA [6] K.Z. Sultana, A. Deo, B.J. Williams, A preliminary study examining relationships between nano-patterns and software security vulnerabilities, 1 (2016), pp. 257–262. IEEE,Washington, DC, USA

23

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al.

[41] S. Marion, R. Jones, C. Ryder, Decrypting the Java gene pool, Proc. 6th Int. Symp. on Memory Management, ISMM ’07, ACM, New York, NY, USA, 2007, pp. 67–78, https://doi.org/10.1145/1296907.1296918. [42] S. Kim, T. Zimmermann, K. Pan, E.J.J. Whitehead, Automatic identification of bugintroducing changes, Proc. 21th Int. Conf. on Aut. Soft. Eng. ASE ’06, IEEE Computer Society, Washington, DC, USA, 2006, pp. 81–90, https://doi.org/10.1109/ASE. 2006.23. [43] S. Bajracharya, T. Ngo, E. Linstead, Y. Dou, P. Rigor, P. Baldi, C. Lopes, Sourcerer: A search engine for open source code supporting structure-based search, Companion to the 21th ACM SIGPLAN Symposium on Object-oriented Programming Systems, Languages, and Applications, OOPSLA ’06, ACM, New York, NY, USA, 2006, pp. 681–682, https://doi.org/10.1145/1176617.1176671. [44] R. Schauer, R.K. Keller, Pattern visualization for software comprehension, Proc. 6th Int. Workshop on Program Comprehension, IWPC ’98, IEEE Computer Society, Washington, DC, USA, 1998. [45] E. Gamma, R. Helm, R. Johnson, J. Vlissides, Design patterns: Abstraction and reuse of object-oriented design, (ECOOP’93), 707 (1993), pp. 406–431. [46] D.A. Watt, Programming Language: Concepts and Paradigms, (1990). [47] B. Lee, R. Grimm, M. Hirzel, K.S. McKinley, Marco: Safe, expressive macros for any language, Proc. of the 26th European Conf. on Object-Oriented Programming, ECOOP’12, Springer-Verlag, Berlin, Heidelberg, 2012, pp. 589–613, https://doi. org/10.1007/978-3-642-31057-7_26. [48] S. Erdweg, T. Rendel, C. Kästner, K. Ostermann, Sugarj: Library-based syntactic language extensibility, SIGPLAN Not. 46 (10) (2011) 391–406, https://doi.org/10. 1145/2076021.2048099. [49] M. Allamanis, C.A. Sutton, Mining source code repositories at massive scale using language modeling, in: T. Zimmermann, M.D. Penta, S. Kim (Eds.), Proc. the 10th Working Conf. Mining Soft. Repositories, MSR ’13, San Francisco, CA, USA, May 1819, 2013, IEEE, NY/USA, 2013, pp. 207–216, , https://doi.org/10.1109/MSR.2013. 6624029. [50] M. Hollander, D.A. Wolfe, E. Chicken, Nonparametric Statistical Methods, Wiley Series in Probability and Statistics, Wiley, 2013. [51] M. Mernik, J. Heering, A.M. Sloane, When and how to develop domain-specific languages, ACM Comput. Surv. 37 (4) (2005) 316–344. [52] T. Kosar, S. Bohra, M. Mernik, Domain-Specific Languages: A Systematic Mapping Study, Information and Software Technology 71 (2016) 77–91, https://doi.org/10. 1016/j.infsof.2015.11.001. [53] A. Tomer, L. Goldin, T. Kuflik, E. Kimchi, S.R. Schach, Evaluating software reuse alternatives: A model and its application to an industrial case study, IEEE Trans. Softw. Eng. 30 (9) (2004) 601–612, https://doi.org/10.1109/TSE.2004.50. [54] L. Egghe, R. Rousseau, An infometric model for the Hirsch-index, Scientometrics 69 (1) (2006) 121–129, https://doi.org/10.1007/s11192-006-0143-8. Springer Publishing, New York City, NY, USA [55] T. Cohen, J. Gil, Self-calibration of metrics of Java methods, TOOLS Pacific’00: 37th Int. Conf. Tech. OO Lang. & Syst. IEEE, Washington, DC, 2000, pp. 94–107, https:// doi.org/10.1109/TOOLS.2000.891361. IEEE,Washington, DC, USA [56] G. Concas, M. Marchesi, S. Pinna, N. Serra, Power-laws in a large oo soft. syst. IEEE 33 (10) (2007) 687–708, https://doi.org/10.1109/TSE.2007.1019. [57] G. Concas, M. Marchesi, A. Murgia, R. Tonelli, I. Turnu, On the distribution of bugs in the Eclipse system, IEEE 37 (6) (2011) 872–877, https://doi.org/10.1109/TSE. 2011.54. [58] I. Turnu, G. Concas, M. Marchesi, S. Pinna, R. Tonelli, A modified yule process to model the evolution of some OO system properties, Inf. Sc. 181 (4) (2011) 883–902, https://doi.org/10.1016/j.ins.2010.10.022. Springer Publishing, New York City, NY, USA [59] Y. Gil, C. Monni, Fitting long-tailed distribution to empirical data, Concurrency and Computation: Practice and Experience 29 (24) (2017), https://doi.org/10.1002/ cpe.4223. [60] A. Decan, T. Mens, M. Claes, An empirical comparison of dependency issues in OSS packaging ecosystems, Proc. 24th Conf. Soft. Analysis, Evolution, & Reengineering. SANER 2017, IEEE, NY, NY, 2017. [61] E. Gamma, Going beyond objects with design patterns, ECOOP’97—OO Prog.: 11th European Conf. Jyväskylä, Finland, Jun 9–13, 1997 Proc. Springer, Berlin, Heidelberg, 1997, https://doi.org/10.1007/BFb0053394. 530–530 [62] P. Van Emde Boas, Resistance is Futile; Formal linguistic observations on design patterns, techreport, The Institute For Logic, Language, and Computation (ILLC), University of Amsterdam, 1997. [63] A.H. Eden, J. Gil, Y. Hirshfeld, A. Yehudai, Motifs in object oriented architecture, 1999. [64] A.H. Eden, J. Gil, A. Yehudai, A formal language for design patterns, (PLoP’96), (1996). [65] A.H. Eden, Formal specification of object-oriented design, Int. Conf. on Multidisciplinary Design in Engineering (CSME-MDE’01), (2001), pp. 21–22. [66] A.H. Eden, A visual formalism for object-oriented architecture, (IDPT’02), (2002). [67] A.H. Eden, Giving “The quality” a name - precise specification of design patterns - a second look at the manuscripts. Journal of Object-Oriented Programming 11 (3) (1998). [68] R. Ferenc, Á. Beszédes, L.J. Fülöp, J. Lele, Design pattern mining enhanced by machine learning, 21th “IEEE” 2005), 25-30 Sep 2005, Budapest, Hungary, IEEE, NY/USA, 2005, pp. 295–304, https://doi.org/10.1109/ICSM.2005.40. [69] Y.-G. Guéhéneuc, J.-Y. Guyomarc’h, H. Sahraoui, Improving design-pattern identification: a new approach & an exploratory study, Soft. Quality J. 18 (1) (2010) 145–174, https://doi.org/10.1007/s11219-009-9082-y. Kluwer Academic Publishers, Hingham, MA, USA [70] M. Zanoni, F.A. Fontana, F. Stella, On applying machine learning techniques for design pattern detection, J. Syst. and Soft. 103 (2015) 102–117, https://doi.org/10.

[13] D. Spinellis, K. Raptis, Component mining: A process and its pattern language, Inf. and Soft. Tech. 42 (9) (2000) 609–617. [14] U. Zdun, Systematic pattern selection using pattern language grammars & design space analysis, Soft. Pract. & Expr. 37 (9) (2007) 983–1016, https://doi.org/10. 1002/spe.799. [15] Y. Gil, M. Orrú, The spartanizer: Massive automatic refactoring, Software Analysis, Evolution and Reengineering (SANER), 2017 IEEE 24th Int. Conf. on, IEEE, 2017, pp. 477–481. [16] E. Gamma, R. Helm, R. Johnson, J. Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software, Professional Computing series, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1995. [17] S.K. Abd-El-Hafiz, V.R. Basili, A knowledge-based approach to the analysis of loops, IEEE Trans. Softw. Eng. 22 (5) (1996) 339–360, https://doi.org/10.1109/32. 502226. [18] J. Gil, I. Maman, Micro patterns in Java code, in: R. Johnson, R.P. Gabriel (Eds.), Proc. the 2th0 Annual OO Prog., Syst., Lang., and App., OOPSLA 2005, 40 ACM, NY/ USA, 2005, pp. 97–116, , https://doi.org/10.1145/1103845.1094819. [19] N. Dragan, M.L. Collard, J.I. Maletic, Reverse engineering method stereotypes, 2006 22nd IEEE Int. Conf. on Software Maintenance, IEEE, Washington, DC, USA, 2006, pp. 24–34, https://doi.org/10.1109/ICSM.2006.54. [20] E.W. Høst, B.M. Østvold, The programmer’s lexicon, volume i: The verbs, Seventh IEEE Int. Working Conf. on Source Code Analysis and Manipulation (SCAM 2007), (2007), pp. 193–202, https://doi.org/10.1109/SCAM.2007.18. IEEE, Washington, DC, USA [21] E.W. Høst, B.M. Østvold, The java prog.’s phrase book, in: D. Gasevic, R. Lämmel, E.V. Wyk (Eds.), Soft. Language Eng., First Int. Conf., SLE 2008, Toulouse, France, Sep 29-30, 2008. Revised Selected Papers, Lecture Notes in Comp. Science, 5452 Springer, NY/USA, 2008, pp. 322–341, , https://doi.org/10.1007/978-3-64200434-620. [22] N. Dragan, M.L. Collard, J.I. Maletic, Using method stereotype distribution as a signature descriptor for software systems, 2009 IEEE International Conference on Software Maintenance, (2009), pp. 567–570, https://doi.org/10.1109/ICSM.2009. 5306394. [23] A. Sutton, R. Holeman, J.I. Maletic, Identification of idiom usage in c++ generic libraries, 2010 IEEE 1th8 Int. Conf. Program Comprehension, IEEE, New York,NY, 2010, pp. 160–169, https://doi.org/10.1109/ICPC.2010.37. [24] G. Sridhara, L. Pollock, K. Vijay-Shanker, Automatically detecting & describing high level actions within methods, Proc. 3th3 Int. Conf. Soft. Eng. - ICSE ’11, (2011). [25] V. Arnaoudova, M. Di Penta, G. Antoniol, Y.-G. Guéhéneuc, A new family of software anti-patterns: Linguistic anti-patterns, 2013 17th European Conference on Software Maintenance and Reengineering, (2013), pp. 187–196, https://doi.org/ 10.1109/CSMR.2013.28. [26] X. Wang, L. Pollock, K. Vijay-Shanker, Developing a model of loop actions by mining loop characteristics from a large code corpus, 2015 IEEE 3th1 Int. Conf. Soft. Maint. & Evolution, ICSME 2015 - Proc. (2015). IEEE,Washington, DC, USA [27] M. Allamanis, E.T. Barr, C. Bird, P. Devanbu, M. Marron, C. Sutton, Mining Semantic Loop Idioms from Big Code, Technical Report, (2016). [28] V. Arnaoudova, M. Di Penta, G. Antoniol, Linguistic antipatterns: what they are and how developers perceive them, Empirical Software Engineering 21 (1) (2016) 104–158, https://doi.org/10.1007/s10664-014-9350-8. [29] I. Lee, S. Kim, S. Park, Y. Cho, Attributes for Characterizing Java Methods, Springer, Berlin, Heidelberg, pp. 185–191. [30] M. Gabel, Z. Su, A study of the uniqueness of source code, Proc. 1th8 ACM SIGSOFT Int. Symp. Found. Soft. Eng. FSE ’10, ACM, NY/USA, 2010, pp. 147–156, https:// doi.org/10.1145/1882291.1882315. [31] Z. Tu, Z. Su, P. Devanbu, On the localness of software, Proc. the 2th2 ACM SIGSOFT Int. Symp. Found. Soft. Eng.- FSE 2014, ACM, NY/USA, 2014, pp. 269–280, https:// doi.org/10.1145/2635868.2635875. [32] A. Hindle, E.T. Barr, Z. Su, M. Gabel, P. Devanbu, On the naturalness soft. Proc. 3th4 Int. Conf. Soft. Eng. ICSE ’12, IEEE, NJ/USA, 2012, pp. 837–847. [33] D. Hou, D.M. Pletcher, Towards a better code completion system by api grouping, filtering, and popularity-based ranking, Proc. of the 2th Int. Workshop on Recommendation Systems for Soft. Eng. RSSE ’10, ACM, New York, NY, USA, 2010, pp. 26–30, https://doi.org/10.1145/1808920.1808926. [34] R. Fiutem, P. Tonella, G. Antoniol, E. Merlo, A cliche-based environment to support architectural reverse engineering, 1996 Proceedings of International Conference on Software Maintenance, (1996), pp. 319–328, https://doi.org/10.1109/ICSM.1996. 565035. [35] Z. Koono, H. Abolhassani, H. Chen, A new way of automatic design of software (simulating human intentional activity), Proc. of the 2006 Conf. on New Trends in Soft. Methodologies, Tools and Techniques: Proc. of the 5th SoMeT_06, IOS Press, Amsterdam, The Netherlands, 2006, pp. 407–420. [36] M.A. Upal, S. Padmanabhuni, Learning how to program, Eng. Solutions for the Next Millennium. 1999 IEEE Canadian Conf. on Electrical and Computer Eng. (Cat. No.99TH8411), 2 (1999), pp. 1052–1056vol.2, https://doi.org/10.1109/CCECE. 1999.808192. [37] L.M. Wills, Automated program recognition: A feasibility demonstration, Artif. Intell. 45 (1-2) (1990) 113–171, https://doi.org/10.1016/0004-3702(90)90039-3. Elsevier Science Publishers Ltd. Essex, UK [38] C. Rich, R.C. Waters, Automatic programming: Myths and prospects, IEEE Computer 21 (1988) 40–51. [39] X. Wang, L. Pollock, K. Vijay-Shanker, Automatic segmentation of method code into meaningful blocks: Design & evaluation, J. Soft.: Evolution & Process, (2014). [40] J. Singer, C. Kirkham, Exploiting the correspondence between micro patterns and class names, Int. Working Conf. on Source Code Analysis and Manipulation, (2008), pp. 67–76, https://doi.org/10.1109/scam.2008.23.

24

Journal of Computer Languages 54 (2019) 100905

Y. Gil, et al. 1016/j.jss.2015.01.037. Springer Publishing, New York City, NY, USA [71] Y. Guéhéneuc, H.A. Sahraoui, F. Zaidi, Fingerprinting design patterns, 11th Working Conf. Rev. Eng., WCRE 2004, Delft, The Netherlands, Nov 8-12, 2004, IEEE, NY/USA, 2004, pp. 172–181, https://doi.org/10.1109/WCRE.2004.21. [72] M. Vokác, An efficient tool for recovering design patterns from c++ code. J. of Object Technology 5 (1) (2006) 139–157. [73] L. Wang, Z. Han, J. He, H. Wang, X. Li, Recovering design patterns to support program comprehension, Proc. of the 2th Int. Workshop on Evidential Assessment of Soft. Technologies, EAST ’12, ACM, New York, NY, USA, 2012, pp. 49–54, https:// doi.org/10.1145/2372233.2372248. [74] E. Linstead, L. Hughes, C. Lopes, P. Baldi, Exploring Java software vocabulary: A search & mining perspective, Proc. Int. Conf. Soft. Eng. (2009), pp. 29–32, https:// doi.org/10.1109/SUITE.2009.5070017. IEEE,Washington, DC, USA [75] E. Enslen, E. Hill, L. Pollock, K. Vijay-Shanker, Mining source code to automatically split identifiers for software analysis, 2009 6th IEEE Int. Working Conf. on Mining Software Repositories, IEEE, Washington, DC, USA, 2009, pp. 71–80, https://doi. org/10.1109/MSR.2009.5069482. [76] S.L. Abebe, S. Haiduc, A. Marcus, P. Tonella, G. Antoniol, Analyzing the evolution of the source code vocabulary, 2009 13th European Conf. on Software Maintenance and Reengineering, IEEE, Washington, DC, USA, 2009, pp. 189–198, https://doi. org/10.1109/CSMR.2009.61. [77] C.D. Newman, R.S. AlSuhaibani, M.L. Collard, J.I. Maletic, Lexical categories for source code identifiers, 2017 IEEE 24th Int. Conf. on Software Analysis, Evolution and Reengineering (SANER), (2017), pp. 228–239, https://doi.org/10.1109/ SANER.2017.7884624. IEEE,Washington, DC, USA [78] E.W. Høst, B.M. Østvold, Debugging method names, Proc. the 23rd European Conf. on ECOOP 2009—Object-Oriented Programming, Genoa, Springer-Verlag, Berlin, Heidelberg, 2009, pp. 294–317, https://doi.org/10.1007/978-3-642-03013-014. [79] Y. Kashiwabara, Y. Onizuka, T. Ishio, Y. Hayase, T. Yamamoto, K. Inoue, Recommending verbs for rename method using association rule mining, 2014 Software Evolution Week - IEEE Conf. on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE), (2014), pp. 323–327, https://doi.org/10. 1109/CSMR-WCRE.2014.6747186. IEEE,Washington, DC, USA [80] Y. Kashiwabara, T. Ishio, H. Hata, K. Inoue, Method verb recommendation using association rule mining in a set of existing projects, IEICE Transactions on Information and Systems E98.D (3) (2015) 627–636, https://doi.org/10.1587/ transinf.2014EDP7276.

[81] L. Moreno, A. Marcus, Jstereocode: automatically identifying method and class stereotypes in java code, 2012 Proc. the 27th IEEE/ACM Int. Conf. on Automated Software Engineering, IEEE, New York, NY, 2012, pp. 358–361, https://doi.org/10. 1145/2351676.2351747. [82] R. Brooks, Towards a theory of the comprehension of computer programs, Int. J. Man-Machine Studies 18 (6) (1983) 543–554. Elsevier Science Publishers, Amsterdam, The Netherlands [83] M.E. Crosby, J. Scholtz, S. Wiedenbeck, The roles beacons play in comprehension for novice & expert programmers, in: J. Kuljis, L. Baldwin, R. Scoble (Eds.), Proc. 14th Ann. Workshop of the Psychology of Programmers Interest Group (PPIG), 2002, pp. 58–73. [84] D. Qiu, B. Li, E.T. Barr, Z. Su, Understanding the syntactic rule usage in Java, J. Syst. and Soft. 123 (2017) 160–172, https://doi.org/10.1016/j.jss.2016.10.017. Springer Publishing, New York City, NY, USA [85] Wikibooks, More C++ Idioms, 2017. https://en.wikibooks.org/wiki/More_C%2B %2B_Idioms. [86] J.I. Editor, Java Idioms, 2017. http://c2.com/ppr/wiki/JavaIdioms/JavaIdioms. html. [87] S. Chuan, JavaScript Patterns Collection, 2014. http://shichuan.github.io/ javascript-patterns/. [88] R. Waldron, Principles of writing consistent, idiomatic JavaScript, 2014. https:// github.com/rwaldron/idiomatic.js/. [89] E. Recommenders-Contributors, Eclipse SnipMatch, 2014. [90] JetBrains, High-speed coding with Custom Live Templates., 2014. http://bit.ly/ 1o8R8Do. [91] N. Gurewich, O. Gurewich, Java Manual of Style, Ziff-Davis Publishing Co., 1996. [92] C. Laffra, Advanced Java: Idioms, Pitfalls, Styles & Prog. Tips, Prentice Hall Ptr, 1996. [93] C. Larman, R. Guthrie, Java 2 Performance & Idiom Guide, Prentice Hall Ptr, NJ/ USA, 1999. [94] A. Koenig, Idiomatic design, Comm. ACM (1995) 14–19. ACM, New York, NY, USA [95] A. Langer, Java programming idioms, Proc. OO Lang. & Syst.. TOOLS 38, (2001), pp. 197–198, https://doi.org/10.1109/TOOLS.2001.911773. IEEE,Washington, DC, USA [96] K. Mens, I. Michiels, R. Wuyts, Supporting software development through declaratively codified programming patterns, Expert Syst. with App. (2002).

25