First International Workshop on Properties of Software Engineering Research (PROPSER)


This workshop aims to bring together researchers at various points in their career and practitioners alike, for discussing the systematic description and evaluation of software engineering research.


The workshop is motivated by two observations:

  1. writing and reviewing software engineering research results is often hindered by the lack of a community understanding what are important questions (hypotheses) to validate research and which methods are suitable, appropriate and needed for demonstrating evidence;

  2. retrieving research results just through syntactical terms of the paper title (or even abstract) is often very hard, and, as a consequence, comprehensively relating research results to each other in terms of support, strengthening or contradiction is rarely done.

Therefore, in PROPSER we want to discuss in particular:

  • properties for describing software engineering research, such as relevance, evidence, applicability, appropriateness, etc;
  • general patterns for suitable empirical validations of research results and hypotheses;
  • suitable classifications of software engineering research, beyond established subject categories;
  • experiences, rules or guidelines, on how to decide on suitable validation questions and subsequently suitable empirical methods;
  • managing and organising research knowledge at a finer grain than paper citations;
  • methods for evaluating the impact of research on software engineering practice.

The workshop aims to discuss these subjects as such, it is not meant to be a platform for original research results in a specific software engineering topic. Therefore, we intentionally do not require original new ideas to be submitted. Of course, for inclusion in the proceedings a paper should be original text.


Workshop Program, 23.06.2021

09:00-10:30 General EASE Keynote Mark Harman
10:30-10:40 Coffee Break  
10:40-10:50 Welcome Introduction
Antonia Bertolino, ISTI-CNR, Italy
Anne Koziolek, Karlsruhe Institute of Technology (KIT), Germany
Ralf Reussner, Karlsruhe Institute of Technology (KIT) / FZI Research Centre
  Paper Session 1 (Session Chair: Willi Hasselbring)  
10:50-11:05 On implicit assumptions underlying software engineering research Lutz Prechelt, Freie Universität Berlin, Germany
11:05-11:20 A classification for managing software engineering knowledge Angelika Kaplan, Karlsruhe Institute of Technology, Germany
Maximilian Walter, Karlsruhe Institute of Technology, Germany
Robert Heinrich, Karlsruhe Institute of Technology, Germany
11:20-11:35 Towards an automated classification approach for software engineering research Angelika Kaplan, Karlsruhe Institute of Technology, Germany
Jan Keim, Karlsruhe Institute of Technology, Germany
11:35-11:50 About the assessment of grey literature in software engineering Guglielmo De Angelis, IASI_CNR, Italy
Francesca Lonetti, ISTI-CNR, Italy
11:50-13:00 Lunch Break  
  Paper Session 2 (Session Chair: Lutz Prechelt)  
13:00-13:15 If you want better empirical research, value your theory: On the importance of strong theories for progress in empirical software engineering research Klaus Schmid, University of Hildesheim, Germany
13:15-13:30 Benchmarking as empirical standard in software engineering research Wilhelm Hasselbring, Kiel University, Germany
13:30-13:45 Towards a more structured peer review process with empirical standards Arham Arshad, Dalhousie University, Canada
Taher Ahemd Ghaleb, Queens University, Canada
Paul Ralph, Dalhousie University, Canada
13:45-14:00 Organization of break-out groups (Session Chair: Antonia Bertolino)  
14:00-14:10 Coffee Break  
14:10-15:45 Break-out group discussion  
15:45-16:15 Presentation of break-out group results (Session Chair: Ralf Reussner)  
16:15-17:00 Discussion, further steps and closing (Session Chair: A. Bertolino, A. Koziolek, R. Reussner et al.)  


Concretely, we invite three types of submissions:

  • taxonomy papers (up to 8 pages): papers presenting or surveying classifications of software engineering subjects, empirical methods, areas of software applications, etc.; including hypotheses about relationships between such categories (e.g., "contributions of type X should be validated with method Y.")

  • method-not-obvious papers (up to 6 pages): papers describing ongoing research that has unsolved questions regarding how to demonstrate the validity and relevance of the work.

  • proposal papers (up to 6 pages): papers that directly address one or a few of the above-mentioned topics.

None of these submission types strictly requires a conventional sequence of sections; please find a suitable section format for your respective case, but make heavy use of headings to help readers with orientation.

Papers will be reviewed with respect of their strength of argumentation and their potential to spark interesting discussions during the workshop regarding the above-mentioned topics.

Submitted papers must be written in English, contain original, unpublished work, and conform to the ACM Proceedings Format ( Note that the new ACM submission template is one-column, whereas the camera-ready version will still be expected in two-column. Word users shall refer to the Table at the bottom of to estimate the paper's number of pages in the camera-ready version. Latex users can use \documentclass[sigconf,authordraft]{acmart} to write the paper and to control the number of pages in the camera-ready version but change it to \documentclass[manuscript]{acmart} for submitting the paper for review.

Submissions are done via the Easychair site of EASE.


The deadlines and important dates for the submissions are as follows:

  • Paper submission: 2 April 2021 (extended from 26 March 2021)
  • Author notification: 19 April 2021
  • Camera-ready: 3 May 2021 (extended from 30 April 2021)
  • Workshop: 23 June 2021

All accepted papers will be published at ACM EASE Companion Proceedings.

At least one author of each accepted paper must register by the camera-ready deadline.



Antonia Bertolino, ISTI-CNR, Italy

Anne Koziolek, Karlsruhe Institute of Technology (KIT), Germany

Ralf Reussner, Karlsruhe Institute of Technology (KIT) / FZI – IT Research Centre, Germany (main contact)

Programme Committee

Robert Feldt, Chalmers University, Sweden

Paul Ralph, Dalhousie University, Canada

Carlo Ghezzi, Politecnico di Milano, Italia

Gustavo Pinto, Federal University of Pará, Brasil

Barbara Kitchenham, Keele University, UK

Lutz Prechelt, FU Berlin, Germany

Walter Tichy, Karlsruhe Institute of Technology, Germany

Sira Vegas, Universidad Politécnica de Madrid, Spain