Repository of resources - Research methods in Computing

Introduction

This repository includes links to educational resources related to the Research Methods in Computer Science -course at the School of Computing, UEF. Most of the resources are open educational resources. The collection of the resources was launched in Spring 2015 during the Research Methods in Computer Science  -course by Jarkko Suhonen. The content of the repository was expanded and updated in 2016, 2017 and 2018.

If you have any questions or want to suggest additions to the repository, please contact Jarkko Suhonen, jarkko.suhonen(at)uef.fi.

General research resources

Articles

• About Computing Science Research Methodology - University of Alberta
• The Level of Scientific Methods Use in Computing Research Paradigms - by Elly Amani Gamukama and Oliver Popov
• Research in Computer Science: An Empirical Study - by V. Ramesh et al.
• Scientific Research Methods and Computer Science - by Ricardo Andre Freitas
• A Letter to Research Students - Practical tips for conducting research by Duane A. Bailey, Williams College
• How to Do a Structured Literature Review in Computer Science - by Anders-Koford Petersen
• Research Methods - Discussion about quantitative, qualitative and mixed methods research by Carrie Williams
• Research Dillemas: Paradigm, Methods and Methodology - by N. Mackenzie and S. Knipe
• The Information Systems Research Cycle - by A.R. Hevner and S.T. March

Websites and resources

• Tutorial: Research Methods in Computer Science - by Serge Demeyer
• Choosing Qualitative or Quantitative Research Methodologies - Audience Dialogue
• Guides for Research Writing - Write.com
• Study Guide for Writing a Research Proposal - Study Guides and Strategies
• Research Methods - Quantitative, Qualitative Methods and Experimental Computer Science, PP-slides, Ryerson University
• When to Use Which User-Experience Research Methods - by Nielsen Norman Group
• Questionnaires as Data Collection Method - The University of Sheffield
• A Brief Guide to Questionnaire Development - Robert B. Frary
• arXiv.org -  archive and distribution server for research articles in various fields, including Computer Science. Hosted by Cornell University.
• Pros and Cons of Inforamtion Gathering Technigues - by Jamie MacDonald

Courses on Research Methods in Computing

• CS2130: Empirical Research Methods for Computer Scientist (2014) - Steve Easterbrook and Barbara Barbosa Neves, University of Toronto.
• Empirical Research Methods - Open Learning Initiative at Carnegie Mellon University
• CDT403 Research Methods in Natural Sciences and Engineering - Mälardalen University

Videos

• Video Course on Research Methodologies in Computer Science - Prasant Mohapatra, University of California
• A Brief Comparison of Qualitative and Quantitative Research Methods - Shawn M. Clankie
• Wide-Field Ethnography: Studying Software Engineering (and Other Things) in 2025 and Beyond - David Socha
• CS6000 Computer Science Research Methods - series of videos done by Prof. Terry Boult related Introduction to Computer Science Research Methods -course at the University of Colorado Colodaro Springs

Quantitative research

Articles

• Post-positivistic approaches to research - by Anne B. Ryan
• Quantitative Research Designs - by Will G. Hopkins

Websites and resources

• The Positivistic Paradigm - Robert Wood Johnson Foundation
• Electronic Statistics Textbook - by StatSoft
• Beginners Guide to the Scientific Method - Explorable.com

Videos

• Developing a Quantitative Research Plan: Choosing a Research Design by Amanda J. Rockinson-Szapwik
• Analysing Questionnaires - John Schultz, University of Southampton

Survey research

Articles

• Survey Research in Software Engineering: Problems and Strategies - by Ahmad Nauman Ghazi et al.

Websites and resources

• Compact Summary of Survey Research - PP-slides by Steve Easterbrooks
• Fundamentals of Survey Research Methodology - Priscilla A. Glasgow
• Guide to Survey Research - Colorado State University
• Survey Design Essentials - video by Jason Husser

Experimental research

Articles and books

• Experiments in Computing: A Survey - by M. Tedre and N. Moisseinen
• What is Experimental Computer Science - by Peter J. Denning
• Experimental Computer Science: The Need for Cultural Change - by Dror G. Feitelson
• Reliable Research: Towards Experimental Standards for Computer Science - by Justin Zobel
• Single-Case Design Technical Documantation - by What Works Clearinghouse
• Experimental and Quasi-Experimental Designs for Research - by Donald T. Campbell and Julian C. Stanley
• Empiricism in Computer Science - by Eike Send and Christopher Oezbek
• Experiment Databases: A Novel Methodology for Experimental Research - by Hendrik Blockeel

Websites and resources

General

• Resources on Experimental Computer Science - by Prof. Dror Feitelson
• Experimental Methods in Human-Computer Interaction - a chapter written by Paul Cairns in online book Encyclopedia of Human-Computer Interaction
• Experimental Design Definition and Examples - StatisticHowTo.Com
• Overview of Experimental Design - Centre for Innovation in Research and Teaching
• Benefits and Limitations of Experimental Research - Centre for Innovation in Research and Teaching
• Experimental Psychology - Psychology World website by Richard Hall
  
  

Specific experimental research setups

• 2X2 Between Subjects Factorial Design - Psychology World website by Richard Hall
• Two-Group Experimental Designs - The Research Methods Knowledge Base
• ABAB Experimental Design - by Christopher L. Heffner
• Solomon Four Group Design - Explorable.com
• Pretest-Posttest Design - Explorable.com

Videos

• Experimental Design - video by Ana Gebejes
• Statistics Introduction to Experimental Research - by Rob Tarrou
• Experimental Research Video Lecture by Dr. Watson (Crystal McDonald, Vanessa Palmer and Hannah Wang)
• Quasi Experimental Designs & True, Quasi, Pre and Non Experimental Designs
• Between and Within Subjects Experimental Design - by Dr. Keon West
• What is A-B-A-B Design? - Terminology Learner
• AB (Baseline) Design - Lecture recording by Prof. Heather Vonderfect
• Correlational Research Design - by Dr. Thelisa Nutt

Qualitative research

Articles

• Using Qualitative Research Methods in Higher Education - by W.C. Savenye & R.S. Robinson
• Guidelines for Conducing and Reporting Case Study Research in Software Engineering - by Per Runeson & Martin Höst
• Five Misunderstandings About Case-Study Research - by Bent Flyvjberg
• Qualitative Case Study Methodology: Study Design and Implementation for Novice Researchers - by Pamela Baxter & Susan Jack
• Case Study as a Research Method - by Zaidah Zainal
• Three Approaches to Qualtitative Content Analysis - by Hsiu-Fang Hsieh, Sarah E. Shannon
• Strategies for Ensuring Trustworthiness in Qualitative Research Project - by Andrew K. Shenton

• Scholarship and Practice: The Contribution of Ethnographic Research Methods to Bridging The Gap  - by Lynda J. Harvey & Michael D. Myers

• Problems of Reliability and Validity in Ethnographic Research - by Margaret LeCompte & Judith Goetz
• Ethnography - A chapter written by Dave Randall and Mark Rouncefield about Ethnography in HCI, the chapter appears in online book Encyclopedia of Human-Computer Interaction
• Wild Interdisciplinarity: Ethnography and Computer Science - by M. Goulden et al.
• Phenomenology - A chapter written by Shaun Gallagher about Phenomenology in HCI, the chapter appears in online book Encyclopedia of Human-Computer Interaction
• Semi-Structured Qualitative Studies - A chapter written by Ann Blandford about applying qualitative studies in HCI, the chapter appears in online book Encyclopedia of Human-Computer Interaction

Presentation

• Qualitative Research in Computer Science - PP-presentation by Edwin Blake
• Qualitative Data Analysis and Codebook Creation - Prezi presentation by Rebecca King

Books, websites and resources

• Qualitative Methods Workbook - by Dr. C. George Boeree
• Qualitative Research in Information Systems - by Michael D. Myers.
• Interview Guide - Robert Wood Johnson Foundation
• Forum: Qualitative Social Research - open access online journal for qualitative research in social research
• Methodology Brief - Introduction to Focus Groups - by Barry Nagel and Nichelle Williams

Videos

• Fundamentals of Qualitative Research - Leslie Curry, Yale University.
• Complete Methods of Qualitative Research - by Dr. Jason J. Campbell, Nova Southeastern University
• Phenomenology - by Chris Flipp
• Digital Ethnography - Interview of Michael Wesch
• Ethnography - by Chris Flipp
• Case Study - by Chris Flipp
• Analysing Interviews - John Schultz, University of Southampton

Design science research

Articles

• Research Through Design - A chapter written by Pieter Stappers and Elisa Giaccardi about design as research, the chapter appears in online book Encyclopedia of Human-Computer Interaction
• A Design Science Research Methodology for Information Systems Research - by Ken Peffers et al.
• A Three Cycle View of Design Science Research - by Alan Hevner
• Design Science Research in Information Systems - by Vijay Vaishnavi et al.
• Action Design Research - by Maung Sein et al.
• Design Science Research: Paradigm or Approach - by Sven Weber
• Design Science Research in Europe - by Robert Winter.
• Action Research and Design Science Research - Seemingly Similar but Decisively Dissimilar - by J. Iivari and J.R. Venable
• Action Research as an Approach in Design Science - by Pertti Järvinen
• Action Research vs. Design Research: Using Practice Research as a Lens for Comparison and Integration - by Göran Goldkuhl
• Towards a Maturity Model for E-Collaboration – A Design Science Research Approach - by Stefanie Hain, Andrea Back

Videos

• Is Design Science Research the Future of Innovation? - by Prof. Alan Hevner
• Design Science and Action Research - UTS Software Engineering
• Robust Processes of Design Science Research - by Prof. Alan Hevner

Presentation and other resources

• Design Science Research in Information Systems - PP-slides by Prof. Alan Hevner
• Collection of Scientific Literature Related to DSR - list of articles especially relevant for application of design science research in educational technology
• Research Methods in Informatics and its Applications: Design-Oriented Research - PP-slides by Baumgart et al.

Action research

Articles

• Action Research - by Avison et al.
• Investigating Information Systems with Action Research - by R.L. Baskerville
• Why Action Research? - by M. Brydon-Miller et al.
• On Various Characteristics of Action Research - by Pertti Järvinen
• An Overview of the Methodological Approach of Action Research - by Rory O'Brian
• The Three Threats of Action Research: a Discussion of Methodological Antidotes in the Context of an Information Systems Study - by Ned Kock
• Themes in Education: Action Research - by Eileen Ferrance
• Using Action research for Complex Research Initiatives - by M. Greef and L. Coetzee
• An Assessment of Scientific Metrics of Action Research - by G.I. Susman and R.D. Evered.
• Towards a Framework for Action Research in Information Systems Studies - by F. Lau
• Mixed Methods and Action Research: Similar or Different - by D. Wisniewska
• Introduction to Action Research (in the Context of Management Research) - by Susan Rose, Nigel Spinks & Ana Isabel Canholo
• Action Research is Similar to Design Science - by Pertti Järvinen

Books, guides, presentations and websites

• Embrace Action Research, Creative Educator
• Action Research - A chapter written by Ned Kock about Action Research in HCI. The chapter appears in online book Encyclopedia of Human-Computer Interaction
• Action Research: A guide to associate lectures - Guide on how to use action research at Open University, UK
• You Want To Do an Action Research Thesis - Guide for conducting and reporting action research written by Bob Dick
• Action Research Tutorial - by Margaret Riel   
• Introduction to Action Research -  PP-slides by Benjamin Fritzsche and Ingo Just
  
• Action Research Intro  - by PP-slide by Hanna Repaz
• Action Research: Practice Guide - by Lydia Arnold and Lin Norton
  

Videos

• What is Action Research - by John Spencer
• Action Research in the Classroom: Part 1 and Part 2 - by Jana Duganzic, Taryn Durrant, Leya Finau, Nicola Firth and Melissa Frank
• Action Research in Education - by Carol Davenport
• Participatory Action Research - by Ade Oudshoorn
• History of Action Research - by Marian Lowry
• Participatory Action Research - by Jason Cambel

Acknowledgement

The following students have helped us to create and expand the knowledge base. Many thanks!