Title: Achievements, open problems and challenges for search based software testing

Abstract

Search-based approaches to software testing formulate testing as an optimisation problem, which can be attacked using computational search techniques from the field of Search Based Software Engineering (SBSE). Test objectives find natural counterparts as the fitness functions used by SBSE to guide automated search, thereby facilitating SBSE formulations of many and diverse testing problems. This keynote will review achievements in Search Based Software Testing and set out challenges and open problems for the future development of this promising approach to automated test data generation and test suite optimisation.

Bio

Mark Harman is professor of Software Engineering in the Department of Computer Science at University College London, where he directs the CREST centre and is Head of Software Systems Engineering. He is widely known for work on source code analysis and testing and co-founded the field of Search Based Software Engineering (SBSE), the topic of this keynote. SBSE research has rapidly grown over the past five years and now includes over 1100 authors, from nearly 300 institutions spread over more than 40 countries. A recent tutorial paper on SBSE can be found here: http://www.cs.ucl.ac.uk/staff/mharman/laser.pdf

---

Title: Perspectives on White-Box Testing: Coverage, Concurrency, and Concolic Execution

Abstract

The last years have seen a fruitful exchange of ideas between automated software verification and white-box software testing; the industrial impact of concolic testing for sequential software is the most notable result of this interdisciplinary effort. While concolic testing is very successful at finding bugs, and even achieves verification in the limit, it is often hard to quantify the progress it achieves towards verification. In this talk, we survey two recent projects which aim to remedy this situation: In the FQL project, we devise a test specification language which facilitates precise specification of coverage criteria, and a separation of concerns between test specification and test case generation. In con2colic testing, we develop a concolic testing methodology for concurrent programs where progress is measured in terms of the data flow between program threads.

Bio

Helmut Veith is a professor at the Faculty of Informatics of Vienna University of Technology (TU Vienna), and an adjunct professor at Carnegie Mellon University. He has a diploma in Computational Logic and a PhD sub auspiciis praesidentis in Computer Science, both from Vienna University of Technology. Prior to his appointment to Vienna, he was holding professor positions at TU Darmstadt and TU Munich.

In his research, Helmut Veith applies formal and logical methods to problems in software technology and engineering. His current work is focusing on model checking, software verification and testing, embedded software and computer security.

---

Title: Testing in a large service based architecture, from unit testing to acceptance testing

Abstract

Twitter was first built as a monolithic codebase on one web framework, rubyon rails. Though this didn’t scale to our traffic or our team size, it was great for code analysis, integration testing, and end-to-end testing. When we became a service based architecture (SBA) with numerous engineering teams, a much better fit for our scale, the existing tools, assumptions and philosophies used for testing and code analysis no longer worked either technically or culturally. We had (and have) to overcome challenges at each stage of testing. This talk will address some of the challenges we have and haven’t solved at the various stages of testing. End-to-end acceptance testing in a polyglot SBA with per-service/team deploy schedules and policies, exhaustive compatibility testing, smoke testing complex systems without always seeing fire, ensuring unit test coverage, communicating the findings of static analysis tools quickly and usefully, etc.

Bio

Nick Green has been a software engineer (and security engineer) at Twitter since 2011. As a security engineer he worked on static analysis, built tools to make static analysis actually useful in a large diverse organization and many other tools to provide actionable and useful feedback during the build-test-release process. More recently he built the front-end service for login/authentication and the tests and test environments to support these systems.