Collaborative Research: Acquiring Accurate Dynamic Field Data Using Lightweight Instrumentation

Research Project

The goal was to design, implement and evaluate light weight instrumentation for software testing and profiling. The underlying concept was that different instances (users' copies) of the software are instrumented to collect information differently but lightly so that performance is not impaired.

Seven critical research challenges formed the core of the project:

  1. Lightweight instrumentation--Develop instrumentation that is virtually transparent to individual users.
  2. Compositional analysis techniques--Develop distributed analysis techniques that decompose traditional analyses into smaller steps, distribute the steps among multiple users, and then fuse each user's results into an accurate solution to the original problem.
  3. Scalability--Develop storage and analysis techniques to deal with the high data volumes we expect to encounter.
  4. Anomaly Detection--Define data-driven techniques to automatically identify anomalous behaviors of deployed software.
  5. Privacy and Security--Incorporate privacy and security safeguards into our data collection and analysis approaches.
  6. Dynamic updating mechanisms--Develop techniques to make runtime changes to the location and function of instrumentation, and to parts of the software itself.
  7. Validate approach on industrial software.
Project Goal: 

The project goal is to give developers unprecedented insight into the actual runtime behavior of their software, allowing developers (and ultimately the software itself) to change, optimize, and adapt the software based on highly accurate field data. Lightweight, collaborative dynamic analyses conducted around-the-world and around-the-clock form the new platform: (1) lightly instrument fielded software (i.e., each program copy performs a small part of the analysis) (2) collect the partial data from many instances of the software, fusing it to conduct the complete analysis, (3) change the running program instances based on the findings and (4) repeat the process.

Research Team: 

Adam Porter, Maryland, Alan Karr, NISS, David Notkin, Washington, Douglas Schmidt, UC-Irvine, Ashish Sanil, NISS

Individual Team Members: 
Alan F. KarrAshish SanilAdam Porter

Funding Sponsors: