Univ. of Tokyo
University of Tokyo

SIMULBED/DESIGNET

 Associate Team Home Page

between 

INRIA - NICT and Keio University 2012-2013 

And

INRIA - NICT and University of Tokyo 2014 

Summary


    The SIMULBED project is an associated team between researchers at INRIA, Keio University and NICT; it is financed by INRIA and the Japanese Society for the Promotion of Science. The SIMULBED associated team started formally for Inria in january 2012 for a duration of three years ending in december 2014. However, from the japanese side, the SIMULBED project started in April 2012 for a duration of two years. The initial SIMULBED project was established with the team of Professor Osamu Nakamura at Keio Univeristy. Professor Nakamura was the PI from the Japanese side for 2012 and 2013. Hitoshi Asaeda and Hajime Tazaki has actively contributed to this collaboration. After the end of the two years AYAME project, we established a new collaboration agreement called DESIGNET (corresponding to the third year of the SIMULBED associated team from the french side), with University of Tokyo (Prof. Yuji Sekiya)  and NICT (Hitoshi Asaeda is the partner PI for 2014).

    SIMULBED focuses on the design of an efficient experimentation platform to evaluate new network protocols and architectures. This project builds on the following state-of-the-art tools and platforms: the open source ns-3 network simulator,  PlanetLab testbed and NEPI. Indeed, the use of both simulators and testbeds to conduct experiments grants a better insight on the behavior of the evaluated network protocols and applications.
 
    The overall objective of the project is to make available to networking research community, the SIMULBED platform whose goal is to (1) allow to easily conduct  mixed simulation-experimentation evaluation of networking protocols and (2) to scale up the size of the PlanetLab experimental testbed, while maintaining a high degree of realism and increasing controllability and reproducibility. We leverage of the experience of both partners in ns-3 and PlanetLab to design and develop this new plarform. We also use the NEPI unified programming environment recently developed in the Planète/Diana research team to help in simplifying the configuration, deployment and run of network scenarios on the platform.

Context and Problem Statement

    Simulators and experimental testbeds are two different and complementary approaches for the evaluation of network protocols and they provide a varying degree of repeatability, scalability, instrumentation and realism.

    PlanetLab is one of the most known experimental testbed in the networking community. It is a globally distributed testbed designed for network experimentation and composed of nodes connected to the Internet across the world. PlanetLab uses container-based virtualization to allow multiple experiments running independently on the same node while sharing its ressources. However, the use of network virtualization has a cost: some tasks are made more complex than in a classical (non-virtualized) environment. In particular, slivers (which are isolation units for experiments running in PlanetLab) do not have full root privileges. Therefore operating system calls to set up routes, tunnels, or IP filters, among others, are forbidden.  One goal of this project is to provide a platform that makes easier for researchers, the setup, running and evaluation of  new protocols and architectures.

    Ns-3 is the first network simulator that converts transparently and efficiently network packets to and from simulation objects, hence enabling transparent support for real-time simulation in every ns-3 model. This provides a high degree of realism. Therefore it will be easy to use it as a real-time emulator in a larger testbed. It will also allows to execute directly within the simulator unmodified network protocol implementations and applications, using the Direct Code Execution (DCE) feature. Using DCE, ns-3 trades off scalability for realism by embedding within the simulator existing protocol implementations that were never designed to run in a simulator. It makes possible to considerably increase the realism of a simulation while retaining its repeatability and ease of debugging. It also makes it trivial to repeat and compare the result of the same experiment in a testbed by using the same protocol implementation in both cases. In DCE, the Virtualizing Dynamic Loader (VDL) Executable and Linkable Format (ELF) loader is used to both virtualize access to every global and static variable and to redirect function calls to system facilities from their normal implementation to a simulation implementation. However, the memory and CPU cost of virtualizing a single external protocol implementation for execution within the simulator should be maintained as low as possible to be able to run simulations which process many packets per second and include a large numbers of instances of this protocol implementation. Moreover, the current DCE library is still incomplete from the perspective of replacing all the functionality provided by normal system libraries. Therefore, in this project, we plan to enhance DCE and increase its functionality coverage by adding the missing wrapper functions.   

    Another objective of this project is to allow to run hybrid networking scenarii involving both simulators and real nodes. While these scenarii are simple to state, the complexity of deploying, and configuring correctly these experiments put them out of reach of most experimenters. Indeed, the complexity of setting them up is very high: users must configure compatibly both their testbed and their simulation, which makes it impossible to use the native setup capabilities of each experimentation tool and thus, requires a lot of manual work. Then, another contribution of this project is to implement mechanisms to simplifying the description and deployment of such mixed network experiments scenarii. Specifically, we plan to extend the NEPI unified programming environment developed at  Planète to fit the specific requirements of the SIMULBED platform. We need also to propose some algorithms that help in distributing a networking simulation scenario in multiple real nodes. This problem is challenging because it is necessary to minimize the traffic exchanged between real nodes and to ensure to keep the CPU load of real nodes as low as possible.


Work Progress 

Note that in Japan the project officially started on June 2012 with the kick-off meeting at INRIA.

Period January 2012 - September 2012:

As part of the efforts to simplify set-up and execution of mixed experiments, NEPI was extended to automate deployment and customization of routing overlays on PlanetLab. Thanks to this ability and to the additional support to deploy and control ns-3 simulations on PlanetLab nodes, it is now possible through NEPI to conduct experiments where PlanetLab nodes and ns-3 simulations are part of a same routing overlay. This extension to NEPI not only allows a better control over the routing between PlanetLab nodes, thanks to the use of customizable tunnels, but it also enables researchers to easily scale PlanetLab experiments using ns-3 simulations. Additionally, support for easy experimentation with CCNx on PlanetLab was added to NEPI. CCNx is an interesting use case for NEPI since it is a promising technology for future Internet architecture with a rapidly growing industrial and academic community. This work was presented to the community on the last CCNx conference.

Work is underway to support a fourth experimentation environment, OMF, in NEPI, to enable wireless network experimentation. NEPI will interact with the OMF resource control layer by implementing the OMF XMPP protocol. In the future we foresee to support the ability to deploy ns-3 simulations in OMF nodes and to integrate the simulated network in a same overlay with OMF nodes. Finally, extensions to support automatic IP address assignment and routing table population during experiment design time are being added to NEPI with the objective of easing the high time expense involved in manual configuration of large experiment topologies.

DNEmu was also introduced to provide smooth experimentation control over distributed PlanetLab nodes with ns-3. As NEPI does, DNEmu is also able to control and synchronize the experimental scenario among multiple nodes, but DNEmu more focuses on the ns-3 specific feature and tries to reduce the dependency of external software and toolset. This work was presented at the WNS3 workshop and on  TridentCom'12 conference.

Concerning Direct Code Execution (DCE) of ns-3, we have added the support the recent 3.4.5 Linux kernel, and different versions of Fedora and Ubuntu Linux distributions. Other works include the creation of an helper to automatically create the static routes for a static network like the helper existing in ns-3, and the support of  Message Passing Interface (MPI)  for two loaders of DCE (CoojaLoader and CopyLoader). Further details on DCE functionalities added and planned to be added can be found in bugzilla.

Period October 2012 - October 2013:

Our work on the Simulbed experimentation platform focused on two main directions: Direct Code Execution (DCE) framework and Network Experiment Programming Interface (NEPI).

We released version 1.1 of DCE on September 1st, 2013. Several new features have been added. First, we added aspect-based tracing, which allows us to use tracing facility in DCE when simulating unmodified code. This feature is important for debugging and protocol analysis purposes as it enables to investigate which function is called or how many messages of a particular protocol are exchanged for example. We introduced a development suite for network protocol development that includes code coverage measurement using the gcov tool, memory analysis under distributed protocol operation with valgrind, and distributed code debug with gdb. In addition to above new feature supports, we wrote a public web page for DCE (see URL http://www.nsnam.org/overview/projects/direct-code-execution/) along with manuals (see URL http://www.nsnam.org/docs/dce/manual/html/index.html), and proposed a stable software release cycle so that users can choose to use either the stable DCE version or the latest experimental version. We also designed the bake building and integration tool to make easier the DCE installation procedure, see URL http://www.nsnam.org/docs/bake/tutorial/html/index.html.

Two papers on DCE have been accepted in 2013. The first one introduces DCE Cradle, a feature that enables to use ns-3 native applications over DCE Linux kernel stack. DCE Cradle utilizes standard Linux kernel network stacks and provides transparent socket interfaces to the ns-3 applications. This allows us to study Linux kernel protocols with various useful applications of ns-3, such as trac generators or routing protocols. This paper presents the architecture, a use case involving the DCCP protocol and some benchmarks. The second paper describes the DCE framework, and has just been accepted at the prestigious ACM CoNEXT conference. It describes in detail the architecture of DCE, reports on packet processing benchmark and showcases key features of the framework with various use cases. This paper also demonstrates that DCE can be used to write runnable papers, i.e., papers where researchers can easily reproduce experiments shown in papers.

A tutorial on how to experiment CCN using the CCNx implementation from PARC with DCE has been presented at the ICN workshop in January 2013. A demonstation on how to use DCE with different use cases has been done in the ACM MSWiM conference in Barcelona on November 4, 2013.

    In September 2013, we released version 3.0 of NEPI, the framework that provides an experiment description language to describe experiments during the design stage, and the mechanisms to automate the deployment and control of experiments and collect results, during the execution stage. The previous version of the experiment description language lacked sufficient expensiveness to describe experimental work flows (i.e. to specify a sequence of events that need to take place during the experiment in a flexible way). Also, to support a larger set of experimentation scenarios with NEPI, we needed a more easily extensible architecture to make it easier the support of new experimentation environments and new types of resource (e.g. virtual machines, switches, tunnels, etc).
    Version 3.0 of NEPI includes a re-design of its architecture to allow expressing work flows as part of the experiment description, and to enable easier extensibility of NEPI to support new experimentation environments and resources. This new architecture adds to NEPI the capability to automate execution of experiments in any SSH-enabled Linux testbed (PlanetLab included), and in OMF-based wireless     testbeds. A full release of NEPI, including enhanced user documentation, a new WEB site, a wiki page and a bugzilla bug reporting page, is planned for December 2013.
    Finally we have used NEPI to evaluate the performance of CCN overlays on top of the Internet. We constructed different overlay topologies on PlanetLab, in which we varied the topology configuration, the trafic patterns and different CCN parameters with the objective to find correlations between these variables and the performance obtained. A paper is in preparation and the capabilities of using NEPI to easily run CCNx experiments in realistic wired and wireless environments has been demonstrated in the CCNx workshop at PARC in September 2013.


Period October 2013 - October 2014:

This year, our work includes enhancements of the DCE and NEPI frameworks,  developement of the CUTEi testbed involving NEPI, and  the design on Contrace, a new measurement tool for information-centric networking architectures as detailed as follows:

We have implemented  missing wrapping code between ns-3 and DCE in order to be able to launch with NEPI hybrid experiments including real application codes running over ns-3 through DCE. Actually, the most common language to write DCE simulation scripts is C++. But C++ implementations may be difficult for new users, and time consuming even for experienced users. We believe that a scripting language like Python is helpful to simplify experimentations. Python bindings are wrapping code required to translate Python calls to corresponding C++ calls. Furthermore, these bindings makes possible running DCE from NEPI.

We added  the support of OpenFlow (OF) protocol through the use of Open vSwitch  in DCE. Using DCE helps in reducing the development cost to port the latest flavor of OpenFlow in ns-3 and  allows to simulate with ns-3 with the latest features of OF with minimal cost. Such a work makes it possible to simulate and evaluate in a realistic way wireless Software Defined Networking solutions. Indeed, the most used SDN emulator tool, mininet, can only emulate point-to-point physical links using virtual Ethernet pairs (e.g., MAC layer is ignored), and it can not provide mobility models for wireless nodes. To make DCE able to run Software Defined Networks, we started to support OpenFlow NOX controller and Open vSwitch. The actual NOX binary is executed on a simulated ns-3 node. OpenFlow wireless routers are simulated using the Open vSwitch distribution with data-path kernel module support as it is widely used. DCE provides a mechanism to incorporate such a kernel module based application execution.  This has been demonstrated at the 17th ACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems, held in Montreal Canada.

In 2014 we released two new versions of DCE: 1.3 (released on July 9th), and 1.4 (released on September 17th). Version 1.3 was a major improvement, introducing the Python bindings and the FreeBSD network stack, along support for new protocols (SCTP and IPv6 sockets for DCE Cradle: TCP6, UDP6, RAW6, DCCP6, SCTP6). It also included a number of minor improvements and bug fixes.  Version 1.4 has been an incremental version, including mainly bug fixes and new API support. Most of the new APIs were required to improve the support to NEPI. The two versions also include improvements on the documentation and user manuals.

The capabilities of using NEPI to easily run CCNx experiments in realistic wired and wireless environments has been demonstrated in the CCNx workshop at PARC in September 2013. Moreover, we have demonstrated NEPI to evaluate the performance of Information-Centric Networking (ICN) solutions. Such a development and evaluation environment  that enables both controllable and realistic experimentation is needed to thoroughly understand how ICN solutions would behave in real life deployment. We demonstrated the benefits of our solution by showing how complete experimental studies can be carried out with minimum manual intervention and experiment set-up overhead, in both emulation and live environments.

We have developed CUTEi, which employs the Linux container (LXC) lightweight virtualization mechanism for node design and enables testbed users to evaluate ICN applications and protocols. It enables testbed users to share the major or common ICN software and applications on top of CUTEi nodes. In addition to OS commands and libraries, CCNx, NDNx, and other software and libraries are currently pre-installed in a Logical Volume (LV) of each CUTEi node as basic ICN components. CUTEi adopts NEPI to  automate experiment description, distributing codes, and executing network experiments on network evaluation platforms: The user first uses a Python-based language to describe the experiment as a graph of interconnected components, including CCN nodes, FIB configuration, and CCN applications. Then, s/he instructs NEPI how to deploy the experiment. In the CUTEi testbed, NEPI uses an SSH-based backend to automate to run CCN applications (e.g., ccncat) on user containers. During and after execution of the experiment, the user can ask NEPI to download CCN logs and other collected results from his/her user containers. CUTEi testbed is currently  deployed in 9 sites, INRIA (FR), NICT, Keio U., Waseda U. (JP), Tsinghua U. (CN), SNU (KR), U. Florida (US), etc.

As information flow in ICN/CCN is based on named data requesting, in-network caching, and forwarding—which are unique and can be independent of IP routing. Hence common IP-based network tools such as ping and traceroute can neither trace a forwarding path in CCN nor feasibly evaluate CCN performance. We have designed  “contrace," a network tool for CCNs that can be used to investigate 1) Round-Trip Time (RTT) between content forwarder and consumer, 2) the states of in-network cache per name prefix, and 3) the forwarding path information per name prefix. The results of experiments conducted using contrace on a CCN topology formed on DCE/NS3 not only confirm the behavior but also that contrace can trace paths via name prefix, measure RTT, and show the caching information in CCN. We submitted a paper describing Contrace with use cases to a journal.
The members of the associated team has worked closely together to ensure high visibility of the project resutls. Several publications published in large audience journal and international conferences and workshops. The associated team members also presented several tutorials, demonstrations and posters at specialized conferences, workshops and summer schools. We also have disseminated our work to increase the DCE and NEPI user community by promoting the use of these two frameworks in classrooms (in particular, Ubinet Master, UPMC and Telecom ParisTech students) and in research institutions through the Fed4Fire European FP7 project. Examples include how to write runanble papers, how to reproduce other researchers results using DCE and evaluate further their protocols in extended scenarios using both DCE and NEPI.

Meetings

Publications


Journals 

Conferences and Workshops

Posters

Demos

Various

Software


People

Contact