G Model
ARTICLE IN PRESS
FUSION-8793; No. of Pages 4
Fusion Engineering and Design xxx (2016) xxx–xxx
Contents lists available at ScienceDirect
Fusion Engineering and Design journal homepage: www.elsevier.com/locate/fusengdes
J-TEXT WebScope: An efficient data access and visualization system for long pulse fusion experiment Wei Zheng a,b,∗ , Kuanhong Wan a,b , Zhi Chen a,b , Feiran Hu a,b , Qiang Liu a,b a b
State Key Laboratory of Advanced Electromagnetic Engineering and Technology in Huazhong University of Science and Technology, Wuhan 430074, China School of Electrical and Electronic Engineering in Huazhong University of Science and Technology, Wuhan 430074, China
h i g h l i g h t s • • • • • •
No matter how large the data is, the response time is always less than 500 milliseconds. It is intelligent and just gives you the data you want. It can be accessed directly over the Internet without installing special client software if you already have a browser. Adopt scale and segment technology to organize data. To support a new database for the WebScope is quite easy. With the configuration stored in user’s profile, you have your own portable WebScope.
a r t i c l e
i n f o
Article history: Received 25 May 2015 Received in revised form 27 May 2016 Accepted 2 June 2016 Available online xxx Keywords: Data access and visualization Long pulse WebScope RESTful MDSplus J-TEXT
a b s t r a c t Fusion research is an international collaboration work. To enable researchers across the world to visualize and analyze the experiment data, a web based data access and visualization tool is quite important [1]. Now, a new WebScope based on RIA (Rich Internet Application) is designed and implemented to meet these requirements. On the browser side, a fluent and intuitive interface is provided for researchers at J-TEXT laboratory and collaborators from all over the world to view experiment data and related metadata. The fusion experiments will feature long pulse and high sampling rate in the future. The data access and visualization system in this work has adopted segment and scale concept. Large data samples are re-sampled in different scales and then split into segments for instant response. It allows users to view extremely large data on the web browser efficiently, without worrying about the limitation on the size of the data. The HTML5 and JavaScript based web front-end can provide intuitive and fluent user experience. On the server side, a RESTful (Representational State Transfer) web API, which is based on ASP.NET MVC (Model View Controller), allows users to access the data and its metadata through HTTP (HyperText Transfer Protocol). An interface to the database has been designed to decouple the data access and visualization system from the data storage. It can be applied upon any data storage system like MDSplus or JTEXTDB, and this system is very easy to deploy as well. A WebScope with data provider both for MDSplus and JTEXTDB is implemented and deployed. The WebScope has been integrated into J-TEXT LogBook, an experimental recording system for J-TEXT. It is now serving J-TEXT daily experiment. In the future more databases as the data source will be supported and it will support visualization of more data types besides waveform. © 2016 Elsevier B.V. All rights reserved.
1. Introduction
∗ Corresponding author at: State Key Laboratory of Advanced Electromagnetic Engineering and Technology in Huazhong University of Science and Technology, Wuhan 430074, China. E-mail address:
[email protected] (W. Zheng).
Along with the increasing sampling rate and pulse length [2], to visualize and analyze the experiment data becomes more and more difficult. The traditional visualization tools, like jScope, fetch data as a whole from the server by default to display signal waveform. But this approach is slow and even stops working when the experiment data becomes quite large [3]. Other web-based visualization
http://dx.doi.org/10.1016/j.fusengdes.2016.06.008 0920-3796/© 2016 Elsevier B.V. All rights reserved.
Please cite this article in press as: W. Zheng, et al., J-TEXT WebScope: An efficient data access and visualization system for long pulse fusion experiment, Fusion Eng. Des. (2016), http://dx.doi.org/10.1016/j.fusengdes.2016.06.008
G Model FUSION-8793; No. of Pages 4
ARTICLE IN PRESS W. Zheng et al. / Fusion Engineering and Design xxx (2016) xxx–xxx
2
Fig. 1. System scheme.
data, such as scale, sample count and temporal information. It is also the protocol among data provider, web API controller and the javascript. In the controller layer, we need two kinds of actions: one deals with the experiment data required by the browsers to show waveform, the other is responsible for the configuration data which is stored in the user’s profile. In GetShotData action, the GetData method of DataProvider is called with parameters from front-end and the result is returned to front-end in form of json through http protocol. In the view model, an intuitive interface is provided by using HTML5, CSS, AJAX and jQuery based on javascript. To shorten the development cycle and ensure software quality, an open source library, called Flot [8], which is based on canvas and responsive for interactive waveform displaying is adopted and enhanced.
Fig. 2. ScopeData class diagram.
tools solve the problem that researchers must install a client on computer to view experiment data and the server side must disable corresponding firewall which means potential security risks. As for situation where users need view the huge data, the existed web-based visualization tools usually have a large time delay, even failed to provide service. To be flexible and highly scalable, we need to decouple the visualization tool from the data storage layer. So it will be easy to change the database, to modify the interface, to extend its functions and to establish integrated systems like ITER remote experimentation center [4]. And to resolve the data size limit, we need to adopt a new framework and technology both on browser side and server side. The rest of the paper is organized as follows: we described the design of this new system in this paper firstly. Then the J-TEXT WebScope with a friendly interface, which supports MDSplus [5] and JTEXTDB [6], is implemented. 2. System design The whole system adopts MVC framework, which is shown in Fig. 1: the model layer is the data provider for database; the control layer is the web API controller which provides RESTful [7] API for experiment data, metadata and personalized configuration; the view layer is responsible for the intuitive interface to users. In model layer, we define a new data type (ScopeData), which is shown in Fig. 2, packing the experiment data and related meta-
3. Web front-end implementation In the J-TEXT LogBook, the WebScope tool is implemented as a subsystem. Inside the Scope page, an impressive and responsive interface, which is shown in Fig. 3, is implemented by HTML and CSS. For logged-in users, they are able to manage their personal configurations, such as adding or deleting a configuration with different numbers of waveform. As long as users input the basic information about the experiment data, the waveform related will display in the window immediately. Then rich functions (navigation, zoom, crosshair, coordinate display, etc) to analysis the experiment data can be used just like using a desktop application. Users just need to save it before leaving, and he or she can easily view the same data in other places by logging in and selecting the exact configuration. Moreover, the researchers are able to auto scale different waveforms together, so that they can easily compare data among different signals. Though the experiment data is resampled and part of it is eliminated, the outline of the whole data has never changed a little. And though the current data is not sufficient, it will automatically load more detailed data when the user zooms in, and will switch back to less detailed data when user zooms out to a certain scale. It only loads a little more data than the currently display region. So no matter how large the data is, it is displayed with little lag. There is a need to illustrate that for quick response the browser always requests one thousand points for each waveform by using AJAX (Asynchronous JavaScript and XML) when data with different scale or different duration is needed. If original data between start and end is less than one thousand points, the server will return the
Please cite this article in press as: W. Zheng, et al., J-TEXT WebScope: An efficient data access and visualization system for long pulse fusion experiment, Fusion Eng. Des. (2016), http://dx.doi.org/10.1016/j.fusengdes.2016.06.008
G Model FUSION-8793; No. of Pages 4
ARTICLE IN PRESS W. Zheng et al. / Fusion Engineering and Design xxx (2016) xxx–xxx
3
Fig. 3. The WebScope page in LogBook.
part of original data directly. Then the browser just need to render the chart using the new data. As a result, the user will get the fluent experience without waiting too long for the new data even when the experiment data is extremely large. On the other hand, it is not restricted by the limitation of the Internet bandwidth and the memory capacity.
4. Loose coupling data interface and RESTful Web API Fig. 4. The tree structure.
4.1. Data provider interface In the data layer, an IDataProvider interface is defined. To support a new database for experiment data, the only thing programmer need to do is to implement a DataProvider class whose interface is the IDataProvider. In the implementation of the GetData method, the first step is to find the metadata about the basic information (supported scales, precisions, etc) by parameter expression, the identifier of the experiment data. Then, the real scale and segment are calculated. Finally, the experiment data needed and related metadata are packed as a ScopeData object to return.
4.2. The RESTful Web API REST (Representational State Transfer) is a software architecture style consisting of guidelines and best practices for scalable web services. REST is a coordinated set of constraints applied to the design of components in a distributed hypermedia system that can lead to a more performant and maintainable architecture. REST has gained widespread acceptance across the Web as a simpler alternative to SOAP and WSDL-based Web services. The J-TEXT WebScope adapts this architecture to gain effective improvement in data transmission. The RESTful URI Paths are shown in Table 1.
4.3. Segment and scale technology for MDSplus MDSplus, which allows complex scientific data to be stored into a single, self-descriptive, hierarchical structure, has widely been used in nuclear fusion research. The old J-TEXT data access system is based on the tools provided by MDSplus. So J-TEXT WebScope must be compatible with the MDSplus at the present. To store experiment data for the long pulse, we use segment pattern supported by MDSplus to store shot data. If we resample data just when the data is requested, like just using segment technology of MDSplus, the outline will change a lot because of noise data, and the response time will be expanded. To avoid these problems, data with different sampling rate and related metadata of information about all scales is also stored in the child nodes of original signal node. Base on the fact that a tiny segment data with high sampling rate looks like a vertical line, we can just use the min value and the max value to represent the whole segment data. So only the min values and max values of every certain amount original data are stored in child nodes with different sampling rate while collecting original experiment data. So the visual effect is the same for naked eyes while our method need less points than just returning the original data. And the response time is shortened by using the resample method at first and the segment technology of MDSplus while reading data. The new designed tree structure is shown in Fig. 4.
Table 1 The RESTful URI Paths. RESTful URI Path
Http Verb
Action
/api/scope/shotdata?shot=&expression=&begin=&end=&points= /api/scope/allconfignames /api/scope/myconfig/configname /api/scope/saveconfig /api/scope/addconfig /api/scope/deleteconfig/configname
Get Get Get Post Post Delete
GetShotData GetAllConfigNames GetScopeConfig SaveCurrentConfig AddConfig DeleteConfig
Please cite this article in press as: W. Zheng, et al., J-TEXT WebScope: An efficient data access and visualization system for long pulse fusion experiment, Fusion Eng. Des. (2016), http://dx.doi.org/10.1016/j.fusengdes.2016.06.008
G Model
ARTICLE IN PRESS
FUSION-8793; No. of Pages 4
W. Zheng et al. / Fusion Engineering and Design xxx (2016) xxx–xxx
4
Table 2 The comparison of viewing signal speed among signal node with different duration or sample rate. Shot
Duration(s)
Sample rate
Response time(s)
1 2 3 4
30 30 300 300
104 Hz 105 Hz 105 Hz 104 Hz
<0.5 <0.5 <0.5 <0.5
always the very small constant no matter how long the pulse is. So the scientists and the engineers can be more effective in analyzing experiment data. For the developers, it is also convenient for them to add new module, modify the user interface or even change the data provider. Because each layer is relatively independent. In the future, more databases will be supported, and a quasi-real time data visualization function for long pulse will be added in.
Tested in WAN environment with 100 Mbps bandwidth.
References For instance, an original signal data, whose sampling rate is 100 KHz and which total has 3 M points, is stored as three thousand segments as scale of zero. With the scale of child node increasing one, the sampling rate of data decrease ten times. 5. Test and analysis For J-TEXT WebScope, we design four pulse shots with different duration or sampling rate, the largest data reaches 30 M points, and the smallest data only has 300 K points. The information about all test data is shown in Table 2. It is worth pointing out that the response time always keeps below 500 milliseconds no matter how long the pulse continues in WAN environment. 6. Summary
[1] G. Manduchi, T. Fredian, J. Stillerman, A new web-based tool for data visualization in MDSplus, Fusion Eng. Des. 89 (2014) 780–783. [2] F. Yang, B.J. Xiao, F. Wang, S. Li, W.J. Huang, A new long-pulse data system for EAST experiments, Fusion Eng. Des. 89 (2014) 674–678. [3] F. Yang, B.J. Xiao, A web based MDSplus data analysis and visualization system for EAST, Fusion Eng. Des. 87 (2012) 2161–2165. [4] T. Ozeki, S.L. Clement, N. Nakajima, Plan of ITER remote experimentation center, Fusion Eng. Des. 89 (2014) 529–531. [5] C. Yang, M. Zhang, W. Zheng, R. Liu, G. Zhuang, The upgrade of the J-TEXT experimental data access and management system, Fusion Eng. Des. 89 (2014) 726–730. [6] Ming Zhang, Qiang Liu, Wei Zheng, Kuanhong Wan, Feiran Hu, Kexun Yu, Utilizing cloud storage architecture for long-pulse fusion experiment data storage, Fusion Eng. Des. (2016), In Press. [7] M. Vaqqas, RESTful Web Services: A Tutorial http://www.drdobbs.com/webdevelopment/restful-web-services-a-tutorial/240169069. [8] Ole Laursen, David Schnur, Flot-Attractive JavaScript Plotting for jQuery http:// www.flotcharts.org.
The J-TEXT WebScope adopt a decoupling and highly scalable MVC framework. Not only does it provide the same strong functions as jScope, but also enables users to easily use their history configuration. It is highlighted to say that the response time is
Please cite this article in press as: W. Zheng, et al., J-TEXT WebScope: An efficient data access and visualization system for long pulse fusion experiment, Fusion Eng. Des. (2016), http://dx.doi.org/10.1016/j.fusengdes.2016.06.008